2023-10-08T15:50:37.888 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T15:50:37.888 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298 -- /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/orig.config.yaml 2023-10-08T15:50:37.931 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T15:50:38.013 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} email: null first_in_suite: false job_id: '7418298' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-07-1508-quincy sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-07-1508-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 targets: smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5J24LseHkEJqG75H9ewnuQ/awSKU1uoWWfrr4V/Kg3ryQWLjAKjqT4KloPDDmyWqC8zjxZYvVOpIw63sAzbkQ= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6W+dSuRSn1eeAOlPjZ9dUCkVo104fXgNegX76mlc9/XMmDaVHiwpXLJotSmstk8zg6q7J4uiG1RHLTRlQ6IRA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestVolumes - tasks.cephfs.test_volumes.TestSubvolumeGroups - tasks.cephfs.test_volumes.TestSubvolumes teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-08_15:41:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T15:50:38.013 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa; will attempt to use it 2023-10-08T15:50:38.014 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks 2023-10-08T15:50:38.014 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T15:50:38.016 INFO:teuthology.task.internal:Checking packages... 2023-10-08T15:50:38.036 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-08T15:50:38.036 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T15:50:38.037 INFO:teuthology.packaging:ref: None 2023-10-08T15:50:38.037 INFO:teuthology.packaging:tag: None 2023-10-08T15:50:38.037 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T15:50:38.037 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T15:50:38.037 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T15:50:38.181 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-08T15:50:38.181 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T15:50:38.194 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T15:50:38.194 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T15:50:38.233 INFO:teuthology.task.internal:Saving configuration 2023-10-08T15:50:38.249 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T15:50:38.258 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T15:50:38.283 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:43:39.490959', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5J24LseHkEJqG75H9ewnuQ/awSKU1uoWWfrr4V/Kg3ryQWLjAKjqT4KloPDDmyWqC8zjxZYvVOpIw63sAzbkQ='} 2023-10-08T15:50:38.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:43:39.492255', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6W+dSuRSn1eeAOlPjZ9dUCkVo104fXgNegX76mlc9/XMmDaVHiwpXLJotSmstk8zg6q7J4uiG1RHLTRlQ6IRA='} 2023-10-08T15:50:38.304 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T15:50:38.310 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-08T15:50:38.310 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-08T15:50:38.310 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T15:50:38.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc30e58ab80>, signals=[15]) 2023-10-08T15:50:38.399 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T15:50:38.405 INFO:teuthology.task.internal:Opening connections... 2023-10-08T15:50:38.405 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T15:50:38.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:50:38.476 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T15:50:38.476 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:50:38.544 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T15:50:38.550 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2023-10-08T15:50:38.566 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2023-10-08T15:50:38.567 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8.4 (Ootpa)" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:ID="rhel" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="fedora" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8.4" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-08T15:50:38.622 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T15:50:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-08T15:50:38.624 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2023-10-08T15:50:38.644 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2023-10-08T15:50:38.661 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2023-10-08T15:50:38.661 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="8.4 (Ootpa)" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:ID="rhel" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="fedora" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="8.4" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el8" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-08T15:50:38.717 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T15:50:38.718 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-08T15:50:38.718 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2023-10-08T15:50:38.736 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T15:50:38.746 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T15:50:38.784 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T15:50:38.784 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2023-10-08T15:50:38.786 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2023-10-08T15:50:38.802 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T15:50:38.808 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T15:50:38.808 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2023-10-08T15:50:38.841 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2023-10-08T15:50:38.854 INFO:teuthology.orchestra.run.smithi170.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-08T15:50:38.858 INFO:teuthology.orchestra.run.smithi191.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-08T15:50:38.858 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T15:50:38.911 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T15:50:38.924 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T15:50:38.925 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-10-08T15:50:38.925 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-10-08T15:50:38.925 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T15:50:38.926 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T15:50:38.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:50:38.940 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-10-08T15:50:38.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:50:38.942 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2023-10-08T15:50:38.996 INFO:teuthology.orchestra.run.smithi170.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-08T15:50:38.996 INFO:teuthology.task.kernel:Running kernel on smithi170: 4.18.0-305.25.1.el8_4.x86_64 2023-10-08T15:50:38.996 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2023-10-08T15:50:38.997 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-08T15:50:38.998 INFO:teuthology.task.kernel:Running kernel on smithi191: 4.18.0-305.25.1.el8_4.x86_64 2023-10-08T15:50:38.998 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2023-10-08T15:50:40.414 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:40.414 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2023-10-08T15:50:40.472 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:40.472 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-08T15:50:41.899 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-08T15:50:42.003 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-08T15:50:42.572 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-08T15:50:43.186 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-10-08T15:50:45.854 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 4.0 MB/s | 24 kB 00:00 2023-10-08T15:50:46.389 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-08T15:50:46.600 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-08T15:50:46.600 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-08T15:50:46.600 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-08T15:50:46.753 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-08T15:50:46.754 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2023-10-08T15:50:46.754 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-08T15:50:46.883 DEBUG:teuthology.orchestra.run.smithi170:> echo no | sudo yum reinstall kernel || true 2023-10-08T15:50:47.085 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-10-08T15:50:47.085 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-10-08T15:50:47.085 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-10-08T15:50:47.227 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:47.227 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2023-10-08T15:50:47.229 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2023-10-08T15:50:47.230 INFO:teuthology.orchestra.run.smithi191.stdout:Nothing to do. 2023-10-08T15:50:47.231 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2023-10-08T15:50:47.331 DEBUG:teuthology.orchestra.run.smithi191:> echo no | sudo yum reinstall kernel || true 2023-10-08T15:50:47.527 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:02 ago on Sun 08 Oct 2023 03:50:45 PM UTC. 2023-10-08T15:50:47.657 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-08T15:50:47.657 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:47.657 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:47.675 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:47.675 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-08T15:50:47.697 INFO:teuthology.orchestra.run.smithi170.stderr:Error: No packages marked for reinstall. 2023-10-08T15:50:47.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum reinstall -y kernel || true 2023-10-08T15:50:47.973 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:01 ago on Sun 08 Oct 2023 03:50:46 PM UTC. 2023-10-08T15:50:48.095 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:48.095 INFO:teuthology.orchestra.run.smithi170.stdout:Unable to read consumer identity 2023-10-08T15:50:48.105 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-08T15:50:48.105 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.105 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.124 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2023-10-08T15:50:48.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum reinstall -y kernel || true 2023-10-08T15:50:48.389 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:03 ago on Sun 08 Oct 2023 03:50:45 PM UTC. 2023-10-08T15:50:48.512 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-08T15:50:48.512 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.512 INFO:teuthology.orchestra.run.smithi170.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.517 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T15:50:48.517 INFO:teuthology.orchestra.run.smithi191.stdout:Unable to read consumer identity 2023-10-08T15:50:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:Error: No packages marked for reinstall. 2023-10-08T15:50:48.584 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T15:50:48.708 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.708 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.709 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.709 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-08T15:50:48.709 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-08T15:50:48.709 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-10-08T15:50:48.709 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2023-10-08T15:50:48.709 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T15:50:48.807 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:00:02 ago on Sun 08 Oct 2023 03:50:46 PM UTC. 2023-10-08T15:50:48.834 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.835 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.835 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2023-10-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.02 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 99.el8_4.1 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Group : System Environment/Base 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 9264631 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-08T15:50:48.862 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Relocations : (not relocatable) 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : Red Hat, Inc. 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : Red Hat, Inc. 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T15:50:48.863 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2023-10-08T15:50:48.864 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T15:50:48.864 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T15:50:48.864 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T15:50:48.864 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:48.865 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T15:50:48.937 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-10-08T15:50:48.937 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.937 INFO:teuthology.orchestra.run.smithi191.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-10-08T15:50:48.949 INFO:teuthology.orchestra.run.smithi191.stderr:Error: No packages marked for reinstall. 2023-10-08T15:50:48.998 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T15:50:49.123 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.123 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.123 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.123 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-08T15:50:49.123 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-08T15:50:49.123 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-10-08T15:50:49.123 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2023-10-08T15:50:49.124 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2023-10-08T15:50:49.248 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.248 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.248 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2023-10-08T15:50:49.275 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2023-10-08T15:50:49.275 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2023-10-08T15:50:49.275 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.02 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 99.el8_4.1 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Group : System Environment/Base 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 9264631 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Relocations : (not relocatable) 2023-10-08T15:50:49.276 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : Red Hat, Inc. 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : Red Hat, Inc. 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T15:50:49.277 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T15:50:49.278 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T15:50:49.278 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:49.278 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T15:50:50.219 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2023-10-08T15:50:50.648 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2023-10-08T15:50:51.996 INFO:teuthology.orchestra.run.smithi170.stderr:done 2023-10-08T15:50:51.997 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-10-08T15:50:52.011 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.nFr9Yn7JeF 2023-10-08T15:50:52.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nFr9Yn7JeF 2023-10-08T15:50:52.085 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.nFr9Yn7JeF 2023-10-08T15:50:52.256 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.nFr9Yn7JeF is 6KB 2023-10-08T15:50:52.304 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.nFr9Yn7JeF 2023-10-08T15:50:52.318 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T15:50:52.343 INFO:teuthology.orchestra.run.smithi191.stderr:done 2023-10-08T15:50:52.345 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-08T15:50:52.376 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.3c4AqmBtdT 2023-10-08T15:50:52.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3c4AqmBtdT 2023-10-08T15:50:52.387 INFO:teuthology.orchestra.run.smithi170.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-08T15:50:52.387 INFO:teuthology.orchestra.run.smithi170.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-08T15:50:52.387 INFO:teuthology.orchestra.run.smithi170.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-08T15:50:52.387 INFO:teuthology.orchestra.run.smithi170.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-08T15:50:52.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:52.444 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.3c4AqmBtdT 2023-10-08T15:50:52.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2023-10-08T15:50:52.604 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.3c4AqmBtdT is 6KB 2023-10-08T15:50:52.654 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.3c4AqmBtdT 2023-10-08T15:50:52.667 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T15:50:52.729 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-08T15:50:52.729 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-08T15:50:52.729 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-08T15:50:52.729 INFO:teuthology.orchestra.run.smithi191.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-08T15:50:52.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-10-08T15:50:52.880 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2023-10-08T15:51:22.567 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-08T15:51:22.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T15:51:22.568 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:51:22.883 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-08T15:51:22.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T15:51:22.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:51:40.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-10-08T15:51:41.438 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-10-08T15:51:49.936 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T15:51:49.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:51:50.440 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T15:51:50.440 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:51:53.018 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2023-10-08T15:51:53.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.191 2023-10-08T15:52:05.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T15:52:05.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:52:05.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T15:52:05.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T15:52:36.901 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-10-08T15:52:37.147 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T15:52:37.147 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-08T15:52:37.147 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2023-10-08T15:52:37.161 INFO:teuthology.orchestra.run.smithi170.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-08T15:52:37.161 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:52:37.161 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T15:52:37.161 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-08T15:52:37.352 DEBUG:teuthology.orchestra.run.smithi191:> true 2023-10-08T15:52:37.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T15:52:37.657 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-10-08T15:52:37.657 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2023-10-08T15:52:37.671 INFO:teuthology.orchestra.run.smithi191.stdout:4.18.0-425.3.1.el8.x86_64 2023-10-08T15:52:37.671 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-10-08T15:52:37.671 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T15:52:37.671 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-08T15:52:38.163 DEBUG:teuthology.parallel:result is None 2023-10-08T15:52:38.672 DEBUG:teuthology.parallel:result is None 2023-10-08T15:52:38.673 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T15:52:38.680 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T15:52:38.680 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T15:52:38.683 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T15:52:38.700 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T15:52:38.717 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T15:52:38.751 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T15:52:38.751 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T15:52:38.753 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T15:52:38.845 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T15:52:38.851 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T15:52:38.852 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T15:52:38.854 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T15:52:39.057 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:52:39.066 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:52:39.085 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:52:39.090 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T15:52:39.093 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T15:52:39.100 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T15:52:39.100 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T15:52:39.112 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T15:52:39.198 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T15:52:39.206 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T15:52:39.207 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T15:52:39.209 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T15:52:39.256 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:52:39.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:52:39.534 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:52:39.719 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:52:40.257 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T15:52:40.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T15:52:40.464 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:52:40.780 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T15:52:41.329 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:52:41.504 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T15:52:41.528 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T15:52:41.528 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T15:52:41.592 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2023-10-08T15:52:41.611 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2023-10-08T15:52:41.699 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T15:52:41.825 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T15:52:42.335 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T15:52:42.343 INFO:teuthology.task.internal:Starting timer... 2023-10-08T15:52:42.343 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T15:52:42.387 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T15:52:42.419 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T15:52:42.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2023-10-08T15:52:42.533 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: [ OK ] 2023-10-08T15:52:42.536 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2023-10-08T15:52:42.615 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: [ OK ] 2023-10-08T15:52:42.617 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T15:52:42.617 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2023-10-08T15:52:42.679 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2023-10-08T15:52:42.679 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2023-10-08T15:52:42.699 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2023-10-08T15:52:42.700 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi170.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2023-10-08T15:52:42.700 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T15:52:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:52:42.727 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T15:52:42.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T15:52:42.764 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T15:52:42.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2023-10-08T15:52:42.799 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2023-10-08T15:52:42.829 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T15:52:42.839 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-08T15:52:42.845 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-08T15:52:42.968 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T15:52:42.984 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T15:52:42.985 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T16:01:17.225 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2023-10-08T16:01:17.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T16:01:17.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T16:01:17.297 DEBUG:teuthology.orchestra.run.smithi170:> true 2023-10-08T16:01:17.320 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2023-10-08T16:01:17.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T16:01:17.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T16:01:17.390 DEBUG:teuthology.orchestra.run.smithi191:> true 2023-10-08T16:01:17.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2023-10-08T16:01:17.457 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T16:01:17.470 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T16:01:17.471 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T16:01:17.471 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T16:01:17.473 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T16:01:17.473 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T16:01:17.500 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T16:01:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T16:01:17.527 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T16:01:17.541 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T16:01:17.550 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2023-10-08T16:01:17.561 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2023-10-08T16:01:17.574 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T16:01:17.579 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2023-10-08T16:01:17.588 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T16:01:17.591 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2023-10-08T16:01:17.604 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T16:01:17.618 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T16:01:17.655 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-10-08T16:01:17.657 INFO:teuthology.orchestra.run.smithi170.stdout:210 Number of sources = 4 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.658 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.682 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2023-10-08T16:01:17.684 INFO:teuthology.orchestra.run.smithi191.stdout:210 Number of sources = 4 2023-10-08T16:01:17.684 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T16:01:17.684 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2023-10-08T16:01:17.684 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.685 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.685 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.685 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:01:17.685 INFO:teuthology.run_tasks:Running task install... 2023-10-08T16:01:17.693 DEBUG:teuthology.task.install:project ceph 2023-10-08T16:01:17.693 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-08T16:01:17.694 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-08T16:01:17.694 INFO:teuthology.task.install:Using flavor: default 2023-10-08T16:01:17.699 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T16:01:17.699 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T16:01:17.699 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T16:01:17.700 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T16:01:17.701 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T16:01:17.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T16:01:17.924 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T16:01:17.924 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T16:01:17.935 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T16:01:17.936 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T16:01:18.256 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T16:01:18.256 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:01:18.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T16:01:18.262 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T16:01:18.262 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:01:18.263 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T16:01:18.282 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T16:01:18.282 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T16:01:18.290 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T16:01:18.290 DEBUG:teuthology.orchestra.run.smithi191:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T16:01:18.347 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T16:01:18.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T16:01:18.421 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T16:01:18.430 DEBUG:teuthology.orchestra.run.smithi191:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T16:01:18.485 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2023-10-08T16:01:18.487 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2023-10-08T16:01:18.493 INFO:teuthology.orchestra.run.smithi191.stdout:check_obsoletes = 1 2023-10-08T16:01:18.495 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum clean all 2023-10-08T16:01:18.880 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-08T16:01:18.886 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T16:01:20.101 INFO:teuthology.orchestra.run.smithi170.stdout:44 files removed 2023-10-08T16:01:20.160 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T16:01:20.253 INFO:teuthology.orchestra.run.smithi191.stdout:44 files removed 2023-10-08T16:01:20.310 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T16:01:20.510 INFO:teuthology.orchestra.run.smithi170.stdout:Updating Subscription Management repositories. 2023-10-08T16:01:20.660 INFO:teuthology.orchestra.run.smithi191.stdout:Updating Subscription Management repositories. 2023-10-08T16:01:22.812 INFO:teuthology.orchestra.run.smithi191.stdout:ceph packages for x86_64 100 kB/s | 78 kB 00:00 2023-10-08T16:01:22.971 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 95 kB/s | 78 kB 00:00 2023-10-08T16:01:23.536 INFO:teuthology.orchestra.run.smithi191.stdout:ceph noarch packages 13 kB/s | 9.2 kB 00:00 2023-10-08T16:01:23.709 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 13 kB/s | 9.2 kB 00:00 2023-10-08T16:01:24.153 INFO:teuthology.orchestra.run.smithi191.stdout:ceph source packages 2.5 kB/s | 1.5 kB 00:00 2023-10-08T16:01:24.287 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-10-08T16:01:24.298 INFO:teuthology.orchestra.run.smithi191.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-10-08T16:01:24.399 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-08T16:01:24.955 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 30 MB/s | 16 MB 00:00 2023-10-08T16:01:28.233 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-08T16:01:28.841 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 73 MB/s | 38 MB 00:00 2023-10-08T16:01:33.694 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 16 MB 00:09 2023-10-08T16:01:34.368 INFO:teuthology.orchestra.run.smithi170.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 108 MB/s | 33 MB 00:00 2023-10-08T16:01:36.941 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-08T16:01:37.341 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 120 MB/s | 38 MB 00:00 2023-10-08T16:01:42.279 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T16:01:42.280 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T16:01:42.291 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T16:01:42.292 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T16:01:42.293 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T16:01:42.293 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T16:01:42.295 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T16:01:42.297 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T16:01:42.298 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T16:01:42.789 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================================================== 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================================================== 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T16:01:42.800 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T16:01:42.801 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T16:01:42.802 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2023-10-08T16:01:42.803 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T16:01:42.804 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T16:01:42.805 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T16:01:42.806 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T16:01:42.807 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T16:01:42.808 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T16:01:42.809 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T16:01:42.810 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout:====================================================================================================================== 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout:Install 118 Packages 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 2 Packages 2023-10-08T16:01:42.811 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:01:42.812 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 202 M 2023-10-08T16:01:42.812 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2023-10-08T16:01:42.910 INFO:teuthology.orchestra.run.smithi191.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 87 MB/s | 33 MB 00:00 2023-10-08T16:01:43.321 INFO:teuthology.orchestra.run.smithi170.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-10-08T16:01:44.065 INFO:teuthology.orchestra.run.smithi170.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.2 MB/s | 876 kB 00:00 2023-10-08T16:01:44.148 INFO:teuthology.orchestra.run.smithi170.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 1.9 MB/s | 162 kB 00:00 2023-10-08T16:01:44.815 INFO:teuthology.orchestra.run.smithi170.stdout:(4/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 2.8 MB/s | 5.2 MB 00:01 2023-10-08T16:01:45.108 INFO:teuthology.orchestra.run.smithi170.stdout:(5/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-10-08T16:01:45.348 INFO:teuthology.orchestra.run.smithi170.stdout:(6/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 2.9 MB/s | 1.6 MB 00:00 2023-10-08T16:01:46.078 INFO:teuthology.orchestra.run.smithi170.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 4.1 MB/s | 3.9 MB 00:00 2023-10-08T16:01:48.664 INFO:teuthology.orchestra.run.smithi170.stdout:(8/120): ceph-radosgw-17.2.6-1320.g08cac018.el8 6.5 MB/s | 17 MB 00:02 2023-10-08T16:01:48.745 INFO:teuthology.orchestra.run.smithi170.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 302 kB/s | 24 kB 00:00 2023-10-08T16:01:49.133 INFO:teuthology.orchestra.run.smithi170.stdout:(10/120): ceph-common-17.2.6-1320.g08cac018.el8 3.8 MB/s | 23 MB 00:06 2023-10-08T16:01:49.210 INFO:teuthology.orchestra.run.smithi170.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 3.3 MB/s | 255 kB 00:00 2023-10-08T16:01:49.278 INFO:teuthology.orchestra.run.smithi170.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 367 kB/s | 25 kB 00:00 2023-10-08T16:01:49.515 INFO:teuthology.orchestra.run.smithi170.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 3.3 MB/s | 797 kB 00:00 2023-10-08T16:01:49.593 INFO:teuthology.orchestra.run.smithi170.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 2.2 MB/s | 174 kB 00:00 2023-10-08T16:01:49.669 INFO:teuthology.orchestra.run.smithi170.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 1.7 MB/s | 130 kB 00:00 2023-10-08T16:01:49.828 INFO:teuthology.orchestra.run.smithi170.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 3.1 MB/s | 507 kB 00:00 2023-10-08T16:01:50.807 INFO:teuthology.orchestra.run.smithi191.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T16:01:50.808 INFO:teuthology.orchestra.run.smithi191.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T16:01:50.819 INFO:teuthology.orchestra.run.smithi191.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T16:01:50.820 INFO:teuthology.orchestra.run.smithi191.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T16:01:50.820 INFO:teuthology.orchestra.run.smithi191.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T16:01:50.821 INFO:teuthology.orchestra.run.smithi191.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T16:01:50.823 INFO:teuthology.orchestra.run.smithi191.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T16:01:50.825 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T16:01:50.826 INFO:teuthology.orchestra.run.smithi191.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T16:01:51.148 INFO:teuthology.orchestra.run.smithi170.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 4.2 MB/s | 5.6 MB 00:01 2023-10-08T16:01:51.228 INFO:teuthology.orchestra.run.smithi170.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 563 kB/s | 45 kB 00:00 2023-10-08T16:01:51.315 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2023-10-08T16:01:51.315 INFO:teuthology.orchestra.run.smithi170.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 1.3 MB/s | 115 kB 00:00 2023-10-08T16:01:51.326 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: Package Arch Version Repository Size 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T16:01:51.327 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T16:01:51.328 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T16:01:51.329 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrading: 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T16:01:51.330 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T16:01:51.331 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T16:01:51.332 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T16:01:51.333 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T16:01:51.334 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T16:01:51.335 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T16:01:51.336 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout:====================================================================================================================== 2023-10-08T16:01:51.337 INFO:teuthology.orchestra.run.smithi191.stdout:Install 118 Packages 2023-10-08T16:01:51.338 INFO:teuthology.orchestra.run.smithi191.stdout:Upgrade 2 Packages 2023-10-08T16:01:51.338 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:01:51.338 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 202 M 2023-10-08T16:01:51.338 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2023-10-08T16:01:51.401 INFO:teuthology.orchestra.run.smithi170.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 2.2 MB/s | 190 kB 00:00 2023-10-08T16:01:51.494 INFO:teuthology.orchestra.run.smithi170.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 3.9 MB/s | 371 kB 00:00 2023-10-08T16:01:51.587 INFO:teuthology.orchestra.run.smithi170.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 3.8 MB/s | 358 kB 00:00 2023-10-08T16:01:51.671 INFO:teuthology.orchestra.run.smithi170.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 1.3 MB/s | 107 kB 00:00 2023-10-08T16:01:51.753 INFO:teuthology.orchestra.run.smithi170.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.1 MB/s | 90 kB 00:00 2023-10-08T16:01:51.781 INFO:teuthology.orchestra.run.smithi191.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-10-08T16:01:52.174 INFO:teuthology.orchestra.run.smithi170.stdout:(25/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 2.4 MB/s | 17 MB 00:06 2023-10-08T16:01:52.321 INFO:teuthology.orchestra.run.smithi170.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 1.2 MB/s | 173 kB 00:00 2023-10-08T16:01:52.390 INFO:teuthology.orchestra.run.smithi170.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 332 kB/s | 23 kB 00:00 2023-10-08T16:01:52.456 INFO:teuthology.orchestra.run.smithi170.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 1.8 MB/s | 121 kB 00:00 2023-10-08T16:01:52.570 INFO:teuthology.orchestra.run.smithi170.stdout:(29/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 4.2 MB/s | 3.5 MB 00:00 2023-10-08T16:01:53.612 INFO:teuthology.orchestra.run.smithi170.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 1.5 MB/s | 1.7 MB 00:01 2023-10-08T16:01:53.733 INFO:teuthology.orchestra.run.smithi170.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 1.9 MB/s | 238 kB 00:00 2023-10-08T16:01:53.793 INFO:teuthology.orchestra.run.smithi170.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 795 kB/s | 47 kB 00:00 2023-10-08T16:01:53.845 INFO:teuthology.orchestra.run.smithi170.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 277 kB/s | 14 kB 00:00 2023-10-08T16:01:53.969 INFO:teuthology.orchestra.run.smithi170.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 2.1 MB/s | 267 kB 00:00 2023-10-08T16:01:54.031 INFO:teuthology.orchestra.run.smithi170.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-08T16:01:54.090 INFO:teuthology.orchestra.run.smithi170.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 307 kB/s | 18 kB 00:00 2023-10-08T16:01:54.226 INFO:teuthology.orchestra.run.smithi191.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 358 kB/s | 876 kB 00:02 2023-10-08T16:01:54.465 INFO:teuthology.orchestra.run.smithi191.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 678 kB/s | 162 kB 00:00 2023-10-08T16:01:54.603 INFO:teuthology.orchestra.run.smithi170.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 596 kB/s | 306 kB 00:00 2023-10-08T16:01:54.686 INFO:teuthology.orchestra.run.smithi170.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 716 kB/s | 59 kB 00:00 2023-10-08T16:01:54.732 INFO:teuthology.orchestra.run.smithi170.stdout:(39/120): ceph-mgr-diskprediction-local-17.2.6- 3.4 MB/s | 7.4 MB 00:02 2023-10-08T16:01:54.771 INFO:teuthology.orchestra.run.smithi170.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 879 kB/s | 75 kB 00:00 2023-10-08T16:01:54.852 INFO:teuthology.orchestra.run.smithi170.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 540 kB/s | 44 kB 00:00 2023-10-08T16:01:54.957 INFO:teuthology.orchestra.run.smithi170.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-10-08T16:01:55.036 INFO:teuthology.orchestra.run.smithi170.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 419 kB/s | 33 kB 00:00 2023-10-08T16:01:55.112 INFO:teuthology.orchestra.run.smithi170.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 215 kB/s | 16 kB 00:00 2023-10-08T16:01:55.215 INFO:teuthology.orchestra.run.smithi170.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-10-08T16:01:55.278 INFO:teuthology.orchestra.run.smithi170.stdout:(46/120): python3-asyncssh-2.7.0-2.el8.noarch.r 872 kB/s | 476 kB 00:00 2023-10-08T16:01:55.354 INFO:teuthology.orchestra.run.smithi170.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2023-10-08T16:01:55.360 INFO:teuthology.orchestra.run.smithi170.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 996 kB/s | 81 kB 00:00 2023-10-08T16:01:55.428 INFO:teuthology.orchestra.run.smithi170.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 150 kB/s | 11 kB 00:00 2023-10-08T16:01:55.434 INFO:teuthology.orchestra.run.smithi170.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 236 kB/s | 18 kB 00:00 2023-10-08T16:01:55.513 INFO:teuthology.orchestra.run.smithi170.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 624 kB/s | 49 kB 00:00 2023-10-08T16:01:55.592 INFO:teuthology.orchestra.run.smithi170.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 741 kB/s | 59 kB 00:00 2023-10-08T16:01:55.646 INFO:teuthology.orchestra.run.smithi170.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 5.4 MB/s | 1.2 MB 00:00 2023-10-08T16:01:55.671 INFO:teuthology.orchestra.run.smithi170.stdout:(54/120): python3-natsort-7.1.1-2.el8.noarch.rp 763 kB/s | 60 kB 00:00 2023-10-08T16:01:55.731 INFO:teuthology.orchestra.run.smithi170.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-08T16:01:55.745 INFO:teuthology.orchestra.run.smithi170.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 211 kB/s | 16 kB 00:00 2023-10-08T16:01:55.805 INFO:teuthology.orchestra.run.smithi170.stdout:(57/120): python3-repoze-lru-0.7-6.el8.noarch.r 454 kB/s | 34 kB 00:00 2023-10-08T16:01:55.839 INFO:teuthology.orchestra.run.smithi170.stdout:(58/120): python3-routes-2.4.1-12.el8.noarch.rp 2.0 MB/s | 196 kB 00:00 2023-10-08T16:01:55.881 INFO:teuthology.orchestra.run.smithi170.stdout:(59/120): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-10-08T16:01:55.914 INFO:teuthology.orchestra.run.smithi170.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 250 kB/s | 19 kB 00:00 2023-10-08T16:01:55.955 INFO:teuthology.orchestra.run.smithi170.stdout:(61/120): python3-singledispatch-3.4.0.3-18.el8 323 kB/s | 24 kB 00:00 2023-10-08T16:01:55.990 INFO:teuthology.orchestra.run.smithi170.stdout:(62/120): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-10-08T16:01:56.029 INFO:teuthology.orchestra.run.smithi170.stdout:(63/120): python3-trustme-0.6.0-4.el8.noarch.rp 371 kB/s | 27 kB 00:00 2023-10-08T16:01:56.541 INFO:teuthology.orchestra.run.smithi191.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 1.1 MB/s | 2.3 MB 00:02 2023-10-08T16:01:56.592 INFO:teuthology.orchestra.run.smithi170.stdout:(64/120): ceph-test-17.2.6-1320.g08cac018.el8.x 6.6 MB/s | 52 MB 00:07 2023-10-08T16:01:56.594 INFO:teuthology.orchestra.run.smithi170.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 423 kB/s | 255 kB 00:00 2023-10-08T16:01:56.595 INFO:teuthology.orchestra.run.smithi170.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 444 kB/s | 251 kB 00:00 2023-10-08T16:01:56.669 INFO:teuthology.orchestra.run.smithi170.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 310 kB/s | 23 kB 00:00 2023-10-08T16:01:56.677 INFO:teuthology.orchestra.run.smithi170.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-10-08T16:01:56.681 INFO:teuthology.orchestra.run.smithi170.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-08T16:01:56.750 INFO:teuthology.orchestra.run.smithi170.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 271 kB/s | 19 kB 00:00 2023-10-08T16:01:56.789 INFO:teuthology.orchestra.run.smithi170.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2023-10-08T16:01:56.817 INFO:teuthology.orchestra.run.smithi170.stdout:(72/120): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-10-08T16:01:56.820 INFO:teuthology.orchestra.run.smithi170.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 4.9 MB/s | 155 kB 00:00 2023-10-08T16:01:56.851 INFO:teuthology.orchestra.run.smithi170.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 1.5 MB/s | 48 kB 00:00 2023-10-08T16:01:56.873 INFO:teuthology.orchestra.run.smithi170.stdout:(75/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.9 MB/s | 220 kB 00:00 2023-10-08T16:01:56.875 INFO:teuthology.orchestra.run.smithi170.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.3 MB/s | 229 kB 00:00 2023-10-08T16:01:56.892 INFO:teuthology.orchestra.run.smithi170.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.3 MB/s | 43 kB 00:00 2023-10-08T16:01:56.911 INFO:teuthology.orchestra.run.smithi170.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 946 kB/s | 18 kB 00:00 2023-10-08T16:01:56.918 INFO:teuthology.orchestra.run.smithi170.stdout:(79/120): python3-websocket-client-0.56.0-5.el8 188 kB/s | 61 kB 00:00 2023-10-08T16:01:56.934 INFO:teuthology.orchestra.run.smithi170.stdout:(80/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 40 MB/s | 2.3 MB 00:00 2023-10-08T16:01:56.947 INFO:teuthology.orchestra.run.smithi170.stdout:(81/120): pcre2-devel-10.32-2.el8.x86_64.rpm 17 MB/s | 605 kB 00:00 2023-10-08T16:01:56.962 INFO:teuthology.orchestra.run.smithi170.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 2.5 MB/s | 59 kB 00:00 2023-10-08T16:01:56.979 INFO:teuthology.orchestra.run.smithi170.stdout:(83/120): fuse-2.9.7-12.el8.x86_64.rpm 1.3 MB/s | 83 kB 00:00 2023-10-08T16:01:56.980 INFO:teuthology.orchestra.run.smithi170.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.1 MB/s | 38 kB 00:00 2023-10-08T16:01:56.999 INFO:teuthology.orchestra.run.smithi170.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 4.2 MB/s | 86 kB 00:00 2023-10-08T16:01:57.016 INFO:teuthology.orchestra.run.smithi170.stdout:(86/120): libquadmath-8.4.1-1.el8.x86_64.rpm 2.4 MB/s | 169 kB 00:00 2023-10-08T16:01:57.021 INFO:teuthology.orchestra.run.smithi170.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.1 MB/s | 47 kB 00:00 2023-10-08T16:01:57.026 INFO:teuthology.orchestra.run.smithi170.stdout:(88/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 1.2 MB/s | 58 kB 00:00 2023-10-08T16:01:57.046 INFO:teuthology.orchestra.run.smithi170.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 3.5 MB/s | 70 kB 00:00 2023-10-08T16:01:57.049 INFO:teuthology.orchestra.run.smithi170.stdout:(90/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 17 MB/s | 559 kB 00:00 2023-10-08T16:01:57.059 INFO:teuthology.orchestra.run.smithi170.stdout:(91/120): libgfortran-8.4.1-1.el8.x86_64.rpm 16 MB/s | 639 kB 00:00 2023-10-08T16:01:57.071 INFO:teuthology.orchestra.run.smithi170.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 7.8 MB/s | 200 kB 00:00 2023-10-08T16:01:57.073 INFO:teuthology.orchestra.run.smithi170.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 3.9 MB/s | 97 kB 00:00 2023-10-08T16:01:57.090 INFO:teuthology.orchestra.run.smithi170.stdout:(94/120): python3-prettytable-0.7.2-14.el8.noar 2.3 MB/s | 44 kB 00:00 2023-10-08T16:01:57.091 INFO:teuthology.orchestra.run.smithi170.stdout:(95/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.5 MB/s | 27 kB 00:00 2023-10-08T16:01:57.113 INFO:teuthology.orchestra.run.smithi170.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 6.8 MB/s | 161 kB 00:00 2023-10-08T16:01:57.116 INFO:teuthology.orchestra.run.smithi170.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 12 MB/s | 320 kB 00:00 2023-10-08T16:01:57.142 INFO:teuthology.orchestra.run.smithi170.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 16 MB/s | 457 kB 00:00 2023-10-08T16:01:57.163 INFO:teuthology.orchestra.run.smithi170.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 1.8 MB/s | 39 kB 00:00 2023-10-08T16:01:57.194 INFO:teuthology.orchestra.run.smithi170.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 22 MB/s | 688 kB 00:00 2023-10-08T16:01:57.210 INFO:teuthology.orchestra.run.smithi170.stdout:(101/120): python3-babel-2.5.1-5.el8.noarch.rpm 51 MB/s | 4.8 MB 00:00 2023-10-08T16:01:57.232 INFO:teuthology.orchestra.run.smithi170.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.7 MB/s | 259 kB 00:00 2023-10-08T16:01:57.249 INFO:teuthology.orchestra.run.smithi170.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 5.8 MB/s | 225 kB 00:00 2023-10-08T16:01:57.341 INFO:teuthology.orchestra.run.smithi191.stdout:(5/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 904 kB/s | 5.2 MB 00:05 2023-10-08T16:01:57.355 INFO:teuthology.orchestra.run.smithi170.stdout:(104/120): python3-scipy-1.0.0-20.module+el8.1. 49 MB/s | 15 MB 00:00 2023-10-08T16:01:57.357 INFO:teuthology.orchestra.run.smithi170.stdout:(105/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2023-10-08T16:01:57.398 INFO:teuthology.orchestra.run.smithi170.stdout:(106/120): python3-numpy-1.14.3-9.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-10-08T16:01:57.401 INFO:teuthology.orchestra.run.smithi170.stdout:(107/120): oniguruma-6.8.2-2.el8.x86_64.rpm 4.0 MB/s | 187 kB 00:00 2023-10-08T16:01:57.402 INFO:teuthology.orchestra.run.smithi191.stdout:(6/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 1.8 MB/s | 1.6 MB 00:00 2023-10-08T16:01:57.403 INFO:teuthology.orchestra.run.smithi170.stdout:(108/120): python3-jinja2-2.10.1-2.el8_0.noarch 12 MB/s | 538 kB 00:00 2023-10-08T16:01:57.420 INFO:teuthology.orchestra.run.smithi170.stdout:(109/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-10-08T16:01:57.422 INFO:teuthology.orchestra.run.smithi170.stdout:(110/120): python3-mako-1.0.6-13.el8.noarch.rpm 7.3 MB/s | 157 kB 00:00 2023-10-08T16:01:57.429 INFO:teuthology.orchestra.run.smithi170.stdout:(111/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.2 MB/s | 110 kB 00:00 2023-10-08T16:01:57.444 INFO:teuthology.orchestra.run.smithi170.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 896 kB/s | 20 kB 00:00 2023-10-08T16:01:57.449 INFO:teuthology.orchestra.run.smithi170.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 12 MB/s | 353 kB 00:00 2023-10-08T16:01:57.475 INFO:teuthology.orchestra.run.smithi170.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 15 MB/s | 709 kB 00:00 2023-10-08T16:01:57.492 INFO:teuthology.orchestra.run.smithi170.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-08T16:01:57.523 INFO:teuthology.orchestra.run.smithi170.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 23 MB/s | 713 kB 00:00 2023-10-08T16:01:57.553 INFO:teuthology.orchestra.run.smithi170.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 46 MB/s | 4.7 MB 00:00 2023-10-08T16:01:57.570 INFO:teuthology.orchestra.run.smithi170.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 48 MB/s | 4.6 MB 00:00 2023-10-08T16:01:58.412 INFO:teuthology.orchestra.run.smithi170.stdout:(119/120): librados2-17.2.6-1320.g08cac018.el8. 3.9 MB/s | 3.5 MB 00:00 2023-10-08T16:01:58.920 INFO:teuthology.orchestra.run.smithi170.stdout:(120/120): librbd1-17.2.6-1320.g08cac018.el8.x8 2.6 MB/s | 3.6 MB 00:01 2023-10-08T16:01:58.923 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2023-10-08T16:01:58.923 INFO:teuthology.orchestra.run.smithi170.stdout:Total 13 MB/s | 202 MB 00:16 2023-10-08T16:01:58.970 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2023-10-08T16:01:59.082 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2023-10-08T16:01:59.082 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2023-10-08T16:01:59.451 INFO:teuthology.orchestra.run.smithi191.stdout:(7/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 1.9 MB/s | 3.9 MB 00:02 2023-10-08T16:01:59.728 INFO:teuthology.orchestra.run.smithi191.stdout:(8/120): ceph-common-17.2.6-1320.g08cac018.el8. 2.8 MB/s | 23 MB 00:08 2023-10-08T16:01:59.786 INFO:teuthology.orchestra.run.smithi191.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 423 kB/s | 24 kB 00:00 2023-10-08T16:02:00.810 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2023-10-08T16:02:00.815 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2023-10-08T16:02:03.826 INFO:teuthology.orchestra.run.smithi191.stdout:(10/120): ceph-osd-17.2.6-1320.g08cac018.el8.x8 2.6 MB/s | 17 MB 00:06 2023-10-08T16:02:03.931 INFO:teuthology.orchestra.run.smithi191.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 2.4 MB/s | 255 kB 00:00 2023-10-08T16:02:03.985 INFO:teuthology.orchestra.run.smithi191.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 458 kB/s | 25 kB 00:00 2023-10-08T16:02:04.238 INFO:teuthology.orchestra.run.smithi191.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 3.1 MB/s | 797 kB 00:00 2023-10-08T16:02:04.356 INFO:teuthology.orchestra.run.smithi191.stdout:(14/120): ceph-radosgw-17.2.6-1320.g08cac018.el 3.4 MB/s | 17 MB 00:04 2023-10-08T16:02:04.358 INFO:teuthology.orchestra.run.smithi191.stdout:(15/120): libcephsqlite-17.2.6-1320.g08cac018.e 1.4 MB/s | 174 kB 00:00 2023-10-08T16:02:04.407 INFO:teuthology.orchestra.run.smithi191.stdout:(16/120): librados-devel-17.2.6-1320.g08cac018. 2.5 MB/s | 130 kB 00:00 2023-10-08T16:02:04.531 INFO:teuthology.orchestra.run.smithi191.stdout:(17/120): libradosstriper1-17.2.6-1320.g08cac01 2.9 MB/s | 507 kB 00:00 2023-10-08T16:02:04.588 INFO:teuthology.orchestra.run.smithi191.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 778 kB/s | 45 kB 00:00 2023-10-08T16:02:04.650 INFO:teuthology.orchestra.run.smithi191.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 1.8 MB/s | 115 kB 00:00 2023-10-08T16:02:04.706 INFO:teuthology.orchestra.run.smithi191.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 3.3 MB/s | 190 kB 00:00 2023-10-08T16:02:04.840 INFO:teuthology.orchestra.run.smithi191.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 2.7 MB/s | 371 kB 00:00 2023-10-08T16:02:04.962 INFO:teuthology.orchestra.run.smithi191.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 2.9 MB/s | 358 kB 00:00 2023-10-08T16:02:05.025 INFO:teuthology.orchestra.run.smithi191.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 1.6 MB/s | 107 kB 00:00 2023-10-08T16:02:05.090 INFO:teuthology.orchestra.run.smithi191.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.4 MB/s | 90 kB 00:00 2023-10-08T16:02:05.684 INFO:teuthology.orchestra.run.smithi191.stdout:(25/120): librgw2-17.2.6-1320.g08cac018.el8.x86 4.3 MB/s | 5.6 MB 00:01 2023-10-08T16:02:05.757 INFO:teuthology.orchestra.run.smithi191.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 2.3 MB/s | 173 kB 00:00 2023-10-08T16:02:05.828 INFO:teuthology.orchestra.run.smithi191.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 328 kB/s | 23 kB 00:00 2023-10-08T16:02:05.895 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2023-10-08T16:02:05.898 INFO:teuthology.orchestra.run.smithi191.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 1.7 MB/s | 121 kB 00:00 2023-10-08T16:02:06.137 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T16:02:06.178 INFO:teuthology.orchestra.run.smithi191.stdout:(29/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 3.2 MB/s | 3.5 MB 00:01 2023-10-08T16:02:06.341 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T16:02:06.352 INFO:teuthology.orchestra.run.smithi191.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 3.7 MB/s | 1.7 MB 00:00 2023-10-08T16:02:06.455 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T16:02:06.466 INFO:teuthology.orchestra.run.smithi191.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 2.0 MB/s | 238 kB 00:00 2023-10-08T16:02:06.515 INFO:teuthology.orchestra.run.smithi191.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 978 kB/s | 47 kB 00:00 2023-10-08T16:02:06.567 INFO:teuthology.orchestra.run.smithi191.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 279 kB/s | 14 kB 00:00 2023-10-08T16:02:06.641 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T16:02:06.688 INFO:teuthology.orchestra.run.smithi191.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 2.2 MB/s | 267 kB 00:00 2023-10-08T16:02:06.750 INFO:teuthology.orchestra.run.smithi191.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-08T16:02:06.810 INFO:teuthology.orchestra.run.smithi191.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 304 kB/s | 18 kB 00:00 2023-10-08T16:02:06.975 INFO:teuthology.orchestra.run.smithi191.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-10-08T16:02:07.001 INFO:teuthology.orchestra.run.smithi191.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 2.3 MB/s | 59 kB 00:00 2023-10-08T16:02:07.028 INFO:teuthology.orchestra.run.smithi191.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 2.7 MB/s | 75 kB 00:00 2023-10-08T16:02:07.093 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T16:02:07.156 INFO:teuthology.orchestra.run.smithi191.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 3.6 MB/s | 476 kB 00:00 2023-10-08T16:02:07.183 INFO:teuthology.orchestra.run.smithi191.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.6 MB/s | 44 kB 00:00 2023-10-08T16:02:07.227 INFO:teuthology.orchestra.run.smithi191.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.1 MB/s | 185 kB 00:00 2023-10-08T16:02:07.246 INFO:teuthology.orchestra.run.smithi191.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.7 MB/s | 33 kB 00:00 2023-10-08T16:02:07.265 INFO:teuthology.orchestra.run.smithi191.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 873 kB/s | 16 kB 00:00 2023-10-08T16:02:07.302 INFO:teuthology.orchestra.run.smithi191.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.5 MB/s | 173 kB 00:00 2023-10-08T16:02:07.376 INFO:teuthology.orchestra.run.smithi191.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 5.1 MB/s | 384 kB 00:00 2023-10-08T16:02:07.395 INFO:teuthology.orchestra.run.smithi191.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 4.1 MB/s | 81 kB 00:00 2023-10-08T16:02:07.414 INFO:teuthology.orchestra.run.smithi191.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 604 kB/s | 11 kB 00:00 2023-10-08T16:02:07.436 INFO:teuthology.orchestra.run.smithi191.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 794 kB/s | 18 kB 00:00 2023-10-08T16:02:07.565 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:02:07.602 INFO:teuthology.orchestra.run.smithi191.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 7.0 MB/s | 1.2 MB 00:00 2023-10-08T16:02:07.621 INFO:teuthology.orchestra.run.smithi191.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 2.5 MB/s | 49 kB 00:00 2023-10-08T16:02:07.641 INFO:teuthology.orchestra.run.smithi191.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 3.0 MB/s | 59 kB 00:00 2023-10-08T16:02:07.660 INFO:teuthology.orchestra.run.smithi191.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 3.1 MB/s | 60 kB 00:00 2023-10-08T16:02:07.698 INFO:teuthology.orchestra.run.smithi191.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 7.2 MB/s | 283 kB 00:00 2023-10-08T16:02:07.717 INFO:teuthology.orchestra.run.smithi191.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 846 kB/s | 16 kB 00:00 2023-10-08T16:02:07.736 INFO:teuthology.orchestra.run.smithi191.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.8 MB/s | 34 kB 00:00 2023-10-08T16:02:07.773 INFO:teuthology.orchestra.run.smithi191.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 5.2 MB/s | 196 kB 00:00 2023-10-08T16:02:07.792 INFO:teuthology.orchestra.run.smithi191.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 3.3 MB/s | 64 kB 00:00 2023-10-08T16:02:07.811 INFO:teuthology.orchestra.run.smithi191.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 1.0 MB/s | 19 kB 00:00 2023-10-08T16:02:07.830 INFO:teuthology.orchestra.run.smithi191.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 1.3 MB/s | 24 kB 00:00 2023-10-08T16:02:07.848 INFO:teuthology.orchestra.run.smithi191.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 1.5 MB/s | 29 kB 00:00 2023-10-08T16:02:07.867 INFO:teuthology.orchestra.run.smithi191.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.4 MB/s | 27 kB 00:00 2023-10-08T16:02:07.905 INFO:teuthology.orchestra.run.smithi191.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 6.6 MB/s | 255 kB 00:00 2023-10-08T16:02:07.943 INFO:teuthology.orchestra.run.smithi191.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 6.5 MB/s | 251 kB 00:00 2023-10-08T16:02:07.962 INFO:teuthology.orchestra.run.smithi191.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 3.2 MB/s | 61 kB 00:00 2023-10-08T16:02:07.976 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:02:07.982 INFO:teuthology.orchestra.run.smithi191.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 4.3 MB/s | 85 kB 00:00 2023-10-08T16:02:08.001 INFO:teuthology.orchestra.run.smithi191.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 1.2 MB/s | 23 kB 00:00 2023-10-08T16:02:08.165 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T16:02:08.172 INFO:teuthology.orchestra.run.smithi191.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 10 MB/s | 1.7 MB 00:00 2023-10-08T16:02:08.176 INFO:teuthology.orchestra.run.smithi191.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-10-08T16:02:08.226 INFO:teuthology.orchestra.run.smithi191.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 377 kB/s | 19 kB 00:00 2023-10-08T16:02:08.258 INFO:teuthology.orchestra.run.smithi191.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-10-08T16:02:08.284 INFO:teuthology.orchestra.run.smithi191.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 5.9 MB/s | 155 kB 00:00 2023-10-08T16:02:08.299 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T16:02:08.305 INFO:teuthology.orchestra.run.smithi191.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 10 MB/s | 220 kB 00:00 2023-10-08T16:02:08.327 INFO:teuthology.orchestra.run.smithi191.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 2.1 MB/s | 48 kB 00:00 2023-10-08T16:02:08.352 INFO:teuthology.orchestra.run.smithi191.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.2 MB/s | 229 kB 00:00 2023-10-08T16:02:08.374 INFO:teuthology.orchestra.run.smithi191.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 1.9 MB/s | 43 kB 00:00 2023-10-08T16:02:08.429 INFO:teuthology.orchestra.run.smithi191.stdout:(77/120): ceph-mgr-diskprediction-local-17.2.6- 3.3 MB/s | 7.4 MB 00:02 2023-10-08T16:02:08.438 INFO:teuthology.orchestra.run.smithi191.stdout:(78/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 36 MB/s | 2.3 MB 00:00 2023-10-08T16:02:08.481 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:02:08.497 INFO:teuthology.orchestra.run.smithi191.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-10-08T16:02:08.514 INFO:teuthology.orchestra.run.smithi191.stdout:(80/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 210 kB/s | 18 kB 00:00 2023-10-08T16:02:08.535 INFO:teuthology.orchestra.run.smithi191.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 2.1 MB/s | 83 kB 00:00 2023-10-08T16:02:08.540 INFO:teuthology.orchestra.run.smithi191.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 2.3 MB/s | 59 kB 00:00 2023-10-08T16:02:08.559 INFO:teuthology.orchestra.run.smithi191.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 7.1 MB/s | 169 kB 00:00 2023-10-08T16:02:08.570 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:02:08.574 INFO:teuthology.orchestra.run.smithi191.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 1.1 MB/s | 38 kB 00:00 2023-10-08T16:02:08.595 INFO:teuthology.orchestra.run.smithi191.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 2.3 MB/s | 86 kB 00:00 2023-10-08T16:02:08.607 INFO:teuthology.orchestra.run.smithi191.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-10-08T16:02:08.647 INFO:teuthology.orchestra.run.smithi191.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 911 kB/s | 47 kB 00:00 2023-10-08T16:02:08.653 INFO:teuthology.orchestra.run.smithi191.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 12 MB/s | 559 kB 00:00 2023-10-08T16:02:08.675 INFO:teuthology.orchestra.run.smithi191.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 22 MB/s | 639 kB 00:00 2023-10-08T16:02:08.687 INFO:teuthology.orchestra.run.smithi191.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-10-08T16:02:08.710 INFO:teuthology.orchestra.run.smithi191.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 5.6 MB/s | 200 kB 00:00 2023-10-08T16:02:08.722 INFO:teuthology.orchestra.run.smithi191.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 2.7 MB/s | 97 kB 00:00 2023-10-08T16:02:08.770 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T16:02:08.781 INFO:teuthology.orchestra.run.smithi191.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 741 kB/s | 44 kB 00:00 2023-10-08T16:02:08.800 INFO:teuthology.orchestra.run.smithi191.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.4 MB/s | 27 kB 00:00 2023-10-08T16:02:08.845 INFO:teuthology.orchestra.run.smithi191.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 3.5 MB/s | 161 kB 00:00 2023-10-08T16:02:08.976 INFO:teuthology.orchestra.run.smithi191.stdout:(96/120): python3-scipy-1.0.0-20.module+el8.1.0 55 MB/s | 15 MB 00:00 2023-10-08T16:02:08.981 INFO:teuthology.orchestra.run.smithi191.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-10-08T16:02:09.007 INFO:teuthology.orchestra.run.smithi191.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 15 MB/s | 457 kB 00:00 2023-10-08T16:02:09.033 INFO:teuthology.orchestra.run.smithi191.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2023-10-08T16:02:09.058 INFO:teuthology.orchestra.run.smithi191.stdout:(100/120): python3-babel-2.5.1-5.el8.noarch.rpm 62 MB/s | 4.8 MB 00:00 2023-10-08T16:02:09.065 INFO:teuthology.orchestra.run.smithi191.stdout:(101/120): bison-3.0.4-10.el8.x86_64.rpm 21 MB/s | 688 kB 00:00 2023-10-08T16:02:09.078 INFO:teuthology.orchestra.run.smithi191.stdout:(102/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 13 MB/s | 259 kB 00:00 2023-10-08T16:02:09.086 INFO:teuthology.orchestra.run.smithi191.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 11 MB/s | 225 kB 00:00 2023-10-08T16:02:09.101 INFO:teuthology.orchestra.run.smithi191.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 5.4 MB/s | 126 kB 00:00 2023-10-08T16:02:09.103 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-08T16:02:09.123 INFO:teuthology.orchestra.run.smithi191.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.4 MB/s | 187 kB 00:00 2023-10-08T16:02:09.150 INFO:teuthology.orchestra.run.smithi191.stdout:(106/120): python3-numpy-1.14.3-9.el8.x86_64.rp 58 MB/s | 3.7 MB 00:00 2023-10-08T16:02:09.156 INFO:teuthology.orchestra.run.smithi191.stdout:(107/120): python3-jinja2-2.10.1-2.el8_0.noarch 16 MB/s | 538 kB 00:00 2023-10-08T16:02:09.172 INFO:teuthology.orchestra.run.smithi191.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-10-08T16:02:09.179 INFO:teuthology.orchestra.run.smithi191.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.6 MB/s | 157 kB 00:00 2023-10-08T16:02:09.198 INFO:teuthology.orchestra.run.smithi191.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 4.1 MB/s | 110 kB 00:00 2023-10-08T16:02:09.202 INFO:teuthology.orchestra.run.smithi191.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 15 MB/s | 353 kB 00:00 2023-10-08T16:02:09.221 INFO:teuthology.orchestra.run.smithi191.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 877 kB/s | 20 kB 00:00 2023-10-08T16:02:09.268 INFO:teuthology.orchestra.run.smithi191.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-10-08T16:02:09.303 INFO:teuthology.orchestra.run.smithi191.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 6.9 MB/s | 709 kB 00:00 2023-10-08T16:02:09.418 INFO:teuthology.orchestra.run.smithi191.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 32 MB/s | 4.7 MB 00:00 2023-10-08T16:02:09.457 INFO:teuthology.orchestra.run.smithi191.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 30 MB/s | 4.6 MB 00:00 2023-10-08T16:02:09.466 INFO:teuthology.orchestra.run.smithi191.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-10-08T16:02:09.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T16:02:09.981 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:02:10.094 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:02:10.318 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T16:02:10.481 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T16:02:10.673 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:02:10.779 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:02:10.948 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T16:02:11.015 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T16:02:11.150 INFO:teuthology.orchestra.run.smithi191.stdout:(118/120): librados2-17.2.6-1320.g08cac018.el8. 2.0 MB/s | 3.5 MB 00:01 2023-10-08T16:02:11.160 INFO:teuthology.orchestra.run.smithi191.stdout:(119/120): librbd1-17.2.6-1320.g08cac018.el8.x8 2.1 MB/s | 3.6 MB 00:01 2023-10-08T16:02:11.210 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-08T16:02:11.477 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-08T16:02:11.704 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-08T16:02:11.868 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-08T16:02:12.014 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T16:02:12.163 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T16:02:12.306 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T16:02:12.844 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T16:02:13.148 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T16:02:13.675 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T16:02:13.881 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T16:02:14.609 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T16:02:14.884 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-08T16:02:15.156 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-08T16:02:15.392 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T16:02:15.545 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T16:02:15.699 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T16:02:15.977 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T16:02:16.195 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-08T16:02:16.498 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-08T16:02:16.722 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T16:02:17.216 INFO:teuthology.orchestra.run.smithi191.stdout:(120/120): ceph-test-17.2.6-1320.g08cac018.el8. 3.0 MB/s | 52 MB 00:17 2023-10-08T16:02:17.220 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2023-10-08T16:02:17.221 INFO:teuthology.orchestra.run.smithi191.stdout:Total 7.8 MB/s | 202 MB 00:25 2023-10-08T16:02:17.271 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2023-10-08T16:02:17.386 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2023-10-08T16:02:17.386 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2023-10-08T16:02:18.796 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T16:02:19.113 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2023-10-08T16:02:19.117 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2023-10-08T16:02:19.153 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-08T16:02:20.169 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-08T16:02:20.500 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-08T16:02:20.766 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-08T16:02:21.011 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T16:02:21.208 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T16:02:21.465 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T16:02:21.669 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T16:02:21.931 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T16:02:22.145 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-08T16:02:22.312 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-08T16:02:22.510 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-08T16:02:22.705 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-08T16:02:22.863 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T16:02:22.923 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T16:02:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-08T16:02:23.251 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-08T16:02:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-08T16:02:23.546 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2023-10-08T16:02:23.592 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-08T16:02:23.740 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-08T16:02:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T16:02:23.890 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-08T16:02:23.968 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T16:02:24.049 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-08T16:02:24.102 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T16:02:24.248 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-08T16:02:24.318 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T16:02:24.416 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-08T16:02:24.706 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T16:02:24.766 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-08T16:02:24.962 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:02:25.008 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-08T16:02:25.258 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-08T16:02:25.348 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:02:25.550 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-08T16:02:25.584 INFO:teuthology.orchestra.run.smithi191.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T16:02:25.712 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T16:02:25.740 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T16:02:25.885 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T16:02:25.926 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:02:26.027 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:02:26.167 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T16:02:26.198 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T16:02:26.379 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T16:02:26.524 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-08T16:02:26.610 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T16:02:26.773 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T16:02:26.939 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T16:02:27.137 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T16:02:27.195 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T16:02:27.359 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:02:27.374 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T16:02:27.469 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:02:27.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T16:02:27.705 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T16:02:27.790 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T16:02:27.883 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T16:02:27.976 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T16:02:28.118 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:02:28.190 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T16:02:28.229 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:02:28.390 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T16:02:28.401 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T16:02:28.483 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T16:02:28.626 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T16:02:28.672 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-08T16:02:28.898 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T16:02:28.964 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-08T16:02:29.177 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T16:02:29.183 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-08T16:02:29.336 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-08T16:02:29.369 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T16:02:29.493 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T16:02:29.593 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T16:02:29.617 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T16:02:29.792 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T16:02:30.346 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T16:02:30.534 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T16:02:30.743 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T16:02:31.034 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T16:02:31.069 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T16:02:31.271 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T16:02:31.301 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T16:02:31.596 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T16:02:31.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T16:02:32.016 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T16:02:32.184 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T16:02:32.278 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-08T16:02:32.417 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T16:02:32.525 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-08T16:02:32.587 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T16:02:32.786 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T16:02:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T16:02:32.949 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T16:02:33.085 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T16:02:33.357 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T16:02:33.581 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-08T16:02:33.849 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-08T16:02:34.050 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T16:02:36.116 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T16:02:36.488 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-08T16:02:37.480 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-08T16:02:37.818 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-08T16:02:37.903 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:38.153 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:38.437 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-08T16:02:38.690 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T16:02:38.729 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:38.931 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T16:02:39.146 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T16:02:39.278 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T16:02:39.338 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T16:02:39.384 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T16:02:39.519 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T16:02:39.646 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T16:02:39.873 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-08T16:02:40.065 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-08T16:02:40.257 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-08T16:02:40.477 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-08T16:02:40.649 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T16:02:40.727 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T16:02:40.899 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-08T16:02:41.047 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-08T16:02:41.256 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-08T16:02:41.411 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-08T16:02:41.584 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-08T16:02:41.743 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-08T16:02:41.902 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-08T16:02:42.098 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-08T16:02:42.260 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-08T16:02:42.563 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-08T16:02:42.852 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-08T16:02:43.128 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-08T16:02:43.440 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-08T16:02:43.590 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T16:02:43.778 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T16:02:44.020 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T16:02:44.273 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T16:02:44.488 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T16:02:44.653 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T16:02:44.809 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T16:02:45.033 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T16:02:45.260 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T16:02:45.492 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T16:02:45.684 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T16:02:45.896 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T16:02:46.115 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T16:02:46.353 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T16:02:46.596 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T16:02:46.896 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T16:02:47.156 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T16:02:47.382 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T16:02:47.634 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T16:02:48.780 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T16:02:49.111 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T16:02:49.341 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T16:02:49.661 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T16:02:49.952 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T16:02:50.286 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T16:02:50.548 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T16:02:50.747 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T16:02:50.948 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T16:02:55.941 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:56.172 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:56.767 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T16:02:57.035 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T16:02:57.137 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T16:02:57.313 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T16:02:57.318 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2023-10-08T16:02:57.319 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:02:57.674 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T16:02:59.162 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T16:02:59.350 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T16:02:59.575 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T16:02:59.748 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T16:03:00.137 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T16:03:00.274 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T16:03:00.290 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T16:03:03.146 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T16:03:03.319 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T16:03:03.479 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T16:03:03.647 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T16:03:03.840 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T16:03:04.055 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T16:03:04.257 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T16:03:05.227 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T16:03:05.398 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T16:03:05.496 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T16:03:05.889 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T16:03:06.057 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T16:03:06.094 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T16:03:06.239 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T16:03:06.250 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T16:03:07.871 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T16:03:08.099 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T16:03:08.189 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T16:03:08.347 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T16:03:08.390 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T16:03:08.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T16:03:08.895 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T16:03:09.081 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T16:03:13.990 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T16:03:14.959 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T16:03:15.111 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T16:03:15.111 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /sys 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /proc 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /run 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /mnt 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /var/tmp 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /home 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /tmp 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout:skipping the directory /dev 2023-10-08T16:03:15.112 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:15.306 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-08T16:03:15.476 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T16:03:15.547 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 110/122 2023-10-08T16:03:15.758 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-08T16:03:15.896 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T16:03:15.907 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T16:03:16.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T16:03:16.226 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T16:03:16.379 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T16:03:16.626 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T16:03:16.950 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T16:03:16.959 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T16:03:17.121 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T16:03:17.123 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T16:03:17.180 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T16:03:17.345 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T16:03:17.387 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T16:03:17.395 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T16:03:17.506 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-08T16:03:17.506 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T16:03:17.530 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T16:03:17.707 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T16:03:17.707 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:17.920 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T16:03:18.067 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T16:03:18.084 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:03:18.622 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T16:03:18.623 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T16:03:18.624 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T16:03:18.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T16:03:18.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T16:03:18.632 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T16:03:18.633 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T16:03:18.634 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T16:03:18.635 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-08T16:03:18.636 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-08T16:03:18.637 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-08T16:03:18.638 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-08T16:03:18.639 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T16:03:18.640 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T16:03:20.799 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:20.799 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:01:46 ago on Sun 08 Oct 2023 04:01:34 PM UTC. 2023-10-08T16:03:20.933 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T16:03:21.075 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T16:03:21.256 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T16:03:21.469 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T16:03:21.650 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T16:03:21.865 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T16:03:22.113 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T16:03:22.906 INFO:teuthology.orchestra.run.smithi170.stdout:Installed products updated. 2023-10-08T16:03:23.081 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T16:03:23.087 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:23.087 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.088 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.089 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T16:03:23.090 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.5-12.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T16:03:23.091 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T16:03:23.092 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T16:03:23.093 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T16:03:23.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T16:03:23.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T16:03:23.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.097 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:23.098 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2023-10-08T16:03:23.241 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T16:03:23.336 DEBUG:teuthology.parallel:result is None 2023-10-08T16:03:23.338 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T16:03:23.694 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T16:03:23.863 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T16:03:23.901 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T16:03:24.082 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T16:03:24.094 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T16:03:25.712 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T16:03:25.988 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T16:03:26.067 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T16:03:26.199 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T16:03:26.242 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T16:03:26.408 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T16:03:26.754 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T16:03:26.924 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T16:03:31.846 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T16:03:32.798 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T16:03:33.174 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-08T16:03:33.377 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 110/122 2023-10-08T16:03:33.599 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-08T16:03:33.764 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T16:03:33.776 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T16:03:33.927 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T16:03:34.061 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T16:03:34.230 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T16:03:34.431 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T16:03:34.686 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T16:03:34.880 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T16:03:34.938 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T16:03:35.156 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T16:03:35.164 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T16:03:35.291 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-08T16:03:35.291 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T16:03:35.492 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T16:03:35.492 INFO:teuthology.orchestra.run.smithi191.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:36.356 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:36.356 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T16:03:36.357 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T16:03:36.358 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T16:03:36.359 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T16:03:36.367 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T16:03:36.368 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T16:03:36.369 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-08T16:03:36.370 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-08T16:03:36.371 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-08T16:03:36.372 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-08T16:03:36.373 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T16:03:36.374 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T16:03:38.501 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T16:03:38.501 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:56 ago on Sun 08 Oct 2023 04:01:42 PM UTC. 2023-10-08T16:03:40.579 INFO:teuthology.orchestra.run.smithi191.stdout:Installed products updated. 2023-10-08T16:03:40.729 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:40.730 INFO:teuthology.orchestra.run.smithi191.stdout:Upgraded: 2023-10-08T16:03:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.730 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2023-10-08T16:03:40.730 INFO:teuthology.orchestra.run.smithi191.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T16:03:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T16:03:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.731 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.732 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.733 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.734 INFO:teuthology.orchestra.run.smithi191.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T16:03:40.735 INFO:teuthology.orchestra.run.smithi191.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: jq-1.5-12.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.736 INFO:teuthology.orchestra.run.smithi191.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T16:03:40.737 INFO:teuthology.orchestra.run.smithi191.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T16:03:40.738 INFO:teuthology.orchestra.run.smithi191.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T16:03:40.739 INFO:teuthology.orchestra.run.smithi191.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T16:03:40.740 INFO:teuthology.orchestra.run.smithi191.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T16:03:40.741 INFO:teuthology.orchestra.run.smithi191.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T16:03:40.742 INFO:teuthology.orchestra.run.smithi191.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T16:03:40.743 INFO:teuthology.orchestra.run.smithi191.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T16:03:40.744 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T16:03:40.745 INFO:teuthology.orchestra.run.smithi191.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T16:03:40.746 INFO:teuthology.orchestra.run.smithi191.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T16:03:40.747 INFO:teuthology.orchestra.run.smithi191.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T16:03:40.748 INFO:teuthology.orchestra.run.smithi191.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T16:03:40.748 INFO:teuthology.orchestra.run.smithi191.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T16:03:40.748 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:40.748 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2023-10-08T16:03:41.025 DEBUG:teuthology.parallel:result is None 2023-10-08T16:03:41.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T16:03:41.244 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T16:03:41.270 INFO:teuthology.orchestra.run.smithi170.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T16:03:41.270 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T16:03:41.270 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T16:03:41.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T16:03:41.407 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T16:03:41.429 INFO:teuthology.orchestra.run.smithi191.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T16:03:41.429 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T16:03:41.429 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T16:03:41.429 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T16:03:41.429 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:41.429 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T16:03:41.474 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:41.474 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T16:03:41.514 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T16:03:41.514 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:41.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T16:03:41.541 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T16:03:41.607 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:41.607 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T16:03:41.631 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T16:03:41.696 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T16:03:41.696 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:41.696 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T16:03:41.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T16:03:41.786 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:41.786 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T16:03:41.810 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T16:03:41.874 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T16:03:41.874 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:41.875 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T16:03:41.899 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T16:03:41.963 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:41.963 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T16:03:41.987 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T16:03:42.051 INFO:teuthology.run_tasks:Running task ceph... 2023-10-08T16:03:42.107 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-08T16:03:42.108 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2023-10-08T16:03:42.110 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /var/log/ceph 2023-10-08T16:03:42.133 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-08T16:03:42.134 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T16:03:42.174 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T16:03:42.198 INFO:tasks.ceph:Creating extra log directories... 2023-10-08T16:03:42.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T16:03:42.238 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T16:03:42.267 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-08T16:03:42.268 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-08T16:03:42.268 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-08T16:03:42.268 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:42.268 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T16:03:42.314 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T16:03:42.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T16:03:42.448 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:42.449 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T16:03:42.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T16:03:42.576 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T16:03:42.606 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-08T16:03:42.606 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-08T16:03:42.606 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}}', 'email': None, 'first_in_suite': False, 'job_id': '7418298', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006', 'targets': {'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5J24LseHkEJqG75H9ewnuQ/awSKU1uoWWfrr4V/Kg3ryQWLjAKjqT4KloPDDmyWqC8zjxZYvVOpIw63sAzbkQ=', 'smithi191.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6W+dSuRSn1eeAOlPjZ9dUCkVo104fXgNegX76mlc9/XMmDaVHiwpXLJotSmstk8zg6q7J4uiG1RHLTRlQ6IRA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestVolumes', 'tasks.cephfs.test_volumes.TestSubvolumeGroups', 'tasks.cephfs.test_volumes.TestSubvolumes']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-08_15:41:30', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-08T16:03:42.607 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T16:03:42.609 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T16:03:42.655 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T16:03:42.663 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T16:03:42.720 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:42.720 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T16:03:42.739 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T16:03:42.740 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 215005 Links: 1 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-08 16:03:18.192799396 +0000 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-08 16:00:45.704848849 +0000 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-08 16:00:45.704848849 +0000 2023-10-08T16:03:42.795 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-08T16:03:42.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T16:03:42.857 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-08T16:03:42.858 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-08T16:03:42.858 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000117882 s, 4.3 MB/s 2023-10-08T16:03:42.859 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T16:03:42.914 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2023-10-08T16:03:42.970 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T16:03:42.970 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 223239 Links: 1 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-08 16:03:18.192799396 +0000 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-08 16:00:46.179837944 +0000 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-08 16:00:46.179837944 +0000 2023-10-08T16:03:42.971 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-08T16:03:42.971 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T16:03:43.033 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-08T16:03:43.033 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-08T16:03:43.033 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000112917 s, 4.5 MB/s 2023-10-08T16:03:43.034 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T16:03:43.090 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2023-10-08T16:03:43.144 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T16:03:43.144 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 224287 Links: 1 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-08 16:03:18.193799373 +0000 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-08 16:00:46.654827038 +0000 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-08 16:00:46.654827038 +0000 2023-10-08T16:03:43.145 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-08T16:03:43.145 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T16:03:43.207 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-08T16:03:43.207 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-08T16:03:43.207 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000127706 s, 4.0 MB/s 2023-10-08T16:03:43.208 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T16:03:43.264 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 220559 Links: 1 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:43.319 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2023-10-08 16:03:18.193799373 +0000 2023-10-08T16:03:43.320 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2023-10-08 16:00:47.138815927 +0000 2023-10-08T16:03:43.320 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2023-10-08 16:00:47.138815927 +0000 2023-10-08T16:03:43.320 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2023-10-08T16:03:43.320 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T16:03:43.384 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2023-10-08T16:03:43.384 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2023-10-08T16:03:43.384 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.4543e-05 s, 5.4 MB/s 2023-10-08T16:03:43.385 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T16:03:43.440 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:43.440 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:43.441 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T16:03:43.456 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T16:03:43.456 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2023-10-08T16:03:43.511 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T16:03:43.511 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:43.511 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 222249 Links: 1 2023-10-08T16:03:43.511 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:43.512 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:43.512 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 16:03:35.946892300 +0000 2023-10-08T16:03:43.512 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 16:00:45.700477253 +0000 2023-10-08T16:03:43.512 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 16:00:45.700477253 +0000 2023-10-08T16:03:43.512 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T16:03:43.512 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T16:03:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T16:03:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T16:03:43.576 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000248904 s, 2.1 MB/s 2023-10-08T16:03:43.577 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T16:03:43.633 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 212638 Links: 1 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:43.689 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 16:03:35.946892300 +0000 2023-10-08T16:03:43.690 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 16:00:46.183469054 +0000 2023-10-08T16:03:43.690 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 16:00:46.183469054 +0000 2023-10-08T16:03:43.690 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T16:03:43.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T16:03:43.752 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T16:03:43.752 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T16:03:43.752 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000229423 s, 2.2 MB/s 2023-10-08T16:03:43.753 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T16:03:43.808 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 216848 Links: 1 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 16:03:35.946892300 +0000 2023-10-08T16:03:43.864 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 16:00:46.668460821 +0000 2023-10-08T16:03:43.865 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 16:00:46.668460821 +0000 2023-10-08T16:03:43.865 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T16:03:43.865 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T16:03:43.926 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T16:03:43.926 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T16:03:43.926 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000238997 s, 2.1 MB/s 2023-10-08T16:03:43.927 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T16:03:43.983 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2023-10-08T16:03:44.038 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T16:03:44.038 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T16:03:44.038 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 6h/6d Inode: 216907 Links: 1 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2023-10-08 16:03:35.946892300 +0000 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2023-10-08 16:00:47.153452589 +0000 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2023-10-08 16:00:47.153452589 +0000 2023-10-08T16:03:44.039 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: - 2023-10-08T16:03:44.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T16:03:44.101 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2023-10-08T16:03:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2023-10-08T16:03:44.102 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.00019854 s, 2.6 MB/s 2023-10-08T16:03:44.103 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T16:03:44.158 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:44.158 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T16:03:44.158 INFO:tasks.ceph:Generating config... 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] debug client = 20 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] debug ms = 1 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] pid file = 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-08T16:03:44.160 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-08T16:03:44.160 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-08T16:03:44.160 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-08T16:03:44.160 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-08T16:03:44.160 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-08T16:03:44.161 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-08T16:03:44.162 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-08T16:03:44.162 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-08T16:03:44.162 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-08T16:03:44.163 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-08T16:03:44.163 INFO:tasks.ceph:Setting up mon.a... 2023-10-08T16:03:44.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-08T16:03:44.278 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.keyring 2023-10-08T16:03:44.281 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-08T16:03:44.327 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T16:03:44.351 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.170'), ('mon.b', '172.21.15.191'), ('mon.c', '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]')] 2023-10-08T16:03:44.352 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-08T16:03:44.353 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:44.353 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T16:03:44.407 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.170 --add b 172.21.15.191 --addv c '[v2:172.21.15.191:3301,v1:172.21.15.191:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:44.519 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: generated fsid a2f97f1a-22f1-43be-8b23-d4161b44866d 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:setting min_mon_release = octopus 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:epoch 0 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:fsid a2f97f1a-22f1-43be-8b23-d4161b44866d 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:last_changed 2023-10-08T16:03:44.520300+0000 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:created 2023-10-08T16:03:44.520300+0000 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:min_mon_release 15 (octopus) 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:election_strategy: 1 2023-10-08T16:03:44.520 INFO:teuthology.orchestra.run.smithi170.stdout:0: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.a 2023-10-08T16:03:44.521 INFO:teuthology.orchestra.run.smithi170.stdout:1: [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] mon.b 2023-10-08T16:03:44.521 INFO:teuthology.orchestra.run.smithi170.stdout:2: [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] mon.c 2023-10-08T16:03:44.521 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-08T16:03:44.522 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T16:03:44.535 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a2f97f1a-22f1-43be-8b23-d4161b44866d... 2023-10-08T16:03:44.537 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T16:03:44.578 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T16:03:44.618 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.699 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2023-10-08T16:03:44.700 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.701 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on skipped message = False 2023-10-08T16:03:44.702 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = a2f97f1a-22f1-43be-8b23-d4161b44866d 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.703 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.781 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2023-10-08T16:03:44.782 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore allocator = bitmap 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2023-10-08T16:03:44.783 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2023-10-08T16:03:44.784 INFO:teuthology.orchestra.run.smithi170.stdout: osd op complaint time = 180 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: debug finisher = 20 2023-10-08T16:03:44.785 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2023-10-08T16:03:44.786 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.787 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: mon op complaint time = 120 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: client mount timeout = 600 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2023-10-08T16:03:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = "" 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout: rados mon op timeout = 900 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout: rados osd op timeout = 900 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout:[mds] 2023-10-08T16:03:44.789 INFO:teuthology.orchestra.run.smithi170.stdout: debug mds = 20 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: mds debug frag = True 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: mds debug scatterstat = True 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: mds op complaint time = 180 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: mds verify scatter = True 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: osd op complaint time = 180 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: rados mon op timeout = 900 2023-10-08T16:03:44.790 INFO:teuthology.orchestra.run.smithi170.stdout: rados osd op timeout = 900 2023-10-08T16:03:44.794 INFO:teuthology.orchestra.run.smithi191.stdout:[global] 2023-10-08T16:03:44.794 INFO:teuthology.orchestra.run.smithi191.stdout: chdir = "" 2023-10-08T16:03:44.794 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T16:03:44.794 INFO:teuthology.orchestra.run.smithi191.stdout: auth supported = cephx 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: filestore xattr use omap = true 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: mon clock drift allowed = 1.000 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: osd crush chooseleaf type = 0 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: auth debug = true 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on old message = true 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on bug = true 2023-10-08T16:03:44.795 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: mon pg warn max object skew = 0 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default size = 2 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow primary affinity = true 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd allow pg remap = true 2023-10-08T16:03:44.796 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on legacy crush tunables = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on crush straw calc version zero = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on no sortbitwise = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on osd down out interval zero = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon warn on too few osds = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: mon_allow_pool_size_one = true 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T16:03:44.797 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: osd default data pool replay window = 5 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: mon allow pool delete = true 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: mon cluster log file level = debug 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: debug asserts on shutdown = true 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: mon health detail to clog = false 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: mon host = "172.21.15.170,172.21.15.191,[v2:172.21.15.191:3301,v1:172.21.15.191:6790]" 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: ms die on skipped message = False 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: fsid = a2f97f1a-22f1-43be-8b23-d4161b44866d 2023-10-08T16:03:44.798 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout:[osd] 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd journal size = 100 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub load threshold = 5.0 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd scrub max interval = 600 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd mclock profile = high_recovery_ops 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd recover clone overlap = true 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd recovery max chunk = 1048576 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.799 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug shutdown = true 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug op order = true 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug verify stray on activate = true 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd open classes on start = true 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug pg log writeout = true 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd deep scrub update digest min age = 30 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: osd map max advance = 10 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.800 INFO:teuthology.orchestra.run.smithi191.stdout: journal zero on create = true 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: filestore ondisk finisher threads = 3 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: filestore apply finisher threads = 3 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: bdev debug aio = true 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: osd debug misdirected ops = true 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: bdev async discard = True 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: bdev enable discard = True 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore allocator = bitmap 2023-10-08T16:03:44.801 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore block size = 96636764160 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: bluestore fsck on mount = True 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluefs = 1/20 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: debug bluestore = 1/20 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: debug osd = 20 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: debug rocksdb = 4/10 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd full ratio = 0.9 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd nearfull ratio = 0.8 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: osd failsafe full ratio = 0.95 2023-10-08T16:03:44.802 INFO:teuthology.orchestra.run.smithi191.stdout: osd objectstore = bluestore 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr] 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: debug mgr = 20 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: mgr/telemetry/nag = false 2023-10-08T16:03:44.803 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: debug finisher = 20 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout:[mon] 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: debug mon = 20 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: debug paxos = 20 2023-10-08T16:03:44.804 INFO:teuthology.orchestra.run.smithi191.stdout: debug auth = 20 2023-10-08T16:03:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: mon data avail warn = 5 2023-10-08T16:03:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: mon mgr mkfs grace = 240 2023-10-08T16:03:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min pgs per osd = 4 2023-10-08T16:03:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd reporter subtree level = osd 2023-10-08T16:03:44.846 INFO:teuthology.orchestra.run.smithi191.stdout: mon osd prime pg temp = true 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: mon reweight min bytes per osd = 10 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: # 1m isn't quite enough 2023-10-08T16:03:44.847 INFO:teuthology.orchestra.run.smithi191.stdout: mon_down_mkfs_grace = 2m 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: mon_warn_on_filestore_osds = false 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: mon op complaint time = 120 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout:[client] 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: rgw cache enabled = true 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable ops log = true 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: rgw enable usage log = true 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T16:03:44.848 INFO:teuthology.orchestra.run.smithi191.stdout: client mount timeout = 600 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: debug client = 20 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: pid file = "" 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.a] 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.b] 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mon.c] 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2023-10-08T16:03:44.849 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout:[mds] 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: debug mds = 20 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: debug ms = 1 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug frag = True 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: mds debug scatterstat = True 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: mds op complaint time = 180 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: mds verify scatter = True 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: osd op complaint time = 180 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: rados mon op timeout = 900 2023-10-08T16:03:44.850 INFO:teuthology.orchestra.run.smithi191.stdout: rados osd op timeout = 900 2023-10-08T16:03:44.851 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-08T16:03:44.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-08T16:03:44.942 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-08T16:03:45.198 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:45.198 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-08T16:03:45.215 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:45.216 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-08T16:03:45.273 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:03:45.273 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:45.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T16:03:45.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T16:03:45.535 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:03:45.536 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:45.589 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:03:45.589 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:45.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T16:03:45.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T16:03:45.626 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:03:45.626 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:45.681 INFO:tasks.ceph:Setting up mon nodes... 2023-10-08T16:03:45.681 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-08T16:03:45.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T16:03:45.737 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T16:03:45.740 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T16:03:45.805 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T16:03:45.808 INFO:tasks.ceph:Setting up mds nodes... 2023-10-08T16:03:45.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T16:03:45.862 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T16:03:45.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-08T16:03:45.889 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T16:03:45.978 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T16:03:45.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-08T16:03:46.004 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T16:03:46.053 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T16:03:46.054 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-08T16:03:46.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T16:03:46.165 INFO:teuthology.orchestra.run.smithi191.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T16:03:46.167 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-08T16:03:46.190 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-08T16:03:46.191 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T16:03:46.228 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-08T16:03:46.241 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-08T16:03:46.318 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-08T16:03:46.331 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-08T16:03:46.368 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-08T16:03:46.381 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-08T16:03:46.458 INFO:teuthology.orchestra.run.smithi191.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-08T16:03:46.470 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-08T16:03:46.470 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi191.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T16:03:46.471 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-08T16:03:46.495 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:47.174 INFO:tasks.ceph:role: osd.0 2023-10-08T16:03:47.174 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:03:47.175 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:03:47.200 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:03:47.201 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:03:47.201 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:03:47.201 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:03:47.201 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:03:47.225 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-08T16:03:47.227 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-08T16:03:47.228 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-08T16:03:47.305 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-08T16:03:47.750 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-08T16:03:47.814 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:47.814 INFO:tasks.ceph:role: osd.1 2023-10-08T16:03:47.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:03:47.815 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:03:47.904 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-08T16:03:47.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-08T16:03:47.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-08T16:03:47.983 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-08T16:03:48.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-08T16:03:48.115 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:48.115 INFO:tasks.ceph:role: osd.2 2023-10-08T16:03:48.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:03:48.115 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T16:03:48.178 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:03:48.179 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:03:48.202 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-08T16:03:48.203 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-08T16:03:48.203 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-08T16:03:48.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-08T16:03:48.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-08T16:03:48.412 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T16:03:48.412 INFO:tasks.ceph:role: osd.3 2023-10-08T16:03:48.412 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:03:48.412 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T16:03:48.475 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:03:48.476 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:03:48.498 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2023-10-08T16:03:48.500 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2023-10-08T16:03:48.500 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-08T16:03:48.576 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-08T16:03:48.644 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.742+0000 7f5f7a833540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.742+0000 7f5f7a833540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.742+0000 7f5f7a833540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.742+0000 7f5f7a833540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.742+0000 7f5f7a833540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T16:03:48.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:48.743+0000 7f5f7a833540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-08T16:03:49.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:49.271+0000 7f5f7a833540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:50.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:50.032+0000 7f5f7a833540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:50.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:50.798+0000 7f5f7a833540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:51.599 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-08T16:03:51.628 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:51.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-08T16:03:51.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-08T16:03:51.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T16:03:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T16:03:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T16:03:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:51.713+0000 7f134eaf3540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-08T16:03:52.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:52.235+0000 7f134eaf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:52.996+0000 7f134eaf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:53.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:53.758+0000 7f134eaf3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:54.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-08T16:03:54.612 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:54.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.690+0000 7f6682ce3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-08T16:03:54.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.690+0000 7f6682ce3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-08T16:03:54.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.690+0000 7f6682ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T16:03:54.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.691+0000 7f6682ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T16:03:54.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.691+0000 7f6682ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T16:03:54.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:54.691+0000 7f6682ce3540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-08T16:03:55.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:55.216+0000 7f6682ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:55.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:55.978+0000 7f6682ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:56.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:56.740+0000 7f6682ce3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:57.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-08T16:03:57.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T16:03:57.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:57.676+0000 7f90c29e7540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-08T16:03:58.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:58.198+0000 7f90c29e7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:58.959+0000 7f90c29e7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:03:59.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:03:59.722+0000 7f90c29e7540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:00.515 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-08T16:04:00.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-08T16:04:00.608 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T16:04:00.609 INFO:tasks.ceph:role: osd.4 2023-10-08T16:04:00.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:04:00.609 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T16:04:00.681 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:04:00.682 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:04:01.112 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T16:04:01.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T16:04:01.114 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-08T16:04:01.303 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-08T16:04:01.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-08T16:04:01.399 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T16:04:01.399 INFO:tasks.ceph:role: osd.5 2023-10-08T16:04:01.400 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:04:01.400 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:04:01.471 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:04:01.472 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:04:01.472 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:04:01.472 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:04:01.895 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T16:04:01.897 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T16:04:01.897 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-08T16:04:01.995 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-08T16:04:02.023 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-08T16:04:02.088 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T16:04:02.088 INFO:tasks.ceph:role: osd.6 2023-10-08T16:04:02.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:04:02.088 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T16:04:02.159 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:04:02.160 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:04:02.601 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T16:04:02.603 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T16:04:02.603 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-08T16:04:02.782 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-08T16:04:02.812 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-08T16:04:02.878 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T16:04:02.878 INFO:tasks.ceph:role: osd.7 2023-10-08T16:04:02.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:04:02.879 DEBUG:teuthology.orchestra.run.smithi191:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T16:04:02.951 INFO:teuthology.orchestra.run.smithi191.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T16:04:02.951 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout: = reflink=1 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout: = sunit=0 swidth=0 blks 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T16:04:02.952 INFO:teuthology.orchestra.run.smithi191.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T16:04:03.391 INFO:teuthology.orchestra.run.smithi191.stdout:Discarding blocks...Done. 2023-10-08T16:04:03.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi191.front.sepia.ceph.com -o noatime 2023-10-08T16:04:03.393 DEBUG:teuthology.orchestra.run.smithi191:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-08T16:04:03.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-08T16:04:03.591 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:04.037 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-08T16:04:04.037 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-08T16:04:04.037 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T16:04:04.038 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T16:04:04.038 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T16:04:04.038 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.037+0000 7f3af967b540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-08T16:04:04.568 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:04.568+0000 7f3af967b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:05.331 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:05.331+0000 7f3af967b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:06.096 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.096+0000 7f3af967b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:06.885 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-08T16:04:06.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:06.996 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.995+0000 7fa343111540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-08T16:04:06.996 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.996+0000 7fa343111540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-08T16:04:06.996 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.996+0000 7fa343111540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T16:04:06.996 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.996+0000 7fa343111540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T16:04:06.997 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.996+0000 7fa343111540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T16:04:06.997 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:06.996+0000 7fa343111540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-08T16:04:07.519 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:07.519+0000 7fa343111540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:08.281 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:08.281+0000 7fa343111540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:09.044 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.044+0000 7fa343111540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:09.838 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-08T16:04:09.904 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.985+0000 7f629e850540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.986+0000 7f629e850540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.986+0000 7f629e850540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.986+0000 7f629e850540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.986+0000 7f629e850540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T16:04:09.986 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:09.986+0000 7f629e850540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-08T16:04:10.511 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:10.511+0000 7f629e850540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:11.278 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:11.278+0000 7f629e850540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:12.041 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.040+0000 7f629e850540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:12.608 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:04:12.610 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:04:12.836 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-08T16:04:12.861 DEBUG:teuthology.orchestra.run.smithi191:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.943+0000 7f1dc37b6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.944+0000 7f1dc37b6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.944+0000 7f1dc37b6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.944+0000 7f1dc37b6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.944+0000 7f1dc37b6540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T16:04:12.944 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:12.944+0000 7f1dc37b6540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-08T16:04:13.467 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:13.467+0000 7f1dc37b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:14.229 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:14.229+0000 7f1dc37b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:14.992 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:14.992+0000 7f1dc37b6540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T16:04:15.786 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-08T16:04:15.850 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-08T16:04:15.851 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:15.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-08T16:04:15.878 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:15.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-08T16:04:15.943 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:15.944 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-08T16:04:16.009 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.009 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-08T16:04:16.074 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.074 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-08T16:04:16.139 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-08T16:04:16.203 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.203 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-08T16:04:16.267 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.267 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-08T16:04:16.291 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.291 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-08T16:04:16.355 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.355 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-08T16:04:16.420 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.420 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-08T16:04:16.483 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-08T16:04:16.547 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.548 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-08T16:04:16.612 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.612 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-08T16:04:16.675 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.676 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-08T16:04:16.690 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:16.690 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-08T16:04:16.745 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.745 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-08T16:04:16.759 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:16.759 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-08T16:04:16.813 INFO:tasks.ceph:Adding keys to all mons... 2023-10-08T16:04:16.813 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T16:04:16.815 DEBUG:teuthology.orchestra.run.smithi191:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T16:04:16.857 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2023-10-08T16:04:16.857 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA== 2023-10-08T16:04:16.857 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2023-10-08T16:04:16.857 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ== 2023-10-08T16:04:16.858 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ== 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw== 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q== 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw== 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg== 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2023-10-08T16:04:16.859 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw== 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg== 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg== 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout:[client.2] 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ== 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg== 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout:[client.3] 2023-10-08T16:04:16.860 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g== 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.y] 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA== 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.a] 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g== 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.c] 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg== 2023-10-08T16:04:16.876 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.0] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.1] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.2] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.3] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[mgr.x] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.b] 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw== 2023-10-08T16:04:16.877 INFO:teuthology.orchestra.run.smithi191.stdout:[mds.d] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q== 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.4] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw== 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.5] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg== 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.6] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw== 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:[osd.7] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg== 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout:[client.0] 2023-10-08T16:04:16.878 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg== 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ== 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout:[client.1] 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg== 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout:[client.3] 2023-10-08T16:04:16.879 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g== 2023-10-08T16:04:16.880 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T16:04:16.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T16:04:16.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:16.969 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.021 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.070 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.120 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.122 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.171 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.219 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T16:04:17.269 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T16:04:17.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.320 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.369 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.371 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T16:04:17.421 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.424 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.470 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.472 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.568 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.570 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T16:04:17.617 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.619 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.671 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.723 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.772 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.775 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T16:04:17.824 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-08T16:04:17.825 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-08T16:04:17.848 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T16:04:17.940 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-08T16:04:17.964 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-08T16:04:17.989 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T16:04:18.079 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-08T16:04:18.103 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-08T16:04:18.167 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T16:04:18.258 DEBUG:teuthology.orchestra.run.smithi191:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-08T16:04:18.281 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:18.283 DEBUG:teuthology.orchestra.run.smithi191:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T16:04:18.336 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-08T16:04:18.336 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-08T16:04:18.336 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-08T16:04:18.338 INFO:tasks.ceph.mon.a:Started 2023-10-08T16:04:18.338 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-08T16:04:18.339 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-08T16:04:18.378 INFO:tasks.ceph.mon.b:Started 2023-10-08T16:04:18.378 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-08T16:04:18.378 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-08T16:04:18.380 INFO:tasks.ceph.mon.c:Started 2023-10-08T16:04:18.380 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-08T16:04:18.380 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T16:04:18.380 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T16:04:18.382 INFO:tasks.ceph.mgr.y:Started 2023-10-08T16:04:18.383 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T16:04:18.383 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T16:04:18.384 INFO:tasks.ceph.mgr.x:Started 2023-10-08T16:04:18.385 INFO:tasks.ceph:Setting crush tunables to default 2023-10-08T16:04:18.385 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd crush tunables default 2023-10-08T16:04:18.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.500+0000 7f4a4759e700 1 Processor -- start 2023-10-08T16:04:18.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.500+0000 7f4a4759e700 1 -- start start 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480cfb00 con 0x7f4a480b16d0 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a480b5c40 con 0x7f4a480b1b40 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b5f20 con 0x7f4a480b4e70 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:51566/0 (socket says 172.21.15.170:51566) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:45012/0 (socket says 172.21.15.170:45012) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a45d9b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f4a480b16d0 0x7f4a4815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:32914/0 (socket says 172.21.15.170:32914) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 -- 172.21.15.170:0/2051125139 learned_addr learned my addr 172.21.15.170:0/2051125139 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 -- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 msgr2=0x7f4a4815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 -- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 msgr2=0x7f4a4815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 --2- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 msgr2=0x7f4a480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 msgr2=0x7f4a480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a46d9d700 1 --2- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T16:04:18.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.501+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T16:04:18.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.685+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999762041 0 0) 0x7f4a480cfb00 con 0x7f4a480b16d0 2023-10-08T16:04:18.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.685+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a34003570 con 0x7f4a480b16d0 2023-10-08T16:04:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3225534017 0 0) 0x7f4a34003570 con 0x7f4a480b16d0 2023-10-08T16:04:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 msgr2=0x7f4a480ccff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T16:04:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a480ccff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 msgr2=0x7f4a4815d950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 --2- 172.21.15.170:0/2051125139 >> v2:172.21.15.191:3300/0 conn(0x7f4a480b1b40 0x7f4a4815d950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480b6200 con 0x7f4a480b16d0 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4a30001470 con 0x7f4a480b16d0 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 >> v1:172.21.15.191:6789/0 conn(0x7f4a480b16d0 legacy=0x7f4a4815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34003ae0 0x7f4a34003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34004810 0x7f4a34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a34005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b5c40 con 0x7f4a34003ae0 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480b5f20 con 0x7f4a34004810 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3400bc70 con 0x7f4a480b4e70 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34003ae0 0x7f4a34003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a45d9b700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34004810 0x7f4a34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a46d9d700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a34005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 msgr2=0x7f4a34005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480b4e70 0x7f4a34005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34004810 msgr2=0x7f4a34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:18.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34004810 0x7f4a34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.688+0000 7f4a4659c700 1 -- 172.21.15.170:0/2051125139 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a34003570 con 0x7f4a34003ae0 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4659c700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34003ae0 0x7f4a34003ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a40007440 tx=0x7f4a40009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68d633c3ea9bf7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40009a30 con 0x7f4a34003ae0 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40007cf0 con 0x7f4a34003ae0 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4559a700 1 -- 172.21.15.170:0/2051125139 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40014630 con 0x7f4a34003ae0 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4759e700 1 -- 172.21.15.170:0/2051125139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34003ae0 msgr2=0x7f4a34003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4759e700 1 --2- 172.21.15.170:0/2051125139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a34003ae0 0x7f4a34003ef0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a40007440 tx=0x7f4a40009720 comp rx=0 tx=0).stop 2023-10-08T16:04:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a46d9d700 1 -- 172.21.15.170:0/2051125139 reap_dead start 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4759e700 1 -- 172.21.15.170:0/2051125139 shutdown_connections 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.689+0000 7f4a4759e700 1 -- 172.21.15.170:0/2051125139 >> 172.21.15.170:0/2051125139 conn(0x7f4a480d10c0 msgr2=0x7f4a480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.690+0000 7f4a4759e700 1 -- 172.21.15.170:0/2051125139 shutdown_connections 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.690+0000 7f4a4759e700 1 -- 172.21.15.170:0/2051125139 wait complete. 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.691+0000 7f4a4759e700 1 Processor -- start 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.691+0000 7f4a4759e700 1 -- start start 2023-10-08T16:04:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a481564a0 0x7f4a480cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480cccd0 0x7f4a480c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480cd150 con 0x7f4a481564a0 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a480c7b50 con 0x7f4a34003ae0 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4816ee30 con 0x7f4a480cccd0 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:51612/0 (socket says 172.21.15.170:51612) 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 -- 172.21.15.170:0/391588238 learned_addr learned my addr 172.21.15.170:0/391588238 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a46d9d700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480cccd0 0x7f4a480c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 -- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480cccd0 msgr2=0x7f4a480c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480cccd0 0x7f4a480c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 -- 172.21.15.170:0/391588238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a481564a0 msgr2=0x7f4a480cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a481564a0 0x7f4a480cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 -- 172.21.15.170:0/391588238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4806b830 con 0x7f4a34003ae0 2023-10-08T16:04:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.692+0000 7f4a4659c700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a340057d0 tx=0x7f4a40017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:18.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.695+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4001f650 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.695+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4001f7d0 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.695+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4806bb10 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.695+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4001fb90 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.695+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4806c0d0 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.696+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4a40003020 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.696+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4a40031740 con 0x7f4a34003ae0 2023-10-08T16:04:18.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.696+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4806c2c0 con 0x7f4a34003ae0 2023-10-08T16:04:18.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.703+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4a400496d0 con 0x7f4a34003ae0 2023-10-08T16:04:18.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.777+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4a4804bb80 con 0x7f4a34003ae0 2023-10-08T16:04:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.781+0000 7f4a3e7fc700 1 -- 172.21.15.170:0/391588238 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4a40038070 con 0x7f4a34003ae0 2023-10-08T16:04:18.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.781+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 msgr2=0x7f4a4806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.781+0000 7f4a4759e700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a340057d0 tx=0x7f4a40017000 comp rx=0 tx=0).stop 2023-10-08T16:04:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 shutdown_connections 2023-10-08T16:04:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a480cccd0 0x7f4a480c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a34003ae0 0x7f4a4806dc90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 --2- 172.21.15.170:0/391588238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a481564a0 0x7f4a480cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:18.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 >> 172.21.15.170:0/391588238 conn(0x7f4a480d10c0 msgr2=0x7f4a480b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:18.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 shutdown_connections 2023-10-08T16:04:18.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:18.782+0000 7f4a4759e700 1 -- 172.21.15.170:0/391588238 wait complete. 2023-10-08T16:04:18.783 INFO:teuthology.orchestra.run.smithi170.stderr:adjusted tunables profile to default 2023-10-08T16:04:18.794 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-08T16:04:18.794 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:18.794 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-08T16:04:18.819 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:18.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-08T16:04:18.885 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:18.885 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-08T16:04:18.950 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2023-10-08T16:04:18.950 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-08T16:04:19.014 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:19.014 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-08T16:04:19.039 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:19.039 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-08T16:04:19.103 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:19.103 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-08T16:04:19.169 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2023-10-08T16:04:19.169 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-08T16:04:19.232 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 3463d550-5fc6-4910-8450-96e64c710dcf 0 2023-10-08T16:04:19.388 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.388+0000 7f15415df700 1 Processor -- start 2023-10-08T16:04:19.389 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.388+0000 7f15415df700 1 -- start start 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f15415df700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 0x7f153c06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f15415df700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f15415df700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c06bdb0 con 0x7f153c0c81f0 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f15415df700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c0c8670 con 0x7f153c0aca80 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f15415df700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0cb370 con 0x7f153c069d60 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f153a7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f153a7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53566/0 (socket says 172.21.15.191:53566) 2023-10-08T16:04:19.391 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f153a7fc700 1 -- 172.21.15.191:0/42425052 learned_addr learned my addr 172.21.15.191:0/42425052 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.389+0000 7f153affd700 1 --2- 172.21.15.191:0/42425052 >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 0x7f153c06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153a7fc700 1 -- 172.21.15.191:0/42425052 >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 msgr2=0x7f153c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153a7fc700 1 --2- 172.21.15.191:0/42425052 >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 0x7f153c06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153a7fc700 1 -- 172.21.15.191:0/42425052 >> v1:172.21.15.170:6789/0 conn(0x7f153c0c81f0 legacy=0x7f153c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153a7fc700 1 -- 172.21.15.191:0/42425052 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c159f60 con 0x7f153c069d60 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153affd700 1 --2- 172.21.15.191:0/42425052 >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 0x7f153c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f153a7fc700 1 --2- 172.21.15.191:0/42425052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f152c0099b0 tx=0x7f152c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e4ff0906b363e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f1539ffb700 1 -- 172.21.15.191:0/42425052 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00b860 con 0x7f153c069d60 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f1539ffb700 1 -- 172.21.15.191:0/42425052 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c00b9c0 con 0x7f153c069d60 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.390+0000 7f1539ffb700 1 -- 172.21.15.191:0/42425052 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00bd60 con 0x7f153c069d60 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 -- 172.21.15.191:0/42425052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 msgr2=0x7f153c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.392 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 --2- 172.21.15.191:0/42425052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f152c0099b0 tx=0x7f152c009250 comp rx=0 tx=0).stop 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 -- 172.21.15.191:0/42425052 shutdown_connections 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 --2- 172.21.15.191:0/42425052 >> v2:172.21.15.170:3300/0 conn(0x7f153c0aca80 0x7f153c06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 --2- 172.21.15.191:0/42425052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c069d60 0x7f153c0c7cb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 -- 172.21.15.191:0/42425052 >> 172.21.15.191:0/42425052 conn(0x7f153c0cd670 msgr2=0x7f153c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 -- 172.21.15.191:0/42425052 shutdown_connections 2023-10-08T16:04:19.393 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.391+0000 7f15415df700 1 -- 172.21.15.191:0/42425052 wait complete. 2023-10-08T16:04:19.395 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.392+0000 7f15415df700 1 Processor -- start 2023-10-08T16:04:19.395 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.392+0000 7f15415df700 1 -- start start 2023-10-08T16:04:19.395 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.395 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f153c0c81f0 0x7f153c0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15f040 con 0x7f153c0c81f0 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15da80 con 0x7f153c069d60 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f15415df700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15dd40 con 0x7f153c0aca80 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153a7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153a7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53576/0 (socket says 172.21.15.191:53576) 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44404/0 (socket says 172.21.15.191:44404) 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153a7fc700 1 -- 172.21.15.191:0/627851860 learned_addr learned my addr 172.21.15.191:0/627851860 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153b7fe700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f153c0c81f0 0x7f153c0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.396 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 -- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 msgr2=0x7f153c1523e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 -- 172.21.15.191:0/627851860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f153c0c81f0 msgr2=0x7f153c0ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f153c0c81f0 0x7f153c0ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 -- 172.21.15.191:0/627851860 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0094f0 con 0x7f153c069d60 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153a7fc700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f153affd700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f152400e8a0 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.393+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524005a00 con 0x7f153c069d60 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524005b60 con 0x7f153c069d60 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c15e020 con 0x7f153c069d60 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524006020 con 0x7f153c069d60 2023-10-08T16:04:19.397 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c15e3b0 con 0x7f153c069d60 2023-10-08T16:04:19.398 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1524006410 con 0x7f153c069d60 2023-10-08T16:04:19.398 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1524013ba0 con 0x7f153c069d60 2023-10-08T16:04:19.398 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.394+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c15e820 con 0x7f153c069d60 2023-10-08T16:04:19.399 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.399+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f152403a6d0 con 0x7f153c069d60 2023-10-08T16:04:19.481 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.481+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3463d550-5fc6-4910-8450-96e64c710dcf", "id": 0} v 0) v1 -- 0x7f153c04c3c0 con 0x7f153c069d60 2023-10-08T16:04:19.485 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.485+0000 7f1532ffd700 1 -- 172.21.15.191:0/627851860 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3463d550-5fc6-4910-8450-96e64c710dcf", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1524029070 con 0x7f153c069d60 2023-10-08T16:04:19.485 INFO:teuthology.orchestra.run.smithi191.stdout:0 2023-10-08T16:04:19.486 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 msgr2=0x7f153c15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.486 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f152400e8a0 tx=0x7f1524008040 comp rx=0 tx=0).stop 2023-10-08T16:04:19.486 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 shutdown_connections 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f153c0c81f0 0x7f153c0ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f153c069d60 0x7f153c15eb00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 --2- 172.21.15.191:0/627851860 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f153c0aca80 0x7f153c1523e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 >> 172.21.15.191:0/627851860 conn(0x7f153c0cd670 msgr2=0x7f153c0c25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 shutdown_connections 2023-10-08T16:04:19.487 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.486+0000 7f15415df700 1 -- 172.21.15.191:0/627851860 wait complete. 2023-10-08T16:04:19.498 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new dbc0cad8-7324-4b72-8d94-7e4937678d1e 1 2023-10-08T16:04:19.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.613+0000 7fd62763b700 1 Processor -- start 2023-10-08T16:04:19.613 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.613+0000 7fd62763b700 1 -- start start 2023-10-08T16:04:19.615 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd62763b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd62015d730 0x7fd62015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.615 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd62763b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd62006a410 0x7fd62006a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd62763b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6200c74b0 con 0x7fd620159b10 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd62763b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200c8700 con 0x7fd62006a410 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd62763b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200c9070 con 0x7fd62015d730 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd62015d730 0x7fd62015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd62015d730 0x7fd62015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44420/0 (socket says 172.21.15.191:44420) 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 -- 172.21.15.191:0/1781889108 learned_addr learned my addr 172.21.15.191:0/1781889108 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd625bd8700 1 --2- 172.21.15.191:0/1781889108 >> v2:172.21.15.170:3300/0 conn(0x7fd62006a410 0x7fd62006a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 -- 172.21.15.191:0/1781889108 >> v2:172.21.15.170:3300/0 conn(0x7fd62006a410 msgr2=0x7fd62006a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 --2- 172.21.15.191:0/1781889108 >> v2:172.21.15.170:3300/0 conn(0x7fd62006a410 0x7fd62006a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 -- 172.21.15.191:0/1781889108 >> v1:172.21.15.170:6789/0 conn(0x7fd620159b10 legacy=0x7fd6200cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 -- 172.21.15.191:0/1781889108 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62015e0e0 con 0x7fd62015d730 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd624bd6700 1 --2- 172.21.15.191:0/1781889108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62015d730 0x7fd62015dba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd62006cce0 tx=0x7fd610009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13175378f4756136 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.614+0000 7fd617fff700 1 -- 172.21.15.191:0/1781889108 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610009e70 con 0x7fd62015d730 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd617fff700 1 -- 172.21.15.191:0/1781889108 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61000b8f0 con 0x7fd62015d730 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd617fff700 1 -- 172.21.15.191:0/1781889108 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61000bc90 con 0x7fd62015d730 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 -- 172.21.15.191:0/1781889108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62015d730 msgr2=0x7fd62015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 --2- 172.21.15.191:0/1781889108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62015d730 0x7fd62015dba0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd62006cce0 tx=0x7fd610009250 comp rx=0 tx=0).stop 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 -- 172.21.15.191:0/1781889108 shutdown_connections 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 --2- 172.21.15.191:0/1781889108 >> v2:172.21.15.170:3300/0 conn(0x7fd62006a410 0x7fd62006a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 --2- 172.21.15.191:0/1781889108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62015d730 0x7fd62015dba0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 -- 172.21.15.191:0/1781889108 >> 172.21.15.191:0/1781889108 conn(0x7fd6200c9870 msgr2=0x7fd6200c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 -- 172.21.15.191:0/1781889108 shutdown_connections 2023-10-08T16:04:19.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.615+0000 7fd62763b700 1 -- 172.21.15.191:0/1781889108 wait complete. 2023-10-08T16:04:19.620 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.616+0000 7fd62763b700 1 Processor -- start 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.616+0000 7fd62763b700 1 -- start start 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd62015d730 0x7fd62015c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c3260 con 0x7fd62015d730 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c2650 con 0x7fd62006a410 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd62763b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c2930 con 0x7fd620159b10 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd6253d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd624bd6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.621 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd624bd6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53582/0 (socket says 172.21.15.191:53582) 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd6253d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44430/0 (socket says 172.21.15.191:44430) 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.617+0000 7fd624bd6700 1 -- 172.21.15.191:0/118003627 learned_addr learned my addr 172.21.15.191:0/118003627 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd625bd8700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd62015d730 0x7fd62015c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 -- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 msgr2=0x7fd62015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 -- 172.21.15.191:0/118003627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd62015d730 msgr2=0x7fd62015c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd62015d730 0x7fd62015c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 -- 172.21.15.191:0/118003627 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100094f0 con 0x7fd62006a410 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd624bd6700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:19.622 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6253d7700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c590 tx=0x7fd61c007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c011590 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c011710 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6200c2c10 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c011ad0 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62015a420 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd61c01c6c0 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.618+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd61c0127e0 con 0x7fd62006a410 2023-10-08T16:04:19.623 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.619+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62004c3c0 con 0x7fd62006a410 2023-10-08T16:04:19.624 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.621+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd61c017070 con 0x7fd62006a410 2023-10-08T16:04:19.704 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.704+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dbc0cad8-7324-4b72-8d94-7e4937678d1e", "id": 1} v 0) v1 -- 0x7fd6200633e0 con 0x7fd62006a410 2023-10-08T16:04:19.707 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.707+0000 7fd6157fa700 1 -- 172.21.15.191:0/118003627 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dbc0cad8-7324-4b72-8d94-7e4937678d1e", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd61c01cba0 con 0x7fd62006a410 2023-10-08T16:04:19.707 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:04:19.708 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 msgr2=0x7fd620159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.708 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c590 tx=0x7fd61c007ec0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 shutdown_connections 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd62015d730 0x7fd62015c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd62006a410 0x7fd620159ee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 --2- 172.21.15.191:0/118003627 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd620159b10 0x7fd62015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 >> 172.21.15.191:0/118003627 conn(0x7fd6200c9870 msgr2=0x7fd6200b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 shutdown_connections 2023-10-08T16:04:19.709 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.708+0000 7fd62763b700 1 -- 172.21.15.191:0/118003627 wait complete. 2023-10-08T16:04:19.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f 2 2023-10-08T16:04:19.839 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 Processor -- start 2023-10-08T16:04:19.839 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 -- start start 2023-10-08T16:04:19.841 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4b6406bcd0 0x7f4b64140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6415c2a0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6415b690 con 0x7f4b6406c0a0 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.839+0000 7f4b6a9d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6415b970 con 0x7f4b640c8e00 2023-10-08T16:04:19.842 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b637fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b637fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44436/0 (socket says 172.21.15.191:44436) 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53594/0 (socket says 172.21.15.191:53594) 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b637fe700 1 -- 172.21.15.191:0/4173864067 learned_addr learned my addr 172.21.15.191:0/4173864067 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b63fff700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.170:3300/0 conn(0x7f4b6406bcd0 0x7f4b64140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 -- 172.21.15.191:0/4173864067 >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 msgr2=0x7f4b64141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 -- 172.21.15.191:0/4173864067 >> v2:172.21.15.170:3300/0 conn(0x7f4b6406bcd0 msgr2=0x7f4b64140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.170:3300/0 conn(0x7f4b6406bcd0 0x7f4b64140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 -- 172.21.15.191:0/4173864067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6415bc50 con 0x7f4b640c8e00 2023-10-08T16:04:19.843 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b637fe700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b68f71700 1 --2- 172.21.15.191:0/4173864067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b6406c6e0 tx=0x7f4b5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440250d6580a2c37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b62ffd700 1 -- 172.21.15.191:0/4173864067 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00e070 con 0x7f4b640c8e00 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b62ffd700 1 -- 172.21.15.191:0/4173864067 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c00bab0 con 0x7f4b640c8e00 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.840+0000 7f4b62ffd700 1 -- 172.21.15.191:0/4173864067 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00be70 con 0x7f4b640c8e00 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/4173864067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 msgr2=0x7f4b640c9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/4173864067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b6406c6e0 tx=0x7f4b5c009780 comp rx=0 tx=0).stop 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/4173864067 shutdown_connections 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/4173864067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b640c9240 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.170:3300/0 conn(0x7f4b6406bcd0 0x7f4b64140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/4173864067 >> v2:172.21.15.191:3300/0 conn(0x7f4b6406c0a0 0x7f4b64141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/4173864067 >> 172.21.15.191:0/4173864067 conn(0x7f4b640ce600 msgr2=0x7f4b640cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/4173864067 shutdown_connections 2023-10-08T16:04:19.845 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.841+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/4173864067 wait complete. 2023-10-08T16:04:19.847 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.842+0000 7f4b6a9d4700 1 Processor -- start 2023-10-08T16:04:19.847 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.842+0000 7f4b6a9d4700 1 -- start start 2023-10-08T16:04:19.847 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.847 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b6406c0a0 0x7f4b641694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.847 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640b35b0 con 0x7f4b6406c0a0 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6413b6a0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b6a9d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6406a600 con 0x7f4b640c8e00 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b68f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b68f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53604/0 (socket says 172.21.15.191:53604) 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44452/0 (socket says 172.21.15.191:44452) 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b68f71700 1 -- 172.21.15.191:0/164374602 learned_addr learned my addr 172.21.15.191:0/164374602 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b637fe700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b6406c0a0 0x7f4b641694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 -- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 msgr2=0x7f4b6406a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.848 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 -- 172.21.15.191:0/164374602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b6406c0a0 msgr2=0x7f4b641694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b6406c0a0 0x7f4b641694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b63fff700 1 -- 172.21.15.191:0/164374602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540094f0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.843+0000 7f4b68f71700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b63fff700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ea50 tx=0x7f4b580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800ee90 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5800c7f0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c009a20 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5800cb90 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.844+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b640b3240 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.845+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4b58005c80 con 0x7f4b6406bcd0 2023-10-08T16:04:19.849 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.845+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b641592b0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.850 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.845+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f4b58013470 con 0x7f4b6406bcd0 2023-10-08T16:04:19.850 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.847+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4b58021060 con 0x7f4b6406bcd0 2023-10-08T16:04:19.856 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:19.856+0000 7f612856c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T16:04:19.879 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:19.879+0000 7f01ddbb4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T16:04:19.930 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.930+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f", "id": 2} v 0) v1 -- 0x7f4b6404c3c0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.933 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.934+0000 7f4b4bfff700 1 -- 172.21.15.191:0/164374602 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4b5802a0b0 con 0x7f4b6406bcd0 2023-10-08T16:04:19.934 INFO:teuthology.orchestra.run.smithi191.stdout:2 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.934+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 msgr2=0x7f4b64069f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.934+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800ea50 tx=0x7f4b580091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 shutdown_connections 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b640c8e00 0x7f4b6406a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b6406bcd0 0x7f4b64069f70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 --2- 172.21.15.191:0/164374602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b6406c0a0 0x7f4b641694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 >> 172.21.15.191:0/164374602 conn(0x7f4b640ce600 msgr2=0x7f4b64156eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 shutdown_connections 2023-10-08T16:04:19.935 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:19.935+0000 7f4b6a9d4700 1 -- 172.21.15.191:0/164374602 wait complete. 2023-10-08T16:04:19.946 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new c1d70f9b-1a2f-4956-b336-b50fecb8d223 3 2023-10-08T16:04:19.988 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:19.988+0000 7f612856c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T16:04:20.018 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.018+0000 7f01ddbb4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T16:04:20.062 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.062+0000 7f1740586700 1 Processor -- start 2023-10-08T16:04:20.063 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.062+0000 7f1740586700 1 -- start start 2023-10-08T16:04:20.063 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.063+0000 7f612856c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T16:04:20.065 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f1740586700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1738159890 0x7f173815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.065 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f1740586700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173815a160 con 0x7f1738159c60 2023-10-08T16:04:20.065 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f1740586700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17380d13a0 con 0x7f1738069560 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f1740586700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173815e140 con 0x7f1738159890 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f173eb23700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1738069560 0x7f17380ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:42192/0 (socket says 172.21.15.191:42192) 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f173eb23700 1 -- 172.21.15.191:0/3840117162 learned_addr learned my addr 172.21.15.191:0/3840117162 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.063+0000 7f173e322700 1 --2- 172.21.15.191:0/3840117162 >> v2:172.21.15.170:3300/0 conn(0x7f1738159890 0x7f173815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612703758 0 0) 0x7f17380d13a0 con 0x7f1738069560 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1724003570 con 0x7f1738069560 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975704967 0 0) 0x7f173815a160 con 0x7f1738159c60 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17380d13a0 con 0x7f1738159c60 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173e322700 1 -- 172.21.15.191:0/3840117162 >> v1:172.21.15.191:6789/0 conn(0x7f1738069560 legacy=0x7f17380ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173e322700 1 -- 172.21.15.191:0/3840117162 >> v1:172.21.15.170:6789/0 conn(0x7f1738159c60 legacy=0x7f173815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.066 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173e322700 1 -- 172.21.15.191:0/3840117162 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173815a3d0 con 0x7f1738159890 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173e322700 1 --2- 172.21.15.191:0/3840117162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173815abd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17280098e0 tx=0x7f17280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9947a88e79fa0d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011630 con 0x7f1738159890 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728011790 con 0x7f1738159890 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.064+0000 7f173d320700 1 -- 172.21.15.191:0/3840117162 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011b30 con 0x7f1738159890 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 -- 172.21.15.191:0/3840117162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 msgr2=0x7f173815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 --2- 172.21.15.191:0/3840117162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173815abd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f17280098e0 tx=0x7f17280091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 -- 172.21.15.191:0/3840117162 shutdown_connections 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 --2- 172.21.15.191:0/3840117162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173815abd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 -- 172.21.15.191:0/3840117162 >> 172.21.15.191:0/3840117162 conn(0x7f1738068e20 msgr2=0x7f17380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 -- 172.21.15.191:0/3840117162 shutdown_connections 2023-10-08T16:04:20.067 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.065+0000 7f1740586700 1 -- 172.21.15.191:0/3840117162 wait complete. 2023-10-08T16:04:20.069 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.066+0000 7f1740586700 1 Processor -- start 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.066+0000 7f1740586700 1 -- start start 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 0x7f1738151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738151950 con 0x7f1738159890 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738152790 con 0x7f1738159c60 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173814d7b0 con 0x7f1738069560 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173e322700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 0x7f1738151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173e322700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 0x7f1738151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53612/0 (socket says 172.21.15.191:53612) 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44458/0 (socket says 172.21.15.191:44458) 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173e322700 1 -- 172.21.15.191:0/890870770 learned_addr learned my addr 172.21.15.191:0/890870770 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173db21700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 -- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 msgr2=0x7f1738151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 0x7f1738151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 -- 172.21.15.191:0/890870770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 msgr2=0x7f173814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 -- 172.21.15.191:0/890870770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009420 con 0x7f1738159c60 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f173eb23700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1730001460 tx=0x7f1730001a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000a840 con 0x7f1738159c60 2023-10-08T16:04:20.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173000a9c0 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738151b80 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000ad80 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738152140 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1730007300 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.067+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f173001b8a0 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.068+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173804c3c0 con 0x7f1738159c60 2023-10-08T16:04:20.072 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.070+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1730007790 con 0x7f1738159c60 2023-10-08T16:04:20.098 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.098+0000 7f01ddbb4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T16:04:20.152 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.152+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c1d70f9b-1a2f-4956-b336-b50fecb8d223", "id": 3} v 0) v1 -- 0x7f17381528d0 con 0x7f1738159c60 2023-10-08T16:04:20.155 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.155+0000 7f172e7fc700 1 -- 172.21.15.191:0/890870770 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c1d70f9b-1a2f-4956-b336-b50fecb8d223", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f173001bb80 con 0x7f1738159c60 2023-10-08T16:04:20.155 INFO:teuthology.orchestra.run.smithi191.stdout:3 2023-10-08T16:04:20.156 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 msgr2=0x7f1738156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.156 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1730001460 tx=0x7f1730001a60 comp rx=0 tx=0).stop 2023-10-08T16:04:20.156 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 shutdown_connections 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1738159c60 0x7f1738156be0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1738069560 0x7f1738151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 --2- 172.21.15.191:0/890870770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1738159890 0x7f173814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 >> 172.21.15.191:0/890870770 conn(0x7f1738068e20 msgr2=0x7f17380b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 shutdown_connections 2023-10-08T16:04:20.157 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.156+0000 7f1740586700 1 -- 172.21.15.191:0/890870770 wait complete. 2023-10-08T16:04:20.168 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 541091da-0d2a-4788-a70a-4943fc488905 4 2023-10-08T16:04:20.250 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.250+0000 7f612856c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T16:04:20.284 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.284+0000 7f294b59e700 1 Processor -- start 2023-10-08T16:04:20.284 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.284+0000 7f294b59e700 1 -- start start 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294b59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f294c0b4e70 0x7f294c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294b59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f294c0b16d0 0x7f294c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294b59e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0b2080 con 0x7f294c15d240 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294b59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0cfb00 con 0x7f294c0b16d0 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294b59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0b5c40 con 0x7f294c0b4e70 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f294c0b4e70 0x7f294c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f294c0b4e70 0x7f294c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44474/0 (socket says 172.21.15.191:44474) 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 -- 172.21.15.191:0/1479922269 learned_addr learned my addr 172.21.15.191:0/1479922269 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f2949d9b700 1 --2- 172.21.15.191:0/1479922269 >> v2:172.21.15.170:3300/0 conn(0x7f294c0b16d0 0x7f294c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.287 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 -- 172.21.15.191:0/1479922269 >> v2:172.21.15.170:3300/0 conn(0x7f294c0b16d0 msgr2=0x7f294c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 --2- 172.21.15.191:0/1479922269 >> v2:172.21.15.170:3300/0 conn(0x7f294c0b16d0 0x7f294c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 -- 172.21.15.191:0/1479922269 >> v1:172.21.15.170:6789/0 conn(0x7f294c15d240 legacy=0x7f294c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.285+0000 7f294a59c700 1 -- 172.21.15.191:0/1479922269 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0b5f20 con 0x7f294c0b4e70 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294a59c700 1 --2- 172.21.15.191:0/1479922269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c0b4e70 0x7f294c0ccff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f294c064910 tx=0x7f2940009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7627e99094546e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294959a700 1 -- 172.21.15.191:0/1479922269 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940009e70 con 0x7f294c0b4e70 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294959a700 1 -- 172.21.15.191:0/1479922269 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294000b860 con 0x7f294c0b4e70 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294959a700 1 -- 172.21.15.191:0/1479922269 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294000bc00 con 0x7f294c0b4e70 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 -- 172.21.15.191:0/1479922269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c0b4e70 msgr2=0x7f294c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 --2- 172.21.15.191:0/1479922269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c0b4e70 0x7f294c0ccff0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f294c064910 tx=0x7f2940009250 comp rx=0 tx=0).stop 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 -- 172.21.15.191:0/1479922269 shutdown_connections 2023-10-08T16:04:20.288 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 --2- 172.21.15.191:0/1479922269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c0b4e70 0x7f294c0ccff0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.289 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 --2- 172.21.15.191:0/1479922269 >> v2:172.21.15.170:3300/0 conn(0x7f294c0b16d0 0x7f294c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.289 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 -- 172.21.15.191:0/1479922269 >> 172.21.15.191:0/1479922269 conn(0x7f294c0d10c0 msgr2=0x7f294c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.289 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 -- 172.21.15.191:0/1479922269 shutdown_connections 2023-10-08T16:04:20.289 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.286+0000 7f294b59e700 1 -- 172.21.15.191:0/1479922269 wait complete. 2023-10-08T16:04:20.291 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.287+0000 7f294b59e700 1 Processor -- start 2023-10-08T16:04:20.291 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.287+0000 7f294b59e700 1 -- start start 2023-10-08T16:04:20.291 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294c0b16d0 0x7f294c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.291 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.291 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c15d240 0x7f294c14b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15afa0 con 0x7f294c0b16d0 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15bda0 con 0x7f294c15d240 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c15b190 con 0x7f294c0b4e70 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f2949d9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f2949d9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53624/0 (socket says 172.21.15.191:53624) 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f2949d9b700 1 -- 172.21.15.191:0/4250814067 learned_addr learned my addr 172.21.15.191:0/4250814067 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294ad9d700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c15d240 0x7f294c14b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.288+0000 7f294a59c700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294c0b16d0 0x7f294c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 -- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c15d240 msgr2=0x7f294c14b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c15d240 0x7f294c14b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 -- 172.21.15.191:0/4250814067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294c0b16d0 msgr2=0x7f294c0cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.292 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294c0b16d0 0x7f294c0cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 -- 172.21.15.191:0/4250814067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29400094f0 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f2949d9b700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f293c00ec80 tx=0x7f293c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00c850 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c00c9b0 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c00cd50 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c15b470 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c15ba30 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f293c005c80 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.289+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f293c013b30 con 0x7f294c0b4e70 2023-10-08T16:04:20.293 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.290+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c088990 con 0x7f294c0b4e70 2023-10-08T16:04:20.294 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.292+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f293c021060 con 0x7f294c0b4e70 2023-10-08T16:04:20.294 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.293+0000 7f01ddbb4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T16:04:20.375 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.375+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "541091da-0d2a-4788-a70a-4943fc488905", "id": 4} v 0) v1 -- 0x7f294c15bee0 con 0x7f294c0b4e70 2023-10-08T16:04:20.378 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.378+0000 7f293a7fc700 1 -- 172.21.15.191:0/4250814067 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "541091da-0d2a-4788-a70a-4943fc488905", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f293c017070 con 0x7f294c0b4e70 2023-10-08T16:04:20.378 INFO:teuthology.orchestra.run.smithi191.stdout:4 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 msgr2=0x7f294c15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f293c00ec80 tx=0x7f293c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 shutdown_connections 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f294c15d240 0x7f294c14b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f294c0b16d0 0x7f294c0cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 --2- 172.21.15.191:0/4250814067 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f294c0b4e70 0x7f294c15aa60 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 >> 172.21.15.191:0/4250814067 conn(0x7f294c0d10c0 msgr2=0x7f294c15caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 shutdown_connections 2023-10-08T16:04:20.382 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.379+0000 7f294b59e700 1 -- 172.21.15.191:0/4250814067 wait complete. 2023-10-08T16:04:20.391 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new dc305d76-f069-4520-ac6c-1d7ca891b94e 5 2023-10-08T16:04:20.422 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.422+0000 7f612856c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T16:04:20.473 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.473+0000 7f01ddbb4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T16:04:20.499 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.498+0000 7f612856c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T16:04:20.507 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.507+0000 7f30d7d75700 1 Processor -- start 2023-10-08T16:04:20.507 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.507+0000 7f30d7d75700 1 -- start start 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d7d75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f30d0159890 0x7f30d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d7d75700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d015a160 con 0x7f30d0069560 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d7d75700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d00d13a0 con 0x7f30d0159c60 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d7d75700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d015e140 con 0x7f30d0159890 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5310700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f30d0159c60 0x7f30d015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:42208/0 (socket says 172.21.15.191:42208) 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5310700 1 -- 172.21.15.191:0/775401903 learned_addr learned my addr 172.21.15.191:0/775401903 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5b11700 1 --2- 172.21.15.191:0/775401903 >> v2:172.21.15.170:3300/0 conn(0x7f30d0159890 0x7f30d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.509 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679520145 0 0) 0x7f30d00d13a0 con 0x7f30d0159c60 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003570 con 0x7f30d0159c60 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019177969 0 0) 0x7f30d015a160 con 0x7f30d0069560 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d00d13a0 con 0x7f30d0069560 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5b11700 1 -- 172.21.15.191:0/775401903 >> v1:172.21.15.191:6789/0 conn(0x7f30d0159c60 legacy=0x7f30d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5b11700 1 -- 172.21.15.191:0/775401903 >> v1:172.21.15.170:6789/0 conn(0x7f30d0069560 legacy=0x7f30d00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.508+0000 7f30d5b11700 1 -- 172.21.15.191:0/775401903 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d015a3d0 con 0x7f30d0159890 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d5b11700 1 --2- 172.21.15.191:0/775401903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0159890 0x7f30d015abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ccd05d73eb3a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc011630 con 0x7f30d0159890 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc011790 con 0x7f30d0159890 2023-10-08T16:04:20.510 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d4b0f700 1 -- 172.21.15.191:0/775401903 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc011b10 con 0x7f30d0159890 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 -- 172.21.15.191:0/775401903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0159890 msgr2=0x7f30d015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 --2- 172.21.15.191:0/775401903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0159890 0x7f30d015abd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30cc0098e0 tx=0x7f30cc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 -- 172.21.15.191:0/775401903 shutdown_connections 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 --2- 172.21.15.191:0/775401903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0159890 0x7f30d015abd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 -- 172.21.15.191:0/775401903 >> 172.21.15.191:0/775401903 conn(0x7f30d0068e20 msgr2=0x7f30d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.513 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 -- 172.21.15.191:0/775401903 shutdown_connections 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.509+0000 7f30d7d75700 1 -- 172.21.15.191:0/775401903 wait complete. 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.510+0000 7f30d7d75700 1 Processor -- start 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 -- start start 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0069560 0x7f30d016d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 0x7f30d0158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d016de20 con 0x7f30d0069560 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00c9150 con 0x7f30d0159c60 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d7d75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0152930 con 0x7f30d0159890 2023-10-08T16:04:20.514 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d5310700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d6312700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 0x7f30d0158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d5310700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53628/0 (socket says 172.21.15.191:53628) 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d5310700 1 -- 172.21.15.191:0/2557446191 learned_addr learned my addr 172.21.15.191:0/2557446191 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.511+0000 7f30d5b11700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0069560 0x7f30d016d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 -- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 msgr2=0x7f30d0158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 0x7f30d0158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 -- 172.21.15.191:0/2557446191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0069560 msgr2=0x7f30d016d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0069560 0x7f30d016d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 -- 172.21.15.191:0/2557446191 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc009420 con 0x7f30d0159890 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d6312700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 0x7f30d0158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d5310700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c790 tx=0x7f30c0007e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.515 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0005b30 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0005cd0 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d0157570 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0007320 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d0157b30 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f30c001c6c0 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.512+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f30c0007710 con 0x7f30d0159890 2023-10-08T16:04:20.516 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.513+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d004c3c0 con 0x7f30d0159890 2023-10-08T16:04:20.517 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.515+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f30c0021060 con 0x7f30d0159890 2023-10-08T16:04:20.565 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.565+0000 7f01ddbb4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T16:04:20.610 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.610+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dc305d76-f069-4520-ac6c-1d7ca891b94e", "id": 5} v 0) v1 -- 0x7f30d006a5b0 con 0x7f30d0159890 2023-10-08T16:04:20.616 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.616+0000 7f30c5ffb700 1 -- 172.21.15.191:0/2557446191 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dc305d76-f069-4520-ac6c-1d7ca891b94e", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f30c002a0b0 con 0x7f30d0159890 2023-10-08T16:04:20.616 INFO:teuthology.orchestra.run.smithi191.stdout:5 2023-10-08T16:04:20.617 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 msgr2=0x7f30d00c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f30c000c790 tx=0x7f30c0007e80 comp rx=0 tx=0).stop 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 shutdown_connections 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30d0159c60 0x7f30d0158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30d0069560 0x7f30d016d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 --2- 172.21.15.191:0/2557446191 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30d0159890 0x7f30d00c8c10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 >> 172.21.15.191:0/2557446191 conn(0x7f30d0068e20 msgr2=0x7f30d0152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 shutdown_connections 2023-10-08T16:04:20.618 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.617+0000 7f30d7d75700 1 -- 172.21.15.191:0/2557446191 wait complete. 2023-10-08T16:04:20.629 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new fae402bd-abf3-4d4b-9c09-5104a8806aa2 6 2023-10-08T16:04:20.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.636+0000 7f612856c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T16:04:20.707 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.707+0000 7f01ddbb4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T16:04:20.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:20.732+0000 7f612856c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T16:04:20.746 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.746+0000 7f1f89b9d700 1 Processor -- start 2023-10-08T16:04:20.746 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.746+0000 7f1f89b9d700 1 -- start start 2023-10-08T16:04:20.748 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.746+0000 7f1f89b9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1f840b4e70 0x7f1f840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.748 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f89b9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.748 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f89b9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 0x7f1f8415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.748 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f89b9d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f840b2080 con 0x7f1f840b4e70 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f89b9d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f840cfb00 con 0x7f1f8415d240 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f89b9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840b5c40 con 0x7f1f840b16d0 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f8939c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 0x7f1f8415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f8939c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 0x7f1f8415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44494/0 (socket says 172.21.15.191:44494) 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53632/0 (socket says 172.21.15.191:53632) 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f8939c700 1 -- 172.21.15.191:0/2308390240 learned_addr learned my addr 172.21.15.191:0/2308390240 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f88b9b700 1 --2- 172.21.15.191:0/2308390240 >> v2:172.21.15.170:3300/0 conn(0x7f1f840b4e70 0x7f1f840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 -- 172.21.15.191:0/2308390240 >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 msgr2=0x7f1f8415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 --2- 172.21.15.191:0/2308390240 >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 0x7f1f8415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 -- 172.21.15.191:0/2308390240 >> v2:172.21.15.170:3300/0 conn(0x7f1f840b4e70 msgr2=0x7f1f840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:20.749 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 --2- 172.21.15.191:0/2308390240 >> v2:172.21.15.170:3300/0 conn(0x7f1f840b4e70 0x7f1f840ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 -- 172.21.15.191:0/2308390240 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840b5f20 con 0x7f1f840b16d0 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f83fff700 1 --2- 172.21.15.191:0/2308390240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f8415d180 tx=0x7f1f700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2179d6da862a904c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f837fe700 1 -- 172.21.15.191:0/2308390240 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70011630 con 0x7f1f840b16d0 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f837fe700 1 -- 172.21.15.191:0/2308390240 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f700117b0 con 0x7f1f840b16d0 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.747+0000 7f1f837fe700 1 -- 172.21.15.191:0/2308390240 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70011b70 con 0x7f1f840b16d0 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 -- 172.21.15.191:0/2308390240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 msgr2=0x7f1f840b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/2308390240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1f8415d180 tx=0x7f1f700091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 -- 172.21.15.191:0/2308390240 shutdown_connections 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/2308390240 >> v2:172.21.15.191:3300/0 conn(0x7f1f8415d240 0x7f1f8415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/2308390240 >> v2:172.21.15.170:3300/0 conn(0x7f1f840b4e70 0x7f1f840ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.750 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/2308390240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b16d0 0x7f1f840b1b40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.751 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 -- 172.21.15.191:0/2308390240 >> 172.21.15.191:0/2308390240 conn(0x7f1f840d10c0 msgr2=0x7f1f840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.751 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 -- 172.21.15.191:0/2308390240 shutdown_connections 2023-10-08T16:04:20.751 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.748+0000 7f1f89b9d700 1 -- 172.21.15.191:0/2308390240 wait complete. 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.749+0000 7f1f89b9d700 1 Processor -- start 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.749+0000 7f1f89b9d700 1 -- start start 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f840b16d0 0x7f1f84145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 0x7f1f840c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840c9630 con 0x7f1f840b16d0 2023-10-08T16:04:20.753 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84145b70 con 0x7f1f8415d240 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f89b9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f840c8a20 con 0x7f1f840b4e70 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f8939c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 0x7f1f840c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f8939c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.191:44502/0 (socket says 172.21.15.191:44502) 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f8939c700 1 -- 172.21.15.191:0/1579284245 learned_addr learned my addr 172.21.15.191:0/1579284245 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f88b9b700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f840b16d0 0x7f1f84145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 -- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 msgr2=0x7f1f84068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 -- 172.21.15.191:0/1579284245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f840b16d0 msgr2=0x7f1f84145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.754 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f840b16d0 0x7f1f84145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f83fff700 1 -- 172.21.15.191:0/1579284245 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009420 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.750+0000 7f1f8939c700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f83fff700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 0x7f1f840c8010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f70016000 tx=0x7f1f70017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7001f070 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70002f30 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f840c8d00 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70011920 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f840c92c0 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1f70023070 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.751+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f1f70011a80 con 0x7f1f840b4e70 2023-10-08T16:04:20.755 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.752+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f84088990 con 0x7f1f840b4e70 2023-10-08T16:04:20.756 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.755+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1f7002a070 con 0x7f1f840b4e70 2023-10-08T16:04:20.809 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:20.809+0000 7f01ddbb4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T16:04:20.838 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.838+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fae402bd-abf3-4d4b-9c09-5104a8806aa2", "id": 6} v 0) v1 -- 0x7f1f8415ce20 con 0x7f1f840b4e70 2023-10-08T16:04:20.844 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.844+0000 7f1f80ff9700 1 -- 172.21.15.191:0/1579284245 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fae402bd-abf3-4d4b-9c09-5104a8806aa2", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1f70015020 con 0x7f1f840b4e70 2023-10-08T16:04:20.844 INFO:teuthology.orchestra.run.smithi191.stdout:6 2023-10-08T16:04:20.845 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 msgr2=0x7f1f840c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 0x7f1f840c8010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1f70016000 tx=0x7f1f70017000 comp rx=0 tx=0).stop 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 shutdown_connections 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f8415d240 0x7f1f84068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f840b16d0 0x7f1f84145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 --2- 172.21.15.191:0/1579284245 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f840b4e70 0x7f1f840c8010 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 >> 172.21.15.191:0/1579284245 conn(0x7f1f840d10c0 msgr2=0x7f1f840b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 shutdown_connections 2023-10-08T16:04:20.846 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.845+0000 7f1f89b9d700 1 -- 172.21.15.191:0/1579284245 wait complete. 2023-10-08T16:04:20.857 DEBUG:teuthology.orchestra.run.smithi191:> sudo ceph --cluster ceph osd new 2221559f-a1b9-470e-9926-439928ae1bce 7 2023-10-08T16:04:20.971 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.971+0000 7f5e36cdc700 1 Processor -- start 2023-10-08T16:04:20.972 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.971+0000 7f5e36cdc700 1 -- start start 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e36cdc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e30159890 0x7f5e3015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e36cdc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 0x7f5e300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e36cdc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e30069ee0 con 0x7f5e30159c60 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e36cdc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3015a160 con 0x7f5e30159890 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e36cdc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300d13a0 con 0x7f5e30069560 2023-10-08T16:04:20.974 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e35279700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 0x7f5e300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e2ffff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f5e30159c60 0x7f5e3015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.191:42214/0 (socket says 172.21.15.191:42214) 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.972+0000 7f5e2ffff700 1 -- 172.21.15.191:0/3691922394 learned_addr learned my addr 172.21.15.191:0/3691922394 (peer_addr_for_me v1:172.21.15.191:0/0) 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e34a78700 1 --2- 172.21.15.191:0/3691922394 >> v2:172.21.15.170:3300/0 conn(0x7f5e30159890 0x7f5e3015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e2f7fe700 1 -- 172.21.15.191:0/3691922394 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675148139 0 0) 0x7f5e30069ee0 con 0x7f5e30159c60 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e2f7fe700 1 -- 172.21.15.191:0/3691922394 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e20003570 con 0x7f5e30159c60 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e35279700 1 -- 172.21.15.191:0/3691922394 >> v2:172.21.15.170:3300/0 conn(0x7f5e30159890 msgr2=0x7f5e3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e35279700 1 --2- 172.21.15.191:0/3691922394 >> v2:172.21.15.170:3300/0 conn(0x7f5e30159890 0x7f5e3015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e35279700 1 -- 172.21.15.191:0/3691922394 >> v1:172.21.15.191:6789/0 conn(0x7f5e30159c60 legacy=0x7f5e3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e35279700 1 -- 172.21.15.191:0/3691922394 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3015e140 con 0x7f5e30069560 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e35279700 1 --2- 172.21.15.191:0/3691922394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 0x7f5e300699a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009e60 tx=0x7f5e28009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a395cd24f3e639 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.975 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e2f7fe700 1 -- 172.21.15.191:0/3691922394 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800e070 con 0x7f5e30069560 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e2f7fe700 1 -- 172.21.15.191:0/3691922394 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2800bab0 con 0x7f5e30069560 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e2f7fe700 1 -- 172.21.15.191:0/3691922394 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800be70 con 0x7f5e30069560 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3691922394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 msgr2=0x7f5e300699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.973+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3691922394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 0x7f5e300699a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009e60 tx=0x7f5e28009780 comp rx=0 tx=0).stop 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3691922394 shutdown_connections 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3691922394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30069560 0x7f5e300699a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3691922394 >> v2:172.21.15.170:3300/0 conn(0x7f5e30159890 0x7f5e3015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3691922394 >> 172.21.15.191:0/3691922394 conn(0x7f5e30068e20 msgr2=0x7f5e300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3691922394 shutdown_connections 2023-10-08T16:04:20.976 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.974+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3691922394 wait complete. 2023-10-08T16:04:20.978 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.975+0000 7f5e36cdc700 1 Processor -- start 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.975+0000 7f5e36cdc700 1 -- start start 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.975+0000 7f5e36cdc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e30069560 0x7f5e30149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.975+0000 7f5e36cdc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 0x7f5e3015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cffc0 con 0x7f5e30069560 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3014a390 con 0x7f5e30159890 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3016b990 con 0x7f5e30159c60 2023-10-08T16:04:20.979 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 0x7f5e3015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.191:53666/0 (socket says 172.21.15.191:53666) 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 -- 172.21.15.191:0/3717428512 learned_addr learned my addr 172.21.15.191:0/3717428512 (peer_addr_for_me v2:172.21.15.191:0/0) 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e34a78700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e30069560 0x7f5e30149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 -- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 msgr2=0x7f5e3015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 0x7f5e3015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 -- 172.21.15.191:0/3717428512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e30069560 msgr2=0x7f5e30149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e30069560 0x7f5e30149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 -- 172.21.15.191:0/3717428512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e28009a20 con 0x7f5e30159c60 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2ffff700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 0x7f5e3015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:20.980 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e35279700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b610 tx=0x7f5e28019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28012dd0 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28027470 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3016b060 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28027830 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3016b620 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5e28024070 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.976+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f5e2801e420 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.977+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3016bb00 con 0x7f5e30159c60 2023-10-08T16:04:20.981 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:20.979+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5e280466d0 con 0x7f5e30159c60 2023-10-08T16:04:21.061 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.061+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2221559f-a1b9-470e-9926-439928ae1bce", "id": 7} v 0) v1 -- 0x7f5e3004c3c0 con 0x7f5e30159c60 2023-10-08T16:04:21.069 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.069+0000 7f5e2cff9700 1 -- 172.21.15.191:0/3717428512 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2221559f-a1b9-470e-9926-439928ae1bce", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5e28035030 con 0x7f5e30159c60 2023-10-08T16:04:21.069 INFO:teuthology.orchestra.run.smithi191.stdout:7 2023-10-08T16:04:21.070 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 msgr2=0x7f5e3015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800b610 tx=0x7f5e28019000 comp rx=0 tx=0).stop 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 shutdown_connections 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e30159c60 0x7f5e3015d5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e30069560 0x7f5e30149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 --2- 172.21.15.191:0/3717428512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e30159890 0x7f5e3015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 >> 172.21.15.191:0/3717428512 conn(0x7f5e30068e20 msgr2=0x7f5e300b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 shutdown_connections 2023-10-08T16:04:21.071 INFO:teuthology.orchestra.run.smithi191.stderr:2023-10-08T16:04:21.070+0000 7f5e36cdc700 1 -- 172.21.15.191:0/3717428512 wait complete. 2023-10-08T16:04:21.082 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-08T16:04:21.082 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-08T16:04:21.085 INFO:tasks.ceph.osd.0:Started 2023-10-08T16:04:21.085 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-08T16:04:21.085 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-08T16:04:21.087 INFO:tasks.ceph.osd.1:Started 2023-10-08T16:04:21.087 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-08T16:04:21.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-08T16:04:21.089 INFO:tasks.ceph.osd.2:Started 2023-10-08T16:04:21.089 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-08T16:04:21.089 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-08T16:04:21.091 INFO:tasks.ceph.osd.3:Started 2023-10-08T16:04:21.091 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-08T16:04:21.091 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-08T16:04:21.094 INFO:tasks.ceph.osd.4:Started 2023-10-08T16:04:21.094 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-08T16:04:21.094 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-08T16:04:21.096 INFO:tasks.ceph.osd.5:Started 2023-10-08T16:04:21.096 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-08T16:04:21.096 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-08T16:04:21.098 INFO:tasks.ceph.osd.6:Started 2023-10-08T16:04:21.098 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-08T16:04:21.098 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-08T16:04:21.100 INFO:tasks.ceph.osd.7:Started 2023-10-08T16:04:21.100 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:21.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.245+0000 7fa481bdf700 1 Processor -- start 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.245+0000 7fa481bdf700 1 -- start start 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.245+0000 7fa481bdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c141500 0x7fa47c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.245+0000 7fa481bdf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa47c06c0a0 0x7fa47c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa481bdf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa47c15c2a0 con 0x7fa47c06bcd0 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa481bdf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47c0c9880 con 0x7fa47c06c0a0 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa481bdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c15b690 con 0x7fa47c141500 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa47c06c0a0 0x7fa47c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa47c06c0a0 0x7fa47c0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57924/0 (socket says 172.21.15.170:57924) 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 -- 172.21.15.170:0/1134707096 learned_addr learned my addr 172.21.15.170:0/1134707096 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47affd700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c141500 0x7fa47c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:21.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 -- 172.21.15.170:0/1134707096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c141500 msgr2=0x7fa47c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c141500 0x7fa47c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 -- 172.21.15.170:0/1134707096 >> v1:172.21.15.191:6789/0 conn(0x7fa47c06bcd0 legacy=0x7fa47c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 -- 172.21.15.170:0/1134707096 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47c15b970 con 0x7fa47c06c0a0 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47bfff700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c0c9340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa474009f20 tx=0x7fa474009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5c9bc78c7de4f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47a7fc700 1 -- 172.21.15.170:0/1134707096 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47400e070 con 0x7fa47c06c0a0 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47a7fc700 1 -- 172.21.15.170:0/1134707096 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47400b9d0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.246+0000 7fa47a7fc700 1 -- 172.21.15.170:0/1134707096 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4740092e0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 -- 172.21.15.170:0/1134707096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 msgr2=0x7fa47c0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c0c9340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa474009f20 tx=0x7fa474009840 comp rx=0 tx=0).stop 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 -- 172.21.15.170:0/1134707096 shutdown_connections 2023-10-08T16:04:21.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c0c9340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 --2- 172.21.15.170:0/1134707096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c141500 0x7fa47c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 -- 172.21.15.170:0/1134707096 >> 172.21.15.170:0/1134707096 conn(0x7fa47c0ce600 msgr2=0x7fa47c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 -- 172.21.15.170:0/1134707096 shutdown_connections 2023-10-08T16:04:21.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.247+0000 7fa481bdf700 1 -- 172.21.15.170:0/1134707096 wait complete. 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.248+0000 7fa481bdf700 1 Processor -- start 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.248+0000 7fa481bdf700 1 -- start start 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c06bcd0 0x7fa47c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa47c141500 0x7fa47c06a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c13edb0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c0b1fa0 con 0x7fa47c141500 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47c13f2d0 con 0x7fa47c06bcd0 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:21.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57936/0 (socket says 172.21.15.170:57936) 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 -- 172.21.15.170:0/2869979976 learned_addr learned my addr 172.21.15.170:0/2869979976 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47b7fe700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c06bcd0 0x7fa47c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47bfff700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa47c141500 0x7fa47c06a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 -- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c06bcd0 msgr2=0x7fa47c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c06bcd0 0x7fa47c0afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 -- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa47c141500 msgr2=0x7fa47c06a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa47c141500 0x7fa47c06a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 -- 172.21.15.170:0/2869979976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa474009ae0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa47affd700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e960 tx=0x7fa4700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470013680 con 0x7fa47c06c0a0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4700137e0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47c06aa50 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.249+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470013b80 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.250+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47c0b19d0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.250+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa470005c80 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.250+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fa4700065c0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.251+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa47c0b2150 con 0x7fa47c06c0a0 2023-10-08T16:04:21.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.253+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa47c0b2150 con 0x7fa47c06c0a0 2023-10-08T16:04:21.294 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:21.293+0000 7f612856c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T16:04:21.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.332+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47c0b1cf0 con 0x7fa47c06c0a0 2023-10-08T16:04:21.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa4637fe700 1 -- 172.21.15.170:0/2869979976 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fa470021060 con 0x7fa47c06c0a0 2023-10-08T16:04:21.333 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:21.333 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":10,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:21.062592+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:21.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 msgr2=0x7fa47c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:21.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa481bdf700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e960 tx=0x7fa4700091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 shutdown_connections 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa481bdf700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa47c141500 0x7fa47c06a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.333+0000 7fa481bdf700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa47c06bcd0 0x7fa47c0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.334+0000 7fa481bdf700 1 --2- 172.21.15.170:0/2869979976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa47c06c0a0 0x7fa47c069e80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.334+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 >> 172.21.15.170:0/2869979976 conn(0x7fa47c0ce600 msgr2=0x7fa47c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.334+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 shutdown_connections 2023-10-08T16:04:21.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:21.334+0000 7fa481bdf700 1 -- 172.21.15.170:0/2869979976 wait complete. 2023-10-08T16:04:21.346 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-08T16:04:21.347 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-08T16:04:21.355 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:21.355+0000 7f01ddbb4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T16:04:21.373 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:21.373+0000 7f612856c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T16:04:21.432 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:21.432+0000 7f01ddbb4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T16:04:21.452 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:21.452+0000 7f612856c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T16:04:21.511 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:21.511+0000 7f01ddbb4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T16:04:21.694 INFO:tasks.ceph.osd.1.smithi170.stderr:2023-10-08T16:04:21.694+0000 7f588efe2540 -1 Falling back to public interface 2023-10-08T16:04:21.695 INFO:tasks.ceph.osd.0.smithi170.stderr:2023-10-08T16:04:21.695+0000 7f12402cc540 -1 Falling back to public interface 2023-10-08T16:04:21.699 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T16:04:21.699+0000 7fd022bf5540 -1 Falling back to public interface 2023-10-08T16:04:21.702 INFO:tasks.ceph.osd.2.smithi170.stderr:2023-10-08T16:04:21.702+0000 7febc855b540 -1 Falling back to public interface 2023-10-08T16:04:21.705 INFO:tasks.ceph.osd.3.smithi170.stderr:2023-10-08T16:04:21.705+0000 7f89d3bb4540 -1 Falling back to public interface 2023-10-08T16:04:21.707 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T16:04:21.707+0000 7f38f7742540 -1 Falling back to public interface 2023-10-08T16:04:21.714 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T16:04:21.714+0000 7f881fad1540 -1 Falling back to public interface 2023-10-08T16:04:21.724 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T16:04:21.725+0000 7f465c4a2540 -1 Falling back to public interface 2023-10-08T16:04:22.029 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:22.029+0000 7f612856c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T16:04:22.101 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:22.101+0000 7f01ddbb4180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T16:04:22.152 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:22.167 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:22.167+0000 7f612856c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T16:04:22.238 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:22.238+0000 7f01ddbb4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T16:04:22.260 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.260+0000 7f5d094f2700 1 Processor -- start 2023-10-08T16:04:22.261 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.260+0000 7f5d094f2700 1 -- start start 2023-10-08T16:04:22.261 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f612856c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f5d094f2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5d040ccd40 0x7f5d0406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f5d094f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 0x7f5d0415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f5d094f2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0406b5f0 con 0x7f5d040b2690 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f5d094f2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0415bca0 con 0x7f5d040ccd40 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.261+0000 7f5d094f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0415bf80 con 0x7f5d0415b250 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d037fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5d040b2690 0x7f5d04067610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60778/0 (socket says 172.21.15.170:60778) 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d037fe700 1 -- 172.21.15.170:0/3264297123 learned_addr learned my addr 172.21.15.170:0/3264297123 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:22.263 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 --2- 172.21.15.170:0/3264297123 >> v2:172.21.15.191:3300/0 conn(0x7f5d040ccd40 0x7f5d0406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d027fc700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 0x7f5d0415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d01ffb700 1 -- 172.21.15.170:0/3264297123 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839551223 0 0) 0x7f5d0406b5f0 con 0x7f5d040b2690 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d01ffb700 1 -- 172.21.15.170:0/3264297123 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cec003570 con 0x7f5d040b2690 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 -- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 msgr2=0x7f5d0415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 0x7f5d0415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 -- 172.21.15.170:0/3264297123 >> v1:172.21.15.170:6789/0 conn(0x7f5d040b2690 legacy=0x7f5d04067610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 -- 172.21.15.170:0/3264297123 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0406b280 con 0x7f5d040ccd40 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d027fc700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 0x7f5d0415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d02ffd700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d040ccd40 0x7f5d0406ba80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80098e0 tx=0x7f5cf80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88c096a994c6a150 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d01ffb700 1 -- 172.21.15.170:0/3264297123 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011630 con 0x7f5d040ccd40 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d01ffb700 1 -- 172.21.15.170:0/3264297123 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8011790 con 0x7f5d040ccd40 2023-10-08T16:04:22.264 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.262+0000 7f5d01ffb700 1 -- 172.21.15.170:0/3264297123 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8011b10 con 0x7f5d040ccd40 2023-10-08T16:04:22.265 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 -- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d040ccd40 msgr2=0x7f5d0406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d040ccd40 0x7f5d0406ba80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80098e0 tx=0x7f5cf80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 -- 172.21.15.170:0/3264297123 shutdown_connections 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d040ccd40 0x7f5d0406ba80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 --2- 172.21.15.170:0/3264297123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d0415b250 0x7f5d0415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 -- 172.21.15.170:0/3264297123 >> 172.21.15.170:0/3264297123 conn(0x7f5d0406e460 msgr2=0x7f5d040aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 -- 172.21.15.170:0/3264297123 shutdown_connections 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.263+0000 7f5d094f2700 1 -- 172.21.15.170:0/3264297123 wait complete. 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.265+0000 7f5d094f2700 1 Processor -- start 2023-10-08T16:04:22.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.265+0000 7f5d094f2700 1 -- start start 2023-10-08T16:04:22.268 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 0x7f5d0414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040b09c0 con 0x7f5d040ccd40 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040b1800 con 0x7f5d0415b250 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d094f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040b0bf0 con 0x7f5d040b2690 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.266+0000 7f5d027fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d02ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d02ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:45116/0 (socket says 172.21.15.170:45116) 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57940/0 (socket says 172.21.15.170:57940) 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d02ffd700 1 -- 172.21.15.170:0/2834447417 learned_addr learned my addr 172.21.15.170:0/2834447417 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:22.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d037fe700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 0x7f5d0414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 -- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 msgr2=0x7f5d0416a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 -- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 msgr2=0x7f5d0414f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 0x7f5d0414f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 -- 172.21.15.170:0/2834447417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf4009420 con 0x7f5d040ccd40 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d02ffd700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d037fe700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 0x7f5d0414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d027fc700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ec80 tx=0x7f5cf40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400c850 con 0x7f5d040ccd40 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400c9b0 con 0x7f5d040ccd40 2023-10-08T16:04:22.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf8009420 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400cd70 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d040b11b0 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5cf4005c80 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.267+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f5cf4006700 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.268+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d040b1970 con 0x7f5d040ccd40 2023-10-08T16:04:22.271 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.270+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5cf4021060 con 0x7f5d040ccd40 2023-10-08T16:04:22.318 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:22.318+0000 7f01ddbb4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T16:04:22.340 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.339+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d04063450 con 0x7f5d040ccd40 2023-10-08T16:04:22.340 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.340+0000 7f5ceaffd700 1 -- 172.21.15.170:0/2834447417 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f5cf402a090 con 0x7f5d040ccd40 2023-10-08T16:04:22.340 INFO:teuthology.misc.health.smithi170.stdout: 2023-10-08T16:04:22.341 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":10,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:21.062592+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:22.341 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 msgr2=0x7f5d040b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:22.341 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400ec80 tx=0x7f5cf40091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 shutdown_connections 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5d0415b250 0x7f5d0414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5d040b2690 0x7f5d0416a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 --2- 172.21.15.170:0/2834447417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d040ccd40 0x7f5d040b0480 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 >> 172.21.15.170:0/2834447417 conn(0x7f5d0406e460 msgr2=0x7f5d040b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 shutdown_connections 2023-10-08T16:04:22.342 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:22.341+0000 7f5d094f2700 1 -- 172.21.15.170:0/2834447417 wait complete. 2023-10-08T16:04:22.353 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T16:04:22.518 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:22.518+0000 7f612856c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T16:04:22.569 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:22.569+0000 7f01ddbb4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T16:04:22.597 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:22.597+0000 7f612856c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T16:04:22.646 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:22.646+0000 7f01ddbb4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T16:04:23.444 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.444+0000 7f612856c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T16:04:23.471 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.470+0000 7f01ddbb4180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T16:04:23.517 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.517+0000 7f612856c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T16:04:23.544 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.544+0000 7f01ddbb4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T16:04:23.593 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.593+0000 7f612856c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T16:04:23.620 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.620+0000 7f01ddbb4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T16:04:23.727 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.727+0000 7f612856c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T16:04:23.754 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.754+0000 7f01ddbb4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T16:04:23.812 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.811+0000 7f612856c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T16:04:23.838 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.838+0000 7f01ddbb4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T16:04:23.966 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:23.966+0000 7f612856c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T16:04:23.993 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:23.993+0000 7f01ddbb4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T16:04:24.041 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:24.041+0000 7f612856c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T16:04:24.068 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:24.068+0000 7f01ddbb4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T16:04:24.117 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:24.117+0000 7f612856c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T16:04:24.144 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:24.144+0000 7f01ddbb4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T16:04:24.204 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:24.204+0000 7f612856c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T16:04:24.228 INFO:tasks.ceph.mgr.x.smithi191.stderr:2023-10-08T16:04:24.228+0000 7f01ddbb4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T16:04:24.970 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T16:04:24.970+0000 7fd022bf5540 -1 osd.4 0 log_to_monitors true 2023-10-08T16:04:25.190 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T16:04:25.190+0000 7f38f7742540 -1 osd.6 0 log_to_monitors true 2023-10-08T16:04:25.230 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:25.230+0000 7f60e196d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T16:04:25.231 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:04:25.231+0000 7f60e196d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-08T16:04:25.232 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T16:04:25.233+0000 7f881fad1540 -1 osd.7 0 log_to_monitors true 2023-10-08T16:04:25.252 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T16:04:25.253+0000 7f465c4a2540 -1 osd.5 0 log_to_monitors true 2023-10-08T16:04:25.257 INFO:tasks.ceph.osd.2.smithi170.stderr:2023-10-08T16:04:25.257+0000 7febc855b540 -1 osd.2 0 log_to_monitors true 2023-10-08T16:04:25.257 INFO:tasks.ceph.osd.3.smithi170.stderr:2023-10-08T16:04:25.257+0000 7f89d3bb4540 -1 osd.3 0 log_to_monitors true 2023-10-08T16:04:25.259 INFO:tasks.ceph.osd.0.smithi170.stderr:2023-10-08T16:04:25.259+0000 7f12402cc540 -1 osd.0 0 log_to_monitors true 2023-10-08T16:04:25.260 INFO:tasks.ceph.osd.1.smithi170.stderr:2023-10-08T16:04:25.260+0000 7f588efe2540 -1 osd.1 0 log_to_monitors true 2023-10-08T16:04:27.711 INFO:tasks.ceph.osd.1.smithi170.stderr:2023-10-08T16:04:27.711+0000 7f5885c04700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T16:04:27.716 INFO:tasks.ceph.osd.1.smithi170.stderr:2023-10-08T16:04:27.716+0000 7f587e9fb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:27.773 INFO:tasks.ceph.osd.2.smithi170.stderr:2023-10-08T16:04:27.773+0000 7febbf17d700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T16:04:27.777 INFO:tasks.ceph.osd.2.smithi170.stderr:2023-10-08T16:04:27.777+0000 7febb7f74700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:27.781 INFO:tasks.ceph.osd.3.smithi170.stderr:2023-10-08T16:04:27.781+0000 7f89ca7d6700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T16:04:27.784 INFO:tasks.ceph.osd.0.smithi170.stderr:2023-10-08T16:04:27.784+0000 7f1236eee700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T16:04:27.788 INFO:tasks.ceph.osd.0.smithi170.stderr:2023-10-08T16:04:27.788+0000 7f122fce5700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:27.791 INFO:tasks.ceph.osd.3.smithi170.stderr:2023-10-08T16:04:27.791+0000 7f89c35cd700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:28.473 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T16:04:28.472+0000 7f8817ef6700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T16:04:28.478 INFO:tasks.ceph.osd.7.smithi191.stderr:2023-10-08T16:04:28.477+0000 7f880f4ea700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:28.488 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T16:04:28.488+0000 7f38ee364700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T16:04:28.492 INFO:tasks.ceph.osd.6.smithi191.stderr:2023-10-08T16:04:28.492+0000 7f38e715b700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:28.497 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T16:04:28.497+0000 7fd019817700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T16:04:28.501 INFO:tasks.ceph.osd.4.smithi191.stderr:2023-10-08T16:04:28.501+0000 7fd01260e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:28.501 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T16:04:28.502+0000 7f46530c4700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T16:04:28.506 INFO:tasks.ceph.osd.5.smithi191.stderr:2023-10-08T16:04:28.506+0000 7f464bebb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T16:04:29.159 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:29.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.265+0000 7fe8f8bde700 1 Processor -- start 2023-10-08T16:04:29.266 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 -- start start 2023-10-08T16:04:29.268 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe8f406d990 0x7fe8f40af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.268 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe8f40aeed0 0x7fe8f40afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f406de20 con 0x7fe8f4159730 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f406e100 con 0x7fe8f406d990 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.266+0000 7fe8f8bde700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f406e3e0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe8f406d990 0x7fe8f40af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe8f406d990 0x7fe8f40af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34608/0 (socket says 172.21.15.170:34608) 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/1958332906 learned_addr learned my addr 172.21.15.170:0/1958332906 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f2d9d700 1 --2- 172.21.15.170:0/1958332906 >> v2:172.21.15.191:3300/0 conn(0x7fe8f40aeed0 0x7fe8f40afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/1958332906 >> v2:172.21.15.191:3300/0 conn(0x7fe8f40aeed0 msgr2=0x7fe8f40afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 --2- 172.21.15.170:0/1958332906 >> v2:172.21.15.191:3300/0 conn(0x7fe8f40aeed0 0x7fe8f40afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/1958332906 >> v1:172.21.15.191:6789/0 conn(0x7fe8f4159730 legacy=0x7fe8f40b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/1958332906 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f406e6c0 con 0x7fe8f406d990 2023-10-08T16:04:29.269 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f1d9b700 1 --2- 172.21.15.170:0/1958332906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f406d990 0x7fe8f40af890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406bef0 tx=0x7fe8e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb38d1ab91bb217d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f159a700 1 -- 172.21.15.170:0/1958332906 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8009e70 con 0x7fe8f406d990 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f159a700 1 -- 172.21.15.170:0/1958332906 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800b8f0 con 0x7fe8f406d990 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.267+0000 7fe8f159a700 1 -- 172.21.15.170:0/1958332906 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800bc90 con 0x7fe8f406d990 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 -- 172.21.15.170:0/1958332906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f406d990 msgr2=0x7fe8f40af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/1958332906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f406d990 0x7fe8f40af890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406bef0 tx=0x7fe8e8009250 comp rx=0 tx=0).stop 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 -- 172.21.15.170:0/1958332906 shutdown_connections 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/1958332906 >> v2:172.21.15.191:3300/0 conn(0x7fe8f40aeed0 0x7fe8f40afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/1958332906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f406d990 0x7fe8f40af890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 -- 172.21.15.170:0/1958332906 >> 172.21.15.170:0/1958332906 conn(0x7fe8f4087ff0 msgr2=0x7fe8f40883c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 -- 172.21.15.170:0/1958332906 shutdown_connections 2023-10-08T16:04:29.270 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.268+0000 7fe8f8bde700 1 -- 172.21.15.170:0/1958332906 wait complete. 2023-10-08T16:04:29.275 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.269+0000 7fe8f8bde700 1 Processor -- start 2023-10-08T16:04:29.275 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.269+0000 7fe8f8bde700 1 -- start start 2023-10-08T16:04:29.275 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8f406d990 0x7fe8f415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.275 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.275 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8f4159730 0x7fe8f416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f415ba50 con 0x7fe8f40aeed0 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f415c420 con 0x7fe8f4159730 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f415ae40 con 0x7fe8f406d990 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34618/0 (socket says 172.21.15.170:34618) 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/2008526205 learned_addr learned my addr 172.21.15.170:0/2008526205 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:29.276 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f259c700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8f406d990 0x7fe8f415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f2d9d700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8f4159730 0x7fe8f416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8f406d990 msgr2=0x7fe8f415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8f406d990 0x7fe8f415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8f4159730 msgr2=0x7fe8f416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8f4159730 0x7fe8f416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 -- 172.21.15.170:0/2008526205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e80094f0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f1d9b700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4064310 tx=0x7fe8e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800bc70 con 0x7fe8f40aeed0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8024440 con 0x7fe8f40aeed0 2023-10-08T16:04:29.277 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f415b120 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80247c0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.270+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f415b6e0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.271+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f415bbc0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.272+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe8e801a420 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.272+0000 7fe8e27fc700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 0x7fe8d4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.272+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fe8e8085cc0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.272+0000 7fe8f259c700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 0x7fe8d4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.278 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.272+0000 7fe8f259c700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 0x7fe8d4069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c4f0 tx=0x7fe8e4001a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.279 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.276+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8e80534e0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.465 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.464+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8f40adcd0 con 0x7fe8f40aeed0 2023-10-08T16:04:29.465 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.465+0000 7fe8e27fc700 1 -- 172.21.15.170:0/2008526205 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10444 (secure 0 0 0) 0x7fe8e8015020 con 0x7fe8f40aeed0 2023-10-08T16:04:29.466 INFO:teuthology.misc.health.smithi170.stdout: 2023-10-08T16:04:29.466 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":13,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:28.513686+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 msgr2=0x7fe8d4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 0x7fe8d4069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400c4f0 tx=0x7fe8e4001a80 comp rx=0 tx=0).stop 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 msgr2=0x7fe8f40992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4064310 tx=0x7fe8e8017000 comp rx=0 tx=0).stop 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 shutdown_connections 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe8f4159730 0x7fe8f416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8f406d990 0x7fe8f415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe8d4066ea0 0x7fe8d4069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.468 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 --2- 172.21.15.170:0/2008526205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f40aeed0 0x7fe8f40992e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.469 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 >> 172.21.15.170:0/2008526205 conn(0x7fe8f4087ff0 msgr2=0x7fe8f40b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:29.469 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 shutdown_connections 2023-10-08T16:04:29.469 INFO:teuthology.misc.health.smithi170.stderr:2023-10-08T16:04:29.467+0000 7fe8f8bde700 1 -- 172.21.15.170:0/2008526205 wait complete. 2023-10-08T16:04:29.478 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-08T16:04:29.478 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-08T16:04:29.479 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:04:29.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:04:29.481 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:04:29.481 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:04:29.481 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:04:29.483 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:04:29.483 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:04:29.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:04:29.486 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:04:29.486 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:04:29.486 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:04:29.488 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:04:29.489 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-08T16:04:29.489 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:29.555 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:04:29.555 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:04:29.555 INFO:tasks.ceph.mds.a.smithi170.stderr:2023-10-08T16:04:29.555+0000 7fa8c4d1d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:04:29.560 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:04:29.561 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:04:29.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.609+0000 7f655fa36700 1 Processor -- start 2023-10-08T16:04:29.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.609+0000 7f655fa36700 1 -- start start 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655fa36700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6558159c60 0x7f655815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655fa36700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655815a160 con 0x7f6558159890 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655fa36700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65580d13a0 con 0x7f655815db10 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655fa36700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655815df50 con 0x7f6558159c60 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655d7d2700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6558159890 0x7f655815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53532/0 (socket says 172.21.15.170:53532) 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655d7d2700 1 -- 172.21.15.170:0/3758240839 learned_addr learned my addr 172.21.15.170:0/3758240839 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:29.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655cfd1700 1 --2- 172.21.15.170:0/3758240839 >> v2:172.21.15.191:3300/0 conn(0x7f6558159c60 0x7f655815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249363212 0 0) 0x7f655815a160 con 0x7f6558159890 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6544003570 con 0x7f6558159890 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655cfd1700 1 -- 172.21.15.170:0/3758240839 >> v1:172.21.15.191:6789/0 conn(0x7f655815db10 legacy=0x7f6558069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485414602 0 0) 0x7f65580d13a0 con 0x7f655815db10 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.610+0000 7f655cfd1700 1 -- 172.21.15.170:0/3758240839 >> v1:172.21.15.170:6789/0 conn(0x7f6558159890 legacy=0x7f655815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655cfd1700 1 -- 172.21.15.170:0/3758240839 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655815e140 con 0x7f6558159c60 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655cfd1700 1 --2- 172.21.15.170:0/3758240839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f655815d5d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65480098e0 tx=0x7f65480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32dd4194ec84eb8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800b640 con 0x7f6558159c60 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654800b7a0 con 0x7f6558159c60 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f654ffff700 1 -- 172.21.15.170:0/3758240839 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800bb40 con 0x7f6558159c60 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 -- 172.21.15.170:0/3758240839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 msgr2=0x7f655815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 --2- 172.21.15.170:0/3758240839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f655815d5d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f65480098e0 tx=0x7f65480091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 -- 172.21.15.170:0/3758240839 shutdown_connections 2023-10-08T16:04:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 --2- 172.21.15.170:0/3758240839 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f655815d5d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 -- 172.21.15.170:0/3758240839 >> 172.21.15.170:0/3758240839 conn(0x7f6558068e20 msgr2=0x7f65580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 -- 172.21.15.170:0/3758240839 shutdown_connections 2023-10-08T16:04:29.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.611+0000 7f655fa36700 1 -- 172.21.15.170:0/3758240839 wait complete. 2023-10-08T16:04:29.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.613+0000 7f655fa36700 1 Processor -- start 2023-10-08T16:04:29.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.613+0000 7f655fa36700 1 -- start start 2023-10-08T16:04:29.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6558159890 0x7f655815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f6558153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655816a1b0 con 0x7f655815db10 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558169880 con 0x7f6558159c60 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6558169b60 con 0x7f6558159890 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34644/0 (socket says 172.21.15.170:34644) 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 -- 172.21.15.170:0/1562226613 learned_addr learned my addr 172.21.15.170:0/1562226613 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655cfd1700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f6558153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655d7d2700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6558159890 0x7f655815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 -- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6558159890 msgr2=0x7f655815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6558159890 0x7f655815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 -- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 msgr2=0x7f6558153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f6558153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 -- 172.21.15.170:0/1562226613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6548009420 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655dfd3700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f655000c890 tx=0x7f6550001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550017070 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6550006ba0 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6558169e40 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550012630 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.614+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65581694e0 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.615+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655816a540 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.615+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6550012a60 con 0x7f655815db10 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.616+0000 7f654d7fa700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 0x7f653c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.616+0000 7f655d7d2700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 0x7f653c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.616+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f6550083d50 con 0x7f655815db10 2023-10-08T16:04:29.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.616+0000 7f655d7d2700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 0x7f653c069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65540058d0 tx=0x7f6554009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.620+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f655816a540 con 0x7f655815db10 2023-10-08T16:04:29.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.804+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f655816a540 con 0x7f655815db10 2023-10-08T16:04:29.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.805+0000 7f654d7fa700 1 -- 172.21.15.170:0/1562226613 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10628 (secure 0 0 0) 0x7f655816a540 con 0x7f655815db10 2023-10-08T16:04:29.806 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:29.806 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":14,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:29.789559+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 msgr2=0x7f653c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 0x7f653c069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65540058d0 tx=0x7f6554009040 comp rx=0 tx=0).stop 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 msgr2=0x7f65580c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f655000c890 tx=0x7f6550001670 comp rx=0 tx=0).stop 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 shutdown_connections 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f653c066ea0 0x7f653c069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f655815db10 0x7f65580c6560 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6558159890 0x7f655815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 --2- 172.21.15.170:0/1562226613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6558159c60 0x7f6558153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 >> 172.21.15.170:0/1562226613 conn(0x7f6558068e20 msgr2=0x7f65581553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 shutdown_connections 2023-10-08T16:04:29.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.807+0000 7f655fa36700 1 -- 172.21.15.170:0/1562226613 wait complete. 2023-10-08T16:04:29.819 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-10-08T16:04:29.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:29.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.934+0000 7fc82c05f700 1 Processor -- start 2023-10-08T16:04:29.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.934+0000 7fc82c05f700 1 -- start start 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82c05f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 0x7fc82415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82c05f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc82415db10 0x7fc824069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82c05f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82415a160 con 0x7fc824159890 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82c05f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240d13a0 con 0x7fc82415db10 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82c05f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82415e140 con 0x7fc824159c60 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc82a5fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc82415db10 0x7fc824069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc829dfb700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc824159890 0x7fc82415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53546/0 (socket says 172.21.15.170:53546) 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.935+0000 7fc829dfb700 1 -- 172.21.15.170:0/2170001847 learned_addr learned my addr 172.21.15.170:0/2170001847 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc8295fa700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 0x7fc82415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82a5fc700 1 -- 172.21.15.170:0/2170001847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 msgr2=0x7fc82415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82a5fc700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 0x7fc82415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82a5fc700 1 -- 172.21.15.170:0/2170001847 >> v1:172.21.15.170:6789/0 conn(0x7fc824159890 legacy=0x7fc82415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82a5fc700 1 -- 172.21.15.170:0/2170001847 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82415a3d0 con 0x7fc82415db10 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc8295fa700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 0x7fc82415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82a5fc700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc82415db10 0x7fc824069560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0098e0 tx=0x7fc81c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6fd7d291c6205d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc828df9700 1 -- 172.21.15.170:0/2170001847 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c009e80 con 0x7fc82415db10 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc828df9700 1 -- 172.21.15.170:0/2170001847 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00b860 con 0x7fc82415db10 2023-10-08T16:04:29.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc828df9700 1 -- 172.21.15.170:0/2170001847 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00bbe0 con 0x7fc82415db10 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 -- 172.21.15.170:0/2170001847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc82415db10 msgr2=0x7fc824069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc82415db10 0x7fc824069560 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0098e0 tx=0x7fc81c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 -- 172.21.15.170:0/2170001847 shutdown_connections 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc82415db10 0x7fc824069560 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 --2- 172.21.15.170:0/2170001847 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159c60 0x7fc82415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 -- 172.21.15.170:0/2170001847 >> 172.21.15.170:0/2170001847 conn(0x7fc824068e20 msgr2=0x7fc8240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.936+0000 7fc82c05f700 1 -- 172.21.15.170:0/2170001847 shutdown_connections 2023-10-08T16:04:29.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.937+0000 7fc82c05f700 1 -- 172.21.15.170:0/2170001847 wait complete. 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 Processor -- start 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 -- start start 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159890 0x7fc8240c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc82415db10 0x7fc8241472d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc824147810 con 0x7fc824159c60 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240cfe70 con 0x7fc82415db10 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc82c05f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c9c40 con 0x7fc824159890 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc8295fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc8295fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34666/0 (socket says 172.21.15.170:34666) 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.938+0000 7fc8295fa700 1 -- 172.21.15.170:0/3220014795 learned_addr learned my addr 172.21.15.170:0/3220014795 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc829dfb700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159890 0x7fc8240c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc82a5fc700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc82415db10 0x7fc8241472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 -- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159890 msgr2=0x7fc8240c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159890 0x7fc8240c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 -- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc82415db10 msgr2=0x7fc8241472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc82415db10 0x7fc8241472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 -- 172.21.15.170:0/3220014795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c009420 con 0x7fc824159c60 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc8295fa700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c880 tx=0x7fc814008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814011600 con 0x7fc824159c60 2023-10-08T16:04:29.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc814011760 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240c9f20 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814011ae0 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.939+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240ca4e0 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.940+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8240ca700 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.940+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc8140193b0 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.941+0000 7fc819ffb700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 0x7fc808069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.941+0000 7fc829dfb700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 0x7fc808069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.941+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4467+0+0 (secure 0 0 0) 0x7fc814083e90 con 0x7fc824159c60 2023-10-08T16:04:29.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.941+0000 7fc829dfb700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 0x7fc808069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8200020c0 tx=0x7fc820001a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:29.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:29.945+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8140516b0 con 0x7fc824159c60 2023-10-08T16:04:30.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.169+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc82404c3c0 con 0x7fc824159c60 2023-10-08T16:04:30.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.170+0000 7fc819ffb700 1 -- 172.21.15.170:0/3220014795 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1259 (secure 0 0 0) 0x7fc81401f500 con 0x7fc824159c60 2023-10-08T16:04:30.170 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:30.170 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 msgr2=0x7fc808069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 0x7fc808069350 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8200020c0 tx=0x7fc820001a10 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 msgr2=0x7fc824146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc81400c880 tx=0x7fc814008040 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 shutdown_connections 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc82415db10 0x7fc8241472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc824159890 0x7fc8240c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc808066ea0 0x7fc808069350 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 --2- 172.21.15.170:0/3220014795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc824159c60 0x7fc824146d90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:30.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 >> 172.21.15.170:0/3220014795 conn(0x7fc824068e20 msgr2=0x7fc824152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 shutdown_connections 2023-10-08T16:04:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:30.172+0000 7fc82c05f700 1 -- 172.21.15.170:0/3220014795 wait complete. 2023-10-08T16:04:30.174 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 2 2023-10-08T16:04:32.186 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:32.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.317+0000 7f7d9ec95700 1 Processor -- start 2023-10-08T16:04:32.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.317+0000 7f7d9ec95700 1 -- start start 2023-10-08T16:04:32.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ec95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d980c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ec95700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7d9806c0a0 0x7f7d980c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ec95700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9815c2a0 con 0x7f7d9806bcd0 2023-10-08T16:04:32.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ec95700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d980c9880 con 0x7f7d9806c0a0 2023-10-08T16:04:32.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ec95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9815b690 con 0x7f7d98141500 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9d232700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7d9806c0a0 0x7f7d980c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ca31700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7d9806bcd0 0x7f7d98140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53554/0 (socket says 172.21.15.170:53554) 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d9ca31700 1 -- 172.21.15.170:0/518770757 learned_addr learned my addr 172.21.15.170:0/518770757 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.318+0000 7f7d97fff700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d980c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d9d232700 1 -- 172.21.15.170:0/518770757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 msgr2=0x7f7d980c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d9d232700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d980c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d9d232700 1 -- 172.21.15.170:0/518770757 >> v1:172.21.15.170:6789/0 conn(0x7f7d9806bcd0 legacy=0x7f7d98140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d9d232700 1 -- 172.21.15.170:0/518770757 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9815b970 con 0x7f7d9806c0a0 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d977fe700 1 -- 172.21.15.170:0/518770757 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699903954 0 0) 0x7f7d9815c2a0 con 0x7f7d9806bcd0 2023-10-08T16:04:32.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d9d232700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c9340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009e60 tx=0x7f7d90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35feaf53ef5764f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d977fe700 1 -- 172.21.15.170:0/518770757 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000e070 con 0x7f7d9806c0a0 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d977fe700 1 -- 172.21.15.170:0/518770757 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9000bb60 con 0x7f7d9806c0a0 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.319+0000 7f7d977fe700 1 -- 172.21.15.170:0/518770757 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90012430 con 0x7f7d9806c0a0 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 -- 172.21.15.170:0/518770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 msgr2=0x7f7d980c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c9340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009e60 tx=0x7f7d90009780 comp rx=0 tx=0).stop 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 -- 172.21.15.170:0/518770757 shutdown_connections 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c9340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/518770757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d980c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 -- 172.21.15.170:0/518770757 >> 172.21.15.170:0/518770757 conn(0x7f7d980ce600 msgr2=0x7f7d980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 -- 172.21.15.170:0/518770757 shutdown_connections 2023-10-08T16:04:32.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.320+0000 7f7d9ec95700 1 -- 172.21.15.170:0/518770757 wait complete. 2023-10-08T16:04:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.321+0000 7f7d9ec95700 1 Processor -- start 2023-10-08T16:04:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.321+0000 7f7d9ec95700 1 -- start start 2023-10-08T16:04:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d9806bcd0 0x7f7d9813dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d9813f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9816a510 con 0x7f7d9806c0a0 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9816a7f0 con 0x7f7d9806bcd0 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ec95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9816aad0 con 0x7f7d98141500 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34684/0 (socket says 172.21.15.170:34684) 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 -- 172.21.15.170:0/2502843029 learned_addr learned my addr 172.21.15.170:0/2502843029 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9d232700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d9813f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d9ca31700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d9806bcd0 0x7f7d9813dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 -- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 msgr2=0x7f7d9813f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d9813f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 -- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d9806bcd0 msgr2=0x7f7d9813dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d9806bcd0 0x7f7d9813dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.322+0000 7f7d97fff700 1 -- 172.21.15.170:0/2502843029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009a20 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d97fff700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e930 tx=0x7f7d840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84013680 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d840137e0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9816adb0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84013b80 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.323+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d980c83c0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.324+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9804c3c0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.325+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f7d8400c3e0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.325+0000 7f7d94ff9700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 0x7f7d7c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.325+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7d84084170 con 0x7f7d9806c0a0 2023-10-08T16:04:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.325+0000 7f7d9ca31700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 0x7f7d7c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.326+0000 7f7d9ca31700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 0x7f7d7c069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c007fd0 tx=0x7f7d8c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.330+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d84051810 con 0x7f7d9806c0a0 2023-10-08T16:04:32.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.577+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d9815bc50 con 0x7f7d9806c0a0 2023-10-08T16:04:32.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.577+0000 7f7d94ff9700 1 -- 172.21.15.170:0/2502843029 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7f7d84017de0 con 0x7f7d9806c0a0 2023-10-08T16:04:32.578 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:32.578 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-08T16:04:32.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 msgr2=0x7f7d7c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 0x7f7d7c069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c007fd0 tx=0x7f7d8c007d30 comp rx=0 tx=0).stop 2023-10-08T16:04:32.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 msgr2=0x7f7d980c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400e930 tx=0x7f7d840091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 shutdown_connections 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7d98141500 0x7f7d9813f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7d9806bcd0 0x7f7d9813dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d7c066ea0 0x7f7d7c069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 --2- 172.21.15.170:0/2502843029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d9806c0a0 0x7f7d980c7e80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.579+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 >> 172.21.15.170:0/2502843029 conn(0x7f7d980ce600 msgr2=0x7f7d980c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.580+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 shutdown_connections 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.580+0000 7f7d9ec95700 1 -- 172.21.15.170:0/2502843029 wait complete. 2023-10-08T16:04:32.581 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 3 2023-10-08T16:04:32.593 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:32.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.710+0000 7f2a9528f700 1 Processor -- start 2023-10-08T16:04:32.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 -- start start 2023-10-08T16:04:32.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a90159890 0x7f2a9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 0x7f2a900699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a90069ee0 con 0x7f2a90159c60 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9015a160 con 0x7f2a90159890 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.711+0000 7f2a9528f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900d13a0 con 0x7f2a90069560 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a90159890 0x7f2a9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a90159890 0x7f2a9015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34712/0 (socket says 172.21.15.170:34712) 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 -- 172.21.15.170:0/28560364 learned_addr learned my addr 172.21.15.170:0/28560364 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8f7fe700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 0x7f2a900699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 -- 172.21.15.170:0/28560364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 msgr2=0x7f2a900699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 0x7f2a900699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 -- 172.21.15.170:0/28560364 >> v1:172.21.15.191:6789/0 conn(0x7f2a90159c60 legacy=0x7f2a9015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 -- 172.21.15.170:0/28560364 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9015e140 con 0x7f2a90159890 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8f7fe700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 0x7f2a900699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8effd700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90159890 0x7f2a9015abd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a800099b0 tx=0x7f2a80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1229d5ada86fe0ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8e7fc700 1 -- 172.21.15.170:0/28560364 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000b860 con 0x7f2a90159890 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8e7fc700 1 -- 172.21.15.170:0/28560364 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8000b9c0 con 0x7f2a90159890 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.712+0000 7f2a8e7fc700 1 -- 172.21.15.170:0/28560364 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000bd60 con 0x7f2a90159890 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 -- 172.21.15.170:0/28560364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90159890 msgr2=0x7f2a9015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90159890 0x7f2a9015abd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a800099b0 tx=0x7f2a80009250 comp rx=0 tx=0).stop 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 -- 172.21.15.170:0/28560364 shutdown_connections 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90069560 0x7f2a900699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 --2- 172.21.15.170:0/28560364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90159890 0x7f2a9015abd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 -- 172.21.15.170:0/28560364 >> 172.21.15.170:0/28560364 conn(0x7f2a90068e20 msgr2=0x7f2a900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:32.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 -- 172.21.15.170:0/28560364 shutdown_connections 2023-10-08T16:04:32.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.713+0000 7f2a9528f700 1 -- 172.21.15.170:0/28560364 wait complete. 2023-10-08T16:04:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.714+0000 7f2a9528f700 1 Processor -- start 2023-10-08T16:04:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.714+0000 7f2a9528f700 1 -- start start 2023-10-08T16:04:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90159890 0x7f2a900b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 0x7f2a90148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900c8b00 con 0x7f2a90069560 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900c7e80 con 0x7f2a90159c60 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900c8160 con 0x7f2a90159890 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34718/0 (socket says 172.21.15.170:34718) 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 -- 172.21.15.170:0/1092200510 learned_addr learned my addr 172.21.15.170:0/1092200510 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8f7fe700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 0x7f2a90148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a867fc700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90159890 0x7f2a900b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 -- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90159890 msgr2=0x7f2a900b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90159890 0x7f2a900b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 -- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 msgr2=0x7f2a90148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 0x7f2a90148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 -- 172.21.15.170:0/1092200510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800094f0 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8f7fe700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 0x7f2a90148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a8effd700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009290 tx=0x7f2a80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80024440 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a800245a0 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a900c8440 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80024940 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.715+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a90148a40 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.716+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9004c3c0 con 0x7f2a90069560 2023-10-08T16:04:32.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.717+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f2a8001a420 con 0x7f2a90069560 2023-10-08T16:04:32.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.717+0000 7f2a877fe700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 0x7f2a70069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:32.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.717+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2a80085f30 con 0x7f2a90069560 2023-10-08T16:04:32.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.717+0000 7f2a867fc700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 0x7f2a70069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:32.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.717+0000 7f2a867fc700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 0x7f2a70069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003a80 tx=0x7f2a78007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:32.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.721+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a800535d0 con 0x7f2a90069560 2023-10-08T16:04:32.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.914+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a9006a5b0 con 0x7f2a90069560 2023-10-08T16:04:32.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.915+0000 7f2a877fe700 1 -- 172.21.15.170:0/1092200510 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f2a80056bf0 con 0x7f2a90069560 2023-10-08T16:04:32.915 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:32.915 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:32.801179+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:32.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.916+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 msgr2=0x7f2a70069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.916+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 0x7f2a70069350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a78003a80 tx=0x7f2a78007ec0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 msgr2=0x7f2a900c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:32.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009290 tx=0x7f2a80017000 comp rx=0 tx=0).stop 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 shutdown_connections 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a90159c60 0x7f2a90148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a70066ea0 0x7f2a70069350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a90069560 0x7f2a900c78a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 --2- 172.21.15.170:0/1092200510 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a90159890 0x7f2a900b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 >> 172.21.15.170:0/1092200510 conn(0x7f2a90068e20 msgr2=0x7f2a90154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 shutdown_connections 2023-10-08T16:04:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:32.917+0000 7f2a9528f700 1 -- 172.21.15.170:0/1092200510 wait complete. 2023-10-08T16:04:32.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:04:32.929 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:04:33.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.048+0000 7f5248869700 1 Processor -- start 2023-10-08T16:04:33.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.048+0000 7f5248869700 1 -- start start 2023-10-08T16:04:33.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5248869700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f52440b4e70 0x7f52440ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5248869700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52440cfb00 con 0x7f52440b1b40 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5248869700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524415e060 con 0x7f52440b16d0 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5248869700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52440b5c40 con 0x7f52440b4e70 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f52437fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f52440b4e70 0x7f52440ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5243fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f52440b1b40 0x7f524415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53570/0 (socket says 172.21.15.170:53570) 2023-10-08T16:04:33.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.049+0000 7f5243fff700 1 -- 172.21.15.170:0/2054732226 learned_addr learned my addr 172.21.15.170:0/2054732226 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f5242ffd700 1 -- 172.21.15.170:0/2054732226 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499934523 0 0) 0x7f524415e060 con 0x7f52440b16d0 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f5242ffd700 1 -- 172.21.15.170:0/2054732226 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5230003570 con 0x7f52440b16d0 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f52437fe700 1 -- 172.21.15.170:0/2054732226 >> v1:172.21.15.191:6789/0 conn(0x7f52440b16d0 legacy=0x7f524415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f52437fe700 1 -- 172.21.15.170:0/2054732226 >> v1:172.21.15.170:6789/0 conn(0x7f52440b1b40 legacy=0x7f524415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f52437fe700 1 -- 172.21.15.170:0/2054732226 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52440b5f20 con 0x7f52440b4e70 2023-10-08T16:04:33.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.050+0000 7f52437fe700 1 --2- 172.21.15.170:0/2054732226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b4e70 0x7f52440ccff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52340098e0 tx=0x7f52340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894495a2c667c0d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.066+0000 7f5242ffd700 1 -- 172.21.15.170:0/2054732226 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400b640 con 0x7f52440b4e70 2023-10-08T16:04:33.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.066+0000 7f5242ffd700 1 -- 172.21.15.170:0/2054732226 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400b7a0 con 0x7f52440b4e70 2023-10-08T16:04:33.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.066+0000 7f5242ffd700 1 -- 172.21.15.170:0/2054732226 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523400bb20 con 0x7f52440b4e70 2023-10-08T16:04:33.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 -- 172.21.15.170:0/2054732226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b4e70 msgr2=0x7f52440ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 --2- 172.21.15.170:0/2054732226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b4e70 0x7f52440ccff0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f52340098e0 tx=0x7f52340091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 -- 172.21.15.170:0/2054732226 shutdown_connections 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 --2- 172.21.15.170:0/2054732226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b4e70 0x7f52440ccff0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 -- 172.21.15.170:0/2054732226 >> 172.21.15.170:0/2054732226 conn(0x7f52440d10c0 msgr2=0x7f52440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 -- 172.21.15.170:0/2054732226 shutdown_connections 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.067+0000 7f5248869700 1 -- 172.21.15.170:0/2054732226 wait complete. 2023-10-08T16:04:33.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.068+0000 7f5248869700 1 Processor -- start 2023-10-08T16:04:33.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.068+0000 7f5248869700 1 -- start start 2023-10-08T16:04:33.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440b1b40 0x7f5244149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52440b4e70 0x7f52440c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440c93c0 con 0x7f52440b16d0 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52440c87b0 con 0x7f52440b1b40 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5248869700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52440c8a90 con 0x7f52440b4e70 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34742/0 (socket says 172.21.15.170:34742) 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 -- 172.21.15.170:0/156279520 learned_addr learned my addr 172.21.15.170:0/156279520 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f5243fff700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52440b4e70 0x7f52440c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f523bfff700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440b1b40 0x7f5244149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 -- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52440b4e70 msgr2=0x7f52440c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52440b4e70 0x7f52440c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 -- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440b1b40 msgr2=0x7f5244149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440b1b40 0x7f5244149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.069+0000 7f52437fe700 1 -- 172.21.15.170:0/156279520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009420 con 0x7f52440b16d0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f52437fe700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5244070dd0 tx=0x7f5234017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234000d10 con 0x7f52440b16d0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400b910 con 0x7f52440b16d0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234025670 con 0x7f52440b16d0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52440c8d70 con 0x7f52440b16d0 2023-10-08T16:04:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.070+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5244146c80 con 0x7f52440b16d0 2023-10-08T16:04:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.071+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524415d0c0 con 0x7f52440b16d0 2023-10-08T16:04:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.071+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f5234023070 con 0x7f52440b16d0 2023-10-08T16:04:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.072+0000 7f523b7fe700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 0x7f5224069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.072+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5234085480 con 0x7f52440b16d0 2023-10-08T16:04:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.072+0000 7f523bfff700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 0x7f5224069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.072+0000 7f523bfff700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 0x7f5224069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52440b3ef0 tx=0x7f522c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.076+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5234052bd0 con 0x7f52440b16d0 2023-10-08T16:04:33.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.261+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f524404bb80 con 0x7f52440b16d0 2023-10-08T16:04:33.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.261+0000 7f523b7fe700 1 -- 172.21.15.170:0/156279520 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52340561f0 con 0x7f52440b16d0 2023-10-08T16:04:33.262 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 msgr2=0x7f5224069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 0x7f5224069350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52440b3ef0 tx=0x7f522c009040 comp rx=0 tx=0).stop 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 msgr2=0x7f52440c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5244070dd0 tx=0x7f5234017000 comp rx=0 tx=0).stop 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 shutdown_connections 2023-10-08T16:04:33.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52440b4e70 0x7f52440c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5224066ea0 0x7f5224069350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52440b16d0 0x7f52440c80f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 --2- 172.21.15.170:0/156279520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52440b1b40 0x7f5244149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 >> 172.21.15.170:0/156279520 conn(0x7f52440d10c0 msgr2=0x7f52440b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 shutdown_connections 2023-10-08T16:04:33.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.263+0000 7f5248869700 1 -- 172.21.15.170:0/156279520 wait complete. 2023-10-08T16:04:33.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:04:33.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.389+0000 7f4f4d62e700 1 Processor -- start 2023-10-08T16:04:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.389+0000 7f4f4d62e700 1 -- start start 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4d62e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f48159420 0x7f4f480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4d62e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4f481597f0 0x7f4f480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4d62e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f480b11d0 con 0x7f4f4815d230 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4d62e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f480b14b0 con 0x7f4f48159420 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4d62e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f480b1790 con 0x7f4f481597f0 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f47fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f48159420 0x7f4f480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4ce2d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4f4815d230 0x7f4f4815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53582/0 (socket says 172.21.15.170:53582) 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f4ce2d700 1 -- 172.21.15.170:0/2216780956 learned_addr learned my addr 172.21.15.170:0/2216780956 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:33.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f477fe700 1 --2- 172.21.15.170:0/2216780956 >> v2:172.21.15.191:3300/0 conn(0x7f4f481597f0 0x7f4f480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f47fff700 1 -- 172.21.15.170:0/2216780956 >> v2:172.21.15.191:3300/0 conn(0x7f4f481597f0 msgr2=0x7f4f480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f47fff700 1 --2- 172.21.15.170:0/2216780956 >> v2:172.21.15.191:3300/0 conn(0x7f4f481597f0 0x7f4f480b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f47fff700 1 -- 172.21.15.170:0/2216780956 >> v1:172.21.15.170:6789/0 conn(0x7f4f4815d230 legacy=0x7f4f4815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f47fff700 1 -- 172.21.15.170:0/2216780956 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480b1a70 con 0x7f4f48159420 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.390+0000 7f4f46ffd700 1 -- 172.21.15.170:0/2216780956 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724986749 0 0) 0x7f4f480b11d0 con 0x7f4f4815d230 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f47fff700 1 --2- 172.21.15.170:0/2216780956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f480b5f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f48089750 tx=0x7f4f380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e60840c2bc03d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f46ffd700 1 -- 172.21.15.170:0/2216780956 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011630 con 0x7f4f48159420 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f46ffd700 1 -- 172.21.15.170:0/2216780956 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38011790 con 0x7f4f48159420 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f46ffd700 1 -- 172.21.15.170:0/2216780956 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011b30 con 0x7f4f48159420 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2216780956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 msgr2=0x7f4f480b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2216780956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f480b5f40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4f48089750 tx=0x7f4f380091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2216780956 shutdown_connections 2023-10-08T16:04:33.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2216780956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f480b5f40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2216780956 >> v2:172.21.15.191:3300/0 conn(0x7f4f481597f0 0x7f4f480b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2216780956 >> 172.21.15.170:0/2216780956 conn(0x7f4f480d0660 msgr2=0x7f4f480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:33.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2216780956 shutdown_connections 2023-10-08T16:04:33.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.391+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2216780956 wait complete. 2023-10-08T16:04:33.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.392+0000 7f4f4d62e700 1 Processor -- start 2023-10-08T16:04:33.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 -- start start 2023-10-08T16:04:33.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f481597f0 0x7f4f480ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4815d230 0x7f4f480c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480c7bd0 con 0x7f4f48159420 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4816d580 con 0x7f4f481597f0 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4d62e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480c6870 con 0x7f4f4815d230 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f47fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f47fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34766/0 (socket says 172.21.15.170:34766) 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f47fff700 1 -- 172.21.15.170:0/2462426717 learned_addr learned my addr 172.21.15.170:0/2462426717 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f4ce2d700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4815d230 0x7f4f480c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.393+0000 7f4f477fe700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f481597f0 0x7f4f480ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 -- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4815d230 msgr2=0x7f4f480c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4815d230 0x7f4f480c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 -- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f481597f0 msgr2=0x7f4f480ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f481597f0 0x7f4f480ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 -- 172.21.15.170:0/2462426717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009420 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f47fff700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f380091e0 tx=0x7f4f38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011900 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38011a60 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f480c6b50 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38025640 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.394+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f480c7860 con 0x7f4f48159420 2023-10-08T16:04:33.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.395+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4804c3c0 con 0x7f4f48159420 2023-10-08T16:04:33.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.395+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f4f38025a70 con 0x7f4f48159420 2023-10-08T16:04:33.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.396+0000 7f4f2ffff700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 0x7f4f34069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.396+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4f380858c0 con 0x7f4f48159420 2023-10-08T16:04:33.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.396+0000 7f4f477fe700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 0x7f4f34069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.396+0000 7f4f477fe700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 0x7f4f34069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f480c5b40 tx=0x7f4f3c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.400+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f38052f60 con 0x7f4f48159420 2023-10-08T16:04:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.586+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f480b1d50 con 0x7f4f48159420 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.586+0000 7f4f2ffff700 1 -- 172.21.15.170:0/2462426717 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4f38015060 con 0x7f4f48159420 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:04:33.587 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:04:33.588 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 msgr2=0x7f4f34069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 0x7f4f34069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4f480c5b40 tx=0x7f4f3c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 msgr2=0x7f4f48156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f380091e0 tx=0x7f4f38017000 comp rx=0 tx=0).stop 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 shutdown_connections 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f4815d230 0x7f4f480c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f34066ea0 0x7f4f34069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f48159420 0x7f4f48156480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 --2- 172.21.15.170:0/2462426717 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f481597f0 0x7f4f480ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 >> 172.21.15.170:0/2462426717 conn(0x7f4f480d0660 msgr2=0x7f4f480cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:33.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 shutdown_connections 2023-10-08T16:04:33.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.588+0000 7f4f4d62e700 1 -- 172.21.15.170:0/2462426717 wait complete. 2023-10-08T16:04:33.601 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:04:33.601 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:04:33.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.718+0000 7f6c0d661700 1 Processor -- start 2023-10-08T16:04:33.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.718+0000 7f6c0d661700 1 -- start start 2023-10-08T16:04:33.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c0d661700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6c0806d840 0x7f6c08164b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c0d661700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c080b4570 con 0x7f6c08153f50 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c0d661700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c080b2710 con 0x7f6c08154320 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c0d661700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0815a730 con 0x7f6c0806d840 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c077fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6c0806d840 0x7f6c08164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c06ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6c08153f50 0x7f6c080b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53588/0 (socket says 172.21.15.170:53588) 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c06ffd700 1 -- 172.21.15.170:0/3210358949 learned_addr learned my addr 172.21.15.170:0/3210358949 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c077fe700 1 -- 172.21.15.170:0/3210358949 >> v1:172.21.15.191:6789/0 conn(0x7f6c08154320 legacy=0x7f6c0806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c077fe700 1 -- 172.21.15.170:0/3210358949 >> v1:172.21.15.170:6789/0 conn(0x7f6c08153f50 legacy=0x7f6c080b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c077fe700 1 -- 172.21.15.170:0/3210358949 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0815aa10 con 0x7f6c0806d840 2023-10-08T16:04:33.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c077fe700 1 --2- 172.21.15.170:0/3210358949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0806d840 0x7f6c08164b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009e90 tx=0x7f6c00009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96b3f3dfa79b3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.719+0000 7f6c067fc700 1 -- 172.21.15.170:0/3210358949 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000e070 con 0x7f6c0806d840 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c067fc700 1 -- 172.21.15.170:0/3210358949 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0000bbb0 con 0x7f6c0806d840 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c067fc700 1 -- 172.21.15.170:0/3210358949 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00012430 con 0x7f6c0806d840 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 -- 172.21.15.170:0/3210358949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0806d840 msgr2=0x7f6c08164b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3210358949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0806d840 0x7f6c08164b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009e90 tx=0x7f6c00009780 comp rx=0 tx=0).stop 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 -- 172.21.15.170:0/3210358949 shutdown_connections 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3210358949 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c0806d840 0x7f6c08164b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 -- 172.21.15.170:0/3210358949 >> 172.21.15.170:0/3210358949 conn(0x7f6c0806baf0 msgr2=0x7f6c0806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 -- 172.21.15.170:0/3210358949 shutdown_connections 2023-10-08T16:04:33.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.720+0000 7f6c0d661700 1 -- 172.21.15.170:0/3210358949 wait complete. 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.721+0000 7f6c0d661700 1 Processor -- start 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.721+0000 7f6c0d661700 1 -- start start 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c0806d840 0x7f6c08151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c08153f50 0x7f6c08158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0816ac00 con 0x7f6c08154320 2023-10-08T16:04:33.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c08169ff0 con 0x7f6c0806d840 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c0d661700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0816a2d0 con 0x7f6c08153f50 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34780/0 (socket says 172.21.15.170:34780) 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 -- 172.21.15.170:0/3145634104 learned_addr learned my addr 172.21.15.170:0/3145634104 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c06ffd700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c0806d840 0x7f6c08151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6bfe7fc700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c08153f50 0x7f6c08158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 -- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c08153f50 msgr2=0x7f6c08158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c08153f50 0x7f6c08158ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 -- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c0806d840 msgr2=0x7f6c08151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:33.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c0806d840 0x7f6c08151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.722+0000 7f6c077fe700 1 -- 172.21.15.170:0/3145634104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00009a20 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6c077fe700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c000099f0 tx=0x7f6c00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0001d640 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0001d7e0 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0816a5b0 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0001dba0 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.723+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c08151fd0 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.724+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c080718b0 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.724+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f6c00024070 con 0x7f6c08154320 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.725+0000 7f6bff7fe700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 0x7f6be8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.725+0000 7f6c06ffd700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 0x7f6be8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:33.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.725+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6c00086bb0 con 0x7f6c08154320 2023-10-08T16:04:33.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.725+0000 7f6c06ffd700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 0x7f6be8069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8007130 tx=0x7f6bf8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:33.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.729+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c00054260 con 0x7f6c08154320 2023-10-08T16:04:33.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:33.913+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6c0815acf0 con 0x7f6c08154320 2023-10-08T16:04:34.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.823+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c00054630 con 0x7f6c08154320 2023-10-08T16:04:34.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.825+0000 7f6bff7fe700 1 -- 172.21.15.170:0/3145634104 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6c00054260 con 0x7f6c08154320 2023-10-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 msgr2=0x7f6be8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 0x7f6be8069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8007130 tx=0x7f6bf8007eb0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 msgr2=0x7f6c080c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c000099f0 tx=0x7f6c00018000 comp rx=0 tx=0).stop 2023-10-08T16:04:34.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 shutdown_connections 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6be8066ea0 0x7f6be8069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c08154320 0x7f6c080c9a20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6c0806d840 0x7f6c08151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 --2- 172.21.15.170:0/3145634104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6c08153f50 0x7f6c08158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.830+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 >> 172.21.15.170:0/3145634104 conn(0x7f6c0806baf0 msgr2=0x7f6c080ae610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.831+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 shutdown_connections 2023-10-08T16:04:34.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.831+0000 7f6c0d661700 1 -- 172.21.15.170:0/3145634104 wait complete. 2023-10-08T16:04:34.833 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:04:34.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:04:34.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.970+0000 7f4020d8a700 1 Processor -- start 2023-10-08T16:04:34.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.970+0000 7f4020d8a700 1 -- start start 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.971+0000 7f4020d8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f401c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.971+0000 7f4020d8a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f401c069c70 0x7f401c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.971+0000 7f4020d8a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0ce380 con 0x7f401c0af320 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.971+0000 7f4020d8a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0cec20 con 0x7f401c069c70 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.971+0000 7f4020d8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0afa90 con 0x7f401c159950 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f4019d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f401c0af320 0x7f401c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53602/0 (socket says 172.21.15.170:53602) 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f4019d9b700 1 -- 172.21.15.170:0/2920984471 learned_addr learned my addr 172.21.15.170:0/2920984471 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401a59c700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f401c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401ad9d700 1 --2- 172.21.15.170:0/2920984471 >> v2:172.21.15.191:3300/0 conn(0x7f401c069c70 0x7f401c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416657488 0 0) 0x7f401c0ce380 con 0x7f401c0af320 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4004003570 con 0x7f401c0af320 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2302374544 0 0) 0x7f4004003570 con 0x7f401c0af320 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 msgr2=0x7f401c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f401c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 >> v2:172.21.15.191:3300/0 conn(0x7f401c069c70 msgr2=0x7f401c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 --2- 172.21.15.170:0/2920984471 >> v2:172.21.15.191:3300/0 conn(0x7f401c069c70 0x7f401c0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0ce5d0 con 0x7f401c0af320 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4010001470 con 0x7f401c0af320 2023-10-08T16:04:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 >> v1:172.21.15.170:6789/0 conn(0x7f401c0af320 legacy=0x7f401c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f4004003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f4004004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4004004d50 0x7f40040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cec20 con 0x7f4004003fa0 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0afa90 con 0x7f4004004d50 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400400ba60 con 0x7f401c159950 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f4019d9b700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f4004004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401a59c700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f4004003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.972+0000 7f401ad9d700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4004004d50 0x7f40040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 -- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 msgr2=0x7f4004003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c159950 0x7f4004003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 -- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4004004d50 msgr2=0x7f40040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4004004d50 0x7f40040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 -- 172.21.15.170:0/2920984471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004003570 con 0x7f4004003fa0 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4019d9b700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f4004004810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4010005260 tx=0x7f4010008cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6773685bf63378 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004ba0 con 0x7f4004003fa0 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010005920 con 0x7f4004003fa0 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f401959a700 1 -- 172.21.15.170:0/2920984471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010005cc0 con 0x7f4004003fa0 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4020d8a700 1 -- 172.21.15.170:0/2920984471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 msgr2=0x7f4004004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4020d8a700 1 --2- 172.21.15.170:0/2920984471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f4004004810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4010005260 tx=0x7f4010008cc0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f401ad9d700 1 -- 172.21.15.170:0/2920984471 reap_dead start 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4020d8a700 1 -- 172.21.15.170:0/2920984471 shutdown_connections 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.973+0000 7f4020d8a700 1 -- 172.21.15.170:0/2920984471 >> 172.21.15.170:0/2920984471 conn(0x7f401c068e00 msgr2=0x7f401c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.974+0000 7f4020d8a700 1 -- 172.21.15.170:0/2920984471 shutdown_connections 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.974+0000 7f4020d8a700 1 -- 172.21.15.170:0/2920984471 wait complete. 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 Processor -- start 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 -- start start 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 0x7f401c0c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c14d610 0x7f401c0c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c1570a0 con 0x7f4004003fa0 2023-10-08T16:04:34.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c155d20 con 0x7f401c14d1a0 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4020d8a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c156000 con 0x7f401c14d610 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f401a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f401a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34808/0 (socket says 172.21.15.170:34808) 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f4019d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 0x7f401c0c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f401a59c700 1 -- 172.21.15.170:0/1160102468 learned_addr learned my addr 172.21.15.170:0/1160102468 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.975+0000 7f401ad9d700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c14d610 0x7f401c0c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 -- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c14d610 msgr2=0x7f401c0c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c14d610 0x7f401c0c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 -- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 msgr2=0x7f401c0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 -- 172.21.15.170:0/1160102468 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c1562e0 con 0x7f401c14d1a0 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f401a59c700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4019d9b700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 0x7f401c0c5550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4010015000 tx=0x7f4010012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010004ba0 con 0x7f401c14d1a0 2023-10-08T16:04:34.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4010001640 con 0x7f401c14d1a0 2023-10-08T16:04:34.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c1565c0 con 0x7f401c14d1a0 2023-10-08T16:04:34.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010008aa0 con 0x7f401c14d1a0 2023-10-08T16:04:34.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.976+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c156b80 con 0x7f401c14d1a0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.977+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c156ef0 con 0x7f401c14d1a0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.977+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f401002a430 con 0x7f401c14d1a0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.978+0000 7f400a7fc700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.978+0000 7f401a59c700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.978+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f40100855b0 con 0x7f401c14d1a0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.978+0000 7f401a59c700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 0x7f3ffc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f400c011ae0 tx=0x7f400c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:34.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:34.983+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4010052960 con 0x7f401c14d1a0 2023-10-08T16:04:35.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.163+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f401c0718b0 con 0x7f401c14d1a0 2023-10-08T16:04:35.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.823+0000 7f400a7fc700 1 -- 172.21.15.170:0/1160102468 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4010026020 con 0x7f401c14d1a0 2023-10-08T16:04:35.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 msgr2=0x7f3ffc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 0x7f3ffc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f400c011ae0 tx=0x7f400c00b040 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 msgr2=0x7f401c0c5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 0x7f401c0c5550 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4010015000 tx=0x7f4010012000 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 shutdown_connections 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f401c14d610 0x7f401c0c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ffc066f30 0x7f3ffc0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4004003fa0 0x7f401c0b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 --2- 172.21.15.170:0/1160102468 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f401c14d1a0 0x7f401c0c5550 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4010015000 tx=0x7f4010012000 comp rx=0 tx=0).stop 2023-10-08T16:04:35.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.833+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 >> 172.21.15.170:0/1160102468 conn(0x7f401c068e00 msgr2=0x7f401c0b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:35.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.834+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 shutdown_connections 2023-10-08T16:04:35.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.834+0000 7f4020d8a700 1 -- 172.21.15.170:0/1160102468 wait complete. 2023-10-08T16:04:35.837 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:04:35.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:04:35.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.967+0000 7fd54d067700 1 Processor -- start 2023-10-08T16:04:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.967+0000 7fd54d067700 1 -- start start 2023-10-08T16:04:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54d067700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5480b4e70 0x7fd5480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54d067700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54d067700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5480cfb00 con 0x7fd5480b16d0 2023-10-08T16:04:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54d067700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5480b5c40 con 0x7fd5480b4e70 2023-10-08T16:04:35.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54d067700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b5f20 con 0x7fd5480b1b40 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd547fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5480b4e70 0x7fd5480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54c866700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd547fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd5480b4e70 0x7fd5480ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38120/0 (socket says 172.21.15.170:38120) 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd547fff700 1 -- 172.21.15.170:0/1838119615 learned_addr learned my addr 172.21.15.170:0/1838119615 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.968+0000 7fd54c866700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35092/0 (socket says 172.21.15.170:35092) 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd547fff700 1 -- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 msgr2=0x7fd54815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd547fff700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd547fff700 1 -- 172.21.15.170:0/1838119615 >> v1:172.21.15.191:6789/0 conn(0x7fd5480b16d0 legacy=0x7fd54815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd547fff700 1 -- 172.21.15.170:0/1838119615 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480b6200 con 0x7fd5480b4e70 2023-10-08T16:04:35.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54c866700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd547fff700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b4e70 0x7fd5480ccff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5380099b0 tx=0x7fd538009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1871c2984776a44f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd546ffd700 1 -- 172.21.15.170:0/1838119615 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538009ea0 con 0x7fd5480b4e70 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd546ffd700 1 -- 172.21.15.170:0/1838119615 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800b860 con 0x7fd5480b4e70 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd546ffd700 1 -- 172.21.15.170:0/1838119615 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800bc00 con 0x7fd5480b4e70 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 -- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b4e70 msgr2=0x7fd5480ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b4e70 0x7fd5480ccff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd5380099b0 tx=0x7fd538009250 comp rx=0 tx=0).stop 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 -- 172.21.15.170:0/1838119615 shutdown_connections 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b1b40 0x7fd54815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 --2- 172.21.15.170:0/1838119615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b4e70 0x7fd5480ccff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.969+0000 7fd54d067700 1 -- 172.21.15.170:0/1838119615 >> 172.21.15.170:0/1838119615 conn(0x7fd5480d10c0 msgr2=0x7fd5480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:35.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.970+0000 7fd54d067700 1 -- 172.21.15.170:0/1838119615 shutdown_connections 2023-10-08T16:04:35.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.970+0000 7fd54d067700 1 -- 172.21.15.170:0/1838119615 wait complete. 2023-10-08T16:04:35.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.971+0000 7fd54d067700 1 Processor -- start 2023-10-08T16:04:35.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.971+0000 7fd54d067700 1 -- start start 2023-10-08T16:04:35.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b1b40 0x7fd5480c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b4e70 0x7fd5481477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c7a20 con 0x7fd5480b16d0 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd548147cf0 con 0x7fd5480b1b40 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c7ca0 con 0x7fd5480b4e70 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46092/0 (socket says 172.21.15.170:46092) 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 -- 172.21.15.170:0/3036272985 learned_addr learned my addr 172.21.15.170:0/3036272985 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54c866700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b4e70 0x7fd5481477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd5477fe700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b1b40 0x7fd5480c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 -- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b4e70 msgr2=0x7fd5481477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b4e70 0x7fd5481477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 -- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b1b40 msgr2=0x7fd5480c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b1b40 0x7fd5480c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 -- 172.21.15.170:0/3036272985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd540009a20 con 0x7fd5480b16d0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd547fff700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd548070dd0 tx=0x7fd538017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800bbe0 con 0x7fd5480b16d0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538011420 con 0x7fd5480b16d0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380117c0 con 0x7fd5480b16d0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5380094f0 con 0x7fd5480b16d0 2023-10-08T16:04:35.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.972+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5480c8260 con 0x7fd5480b16d0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.973+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd548088990 con 0x7fd5480b16d0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.974+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd538011bf0 con 0x7fd5480b16d0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.974+0000 7fd52ffff700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 0x7fd5340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.974+0000 7fd5477fe700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 0x7fd5340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.974+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fd538085910 con 0x7fd5480b16d0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.975+0000 7fd5477fe700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 0x7fd5340693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0039e0 tx=0x7fd53c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:35.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:35.979+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd538052d20 con 0x7fd5480b16d0 2023-10-08T16:04:36.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.201+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd548069b20 con 0x7fd5480b16d0 2023-10-08T16:04:36.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.846+0000 7fd52ffff700 1 -- 172.21.15.170:0/3036272985 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd538056340 con 0x7fd5480b16d0 2023-10-08T16:04:36.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.850+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 msgr2=0x7fd5340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:36.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.850+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 0x7fd5340693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0039e0 tx=0x7fd53c007ec0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.850+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 msgr2=0x7fd54816abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:36.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.850+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd548070dd0 tx=0x7fd538017000 comp rx=0 tx=0).stop 2023-10-08T16:04:36.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 shutdown_connections 2023-10-08T16:04:36.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5480b4e70 0x7fd5481477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd534066f30 0x7fd5340693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5480b16d0 0x7fd54816abc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 --2- 172.21.15.170:0/3036272985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5480b1b40 0x7fd5480c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 >> 172.21.15.170:0/3036272985 conn(0x7fd5480d10c0 msgr2=0x7fd5480b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:36.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 shutdown_connections 2023-10-08T16:04:36.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.854+0000 7fd54d067700 1 -- 172.21.15.170:0/3036272985 wait complete. 2023-10-08T16:04:36.858 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T16:04:36.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:36.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.983+0000 7f22b2704700 1 Processor -- start 2023-10-08T16:04:36.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 -- start start 2023-10-08T16:04:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f22ac0aca80 0x7f22ac06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f22ac069d60 0x7f22ac0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0c81f0 0x7f22ac068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0c8650 con 0x7f22ac069d60 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac06bdb0 con 0x7f22ac0aca80 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22b2704700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0cb370 con 0x7f22ac0c81f0 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.984+0000 7f22ab7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f22ac069d60 0x7f22ac0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f22ac069d60 0x7f22ac0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46100/0 (socket says 172.21.15.170:46100) 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 -- 172.21.15.170:0/2067001473 learned_addr learned my addr 172.21.15.170:0/2067001473 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:36.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22abfff700 1 --2- 172.21.15.170:0/2067001473 >> v2:172.21.15.191:3300/0 conn(0x7f22ac0aca80 0x7f22ac06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22b0ca1700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0c81f0 0x7f22ac068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 -- 172.21.15.170:0/2067001473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0c81f0 msgr2=0x7f22ac068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0c81f0 0x7f22ac068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 -- 172.21.15.170:0/2067001473 >> v2:172.21.15.191:3300/0 conn(0x7f22ac0aca80 msgr2=0x7f22ac06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 --2- 172.21.15.170:0/2067001473 >> v2:172.21.15.191:3300/0 conn(0x7f22ac0aca80 0x7f22ac06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 -- 172.21.15.170:0/2067001473 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac159f60 con 0x7f22ac069d60 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22ab7fe700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac069d60 0x7f22ac0c7cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e570 tx=0x7f22a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ccb09d3f8e1db1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22aaffd700 1 -- 172.21.15.170:0/2067001473 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000b860 con 0x7f22ac069d60 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22aaffd700 1 -- 172.21.15.170:0/2067001473 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000b9c0 con 0x7f22ac069d60 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.985+0000 7f22aaffd700 1 -- 172.21.15.170:0/2067001473 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000bd60 con 0x7f22ac069d60 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 -- 172.21.15.170:0/2067001473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac069d60 msgr2=0x7f22ac0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:36.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac069d60 0x7f22ac0c7cb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06e570 tx=0x7f22a0009250 comp rx=0 tx=0).stop 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 -- 172.21.15.170:0/2067001473 shutdown_connections 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0c81f0 0x7f22ac068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 --2- 172.21.15.170:0/2067001473 >> v2:172.21.15.191:3300/0 conn(0x7f22ac0aca80 0x7f22ac06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 --2- 172.21.15.170:0/2067001473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac069d60 0x7f22ac0c7cb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 -- 172.21.15.170:0/2067001473 >> 172.21.15.170:0/2067001473 conn(0x7f22ac0cd670 msgr2=0x7f22ac0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 -- 172.21.15.170:0/2067001473 shutdown_connections 2023-10-08T16:04:36.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.986+0000 7f22b2704700 1 -- 172.21.15.170:0/2067001473 wait complete. 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.987+0000 7f22b2704700 1 Processor -- start 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.987+0000 7f22b2704700 1 -- start start 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22ac069d60 0x7f22ac0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0aca80 0x7f22ac0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0ad710 con 0x7f22ac0c81f0 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0ad9f0 con 0x7f22ac069d60 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac0adcd0 con 0x7f22ac0aca80 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46102/0 (socket says 172.21.15.170:46102) 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 -- 172.21.15.170:0/1443355406 learned_addr learned my addr 172.21.15.170:0/1443355406 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:36.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22ab7fe700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0aca80 0x7f22ac0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22abfff700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22ac069d60 0x7f22ac0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 -- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0aca80 msgr2=0x7f22ac0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0aca80 0x7f22ac0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 -- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22ac069d60 msgr2=0x7f22ac0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22ac069d60 0x7f22ac0b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 -- 172.21.15.170:0/1443355406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00094f0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b0ca1700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a4007f20 tx=0x7f22a4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a400eca0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a400ee20 con 0x7f22ac0c81f0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac0adfb0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4014670 con 0x7f22ac0c81f0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.988+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac0ad420 con 0x7f22ac0c81f0 2023-10-08T16:04:36.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.989+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac0ae4d0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.989+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22a40149e0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.990+0000 7f2293fff700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 0x7f2298069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:36.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.990+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f22a4084cc0 con 0x7f22ac0c81f0 2023-10-08T16:04:36.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.990+0000 7f22abfff700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 0x7f2298069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:36.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.990+0000 7f22abfff700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 0x7f2298069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f22ac15aad0 tx=0x7f229c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:36.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:36.994+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22ac0ae4d0 con 0x7f22ac0c81f0 2023-10-08T16:04:37.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.177+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22ac0ae4d0 con 0x7f22ac0c81f0 2023-10-08T16:04:37.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.177+0000 7f2293fff700 1 -- 172.21.15.170:0/1443355406 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f22a4070c50 con 0x7f22ac0c81f0 2023-10-08T16:04:37.178 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:37.178 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":20,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:36.202889+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:37.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.179+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 msgr2=0x7f2298069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.179+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 0x7f2298069460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f22ac15aad0 tx=0x7f229c008040 comp rx=0 tx=0).stop 2023-10-08T16:04:37.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.179+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 msgr2=0x7f22ac157ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.179+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f22a4007f20 tx=0x7f22a4009860 comp rx=0 tx=0).stop 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.179+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 shutdown_connections 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2298066fb0 0x7f2298069460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f22ac0c81f0 0x7f22ac157ac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f22ac069d60 0x7f22ac0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 --2- 172.21.15.170:0/1443355406 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f22ac0aca80 0x7f22ac0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 >> 172.21.15.170:0/1443355406 conn(0x7f22ac0cd670 msgr2=0x7f22ac068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:37.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 shutdown_connections 2023-10-08T16:04:37.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.180+0000 7f22b2704700 1 -- 172.21.15.170:0/1443355406 wait complete. 2023-10-08T16:04:37.192 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.305+0000 7f06ff59e700 1 Processor -- start 2023-10-08T16:04:37.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.305+0000 7f06ff59e700 1 -- start start 2023-10-08T16:04:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06ff59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 0x7f07000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06ff59e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000b11d0 con 0x7f070015d230 2023-10-08T16:04:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06ff59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07000b14b0 con 0x7f0700159420 2023-10-08T16:04:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06ff59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b1790 con 0x7f07001597f0 2023-10-08T16:04:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f070015d230 0x7f07000b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54646/0 (socket says 172.21.15.170:54646) 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fed9d700 1 -- 172.21.15.170:0/1626891290 learned_addr learned my addr 172.21.15.170:0/1626891290 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06f5d9b700 1 --2- 172.21.15.170:0/1626891290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 0x7f07000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186187005 0 0) 0x7f07000b11d0 con 0x7f070015d230 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06ec003570 con 0x7f070015d230 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536680581 0 0) 0x7f07000b14b0 con 0x7f0700159420 2023-10-08T16:04:37.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07000b11d0 con 0x7f0700159420 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06f5d9b700 1 -- 172.21.15.170:0/1626891290 >> v1:172.21.15.191:6789/0 conn(0x7f0700159420 legacy=0x7f07000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06f5d9b700 1 -- 172.21.15.170:0/1626891290 >> v1:172.21.15.170:6789/0 conn(0x7f070015d230 legacy=0x7f07000b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06f5d9b700 1 -- 172.21.15.170:0/1626891290 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000b1a70 con 0x7f07001597f0 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.306+0000 7f06f5d9b700 1 --2- 172.21.15.170:0/1626891290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 0x7f07000b2270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009910 tx=0x7f06e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5652783ea7605e5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011630 con 0x7f07001597f0 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8011790 con 0x7f07001597f0 2023-10-08T16:04:37.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06fdd9b700 1 -- 172.21.15.170:0/1626891290 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011b30 con 0x7f07001597f0 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 -- 172.21.15.170:0/1626891290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 msgr2=0x7f07000b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 --2- 172.21.15.170:0/1626891290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 0x7f07000b2270 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009910 tx=0x7f06e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 -- 172.21.15.170:0/1626891290 shutdown_connections 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 --2- 172.21.15.170:0/1626891290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07001597f0 0x7f07000b2270 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 -- 172.21.15.170:0/1626891290 >> 172.21.15.170:0/1626891290 conn(0x7f07000d0660 msgr2=0x7f07000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 -- 172.21.15.170:0/1626891290 shutdown_connections 2023-10-08T16:04:37.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.307+0000 7f06ff59e700 1 -- 172.21.15.170:0/1626891290 wait complete. 2023-10-08T16:04:37.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.308+0000 7f06ff59e700 1 Processor -- start 2023-10-08T16:04:37.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.308+0000 7f06ff59e700 1 -- start start 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0700159420 0x7f070016a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07001597f0 0x7f0700156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000c9170 con 0x7f070015d230 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000cc040 con 0x7f07001597f0 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06ff59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070016ad20 con 0x7f0700159420 2023-10-08T16:04:37.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46110/0 (socket says 172.21.15.170:46110) 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 -- 172.21.15.170:0/2292451644 learned_addr learned my addr 172.21.15.170:0/2292451644 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06f5d9b700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07001597f0 0x7f0700156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fe59c700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0700159420 0x7f070016a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 -- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0700159420 msgr2=0x7f070016a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0700159420 0x7f070016a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 -- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07001597f0 msgr2=0x7f0700156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07001597f0 0x7f0700156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.309+0000 7f06fed9d700 1 -- 172.21.15.170:0/2292451644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009420 con 0x7f070015d230 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06fed9d700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06f800cc50 tx=0x7f06f8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800a810 con 0x7f070015d230 2023-10-08T16:04:37.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f800a990 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000cb430 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f800ad50 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.310+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000cb9f0 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.311+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07000b3400 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.311+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f06f8007300 con 0x7f070015d230 2023-10-08T16:04:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.311+0000 7f06f6ffd700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.312+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06f80847a0 con 0x7f070015d230 2023-10-08T16:04:37.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.312+0000 7f06fe59c700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.312+0000 7f06fe59c700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06f0007fd0 tx=0x7f06f0007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.316+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f06f8051a90 con 0x7f070015d230 2023-10-08T16:04:37.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.497+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f070004c3c0 con 0x7f070015d230 2023-10-08T16:04:37.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.497+0000 7f06f6ffd700 1 -- 172.21.15.170:0/2292451644 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f06f80550b0 con 0x7f070015d230 2023-10-08T16:04:37.499 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:37.499 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":20,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:36.202889+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:37.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 msgr2=0x7f06e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 0x7f06e00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06f0007fd0 tx=0x7f06f0007d50 comp rx=0 tx=0).stop 2023-10-08T16:04:37.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 msgr2=0x7f07000ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f06f800cc50 tx=0x7f06f8001a40 comp rx=0 tx=0).stop 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 shutdown_connections 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f06e0066f30 0x7f06e00693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f070015d230 0x7f07000ce980 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0700159420 0x7f070016a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 --2- 172.21.15.170:0/2292451644 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07001597f0 0x7f0700156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 >> 172.21.15.170:0/2292451644 conn(0x7f07000d0660 msgr2=0x7f0700158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:37.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 shutdown_connections 2023-10-08T16:04:37.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.500+0000 7f06ff59e700 1 -- 172.21.15.170:0/2292451644 wait complete. 2023-10-08T16:04:37.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:04:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.629+0000 7f40fe72f700 1 Processor -- start 2023-10-08T16:04:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.629+0000 7f40fe72f700 1 -- start start 2023-10-08T16:04:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fe72f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40f806bcd0 0x7f40f8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fe72f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 0x7f40f8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fe72f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f815c2a0 con 0x7f40f80c8e00 2023-10-08T16:04:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fe72f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f815b690 con 0x7f40f806bcd0 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fe72f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f815b970 con 0x7f40f806c0a0 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fcccc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f40f80c8e00 0x7f40f80c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54660/0 (socket says 172.21.15.170:54660) 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40fcccc700 1 -- 172.21.15.170:0/220201543 learned_addr learned my addr 172.21.15.170:0/220201543 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40f7fff700 1 --2- 172.21.15.170:0/220201543 >> v2:172.21.15.191:3300/0 conn(0x7f40f806bcd0 0x7f40f8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40f77fe700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 0x7f40f8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40f6ffd700 1 -- 172.21.15.170:0/220201543 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282356153 0 0) 0x7f40f815c2a0 con 0x7f40f80c8e00 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.630+0000 7f40f6ffd700 1 -- 172.21.15.170:0/220201543 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e0003570 con 0x7f40f80c8e00 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f7fff700 1 -- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 msgr2=0x7f40f8141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f7fff700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 0x7f40f8141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f7fff700 1 -- 172.21.15.170:0/220201543 >> v1:172.21.15.170:6789/0 conn(0x7f40f80c8e00 legacy=0x7f40f80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f7fff700 1 -- 172.21.15.170:0/220201543 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f815bc50 con 0x7f40f806bcd0 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f77fe700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 0x7f40f8141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f7fff700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f806bcd0 0x7f40f8140df0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098e0 tx=0x7f40e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=184f667928642636 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f6ffd700 1 -- 172.21.15.170:0/220201543 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8011630 con 0x7f40f806bcd0 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f6ffd700 1 -- 172.21.15.170:0/220201543 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8011790 con 0x7f40f806bcd0 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40f6ffd700 1 -- 172.21.15.170:0/220201543 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8011b30 con 0x7f40f806bcd0 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 -- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f806bcd0 msgr2=0x7f40f8140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f806bcd0 0x7f40f8140df0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098e0 tx=0x7f40e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 -- 172.21.15.170:0/220201543 shutdown_connections 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f806bcd0 0x7f40f8140df0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 --2- 172.21.15.170:0/220201543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806c0a0 0x7f40f8141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 -- 172.21.15.170:0/220201543 >> 172.21.15.170:0/220201543 conn(0x7f40f80ce600 msgr2=0x7f40f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 -- 172.21.15.170:0/220201543 shutdown_connections 2023-10-08T16:04:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.631+0000 7f40fe72f700 1 -- 172.21.15.170:0/220201543 wait complete. 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.632+0000 7f40fe72f700 1 Processor -- start 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 -- start start 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806bcd0 0x7f40f80b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f80c8e00 0x7f40f80c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80b2e80 con 0x7f40f806c0a0 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f813da60 con 0x7f40f80c8e00 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40fe72f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f813dd20 con 0x7f40f806bcd0 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.633+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46120/0 (socket says 172.21.15.170:46120) 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 -- 172.21.15.170:0/671575119 learned_addr learned my addr 172.21.15.170:0/671575119 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f7fff700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806bcd0 0x7f40f80b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40fcccc700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f80c8e00 0x7f40f80c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 -- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806bcd0 msgr2=0x7f40f80b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806bcd0 0x7f40f80b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 -- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f80c8e00 msgr2=0x7f40f80c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f80c8e00 0x7f40f80c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 -- 172.21.15.170:0/671575119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec009420 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40f77fe700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ec80 tx=0x7f40ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00c920 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec00ca80 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e8009420 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec00ce00 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.634+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f80b2ba0 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.635+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f804c3c0 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.635+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40ec013680 con 0x7f40f806c0a0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.636+0000 7f40dffff700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 0x7f40e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.636+0000 7f40f7fff700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 0x7f40e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.636+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40ec084c40 con 0x7f40f806c0a0 2023-10-08T16:04:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.636+0000 7f40f7fff700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 0x7f40e40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40e80091e0 tx=0x7f40e8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.640+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40ec051eb0 con 0x7f40f806c0a0 2023-10-08T16:04:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:37.869+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f40f806c9d0 con 0x7f40f806c0a0 2023-10-08T16:04:38.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.853+0000 7f40dffff700 1 -- 172.21.15.170:0/671575119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f40ec0170b0 con 0x7f40f806c0a0 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 msgr2=0x7f40e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 0x7f40e40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f40e80091e0 tx=0x7f40e8017000 comp rx=0 tx=0).stop 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 msgr2=0x7f40f81572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ec80 tx=0x7f40ec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 shutdown_connections 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40f80c8e00 0x7f40f80c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40f806bcd0 0x7f40f80b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:38.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40e4066f30 0x7f40e40693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:38.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 --2- 172.21.15.170:0/671575119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40f806c0a0 0x7f40f81572a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:38.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 >> 172.21.15.170:0/671575119 conn(0x7f40f80ce600 msgr2=0x7f40f80c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:38.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 shutdown_connections 2023-10-08T16:04:38.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.856+0000 7f40fe72f700 1 -- 172.21.15.170:0/671575119 wait complete. 2023-10-08T16:04:38.872 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:04:38.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.994+0000 7f4d015b9700 1 Processor -- start 2023-10-08T16:04:38.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.994+0000 7f4d015b9700 1 -- start start 2023-10-08T16:04:38.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d015b9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cfc0b4e70 0x7f4cfc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d015b9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4cfc0b16d0 0x7f4cfc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d015b9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0b2080 con 0x7f4cfc15d240 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d015b9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0cfb00 con 0x7f4cfc0b4e70 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d015b9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0b5c40 con 0x7f4cfc0b16d0 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4cfc0b4e70 0x7f4cfc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d00db8700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4cfc15d240 0x7f4cfc15d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54664/0 (socket says 172.21.15.170:54664) 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4d00db8700 1 -- 172.21.15.170:0/1204513439 learned_addr learned my addr 172.21.15.170:0/1204513439 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfb7fe700 1 --2- 172.21.15.170:0/1204513439 >> v2:172.21.15.191:3300/0 conn(0x7f4cfc0b16d0 0x7f4cfc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfbfff700 1 -- 172.21.15.170:0/1204513439 >> v2:172.21.15.191:3300/0 conn(0x7f4cfc0b16d0 msgr2=0x7f4cfc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:38.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfbfff700 1 --2- 172.21.15.170:0/1204513439 >> v2:172.21.15.191:3300/0 conn(0x7f4cfc0b16d0 0x7f4cfc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfbfff700 1 -- 172.21.15.170:0/1204513439 >> v1:172.21.15.170:6789/0 conn(0x7f4cfc15d240 legacy=0x7f4cfc15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfaffd700 1 -- 172.21.15.170:0/1204513439 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261633652 0 0) 0x7f4cfc0b2080 con 0x7f4cfc15d240 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.995+0000 7f4cfbfff700 1 -- 172.21.15.170:0/1204513439 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0b5f20 con 0x7f4cfc0b4e70 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4cfbfff700 1 --2- 172.21.15.170:0/1204513439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc0b4e70 0x7f4cfc0ccff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc089830 tx=0x7f4cec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d27be1d2403e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4cfaffd700 1 -- 172.21.15.170:0/1204513439 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec011630 con 0x7f4cfc0b4e70 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4cfaffd700 1 -- 172.21.15.170:0/1204513439 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec011790 con 0x7f4cfc0b4e70 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4cfaffd700 1 -- 172.21.15.170:0/1204513439 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec011b30 con 0x7f4cfc0b4e70 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 -- 172.21.15.170:0/1204513439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc0b4e70 msgr2=0x7f4cfc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1204513439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc0b4e70 0x7f4cfc0ccff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc089830 tx=0x7f4cec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 -- 172.21.15.170:0/1204513439 shutdown_connections 2023-10-08T16:04:39.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1204513439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc0b4e70 0x7f4cfc0ccff0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1204513439 >> v2:172.21.15.191:3300/0 conn(0x7f4cfc0b16d0 0x7f4cfc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 -- 172.21.15.170:0/1204513439 >> 172.21.15.170:0/1204513439 conn(0x7f4cfc0d10c0 msgr2=0x7f4cfc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 -- 172.21.15.170:0/1204513439 shutdown_connections 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.996+0000 7f4d015b9700 1 -- 172.21.15.170:0/1204513439 wait complete. 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.997+0000 7f4d015b9700 1 Processor -- start 2023-10-08T16:04:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.997+0000 7f4d015b9700 1 -- start start 2023-10-08T16:04:39.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cfc0b16d0 0x7f4cfc16a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc0b4e70 0x7f4cfc15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc16a740 con 0x7f4cfc15d240 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc16aa20 con 0x7f4cfc0b16d0 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d015b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc16ad00 con 0x7f4cfc0b4e70 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d00db8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d00db8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46134/0 (socket says 172.21.15.170:46134) 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4d00db8700 1 -- 172.21.15.170:0/1401516242 learned_addr learned my addr 172.21.15.170:0/1401516242 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4cfbfff700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cfc0b16d0 0x7f4cfc16a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.998+0000 7f4cfb7fe700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc0b4e70 0x7f4cfc15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 -- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc0b4e70 msgr2=0x7f4cfc15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc0b4e70 0x7f4cfc15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 -- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cfc0b16d0 msgr2=0x7f4cfc16a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cfc0b16d0 0x7f4cfc16a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 -- 172.21.15.170:0/1401516242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec009420 con 0x7f4cfc15d240 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d00db8700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400cbf0 tx=0x7f4cf40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf400a9c0 con 0x7f4cfc15d240 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf400ab20 con 0x7f4cfc15d240 2023-10-08T16:04:39.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40070f0 con 0x7f4cfc15d240 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc16afe0 con 0x7f4cfc15d240 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:38.999+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc158860 con 0x7f4cfc15d240 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.000+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc088990 con 0x7f4cfc15d240 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.000+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cf4007520 con 0x7f4cfc15d240 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.001+0000 7f4ce3fff700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 0x7f4ce8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.001+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4cf40849b0 con 0x7f4cfc15d240 2023-10-08T16:04:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.001+0000 7f4cfbfff700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 0x7f4ce8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.001+0000 7f4cfbfff700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 0x7f4ce8069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0093f0 tx=0x7f4cec017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.005+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cf4051c00 con 0x7f4cfc15d240 2023-10-08T16:04:39.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.230+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4cfc04bb80 con 0x7f4cfc15d240 2023-10-08T16:04:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.853+0000 7f4ce3fff700 1 -- 172.21.15.170:0/1401516242 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4cf4055220 con 0x7f4cfc15d240 2023-10-08T16:04:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 msgr2=0x7f4ce8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 0x7f4ce8069460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0093f0 tx=0x7f4cec017000 comp rx=0 tx=0).stop 2023-10-08T16:04:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 msgr2=0x7f4cfc15ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400cbf0 tx=0x7f4cf40019b0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 shutdown_connections 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ce8066fb0 0x7f4ce8069460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4cfc15d240 0x7f4cfc15ee40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4cfc0b16d0 0x7f4cfc16a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 --2- 172.21.15.170:0/1401516242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4cfc0b4e70 0x7f4cfc15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 >> 172.21.15.170:0/1401516242 conn(0x7f4cfc0d10c0 msgr2=0x7f4cfc158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 shutdown_connections 2023-10-08T16:04:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.856+0000 7f4d015b9700 1 -- 172.21.15.170:0/1401516242 wait complete. 2023-10-08T16:04:39.871 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:39.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.987+0000 7f2837250700 1 Processor -- start 2023-10-08T16:04:39.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.987+0000 7f2837250700 1 -- start start 2023-10-08T16:04:39.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2837250700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f28301597f0 0x7f28300b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2837250700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300b11d0 con 0x7f283015d230 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2837250700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300b14b0 con 0x7f2830159420 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2837250700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28300b1790 con 0x7f28301597f0 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2835a4d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f28301597f0 0x7f28300b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2836a4f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f283015d230 0x7f283015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54672/0 (socket says 172.21.15.170:54672) 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2836a4f700 1 -- 172.21.15.170:0/3486852164 learned_addr learned my addr 172.21.15.170:0/3486852164 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2835a4d700 1 -- 172.21.15.170:0/3486852164 >> v1:172.21.15.191:6789/0 conn(0x7f2830159420 legacy=0x7f28300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2835a4d700 1 -- 172.21.15.170:0/3486852164 >> v1:172.21.15.170:6789/0 conn(0x7f283015d230 legacy=0x7f283015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2835a4d700 1 -- 172.21.15.170:0/3486852164 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28300b1a70 con 0x7f28301597f0 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f283524c700 1 -- 172.21.15.170:0/3486852164 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258179844 0 0) 0x7f28300b11d0 con 0x7f283015d230 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f2835a4d700 1 --2- 172.21.15.170:0/3486852164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28301597f0 0x7f28300b2270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28200098e0 tx=0x7f28200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9778d37e1d290356 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f283524c700 1 -- 172.21.15.170:0/3486852164 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820009e80 con 0x7f28301597f0 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f283524c700 1 -- 172.21.15.170:0/3486852164 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282000b820 con 0x7f28301597f0 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.988+0000 7f283524c700 1 -- 172.21.15.170:0/3486852164 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282000bba0 con 0x7f28301597f0 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 -- 172.21.15.170:0/3486852164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28301597f0 msgr2=0x7f28300b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 --2- 172.21.15.170:0/3486852164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28301597f0 0x7f28300b2270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28200098e0 tx=0x7f28200091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 -- 172.21.15.170:0/3486852164 shutdown_connections 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 --2- 172.21.15.170:0/3486852164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28301597f0 0x7f28300b2270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 -- 172.21.15.170:0/3486852164 >> 172.21.15.170:0/3486852164 conn(0x7f28300d0660 msgr2=0x7f28300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 -- 172.21.15.170:0/3486852164 shutdown_connections 2023-10-08T16:04:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.989+0000 7f2837250700 1 -- 172.21.15.170:0/3486852164 wait complete. 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.990+0000 7f2837250700 1 Processor -- start 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.990+0000 7f2837250700 1 -- start start 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2830159420 0x7f28301544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28301597f0 0x7f28300c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283015b5a0 con 0x7f283015d230 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283015aa20 con 0x7f2830159420 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283015ad00 con 0x7f28301597f0 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46148/0 (socket says 172.21.15.170:46148) 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 -- 172.21.15.170:0/3464727651 learned_addr learned my addr 172.21.15.170:0/3464727651 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2835a4d700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28301597f0 0x7f28300c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 -- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28301597f0 msgr2=0x7f28300c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f283624e700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2830159420 0x7f28301544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28301597f0 0x7f28300c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 -- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2830159420 msgr2=0x7f28301544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2830159420 0x7f28301544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:39.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 -- 172.21.15.170:0/3464727651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009420 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2836a4f700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f282800cbc0 tx=0x7f2828001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800a940 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800aaa0 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800ae60 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283015afe0 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.991+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283015a660 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.992+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283004c3c0 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.993+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2828007300 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.993+0000 7f28267fc700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.993+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2828084910 con 0x7f283015d230 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.993+0000 7f283624e700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:39.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.993+0000 7f283624e700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f282c0027e0 tx=0x7f282c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:39.997+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2828051c00 con 0x7f283015d230 2023-10-08T16:04:40.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.238+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28300b1d50 con 0x7f283015d230 2023-10-08T16:04:40.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.239+0000 7f28267fc700 1 -- 172.21.15.170:0/3464727651 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f2828055220 con 0x7f283015d230 2023-10-08T16:04:40.241 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:40.241 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:04:40.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 msgr2=0x7f28140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f282c0027e0 tx=0x7f282c009040 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 msgr2=0x7f28300c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f282800cbc0 tx=0x7f2828001980 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 shutdown_connections 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f283015d230 0x7f28300c7560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2830159420 0x7f28301544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 --2- 172.21.15.170:0/3464727651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f28301597f0 0x7f28300c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 >> 172.21.15.170:0/3464727651 conn(0x7f28300d0660 msgr2=0x7f2830155450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 shutdown_connections 2023-10-08T16:04:40.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.241+0000 7f2837250700 1 -- 172.21.15.170:0/3464727651 wait complete. 2023-10-08T16:04:40.244 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:04:40.254 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:40.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.368+0000 7f49aebd9700 1 Processor -- start 2023-10-08T16:04:40.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.368+0000 7f49aebd9700 1 -- start start 2023-10-08T16:04:40.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49aebd9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49aebd9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f49a815db10 0x7f49a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49aebd9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a815a160 con 0x7f49a8159890 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49aebd9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a80d13a0 con 0x7f49a815db10 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49aebd9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a815e140 con 0x7f49a8159c60 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49ac975700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f49a8159890 0x7f49a815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54682/0 (socket says 172.21.15.170:54682) 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49ac975700 1 -- 172.21.15.170:0/568987052 learned_addr learned my addr 172.21.15.170:0/568987052 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49ad176700 1 --2- 172.21.15.170:0/568987052 >> v2:172.21.15.191:3300/0 conn(0x7f49a815db10 0x7f49a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241294464 0 0) 0x7f49a815a160 con 0x7f49a8159890 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4998003570 con 0x7f49a8159890 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3788940991 0 0) 0x7f4998003570 con 0x7f49a8159890 2023-10-08T16:04:40.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 msgr2=0x7f49a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 >> v2:172.21.15.191:3300/0 conn(0x7f49a815db10 msgr2=0x7f49a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 --2- 172.21.15.170:0/568987052 >> v2:172.21.15.191:3300/0 conn(0x7f49a815db10 0x7f49a8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a815a3d0 con 0x7f49a8159890 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f499c001470 con 0x7f49a8159890 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49ad176700 1 --2- 172.21.15.170:0/568987052 >> v2:172.21.15.191:3300/0 conn(0x7f49a815db10 0x7f49a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 >> v1:172.21.15.170:6789/0 conn(0x7f49a8159890 legacy=0x7f49a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998003a60 0x7f4998003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4998004810 0x7f4998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80d13a0 con 0x7f4998004810 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a815e140 con 0x7f4998003a60 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.369+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499800ba60 con 0x7f49a8159c60 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4998004810 0x7f4998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49ac975700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998003a60 0x7f4998003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49ad176700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 -- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 msgr2=0x7f49980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a8159c60 0x7f49980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 -- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998003a60 msgr2=0x7f4998003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998003a60 0x7f4998003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 -- 172.21.15.170:0/568987052 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009420 con 0x7f4998004810 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a7fff700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4998004810 0x7f4998004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f49a815a090 tx=0x7f49940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3b56dc069c1b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994009740 con 0x7f4998004810 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499400e920 con 0x7f4998004810 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49a77fe700 1 -- 172.21.15.170:0/568987052 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400ece0 con 0x7f4998004810 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49aebd9700 1 -- 172.21.15.170:0/568987052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4998004810 msgr2=0x7f4998004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49aebd9700 1 --2- 172.21.15.170:0/568987052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4998004810 0x7f4998004c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f49a815a090 tx=0x7f49940091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.370+0000 7f49ad176700 1 -- 172.21.15.170:0/568987052 reap_dead start 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.371+0000 7f49aebd9700 1 -- 172.21.15.170:0/568987052 shutdown_connections 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.371+0000 7f49aebd9700 1 -- 172.21.15.170:0/568987052 >> 172.21.15.170:0/568987052 conn(0x7f49a8068e20 msgr2=0x7f49a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:40.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.371+0000 7f49aebd9700 1 -- 172.21.15.170:0/568987052 shutdown_connections 2023-10-08T16:04:40.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.371+0000 7f49aebd9700 1 -- 172.21.15.170:0/568987052 wait complete. 2023-10-08T16:04:40.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 Processor -- start 2023-10-08T16:04:40.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 -- start start 2023-10-08T16:04:40.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 0x7f49a80cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a816df50 0x7f49a8147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80b3e90 con 0x7f49a816d2c0 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a816dc70 con 0x7f4998004810 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.372+0000 7f49aebd9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a80c78a0 con 0x7f49a816df50 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ac975700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 0x7f49a80cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49a7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a816df50 0x7f49a8147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ac975700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 0x7f49a80cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38226/0 (socket says 172.21.15.170:38226) 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ac975700 1 -- 172.21.15.170:0/2614840928 learned_addr learned my addr 172.21.15.170:0/2614840928 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46170/0 (socket says 172.21.15.170:46170) 2023-10-08T16:04:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 -- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a816df50 msgr2=0x7f49a8147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a816df50 0x7f49a8147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 -- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 msgr2=0x7f49a80cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 0x7f49a80cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 -- 172.21.15.170:0/2614840928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80c7af0 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49ad176700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49a815dee0 tx=0x7f49a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000e040 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a000f040 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a80c7dd0 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a000fea0 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.373+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a80b3b20 con 0x7f49a816d2c0 2023-10-08T16:04:40.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.374+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a804c3c0 con 0x7f49a816d2c0 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.374+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49a0015640 con 0x7f49a816d2c0 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.375+0000 7f49a4ff9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.375+0000 7f49ac975700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.375+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49a0084dd0 con 0x7f49a816d2c0 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.375+0000 7f49ac975700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 0x7f498c0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f499c000e10 tx=0x7f499c001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.379+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49a0052040 con 0x7f49a816d2c0 2023-10-08T16:04:40.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.561+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49a80b5f30 con 0x7f49a816d2c0 2023-10-08T16:04:40.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.562+0000 7f49a4ff9700 1 -- 172.21.15.170:0/2614840928 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f49a0055660 con 0x7f49a816d2c0 2023-10-08T16:04:40.564 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:40.564 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:40.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 msgr2=0x7f498c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 0x7f498c0693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f499c000e10 tx=0x7f499c001e90 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 msgr2=0x7f49a816d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49a815dee0 tx=0x7f49a000b040 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 shutdown_connections 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f498c066f30 0x7f498c0693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49a816d2c0 0x7f49a816d730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4998004810 0x7f49a80cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 --2- 172.21.15.170:0/2614840928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49a816df50 0x7f49a8147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 >> 172.21.15.170:0/2614840928 conn(0x7f49a8068e20 msgr2=0x7f49a81584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 shutdown_connections 2023-10-08T16:04:40.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.565+0000 7f49aebd9700 1 -- 172.21.15.170:0/2614840928 wait complete. 2023-10-08T16:04:40.571 INFO:tasks.ceph.mds.a.smithi170.stderr:2023-10-08T16:04:40.570+0000 7fa8bed11700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:04:40.578 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:04:40.579 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:04:40.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.691+0000 7f0674ffc700 1 Processor -- start 2023-10-08T16:04:40.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.691+0000 7f0674ffc700 1 -- start start 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f0674ffc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0670159890 0x7f067015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f0674ffc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f067015a160 con 0x7f0670069560 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f0674ffc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06700d13a0 con 0x7f0670159c60 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f0674ffc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067015e140 con 0x7f0670159890 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f066e59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0670159890 0x7f067015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f066ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0670069560 0x7f06700ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54698/0 (socket says 172.21.15.170:54698) 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f066ed9d700 1 -- 172.21.15.170:0/3047074360 learned_addr learned my addr 172.21.15.170:0/3047074360 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f066e59c700 1 -- 172.21.15.170:0/3047074360 >> v1:172.21.15.191:6789/0 conn(0x7f0670159c60 legacy=0x7f067015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.692+0000 7f066e59c700 1 -- 172.21.15.170:0/3047074360 >> v1:172.21.15.170:6789/0 conn(0x7f0670069560 legacy=0x7f06700ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066dd9b700 1 -- 172.21.15.170:0/3047074360 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534048717 0 0) 0x7f067015a160 con 0x7f0670069560 2023-10-08T16:04:40.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066e59c700 1 -- 172.21.15.170:0/3047074360 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067015a3d0 con 0x7f0670159890 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066e59c700 1 --2- 172.21.15.170:0/3047074360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015abd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06600098e0 tx=0x7f06600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15cb1c5b3c636b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066dd9b700 1 -- 172.21.15.170:0/3047074360 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660009e80 con 0x7f0670159890 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066dd9b700 1 -- 172.21.15.170:0/3047074360 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066000b860 con 0x7f0670159890 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f066dd9b700 1 -- 172.21.15.170:0/3047074360 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066000bbe0 con 0x7f0670159890 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 -- 172.21.15.170:0/3047074360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 msgr2=0x7f067015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 --2- 172.21.15.170:0/3047074360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015abd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06600098e0 tx=0x7f06600091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 -- 172.21.15.170:0/3047074360 shutdown_connections 2023-10-08T16:04:40.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 --2- 172.21.15.170:0/3047074360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015abd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 -- 172.21.15.170:0/3047074360 >> 172.21.15.170:0/3047074360 conn(0x7f0670068e20 msgr2=0x7f06700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:40.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 -- 172.21.15.170:0/3047074360 shutdown_connections 2023-10-08T16:04:40.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.693+0000 7f0674ffc700 1 -- 172.21.15.170:0/3047074360 wait complete. 2023-10-08T16:04:40.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 Processor -- start 2023-10-08T16:04:40.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 -- start start 2023-10-08T16:04:40.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0670069560 0x7f0670154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0670159c60 0x7f067015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067014cba0 con 0x7f0670159890 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067016b440 con 0x7f0670159c60 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0674ffc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067015ca30 con 0x7f0670069560 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0665d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0665d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46196/0 (socket says 172.21.15.170:46196) 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.695+0000 7f0665d9b700 1 -- 172.21.15.170:0/2686045516 learned_addr learned my addr 172.21.15.170:0/2686045516 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f066ed9d700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0670159c60 0x7f067015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f066e59c700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0670069560 0x7f0670154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 -- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0670069560 msgr2=0x7f0670154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0670069560 0x7f0670154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 -- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0670159c60 msgr2=0x7f067015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0670159c60 0x7f067015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 -- 172.21.15.170:0/2686045516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660009420 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0665d9b700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065800c4e0 tx=0x7f065800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658011590 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0658011710 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067016b160 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658011ad0 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.696+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067015cfb0 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.697+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067004c3c0 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.697+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f065801b3b0 con 0x7f0670159890 2023-10-08T16:04:40.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.698+0000 7f0666ffd700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:40.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.698+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06580838c0 con 0x7f0670159890 2023-10-08T16:04:40.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.698+0000 7f066e59c700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:40.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.698+0000 7f066e59c700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 0x7f06500693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06600093f0 tx=0x7f0660017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:40.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.702+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0658050b30 con 0x7f0670159890 2023-10-08T16:04:40.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.882+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0670158ef0 con 0x7f0650066f30 2023-10-08T16:04:40.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.890+0000 7f0666ffd700 1 -- 172.21.15.170:0/2686045516 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264605 (secure 0 0 0) 0x7f0670158ef0 con 0x7f0650066f30 2023-10-08T16:04:40.894 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:40.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 msgr2=0x7f06500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 0x7f06500693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f06600093f0 tx=0x7f0660017000 comp rx=0 tx=0).stop 2023-10-08T16:04:40.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 msgr2=0x7f067015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f065800c4e0 tx=0x7f065800c7f0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 shutdown_connections 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0670159c60 0x7f067015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0670069560 0x7f0670154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0650066f30 0x7f06500693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 --2- 172.21.15.170:0/2686045516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0670159890 0x7f067015d4d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 >> 172.21.15.170:0/2686045516 conn(0x7f0670068e20 msgr2=0x7f0670152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 shutdown_connections 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:40.893+0000 7f0674ffc700 1 -- 172.21.15.170:0/2686045516 wait complete. 2023-10-08T16:04:40.897 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:04:40.906 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T16:04:40.225053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.406655"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854867+0000","last_change":"2023-10-08T16:04:36.847335+0000","last_active":"2023-10-08T16:04:36.854867+0000","last_peered":"2023-10-08T16:04:36.854867+0000","last_clean":"2023-10-08T16:04:36.854867+0000","last_became_active":"2023-10-08T16:04:36.846900+0000","last_became_peered":"2023-10-08T16:04:36.846900+0000","last_unstale":"2023-10-08T16:04:36.854867+0000","last_undegraded":"2023-10-08T16:04:36.854867+0000","last_fullsized":"2023-10-08T16:04:36.854867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.847766+0000","last_change":"2023-10-08T16:04:36.841973+0000","last_active":"2023-10-08T16:04:36.847766+0000","last_peered":"2023-10-08T16:04:36.847766+0000","last_clean":"2023-10-08T16:04:36.847766+0000","last_became_active":"2023-10-08T16:04:36.841752+0000","last_became_peered":"2023-10-08T16:04:36.841752+0000","last_unstale":"2023-10-08T16:04:36.847766+0000","last_undegraded":"2023-10-08T16:04:36.847766+0000","last_fullsized":"2023-10-08T16:04:36.847766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844363+0000","last_change":"2023-10-08T16:04:36.841982+0000","last_active":"2023-10-08T16:04:36.844363+0000","last_peered":"2023-10-08T16:04:36.844363+0000","last_clean":"2023-10-08T16:04:36.844363+0000","last_became_active":"2023-10-08T16:04:36.841738+0000","last_became_peered":"2023-10-08T16:04:36.841738+0000","last_unstale":"2023-10-08T16:04:36.844363+0000","last_undegraded":"2023-10-08T16:04:36.844363+0000","last_fullsized":"2023-10-08T16:04:36.844363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:28.994728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861145+0000","last_change":"2023-10-08T16:04:36.857688+0000","last_active":"2023-10-08T16:04:36.861145+0000","last_peered":"2023-10-08T16:04:36.861145+0000","last_clean":"2023-10-08T16:04:36.861145+0000","last_became_active":"2023-10-08T16:04:36.857001+0000","last_became_peered":"2023-10-08T16:04:36.857001+0000","last_unstale":"2023-10-08T16:04:36.861145+0000","last_undegraded":"2023-10-08T16:04:36.861145+0000","last_fullsized":"2023-10-08T16:04:36.861145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848398+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848398+0000","last_peered":"2023-10-08T16:04:36.848398+0000","last_clean":"2023-10-08T16:04:36.848398+0000","last_became_active":"2023-10-08T16:04:36.842806+0000","last_became_peered":"2023-10-08T16:04:36.842806+0000","last_unstale":"2023-10-08T16:04:36.848398+0000","last_undegraded":"2023-10-08T16:04:36.848398+0000","last_fullsized":"2023-10-08T16:04:36.848398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854094+0000","last_change":"2023-10-08T16:04:36.845475+0000","last_active":"2023-10-08T16:04:36.854094+0000","last_peered":"2023-10-08T16:04:36.854094+0000","last_clean":"2023-10-08T16:04:36.854094+0000","last_became_active":"2023-10-08T16:04:36.845053+0000","last_became_peered":"2023-10-08T16:04:36.845053+0000","last_unstale":"2023-10-08T16:04:36.854094+0000","last_undegraded":"2023-10-08T16:04:36.854094+0000","last_fullsized":"2023-10-08T16:04:36.854094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848364+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848364+0000","last_peered":"2023-10-08T16:04:36.848364+0000","last_clean":"2023-10-08T16:04:36.848364+0000","last_became_active":"2023-10-08T16:04:36.842841+0000","last_became_peered":"2023-10-08T16:04:36.842841+0000","last_unstale":"2023-10-08T16:04:36.848364+0000","last_undegraded":"2023-10-08T16:04:36.848364+0000","last_fullsized":"2023-10-08T16:04:36.848364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855557+0000","last_change":"2023-10-08T16:04:36.850496+0000","last_active":"2023-10-08T16:04:36.855557+0000","last_peered":"2023-10-08T16:04:36.855557+0000","last_clean":"2023-10-08T16:04:36.855557+0000","last_became_active":"2023-10-08T16:04:36.850237+0000","last_became_peered":"2023-10-08T16:04:36.850237+0000","last_unstale":"2023-10-08T16:04:36.855557+0000","last_undegraded":"2023-10-08T16:04:36.855557+0000","last_fullsized":"2023-10-08T16:04:36.855557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854848+0000","last_change":"2023-10-08T16:04:36.847152+0000","last_active":"2023-10-08T16:04:36.854848+0000","last_peered":"2023-10-08T16:04:36.854848+0000","last_clean":"2023-10-08T16:04:36.854848+0000","last_became_active":"2023-10-08T16:04:36.846573+0000","last_became_peered":"2023-10-08T16:04:36.846573+0000","last_unstale":"2023-10-08T16:04:36.854848+0000","last_undegraded":"2023-10-08T16:04:36.854848+0000","last_fullsized":"2023-10-08T16:04:36.854848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853411+0000","last_change":"2023-10-08T16:04:36.847416+0000","last_active":"2023-10-08T16:04:36.853411+0000","last_peered":"2023-10-08T16:04:36.853411+0000","last_clean":"2023-10-08T16:04:36.853411+0000","last_became_active":"2023-10-08T16:04:36.847092+0000","last_became_peered":"2023-10-08T16:04:36.847092+0000","last_unstale":"2023-10-08T16:04:36.853411+0000","last_undegraded":"2023-10-08T16:04:36.853411+0000","last_fullsized":"2023-10-08T16:04:36.853411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860614+0000","last_change":"2023-10-08T16:04:36.854204+0000","last_active":"2023-10-08T16:04:36.860614+0000","last_peered":"2023-10-08T16:04:36.860614+0000","last_clean":"2023-10-08T16:04:36.860614+0000","last_became_active":"2023-10-08T16:04:36.853963+0000","last_became_peered":"2023-10-08T16:04:36.853963+0000","last_unstale":"2023-10-08T16:04:36.860614+0000","last_undegraded":"2023-10-08T16:04:36.860614+0000","last_fullsized":"2023-10-08T16:04:36.860614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859253+0000","last_change":"2023-10-08T16:04:36.855269+0000","last_active":"2023-10-08T16:04:36.859253+0000","last_peered":"2023-10-08T16:04:36.859253+0000","last_clean":"2023-10-08T16:04:36.859253+0000","last_became_active":"2023-10-08T16:04:36.854443+0000","last_became_peered":"2023-10-08T16:04:36.854443+0000","last_unstale":"2023-10-08T16:04:36.859253+0000","last_undegraded":"2023-10-08T16:04:36.859253+0000","last_fullsized":"2023-10-08T16:04:36.859253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844837+0000","last_change":"2023-10-08T16:04:36.842900+0000","last_active":"2023-10-08T16:04:36.844837+0000","last_peered":"2023-10-08T16:04:36.844837+0000","last_clean":"2023-10-08T16:04:36.844837+0000","last_became_active":"2023-10-08T16:04:36.842642+0000","last_became_peered":"2023-10-08T16:04:36.842642+0000","last_unstale":"2023-10-08T16:04:36.844837+0000","last_undegraded":"2023-10-08T16:04:36.844837+0000","last_fullsized":"2023-10-08T16:04:36.844837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844389+0000","last_change":"2023-10-08T16:04:36.842577+0000","last_active":"2023-10-08T16:04:36.844389+0000","last_peered":"2023-10-08T16:04:36.844389+0000","last_clean":"2023-10-08T16:04:36.844389+0000","last_became_active":"2023-10-08T16:04:36.841699+0000","last_became_peered":"2023-10-08T16:04:36.841699+0000","last_unstale":"2023-10-08T16:04:36.844389+0000","last_undegraded":"2023-10-08T16:04:36.844389+0000","last_fullsized":"2023-10-08T16:04:36.844389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850510+0000","last_change":"2023-10-08T16:04:36.845580+0000","last_active":"2023-10-08T16:04:36.850510+0000","last_peered":"2023-10-08T16:04:36.850510+0000","last_clean":"2023-10-08T16:04:36.850510+0000","last_became_active":"2023-10-08T16:04:36.845348+0000","last_became_peered":"2023-10-08T16:04:36.845348+0000","last_unstale":"2023-10-08T16:04:36.850510+0000","last_undegraded":"2023-10-08T16:04:36.850510+0000","last_fullsized":"2023-10-08T16:04:36.850510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854086+0000","last_change":"2023-10-08T16:04:36.845629+0000","last_active":"2023-10-08T16:04:36.854086+0000","last_peered":"2023-10-08T16:04:36.854086+0000","last_clean":"2023-10-08T16:04:36.854086+0000","last_became_active":"2023-10-08T16:04:36.845283+0000","last_became_peered":"2023-10-08T16:04:36.845283+0000","last_unstale":"2023-10-08T16:04:36.854086+0000","last_undegraded":"2023-10-08T16:04:36.854086+0000","last_fullsized":"2023-10-08T16:04:36.854086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856506+0000","last_change":"2023-10-08T16:04:36.852540+0000","last_active":"2023-10-08T16:04:36.856506+0000","last_peered":"2023-10-08T16:04:36.856506+0000","last_clean":"2023-10-08T16:04:36.856506+0000","last_became_active":"2023-10-08T16:04:36.852294+0000","last_became_peered":"2023-10-08T16:04:36.852294+0000","last_unstale":"2023-10-08T16:04:36.856506+0000","last_undegraded":"2023-10-08T16:04:36.856506+0000","last_fullsized":"2023-10-08T16:04:36.856506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850863+0000","last_change":"2023-10-08T16:04:36.845236+0000","last_active":"2023-10-08T16:04:36.850863+0000","last_peered":"2023-10-08T16:04:36.850863+0000","last_clean":"2023-10-08T16:04:36.850863+0000","last_became_active":"2023-10-08T16:04:36.844966+0000","last_became_peered":"2023-10-08T16:04:36.844966+0000","last_unstale":"2023-10-08T16:04:36.850863+0000","last_undegraded":"2023-10-08T16:04:36.850863+0000","last_fullsized":"2023-10-08T16:04:36.850863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849074+0000","last_change":"2023-10-08T16:04:36.844871+0000","last_active":"2023-10-08T16:04:36.849074+0000","last_peered":"2023-10-08T16:04:36.849074+0000","last_clean":"2023-10-08T16:04:36.849074+0000","last_became_active":"2023-10-08T16:04:36.844378+0000","last_became_peered":"2023-10-08T16:04:36.844378+0000","last_unstale":"2023-10-08T16:04:36.849074+0000","last_undegraded":"2023-10-08T16:04:36.849074+0000","last_fullsized":"2023-10-08T16:04:36.849074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861810+0000","last_change":"2023-10-08T16:04:35.842581+0000","last_active":"2023-10-08T16:04:36.861810+0000","last_peered":"2023-10-08T16:04:36.861810+0000","last_clean":"2023-10-08T16:04:36.861810+0000","last_became_active":"2023-10-08T16:04:35.842258+0000","last_became_peered":"2023-10-08T16:04:35.842258+0000","last_unstale":"2023-10-08T16:04:36.861810+0000","last_undegraded":"2023-10-08T16:04:36.861810+0000","last_fullsized":"2023-10-08T16:04:36.861810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:04.459691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844626+0000","last_change":"2023-10-08T16:04:36.842889+0000","last_active":"2023-10-08T16:04:36.844626+0000","last_peered":"2023-10-08T16:04:36.844626+0000","last_clean":"2023-10-08T16:04:36.844626+0000","last_became_active":"2023-10-08T16:04:36.842161+0000","last_became_peered":"2023-10-08T16:04:36.842161+0000","last_unstale":"2023-10-08T16:04:36.844626+0000","last_undegraded":"2023-10-08T16:04:36.844626+0000","last_fullsized":"2023-10-08T16:04:36.844626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856629+0000","last_change":"2023-10-08T16:04:35.835973+0000","last_active":"2023-10-08T16:04:36.856629+0000","last_peered":"2023-10-08T16:04:36.856629+0000","last_clean":"2023-10-08T16:04:36.856629+0000","last_became_active":"2023-10-08T16:04:35.835785+0000","last_became_peered":"2023-10-08T16:04:35.835785+0000","last_unstale":"2023-10-08T16:04:36.856629+0000","last_undegraded":"2023-10-08T16:04:36.856629+0000","last_fullsized":"2023-10-08T16:04:36.856629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849375+0000","last_change":"2023-10-08T16:04:36.845806+0000","last_active":"2023-10-08T16:04:36.849375+0000","last_peered":"2023-10-08T16:04:36.849375+0000","last_clean":"2023-10-08T16:04:36.849375+0000","last_became_active":"2023-10-08T16:04:36.845574+0000","last_became_peered":"2023-10-08T16:04:36.845574+0000","last_unstale":"2023-10-08T16:04:36.849375+0000","last_undegraded":"2023-10-08T16:04:36.849375+0000","last_fullsized":"2023-10-08T16:04:36.849375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860925+0000","last_change":"2023-10-08T16:04:35.844449+0000","last_active":"2023-10-08T16:04:36.860925+0000","last_peered":"2023-10-08T16:04:36.860925+0000","last_clean":"2023-10-08T16:04:36.860925+0000","last_became_active":"2023-10-08T16:04:35.844192+0000","last_became_peered":"2023-10-08T16:04:35.844192+0000","last_unstale":"2023-10-08T16:04:36.860925+0000","last_undegraded":"2023-10-08T16:04:36.860925+0000","last_fullsized":"2023-10-08T16:04:36.860925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856262+0000","last_change":"2023-10-08T16:04:36.853135+0000","last_active":"2023-10-08T16:04:36.856262+0000","last_peered":"2023-10-08T16:04:36.856262+0000","last_clean":"2023-10-08T16:04:36.856262+0000","last_became_active":"2023-10-08T16:04:36.852832+0000","last_became_peered":"2023-10-08T16:04:36.852832+0000","last_unstale":"2023-10-08T16:04:36.856262+0000","last_undegraded":"2023-10-08T16:04:36.856262+0000","last_fullsized":"2023-10-08T16:04:36.856262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861744+0000","last_change":"2023-10-08T16:04:35.845418+0000","last_active":"2023-10-08T16:04:36.861744+0000","last_peered":"2023-10-08T16:04:36.861744+0000","last_clean":"2023-10-08T16:04:36.861744+0000","last_became_active":"2023-10-08T16:04:35.845104+0000","last_became_peered":"2023-10-08T16:04:35.845104+0000","last_unstale":"2023-10-08T16:04:36.861744+0000","last_undegraded":"2023-10-08T16:04:36.861744+0000","last_fullsized":"2023-10-08T16:04:36.861744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856172+0000","last_change":"2023-10-08T16:04:36.851677+0000","last_active":"2023-10-08T16:04:36.856172+0000","last_peered":"2023-10-08T16:04:36.856172+0000","last_clean":"2023-10-08T16:04:36.856172+0000","last_became_active":"2023-10-08T16:04:36.847960+0000","last_became_peered":"2023-10-08T16:04:36.847960+0000","last_unstale":"2023-10-08T16:04:36.856172+0000","last_undegraded":"2023-10-08T16:04:36.856172+0000","last_fullsized":"2023-10-08T16:04:36.856172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861300+0000","last_change":"2023-10-08T16:04:35.833685+0000","last_active":"2023-10-08T16:04:36.861300+0000","last_peered":"2023-10-08T16:04:36.861300+0000","last_clean":"2023-10-08T16:04:36.861300+0000","last_became_active":"2023-10-08T16:04:35.833403+0000","last_became_peered":"2023-10-08T16:04:35.833403+0000","last_unstale":"2023-10-08T16:04:36.861300+0000","last_undegraded":"2023-10-08T16:04:36.861300+0000","last_fullsized":"2023-10-08T16:04:36.861300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851430+0000","last_change":"2023-10-08T16:04:36.845487+0000","last_active":"2023-10-08T16:04:36.851430+0000","last_peered":"2023-10-08T16:04:36.851430+0000","last_clean":"2023-10-08T16:04:36.851430+0000","last_became_active":"2023-10-08T16:04:36.845204+0000","last_became_peered":"2023-10-08T16:04:36.845204+0000","last_unstale":"2023-10-08T16:04:36.851430+0000","last_undegraded":"2023-10-08T16:04:36.851430+0000","last_fullsized":"2023-10-08T16:04:36.851430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:37.980506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861219+0000","last_change":"2023-10-08T16:04:35.833842+0000","last_active":"2023-10-08T16:04:36.861219+0000","last_peered":"2023-10-08T16:04:36.861219+0000","last_clean":"2023-10-08T16:04:36.861219+0000","last_became_active":"2023-10-08T16:04:35.833461+0000","last_became_peered":"2023-10-08T16:04:35.833461+0000","last_unstale":"2023-10-08T16:04:36.861219+0000","last_undegraded":"2023-10-08T16:04:36.861219+0000","last_fullsized":"2023-10-08T16:04:36.861219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855228+0000","last_change":"2023-10-08T16:04:36.849156+0000","last_active":"2023-10-08T16:04:36.855228+0000","last_peered":"2023-10-08T16:04:36.855228+0000","last_clean":"2023-10-08T16:04:36.855228+0000","last_became_active":"2023-10-08T16:04:36.848907+0000","last_became_peered":"2023-10-08T16:04:36.848907+0000","last_unstale":"2023-10-08T16:04:36.855228+0000","last_undegraded":"2023-10-08T16:04:36.855228+0000","last_fullsized":"2023-10-08T16:04:36.855228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:04.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834426+0000","last_change":"2023-10-08T16:04:35.838361+0000","last_active":"2023-10-08T16:04:36.834426+0000","last_peered":"2023-10-08T16:04:36.834426+0000","last_clean":"2023-10-08T16:04:36.834426+0000","last_became_active":"2023-10-08T16:04:35.838175+0000","last_became_peered":"2023-10-08T16:04:35.838175+0000","last_unstale":"2023-10-08T16:04:36.834426+0000","last_undegraded":"2023-10-08T16:04:36.834426+0000","last_fullsized":"2023-10-08T16:04:36.834426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851367+0000","last_change":"2023-10-08T16:04:36.843489+0000","last_active":"2023-10-08T16:04:36.851367+0000","last_peered":"2023-10-08T16:04:36.851367+0000","last_clean":"2023-10-08T16:04:36.851367+0000","last_became_active":"2023-10-08T16:04:36.843221+0000","last_became_peered":"2023-10-08T16:04:36.843221+0000","last_unstale":"2023-10-08T16:04:36.851367+0000","last_undegraded":"2023-10-08T16:04:36.851367+0000","last_fullsized":"2023-10-08T16:04:36.851367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840321+0000","last_change":"2023-10-08T16:04:35.839307+0000","last_active":"2023-10-08T16:04:36.840321+0000","last_peered":"2023-10-08T16:04:36.840321+0000","last_clean":"2023-10-08T16:04:36.840321+0000","last_became_active":"2023-10-08T16:04:35.838997+0000","last_became_peered":"2023-10-08T16:04:35.838997+0000","last_unstale":"2023-10-08T16:04:36.840321+0000","last_undegraded":"2023-10-08T16:04:36.840321+0000","last_fullsized":"2023-10-08T16:04:36.840321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840059+0000","last_change":"2023-10-08T16:04:36.838211+0000","last_active":"2023-10-08T16:04:36.840059+0000","last_peered":"2023-10-08T16:04:36.840059+0000","last_clean":"2023-10-08T16:04:36.840059+0000","last_became_active":"2023-10-08T16:04:36.837948+0000","last_became_peered":"2023-10-08T16:04:36.837948+0000","last_unstale":"2023-10-08T16:04:36.840059+0000","last_undegraded":"2023-10-08T16:04:36.840059+0000","last_fullsized":"2023-10-08T16:04:36.840059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838498+0000","last_change":"2023-10-08T16:04:35.841004+0000","last_active":"2023-10-08T16:04:36.838498+0000","last_peered":"2023-10-08T16:04:36.838498+0000","last_clean":"2023-10-08T16:04:36.838498+0000","last_became_active":"2023-10-08T16:04:35.840735+0000","last_became_peered":"2023-10-08T16:04:35.840735+0000","last_unstale":"2023-10-08T16:04:36.838498+0000","last_undegraded":"2023-10-08T16:04:36.838498+0000","last_fullsized":"2023-10-08T16:04:36.838498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855922+0000","last_change":"2023-10-08T16:04:36.850043+0000","last_active":"2023-10-08T16:04:36.855922+0000","last_peered":"2023-10-08T16:04:36.855922+0000","last_clean":"2023-10-08T16:04:36.855922+0000","last_became_active":"2023-10-08T16:04:36.849743+0000","last_became_peered":"2023-10-08T16:04:36.849743+0000","last_unstale":"2023-10-08T16:04:36.855922+0000","last_undegraded":"2023-10-08T16:04:36.855922+0000","last_fullsized":"2023-10-08T16:04:36.855922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838528+0000","last_change":"2023-10-08T16:04:35.844419+0000","last_active":"2023-10-08T16:04:36.838528+0000","last_peered":"2023-10-08T16:04:36.838528+0000","last_clean":"2023-10-08T16:04:36.838528+0000","last_became_active":"2023-10-08T16:04:35.844125+0000","last_became_peered":"2023-10-08T16:04:35.844125+0000","last_unstale":"2023-10-08T16:04:36.838528+0000","last_undegraded":"2023-10-08T16:04:36.838528+0000","last_fullsized":"2023-10-08T16:04:36.838528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849356+0000","last_change":"2023-10-08T16:04:36.845771+0000","last_active":"2023-10-08T16:04:36.849356+0000","last_peered":"2023-10-08T16:04:36.849356+0000","last_clean":"2023-10-08T16:04:36.849356+0000","last_became_active":"2023-10-08T16:04:36.845274+0000","last_became_peered":"2023-10-08T16:04:36.845274+0000","last_unstale":"2023-10-08T16:04:36.849356+0000","last_undegraded":"2023-10-08T16:04:36.849356+0000","last_fullsized":"2023-10-08T16:04:36.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856530+0000","last_change":"2023-10-08T16:04:35.840602+0000","last_active":"2023-10-08T16:04:36.856530+0000","last_peered":"2023-10-08T16:04:36.856530+0000","last_clean":"2023-10-08T16:04:36.856530+0000","last_became_active":"2023-10-08T16:04:35.840299+0000","last_became_peered":"2023-10-08T16:04:35.840299+0000","last_unstale":"2023-10-08T16:04:36.856530+0000","last_undegraded":"2023-10-08T16:04:36.856530+0000","last_fullsized":"2023-10-08T16:04:36.856530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853166+0000","last_change":"2023-10-08T16:04:36.845842+0000","last_active":"2023-10-08T16:04:36.853166+0000","last_peered":"2023-10-08T16:04:36.853166+0000","last_clean":"2023-10-08T16:04:36.853166+0000","last_became_active":"2023-10-08T16:04:36.844954+0000","last_became_peered":"2023-10-08T16:04:36.844954+0000","last_unstale":"2023-10-08T16:04:36.853166+0000","last_undegraded":"2023-10-08T16:04:36.853166+0000","last_fullsized":"2023-10-08T16:04:36.853166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:34.979889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.863226+0000","last_change":"2023-10-08T16:04:35.838972+0000","last_active":"2023-10-08T16:04:36.863226+0000","last_peered":"2023-10-08T16:04:36.863226+0000","last_clean":"2023-10-08T16:04:36.863226+0000","last_became_active":"2023-10-08T16:04:35.838694+0000","last_became_peered":"2023-10-08T16:04:35.838694+0000","last_unstale":"2023-10-08T16:04:36.863226+0000","last_undegraded":"2023-10-08T16:04:36.863226+0000","last_fullsized":"2023-10-08T16:04:36.863226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849056+0000","last_change":"2023-10-08T16:04:36.844679+0000","last_active":"2023-10-08T16:04:36.849056+0000","last_peered":"2023-10-08T16:04:36.849056+0000","last_clean":"2023-10-08T16:04:36.849056+0000","last_became_active":"2023-10-08T16:04:36.844115+0000","last_became_peered":"2023-10-08T16:04:36.844115+0000","last_unstale":"2023-10-08T16:04:36.849056+0000","last_undegraded":"2023-10-08T16:04:36.849056+0000","last_fullsized":"2023-10-08T16:04:36.849056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859530+0000","last_change":"2023-10-08T16:04:35.837996+0000","last_active":"2023-10-08T16:04:36.859530+0000","last_peered":"2023-10-08T16:04:36.859530+0000","last_clean":"2023-10-08T16:04:36.859530+0000","last_became_active":"2023-10-08T16:04:35.837567+0000","last_became_peered":"2023-10-08T16:04:35.837567+0000","last_unstale":"2023-10-08T16:04:36.859530+0000","last_undegraded":"2023-10-08T16:04:36.859530+0000","last_fullsized":"2023-10-08T16:04:36.859530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855385+0000","last_change":"2023-10-08T16:04:36.849839+0000","last_active":"2023-10-08T16:04:36.855385+0000","last_peered":"2023-10-08T16:04:36.855385+0000","last_clean":"2023-10-08T16:04:36.855385+0000","last_became_active":"2023-10-08T16:04:36.849577+0000","last_became_peered":"2023-10-08T16:04:36.849577+0000","last_unstale":"2023-10-08T16:04:36.855385+0000","last_undegraded":"2023-10-08T16:04:36.855385+0000","last_fullsized":"2023-10-08T16:04:36.855385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:55.787779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838421+0000","last_change":"2023-10-08T16:04:35.841168+0000","last_active":"2023-10-08T16:04:36.838421+0000","last_peered":"2023-10-08T16:04:36.838421+0000","last_clean":"2023-10-08T16:04:36.838421+0000","last_became_active":"2023-10-08T16:04:35.840667+0000","last_became_peered":"2023-10-08T16:04:35.840667+0000","last_unstale":"2023-10-08T16:04:36.838421+0000","last_undegraded":"2023-10-08T16:04:36.838421+0000","last_fullsized":"2023-10-08T16:04:36.838421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859673+0000","last_change":"2023-10-08T16:04:36.849407+0000","last_active":"2023-10-08T16:04:36.859673+0000","last_peered":"2023-10-08T16:04:36.859673+0000","last_clean":"2023-10-08T16:04:36.859673+0000","last_became_active":"2023-10-08T16:04:36.849157+0000","last_became_peered":"2023-10-08T16:04:36.849157+0000","last_unstale":"2023-10-08T16:04:36.859673+0000","last_undegraded":"2023-10-08T16:04:36.859673+0000","last_fullsized":"2023-10-08T16:04:36.859673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836282+0000","last_change":"2023-10-08T16:04:35.835032+0000","last_active":"2023-10-08T16:04:36.836282+0000","last_peered":"2023-10-08T16:04:36.836282+0000","last_clean":"2023-10-08T16:04:36.836282+0000","last_became_active":"2023-10-08T16:04:35.834550+0000","last_became_peered":"2023-10-08T16:04:35.834550+0000","last_unstale":"2023-10-08T16:04:36.836282+0000","last_undegraded":"2023-10-08T16:04:36.836282+0000","last_fullsized":"2023-10-08T16:04:36.836282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855292+0000","last_change":"2023-10-08T16:04:36.848762+0000","last_active":"2023-10-08T16:04:36.855292+0000","last_peered":"2023-10-08T16:04:36.855292+0000","last_clean":"2023-10-08T16:04:36.855292+0000","last_became_active":"2023-10-08T16:04:36.848407+0000","last_became_peered":"2023-10-08T16:04:36.848407+0000","last_unstale":"2023-10-08T16:04:36.855292+0000","last_undegraded":"2023-10-08T16:04:36.855292+0000","last_fullsized":"2023-10-08T16:04:36.855292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856015+0000","last_change":"2023-10-08T16:04:35.834021+0000","last_active":"2023-10-08T16:04:36.856015+0000","last_peered":"2023-10-08T16:04:36.856015+0000","last_clean":"2023-10-08T16:04:36.856015+0000","last_became_active":"2023-10-08T16:04:35.833739+0000","last_became_peered":"2023-10-08T16:04:35.833739+0000","last_unstale":"2023-10-08T16:04:36.856015+0000","last_undegraded":"2023-10-08T16:04:36.856015+0000","last_fullsized":"2023-10-08T16:04:36.856015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859686+0000","last_change":"2023-10-08T16:04:36.850254+0000","last_active":"2023-10-08T16:04:36.859686+0000","last_peered":"2023-10-08T16:04:36.859686+0000","last_clean":"2023-10-08T16:04:36.859686+0000","last_became_active":"2023-10-08T16:04:36.849843+0000","last_became_peered":"2023-10-08T16:04:36.849843+0000","last_unstale":"2023-10-08T16:04:36.859686+0000","last_undegraded":"2023-10-08T16:04:36.859686+0000","last_fullsized":"2023-10-08T16:04:36.859686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859275+0000","last_change":"2023-10-08T16:04:35.837065+0000","last_active":"2023-10-08T16:04:36.859275+0000","last_peered":"2023-10-08T16:04:36.859275+0000","last_clean":"2023-10-08T16:04:36.859275+0000","last_became_active":"2023-10-08T16:04:35.836764+0000","last_became_peered":"2023-10-08T16:04:35.836764+0000","last_unstale":"2023-10-08T16:04:36.859275+0000","last_undegraded":"2023-10-08T16:04:36.859275+0000","last_fullsized":"2023-10-08T16:04:36.859275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:43.449593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848948+0000","last_change":"2023-10-08T16:04:36.845465+0000","last_active":"2023-10-08T16:04:36.848948+0000","last_peered":"2023-10-08T16:04:36.848948+0000","last_clean":"2023-10-08T16:04:36.848948+0000","last_became_active":"2023-10-08T16:04:36.845207+0000","last_became_peered":"2023-10-08T16:04:36.845207+0000","last_unstale":"2023-10-08T16:04:36.848948+0000","last_undegraded":"2023-10-08T16:04:36.848948+0000","last_fullsized":"2023-10-08T16:04:36.848948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831893+0000","last_change":"2023-10-08T16:04:35.836100+0000","last_active":"2023-10-08T16:04:36.831893+0000","last_peered":"2023-10-08T16:04:36.831893+0000","last_clean":"2023-10-08T16:04:36.831893+0000","last_became_active":"2023-10-08T16:04:35.835832+0000","last_became_peered":"2023-10-08T16:04:35.835832+0000","last_unstale":"2023-10-08T16:04:36.831893+0000","last_undegraded":"2023-10-08T16:04:36.831893+0000","last_fullsized":"2023-10-08T16:04:36.831893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860666+0000","last_change":"2023-10-08T16:04:36.853664+0000","last_active":"2023-10-08T16:04:36.860666+0000","last_peered":"2023-10-08T16:04:36.860666+0000","last_clean":"2023-10-08T16:04:36.860666+0000","last_became_active":"2023-10-08T16:04:36.853412+0000","last_became_peered":"2023-10-08T16:04:36.853412+0000","last_unstale":"2023-10-08T16:04:36.860666+0000","last_undegraded":"2023-10-08T16:04:36.860666+0000","last_fullsized":"2023-10-08T16:04:36.860666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838343+0000","last_change":"2023-10-08T16:04:35.838075+0000","last_active":"2023-10-08T16:04:36.838343+0000","last_peered":"2023-10-08T16:04:36.838343+0000","last_clean":"2023-10-08T16:04:36.838343+0000","last_became_active":"2023-10-08T16:04:35.837821+0000","last_became_peered":"2023-10-08T16:04:35.837821+0000","last_unstale":"2023-10-08T16:04:36.838343+0000","last_undegraded":"2023-10-08T16:04:36.838343+0000","last_fullsized":"2023-10-08T16:04:36.838343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850970+0000","last_change":"2023-10-08T16:04:36.845413+0000","last_active":"2023-10-08T16:04:36.850970+0000","last_peered":"2023-10-08T16:04:36.850970+0000","last_clean":"2023-10-08T16:04:36.850970+0000","last_became_active":"2023-10-08T16:04:36.845152+0000","last_became_peered":"2023-10-08T16:04:36.845152+0000","last_unstale":"2023-10-08T16:04:36.850970+0000","last_undegraded":"2023-10-08T16:04:36.850970+0000","last_fullsized":"2023-10-08T16:04:36.850970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839476+0000","last_change":"2023-10-08T16:04:36.837273+0000","last_active":"2023-10-08T16:04:36.839476+0000","last_peered":"2023-10-08T16:04:36.839476+0000","last_clean":"2023-10-08T16:04:36.839476+0000","last_became_active":"2023-10-08T16:04:36.837025+0000","last_became_peered":"2023-10-08T16:04:36.837025+0000","last_unstale":"2023-10-08T16:04:36.839476+0000","last_undegraded":"2023-10-08T16:04:36.839476+0000","last_fullsized":"2023-10-08T16:04:36.839476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833690+0000","last_change":"2023-10-08T16:04:35.831326+0000","last_active":"2023-10-08T16:04:36.833690+0000","last_peered":"2023-10-08T16:04:36.833690+0000","last_clean":"2023-10-08T16:04:36.833690+0000","last_became_active":"2023-10-08T16:04:35.831088+0000","last_became_peered":"2023-10-08T16:04:35.831088+0000","last_unstale":"2023-10-08T16:04:36.833690+0000","last_undegraded":"2023-10-08T16:04:36.833690+0000","last_fullsized":"2023-10-08T16:04:36.833690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855327+0000","last_change":"2023-10-08T16:04:36.847842+0000","last_active":"2023-10-08T16:04:36.855327+0000","last_peered":"2023-10-08T16:04:36.855327+0000","last_clean":"2023-10-08T16:04:36.855327+0000","last_became_active":"2023-10-08T16:04:36.847222+0000","last_became_peered":"2023-10-08T16:04:36.847222+0000","last_unstale":"2023-10-08T16:04:36.855327+0000","last_undegraded":"2023-10-08T16:04:36.855327+0000","last_fullsized":"2023-10-08T16:04:36.855327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835188+0000","last_change":"2023-10-08T16:04:35.837295+0000","last_active":"2023-10-08T16:04:36.835188+0000","last_peered":"2023-10-08T16:04:36.835188+0000","last_clean":"2023-10-08T16:04:36.835188+0000","last_became_active":"2023-10-08T16:04:35.836609+0000","last_became_peered":"2023-10-08T16:04:35.836609+0000","last_unstale":"2023-10-08T16:04:36.835188+0000","last_undegraded":"2023-10-08T16:04:36.835188+0000","last_fullsized":"2023-10-08T16:04:36.835188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835250+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:04:36.835250+0000","last_peered":"2023-10-08T16:04:36.835250+0000","last_clean":"2023-10-08T16:04:36.835250+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:04:36.835250+0000","last_undegraded":"2023-10-08T16:04:36.835250+0000","last_fullsized":"2023-10-08T16:04:36.835250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855399+0000","last_change":"2023-10-08T16:04:36.849109+0000","last_active":"2023-10-08T16:04:36.855399+0000","last_peered":"2023-10-08T16:04:36.855399+0000","last_clean":"2023-10-08T16:04:36.855399+0000","last_became_active":"2023-10-08T16:04:36.848847+0000","last_became_peered":"2023-10-08T16:04:36.848847+0000","last_unstale":"2023-10-08T16:04:36.855399+0000","last_undegraded":"2023-10-08T16:04:36.855399+0000","last_fullsized":"2023-10-08T16:04:36.855399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:34:26.769453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835538+0000","last_change":"2023-10-08T16:04:35.840218+0000","last_active":"2023-10-08T16:04:36.835538+0000","last_peered":"2023-10-08T16:04:36.835538+0000","last_clean":"2023-10-08T16:04:36.835538+0000","last_became_active":"2023-10-08T16:04:35.839272+0000","last_became_peered":"2023-10-08T16:04:35.839272+0000","last_unstale":"2023-10-08T16:04:36.835538+0000","last_undegraded":"2023-10-08T16:04:36.835538+0000","last_fullsized":"2023-10-08T16:04:36.835538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836042+0000","last_change":"2023-10-08T16:04:35.842423+0000","last_active":"2023-10-08T16:04:36.836042+0000","last_peered":"2023-10-08T16:04:36.836042+0000","last_clean":"2023-10-08T16:04:36.836042+0000","last_became_active":"2023-10-08T16:04:35.842183+0000","last_became_peered":"2023-10-08T16:04:35.842183+0000","last_unstale":"2023-10-08T16:04:36.836042+0000","last_undegraded":"2023-10-08T16:04:36.836042+0000","last_fullsized":"2023-10-08T16:04:36.836042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839745+0000","last_change":"2023-10-08T16:04:36.837348+0000","last_active":"2023-10-08T16:04:36.839745+0000","last_peered":"2023-10-08T16:04:36.839745+0000","last_clean":"2023-10-08T16:04:36.839745+0000","last_became_active":"2023-10-08T16:04:36.837116+0000","last_became_peered":"2023-10-08T16:04:36.837116+0000","last_unstale":"2023-10-08T16:04:36.839745+0000","last_undegraded":"2023-10-08T16:04:36.839745+0000","last_fullsized":"2023-10-08T16:04:36.839745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.832360+0000","last_change":"2023-10-08T16:04:35.834800+0000","last_active":"2023-10-08T16:04:36.832360+0000","last_peered":"2023-10-08T16:04:36.832360+0000","last_clean":"2023-10-08T16:04:36.832360+0000","last_became_active":"2023-10-08T16:04:35.834516+0000","last_became_peered":"2023-10-08T16:04:35.834516+0000","last_unstale":"2023-10-08T16:04:36.832360+0000","last_undegraded":"2023-10-08T16:04:36.832360+0000","last_fullsized":"2023-10-08T16:04:36.832360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855314+0000","last_change":"2023-10-08T16:04:36.850191+0000","last_active":"2023-10-08T16:04:36.855314+0000","last_peered":"2023-10-08T16:04:36.855314+0000","last_clean":"2023-10-08T16:04:36.855314+0000","last_became_active":"2023-10-08T16:04:36.849886+0000","last_became_peered":"2023-10-08T16:04:36.849886+0000","last_unstale":"2023-10-08T16:04:36.855314+0000","last_undegraded":"2023-10-08T16:04:36.855314+0000","last_fullsized":"2023-10-08T16:04:36.855314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861589+0000","last_change":"2023-10-08T16:04:35.842733+0000","last_active":"2023-10-08T16:04:36.861589+0000","last_peered":"2023-10-08T16:04:36.861589+0000","last_clean":"2023-10-08T16:04:36.861589+0000","last_became_active":"2023-10-08T16:04:35.842431+0000","last_became_peered":"2023-10-08T16:04:35.842431+0000","last_unstale":"2023-10-08T16:04:36.861589+0000","last_undegraded":"2023-10-08T16:04:36.861589+0000","last_fullsized":"2023-10-08T16:04:36.861589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843518+0000","last_change":"2023-10-08T16:04:36.841209+0000","last_active":"2023-10-08T16:04:36.843518+0000","last_peered":"2023-10-08T16:04:36.843518+0000","last_clean":"2023-10-08T16:04:36.843518+0000","last_became_active":"2023-10-08T16:04:36.840900+0000","last_became_peered":"2023-10-08T16:04:36.840900+0000","last_unstale":"2023-10-08T16:04:36.843518+0000","last_undegraded":"2023-10-08T16:04:36.843518+0000","last_fullsized":"2023-10-08T16:04:36.843518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834548+0000","last_change":"2023-10-08T16:04:35.840034+0000","last_active":"2023-10-08T16:04:36.834548+0000","last_peered":"2023-10-08T16:04:36.834548+0000","last_clean":"2023-10-08T16:04:36.834548+0000","last_became_active":"2023-10-08T16:04:35.839696+0000","last_became_peered":"2023-10-08T16:04:35.839696+0000","last_unstale":"2023-10-08T16:04:36.834548+0000","last_undegraded":"2023-10-08T16:04:36.834548+0000","last_fullsized":"2023-10-08T16:04:36.834548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:51.871435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853454+0000","last_change":"2023-10-08T16:04:36.847014+0000","last_active":"2023-10-08T16:04:36.853454+0000","last_peered":"2023-10-08T16:04:36.853454+0000","last_clean":"2023-10-08T16:04:36.853454+0000","last_became_active":"2023-10-08T16:04:36.846715+0000","last_became_peered":"2023-10-08T16:04:36.846715+0000","last_unstale":"2023-10-08T16:04:36.853454+0000","last_undegraded":"2023-10-08T16:04:36.853454+0000","last_fullsized":"2023-10-08T16:04:36.853454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:13.602001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831854+0000","last_change":"2023-10-08T16:04:35.833211+0000","last_active":"2023-10-08T16:04:36.831854+0000","last_peered":"2023-10-08T16:04:36.831854+0000","last_clean":"2023-10-08T16:04:36.831854+0000","last_became_active":"2023-10-08T16:04:35.832926+0000","last_became_peered":"2023-10-08T16:04:35.832926+0000","last_unstale":"2023-10-08T16:04:36.831854+0000","last_undegraded":"2023-10-08T16:04:36.831854+0000","last_fullsized":"2023-10-08T16:04:36.831854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840582+0000","last_change":"2023-10-08T16:04:36.839516+0000","last_active":"2023-10-08T16:04:36.840582+0000","last_peered":"2023-10-08T16:04:36.840582+0000","last_clean":"2023-10-08T16:04:36.840582+0000","last_became_active":"2023-10-08T16:04:36.839275+0000","last_became_peered":"2023-10-08T16:04:36.839275+0000","last_unstale":"2023-10-08T16:04:36.840582+0000","last_undegraded":"2023-10-08T16:04:36.840582+0000","last_fullsized":"2023-10-08T16:04:36.840582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831607+0000","last_change":"2023-10-08T16:04:35.826206+0000","last_active":"2023-10-08T16:04:36.831607+0000","last_peered":"2023-10-08T16:04:36.831607+0000","last_clean":"2023-10-08T16:04:36.831607+0000","last_became_active":"2023-10-08T16:04:35.825925+0000","last_became_peered":"2023-10-08T16:04:35.825925+0000","last_unstale":"2023-10-08T16:04:36.831607+0000","last_undegraded":"2023-10-08T16:04:36.831607+0000","last_fullsized":"2023-10-08T16:04:36.831607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855072+0000","last_change":"2023-10-08T16:04:36.845340+0000","last_active":"2023-10-08T16:04:36.855072+0000","last_peered":"2023-10-08T16:04:36.855072+0000","last_clean":"2023-10-08T16:04:36.855072+0000","last_became_active":"2023-10-08T16:04:36.845040+0000","last_became_peered":"2023-10-08T16:04:36.845040+0000","last_unstale":"2023-10-08T16:04:36.855072+0000","last_undegraded":"2023-10-08T16:04:36.855072+0000","last_fullsized":"2023-10-08T16:04:36.855072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860563+0000","last_change":"2023-10-08T16:04:35.841423+0000","last_active":"2023-10-08T16:04:36.860563+0000","last_peered":"2023-10-08T16:04:36.860563+0000","last_clean":"2023-10-08T16:04:36.860563+0000","last_became_active":"2023-10-08T16:04:35.841174+0000","last_became_peered":"2023-10-08T16:04:35.841174+0000","last_unstale":"2023-10-08T16:04:36.860563+0000","last_undegraded":"2023-10-08T16:04:36.860563+0000","last_fullsized":"2023-10-08T16:04:36.860563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860494+0000","last_change":"2023-10-08T16:04:36.851799+0000","last_active":"2023-10-08T16:04:36.860494+0000","last_peered":"2023-10-08T16:04:36.860494+0000","last_clean":"2023-10-08T16:04:36.860494+0000","last_became_active":"2023-10-08T16:04:36.851458+0000","last_became_peered":"2023-10-08T16:04:36.851458+0000","last_unstale":"2023-10-08T16:04:36.860494+0000","last_undegraded":"2023-10-08T16:04:36.860494+0000","last_fullsized":"2023-10-08T16:04:36.860494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833797+0000","last_change":"2023-10-08T16:04:35.840122+0000","last_active":"2023-10-08T16:04:36.833797+0000","last_peered":"2023-10-08T16:04:36.833797+0000","last_clean":"2023-10-08T16:04:36.833797+0000","last_became_active":"2023-10-08T16:04:35.839817+0000","last_became_peered":"2023-10-08T16:04:35.839817+0000","last_unstale":"2023-10-08T16:04:36.833797+0000","last_undegraded":"2023-10-08T16:04:36.833797+0000","last_fullsized":"2023-10-08T16:04:36.833797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855128+0000","last_change":"2023-10-08T16:04:36.847797+0000","last_active":"2023-10-08T16:04:36.855128+0000","last_peered":"2023-10-08T16:04:36.855128+0000","last_clean":"2023-10-08T16:04:36.855128+0000","last_became_active":"2023-10-08T16:04:36.847542+0000","last_became_peered":"2023-10-08T16:04:36.847542+0000","last_unstale":"2023-10-08T16:04:36.855128+0000","last_undegraded":"2023-10-08T16:04:36.855128+0000","last_fullsized":"2023-10-08T16:04:36.855128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859329+0000","last_change":"2023-10-08T16:04:35.826935+0000","last_active":"2023-10-08T16:04:36.859329+0000","last_peered":"2023-10-08T16:04:36.859329+0000","last_clean":"2023-10-08T16:04:36.859329+0000","last_became_active":"2023-10-08T16:04:35.826668+0000","last_became_peered":"2023-10-08T16:04:35.826668+0000","last_unstale":"2023-10-08T16:04:36.859329+0000","last_undegraded":"2023-10-08T16:04:36.859329+0000","last_fullsized":"2023-10-08T16:04:36.859329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861167+0000","last_change":"2023-10-08T16:04:36.857918+0000","last_active":"2023-10-08T16:04:36.861167+0000","last_peered":"2023-10-08T16:04:36.861167+0000","last_clean":"2023-10-08T16:04:36.861167+0000","last_became_active":"2023-10-08T16:04:36.857368+0000","last_became_peered":"2023-10-08T16:04:36.857368+0000","last_unstale":"2023-10-08T16:04:36.861167+0000","last_undegraded":"2023-10-08T16:04:36.861167+0000","last_fullsized":"2023-10-08T16:04:36.861167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855827+0000","last_change":"2023-10-08T16:04:35.836755+0000","last_active":"2023-10-08T16:04:36.855827+0000","last_peered":"2023-10-08T16:04:36.855827+0000","last_clean":"2023-10-08T16:04:36.855827+0000","last_became_active":"2023-10-08T16:04:35.836467+0000","last_became_peered":"2023-10-08T16:04:35.836467+0000","last_unstale":"2023-10-08T16:04:36.855827+0000","last_undegraded":"2023-10-08T16:04:36.855827+0000","last_fullsized":"2023-10-08T16:04:36.855827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860389+0000","last_change":"2023-10-08T16:04:36.851314+0000","last_active":"2023-10-08T16:04:36.860389+0000","last_peered":"2023-10-08T16:04:36.860389+0000","last_clean":"2023-10-08T16:04:36.860389+0000","last_became_active":"2023-10-08T16:04:36.850726+0000","last_became_peered":"2023-10-08T16:04:36.850726+0000","last_unstale":"2023-10-08T16:04:36.860389+0000","last_undegraded":"2023-10-08T16:04:36.860389+0000","last_fullsized":"2023-10-08T16:04:36.860389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853164+0000","last_change":"2023-10-08T16:04:36.843832+0000","last_active":"2023-10-08T16:04:36.853164+0000","last_peered":"2023-10-08T16:04:36.853164+0000","last_clean":"2023-10-08T16:04:36.853164+0000","last_became_active":"2023-10-08T16:04:36.843552+0000","last_became_peered":"2023-10-08T16:04:36.843552+0000","last_unstale":"2023-10-08T16:04:36.853164+0000","last_undegraded":"2023-10-08T16:04:36.853164+0000","last_fullsized":"2023-10-08T16:04:36.853164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848791+0000","last_change":"2023-10-08T16:04:36.846568+0000","last_active":"2023-10-08T16:04:36.848791+0000","last_peered":"2023-10-08T16:04:36.848791+0000","last_clean":"2023-10-08T16:04:36.848791+0000","last_became_active":"2023-10-08T16:04:36.846344+0000","last_became_peered":"2023-10-08T16:04:36.846344+0000","last_unstale":"2023-10-08T16:04:36.848791+0000","last_undegraded":"2023-10-08T16:04:36.848791+0000","last_fullsized":"2023-10-08T16:04:36.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861183+0000","last_change":"2023-10-08T16:04:36.857746+0000","last_active":"2023-10-08T16:04:36.861183+0000","last_peered":"2023-10-08T16:04:36.861183+0000","last_clean":"2023-10-08T16:04:36.861183+0000","last_became_active":"2023-10-08T16:04:36.856907+0000","last_became_peered":"2023-10-08T16:04:36.856907+0000","last_unstale":"2023-10-08T16:04:36.861183+0000","last_undegraded":"2023-10-08T16:04:36.861183+0000","last_fullsized":"2023-10-08T16:04:36.861183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.846189+0000","last_change":"2023-10-08T16:04:36.843885+0000","last_active":"2023-10-08T16:04:36.846189+0000","last_peered":"2023-10-08T16:04:36.846189+0000","last_clean":"2023-10-08T16:04:36.846189+0000","last_became_active":"2023-10-08T16:04:36.843640+0000","last_became_peered":"2023-10-08T16:04:36.843640+0000","last_unstale":"2023-10-08T16:04:36.846189+0000","last_undegraded":"2023-10-08T16:04:36.846189+0000","last_fullsized":"2023-10-08T16:04:36.846189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855356+0000","last_change":"2023-10-08T16:04:36.851431+0000","last_active":"2023-10-08T16:04:36.855356+0000","last_peered":"2023-10-08T16:04:36.855356+0000","last_clean":"2023-10-08T16:04:36.855356+0000","last_became_active":"2023-10-08T16:04:36.851215+0000","last_became_peered":"2023-10-08T16:04:36.851215+0000","last_unstale":"2023-10-08T16:04:36.855356+0000","last_undegraded":"2023-10-08T16:04:36.855356+0000","last_fullsized":"2023-10-08T16:04:36.855356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840134+0000","last_change":"2023-10-08T16:04:36.837981+0000","last_active":"2023-10-08T16:04:36.840134+0000","last_peered":"2023-10-08T16:04:36.840134+0000","last_clean":"2023-10-08T16:04:36.840134+0000","last_became_active":"2023-10-08T16:04:36.837478+0000","last_became_peered":"2023-10-08T16:04:36.837478+0000","last_unstale":"2023-10-08T16:04:36.840134+0000","last_undegraded":"2023-10-08T16:04:36.840134+0000","last_fullsized":"2023-10-08T16:04:36.840134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855950+0000","last_change":"2023-10-08T16:04:36.850056+0000","last_active":"2023-10-08T16:04:36.855950+0000","last_peered":"2023-10-08T16:04:36.855950+0000","last_clean":"2023-10-08T16:04:36.855950+0000","last_became_active":"2023-10-08T16:04:36.849732+0000","last_became_peered":"2023-10-08T16:04:36.849732+0000","last_unstale":"2023-10-08T16:04:36.855950+0000","last_undegraded":"2023-10-08T16:04:36.855950+0000","last_fullsized":"2023-10-08T16:04:36.855950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843571+0000","last_change":"2023-10-08T16:04:36.842352+0000","last_active":"2023-10-08T16:04:36.843571+0000","last_peered":"2023-10-08T16:04:36.843571+0000","last_clean":"2023-10-08T16:04:36.843571+0000","last_became_active":"2023-10-08T16:04:36.842078+0000","last_became_peered":"2023-10-08T16:04:36.842078+0000","last_unstale":"2023-10-08T16:04:36.843571+0000","last_undegraded":"2023-10-08T16:04:36.843571+0000","last_fullsized":"2023-10-08T16:04:36.843571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861173+0000","last_change":"2023-10-08T16:04:36.858170+0000","last_active":"2023-10-08T16:04:36.861173+0000","last_peered":"2023-10-08T16:04:36.861173+0000","last_clean":"2023-10-08T16:04:36.861173+0000","last_became_active":"2023-10-08T16:04:36.857822+0000","last_became_peered":"2023-10-08T16:04:36.857822+0000","last_unstale":"2023-10-08T16:04:36.861173+0000","last_undegraded":"2023-10-08T16:04:36.861173+0000","last_fullsized":"2023-10-08T16:04:36.861173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855132+0000","last_change":"2023-10-08T16:04:36.847659+0000","last_active":"2023-10-08T16:04:36.855132+0000","last_peered":"2023-10-08T16:04:36.855132+0000","last_clean":"2023-10-08T16:04:36.855132+0000","last_became_active":"2023-10-08T16:04:36.847337+0000","last_became_peered":"2023-10-08T16:04:36.847337+0000","last_unstale":"2023-10-08T16:04:36.855132+0000","last_undegraded":"2023-10-08T16:04:36.855132+0000","last_fullsized":"2023-10-08T16:04:36.855132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855842+0000","last_change":"2023-10-08T16:04:36.850206+0000","last_active":"2023-10-08T16:04:36.855842+0000","last_peered":"2023-10-08T16:04:36.855842+0000","last_clean":"2023-10-08T16:04:36.855842+0000","last_became_active":"2023-10-08T16:04:36.849892+0000","last_became_peered":"2023-10-08T16:04:36.849892+0000","last_unstale":"2023-10-08T16:04:36.855842+0000","last_undegraded":"2023-10-08T16:04:36.855842+0000","last_fullsized":"2023-10-08T16:04:36.855842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848345+0000","last_change":"2023-10-08T16:04:36.846291+0000","last_active":"2023-10-08T16:04:36.848345+0000","last_peered":"2023-10-08T16:04:36.848345+0000","last_clean":"2023-10-08T16:04:36.848345+0000","last_became_active":"2023-10-08T16:04:36.846030+0000","last_became_peered":"2023-10-08T16:04:36.846030+0000","last_unstale":"2023-10-08T16:04:36.848345+0000","last_undegraded":"2023-10-08T16:04:36.848345+0000","last_fullsized":"2023-10-08T16:04:36.848345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860629+0000","last_change":"2023-10-08T16:04:36.852345+0000","last_active":"2023-10-08T16:04:36.860629+0000","last_peered":"2023-10-08T16:04:36.860629+0000","last_clean":"2023-10-08T16:04:36.860629+0000","last_became_active":"2023-10-08T16:04:36.852087+0000","last_became_peered":"2023-10-08T16:04:36.852087+0000","last_unstale":"2023-10-08T16:04:36.860629+0000","last_undegraded":"2023-10-08T16:04:36.860629+0000","last_fullsized":"2023-10-08T16:04:36.860629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":666235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":666291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":75439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:04:40.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:04:41.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.058+0000 7f8a9d3c8700 1 Processor -- start 2023-10-08T16:04:41.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.058+0000 7f8a9d3c8700 1 -- start start 2023-10-08T16:04:41.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a9d3c8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a9d3c8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a9d3c8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9815a160 con 0x7f8a98159890 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a9d3c8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a980d13a0 con 0x7f8a9815db10 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a9d3c8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9815e140 con 0x7f8a98159c60 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a977fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a8e7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a977fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38252/0 (socket says 172.21.15.170:38252) 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a8e7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35212/0 (socket says 172.21.15.170:35212) 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.059+0000 7f8a977fe700 1 -- 172.21.15.170:0/470970392 learned_addr learned my addr 172.21.15.170:0/470970392 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a8e7fc700 1 -- 172.21.15.170:0/470970392 >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 msgr2=0x7f8a98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a8e7fc700 1 --2- 172.21.15.170:0/470970392 >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a8e7fc700 1 -- 172.21.15.170:0/470970392 >> v1:172.21.15.191:6789/0 conn(0x7f8a98159890 legacy=0x7f8a9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a8e7fc700 1 -- 172.21.15.170:0/470970392 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9815a3d0 con 0x7f8a98159c60 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a977fe700 1 --2- 172.21.15.170:0/470970392 >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a8e7fc700 1 --2- 172.21.15.170:0/470970392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8a800098e0 tx=0x7f8a800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c759d2f596edf870 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a967fc700 1 -- 172.21.15.170:0/470970392 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000b640 con 0x7f8a98159c60 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a967fc700 1 -- 172.21.15.170:0/470970392 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8000b7a0 con 0x7f8a98159c60 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a967fc700 1 -- 172.21.15.170:0/470970392 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000bb40 con 0x7f8a98159c60 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/470970392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 msgr2=0x7f8a9815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.060+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/470970392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8a800098e0 tx=0x7f8a800091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/470970392 shutdown_connections 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/470970392 >> v2:172.21.15.191:3300/0 conn(0x7f8a9815db10 0x7f8a98069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/470970392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a9815d5d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/470970392 >> 172.21.15.170:0/470970392 conn(0x7f8a98068e20 msgr2=0x7f8a980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/470970392 shutdown_connections 2023-10-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.061+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/470970392 wait complete. 2023-10-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.062+0000 7f8a9d3c8700 1 Processor -- start 2023-10-08T16:04:41.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.062+0000 7f8a9d3c8700 1 -- start start 2023-10-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.062+0000 7f8a9d3c8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 0x7f8a980b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a980ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980c7b10 con 0x7f8a9815db10 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980b5260 con 0x7f8a98159890 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980b5540 con 0x7f8a98159c60 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46212/0 (socket says 172.21.15.170:46212) 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 -- 172.21.15.170:0/3699668339 learned_addr learned my addr 172.21.15.170:0/3699668339 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a96ffd700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 0x7f8a980b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a8e7fc700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a980ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 -- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 msgr2=0x7f8a980ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a980ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 -- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 msgr2=0x7f8a980b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 0x7f8a980b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 -- 172.21.15.170:0/3699668339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90009ae0 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a96ffd700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 0x7f8a980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a977fe700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f30 tx=0x7f8a9000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90018070 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9000f040 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000ccd0 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a80009420 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.063+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a980b5b00 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.064+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9804c3c0 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.064+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a90003020 con 0x7f8a9815db10 2023-10-08T16:04:41.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.065+0000 7f8a8f7fe700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.065+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8a90084a60 con 0x7f8a9815db10 2023-10-08T16:04:41.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.065+0000 7f8a96ffd700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.065+0000 7f8a96ffd700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 0x7f8a780693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a88007fd0 tx=0x7f8a88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.069+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a90055760 con 0x7f8a9815db10 2023-10-08T16:04:41.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.249+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a9806a5b0 con 0x7f8a78066f30 2023-10-08T16:04:41.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.256+0000 7f8a8f7fe700 1 -- 172.21.15.170:0/3699668339 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264605 (secure 0 0 0) 0x7f8a9806a5b0 con 0x7f8a78066f30 2023-10-08T16:04:41.261 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.259+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 msgr2=0x7f8a780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.259+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 0x7f8a780693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a88007fd0 tx=0x7f8a88009040 comp rx=0 tx=0).stop 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.259+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 msgr2=0x7f8a981471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.259+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009f30 tx=0x7f8a9000b040 comp rx=0 tx=0).stop 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 shutdown_connections 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8a78066f30 0x7f8a780693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a9815db10 0x7f8a981471f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a98159890 0x7f8a980b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 --2- 172.21.15.170:0/3699668339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a98159c60 0x7f8a980ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 >> 172.21.15.170:0/3699668339 conn(0x7f8a98068e20 msgr2=0x7f8a9815d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 shutdown_connections 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.260+0000 7f8a9d3c8700 1 -- 172.21.15.170:0/3699668339 wait complete. 2023-10-08T16:04:41.264 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:04:41.273 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T16:04:40.225053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.406655"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854867+0000","last_change":"2023-10-08T16:04:36.847335+0000","last_active":"2023-10-08T16:04:36.854867+0000","last_peered":"2023-10-08T16:04:36.854867+0000","last_clean":"2023-10-08T16:04:36.854867+0000","last_became_active":"2023-10-08T16:04:36.846900+0000","last_became_peered":"2023-10-08T16:04:36.846900+0000","last_unstale":"2023-10-08T16:04:36.854867+0000","last_undegraded":"2023-10-08T16:04:36.854867+0000","last_fullsized":"2023-10-08T16:04:36.854867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.847766+0000","last_change":"2023-10-08T16:04:36.841973+0000","last_active":"2023-10-08T16:04:36.847766+0000","last_peered":"2023-10-08T16:04:36.847766+0000","last_clean":"2023-10-08T16:04:36.847766+0000","last_became_active":"2023-10-08T16:04:36.841752+0000","last_became_peered":"2023-10-08T16:04:36.841752+0000","last_unstale":"2023-10-08T16:04:36.847766+0000","last_undegraded":"2023-10-08T16:04:36.847766+0000","last_fullsized":"2023-10-08T16:04:36.847766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844363+0000","last_change":"2023-10-08T16:04:36.841982+0000","last_active":"2023-10-08T16:04:36.844363+0000","last_peered":"2023-10-08T16:04:36.844363+0000","last_clean":"2023-10-08T16:04:36.844363+0000","last_became_active":"2023-10-08T16:04:36.841738+0000","last_became_peered":"2023-10-08T16:04:36.841738+0000","last_unstale":"2023-10-08T16:04:36.844363+0000","last_undegraded":"2023-10-08T16:04:36.844363+0000","last_fullsized":"2023-10-08T16:04:36.844363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:28.994728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861145+0000","last_change":"2023-10-08T16:04:36.857688+0000","last_active":"2023-10-08T16:04:36.861145+0000","last_peered":"2023-10-08T16:04:36.861145+0000","last_clean":"2023-10-08T16:04:36.861145+0000","last_became_active":"2023-10-08T16:04:36.857001+0000","last_became_peered":"2023-10-08T16:04:36.857001+0000","last_unstale":"2023-10-08T16:04:36.861145+0000","last_undegraded":"2023-10-08T16:04:36.861145+0000","last_fullsized":"2023-10-08T16:04:36.861145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848398+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848398+0000","last_peered":"2023-10-08T16:04:36.848398+0000","last_clean":"2023-10-08T16:04:36.848398+0000","last_became_active":"2023-10-08T16:04:36.842806+0000","last_became_peered":"2023-10-08T16:04:36.842806+0000","last_unstale":"2023-10-08T16:04:36.848398+0000","last_undegraded":"2023-10-08T16:04:36.848398+0000","last_fullsized":"2023-10-08T16:04:36.848398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854094+0000","last_change":"2023-10-08T16:04:36.845475+0000","last_active":"2023-10-08T16:04:36.854094+0000","last_peered":"2023-10-08T16:04:36.854094+0000","last_clean":"2023-10-08T16:04:36.854094+0000","last_became_active":"2023-10-08T16:04:36.845053+0000","last_became_peered":"2023-10-08T16:04:36.845053+0000","last_unstale":"2023-10-08T16:04:36.854094+0000","last_undegraded":"2023-10-08T16:04:36.854094+0000","last_fullsized":"2023-10-08T16:04:36.854094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848364+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848364+0000","last_peered":"2023-10-08T16:04:36.848364+0000","last_clean":"2023-10-08T16:04:36.848364+0000","last_became_active":"2023-10-08T16:04:36.842841+0000","last_became_peered":"2023-10-08T16:04:36.842841+0000","last_unstale":"2023-10-08T16:04:36.848364+0000","last_undegraded":"2023-10-08T16:04:36.848364+0000","last_fullsized":"2023-10-08T16:04:36.848364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855557+0000","last_change":"2023-10-08T16:04:36.850496+0000","last_active":"2023-10-08T16:04:36.855557+0000","last_peered":"2023-10-08T16:04:36.855557+0000","last_clean":"2023-10-08T16:04:36.855557+0000","last_became_active":"2023-10-08T16:04:36.850237+0000","last_became_peered":"2023-10-08T16:04:36.850237+0000","last_unstale":"2023-10-08T16:04:36.855557+0000","last_undegraded":"2023-10-08T16:04:36.855557+0000","last_fullsized":"2023-10-08T16:04:36.855557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854848+0000","last_change":"2023-10-08T16:04:36.847152+0000","last_active":"2023-10-08T16:04:36.854848+0000","last_peered":"2023-10-08T16:04:36.854848+0000","last_clean":"2023-10-08T16:04:36.854848+0000","last_became_active":"2023-10-08T16:04:36.846573+0000","last_became_peered":"2023-10-08T16:04:36.846573+0000","last_unstale":"2023-10-08T16:04:36.854848+0000","last_undegraded":"2023-10-08T16:04:36.854848+0000","last_fullsized":"2023-10-08T16:04:36.854848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853411+0000","last_change":"2023-10-08T16:04:36.847416+0000","last_active":"2023-10-08T16:04:36.853411+0000","last_peered":"2023-10-08T16:04:36.853411+0000","last_clean":"2023-10-08T16:04:36.853411+0000","last_became_active":"2023-10-08T16:04:36.847092+0000","last_became_peered":"2023-10-08T16:04:36.847092+0000","last_unstale":"2023-10-08T16:04:36.853411+0000","last_undegraded":"2023-10-08T16:04:36.853411+0000","last_fullsized":"2023-10-08T16:04:36.853411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860614+0000","last_change":"2023-10-08T16:04:36.854204+0000","last_active":"2023-10-08T16:04:36.860614+0000","last_peered":"2023-10-08T16:04:36.860614+0000","last_clean":"2023-10-08T16:04:36.860614+0000","last_became_active":"2023-10-08T16:04:36.853963+0000","last_became_peered":"2023-10-08T16:04:36.853963+0000","last_unstale":"2023-10-08T16:04:36.860614+0000","last_undegraded":"2023-10-08T16:04:36.860614+0000","last_fullsized":"2023-10-08T16:04:36.860614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859253+0000","last_change":"2023-10-08T16:04:36.855269+0000","last_active":"2023-10-08T16:04:36.859253+0000","last_peered":"2023-10-08T16:04:36.859253+0000","last_clean":"2023-10-08T16:04:36.859253+0000","last_became_active":"2023-10-08T16:04:36.854443+0000","last_became_peered":"2023-10-08T16:04:36.854443+0000","last_unstale":"2023-10-08T16:04:36.859253+0000","last_undegraded":"2023-10-08T16:04:36.859253+0000","last_fullsized":"2023-10-08T16:04:36.859253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844837+0000","last_change":"2023-10-08T16:04:36.842900+0000","last_active":"2023-10-08T16:04:36.844837+0000","last_peered":"2023-10-08T16:04:36.844837+0000","last_clean":"2023-10-08T16:04:36.844837+0000","last_became_active":"2023-10-08T16:04:36.842642+0000","last_became_peered":"2023-10-08T16:04:36.842642+0000","last_unstale":"2023-10-08T16:04:36.844837+0000","last_undegraded":"2023-10-08T16:04:36.844837+0000","last_fullsized":"2023-10-08T16:04:36.844837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844389+0000","last_change":"2023-10-08T16:04:36.842577+0000","last_active":"2023-10-08T16:04:36.844389+0000","last_peered":"2023-10-08T16:04:36.844389+0000","last_clean":"2023-10-08T16:04:36.844389+0000","last_became_active":"2023-10-08T16:04:36.841699+0000","last_became_peered":"2023-10-08T16:04:36.841699+0000","last_unstale":"2023-10-08T16:04:36.844389+0000","last_undegraded":"2023-10-08T16:04:36.844389+0000","last_fullsized":"2023-10-08T16:04:36.844389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850510+0000","last_change":"2023-10-08T16:04:36.845580+0000","last_active":"2023-10-08T16:04:36.850510+0000","last_peered":"2023-10-08T16:04:36.850510+0000","last_clean":"2023-10-08T16:04:36.850510+0000","last_became_active":"2023-10-08T16:04:36.845348+0000","last_became_peered":"2023-10-08T16:04:36.845348+0000","last_unstale":"2023-10-08T16:04:36.850510+0000","last_undegraded":"2023-10-08T16:04:36.850510+0000","last_fullsized":"2023-10-08T16:04:36.850510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854086+0000","last_change":"2023-10-08T16:04:36.845629+0000","last_active":"2023-10-08T16:04:36.854086+0000","last_peered":"2023-10-08T16:04:36.854086+0000","last_clean":"2023-10-08T16:04:36.854086+0000","last_became_active":"2023-10-08T16:04:36.845283+0000","last_became_peered":"2023-10-08T16:04:36.845283+0000","last_unstale":"2023-10-08T16:04:36.854086+0000","last_undegraded":"2023-10-08T16:04:36.854086+0000","last_fullsized":"2023-10-08T16:04:36.854086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856506+0000","last_change":"2023-10-08T16:04:36.852540+0000","last_active":"2023-10-08T16:04:36.856506+0000","last_peered":"2023-10-08T16:04:36.856506+0000","last_clean":"2023-10-08T16:04:36.856506+0000","last_became_active":"2023-10-08T16:04:36.852294+0000","last_became_peered":"2023-10-08T16:04:36.852294+0000","last_unstale":"2023-10-08T16:04:36.856506+0000","last_undegraded":"2023-10-08T16:04:36.856506+0000","last_fullsized":"2023-10-08T16:04:36.856506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850863+0000","last_change":"2023-10-08T16:04:36.845236+0000","last_active":"2023-10-08T16:04:36.850863+0000","last_peered":"2023-10-08T16:04:36.850863+0000","last_clean":"2023-10-08T16:04:36.850863+0000","last_became_active":"2023-10-08T16:04:36.844966+0000","last_became_peered":"2023-10-08T16:04:36.844966+0000","last_unstale":"2023-10-08T16:04:36.850863+0000","last_undegraded":"2023-10-08T16:04:36.850863+0000","last_fullsized":"2023-10-08T16:04:36.850863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849074+0000","last_change":"2023-10-08T16:04:36.844871+0000","last_active":"2023-10-08T16:04:36.849074+0000","last_peered":"2023-10-08T16:04:36.849074+0000","last_clean":"2023-10-08T16:04:36.849074+0000","last_became_active":"2023-10-08T16:04:36.844378+0000","last_became_peered":"2023-10-08T16:04:36.844378+0000","last_unstale":"2023-10-08T16:04:36.849074+0000","last_undegraded":"2023-10-08T16:04:36.849074+0000","last_fullsized":"2023-10-08T16:04:36.849074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861810+0000","last_change":"2023-10-08T16:04:35.842581+0000","last_active":"2023-10-08T16:04:36.861810+0000","last_peered":"2023-10-08T16:04:36.861810+0000","last_clean":"2023-10-08T16:04:36.861810+0000","last_became_active":"2023-10-08T16:04:35.842258+0000","last_became_peered":"2023-10-08T16:04:35.842258+0000","last_unstale":"2023-10-08T16:04:36.861810+0000","last_undegraded":"2023-10-08T16:04:36.861810+0000","last_fullsized":"2023-10-08T16:04:36.861810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:04.459691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844626+0000","last_change":"2023-10-08T16:04:36.842889+0000","last_active":"2023-10-08T16:04:36.844626+0000","last_peered":"2023-10-08T16:04:36.844626+0000","last_clean":"2023-10-08T16:04:36.844626+0000","last_became_active":"2023-10-08T16:04:36.842161+0000","last_became_peered":"2023-10-08T16:04:36.842161+0000","last_unstale":"2023-10-08T16:04:36.844626+0000","last_undegraded":"2023-10-08T16:04:36.844626+0000","last_fullsized":"2023-10-08T16:04:36.844626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856629+0000","last_change":"2023-10-08T16:04:35.835973+0000","last_active":"2023-10-08T16:04:36.856629+0000","last_peered":"2023-10-08T16:04:36.856629+0000","last_clean":"2023-10-08T16:04:36.856629+0000","last_became_active":"2023-10-08T16:04:35.835785+0000","last_became_peered":"2023-10-08T16:04:35.835785+0000","last_unstale":"2023-10-08T16:04:36.856629+0000","last_undegraded":"2023-10-08T16:04:36.856629+0000","last_fullsized":"2023-10-08T16:04:36.856629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849375+0000","last_change":"2023-10-08T16:04:36.845806+0000","last_active":"2023-10-08T16:04:36.849375+0000","last_peered":"2023-10-08T16:04:36.849375+0000","last_clean":"2023-10-08T16:04:36.849375+0000","last_became_active":"2023-10-08T16:04:36.845574+0000","last_became_peered":"2023-10-08T16:04:36.845574+0000","last_unstale":"2023-10-08T16:04:36.849375+0000","last_undegraded":"2023-10-08T16:04:36.849375+0000","last_fullsized":"2023-10-08T16:04:36.849375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860925+0000","last_change":"2023-10-08T16:04:35.844449+0000","last_active":"2023-10-08T16:04:36.860925+0000","last_peered":"2023-10-08T16:04:36.860925+0000","last_clean":"2023-10-08T16:04:36.860925+0000","last_became_active":"2023-10-08T16:04:35.844192+0000","last_became_peered":"2023-10-08T16:04:35.844192+0000","last_unstale":"2023-10-08T16:04:36.860925+0000","last_undegraded":"2023-10-08T16:04:36.860925+0000","last_fullsized":"2023-10-08T16:04:36.860925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856262+0000","last_change":"2023-10-08T16:04:36.853135+0000","last_active":"2023-10-08T16:04:36.856262+0000","last_peered":"2023-10-08T16:04:36.856262+0000","last_clean":"2023-10-08T16:04:36.856262+0000","last_became_active":"2023-10-08T16:04:36.852832+0000","last_became_peered":"2023-10-08T16:04:36.852832+0000","last_unstale":"2023-10-08T16:04:36.856262+0000","last_undegraded":"2023-10-08T16:04:36.856262+0000","last_fullsized":"2023-10-08T16:04:36.856262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861744+0000","last_change":"2023-10-08T16:04:35.845418+0000","last_active":"2023-10-08T16:04:36.861744+0000","last_peered":"2023-10-08T16:04:36.861744+0000","last_clean":"2023-10-08T16:04:36.861744+0000","last_became_active":"2023-10-08T16:04:35.845104+0000","last_became_peered":"2023-10-08T16:04:35.845104+0000","last_unstale":"2023-10-08T16:04:36.861744+0000","last_undegraded":"2023-10-08T16:04:36.861744+0000","last_fullsized":"2023-10-08T16:04:36.861744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856172+0000","last_change":"2023-10-08T16:04:36.851677+0000","last_active":"2023-10-08T16:04:36.856172+0000","last_peered":"2023-10-08T16:04:36.856172+0000","last_clean":"2023-10-08T16:04:36.856172+0000","last_became_active":"2023-10-08T16:04:36.847960+0000","last_became_peered":"2023-10-08T16:04:36.847960+0000","last_unstale":"2023-10-08T16:04:36.856172+0000","last_undegraded":"2023-10-08T16:04:36.856172+0000","last_fullsized":"2023-10-08T16:04:36.856172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861300+0000","last_change":"2023-10-08T16:04:35.833685+0000","last_active":"2023-10-08T16:04:36.861300+0000","last_peered":"2023-10-08T16:04:36.861300+0000","last_clean":"2023-10-08T16:04:36.861300+0000","last_became_active":"2023-10-08T16:04:35.833403+0000","last_became_peered":"2023-10-08T16:04:35.833403+0000","last_unstale":"2023-10-08T16:04:36.861300+0000","last_undegraded":"2023-10-08T16:04:36.861300+0000","last_fullsized":"2023-10-08T16:04:36.861300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851430+0000","last_change":"2023-10-08T16:04:36.845487+0000","last_active":"2023-10-08T16:04:36.851430+0000","last_peered":"2023-10-08T16:04:36.851430+0000","last_clean":"2023-10-08T16:04:36.851430+0000","last_became_active":"2023-10-08T16:04:36.845204+0000","last_became_peered":"2023-10-08T16:04:36.845204+0000","last_unstale":"2023-10-08T16:04:36.851430+0000","last_undegraded":"2023-10-08T16:04:36.851430+0000","last_fullsized":"2023-10-08T16:04:36.851430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:37.980506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861219+0000","last_change":"2023-10-08T16:04:35.833842+0000","last_active":"2023-10-08T16:04:36.861219+0000","last_peered":"2023-10-08T16:04:36.861219+0000","last_clean":"2023-10-08T16:04:36.861219+0000","last_became_active":"2023-10-08T16:04:35.833461+0000","last_became_peered":"2023-10-08T16:04:35.833461+0000","last_unstale":"2023-10-08T16:04:36.861219+0000","last_undegraded":"2023-10-08T16:04:36.861219+0000","last_fullsized":"2023-10-08T16:04:36.861219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855228+0000","last_change":"2023-10-08T16:04:36.849156+0000","last_active":"2023-10-08T16:04:36.855228+0000","last_peered":"2023-10-08T16:04:36.855228+0000","last_clean":"2023-10-08T16:04:36.855228+0000","last_became_active":"2023-10-08T16:04:36.848907+0000","last_became_peered":"2023-10-08T16:04:36.848907+0000","last_unstale":"2023-10-08T16:04:36.855228+0000","last_undegraded":"2023-10-08T16:04:36.855228+0000","last_fullsized":"2023-10-08T16:04:36.855228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:04.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834426+0000","last_change":"2023-10-08T16:04:35.838361+0000","last_active":"2023-10-08T16:04:36.834426+0000","last_peered":"2023-10-08T16:04:36.834426+0000","last_clean":"2023-10-08T16:04:36.834426+0000","last_became_active":"2023-10-08T16:04:35.838175+0000","last_became_peered":"2023-10-08T16:04:35.838175+0000","last_unstale":"2023-10-08T16:04:36.834426+0000","last_undegraded":"2023-10-08T16:04:36.834426+0000","last_fullsized":"2023-10-08T16:04:36.834426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851367+0000","last_change":"2023-10-08T16:04:36.843489+0000","last_active":"2023-10-08T16:04:36.851367+0000","last_peered":"2023-10-08T16:04:36.851367+0000","last_clean":"2023-10-08T16:04:36.851367+0000","last_became_active":"2023-10-08T16:04:36.843221+0000","last_became_peered":"2023-10-08T16:04:36.843221+0000","last_unstale":"2023-10-08T16:04:36.851367+0000","last_undegraded":"2023-10-08T16:04:36.851367+0000","last_fullsized":"2023-10-08T16:04:36.851367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840321+0000","last_change":"2023-10-08T16:04:35.839307+0000","last_active":"2023-10-08T16:04:36.840321+0000","last_peered":"2023-10-08T16:04:36.840321+0000","last_clean":"2023-10-08T16:04:36.840321+0000","last_became_active":"2023-10-08T16:04:35.838997+0000","last_became_peered":"2023-10-08T16:04:35.838997+0000","last_unstale":"2023-10-08T16:04:36.840321+0000","last_undegraded":"2023-10-08T16:04:36.840321+0000","last_fullsized":"2023-10-08T16:04:36.840321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840059+0000","last_change":"2023-10-08T16:04:36.838211+0000","last_active":"2023-10-08T16:04:36.840059+0000","last_peered":"2023-10-08T16:04:36.840059+0000","last_clean":"2023-10-08T16:04:36.840059+0000","last_became_active":"2023-10-08T16:04:36.837948+0000","last_became_peered":"2023-10-08T16:04:36.837948+0000","last_unstale":"2023-10-08T16:04:36.840059+0000","last_undegraded":"2023-10-08T16:04:36.840059+0000","last_fullsized":"2023-10-08T16:04:36.840059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838498+0000","last_change":"2023-10-08T16:04:35.841004+0000","last_active":"2023-10-08T16:04:36.838498+0000","last_peered":"2023-10-08T16:04:36.838498+0000","last_clean":"2023-10-08T16:04:36.838498+0000","last_became_active":"2023-10-08T16:04:35.840735+0000","last_became_peered":"2023-10-08T16:04:35.840735+0000","last_unstale":"2023-10-08T16:04:36.838498+0000","last_undegraded":"2023-10-08T16:04:36.838498+0000","last_fullsized":"2023-10-08T16:04:36.838498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855922+0000","last_change":"2023-10-08T16:04:36.850043+0000","last_active":"2023-10-08T16:04:36.855922+0000","last_peered":"2023-10-08T16:04:36.855922+0000","last_clean":"2023-10-08T16:04:36.855922+0000","last_became_active":"2023-10-08T16:04:36.849743+0000","last_became_peered":"2023-10-08T16:04:36.849743+0000","last_unstale":"2023-10-08T16:04:36.855922+0000","last_undegraded":"2023-10-08T16:04:36.855922+0000","last_fullsized":"2023-10-08T16:04:36.855922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838528+0000","last_change":"2023-10-08T16:04:35.844419+0000","last_active":"2023-10-08T16:04:36.838528+0000","last_peered":"2023-10-08T16:04:36.838528+0000","last_clean":"2023-10-08T16:04:36.838528+0000","last_became_active":"2023-10-08T16:04:35.844125+0000","last_became_peered":"2023-10-08T16:04:35.844125+0000","last_unstale":"2023-10-08T16:04:36.838528+0000","last_undegraded":"2023-10-08T16:04:36.838528+0000","last_fullsized":"2023-10-08T16:04:36.838528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849356+0000","last_change":"2023-10-08T16:04:36.845771+0000","last_active":"2023-10-08T16:04:36.849356+0000","last_peered":"2023-10-08T16:04:36.849356+0000","last_clean":"2023-10-08T16:04:36.849356+0000","last_became_active":"2023-10-08T16:04:36.845274+0000","last_became_peered":"2023-10-08T16:04:36.845274+0000","last_unstale":"2023-10-08T16:04:36.849356+0000","last_undegraded":"2023-10-08T16:04:36.849356+0000","last_fullsized":"2023-10-08T16:04:36.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856530+0000","last_change":"2023-10-08T16:04:35.840602+0000","last_active":"2023-10-08T16:04:36.856530+0000","last_peered":"2023-10-08T16:04:36.856530+0000","last_clean":"2023-10-08T16:04:36.856530+0000","last_became_active":"2023-10-08T16:04:35.840299+0000","last_became_peered":"2023-10-08T16:04:35.840299+0000","last_unstale":"2023-10-08T16:04:36.856530+0000","last_undegraded":"2023-10-08T16:04:36.856530+0000","last_fullsized":"2023-10-08T16:04:36.856530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853166+0000","last_change":"2023-10-08T16:04:36.845842+0000","last_active":"2023-10-08T16:04:36.853166+0000","last_peered":"2023-10-08T16:04:36.853166+0000","last_clean":"2023-10-08T16:04:36.853166+0000","last_became_active":"2023-10-08T16:04:36.844954+0000","last_became_peered":"2023-10-08T16:04:36.844954+0000","last_unstale":"2023-10-08T16:04:36.853166+0000","last_undegraded":"2023-10-08T16:04:36.853166+0000","last_fullsized":"2023-10-08T16:04:36.853166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:34.979889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.863226+0000","last_change":"2023-10-08T16:04:35.838972+0000","last_active":"2023-10-08T16:04:36.863226+0000","last_peered":"2023-10-08T16:04:36.863226+0000","last_clean":"2023-10-08T16:04:36.863226+0000","last_became_active":"2023-10-08T16:04:35.838694+0000","last_became_peered":"2023-10-08T16:04:35.838694+0000","last_unstale":"2023-10-08T16:04:36.863226+0000","last_undegraded":"2023-10-08T16:04:36.863226+0000","last_fullsized":"2023-10-08T16:04:36.863226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849056+0000","last_change":"2023-10-08T16:04:36.844679+0000","last_active":"2023-10-08T16:04:36.849056+0000","last_peered":"2023-10-08T16:04:36.849056+0000","last_clean":"2023-10-08T16:04:36.849056+0000","last_became_active":"2023-10-08T16:04:36.844115+0000","last_became_peered":"2023-10-08T16:04:36.844115+0000","last_unstale":"2023-10-08T16:04:36.849056+0000","last_undegraded":"2023-10-08T16:04:36.849056+0000","last_fullsized":"2023-10-08T16:04:36.849056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859530+0000","last_change":"2023-10-08T16:04:35.837996+0000","last_active":"2023-10-08T16:04:36.859530+0000","last_peered":"2023-10-08T16:04:36.859530+0000","last_clean":"2023-10-08T16:04:36.859530+0000","last_became_active":"2023-10-08T16:04:35.837567+0000","last_became_peered":"2023-10-08T16:04:35.837567+0000","last_unstale":"2023-10-08T16:04:36.859530+0000","last_undegraded":"2023-10-08T16:04:36.859530+0000","last_fullsized":"2023-10-08T16:04:36.859530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855385+0000","last_change":"2023-10-08T16:04:36.849839+0000","last_active":"2023-10-08T16:04:36.855385+0000","last_peered":"2023-10-08T16:04:36.855385+0000","last_clean":"2023-10-08T16:04:36.855385+0000","last_became_active":"2023-10-08T16:04:36.849577+0000","last_became_peered":"2023-10-08T16:04:36.849577+0000","last_unstale":"2023-10-08T16:04:36.855385+0000","last_undegraded":"2023-10-08T16:04:36.855385+0000","last_fullsized":"2023-10-08T16:04:36.855385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:55.787779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838421+0000","last_change":"2023-10-08T16:04:35.841168+0000","last_active":"2023-10-08T16:04:36.838421+0000","last_peered":"2023-10-08T16:04:36.838421+0000","last_clean":"2023-10-08T16:04:36.838421+0000","last_became_active":"2023-10-08T16:04:35.840667+0000","last_became_peered":"2023-10-08T16:04:35.840667+0000","last_unstale":"2023-10-08T16:04:36.838421+0000","last_undegraded":"2023-10-08T16:04:36.838421+0000","last_fullsized":"2023-10-08T16:04:36.838421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859673+0000","last_change":"2023-10-08T16:04:36.849407+0000","last_active":"2023-10-08T16:04:36.859673+0000","last_peered":"2023-10-08T16:04:36.859673+0000","last_clean":"2023-10-08T16:04:36.859673+0000","last_became_active":"2023-10-08T16:04:36.849157+0000","last_became_peered":"2023-10-08T16:04:36.849157+0000","last_unstale":"2023-10-08T16:04:36.859673+0000","last_undegraded":"2023-10-08T16:04:36.859673+0000","last_fullsized":"2023-10-08T16:04:36.859673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836282+0000","last_change":"2023-10-08T16:04:35.835032+0000","last_active":"2023-10-08T16:04:36.836282+0000","last_peered":"2023-10-08T16:04:36.836282+0000","last_clean":"2023-10-08T16:04:36.836282+0000","last_became_active":"2023-10-08T16:04:35.834550+0000","last_became_peered":"2023-10-08T16:04:35.834550+0000","last_unstale":"2023-10-08T16:04:36.836282+0000","last_undegraded":"2023-10-08T16:04:36.836282+0000","last_fullsized":"2023-10-08T16:04:36.836282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855292+0000","last_change":"2023-10-08T16:04:36.848762+0000","last_active":"2023-10-08T16:04:36.855292+0000","last_peered":"2023-10-08T16:04:36.855292+0000","last_clean":"2023-10-08T16:04:36.855292+0000","last_became_active":"2023-10-08T16:04:36.848407+0000","last_became_peered":"2023-10-08T16:04:36.848407+0000","last_unstale":"2023-10-08T16:04:36.855292+0000","last_undegraded":"2023-10-08T16:04:36.855292+0000","last_fullsized":"2023-10-08T16:04:36.855292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856015+0000","last_change":"2023-10-08T16:04:35.834021+0000","last_active":"2023-10-08T16:04:36.856015+0000","last_peered":"2023-10-08T16:04:36.856015+0000","last_clean":"2023-10-08T16:04:36.856015+0000","last_became_active":"2023-10-08T16:04:35.833739+0000","last_became_peered":"2023-10-08T16:04:35.833739+0000","last_unstale":"2023-10-08T16:04:36.856015+0000","last_undegraded":"2023-10-08T16:04:36.856015+0000","last_fullsized":"2023-10-08T16:04:36.856015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859686+0000","last_change":"2023-10-08T16:04:36.850254+0000","last_active":"2023-10-08T16:04:36.859686+0000","last_peered":"2023-10-08T16:04:36.859686+0000","last_clean":"2023-10-08T16:04:36.859686+0000","last_became_active":"2023-10-08T16:04:36.849843+0000","last_became_peered":"2023-10-08T16:04:36.849843+0000","last_unstale":"2023-10-08T16:04:36.859686+0000","last_undegraded":"2023-10-08T16:04:36.859686+0000","last_fullsized":"2023-10-08T16:04:36.859686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859275+0000","last_change":"2023-10-08T16:04:35.837065+0000","last_active":"2023-10-08T16:04:36.859275+0000","last_peered":"2023-10-08T16:04:36.859275+0000","last_clean":"2023-10-08T16:04:36.859275+0000","last_became_active":"2023-10-08T16:04:35.836764+0000","last_became_peered":"2023-10-08T16:04:35.836764+0000","last_unstale":"2023-10-08T16:04:36.859275+0000","last_undegraded":"2023-10-08T16:04:36.859275+0000","last_fullsized":"2023-10-08T16:04:36.859275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:43.449593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848948+0000","last_change":"2023-10-08T16:04:36.845465+0000","last_active":"2023-10-08T16:04:36.848948+0000","last_peered":"2023-10-08T16:04:36.848948+0000","last_clean":"2023-10-08T16:04:36.848948+0000","last_became_active":"2023-10-08T16:04:36.845207+0000","last_became_peered":"2023-10-08T16:04:36.845207+0000","last_unstale":"2023-10-08T16:04:36.848948+0000","last_undegraded":"2023-10-08T16:04:36.848948+0000","last_fullsized":"2023-10-08T16:04:36.848948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831893+0000","last_change":"2023-10-08T16:04:35.836100+0000","last_active":"2023-10-08T16:04:36.831893+0000","last_peered":"2023-10-08T16:04:36.831893+0000","last_clean":"2023-10-08T16:04:36.831893+0000","last_became_active":"2023-10-08T16:04:35.835832+0000","last_became_peered":"2023-10-08T16:04:35.835832+0000","last_unstale":"2023-10-08T16:04:36.831893+0000","last_undegraded":"2023-10-08T16:04:36.831893+0000","last_fullsized":"2023-10-08T16:04:36.831893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860666+0000","last_change":"2023-10-08T16:04:36.853664+0000","last_active":"2023-10-08T16:04:36.860666+0000","last_peered":"2023-10-08T16:04:36.860666+0000","last_clean":"2023-10-08T16:04:36.860666+0000","last_became_active":"2023-10-08T16:04:36.853412+0000","last_became_peered":"2023-10-08T16:04:36.853412+0000","last_unstale":"2023-10-08T16:04:36.860666+0000","last_undegraded":"2023-10-08T16:04:36.860666+0000","last_fullsized":"2023-10-08T16:04:36.860666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838343+0000","last_change":"2023-10-08T16:04:35.838075+0000","last_active":"2023-10-08T16:04:36.838343+0000","last_peered":"2023-10-08T16:04:36.838343+0000","last_clean":"2023-10-08T16:04:36.838343+0000","last_became_active":"2023-10-08T16:04:35.837821+0000","last_became_peered":"2023-10-08T16:04:35.837821+0000","last_unstale":"2023-10-08T16:04:36.838343+0000","last_undegraded":"2023-10-08T16:04:36.838343+0000","last_fullsized":"2023-10-08T16:04:36.838343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850970+0000","last_change":"2023-10-08T16:04:36.845413+0000","last_active":"2023-10-08T16:04:36.850970+0000","last_peered":"2023-10-08T16:04:36.850970+0000","last_clean":"2023-10-08T16:04:36.850970+0000","last_became_active":"2023-10-08T16:04:36.845152+0000","last_became_peered":"2023-10-08T16:04:36.845152+0000","last_unstale":"2023-10-08T16:04:36.850970+0000","last_undegraded":"2023-10-08T16:04:36.850970+0000","last_fullsized":"2023-10-08T16:04:36.850970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839476+0000","last_change":"2023-10-08T16:04:36.837273+0000","last_active":"2023-10-08T16:04:36.839476+0000","last_peered":"2023-10-08T16:04:36.839476+0000","last_clean":"2023-10-08T16:04:36.839476+0000","last_became_active":"2023-10-08T16:04:36.837025+0000","last_became_peered":"2023-10-08T16:04:36.837025+0000","last_unstale":"2023-10-08T16:04:36.839476+0000","last_undegraded":"2023-10-08T16:04:36.839476+0000","last_fullsized":"2023-10-08T16:04:36.839476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833690+0000","last_change":"2023-10-08T16:04:35.831326+0000","last_active":"2023-10-08T16:04:36.833690+0000","last_peered":"2023-10-08T16:04:36.833690+0000","last_clean":"2023-10-08T16:04:36.833690+0000","last_became_active":"2023-10-08T16:04:35.831088+0000","last_became_peered":"2023-10-08T16:04:35.831088+0000","last_unstale":"2023-10-08T16:04:36.833690+0000","last_undegraded":"2023-10-08T16:04:36.833690+0000","last_fullsized":"2023-10-08T16:04:36.833690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855327+0000","last_change":"2023-10-08T16:04:36.847842+0000","last_active":"2023-10-08T16:04:36.855327+0000","last_peered":"2023-10-08T16:04:36.855327+0000","last_clean":"2023-10-08T16:04:36.855327+0000","last_became_active":"2023-10-08T16:04:36.847222+0000","last_became_peered":"2023-10-08T16:04:36.847222+0000","last_unstale":"2023-10-08T16:04:36.855327+0000","last_undegraded":"2023-10-08T16:04:36.855327+0000","last_fullsized":"2023-10-08T16:04:36.855327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835188+0000","last_change":"2023-10-08T16:04:35.837295+0000","last_active":"2023-10-08T16:04:36.835188+0000","last_peered":"2023-10-08T16:04:36.835188+0000","last_clean":"2023-10-08T16:04:36.835188+0000","last_became_active":"2023-10-08T16:04:35.836609+0000","last_became_peered":"2023-10-08T16:04:35.836609+0000","last_unstale":"2023-10-08T16:04:36.835188+0000","last_undegraded":"2023-10-08T16:04:36.835188+0000","last_fullsized":"2023-10-08T16:04:36.835188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835250+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:04:36.835250+0000","last_peered":"2023-10-08T16:04:36.835250+0000","last_clean":"2023-10-08T16:04:36.835250+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:04:36.835250+0000","last_undegraded":"2023-10-08T16:04:36.835250+0000","last_fullsized":"2023-10-08T16:04:36.835250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855399+0000","last_change":"2023-10-08T16:04:36.849109+0000","last_active":"2023-10-08T16:04:36.855399+0000","last_peered":"2023-10-08T16:04:36.855399+0000","last_clean":"2023-10-08T16:04:36.855399+0000","last_became_active":"2023-10-08T16:04:36.848847+0000","last_became_peered":"2023-10-08T16:04:36.848847+0000","last_unstale":"2023-10-08T16:04:36.855399+0000","last_undegraded":"2023-10-08T16:04:36.855399+0000","last_fullsized":"2023-10-08T16:04:36.855399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:34:26.769453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835538+0000","last_change":"2023-10-08T16:04:35.840218+0000","last_active":"2023-10-08T16:04:36.835538+0000","last_peered":"2023-10-08T16:04:36.835538+0000","last_clean":"2023-10-08T16:04:36.835538+0000","last_became_active":"2023-10-08T16:04:35.839272+0000","last_became_peered":"2023-10-08T16:04:35.839272+0000","last_unstale":"2023-10-08T16:04:36.835538+0000","last_undegraded":"2023-10-08T16:04:36.835538+0000","last_fullsized":"2023-10-08T16:04:36.835538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836042+0000","last_change":"2023-10-08T16:04:35.842423+0000","last_active":"2023-10-08T16:04:36.836042+0000","last_peered":"2023-10-08T16:04:36.836042+0000","last_clean":"2023-10-08T16:04:36.836042+0000","last_became_active":"2023-10-08T16:04:35.842183+0000","last_became_peered":"2023-10-08T16:04:35.842183+0000","last_unstale":"2023-10-08T16:04:36.836042+0000","last_undegraded":"2023-10-08T16:04:36.836042+0000","last_fullsized":"2023-10-08T16:04:36.836042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839745+0000","last_change":"2023-10-08T16:04:36.837348+0000","last_active":"2023-10-08T16:04:36.839745+0000","last_peered":"2023-10-08T16:04:36.839745+0000","last_clean":"2023-10-08T16:04:36.839745+0000","last_became_active":"2023-10-08T16:04:36.837116+0000","last_became_peered":"2023-10-08T16:04:36.837116+0000","last_unstale":"2023-10-08T16:04:36.839745+0000","last_undegraded":"2023-10-08T16:04:36.839745+0000","last_fullsized":"2023-10-08T16:04:36.839745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.832360+0000","last_change":"2023-10-08T16:04:35.834800+0000","last_active":"2023-10-08T16:04:36.832360+0000","last_peered":"2023-10-08T16:04:36.832360+0000","last_clean":"2023-10-08T16:04:36.832360+0000","last_became_active":"2023-10-08T16:04:35.834516+0000","last_became_peered":"2023-10-08T16:04:35.834516+0000","last_unstale":"2023-10-08T16:04:36.832360+0000","last_undegraded":"2023-10-08T16:04:36.832360+0000","last_fullsized":"2023-10-08T16:04:36.832360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855314+0000","last_change":"2023-10-08T16:04:36.850191+0000","last_active":"2023-10-08T16:04:36.855314+0000","last_peered":"2023-10-08T16:04:36.855314+0000","last_clean":"2023-10-08T16:04:36.855314+0000","last_became_active":"2023-10-08T16:04:36.849886+0000","last_became_peered":"2023-10-08T16:04:36.849886+0000","last_unstale":"2023-10-08T16:04:36.855314+0000","last_undegraded":"2023-10-08T16:04:36.855314+0000","last_fullsized":"2023-10-08T16:04:36.855314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861589+0000","last_change":"2023-10-08T16:04:35.842733+0000","last_active":"2023-10-08T16:04:36.861589+0000","last_peered":"2023-10-08T16:04:36.861589+0000","last_clean":"2023-10-08T16:04:36.861589+0000","last_became_active":"2023-10-08T16:04:35.842431+0000","last_became_peered":"2023-10-08T16:04:35.842431+0000","last_unstale":"2023-10-08T16:04:36.861589+0000","last_undegraded":"2023-10-08T16:04:36.861589+0000","last_fullsized":"2023-10-08T16:04:36.861589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843518+0000","last_change":"2023-10-08T16:04:36.841209+0000","last_active":"2023-10-08T16:04:36.843518+0000","last_peered":"2023-10-08T16:04:36.843518+0000","last_clean":"2023-10-08T16:04:36.843518+0000","last_became_active":"2023-10-08T16:04:36.840900+0000","last_became_peered":"2023-10-08T16:04:36.840900+0000","last_unstale":"2023-10-08T16:04:36.843518+0000","last_undegraded":"2023-10-08T16:04:36.843518+0000","last_fullsized":"2023-10-08T16:04:36.843518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834548+0000","last_change":"2023-10-08T16:04:35.840034+0000","last_active":"2023-10-08T16:04:36.834548+0000","last_peered":"2023-10-08T16:04:36.834548+0000","last_clean":"2023-10-08T16:04:36.834548+0000","last_became_active":"2023-10-08T16:04:35.839696+0000","last_became_peered":"2023-10-08T16:04:35.839696+0000","last_unstale":"2023-10-08T16:04:36.834548+0000","last_undegraded":"2023-10-08T16:04:36.834548+0000","last_fullsized":"2023-10-08T16:04:36.834548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:51.871435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853454+0000","last_change":"2023-10-08T16:04:36.847014+0000","last_active":"2023-10-08T16:04:36.853454+0000","last_peered":"2023-10-08T16:04:36.853454+0000","last_clean":"2023-10-08T16:04:36.853454+0000","last_became_active":"2023-10-08T16:04:36.846715+0000","last_became_peered":"2023-10-08T16:04:36.846715+0000","last_unstale":"2023-10-08T16:04:36.853454+0000","last_undegraded":"2023-10-08T16:04:36.853454+0000","last_fullsized":"2023-10-08T16:04:36.853454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:13.602001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831854+0000","last_change":"2023-10-08T16:04:35.833211+0000","last_active":"2023-10-08T16:04:36.831854+0000","last_peered":"2023-10-08T16:04:36.831854+0000","last_clean":"2023-10-08T16:04:36.831854+0000","last_became_active":"2023-10-08T16:04:35.832926+0000","last_became_peered":"2023-10-08T16:04:35.832926+0000","last_unstale":"2023-10-08T16:04:36.831854+0000","last_undegraded":"2023-10-08T16:04:36.831854+0000","last_fullsized":"2023-10-08T16:04:36.831854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840582+0000","last_change":"2023-10-08T16:04:36.839516+0000","last_active":"2023-10-08T16:04:36.840582+0000","last_peered":"2023-10-08T16:04:36.840582+0000","last_clean":"2023-10-08T16:04:36.840582+0000","last_became_active":"2023-10-08T16:04:36.839275+0000","last_became_peered":"2023-10-08T16:04:36.839275+0000","last_unstale":"2023-10-08T16:04:36.840582+0000","last_undegraded":"2023-10-08T16:04:36.840582+0000","last_fullsized":"2023-10-08T16:04:36.840582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831607+0000","last_change":"2023-10-08T16:04:35.826206+0000","last_active":"2023-10-08T16:04:36.831607+0000","last_peered":"2023-10-08T16:04:36.831607+0000","last_clean":"2023-10-08T16:04:36.831607+0000","last_became_active":"2023-10-08T16:04:35.825925+0000","last_became_peered":"2023-10-08T16:04:35.825925+0000","last_unstale":"2023-10-08T16:04:36.831607+0000","last_undegraded":"2023-10-08T16:04:36.831607+0000","last_fullsized":"2023-10-08T16:04:36.831607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855072+0000","last_change":"2023-10-08T16:04:36.845340+0000","last_active":"2023-10-08T16:04:36.855072+0000","last_peered":"2023-10-08T16:04:36.855072+0000","last_clean":"2023-10-08T16:04:36.855072+0000","last_became_active":"2023-10-08T16:04:36.845040+0000","last_became_peered":"2023-10-08T16:04:36.845040+0000","last_unstale":"2023-10-08T16:04:36.855072+0000","last_undegraded":"2023-10-08T16:04:36.855072+0000","last_fullsized":"2023-10-08T16:04:36.855072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860563+0000","last_change":"2023-10-08T16:04:35.841423+0000","last_active":"2023-10-08T16:04:36.860563+0000","last_peered":"2023-10-08T16:04:36.860563+0000","last_clean":"2023-10-08T16:04:36.860563+0000","last_became_active":"2023-10-08T16:04:35.841174+0000","last_became_peered":"2023-10-08T16:04:35.841174+0000","last_unstale":"2023-10-08T16:04:36.860563+0000","last_undegraded":"2023-10-08T16:04:36.860563+0000","last_fullsized":"2023-10-08T16:04:36.860563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860494+0000","last_change":"2023-10-08T16:04:36.851799+0000","last_active":"2023-10-08T16:04:36.860494+0000","last_peered":"2023-10-08T16:04:36.860494+0000","last_clean":"2023-10-08T16:04:36.860494+0000","last_became_active":"2023-10-08T16:04:36.851458+0000","last_became_peered":"2023-10-08T16:04:36.851458+0000","last_unstale":"2023-10-08T16:04:36.860494+0000","last_undegraded":"2023-10-08T16:04:36.860494+0000","last_fullsized":"2023-10-08T16:04:36.860494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833797+0000","last_change":"2023-10-08T16:04:35.840122+0000","last_active":"2023-10-08T16:04:36.833797+0000","last_peered":"2023-10-08T16:04:36.833797+0000","last_clean":"2023-10-08T16:04:36.833797+0000","last_became_active":"2023-10-08T16:04:35.839817+0000","last_became_peered":"2023-10-08T16:04:35.839817+0000","last_unstale":"2023-10-08T16:04:36.833797+0000","last_undegraded":"2023-10-08T16:04:36.833797+0000","last_fullsized":"2023-10-08T16:04:36.833797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855128+0000","last_change":"2023-10-08T16:04:36.847797+0000","last_active":"2023-10-08T16:04:36.855128+0000","last_peered":"2023-10-08T16:04:36.855128+0000","last_clean":"2023-10-08T16:04:36.855128+0000","last_became_active":"2023-10-08T16:04:36.847542+0000","last_became_peered":"2023-10-08T16:04:36.847542+0000","last_unstale":"2023-10-08T16:04:36.855128+0000","last_undegraded":"2023-10-08T16:04:36.855128+0000","last_fullsized":"2023-10-08T16:04:36.855128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859329+0000","last_change":"2023-10-08T16:04:35.826935+0000","last_active":"2023-10-08T16:04:36.859329+0000","last_peered":"2023-10-08T16:04:36.859329+0000","last_clean":"2023-10-08T16:04:36.859329+0000","last_became_active":"2023-10-08T16:04:35.826668+0000","last_became_peered":"2023-10-08T16:04:35.826668+0000","last_unstale":"2023-10-08T16:04:36.859329+0000","last_undegraded":"2023-10-08T16:04:36.859329+0000","last_fullsized":"2023-10-08T16:04:36.859329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861167+0000","last_change":"2023-10-08T16:04:36.857918+0000","last_active":"2023-10-08T16:04:36.861167+0000","last_peered":"2023-10-08T16:04:36.861167+0000","last_clean":"2023-10-08T16:04:36.861167+0000","last_became_active":"2023-10-08T16:04:36.857368+0000","last_became_peered":"2023-10-08T16:04:36.857368+0000","last_unstale":"2023-10-08T16:04:36.861167+0000","last_undegraded":"2023-10-08T16:04:36.861167+0000","last_fullsized":"2023-10-08T16:04:36.861167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855827+0000","last_change":"2023-10-08T16:04:35.836755+0000","last_active":"2023-10-08T16:04:36.855827+0000","last_peered":"2023-10-08T16:04:36.855827+0000","last_clean":"2023-10-08T16:04:36.855827+0000","last_became_active":"2023-10-08T16:04:35.836467+0000","last_became_peered":"2023-10-08T16:04:35.836467+0000","last_unstale":"2023-10-08T16:04:36.855827+0000","last_undegraded":"2023-10-08T16:04:36.855827+0000","last_fullsized":"2023-10-08T16:04:36.855827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860389+0000","last_change":"2023-10-08T16:04:36.851314+0000","last_active":"2023-10-08T16:04:36.860389+0000","last_peered":"2023-10-08T16:04:36.860389+0000","last_clean":"2023-10-08T16:04:36.860389+0000","last_became_active":"2023-10-08T16:04:36.850726+0000","last_became_peered":"2023-10-08T16:04:36.850726+0000","last_unstale":"2023-10-08T16:04:36.860389+0000","last_undegraded":"2023-10-08T16:04:36.860389+0000","last_fullsized":"2023-10-08T16:04:36.860389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853164+0000","last_change":"2023-10-08T16:04:36.843832+0000","last_active":"2023-10-08T16:04:36.853164+0000","last_peered":"2023-10-08T16:04:36.853164+0000","last_clean":"2023-10-08T16:04:36.853164+0000","last_became_active":"2023-10-08T16:04:36.843552+0000","last_became_peered":"2023-10-08T16:04:36.843552+0000","last_unstale":"2023-10-08T16:04:36.853164+0000","last_undegraded":"2023-10-08T16:04:36.853164+0000","last_fullsized":"2023-10-08T16:04:36.853164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848791+0000","last_change":"2023-10-08T16:04:36.846568+0000","last_active":"2023-10-08T16:04:36.848791+0000","last_peered":"2023-10-08T16:04:36.848791+0000","last_clean":"2023-10-08T16:04:36.848791+0000","last_became_active":"2023-10-08T16:04:36.846344+0000","last_became_peered":"2023-10-08T16:04:36.846344+0000","last_unstale":"2023-10-08T16:04:36.848791+0000","last_undegraded":"2023-10-08T16:04:36.848791+0000","last_fullsized":"2023-10-08T16:04:36.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861183+0000","last_change":"2023-10-08T16:04:36.857746+0000","last_active":"2023-10-08T16:04:36.861183+0000","last_peered":"2023-10-08T16:04:36.861183+0000","last_clean":"2023-10-08T16:04:36.861183+0000","last_became_active":"2023-10-08T16:04:36.856907+0000","last_became_peered":"2023-10-08T16:04:36.856907+0000","last_unstale":"2023-10-08T16:04:36.861183+0000","last_undegraded":"2023-10-08T16:04:36.861183+0000","last_fullsized":"2023-10-08T16:04:36.861183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.846189+0000","last_change":"2023-10-08T16:04:36.843885+0000","last_active":"2023-10-08T16:04:36.846189+0000","last_peered":"2023-10-08T16:04:36.846189+0000","last_clean":"2023-10-08T16:04:36.846189+0000","last_became_active":"2023-10-08T16:04:36.843640+0000","last_became_peered":"2023-10-08T16:04:36.843640+0000","last_unstale":"2023-10-08T16:04:36.846189+0000","last_undegraded":"2023-10-08T16:04:36.846189+0000","last_fullsized":"2023-10-08T16:04:36.846189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855356+0000","last_change":"2023-10-08T16:04:36.851431+0000","last_active":"2023-10-08T16:04:36.855356+0000","last_peered":"2023-10-08T16:04:36.855356+0000","last_clean":"2023-10-08T16:04:36.855356+0000","last_became_active":"2023-10-08T16:04:36.851215+0000","last_became_peered":"2023-10-08T16:04:36.851215+0000","last_unstale":"2023-10-08T16:04:36.855356+0000","last_undegraded":"2023-10-08T16:04:36.855356+0000","last_fullsized":"2023-10-08T16:04:36.855356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840134+0000","last_change":"2023-10-08T16:04:36.837981+0000","last_active":"2023-10-08T16:04:36.840134+0000","last_peered":"2023-10-08T16:04:36.840134+0000","last_clean":"2023-10-08T16:04:36.840134+0000","last_became_active":"2023-10-08T16:04:36.837478+0000","last_became_peered":"2023-10-08T16:04:36.837478+0000","last_unstale":"2023-10-08T16:04:36.840134+0000","last_undegraded":"2023-10-08T16:04:36.840134+0000","last_fullsized":"2023-10-08T16:04:36.840134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855950+0000","last_change":"2023-10-08T16:04:36.850056+0000","last_active":"2023-10-08T16:04:36.855950+0000","last_peered":"2023-10-08T16:04:36.855950+0000","last_clean":"2023-10-08T16:04:36.855950+0000","last_became_active":"2023-10-08T16:04:36.849732+0000","last_became_peered":"2023-10-08T16:04:36.849732+0000","last_unstale":"2023-10-08T16:04:36.855950+0000","last_undegraded":"2023-10-08T16:04:36.855950+0000","last_fullsized":"2023-10-08T16:04:36.855950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843571+0000","last_change":"2023-10-08T16:04:36.842352+0000","last_active":"2023-10-08T16:04:36.843571+0000","last_peered":"2023-10-08T16:04:36.843571+0000","last_clean":"2023-10-08T16:04:36.843571+0000","last_became_active":"2023-10-08T16:04:36.842078+0000","last_became_peered":"2023-10-08T16:04:36.842078+0000","last_unstale":"2023-10-08T16:04:36.843571+0000","last_undegraded":"2023-10-08T16:04:36.843571+0000","last_fullsized":"2023-10-08T16:04:36.843571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861173+0000","last_change":"2023-10-08T16:04:36.858170+0000","last_active":"2023-10-08T16:04:36.861173+0000","last_peered":"2023-10-08T16:04:36.861173+0000","last_clean":"2023-10-08T16:04:36.861173+0000","last_became_active":"2023-10-08T16:04:36.857822+0000","last_became_peered":"2023-10-08T16:04:36.857822+0000","last_unstale":"2023-10-08T16:04:36.861173+0000","last_undegraded":"2023-10-08T16:04:36.861173+0000","last_fullsized":"2023-10-08T16:04:36.861173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855132+0000","last_change":"2023-10-08T16:04:36.847659+0000","last_active":"2023-10-08T16:04:36.855132+0000","last_peered":"2023-10-08T16:04:36.855132+0000","last_clean":"2023-10-08T16:04:36.855132+0000","last_became_active":"2023-10-08T16:04:36.847337+0000","last_became_peered":"2023-10-08T16:04:36.847337+0000","last_unstale":"2023-10-08T16:04:36.855132+0000","last_undegraded":"2023-10-08T16:04:36.855132+0000","last_fullsized":"2023-10-08T16:04:36.855132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855842+0000","last_change":"2023-10-08T16:04:36.850206+0000","last_active":"2023-10-08T16:04:36.855842+0000","last_peered":"2023-10-08T16:04:36.855842+0000","last_clean":"2023-10-08T16:04:36.855842+0000","last_became_active":"2023-10-08T16:04:36.849892+0000","last_became_peered":"2023-10-08T16:04:36.849892+0000","last_unstale":"2023-10-08T16:04:36.855842+0000","last_undegraded":"2023-10-08T16:04:36.855842+0000","last_fullsized":"2023-10-08T16:04:36.855842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848345+0000","last_change":"2023-10-08T16:04:36.846291+0000","last_active":"2023-10-08T16:04:36.848345+0000","last_peered":"2023-10-08T16:04:36.848345+0000","last_clean":"2023-10-08T16:04:36.848345+0000","last_became_active":"2023-10-08T16:04:36.846030+0000","last_became_peered":"2023-10-08T16:04:36.846030+0000","last_unstale":"2023-10-08T16:04:36.848345+0000","last_undegraded":"2023-10-08T16:04:36.848345+0000","last_fullsized":"2023-10-08T16:04:36.848345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860629+0000","last_change":"2023-10-08T16:04:36.852345+0000","last_active":"2023-10-08T16:04:36.860629+0000","last_peered":"2023-10-08T16:04:36.860629+0000","last_clean":"2023-10-08T16:04:36.860629+0000","last_became_active":"2023-10-08T16:04:36.852087+0000","last_became_peered":"2023-10-08T16:04:36.852087+0000","last_unstale":"2023-10-08T16:04:36.860629+0000","last_undegraded":"2023-10-08T16:04:36.860629+0000","last_fullsized":"2023-10-08T16:04:36.860629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":666235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":666291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":75439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:04:41.278 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:04:41.278 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T16:04:41.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-08T16:04:41.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T16:04:41.278 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-08T16:04:41.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 Processor -- start 2023-10-08T16:04:41.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 -- start start 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2e54069560 0x7f2e540699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5415a160 con 0x7f2e54159c60 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e54069f20 con 0x7f2e54069560 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.427+0000 7f2e591a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e540d13a0 con 0x7f2e54159890 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2e54159c60 0x7f2e5415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54710/0 (socket says 172.21.15.170:54710) 2023-10-08T16:04:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5259c700 1 -- 172.21.15.170:0/2485303233 learned_addr learned my addr 172.21.15.170:0/2485303233 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 --2- 172.21.15.170:0/2485303233 >> v2:172.21.15.191:3300/0 conn(0x7f2e54069560 0x7f2e540699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e52d9d700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950521920 0 0) 0x7f2e5415a160 con 0x7f2e54159c60 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e40003570 con 0x7f2e54159c60 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958399201 0 0) 0x7f2e40003570 con 0x7f2e54159c60 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 -- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 msgr2=0x7f2e5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 -- 172.21.15.170:0/2485303233 >> v1:172.21.15.170:6789/0 conn(0x7f2e54159c60 legacy=0x7f2e5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 -- 172.21.15.170:0/2485303233 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5415e140 con 0x7f2e54069560 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e52d9d700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:41.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e5359e700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54069560 0x7f2e540699e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e440098e0 tx=0x7f2e440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6af328cb5863076 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c001470 con 0x7f2e54069560 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e44011420 con 0x7f2e54069560 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.428+0000 7f2e51d9b700 1 -- 172.21.15.170:0/2485303233 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e440117c0 con 0x7f2e54069560 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 -- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54069560 msgr2=0x7f2e540699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54069560 0x7f2e540699e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2e440098e0 tx=0x7f2e440091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 -- 172.21.15.170:0/2485303233 shutdown_connections 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54069560 0x7f2e540699e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 --2- 172.21.15.170:0/2485303233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 -- 172.21.15.170:0/2485303233 >> 172.21.15.170:0/2485303233 conn(0x7f2e54068e20 msgr2=0x7f2e540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 -- 172.21.15.170:0/2485303233 shutdown_connections 2023-10-08T16:04:41.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.429+0000 7f2e591a6700 1 -- 172.21.15.170:0/2485303233 wait complete. 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.430+0000 7f2e591a6700 1 Processor -- start 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.430+0000 7f2e591a6700 1 -- start start 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5416a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54159c60 0x7f2e540b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5415ee50 con 0x7f2e54069560 2023-10-08T16:04:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5416c840 con 0x7f2e54159c60 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e591a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5415e520 con 0x7f2e54159890 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46224/0 (socket says 172.21.15.170:46224) 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 -- 172.21.15.170:0/3686162364 learned_addr learned my addr 172.21.15.170:0/3686162364 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e5359e700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54159c60 0x7f2e540b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e5259c700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5416a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 -- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 msgr2=0x7f2e5416a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5416a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 -- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54159c60 msgr2=0x7f2e540b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54159c60 0x7f2e540b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 -- 172.21.15.170:0/3686162364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c009a20 con 0x7f2e54069560 2023-10-08T16:04:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e52d9d700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c170 tx=0x7f2e4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.431+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00ce10 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.432+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c00f040 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.432+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e44009420 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.432+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c014690 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.432+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5415eae0 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.433+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5404c3c0 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.433+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e4c014a00 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.433+0000 7f2e4affd700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 0x7f2e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.433+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2e4c085410 con 0x7f2e54069560 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.433+0000 7f2e5259c700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 0x7f2e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.434+0000 7f2e5259c700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 0x7f2e340693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001620 tx=0x7f2e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.437+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e4c052680 con 0x7f2e54069560 2023-10-08T16:04:41.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.675+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2e5415f130 con 0x7f2e54069560 2023-10-08T16:04:41.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.678+0000 7f2e4affd700 1 -- 172.21.15.170:0/3686162364 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163704 (secure 0 0 0) 0x7f2e4c014cb0 con 0x7f2e54069560 2023-10-08T16:04:41.681 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 msgr2=0x7f2e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 0x7f2e340693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c001620 tx=0x7f2e3c009040 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 msgr2=0x7f2e5416c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00c170 tx=0x7f2e4c00b040 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 shutdown_connections 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2e54159c60 0x7f2e540b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2e34066f30 0x7f2e340693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e54069560 0x7f2e5416c300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 --2- 172.21.15.170:0/3686162364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2e54159890 0x7f2e5416a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 >> 172.21.15.170:0/3686162364 conn(0x7f2e54068e20 msgr2=0x7f2e54157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 shutdown_connections 2023-10-08T16:04:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.681+0000 7f2e591a6700 1 -- 172.21.15.170:0/3686162364 wait complete. 2023-10-08T16:04:41.694 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":90636},{"type":"v1","addr":"172.21.15.170:6833","nonce":90636}]},"active_addr":"172.21.15.170:6833/90636","active_change":"2023-10-08T16:04:24.217579+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3709498577}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":389076363}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":392794031}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.170:0","nonce":3384050419}]}]} 2023-10-08T16:04:41.699 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-08T16:04:41.699 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-08T16:04:41.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:41.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.811+0000 7fc2a694c700 1 Processor -- start 2023-10-08T16:04:41.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.811+0000 7fc2a694c700 1 -- start start 2023-10-08T16:04:41.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a694c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a694c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 0x7fc2a00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a694c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a015a160 con 0x7fc2a0159c60 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a694c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a0069f20 con 0x7fc2a0069560 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a694c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00d13a0 con 0x7fc2a0159890 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a4ee9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 0x7fc2a00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a4ee9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 0x7fc2a00699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38304/0 (socket says 172.21.15.170:38304) 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35242/0 (socket says 172.21.15.170:35242) 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.812+0000 7fc2a4ee9700 1 -- 172.21.15.170:0/3600019237 learned_addr learned my addr 172.21.15.170:0/3600019237 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29ffff700 1 -- 172.21.15.170:0/3600019237 >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 msgr2=0x7fc2a00699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29ffff700 1 --2- 172.21.15.170:0/3600019237 >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 0x7fc2a00699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29ffff700 1 -- 172.21.15.170:0/3600019237 >> v1:172.21.15.191:6789/0 conn(0x7fc2a0159c60 legacy=0x7fc2a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29ffff700 1 -- 172.21.15.170:0/3600019237 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a015e140 con 0x7fc2a0159890 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29ffff700 1 --2- 172.21.15.170:0/3600019237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2900098e0 tx=0x7fc2900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30594337273347ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29effd700 1 -- 172.21.15.170:0/3600019237 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000b640 con 0x7fc2a0159890 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29effd700 1 -- 172.21.15.170:0/3600019237 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29000b7a0 con 0x7fc2a0159890 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.813+0000 7fc29effd700 1 -- 172.21.15.170:0/3600019237 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000bb40 con 0x7fc2a0159890 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 -- 172.21.15.170:0/3600019237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 msgr2=0x7fc2a015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 --2- 172.21.15.170:0/3600019237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc2900098e0 tx=0x7fc2900091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 -- 172.21.15.170:0/3600019237 shutdown_connections 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 --2- 172.21.15.170:0/3600019237 >> v2:172.21.15.191:3300/0 conn(0x7fc2a0069560 0x7fc2a00699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 --2- 172.21.15.170:0/3600019237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159890 0x7fc2a015abd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 -- 172.21.15.170:0/3600019237 >> 172.21.15.170:0/3600019237 conn(0x7fc2a0068e20 msgr2=0x7fc2a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:41.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 -- 172.21.15.170:0/3600019237 shutdown_connections 2023-10-08T16:04:41.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.814+0000 7fc2a694c700 1 -- 172.21.15.170:0/3600019237 wait complete. 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.815+0000 7fc2a694c700 1 Processor -- start 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.815+0000 7fc2a694c700 1 -- start start 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.815+0000 7fc2a694c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2a0159890 0x7fc2a014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159c60 0x7fc2a014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00b3620 con 0x7fc2a0069560 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00b4480 con 0x7fc2a0159890 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00b3870 con 0x7fc2a0159c60 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46226/0 (socket says 172.21.15.170:46226) 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 -- 172.21.15.170:0/461465725 learned_addr learned my addr 172.21.15.170:0/461465725 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29f7fe700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2a0159890 0x7fc2a014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a4ee9700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159c60 0x7fc2a014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 -- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159c60 msgr2=0x7fc2a014d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159c60 0x7fc2a014d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 -- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2a0159890 msgr2=0x7fc2a014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2a0159890 0x7fc2a014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 -- 172.21.15.170:0/461465725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc290009420 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc29ffff700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc290009f80 tx=0x7fc290017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29000bb20 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc290011420 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a00b3b50 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290011800 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.816+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a00b4110 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.817+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a004c3c0 con 0x7fc2a0069560 2023-10-08T16:04:41.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.818+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc290011c30 con 0x7fc2a0069560 2023-10-08T16:04:41.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.818+0000 7fc287fff700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 0x7fc28c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:41.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.818+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2900860a0 con 0x7fc2a0069560 2023-10-08T16:04:41.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.818+0000 7fc29f7fe700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 0x7fc28c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:41.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.818+0000 7fc29f7fe700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 0x7fc28c0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0064f70 tx=0x7fc294009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:41.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:41.822+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc290053310 con 0x7fc2a0069560 2023-10-08T16:04:42.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.007+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a0089560 con 0x7fc2a0069560 2023-10-08T16:04:42.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.008+0000 7fc287fff700 1 -- 172.21.15.170:0/461465725 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc290056930 con 0x7fc2a0069560 2023-10-08T16:04:42.009 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:42.009 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 msgr2=0x7fc28c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 0x7fc28c0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0064f70 tx=0x7fc294009040 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 msgr2=0x7fc2a014c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc290009f80 tx=0x7fc290017000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 shutdown_connections 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc2a0159c60 0x7fc2a014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc28c066f30 0x7fc28c0693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc2a0069560 0x7fc2a014c4e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 --2- 172.21.15.170:0/461465725 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc2a0159890 0x7fc2a014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 >> 172.21.15.170:0/461465725 conn(0x7fc2a0068e20 msgr2=0x7fc2a015c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 shutdown_connections 2023-10-08T16:04:42.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.010+0000 7fc2a694c700 1 -- 172.21.15.170:0/461465725 wait complete. 2023-10-08T16:04:42.023 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-08T16:04:42.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.136+0000 7efd90e46700 1 Processor -- start 2023-10-08T16:04:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.136+0000 7efd90e46700 1 -- start start 2023-10-08T16:04:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd90e46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd8c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd90e46700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c15a160 con 0x7efd8c159890 2023-10-08T16:04:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd90e46700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0d13a0 con 0x7efd8c15db10 2023-10-08T16:04:42.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd90e46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c15e140 con 0x7efd8c159c60 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd8a59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efd8c159890 0x7efd8c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54724/0 (socket says 172.21.15.170:54724) 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd8a59c700 1 -- 172.21.15.170:0/686288776 learned_addr learned my addr 172.21.15.170:0/686288776 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd81d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd8c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343480063 0 0) 0x7efd8c15a160 con 0x7efd8c159890 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd78003570 con 0x7efd8c159890 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212564210 0 0) 0x7efd8c0d13a0 con 0x7efd8c15db10 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd8c15a160 con 0x7efd8c15db10 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.137+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 371829584 0 0) 0x7efd78003570 con 0x7efd8c159890 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 msgr2=0x7efd8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd8c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 >> v1:172.21.15.191:6789/0 conn(0x7efd8c15db10 legacy=0x7efd8c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c15a3d0 con 0x7efd8c159890 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7efd7c001470 con 0x7efd8c159890 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd81d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd8c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 >> v1:172.21.15.170:6789/0 conn(0x7efd8c159890 legacy=0x7efd8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd78004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd78004810 0x7efd78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c15e140 con 0x7efd78003c20 2023-10-08T16:04:42.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7800bb10 con 0x7efd78004810 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7800bcd0 con 0x7efd8c159c60 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd78004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd81d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd78004810 0x7efd78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8ad9d700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 -- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 msgr2=0x7efd780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c159c60 0x7efd780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 -- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd78004810 msgr2=0x7efd78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd78004810 0x7efd78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 -- 172.21.15.170:0/686288776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009420 con 0x7efd78003c20 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd81d9b700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd78004810 0x7efd78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:42.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd8a59c700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd78004010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c2d0 tx=0x7efd7c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20561d4a69e1f9db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c00cba0 con 0x7efd78003c20 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c00cd00 con 0x7efd78003c20 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.138+0000 7efd89d9b700 1 -- 172.21.15.170:0/686288776 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c005b70 con 0x7efd78003c20 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 -- 172.21.15.170:0/686288776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 msgr2=0x7efd78004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 --2- 172.21.15.170:0/686288776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd78004010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd7c00c2d0 tx=0x7efd7c001e10 comp rx=0 tx=0).stop 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd8ad9d700 1 -- 172.21.15.170:0/686288776 reap_dead start 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 -- 172.21.15.170:0/686288776 shutdown_connections 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 -- 172.21.15.170:0/686288776 >> 172.21.15.170:0/686288776 conn(0x7efd8c068e20 msgr2=0x7efd8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 -- 172.21.15.170:0/686288776 shutdown_connections 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.139+0000 7efd90e46700 1 -- 172.21.15.170:0/686288776 wait complete. 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.140+0000 7efd90e46700 1 Processor -- start 2023-10-08T16:04:42.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.140+0000 7efd90e46700 1 -- start start 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd8c0ce020 0x7efd8c14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 0x7efd8c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c153100 con 0x7efd78003c20 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0ce490 con 0x7efd8c0ce020 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0b4f90 con 0x7efd8c152720 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46242/0 (socket says 172.21.15.170:46242) 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 -- 172.21.15.170:0/851813234 learned_addr learned my addr 172.21.15.170:0/851813234 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd81d9b700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd8c0ce020 0x7efd8c14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8ad9d700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 0x7efd8c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 -- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 msgr2=0x7efd8c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 0x7efd8c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 -- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd8c0ce020 msgr2=0x7efd8c14a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd8c0ce020 0x7efd8c14a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 -- 172.21.15.170:0/851813234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0b5270 con 0x7efd78003c20 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8ad9d700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 0x7efd8c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd8a59c700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd7c007c90 tx=0x7efd7c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0070d0 con 0x7efd78003c20 2023-10-08T16:04:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c007230 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c0b5550 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0075f0 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.141+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c0b5b10 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.142+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c0b5cd0 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.143+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd7c019030 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.143+0000 7efd82ffd700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.143+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd7c085900 con 0x7efd78003c20 2023-10-08T16:04:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.143+0000 7efd81d9b700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.143+0000 7efd81d9b700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 0x7efd6c0693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd740093f0 tx=0x7efd74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.147+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd7c052b70 con 0x7efd78003c20 2023-10-08T16:04:42.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.331+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd8c04c3c0 con 0x7efd78003c20 2023-10-08T16:04:42.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.332+0000 7efd82ffd700 1 -- 172.21.15.170:0/851813234 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7efd7c056190 con 0x7efd78003c20 2023-10-08T16:04:42.333 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:42.333 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:42.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 msgr2=0x7efd6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 0x7efd6c0693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd740093f0 tx=0x7efd74008040 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 msgr2=0x7efd8c0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd7c007c90 tx=0x7efd7c015000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 shutdown_connections 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd8c152720 0x7efd8c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd6c066f30 0x7efd6c0693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd78003c20 0x7efd8c0b48b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 --2- 172.21.15.170:0/851813234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd8c0ce020 0x7efd8c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 >> 172.21.15.170:0/851813234 conn(0x7efd8c068e20 msgr2=0x7efd8c14d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 shutdown_connections 2023-10-08T16:04:42.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.334+0000 7efd90e46700 1 -- 172.21.15.170:0/851813234 wait complete. 2023-10-08T16:04:42.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-08T16:04:42.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-08T16:04:42.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-08T16:04:42.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-08T16:04:42.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-08T16:04:42.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-08T16:04:42.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-08T16:04:42.350 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-08T16:04:42.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.546+0000 7faa3b319700 1 Processor -- start 2023-10-08T16:04:42.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.546+0000 7faa3b319700 1 -- start start 2023-10-08T16:04:42.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.546+0000 7f82d9e98700 1 Processor -- start 2023-10-08T16:04:42.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7f82d9e98700 1 -- start start 2023-10-08T16:04:42.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7f5eee9b9700 1 Processor -- start 2023-10-08T16:04:42.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7f5eee9b9700 1 -- start start 2023-10-08T16:04:42.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d9e98700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f82d415d230 0x7f82d415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d9e98700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40b2980 con 0x7f82d4159420 2023-10-08T16:04:42.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d9e98700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d415db70 con 0x7f82d41597f0 2023-10-08T16:04:42.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d9e98700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d40b11d0 con 0x7f82d415d230 2023-10-08T16:04:42.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7fdec3772700 1 Processor -- start 2023-10-08T16:04:42.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7f1572404700 1 Processor -- start 2023-10-08T16:04:42.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7faa3b319700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7faa3b319700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa34069d30 con 0x7faa3406ab20 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7faa3b319700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa340c8110 con 0x7faa340cb5a0 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7faa3b319700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa34063940 con 0x7faa34156fb0 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35308/0 (socket says 172.21.15.170:35308) 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 -- 172.21.15.170:0/2122863914 learned_addr learned my addr 172.21.15.170:0/2122863914 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 -- 172.21.15.170:0/2122863914 >> v1:172.21.15.191:6789/0 conn(0x7faa340cb5a0 legacy=0x7faa340c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7fdec3772700 1 -- start start 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdebc068aa0 0x7fdebc068f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc0aceb0 con 0x7fdebc068aa0 2023-10-08T16:04:42.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc069450 con 0x7fdebc156050 2023-10-08T16:04:42.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7fdec3772700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0b2490 con 0x7fdebc0cba20 2023-10-08T16:04:42.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 -- 172.21.15.170:0/2122863914 >> v1:172.21.15.170:6789/0 conn(0x7faa3406ab20 legacy=0x7faa34063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 -- 172.21.15.170:0/2122863914 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa341568e0 con 0x7faa34156fb0 2023-10-08T16:04:42.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5ee8159b10 0x7f5ee80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5ee815d730 0x7f5ee815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee80c74b0 con 0x7f5ee8159b10 2023-10-08T16:04:42.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee815e0e0 con 0x7f5ee815d730 2023-10-08T16:04:42.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f5eee9b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80c8700 con 0x7f5ee8159ee0 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d3fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f82d41597f0 0x7f82d40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55960/0 (socket says 172.21.15.170:55960) 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f82d3fff700 1 -- 172.21.15.170:0/957075315 learned_addr learned my addr 172.21.15.170:0/957075315 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.547+0000 7f1572404700 1 -- start start 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f1572404700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f156c0afc40 0x7f156c0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f1572404700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f156c0b0010 0x7f156c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f1572404700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b0750 con 0x7f156c0b2140 2023-10-08T16:04:42.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f1572404700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0aca70 con 0x7f156c0b0010 2023-10-08T16:04:42.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7f1572404700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0acd50 con 0x7f156c0afc40 2023-10-08T16:04:42.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.548+0000 7faa390b5700 1 --2- 172.21.15.170:0/2122863914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa300099e0 tx=0x7faa30009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5aa6d2bf9247e60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35310/0 (socket says 172.21.15.170:35310) 2023-10-08T16:04:42.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 -- 172.21.15.170:0/3944674793 learned_addr learned my addr 172.21.15.170:0/3944674793 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5ee77fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5ee815d730 0x7f5ee815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec1d0f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f156c0afc40 0x7f156c0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec1d0f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38350/0 (socket says 172.21.15.170:38350) 2023-10-08T16:04:42.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec1d0f700 1 -- 172.21.15.170:0/3629547814 learned_addr learned my addr 172.21.15.170:0/3629547814 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f156c0afc40 0x7f156c0b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38352/0 (socket says 172.21.15.170:38352) 2023-10-08T16:04:42.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156b7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f156c0b2140 0x7f156c15d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55962/0 (socket says 172.21.15.170:55962) 2023-10-08T16:04:42.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 -- 172.21.15.170:0/4026439348 learned_addr learned my addr 172.21.15.170:0/4026439348 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7faa2bfff700 1 -- 172.21.15.170:0/2122863914 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000b7e0 con 0x7faa34156fb0 2023-10-08T16:04:42.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7faa2bfff700 1 -- 172.21.15.170:0/2122863914 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3000b940 con 0x7faa34156fb0 2023-10-08T16:04:42.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7faa2bfff700 1 -- 172.21.15.170:0/2122863914 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3000bce0 con 0x7faa34156fb0 2023-10-08T16:04:42.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 -- 172.21.15.170:0/3944674793 >> v2:172.21.15.191:3300/0 conn(0x7f5ee815d730 msgr2=0x7f5ee815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 --2- 172.21.15.170:0/3944674793 >> v2:172.21.15.191:3300/0 conn(0x7f5ee815d730 0x7f5ee815dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 -- 172.21.15.170:0/3944674793 >> v2:172.21.15.170:3300/0 conn(0x7f5ee8159b10 msgr2=0x7f5ee80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 --2- 172.21.15.170:0/3944674793 >> v2:172.21.15.170:3300/0 conn(0x7f5ee8159b10 0x7f5ee80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 -- 172.21.15.170:0/3944674793 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80c9070 con 0x7f5ee8159ee0 2023-10-08T16:04:42.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 -- 172.21.15.170:0/3629547814 >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 msgr2=0x7fdebc1564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 --2- 172.21.15.170:0/3629547814 >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 -- 172.21.15.170:0/3629547814 >> v2:172.21.15.170:3300/0 conn(0x7fdebc068aa0 msgr2=0x7fdebc068f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 --2- 172.21.15.170:0/3629547814 >> v2:172.21.15.170:3300/0 conn(0x7fdebc068aa0 0x7fdebc068f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 -- 172.21.15.170:0/3629547814 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0ac4e0 con 0x7fdebc0cba20 2023-10-08T16:04:42.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 -- 172.21.15.170:0/4026439348 >> v2:172.21.15.170:3300/0 conn(0x7f156c0b0010 msgr2=0x7f156c15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 --2- 172.21.15.170:0/4026439348 >> v2:172.21.15.170:3300/0 conn(0x7f156c0b0010 0x7f156c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 -- 172.21.15.170:0/4026439348 >> v1:172.21.15.191:6789/0 conn(0x7f156c0b2140 legacy=0x7f156c15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156affd700 1 -- 172.21.15.170:0/4026439348 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307718194 0 0) 0x7f156c0b0750 con 0x7f156c0b2140 2023-10-08T16:04:42.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 -- 172.21.15.170:0/4026439348 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0ad030 con 0x7f156c0afc40 2023-10-08T16:04:42.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec1d0f700 1 --2- 172.21.15.170:0/3629547814 >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:42.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f5eecf56700 1 --2- 172.21.15.170:0/3944674793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806cce0 tx=0x7f5ee0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a982dde8f767abf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7fdec150e700 1 --2- 172.21.15.170:0/3629547814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc0ac970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0afc30 tx=0x7fdeac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63fda3b3b217db32 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.549+0000 7f156bfff700 1 --2- 172.21.15.170:0/4026439348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0afc40 0x7f156c0b1c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f156c063d40 tx=0x7f155c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6693caf54795bb48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f5ee6ffd700 1 -- 172.21.15.170:0/3944674793 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000e070 con 0x7f5ee8159ee0 2023-10-08T16:04:42.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f5ee6ffd700 1 -- 172.21.15.170:0/3944674793 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000bab0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f5ee6ffd700 1 -- 172.21.15.170:0/3944674793 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00092e0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7faa3b319700 1 -- 172.21.15.170:0/2122863914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 msgr2=0x7faa3406a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7faa3b319700 1 --2- 172.21.15.170:0/2122863914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faa300099e0 tx=0x7faa30009250 comp rx=0 tx=0).stop 2023-10-08T16:04:42.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f82d9697700 1 --2- 172.21.15.170:0/957075315 >> v2:172.21.15.170:3300/0 conn(0x7f82d415d230 0x7f82d415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7fdeb3fff700 1 -- 172.21.15.170:0/3629547814 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac010600 con 0x7fdebc0cba20 2023-10-08T16:04:42.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7fdeb3fff700 1 -- 172.21.15.170:0/3629547814 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac010760 con 0x7fdebc0cba20 2023-10-08T16:04:42.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7fdeb3fff700 1 -- 172.21.15.170:0/3629547814 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac010b00 con 0x7fdebc0cba20 2023-10-08T16:04:42.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f156affd700 1 -- 172.21.15.170:0/4026439348 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00b860 con 0x7f156c0afc40 2023-10-08T16:04:42.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f156affd700 1 -- 172.21.15.170:0/4026439348 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00b9c0 con 0x7f156c0afc40 2023-10-08T16:04:42.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f156affd700 1 -- 172.21.15.170:0/4026439348 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00bd60 con 0x7f156c0afc40 2023-10-08T16:04:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f82d9697700 1 -- 172.21.15.170:0/957075315 >> v1:172.21.15.191:6789/0 conn(0x7f82d41597f0 legacy=0x7f82d40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f82d9697700 1 -- 172.21.15.170:0/957075315 >> v1:172.21.15.170:6789/0 conn(0x7f82d4159420 legacy=0x7f82d40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f82d9697700 1 -- 172.21.15.170:0/957075315 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40b14b0 con 0x7f82d415d230 2023-10-08T16:04:42.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.550+0000 7f82d9697700 1 --2- 172.21.15.170:0/957075315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d415d230 0x7f82d415d630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009de0 tx=0x7f82cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbc8c8f006f3308 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 -- 172.21.15.170:0/3944674793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 msgr2=0x7f5ee815a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/3944674793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ee806cce0 tx=0x7f5ee0009840 comp rx=0 tx=0).stop 2023-10-08T16:04:42.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7fdec3772700 1 -- 172.21.15.170:0/3629547814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 msgr2=0x7fdebc0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7fdec3772700 1 --2- 172.21.15.170:0/3629547814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc0ac970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0afc30 tx=0x7fdeac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7faa3b319700 1 -- 172.21.15.170:0/2122863914 shutdown_connections 2023-10-08T16:04:42.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7faa3b319700 1 --2- 172.21.15.170:0/2122863914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa3406a5e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7faa3b319700 1 -- 172.21.15.170:0/2122863914 >> 172.21.15.170:0/2122863914 conn(0x7faa340c9870 msgr2=0x7faa340c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7faa3b319700 1 -- 172.21.15.170:0/2122863914 shutdown_connections 2023-10-08T16:04:42.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7faa3b319700 1 -- 172.21.15.170:0/2122863914 wait complete. 2023-10-08T16:04:42.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 -- 172.21.15.170:0/4026439348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0afc40 msgr2=0x7f156c0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 --2- 172.21.15.170:0/4026439348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0afc40 0x7f156c0b1c00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f156c063d40 tx=0x7f155c009250 comp rx=0 tx=0).stop 2023-10-08T16:04:42.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 -- 172.21.15.170:0/4026439348 shutdown_connections 2023-10-08T16:04:42.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 --2- 172.21.15.170:0/4026439348 >> v2:172.21.15.170:3300/0 conn(0x7f156c0b0010 0x7f156c15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 --2- 172.21.15.170:0/4026439348 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0afc40 0x7f156c0b1c00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f1572404700 1 -- 172.21.15.170:0/4026439348 >> 172.21.15.170:0/4026439348 conn(0x7f156c06a320 msgr2=0x7f156c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 -- 172.21.15.170:0/3944674793 shutdown_connections 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/3944674793 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159ee0 0x7f5ee815a340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/3944674793 >> v2:172.21.15.170:3300/0 conn(0x7f5ee8159b10 0x7f5ee80cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/3944674793 >> v2:172.21.15.191:3300/0 conn(0x7f5ee815d730 0x7f5ee815dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 -- 172.21.15.170:0/3944674793 >> 172.21.15.170:0/3944674793 conn(0x7f5ee80c9870 msgr2=0x7f5ee80c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 -- 172.21.15.170:0/3944674793 shutdown_connections 2023-10-08T16:04:42.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7f5eee9b9700 1 -- 172.21.15.170:0/3944674793 wait complete. 2023-10-08T16:04:42.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7f82d37fe700 1 -- 172.21.15.170:0/957075315 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00e070 con 0x7f82d415d230 2023-10-08T16:04:42.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7f82d37fe700 1 -- 172.21.15.170:0/957075315 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc00bbb0 con 0x7f82d415d230 2023-10-08T16:04:42.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7f82d37fe700 1 -- 172.21.15.170:0/957075315 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc012430 con 0x7f82d415d230 2023-10-08T16:04:42.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7f5eee9b9700 1 Processor -- start 2023-10-08T16:04:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7fe0bf2e9700 1 Processor -- start 2023-10-08T16:04:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.551+0000 7fe0bf2e9700 1 -- start start 2023-10-08T16:04:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7faa3b319700 1 Processor -- start 2023-10-08T16:04:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f5eee9b9700 1 -- start start 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 -- 172.21.15.170:0/957075315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d415d230 msgr2=0x7f82d415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 --2- 172.21.15.170:0/957075315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d415d230 0x7f82d415d630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009de0 tx=0x7f82cc009780 comp rx=0 tx=0).stop 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7fdec3772700 1 -- 172.21.15.170:0/3629547814 shutdown_connections 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7fdec3772700 1 --2- 172.21.15.170:0/3629547814 >> v2:172.21.15.191:3300/0 conn(0x7fdebc156050 0x7fdebc1564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7fdec3772700 1 --2- 172.21.15.170:0/3629547814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc0ac970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7fdec3772700 1 --2- 172.21.15.170:0/3629547814 >> v2:172.21.15.170:3300/0 conn(0x7fdebc068aa0 0x7fdebc068f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.552+0000 7fdec3772700 1 -- 172.21.15.170:0/3629547814 >> 172.21.15.170:0/3629547814 conn(0x7fdebc0cd000 msgr2=0x7fdebc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7faa3b319700 1 -- start start 2023-10-08T16:04:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 -- 172.21.15.170:0/957075315 shutdown_connections 2023-10-08T16:04:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 --2- 172.21.15.170:0/957075315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d415d230 0x7f82d415d630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 -- 172.21.15.170:0/957075315 >> 172.21.15.170:0/957075315 conn(0x7f82d40d0660 msgr2=0x7f82d40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 -- 172.21.15.170:0/957075315 shutdown_connections 2023-10-08T16:04:42.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f1572404700 1 -- 172.21.15.170:0/4026439348 shutdown_connections 2023-10-08T16:04:42.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f1572404700 1 -- 172.21.15.170:0/4026439348 wait complete. 2023-10-08T16:04:42.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f82d9e98700 1 -- 172.21.15.170:0/957075315 wait complete. 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fdec3772700 1 -- 172.21.15.170:0/3629547814 shutdown_connections 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bf2e9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bf2e9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe0b80875d0 0x7fe0b8087a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bf2e9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b80acdf0 con 0x7fe0b8068aa0 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bf2e9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b80ac4c0 con 0x7fe0b80b1c50 2023-10-08T16:04:42.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bf2e9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b80ac7a0 con 0x7fe0b80875d0 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bd085700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bd085700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46292/0 (socket says 172.21.15.170:46292) 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bd085700 1 -- 172.21.15.170:0/2561437637 learned_addr learned my addr 172.21.15.170:0/2561437637 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fe0bc884700 1 --2- 172.21.15.170:0/2561437637 >> v2:172.21.15.191:3300/0 conn(0x7fe0b80875d0 0x7fe0b8087a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159b10 0x7f5ee8154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 0x7f5ee815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee815d730 0x7f5ee80b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee815b4a0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee815b780 con 0x7f5ee815d730 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5eee9b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee815ba60 con 0x7f5ee8159b10 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 0x7f5ee815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 0x7f5ee815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46304/0 (socket says 172.21.15.170:46304) 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 -- 172.21.15.170:0/1781779751 learned_addr learned my addr 172.21.15.170:0/1781779751 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee7fff700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159b10 0x7f5ee8154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 -- 172.21.15.170:0/1781779751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159b10 msgr2=0x7f5ee8154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ee8159b10 0x7f5ee8154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 -- 172.21.15.170:0/1781779751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee815d730 msgr2=0x7f5ee80b1ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee815d730 0x7f5ee80b1ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7f5ee77fe700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0009ae0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7fe0bd085700 1 -- 172.21.15.170:0/2561437637 >> v2:172.21.15.191:3300/0 conn(0x7fe0b80875d0 msgr2=0x7fe0b8087a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7fe0bd085700 1 --2- 172.21.15.170:0/2561437637 >> v2:172.21.15.191:3300/0 conn(0x7fe0b80875d0 0x7fe0b8087a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7fe0bd085700 1 -- 172.21.15.170:0/2561437637 >> v1:172.21.15.170:6789/0 conn(0x7fe0b8068aa0 legacy=0x7fe0b816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7fe0bd085700 1 -- 172.21.15.170:0/2561437637 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80aca80 con 0x7fe0b80b1c50 2023-10-08T16:04:42.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7fe0bd085700 1 --2- 172.21.15.170:0/2561437637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8087f90 tx=0x7fe0b400fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d48e6a6651a9161 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7fe0affff700 1 -- 172.21.15.170:0/2561437637 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4015700 con 0x7fe0b80b1c50 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7fe0affff700 1 -- 172.21.15.170:0/2561437637 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4015860 con 0x7fe0b80b1c50 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7fe0affff700 1 -- 172.21.15.170:0/2561437637 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4015c00 con 0x7fe0b80b1c50 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f82d9e98700 1 Processor -- start 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f82d9e98700 1 -- start start 2023-10-08T16:04:42.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f1572404700 1 Processor -- start 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f1572404700 1 -- start start 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 0x7faa340b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa340cb5a0 0x7faa341533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa34145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa340b2b70 con 0x7faa3406ab20 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa34146940 con 0x7faa340cb5a0 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa3b319700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa34146be0 con 0x7faa34156fb0 2023-10-08T16:04:42.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa390b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 0x7faa340b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa390b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 0x7faa340b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46320/0 (socket says 172.21.15.170:46320) 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa390b5700 1 -- 172.21.15.170:0/816257408 learned_addr learned my addr 172.21.15.170:0/816257408 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa388b4700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa340cb5a0 0x7faa341533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.554+0000 7faa398b6700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa34145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 -- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 msgr2=0x7faa34145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa34156fb0 0x7faa34145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 -- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa340cb5a0 msgr2=0x7faa341533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa340cb5a0 0x7faa341533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa300094f0 con 0x7faa3406ab20 2023-10-08T16:04:42.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa390b5700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 0x7faa340b1cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa30003080 tx=0x7faa30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f0f2721f700 1 Processor -- start 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7f0f2721f700 1 -- start start 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f2721f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0f200cba20 0x7f0f200ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f2721f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0f20156760 0x7f0f20068ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f2721f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f200b2490 con 0x7f0f20068aa0 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f2721f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f200aceb0 con 0x7f0f200cba20 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f2721f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f200ac4e0 con 0x7f0f20156760 2023-10-08T16:04:42.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f1ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0f20068aa0 0x7f0f20156050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54766/0 (socket says 172.21.15.170:54766) 2023-10-08T16:04:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f0f1ffff700 1 -- 172.21.15.170:0/2931419786 learned_addr learned my addr 172.21.15.170:0/2931419786 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.553+0000 7fdec3772700 1 -- 172.21.15.170:0/3629547814 wait complete. 2023-10-08T16:04:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7fdec3772700 1 Processor -- start 2023-10-08T16:04:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7fdec3772700 1 -- start start 2023-10-08T16:04:42.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3001c460 con 0x7faa3406ab20 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3001c5c0 con 0x7faa3406ab20 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa340b22d0 con 0x7faa3406ab20 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3001c980 con 0x7faa3406ab20 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f1f7fe700 1 -- 172.21.15.170:0/2931419786 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157032671 0 0) 0x7f0f0c001110 con 0x7f0f20068aa0 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f1f7fe700 1 -- 172.21.15.170:0/2931419786 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f10003570 con 0x7f0f20068aa0 2023-10-08T16:04:42.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.555+0000 7f5ee77fe700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 0x7f5ee815c540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e930 tx=0x7f5ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 --2- 172.21.15.170:0/2931419786 >> v2:172.21.15.191:3300/0 conn(0x7f0f20156760 0x7f0f20068ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d4159420 0x7f82d4179520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d41597f0 0x7f82d41419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82d415d230 0x7f82d406e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415c770 con 0x7f82d41597f0 2023-10-08T16:04:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d406cff0 con 0x7f82d4159420 2023-10-08T16:04:42.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9e98700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d406c3e0 con 0x7f82d415d230 2023-10-08T16:04:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f192ccc9700 1 Processor -- start 2023-10-08T16:04:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 -- 172.21.15.170:0/2931419786 >> v2:172.21.15.170:3300/0 conn(0x7f0f200cba20 msgr2=0x7f0f200ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 --2- 172.21.15.170:0/2931419786 >> v2:172.21.15.170:3300/0 conn(0x7f0f200cba20 0x7f0f200ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 -- 172.21.15.170:0/2931419786 >> v1:172.21.15.170:6789/0 conn(0x7f0f20068aa0 legacy=0x7f0f20156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 -- 172.21.15.170:0/2931419786 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20069430 con 0x7f0f20156760 2023-10-08T16:04:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f156c0afc40 0x7f156c143c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c0b0010 0x7f156c161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 0x7f156c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c144180 con 0x7f156c0b0010 2023-10-08T16:04:42.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c1437b0 con 0x7f156c0b2140 2023-10-08T16:04:42.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f1572404700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c142ba0 con 0x7f156c0afc40 2023-10-08T16:04:42.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f0f257bc700 1 --2- 172.21.15.170:0/2931419786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f20068ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f20064300 tx=0x7f0f18009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431a2bf8ca122ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d8e96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d4159420 0x7f82d4179520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d9697700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82d415d230 0x7f82d406e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d8e96700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d4159420 0x7f82d4179520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38402/0 (socket says 172.21.15.170:38402) 2023-10-08T16:04:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f82d8e96700 1 -- 172.21.15.170:0/1001265673 learned_addr learned my addr 172.21.15.170:0/1001265673 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7fdec3772700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdebc068aa0 0x7fdebc0aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7fdec3772700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec3772700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdebc156050 0x7fdebc16c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec3772700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc16b720 con 0x7fdebc156050 2023-10-08T16:04:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec3772700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc14bfc0 con 0x7fdebc068aa0 2023-10-08T16:04:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec3772700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc14c2a0 con 0x7fdebc0cba20 2023-10-08T16:04:42.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7f156bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f156c0afc40 0x7f156c143c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec150e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdebc068aa0 0x7fdebc0aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec150e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdebc068aa0 0x7fdebc0aa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38406/0 (socket says 172.21.15.170:38406) 2023-10-08T16:04:42.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec150e700 1 -- 172.21.15.170:0/2379605134 learned_addr learned my addr 172.21.15.170:0/2379605134 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d3fff700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d41597f0 0x7f82d41419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa340b2890 con 0x7faa3406ab20 2023-10-08T16:04:42.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/2561437637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80b1c50 msgr2=0x7fe0b80ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.556+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/2561437637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8087f90 tx=0x7fe0b400fa80 comp rx=0 tx=0).stop 2023-10-08T16:04:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 0x7f156c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 0x7f156c1619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38418/0 (socket says 172.21.15.170:38418) 2023-10-08T16:04:42.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 -- 172.21.15.170:0/3768005662 learned_addr learned my addr 172.21.15.170:0/3768005662 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 -- 172.21.15.170:0/1001265673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d4159420 msgr2=0x7f82d4179520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f82d4159420 0x7f82d4179520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 -- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d41597f0 msgr2=0x7f82d41419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82d41597f0 0x7f82d41419f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc009a20 con 0x7f82d415d230 2023-10-08T16:04:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f82d9697700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82d415d230 0x7f82d406e070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82d40b2f20 tx=0x7f82cc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f192ccc9700 1 -- start start 2023-10-08T16:04:42.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f0f1f7fe700 1 -- 172.21.15.170:0/2931419786 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1800e070 con 0x7f0f20156760 2023-10-08T16:04:42.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f0f1f7fe700 1 -- 172.21.15.170:0/2931419786 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1800bb20 con 0x7f0f20156760 2023-10-08T16:04:42.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f0f1f7fe700 1 -- 172.21.15.170:0/2931419786 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18012430 con 0x7f0f20156760 2023-10-08T16:04:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 -- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f156c0afc40 msgr2=0x7f156c143c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f156c0afc40 0x7f156c143c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 -- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c0b0010 msgr2=0x7f156c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f156c0b0010 0x7f156c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f15709a1700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0094f0 con 0x7f156c0b2140 2023-10-08T16:04:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8013680 con 0x7f5ee8159ee0 2023-10-08T16:04:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 -- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdebc068aa0 msgr2=0x7fdebc0aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdebc068aa0 0x7fdebc0aa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 -- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdebc156050 msgr2=0x7fdebc16c2e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T16:04:42.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdebc156050 0x7fdebc16c2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fdec0d0d700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8009d80 con 0x7fdebc0cba20 2023-10-08T16:04:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed80137e0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee815bd40 con 0x7f5ee8159ee0 2023-10-08T16:04:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/2561437637 shutdown_connections 2023-10-08T16:04:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/2561437637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80b1c50 0x7fe0b80ab8c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/2561437637 >> v2:172.21.15.191:3300/0 conn(0x7fe0b80875d0 0x7fe0b8087a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/2561437637 >> 172.21.15.170:0/2561437637 conn(0x7fe0b80cd000 msgr2=0x7fe0b8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.557+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa3404c3c0 con 0x7faa3406ab20 2023-10-08T16:04:42.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee8154890 con 0x7f5ee8159ee0 2023-10-08T16:04:42.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8013b60 con 0x7f5ee8159ee0 2023-10-08T16:04:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdec0d0d700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc14b750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400bad0 tx=0x7fdeb4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400c8c0 con 0x7fdebc0cba20 2023-10-08T16:04:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc012e10 con 0x7f82d415d230 2023-10-08T16:04:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc01e660 con 0x7f82d415d230 2023-10-08T16:04:42.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc01ea20 con 0x7f82d415d230 2023-10-08T16:04:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d406c6c0 con 0x7f82d415d230 2023-10-08T16:04:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d406cc80 con 0x7f82d415d230 2023-10-08T16:04:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb400ca40 con 0x7fdebc0cba20 2023-10-08T16:04:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeac009420 con 0x7fdebc0cba20 2023-10-08T16:04:42.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb400ce00 con 0x7fdebc0cba20 2023-10-08T16:04:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc14c8e0 con 0x7fdebc0cba20 2023-10-08T16:04:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f0f2721f700 1 -- 172.21.15.170:0/2931419786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 msgr2=0x7f0f20068ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f0f2721f700 1 --2- 172.21.15.170:0/2931419786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f20068ef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f20064300 tx=0x7f0f18009780 comp rx=0 tx=0).stop 2023-10-08T16:04:42.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ec4000fd0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/2561437637 shutdown_connections 2023-10-08T16:04:42.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/2561437637 wait complete. 2023-10-08T16:04:42.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f15709a1700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 0x7f156c1619c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f156400bc30 tx=0x7f1564009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f192ccc9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f192ccc9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f192815df40 0x7f19280caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.558+0000 7f192ccc9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280cffc0 con 0x7f19280cbac0 2023-10-08T16:04:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192ccc9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19280cb4c0 con 0x7f192815df40 2023-10-08T16:04:42.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192ccc9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19280cf3b0 con 0x7f192815db70 2023-10-08T16:04:42.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f1926d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f19280cbac0 0x7f1928069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54774/0 (socket says 172.21.15.170:54774) 2023-10-08T16:04:42.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f1926d9d700 1 -- 172.21.15.170:0/2395403147 learned_addr learned my addr 172.21.15.170:0/2395403147 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192659c700 1 --2- 172.21.15.170:0/2395403147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35380/0 (socket says 172.21.15.170:35380) 2023-10-08T16:04:42.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 -- 172.21.15.170:0/2931419786 shutdown_connections 2023-10-08T16:04:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 --2- 172.21.15.170:0/2931419786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f20068ef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 --2- 172.21.15.170:0/2931419786 >> v2:172.21.15.170:3300/0 conn(0x7f0f200cba20 0x7f0f200ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 -- 172.21.15.170:0/2931419786 >> 172.21.15.170:0/2931419786 conn(0x7f0f200cd000 msgr2=0x7f0f20068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 -- 172.21.15.170:0/2931419786 shutdown_connections 2023-10-08T16:04:42.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f1925d9b700 1 --2- 172.21.15.170:0/2395403147 >> v2:172.21.15.191:3300/0 conn(0x7f192815df40 0x7f19280caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f0f2721f700 1 -- 172.21.15.170:0/2931419786 wait complete. 2023-10-08T16:04:42.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192659c700 1 -- 172.21.15.170:0/2395403147 >> v2:172.21.15.191:3300/0 conn(0x7f192815df40 msgr2=0x7f19280caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192659c700 1 --2- 172.21.15.170:0/2395403147 >> v2:172.21.15.191:3300/0 conn(0x7f192815df40 0x7f19280caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f192659c700 1 -- 172.21.15.170:0/2395403147 >> v1:172.21.15.170:6789/0 conn(0x7f19280cbac0 legacy=0x7f1928069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f192659c700 1 -- 172.21.15.170:0/2395403147 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280cf690 con 0x7f192815db70 2023-10-08T16:04:42.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15640092e0 con 0x7f156c0b2140 2023-10-08T16:04:42.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156400c7a0 con 0x7f156c0b2140 2023-10-08T16:04:42.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c142e80 con 0x7f156c0b2140 2023-10-08T16:04:42.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdebc14bdb0 con 0x7fdebc0cba20 2023-10-08T16:04:42.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156400cb80 con 0x7f156c0b2140 2023-10-08T16:04:42.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7fe0bf2e9700 1 Processor -- start 2023-10-08T16:04:42.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.559+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82d404c3c0 con 0x7f82d415d230 2023-10-08T16:04:42.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c143440 con 0x7f156c0b2140 2023-10-08T16:04:42.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa3001ccf0 con 0x7faa3406ab20 2023-10-08T16:04:42.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa18067080 0x7faa18069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa300861c0 con 0x7faa3406ab20 2023-10-08T16:04:42.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7faa1806f290 0x7faa180716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa18071d50 con 0x7faa1806f290 2023-10-08T16:04:42.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa30087dd0 con 0x7faa3406ab20 2023-10-08T16:04:42.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7faa398b6700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7faa1806f290 0x7faa180716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeb4014620 con 0x7fdebc0cba20 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ed800c460 con 0x7f5ee8159ee0 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ecc067050 0x7f5ecc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ed8084a90 con 0x7f5ee8159ee0 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7f5ecc06f260 0x7f5ecc071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ecc071d20 con 0x7f5ecc06f260 2023-10-08T16:04:42.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.560+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed8084dc0 con 0x7f5ee8159ee0 2023-10-08T16:04:42.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7f5ee7fff700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ecc067050 0x7f5ecc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7f0f2721f700 1 Processor -- start 2023-10-08T16:04:42.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7f192659c700 1 --2- 172.21.15.170:0/2395403147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4c483f77072899 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7f5ee7fff700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ecc067050 0x7f5ecc069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009bf0 tx=0x7f5edc0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7fdeb17fa700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdea0066f00 0x7fdea00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdeb40844c0 con 0x7fdebc0cba20 2023-10-08T16:04:42.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fdeb17fa700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fdea006f090 0x7fdea00714a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdea0071b50 con 0x7fdea006f090 2023-10-08T16:04:42.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7faa388b4700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa18067080 0x7faa18069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f5eecf56700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7f5ecc06f260 0x7f5ecc071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f192559a700 1 -- 172.21.15.170:0/2395403147 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00b640 con 0x7f192815db70 2023-10-08T16:04:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f192559a700 1 -- 172.21.15.170:0/2395403147 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c00b7a0 con 0x7f192815db70 2023-10-08T16:04:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f192559a700 1 -- 172.21.15.170:0/2395403147 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c00bb40 con 0x7f192815db70 2023-10-08T16:04:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82cc01c3e0 con 0x7f82d415d230 2023-10-08T16:04:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.561+0000 7fe0bf2e9700 1 -- start start 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d0ff9700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82b8067000 0x7f82b80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0b8068aa0 0x7fe0b815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80875d0 0x7fe0b8176b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0b80b1c50 0x7fe0b815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b81773c0 con 0x7fe0b80875d0 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b81776a0 con 0x7fe0b80b1c50 2023-10-08T16:04:42.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bf2e9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b8177980 con 0x7fe0b8068aa0 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bd085700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0b8068aa0 0x7fe0b815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82cc0868d0 con 0x7f82d415d230 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d0ff9700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7f82b806f190 0x7f82b80715a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82b8071c50 con 0x7f82b806f190 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fdec150e700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdea0066f00 0x7fdea00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeb4051730 con 0x7fdebc0cba20 2023-10-08T16:04:42.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f156c04c3c0 con 0x7f156c0b2140 2023-10-08T16:04:42.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bd085700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0b8068aa0 0x7fe0b815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35394/0 (socket says 172.21.15.170:35394) 2023-10-08T16:04:42.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7fe0bd085700 1 -- 172.21.15.170:0/3695934467 learned_addr learned my addr 172.21.15.170:0/3695934467 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d3fff700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7f82b806f190 0x7f82b80715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7faa398b6700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7faa1806f290 0x7faa180716a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f192ccc9700 1 -- 172.21.15.170:0/2395403147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 msgr2=0x7f19280ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2395403147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f191c0098e0 tx=0x7f191c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f82d8e96700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82b8067000 0x7f82b80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f82d3fff700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7f82b806f190 0x7f82b80715a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7faa388b4700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa18067080 0x7faa18069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa340c34a0 tx=0x7faa240090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.562+0000 7f0f2721f700 1 -- start start 2023-10-08T16:04:42.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82cc053bc0 con 0x7f82d415d230 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f20068aa0 0x7f0f20150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 0x7f0f200c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f200ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200a6470 con 0x7f0f200cba20 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f200cb000 con 0x7f0f20156760 2023-10-08T16:04:42.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f2721f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f201575f0 con 0x7f0f20068aa0 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80875d0 0x7fe0b8176b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 -- 172.21.15.170:0/3695934467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0b8068aa0 msgr2=0x7fe0b815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0b8068aa0 0x7fe0b815c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 -- 172.21.15.170:0/3695934467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0b80b1c50 msgr2=0x7fe0b815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0b80b1c50 0x7fe0b815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b400fd20 con 0x7fe0b80875d0 2023-10-08T16:04:42.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f1ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 0x7f0f200c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f5eecf56700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7f5ecc06f260 0x7f5ecc071670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f1ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 0x7f0f200c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46376/0 (socket says 172.21.15.170:46376) 2023-10-08T16:04:42.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f257bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f200ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7f0f1ffff700 1 -- 172.21.15.170:0/36631385 learned_addr learned my addr 172.21.15.170:0/36631385 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fe0bc884700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80875d0 0x7fe0b8176b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d890 tx=0x7fe0a8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== osd.5 v2:172.21.15.191:6824/91096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faa18071d50 con 0x7faa1806f290 2023-10-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 -- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f20068aa0 msgr2=0x7f0f20150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f20068aa0 0x7f0f20150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 -- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 msgr2=0x7f0f200ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fdec1d0f700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fdea006f090 0x7fdea00714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.563+0000 7fdec150e700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdea0066f00 0x7fdea00693b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0093c0 tx=0x7fdeac018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 -- 172.21.15.170:0/2395403147 shutdown_connections 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2395403147 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f192815db70 0x7f19280ced80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2395403147 >> v2:172.21.15.191:3300/0 conn(0x7f192815df40 0x7f19280caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 -- 172.21.15.170:0/2395403147 >> 172.21.15.170:0/2395403147 conn(0x7f19280d07f0 msgr2=0x7f19280d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f82d8e96700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82b8067000 0x7f82b80694b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009aa0 tx=0x7f82c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f20156760 0x7f0f200ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 -- 172.21.15.170:0/36631385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f18009a20 con 0x7f0f200cba20 2023-10-08T16:04:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1ffff700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 0x7f0f200c9ae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c590 tx=0x7f0f0c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 -- 172.21.15.170:0/2395403147 shutdown_connections 2023-10-08T16:04:42.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7fdec1d0f700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fdea006f090 0x7fdea00714a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== osd.0 v2:172.21.15.170:6808/90868 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f5ecc071d20 con 0x7f5ecc06f260 2023-10-08T16:04:42.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c005b30 con 0x7f0f200cba20 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1564012070 con 0x7f156c0b2140 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f1553fff700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1558067000 0x7f15580694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1564084650 con 0x7f156c0b2140 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f1553fff700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f155806f210 0x7f1558071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1558071cd0 con 0x7f155806f210 2023-10-08T16:04:42.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1564084a20 con 0x7f156c0b2140 2023-10-08T16:04:42.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a80125f0 con 0x7fe0b80875d0 2023-10-08T16:04:42.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a8012750 con 0x7fe0b80875d0 2023-10-08T16:04:42.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8012b10 con 0x7fe0b80875d0 2023-10-08T16:04:42.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f156b7fe700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f155806f210 0x7f1558071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f156bfff700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1558067000 0x7f15580694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== osd.3 v2:172.21.15.170:6824/90884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdea0071b50 con 0x7fdea006f090 2023-10-08T16:04:42.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f156b7fe700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f155806f210 0x7f1558071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c005c90 con 0x7f0f200cba20 2023-10-08T16:04:42.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c007320 con 0x7f0f200cba20 2023-10-08T16:04:42.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f200cad20 con 0x7f0f200cba20 2023-10-08T16:04:42.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.565+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f20157280 con 0x7f0f200cba20 2023-10-08T16:04:42.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.566+0000 7f156bfff700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1558067000 0x7f15580694b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f156c063d40 tx=0x7f155c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.564+0000 7f192ccc9700 1 -- 172.21.15.170:0/2395403147 wait complete. 2023-10-08T16:04:42.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.566+0000 7f192ccc9700 1 Processor -- start 2023-10-08T16:04:42.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.566+0000 7f192ccc9700 1 -- start start 2023-10-08T16:04:42.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.566+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== osd.4 v2:172.21.15.191:6800/91066 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1558071cd0 con 0x7f155806f210 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 0x7f19281630f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f192815db70 0x7f1928170fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192815df40 0x7f19280c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19281620c0 con 0x7f192815df40 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1928161440 con 0x7f192815db70 2023-10-08T16:04:42.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192ccc9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1928161720 con 0x7f19280cbac0 2023-10-08T16:04:42.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b815eb30 con 0x7fe0b80875d0 2023-10-08T16:04:42.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b8177040 con 0x7fe0b80875d0 2023-10-08T16:04:42.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f0c01b3b0 con 0x7f0f200cba20 2023-10-08T16:04:42.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 0x7f19281630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 0x7f19281630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35416/0 (socket says 172.21.15.170:35416) 2023-10-08T16:04:42.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 -- 172.21.15.170:0/2356771035 learned_addr learned my addr 172.21.15.170:0/2356771035 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f0f1cff9700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0f04067080 0x7f0f04069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 -- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f192815db70 msgr2=0x7f1928170fe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T16:04:42.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f192815db70 0x7f1928170fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 -- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192815df40 msgr2=0x7f19280c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f192815df40 0x7f19280c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f192659c700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009420 con 0x7f19280cbac0 2023-10-08T16:04:42.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f0c083870 con 0x7f0f200cba20 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.568+0000 7f0f24fbb700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0f04067080 0x7f0f04069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.568+0000 7f0f24fbb700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0f04067080 0x7f0f04069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006fd0 tx=0x7f0f14006db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.568+0000 7f0f2721f700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7f0f000015f0 0x7f0f00003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.568+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 --> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f00006b90 con 0x7f0f000015f0 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.568+0000 7f192659c700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 0x7f19281630f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f191c0093f0 tx=0x7f191c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f0f257bc700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7f0f000015f0 0x7f0f00003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c01f070 con 0x7f19280cbac0 2023-10-08T16:04:42.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1928161a00 con 0x7f19280cbac0 2023-10-08T16:04:42.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1928171520 con 0x7f19280cbac0 2023-10-08T16:04:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c01ab40 con 0x7f19280cbac0 2023-10-08T16:04:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.569+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c025420 con 0x7f19280cbac0 2023-10-08T16:04:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.570+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f190c000fd0 con 0x7f19280cbac0 2023-10-08T16:04:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.570+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f191c023070 con 0x7f19280cbac0 2023-10-08T16:04:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.570+0000 7f0f257bc700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7f0f000015f0 0x7f0f00003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f19167fc700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1908066f30 0x7f19080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f191c085a20 con 0x7f19280cbac0 2023-10-08T16:04:42.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f19167fc700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f190806f140 0x7f1908071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1908071c00 con 0x7f190806f140 2023-10-08T16:04:42.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191c08a240 con 0x7f19280cbac0 2023-10-08T16:04:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== osd.7 v2:172.21.15.191:6816/91097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0f00006b90 con 0x7f0f000015f0 2023-10-08T16:04:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.571+0000 7f1926d9d700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f190806f140 0x7f1908071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.572+0000 7f1926d9d700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f190806f140 0x7f1908071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.572+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== osd.6 v2:172.21.15.191:6808/91067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1908071c00 con 0x7f190806f140 2023-10-08T16:04:42.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.577+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe0b815e9b0 con 0x7fe0b80875d0 2023-10-08T16:04:42.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.578+0000 7f1925d9b700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1908066f30 0x7f19080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.567+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== osd.2 v2:172.21.15.170:6816/90869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f82b8071c50 con 0x7f82b806f190 2023-10-08T16:04:42.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.578+0000 7f1925d9b700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1908066f30 0x7f19080693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19280c9560 tx=0x7f192000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.587+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0a801e460 con 0x7fe0b80875d0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.588+0000 7fe0ad7fa700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe09c066fb0 0x7fe09c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.588+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe0a8085620 con 0x7fe0b80875d0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.588+0000 7fe0ad7fa700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7fe09c06f1c0 0x7fe09c0715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.588+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe09c071c80 con 0x7fe09c06f1c0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.588+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0a8085950 con 0x7fe0b80875d0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.589+0000 7fe0bd886700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7fe09c06f1c0 0x7fe09c0715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.589+0000 7fe0bd085700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe09c066fb0 0x7fe09c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.590+0000 7fe0bd886700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7fe09c06f1c0 0x7fe09c0715d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.590+0000 7fe0bd085700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe09c066fb0 0x7fe09c069460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4015830 tx=0x7fe0b401f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.591+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== osd.1 v2:172.21.15.170:6800/90867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe09c071c80 con 0x7fe09c06f1c0 2023-10-08T16:04:42.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.592+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 --> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa3406a3c0 con 0x7faa1806f290 2023-10-08T16:04:42.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.592+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 --> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ec4002d30 con 0x7f5ecc06f260 2023-10-08T16:04:42.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa297fa700 1 -- 172.21.15.170:0/816257408 <== osd.5 v2:172.21.15.191:6824/91096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faa3406a3c0 con 0x7faa1806f290 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7faa1806f290 msgr2=0x7faa180716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7faa1806f290 0x7faa180716a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa18067080 msgr2=0x7faa18069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa18067080 0x7faa18069530 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faa340c34a0 tx=0x7faa240090f0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 msgr2=0x7faa340b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.593+0000 7faa3b319700 1 --2- 172.21.15.170:0/816257408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa3406ab20 0x7faa340b1cf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa30003080 tx=0x7faa30016000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.594+0000 7faa398b6700 1 -- 172.21.15.170:0/816257408 reap_dead start 2023-10-08T16:04:42.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.595+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 --> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdebc04c3c0 con 0x7fdea006f090 2023-10-08T16:04:42.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.595+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 shutdown_connections 2023-10-08T16:04:42.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.595+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 >> 172.21.15.170:0/816257408 conn(0x7faa340c9870 msgr2=0x7faa34149450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdeb17fa700 1 -- 172.21.15.170:0/2379605134 <== osd.3 v2:172.21.15.170:6824/90884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdebc04c3c0 con 0x7fdea006f090 2023-10-08T16:04:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fdea006f090 msgr2=0x7fdea00714a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fdea006f090 0x7fdea00714a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdea0066f00 msgr2=0x7fdea00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdea0066f00 0x7fdea00693b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0093c0 tx=0x7fdeac018000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 msgr2=0x7fdebc14b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 --2- 172.21.15.170:0/2379605134 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdebc0cba20 0x7fdebc14b750 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400bad0 tx=0x7fdeb4009780 comp rx=0 tx=0).stop 2023-10-08T16:04:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec1d0f700 1 -- 172.21.15.170:0/2379605134 reap_dead start 2023-10-08T16:04:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 shutdown_connections 2023-10-08T16:04:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.596+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 >> 172.21.15.170:0/2379605134 conn(0x7fdebc0cd000 msgr2=0x7fdebc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.597+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 shutdown_connections 2023-10-08T16:04:42.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.597+0000 7faa3b319700 1 -- 172.21.15.170:0/816257408 wait complete. 2023-10-08T16:04:42.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.597+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 shutdown_connections 2023-10-08T16:04:42.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.597+0000 7fdec3772700 1 -- 172.21.15.170:0/2379605134 wait complete. 2023-10-08T16:04:42.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.598+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 --> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f156c0ad310 con 0x7f155806f210 2023-10-08T16:04:42.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.598+0000 7f1553fff700 1 -- 172.21.15.170:0/3768005662 <== osd.4 v2:172.21.15.191:6800/91066 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f156c0ad310 con 0x7f155806f210 2023-10-08T16:04:42.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f5ed7fff700 1 -- 172.21.15.170:0/1781779751 <== osd.0 v2:172.21.15.170:6808/90868 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ec4002d30 con 0x7f5ecc06f260 2023-10-08T16:04:42.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f155806f210 msgr2=0x7f1558071620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f1572404700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f155806f210 0x7f1558071620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 --> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82d40b1790 con 0x7f82b806f190 2023-10-08T16:04:42.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f82d0ff9700 1 -- 172.21.15.170:0/1001265673 <== osd.2 v2:172.21.15.170:6816/90869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82d40b1790 con 0x7f82b806f190 2023-10-08T16:04:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.599+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7f5ecc06f260 msgr2=0x7f5ecc071670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7f5ecc06f260 0x7f5ecc071670 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 --> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f00005c20 con 0x7f0f000015f0 2023-10-08T16:04:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 --> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f190c002d30 con 0x7f190806f140 2023-10-08T16:04:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1558067000 msgr2=0x7f15580694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1558067000 0x7f15580694b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f156c063d40 tx=0x7f155c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 msgr2=0x7f156c1619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 --2- 172.21.15.170:0/3768005662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f156c0b2140 0x7f156c1619c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f156400bc30 tx=0x7f1564009840 comp rx=0 tx=0).stop 2023-10-08T16:04:42.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f15709a1700 1 -- 172.21.15.170:0/3768005662 reap_dead start 2023-10-08T16:04:42.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ecc067050 msgr2=0x7f5ecc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ecc067050 0x7f5ecc069500 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009bf0 tx=0x7f5edc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 msgr2=0x7f5ee815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eee9b9700 1 --2- 172.21.15.170:0/1781779751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8159ee0 0x7f5ee815c540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800e930 tx=0x7f5ed80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f5eecf56700 1 -- 172.21.15.170:0/1781779751 reap_dead start 2023-10-08T16:04:42.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 shutdown_connections 2023-10-08T16:04:42.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 >> 172.21.15.170:0/3768005662 conn(0x7f156c06a320 msgr2=0x7f156c0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f0f1cff9700 1 -- 172.21.15.170:0/36631385 <== osd.7 v2:172.21.15.191:6816/91097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0f00005c20 con 0x7f0f000015f0 2023-10-08T16:04:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f19167fc700 1 -- 172.21.15.170:0/2356771035 <== osd.6 v2:172.21.15.191:6808/91067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f190c002d30 con 0x7f190806f140 2023-10-08T16:04:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7f82b806f190 msgr2=0x7f82b80715a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.600+0000 7f82d9e98700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7f82b806f190 0x7f82b80715a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82b8067000 msgr2=0x7f82b80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82b8067000 0x7f82b80694b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009aa0 tx=0x7f82c0008040 comp rx=0 tx=0).stop 2023-10-08T16:04:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82d415d230 msgr2=0x7f82d406e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 --2- 172.21.15.170:0/1001265673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f82d415d230 0x7f82d406e070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82d40b2f20 tx=0x7f82cc017000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 shutdown_connections 2023-10-08T16:04:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f1572404700 1 -- 172.21.15.170:0/3768005662 wait complete. 2023-10-08T16:04:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7f0f000015f0 msgr2=0x7f0f00003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7f0f000015f0 0x7f0f00003aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f190806f140 msgr2=0x7f1908071550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f190806f140 0x7f1908071550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0f04067080 msgr2=0x7f0f04069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0f04067080 0x7f0f04069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f14006fd0 tx=0x7f0f14006db0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 msgr2=0x7f0f200c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 --2- 172.21.15.170:0/36631385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f200cba20 0x7f0f200c9ae0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c590 tx=0x7f0f0c000fe0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f257bc700 1 -- 172.21.15.170:0/36631385 reap_dead start 2023-10-08T16:04:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 shutdown_connections 2023-10-08T16:04:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 >> 172.21.15.170:0/36631385 conn(0x7f0f200cd000 msgr2=0x7f0f20153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1908066f30 msgr2=0x7f19080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1908066f30 0x7f19080693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19280c9560 tx=0x7f192000b040 comp rx=0 tx=0).stop 2023-10-08T16:04:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 msgr2=0x7f19281630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f192ccc9700 1 --2- 172.21.15.170:0/2356771035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f19280cbac0 0x7f19281630f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f191c0093f0 tx=0x7f191c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f1926d9d700 1 -- 172.21.15.170:0/2356771035 reap_dead start 2023-10-08T16:04:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9697700 1 -- 172.21.15.170:0/1001265673 reap_dead start 2023-10-08T16:04:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 shutdown_connections 2023-10-08T16:04:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 >> 172.21.15.170:0/1001265673 conn(0x7f82d40d0660 msgr2=0x7f82d40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 shutdown_connections 2023-10-08T16:04:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.601+0000 7f82d9e98700 1 -- 172.21.15.170:0/1001265673 wait complete. 2023-10-08T16:04:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.602+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 shutdown_connections 2023-10-08T16:04:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.602+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 >> 172.21.15.170:0/2356771035 conn(0x7f19280d07f0 msgr2=0x7f19280b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.602+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 shutdown_connections 2023-10-08T16:04:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.602+0000 7f192ccc9700 1 -- 172.21.15.170:0/2356771035 wait complete. 2023-10-08T16:04:42.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.603+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 shutdown_connections 2023-10-08T16:04:42.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.603+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 >> 172.21.15.170:0/1781779751 conn(0x7f5ee80c9870 msgr2=0x7f5ee806a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.604+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 shutdown_connections 2023-10-08T16:04:42.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.605+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 shutdown_connections 2023-10-08T16:04:42.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.611+0000 7f0f2721f700 1 -- 172.21.15.170:0/36631385 wait complete. 2023-10-08T16:04:42.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.611+0000 7f5eee9b9700 1 -- 172.21.15.170:0/1781779751 wait complete. 2023-10-08T16:04:42.677 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:42.678 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T16:04:42.679 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:42.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T16:04:42.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.619+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 --> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe0b80872f0 con 0x7fe09c06f1c0 2023-10-08T16:04:42.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.625+0000 7fe0ad7fa700 1 -- 172.21.15.170:0/3695934467 <== osd.1 v2:172.21.15.170:6800/90867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe0b80872f0 con 0x7fe09c06f1c0 2023-10-08T16:04:42.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7fe09c06f1c0 msgr2=0x7fe09c0715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7fe09c06f1c0 0x7fe09c0715d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe09c066fb0 msgr2=0x7fe09c069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe09c066fb0 0x7fe09c069460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4015830 tx=0x7fe0b401f000 comp rx=0 tx=0).stop 2023-10-08T16:04:42.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80875d0 msgr2=0x7fe0b8176b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 --2- 172.21.15.170:0/3695934467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0b80875d0 0x7fe0b8176b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d890 tx=0x7fe0a8008040 comp rx=0 tx=0).stop 2023-10-08T16:04:42.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bd886700 1 -- 172.21.15.170:0/3695934467 reap_dead start 2023-10-08T16:04:42.682 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:42.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T16:04:42.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 shutdown_connections 2023-10-08T16:04:42.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.626+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 >> 172.21.15.170:0/3695934467 conn(0x7fe0b80cd000 msgr2=0x7fe0b8165ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.684 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:42.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T16:04:42.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.627+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 shutdown_connections 2023-10-08T16:04:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.627+0000 7fe0bf2e9700 1 -- 172.21.15.170:0/3695934467 wait complete. 2023-10-08T16:04:42.685 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:42.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T16:04:42.686 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:42.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T16:04:42.688 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:42.688 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T16:04:42.689 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:42.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T16:04:42.694 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:04:42.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:04:42.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.917+0000 7f2345e11700 1 Processor -- start 2023-10-08T16:04:42.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.917+0000 7f2345e11700 1 -- start start 2023-10-08T16:04:42.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.918+0000 7f2345e11700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2340140350 0x7f2340069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.918+0000 7f2345e11700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f234006a1c0 0x7f2340068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.918+0000 7f2345e11700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23400c7de0 con 0x7f2340140720 2023-10-08T16:04:42.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.918+0000 7f2345e11700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2340140c10 con 0x7f234006a1c0 2023-10-08T16:04:42.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.918+0000 7f2345e11700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2340141490 con 0x7f2340140350 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2340140350 0x7f2340069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2340140350 0x7f2340069c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38460/0 (socket says 172.21.15.170:38460) 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 -- 172.21.15.170:0/2898611460 learned_addr learned my addr 172.21.15.170:0/2898611460 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 -- 172.21.15.170:0/2898611460 >> v2:172.21.15.170:3300/0 conn(0x7f234006a1c0 msgr2=0x7f2340068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 --2- 172.21.15.170:0/2898611460 >> v2:172.21.15.170:3300/0 conn(0x7f234006a1c0 0x7f2340068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 -- 172.21.15.170:0/2898611460 >> v1:172.21.15.191:6789/0 conn(0x7f2340140720 legacy=0x7f234015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 -- 172.21.15.170:0/2898611460 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340140ed0 con 0x7f2340140350 2023-10-08T16:04:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233f7fe700 1 --2- 172.21.15.170:0/2898611460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f2340069c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f234006d4b0 tx=0x7f2334009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e2aee07d04bdb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f233e7fc700 1 -- 172.21.15.170:0/2898611460 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334005c10 con 0x7f2340140350 2023-10-08T16:04:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f233e7fc700 1 -- 172.21.15.170:0/2898611460 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334005d70 con 0x7f2340140350 2023-10-08T16:04:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f233e7fc700 1 -- 172.21.15.170:0/2898611460 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400b840 con 0x7f2340140350 2023-10-08T16:04:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f2345e11700 1 -- 172.21.15.170:0/2898611460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 msgr2=0x7f2340069c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f2345e11700 1 --2- 172.21.15.170:0/2898611460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f2340069c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f234006d4b0 tx=0x7f2334009220 comp rx=0 tx=0).stop 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f0ec6088700 1 Processor -- start 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.919+0000 7f0ec6088700 1 -- start start 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec00879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0ec00b1c50 0x7f0ec00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0ec0068aa0 0x7f0ec0068f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec0069460 con 0x7f0ec00b1c50 2023-10-08T16:04:42.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec00acdf0 con 0x7f0ec0068aa0 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ec6088700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00ac1e0 con 0x7f0ec00875d0 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ebf7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec00879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ebf7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec00879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35420/0 (socket says 172.21.15.170:35420) 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ebf7fe700 1 -- 172.21.15.170:0/2516875676 learned_addr learned my addr 172.21.15.170:0/2516875676 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.920+0000 7f0ebffff700 1 --2- 172.21.15.170:0/2516875676 >> v2:172.21.15.191:3300/0 conn(0x7f0ec0068aa0 0x7f0ec0068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 -- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 msgr2=0x7f0ec00879a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 --2- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec00879a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 -- 172.21.15.170:0/2516875676 >> v2:172.21.15.170:3300/0 conn(0x7f0ec00b1c50 msgr2=0x7f0ec00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 --2- 172.21.15.170:0/2516875676 >> v2:172.21.15.170:3300/0 conn(0x7f0ec00b1c50 0x7f0ec00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 -- 172.21.15.170:0/2516875676 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00ac4c0 con 0x7f0ec0068aa0 2023-10-08T16:04:42.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f2345e11700 1 -- 172.21.15.170:0/2898611460 shutdown_connections 2023-10-08T16:04:42.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f2345e11700 1 --2- 172.21.15.170:0/2898611460 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f2340069c80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f2345e11700 1 --2- 172.21.15.170:0/2898611460 >> v2:172.21.15.170:3300/0 conn(0x7f234006a1c0 0x7f2340068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f2345e11700 1 -- 172.21.15.170:0/2898611460 >> 172.21.15.170:0/2898611460 conn(0x7f23400cd670 msgr2=0x7f23400cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.917+0000 7f915f0ab700 1 Processor -- start 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f915f0ab700 1 -- start start 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.922+0000 7f915f0ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580b1f40 0x7f91580ac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915f0ab700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580accf0 con 0x7f9158159fb0 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915f0ab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f915815a440 con 0x7f9158068aa0 2023-10-08T16:04:42.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915f0ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915815a720 con 0x7f91580b1f40 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915d648700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f9158068aa0 0x7f915815d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55968/0 (socket says 172.21.15.170:55968) 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915d648700 1 -- 172.21.15.170:0/2713035758 learned_addr learned my addr 172.21.15.170:0/2713035758 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebffff700 1 --2- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec0068f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ec016b340 tx=0x7f0eb8006500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b889ec1c6bb3a038 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebe7fc700 1 -- 172.21.15.170:0/2516875676 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8007e30 con 0x7f0ec0068aa0 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebe7fc700 1 -- 172.21.15.170:0/2516875676 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb801a440 con 0x7f0ec0068aa0 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.921+0000 7f0ebe7fc700 1 -- 172.21.15.170:0/2516875676 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb801a7e0 con 0x7f0ec0068aa0 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.922+0000 7f0ec6088700 1 -- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 msgr2=0x7f0ec0068f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.922+0000 7f0ec6088700 1 --2- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec0068f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ec016b340 tx=0x7f0eb8006500 comp rx=0 tx=0).stop 2023-10-08T16:04:42.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f0ec6088700 1 -- 172.21.15.170:0/2516875676 shutdown_connections 2023-10-08T16:04:42.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f0ec6088700 1 --2- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec0068f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f0ec6088700 1 --2- 172.21.15.170:0/2516875676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec00879a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f0ec6088700 1 --2- 172.21.15.170:0/2516875676 >> v2:172.21.15.170:3300/0 conn(0x7f0ec00b1c50 0x7f0ec00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f0ec6088700 1 -- 172.21.15.170:0/2516875676 >> 172.21.15.170:0/2516875676 conn(0x7f0ec00cd000 msgr2=0x7f0ec0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935466175 0 0) 0x7f915815a440 con 0x7f9158068aa0 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9140003570 con 0x7f9158068aa0 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064981289 0 0) 0x7f91580accf0 con 0x7f9158159fb0 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f915815a440 con 0x7f9158159fb0 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f915ce47700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580b1f40 0x7f91580ac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f3f56283700 1 Processor -- start 2023-10-08T16:04:42.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.923+0000 7f3f56283700 1 -- start start 2023-10-08T16:04:42.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f2345e11700 1 -- 172.21.15.170:0/2898611460 shutdown_connections 2023-10-08T16:04:42.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f0891cce700 1 Processor -- start 2023-10-08T16:04:42.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129449625 0 0) 0x7f9140003570 con 0x7f9158068aa0 2023-10-08T16:04:42.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580b1f40 msgr2=0x7f91580ac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91580b1f40 0x7f91580ac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 >> v1:172.21.15.170:6789/0 conn(0x7f9158159fb0 legacy=0x7f91580aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f0891cce700 1 -- start start 2023-10-08T16:04:42.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915815aa00 con 0x7f9158068aa0 2023-10-08T16:04:42.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f9148001470 con 0x7f9158068aa0 2023-10-08T16:04:42.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 >> v1:172.21.15.191:6789/0 conn(0x7f9158068aa0 legacy=0x7f915815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9140003c20 0x7f9140003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140004810 0x7f9140004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f914000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914c001230 con 0x7f9140003c20 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915815a720 con 0x7f9140004810 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914000c2b0 con 0x7f91400052e0 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f9157fff700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140004810 0x7f9140004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f915d648700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f914000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f2345e11700 1 -- 172.21.15.170:0/2898611460 wait complete. 2023-10-08T16:04:42.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.924+0000 7f0ec6088700 1 -- 172.21.15.170:0/2516875676 shutdown_connections 2023-10-08T16:04:42.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f56283700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3f500691b0 0x7f3f50158510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f56283700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500cb000 con 0x7f3f50158fd0 2023-10-08T16:04:42.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f56283700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f500ca710 con 0x7f3f50151c50 2023-10-08T16:04:42.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f56283700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f50158e10 con 0x7f3f500691b0 2023-10-08T16:04:42.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0ec6088700 1 -- 172.21.15.170:0/2516875676 wait complete. 2023-10-08T16:04:42.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 -- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140004810 msgr2=0x7f9140004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9140004810 0x7f9140004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 -- 172.21.15.170:0/2713035758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9140003c20 msgr2=0x7f9140003ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T16:04:42.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9140003c20 0x7f9140003ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 -- 172.21.15.170:0/2713035758 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140003570 con 0x7f91400052e0 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0891cce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0891cce700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f088c0ad250 0x7f088c0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0891cce700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0adbf0 con 0x7f088c159720 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0891cce700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0aba20 con 0x7f088c0ad250 2023-10-08T16:04:42.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f0891cce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0ac020 con 0x7f088c159af0 2023-10-08T16:04:42.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f915d648700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f914000ba60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f914800c1e0 tx=0x7f9148001d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e478faf4f367a650 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800cb10 con 0x7f91400052e0 2023-10-08T16:04:42.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3f50158fd0 0x7f3f50157e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54796/0 (socket says 172.21.15.170:54796) 2023-10-08T16:04:42.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4f7fe700 1 -- 172.21.15.170:0/2711568199 learned_addr learned my addr 172.21.15.170:0/2711568199 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894056449 0 0) 0x7f3f500ca710 con 0x7f3f50151c50 2023-10-08T16:04:42.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f38003570 con 0x7f3f50151c50 2023-10-08T16:04:42.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269270330 0 0) 0x7f3f38003570 con 0x7f3f50151c50 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 >> v2:172.21.15.170:3300/0 conn(0x7f3f500691b0 msgr2=0x7f3f50158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 --2- 172.21.15.170:0/2711568199 >> v2:172.21.15.170:3300/0 conn(0x7f3f500691b0 0x7f3f50158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 >> v1:172.21.15.170:6789/0 conn(0x7f3f50158fd0 legacy=0x7f3f50157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f50158b30 con 0x7f3f50151c50 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f3f44001470 con 0x7f3f50151c50 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 >> v1:172.21.15.191:6789/0 conn(0x7f3f50151c50 legacy=0x7f3f50068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f38003ae0 0x7f3f38003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 0x7f3f38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.925+0000 7f3f4effd700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f380053c0 0x7f3f3800ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50158e10 con 0x7f3f38003ae0 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800c360 con 0x7f3f38004810 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3800c520 con 0x7f3f380053c0 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 0x7f3f38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f54820700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f380053c0 0x7f3f3800ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800cc90 con 0x7f91400052e0 2023-10-08T16:04:42.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f91577fe700 1 -- 172.21.15.170:0/2713035758 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148005b70 con 0x7f91400052e0 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35468/0 (socket says 172.21.15.170:35468) 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 -- 172.21.15.170:0/2224894401 learned_addr learned my addr 172.21.15.170:0/2224894401 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 -- 172.21.15.170:0/2224894401 >> v2:172.21.15.170:3300/0 conn(0x7f088c0ad250 msgr2=0x7f088c0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 --2- 172.21.15.170:0/2224894401 >> v2:172.21.15.170:3300/0 conn(0x7f088c0ad250 0x7f088c0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 -- 172.21.15.170:0/2224894401 >> v1:172.21.15.170:6789/0 conn(0x7f088c159720 legacy=0x7f088c0a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 -- 172.21.15.170:0/2224894401 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0aa440 con 0x7f088c159af0 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 -- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f380053c0 msgr2=0x7f3f3800ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f380053c0 0x7f3f3800ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 -- 172.21.15.170:0/2711568199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f38003ae0 msgr2=0x7f3f38003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f38003ae0 0x7f3f38003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f4f7fe700 1 -- 172.21.15.170:0/2711568199 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f50158b30 con 0x7f3f38004810 2023-10-08T16:04:42.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f3f54820700 1 -- 172.21.15.170:0/2711568199 reap_dead start 2023-10-08T16:04:42.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f2345e11700 1 Processor -- start 2023-10-08T16:04:42.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f088affd700 1 --2- 172.21.15.170:0/2224894401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f08800099b0 tx=0x7f0880009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=449932dab94fcf9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f0ec6088700 1 Processor -- start 2023-10-08T16:04:42.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f0ec6088700 1 -- start start 2023-10-08T16:04:42.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f915f0ab700 1 -- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 msgr2=0x7f914000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f915f0ab700 1 --2- 172.21.15.170:0/2713035758 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f914000ba60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f914800c1e0 tx=0x7f9148001d40 comp rx=0 tx=0).stop 2023-10-08T16:04:42.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f915d648700 1 -- 172.21.15.170:0/2713035758 reap_dead start 2023-10-08T16:04:42.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 0x7f3f38004c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f400052c0 tx=0x7f3f40007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abed9a1848a42ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.926+0000 7f2345e11700 1 -- start start 2023-10-08T16:04:42.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f088a7fc700 1 -- 172.21.15.170:0/2224894401 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088000b820 con 0x7f088c159af0 2023-10-08T16:04:42.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f915f0ab700 1 -- 172.21.15.170:0/2713035758 shutdown_connections 2023-10-08T16:04:42.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f915f0ab700 1 -- 172.21.15.170:0/2713035758 >> 172.21.15.170:0/2713035758 conn(0x7f91580cd000 msgr2=0x7f9158068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f088a7fc700 1 -- 172.21.15.170:0/2224894401 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088000b980 con 0x7f088c159af0 2023-10-08T16:04:42.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f088a7fc700 1 -- 172.21.15.170:0/2224894401 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088000bd20 con 0x7f088c159af0 2023-10-08T16:04:42.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400058e0 con 0x7f3f38004810 2023-10-08T16:04:42.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40005a40 con 0x7f3f38004810 2023-10-08T16:04:42.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f3f4effd700 1 -- 172.21.15.170:0/2711568199 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40005e00 con 0x7f3f38004810 2023-10-08T16:04:42.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.927+0000 7f0ec6088700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec017a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f23400c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2340140720 0x7f234015e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234013d9f0 con 0x7f234006a1c0 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400b16f0 con 0x7f2340140350 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f2345e11700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23400b0b20 con 0x7f2340140720 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ec6088700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ec6088700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 0x7f0ec0165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f915f0ab700 1 -- 172.21.15.170:0/2713035758 shutdown_connections 2023-10-08T16:04:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ec6088700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0164d90 con 0x7f0ec00b1c50 2023-10-08T16:04:42.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ec6088700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec017a8d0 con 0x7f0ec0068aa0 2023-10-08T16:04:42.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ec6088700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0164180 con 0x7f0ec00875d0 2023-10-08T16:04:42.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 -- 172.21.15.170:0/2224894401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 msgr2=0x7f088c0ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 --2- 172.21.15.170:0/2224894401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f08800099b0 tx=0x7f0880009250 comp rx=0 tx=0).stop 2023-10-08T16:04:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 0x7f0ec0165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 0x7f0ec0165220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46428/0 (socket says 172.21.15.170:46428) 2023-10-08T16:04:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35472/0 (socket says 172.21.15.170:35472) 2023-10-08T16:04:42.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebffff700 1 -- 172.21.15.170:0/3339745374 learned_addr learned my addr 172.21.15.170:0/3339745374 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46442/0 (socket says 172.21.15.170:46442) 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 -- 172.21.15.170:0/1326895222 learned_addr learned my addr 172.21.15.170:0/1326895222 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 -- 172.21.15.170:0/2224894401 shutdown_connections 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 --2- 172.21.15.170:0/2224894401 >> v2:172.21.15.170:3300/0 conn(0x7f088c0ad250 0x7f088c0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 --2- 172.21.15.170:0/2224894401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c159af0 0x7f088c0ab4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 -- 172.21.15.170:0/2224894401 >> 172.21.15.170:0/2224894401 conn(0x7f088c06a320 msgr2=0x7f088c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 -- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 msgr2=0x7f0ec017a390 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec017a390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 -- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 msgr2=0x7f0ec0165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 0x7f0ec0165220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0ebeffd700 1 -- 172.21.15.170:0/3339745374 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80116f0 con 0x7f0ec00875d0 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f3f56283700 1 -- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 msgr2=0x7f3f38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f3f56283700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 0x7f3f38004c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3f400052c0 tx=0x7f3f40007e20 comp rx=0 tx=0).stop 2023-10-08T16:04:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f915f0ab700 1 -- 172.21.15.170:0/2713035758 wait complete. 2023-10-08T16:04:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 -- 172.21.15.170:0/2224894401 shutdown_connections 2023-10-08T16:04:42.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f0891cce700 1 -- 172.21.15.170:0/2224894401 wait complete. 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 -- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2340140720 msgr2=0x7f234015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2340140720 0x7f234015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 -- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 msgr2=0x7f23400c5130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f23400c5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.928+0000 7f233f7fe700 1 -- 172.21.15.170:0/1326895222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334009480 con 0x7f234006a1c0 2023-10-08T16:04:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ebeffd700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400b630 tx=0x7f0eb40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f233f7fe700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2334003080 tx=0x7f2334015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400f510 con 0x7f0ec00875d0 2023-10-08T16:04:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb400f670 con 0x7f0ec00875d0 2023-10-08T16:04:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb400fa30 con 0x7f0ec00875d0 2023-10-08T16:04:42.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec0164460 con 0x7f0ec00875d0 2023-10-08T16:04:42.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec0164a20 con 0x7f0ec00875d0 2023-10-08T16:04:42.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334011bb0 con 0x7f234006a1c0 2023-10-08T16:04:42.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334011d10 con 0x7f234006a1c0 2023-10-08T16:04:42.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.929+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233401d660 con 0x7f234006a1c0 2023-10-08T16:04:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f915f0ab700 1 Processor -- start 2023-10-08T16:04:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f915f0ab700 1 -- start start 2023-10-08T16:04:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f0891cce700 1 Processor -- start 2023-10-08T16:04:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23400b0dc0 con 0x7f234006a1c0 2023-10-08T16:04:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23400b1380 con 0x7f234006a1c0 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f0891cce700 1 -- start start 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 0x7f088c14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 0x7f088c14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f088c159af0 0x7f088c14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0cadb0 con 0x7f088c159720 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c15ccc0 con 0x7f088c159af0 2023-10-08T16:04:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0891cce700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c14ca60 con 0x7f088c0ad250 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.930+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec004c3c0 con 0x7f0ec00875d0 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0eb401a400 con 0x7f0ec00875d0 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f0ea77fe700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 0x7f0eac0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0eb40840c0 con 0x7f0ec00875d0 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f91580cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 0x7f91580c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 0x7f91580a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580c9470 con 0x7f91580ca740 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580ca2b0 con 0x7f91581410a0 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915f0ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580c96a0 con 0x7f91400052e0 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915ce47700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f91580cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915ce47700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f91580cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35498/0 (socket says 172.21.15.170:35498) 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915ce47700 1 -- 172.21.15.170:0/130394817 learned_addr learned my addr 172.21.15.170:0/130394817 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f915d648700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 0x7f91580a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.931+0000 7f9157fff700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 0x7f91580c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f3f56283700 1 -- 172.21.15.170:0/2711568199 shutdown_connections 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f3f56283700 1 --2- 172.21.15.170:0/2711568199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f38004810 0x7f3f38004c80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f3f56283700 1 -- 172.21.15.170:0/2711568199 >> 172.21.15.170:0/2711568199 conn(0x7f3f500cd000 msgr2=0x7f3f50068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 0x7f088c14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f088affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 0x7f088c14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f088b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 0x7f088c14c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35514/0 (socket says 172.21.15.170:35514) 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f088b7fe700 1 -- 172.21.15.170:0/823453960 learned_addr learned my addr 172.21.15.170:0/823453960 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 -- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 msgr2=0x7f91580cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f91580cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 -- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 msgr2=0x7f91580a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 0x7f91580a73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 -- 172.21.15.170:0/130394817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580c9980 con 0x7f91580ca740 2023-10-08T16:04:42.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f915d648700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 0x7f91580a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:42.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f0ebf7fe700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 0x7f0eac0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23400b1c40 con 0x7f234006a1c0 2023-10-08T16:04:42.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f9157fff700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 0x7f91580c8f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f914c020b00 tx=0x7f914c001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.932+0000 7f0ebf7fe700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 0x7f0eac0694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006fd0 tx=0x7f0eb000a330 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f233401da50 con 0x7f234006a1c0 2023-10-08T16:04:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f560f03d700 1 Processor -- start 2023-10-08T16:04:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 -- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 msgr2=0x7f088c14c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 0x7f088c14c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 -- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f088c159af0 msgr2=0x7f088c14d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f088c159af0 0x7f088c14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 -- 172.21.15.170:0/823453960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800094f0 con 0x7f088c159720 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01e860 con 0x7f91580ca740 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c01e9c0 con 0x7f91580ca740 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01ed80 con 0x7f91580ca740 2023-10-08T16:04:42.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91580c9c60 con 0x7f91580ca740 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9158141500 con 0x7f91580ca740 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f088affd700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 0x7f088c14cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f088c063d40 tx=0x7f0880016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f23277fe700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 0x7f232c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2334086690 con 0x7f234006a1c0 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f233effd700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 0x7f232c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f233effd700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 0x7f232c0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2330006fd0 tx=0x7f2330006d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91580ca460 con 0x7f91580ca740 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.933+0000 7f560f03d700 1 -- start start 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 0x7f5608161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5608162080 0x7f560815e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080c74b0 con 0x7f5608162080 2023-10-08T16:04:42.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560806bc00 con 0x7f560806b830 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560f03d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5608158a30 con 0x7f56080cb440 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560cdd9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f5607fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 0x7f5608161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560cdd9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38516/0 (socket says 172.21.15.170:38516) 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f5607fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 0x7f5608161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35516/0 (socket says 172.21.15.170:35516) 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560cdd9700 1 -- 172.21.15.170:0/2047799467 learned_addr learned my addr 172.21.15.170:0/2047799467 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 --2- 172.21.15.170:0/2047799467 >> v2:172.21.15.170:3300/0 conn(0x7f5608162080 0x7f560815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 -- 172.21.15.170:0/2047799467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 msgr2=0x7f5608161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 --2- 172.21.15.170:0/2047799467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 0x7f5608161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 -- 172.21.15.170:0/2047799467 >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 msgr2=0x7f56080c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 --2- 172.21.15.170:0/2047799467 >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560d5da700 1 -- 172.21.15.170:0/2047799467 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56081624a0 con 0x7f5608162080 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.934+0000 7f560cdd9700 1 --2- 172.21.15.170:0/2047799467 >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.935+0000 7f560d5da700 1 --2- 172.21.15.170:0/2047799467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5608162080 0x7f560815e540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5608063e90 tx=0x7f55fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073f1d0b6455023 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.935+0000 7f3f56283700 1 -- 172.21.15.170:0/2711568199 shutdown_connections 2023-10-08T16:04:42.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.935+0000 7f3f56283700 1 -- 172.21.15.170:0/2711568199 wait complete. 2023-10-08T16:04:42.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.936+0000 7ff285538700 1 Processor -- start 2023-10-08T16:04:42.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.936+0000 7ff285538700 1 -- start start 2023-10-08T16:04:42.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.936+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eb4051390 con 0x7f0ec00875d0 2023-10-08T16:04:42.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7f3f56283700 1 Processor -- start 2023-10-08T16:04:42.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7f3f56283700 1 -- start start 2023-10-08T16:04:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff285538700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 0x7ff2800cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff285538700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff2800c91e0 0x7ff280160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff285538700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2800cc210 con 0x7ff28006bac0 2023-10-08T16:04:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff285538700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2800cc4f0 con 0x7ff2800c91e0 2023-10-08T16:04:42.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff285538700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2800cc7d0 con 0x7ff28006aa40 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff27e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff28006bac0 0x7ff28006ae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54816/0 (socket says 172.21.15.170:54816) 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff27e7fc700 1 -- 172.21.15.170:0/1721755667 learned_addr learned my addr 172.21.15.170:0/1721755667 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.937+0000 7ff27effd700 1 --2- 172.21.15.170:0/1721755667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 0x7ff2800cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27effd700 1 -- 172.21.15.170:0/1721755667 >> v2:172.21.15.191:3300/0 conn(0x7ff2800c91e0 msgr2=0x7ff280160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27f7fe700 1 --2- 172.21.15.170:0/1721755667 >> v2:172.21.15.191:3300/0 conn(0x7ff2800c91e0 0x7ff280160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27effd700 1 --2- 172.21.15.170:0/1721755667 >> v2:172.21.15.191:3300/0 conn(0x7ff2800c91e0 0x7ff280160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27effd700 1 -- 172.21.15.170:0/1721755667 >> v1:172.21.15.170:6789/0 conn(0x7ff28006bac0 legacy=0x7ff28006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27effd700 1 -- 172.21.15.170:0/1721755667 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2800ccab0 con 0x7ff28006aa40 2023-10-08T16:04:42.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27effd700 1 --2- 172.21.15.170:0/1721755667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 0x7ff2800cbc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff2700098d0 tx=0x7ff270009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04320154cd508b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 0x7f3f50141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f50153b80 0x7f3f50153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f501418b0 con 0x7f3f50153b80 2023-10-08T16:04:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50154530 con 0x7f3f50155b70 2023-10-08T16:04:42.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f56283700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50156cf0 con 0x7f3f38004810 2023-10-08T16:04:42.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27dffb700 1 -- 172.21.15.170:0/1721755667 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff270009e70 con 0x7ff28006aa40 2023-10-08T16:04:42.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27dffb700 1 -- 172.21.15.170:0/1721755667 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff270005c10 con 0x7ff28006aa40 2023-10-08T16:04:42.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7ff27dffb700 1 -- 172.21.15.170:0/1721755667 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27000b650 con 0x7ff28006aa40 2023-10-08T16:04:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35532/0 (socket says 172.21.15.170:35532) 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 -- 172.21.15.170:0/2368991773 learned_addr learned my addr 172.21.15.170:0/2368991773 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 0x7f3f50141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f54820700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f50153b80 0x7f3f50153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 -- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 msgr2=0x7f3f50141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 0x7f3f50141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 -- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f50153b80 msgr2=0x7f3f50153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f50153b80 0x7f3f50153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4ffff700 1 -- 172.21.15.170:0/2368991773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f501560e0 con 0x7f3f38004810 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.938+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 0x7f3f50141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:42.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f4ffff700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400cec0 tx=0x7f3f44007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400ad10 con 0x7f3f38004810 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400ae90 con 0x7f3f38004810 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f501563c0 con 0x7f3f38004810 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44001b90 con 0x7f3f38004810 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 -- 172.21.15.170:0/1721755667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 msgr2=0x7ff2800cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 --2- 172.21.15.170:0/1721755667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 0x7ff2800cbc00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff2700098d0 tx=0x7ff270009120 comp rx=0 tx=0).stop 2023-10-08T16:04:42.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f50156980 con 0x7f3f38004810 2023-10-08T16:04:42.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 -- 172.21.15.170:0/1721755667 shutdown_connections 2023-10-08T16:04:42.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 --2- 172.21.15.170:0/1721755667 >> v2:172.21.15.191:3300/0 conn(0x7ff2800c91e0 0x7ff280160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 --2- 172.21.15.170:0/1721755667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006aa40 0x7ff2800cbc00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 -- 172.21.15.170:0/1721755667 >> 172.21.15.170:0/1721755667 conn(0x7ff2800cd670 msgr2=0x7ff2800cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 -- 172.21.15.170:0/1721755667 shutdown_connections 2023-10-08T16:04:42.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7ff285538700 1 -- 172.21.15.170:0/1721755667 wait complete. 2023-10-08T16:04:42.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.939+0000 7f56077fe700 1 -- 172.21.15.170:0/2047799467 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc011630 con 0x7f5608162080 2023-10-08T16:04:42.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880026480 con 0x7f088c159720 2023-10-08T16:04:42.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088001f070 con 0x7f088c159720 2023-10-08T16:04:42.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c0caa80 con 0x7f088c159720 2023-10-08T16:04:42.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880026810 con 0x7f088c159720 2023-10-08T16:04:42.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c14dac0 con 0x7f088c159720 2023-10-08T16:04:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.940+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f440030f0 con 0x7f3f38004810 2023-10-08T16:04:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7f3f37fff700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 0x7f3f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f440849c0 con 0x7f3f38004810 2023-10-08T16:04:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7ff285538700 1 Processor -- start 2023-10-08T16:04:42.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7ff285538700 1 -- start start 2023-10-08T16:04:42.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 0x7f3f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c04c3c0 con 0x7f088c159720 2023-10-08T16:04:42.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.941+0000 7f3f4f7fe700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 0x7f3f3c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f40008fd0 tx=0x7f3f40015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f56077fe700 1 -- 172.21.15.170:0/2047799467 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0117b0 con 0x7f5608162080 2023-10-08T16:04:42.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f56077fe700 1 -- 172.21.15.170:0/2047799467 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc011b70 con 0x7f5608162080 2023-10-08T16:04:42.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28006aa40 0x7ff28015b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006bac0 0x7ff280167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28015be10 con 0x7ff28006aa40 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff280166660 con 0x7ff2800c91e0 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff285538700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff280166be0 con 0x7ff28006bac0 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2334053900 con 0x7f234006a1c0 2023-10-08T16:04:42.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5004c3c0 con 0x7f3f38004810 2023-10-08T16:04:42.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f004a6e7700 1 Processor -- start 2023-10-08T16:04:42.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:38542/0 (socket says 172.21.15.170:38542) 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 -- 172.21.15.170:0/1886917436 learned_addr learned my addr 172.21.15.170:0/1886917436 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 -- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006bac0 msgr2=0x7ff280167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006bac0 0x7ff280167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 -- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28006aa40 msgr2=0x7ff28015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28006aa40 0x7ff28015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7ff27f7fe700 1 -- 172.21.15.170:0/1886917436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2700093c0 con 0x7ff2800c91e0 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f560f03d700 1 -- 172.21.15.170:0/2047799467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5608162080 msgr2=0x7f560815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f560f03d700 1 --2- 172.21.15.170:0/2047799467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5608162080 0x7f560815e540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5608063e90 tx=0x7f55fc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 -- 172.21.15.170:0/2047799467 shutdown_connections 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 --2- 172.21.15.170:0/2047799467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5608162080 0x7f560815e540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 --2- 172.21.15.170:0/2047799467 >> v2:172.21.15.191:3300/0 conn(0x7f560806b830 0x7f56080c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 --2- 172.21.15.170:0/2047799467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56080cb440 0x7f5608161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 -- 172.21.15.170:0/2047799467 >> 172.21.15.170:0/2047799467 conn(0x7f56080c9870 msgr2=0x7f56080c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:42.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff27f7fe700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff27800cdb0 tx=0x7ff2780077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 -- 172.21.15.170:0/2047799467 shutdown_connections 2023-10-08T16:04:42.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f560f03d700 1 -- 172.21.15.170:0/2047799467 wait complete. 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27800aa80 con 0x7ff2800c91e0 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27800abe0 con 0x7ff2800c91e0 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff28015b5d0 con 0x7ff2800c91e0 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28015b850 con 0x7ff2800c91e0 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.942+0000 7f004a6e7700 1 -- start start 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f004a6e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0044162250 0x7f0044168ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f004a6e7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00440c74b0 con 0x7f00440c8f50 2023-10-08T16:04:42.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f004a6e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004406bc00 con 0x7f004406b830 2023-10-08T16:04:42.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f004a6e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00440c93d0 con 0x7f0044162250 2023-10-08T16:04:42.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f0043fff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f004406b830 0x7f00440cb440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55988/0 (socket says 172.21.15.170:55988) 2023-10-08T16:04:42.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7f0043fff700 1 -- 172.21.15.170:0/2532069015 learned_addr learned my addr 172.21.15.170:0/2532069015 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:42.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.943+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff278017430 con 0x7ff2800c91e0 2023-10-08T16:04:42.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2600052b0 con 0x7ff2800c91e0 2023-10-08T16:04:42.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f0048c84700 1 --2- 172.21.15.170:0/2532069015 >> v2:172.21.15.170:3300/0 conn(0x7f0044162250 0x7f0044168ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f560f03d700 1 Processor -- start 2023-10-08T16:04:42.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f0042ffd700 1 -- 172.21.15.170:0/2532069015 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855700662 0 0) 0x7f004406bc00 con 0x7f004406b830 2023-10-08T16:04:42.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f0042ffd700 1 -- 172.21.15.170:0/2532069015 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c003570 con 0x7f004406b830 2023-10-08T16:04:42.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f0048c84700 1 -- 172.21.15.170:0/2532069015 >> v1:172.21.15.191:6789/0 conn(0x7f004406b830 legacy=0x7f00440cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.944+0000 7f0048c84700 1 -- 172.21.15.170:0/2532069015 >> v1:172.21.15.170:6789/0 conn(0x7f00440c8f50 legacy=0x7f0044161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.945+0000 7f0048c84700 1 -- 172.21.15.170:0/2532069015 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0044158a30 con 0x7f0044162250 2023-10-08T16:04:42.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.945+0000 7f560f03d700 1 -- start start 2023-10-08T16:04:42.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.945+0000 7f0048c84700 1 --2- 172.21.15.170:0/2532069015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044168ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f003c009e60 tx=0x7f003c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427c9ec87d47ca41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:42.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f560806b830 0x7f5608142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5608162080 0x7f56080c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c6f30 con 0x7f56080cb440 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56081401b0 con 0x7f5608162080 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560f03d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c63b0 con 0x7f560806b830 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46502/0 (socket says 172.21.15.170:46502) 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 -- 172.21.15.170:0/3263574670 learned_addr learned my addr 172.21.15.170:0/3263574670 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f560d5da700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5608162080 0x7f56080c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 -- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f560806b830 msgr2=0x7f5608142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:42.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f560806b830 0x7f5608142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 -- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5608162080 msgr2=0x7f56080c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5608162080 0x7f56080c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7f5607fff700 1 -- 172.21.15.170:0/3263574670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009ae0 con 0x7f56080cb440 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff278003020 con 0x7ff2800c91e0 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7ff266ffd700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 0x7ff26c06b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.946+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff278013070 con 0x7ff2800c91e0 2023-10-08T16:04:43.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.947+0000 7ff27effd700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 0x7ff26c06b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.947+0000 7f5607fff700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55f800ec80 tx=0x7f55f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.947+0000 7ff27effd700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 0x7ff26c06b5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff270009160 tx=0x7ff270016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.947+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f4401b080 con 0x7f3f38004810 2023-10-08T16:04:43.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.948+0000 7f0042ffd700 1 -- 172.21.15.170:0/2532069015 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00e070 con 0x7f0044162250 2023-10-08T16:04:43.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.948+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f914c024610 con 0x7f91580ca740 2023-10-08T16:04:43.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.949+0000 7f9154ff9700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 0x7f91440694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.949+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f914c002e40 con 0x7f91580ca740 2023-10-08T16:04:43.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.949+0000 7f915ce47700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 0x7f91440694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.949+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f914c096110 con 0x7f91580ca740 2023-10-08T16:04:43.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.949+0000 7f915ce47700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 0x7f91440694b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f915000f5c0 tx=0x7f9150007740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.950+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff278052330 con 0x7ff2800c91e0 2023-10-08T16:04:43.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.953+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800c850 con 0x7f56080cb440 2023-10-08T16:04:43.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.953+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f800c9b0 con 0x7f56080cb440 2023-10-08T16:04:43.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.953+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800cd50 con 0x7f56080cb440 2023-10-08T16:04:43.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc009420 con 0x7f56080cb440 2023-10-08T16:04:43.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56080c6970 con 0x7f56080cb440 2023-10-08T16:04:43.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f088001a440 con 0x7f088c159720 2023-10-08T16:04:43.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f08737fe700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 0x7f0878069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0880021070 con 0x7f088c159720 2023-10-08T16:04:43.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f088b7fe700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 0x7f0878069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.955+0000 7f088b7fe700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 0x7f0878069460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087c009b80 tx=0x7f087c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f0042ffd700 1 -- 172.21.15.170:0/2532069015 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c00bbf0 con 0x7f0044162250 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.954+0000 7f0042ffd700 1 -- 172.21.15.170:0/2532069015 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c012430 con 0x7f0044162250 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.956+0000 7f004a6e7700 1 -- 172.21.15.170:0/2532069015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 msgr2=0x7f0044168ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.956+0000 7f004a6e7700 1 --2- 172.21.15.170:0/2532069015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044168ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f003c009e60 tx=0x7f003c009780 comp rx=0 tx=0).stop 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.957+0000 7f004a6e7700 1 -- 172.21.15.170:0/2532069015 shutdown_connections 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.957+0000 7f004a6e7700 1 --2- 172.21.15.170:0/2532069015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044168ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.957+0000 7f004a6e7700 1 -- 172.21.15.170:0/2532069015 >> 172.21.15.170:0/2532069015 conn(0x7f00440c9870 msgr2=0x7f00440c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.957+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0880053cb0 con 0x7f088c159720 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.959+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55f8013680 con 0x7f56080cb440 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.959+0000 7f5604ff9700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 0x7f55f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.960+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55f8084830 con 0x7f56080cb440 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.960+0000 7f560cdd9700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 0x7f55f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.960+0000 7f560cdd9700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 0x7f55f40693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f560000e400 tx=0x7f560000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.961+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560804c3c0 con 0x7f56080cb440 2023-10-08T16:04:43.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.969+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55f8084020 con 0x7f56080cb440 2023-10-08T16:04:43.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.970+0000 7f004a6e7700 1 -- 172.21.15.170:0/2532069015 shutdown_connections 2023-10-08T16:04:43.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.970+0000 7f004a6e7700 1 -- 172.21.15.170:0/2532069015 wait complete. 2023-10-08T16:04:43.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.972+0000 7f004a6e7700 1 Processor -- start 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.972+0000 7f004a6e7700 1 -- start start 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004406b830 0x7f004415e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 0x7f0044148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044144130 con 0x7f0044162250 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044143800 con 0x7f004406b830 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.973+0000 7f004a6e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0044143ae0 con 0x7f00440c8f50 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.974+0000 7f00437fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 0x7f0044148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.974+0000 7f00437fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 0x7f0044148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35548/0 (socket says 172.21.15.170:35548) 2023-10-08T16:04:43.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.974+0000 7f00437fe700 1 -- 172.21.15.170:0/1213447642 learned_addr learned my addr 172.21.15.170:0/1213447642 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.974+0000 7f0048c84700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0043fff700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004406b830 0x7f004415e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 -- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 msgr2=0x7f0044148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 0x7f0044148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 -- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004406b830 msgr2=0x7f004415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004406b830 0x7f004415e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 -- 172.21.15.170:0/1213447642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009a20 con 0x7f0044162250 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.975+0000 7f0048c84700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044148ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f003c016000 tx=0x7f003c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.976+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c012de0 con 0x7f0044162250 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.976+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c021070 con 0x7f0044162250 2023-10-08T16:04:43.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.976+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c026680 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.976+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0044143dc0 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.976+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004415eb40 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.980+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f003c023030 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.980+0000 7f002bfff700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 0x7f0030069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.980+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f003c0866d0 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.981+0000 7f0043fff700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 0x7f0030069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.981+0000 7f0043fff700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 0x7f0030069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00380168e0 tx=0x7f003801a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.983+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00240052b0 con 0x7f0044162250 2023-10-08T16:04:43.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:42.992+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f003c053920 con 0x7f0044162250 2023-10-08T16:04:43.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.326+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f234004c3c0 con 0x7f234006a1c0 2023-10-08T16:04:43.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.327+0000 7f23277fe700 1 -- 172.21.15.170:0/1326895222 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2334056f20 con 0x7f234006a1c0 2023-10-08T16:04:43.332 INFO:teuthology.orchestra.run.smithi170.stdout:55834574852 2023-10-08T16:04:43.332 INFO:teuthology.orchestra.run.smithi170.stdout:55834574852 2023-10-08T16:04:43.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.327+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0ec00acfd0 con 0x7f0ec00875d0 2023-10-08T16:04:43.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.327+0000 7f0ea77fe700 1 -- 172.21.15.170:0/3339745374 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0eb40549b0 con 0x7f0ec00875d0 2023-10-08T16:04:43.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 msgr2=0x7f232c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 0x7f232c0693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2330006fd0 tx=0x7f2330006d30 comp rx=0 tx=0).stop 2023-10-08T16:04:43.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 msgr2=0x7f234013d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2334003080 tx=0x7f2334015000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 shutdown_connections 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2340140720 0x7f234015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f232c066f30 0x7f232c0693e0 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2330006fd0 tx=0x7f2330006d30 comp rx=0 tx=0).stop 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f234006a1c0 0x7f234013d4b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 --2- 172.21.15.170:0/1326895222 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2340140350 0x7f23400c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 >> 172.21.15.170:0/1326895222 conn(0x7f23400cd670 msgr2=0x7f23400c3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 msgr2=0x7f0eac0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 0x7f0eac0694b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006fd0 tx=0x7f0eb000a330 comp rx=0 tx=0).stop 2023-10-08T16:04:43.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 msgr2=0x7f0ec0175ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.331+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400b630 tx=0x7f0eb40091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.332+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 shutdown_connections 2023-10-08T16:04:43.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.332+0000 7f2345e11700 1 -- 172.21.15.170:0/1326895222 wait complete. 2023-10-08T16:04:43.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.334+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f91580ca420 con 0x7f91580ca740 2023-10-08T16:04:43.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.335+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f3f500cd470 con 0x7f3f38004810 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 shutdown_connections 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0eac067000 0x7f0eac0694b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec00b1c50 0x7f0ec0165220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0ec0068aa0 0x7f0ec017a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 --2- 172.21.15.170:0/3339745374 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0ec00875d0 0x7f0ec0175ca0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 >> 172.21.15.170:0/3339745374 conn(0x7f0ec00cd000 msgr2=0x7f0ec00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f3f37fff700 1 -- 172.21.15.170:0/2368991773 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3f44051cb0 con 0x7f3f38004810 2023-10-08T16:04:43.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 shutdown_connections 2023-10-08T16:04:43.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.336+0000 7f0ec6088700 1 -- 172.21.15.170:0/3339745374 wait complete. 2023-10-08T16:04:43.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.338+0000 7f9154ff9700 1 -- 172.21.15.170:0/130394817 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f914c028020 con 0x7f91580ca740 2023-10-08T16:04:43.339 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:43.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.338+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff260005830 con 0x7ff2800c91e0 2023-10-08T16:04:43.341 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:43.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.342+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 msgr2=0x7f3f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.342+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 0x7f3f3c0693e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f40008fd0 tx=0x7f3f40015000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.342+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 msgr2=0x7f3f50153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.342+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400cec0 tx=0x7f3f44007780 comp rx=0 tx=0).stop 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 shutdown_connections 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f3c066f30 0x7f3f3c0693e0 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3f40008fd0 tx=0x7f3f40015000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f50153b80 0x7f3f50153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f38004810 0x7f3f50153570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 --2- 172.21.15.170:0/2368991773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f50155b70 0x7f3f50141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.343+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 >> 172.21.15.170:0/2368991773 conn(0x7f3f500cd000 msgr2=0x7f3f5015b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 msgr2=0x7f91440694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 0x7f91440694b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f915000f5c0 tx=0x7f9150007740 comp rx=0 tx=0).stop 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 msgr2=0x7f91580c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 0x7f91580c8f30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f914c020b00 tx=0x7f914c001da0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 shutdown_connections 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f91581410a0 0x7f91580a73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f91400052e0 0x7f91580cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9144067000 0x7f91440694b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 --2- 172.21.15.170:0/130394817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91580ca740 0x7f91580c8f30 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f914c020b00 tx=0x7f914c001da0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 >> 172.21.15.170:0/130394817 conn(0x7f91580cd000 msgr2=0x7f9158142a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f088c0adbf0 con 0x7f088c159720 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 shutdown_connections 2023-10-08T16:04:43.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f3f56283700 1 -- 172.21.15.170:0/2368991773 wait complete. 2023-10-08T16:04:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7f08737fe700 1 -- 172.21.15.170:0/823453960 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f088c0adbf0 con 0x7f088c159720 2023-10-08T16:04:43.349 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:43.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.346+0000 7ff266ffd700 1 -- 172.21.15.170:0/1886917436 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff278055950 con 0x7ff2800c91e0 2023-10-08T16:04:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.349+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 msgr2=0x7f0878069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.349+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 0x7f0878069460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f087c009b80 tx=0x7f087c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.349+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 msgr2=0x7f088c14cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.349+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 0x7f088c14cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f088c063d40 tx=0x7f0880016000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 shutdown_connections 2023-10-08T16:04:43.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f088c159af0 0x7f088c14d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f088c0ad250 0x7f088c14c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0878066fb0 0x7f0878069460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 --2- 172.21.15.170:0/823453960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f088c159720 0x7f088c14cc70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 >> 172.21.15.170:0/823453960 conn(0x7f088c06a320 msgr2=0x7f088c0ac260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 shutdown_connections 2023-10-08T16:04:43.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.350+0000 7f0891cce700 1 -- 172.21.15.170:0/823453960 wait complete. 2023-10-08T16:04:43.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.352+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 shutdown_connections 2023-10-08T16:04:43.353 INFO:teuthology.orchestra.run.smithi170.stdout:55834574852 2023-10-08T16:04:43.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.352+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5608161630 con 0x7f56080cb440 2023-10-08T16:04:43.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.352+0000 7f915f0ab700 1 -- 172.21.15.170:0/130394817 wait complete. 2023-10-08T16:04:43.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.354+0000 7f5604ff9700 1 -- 172.21.15.170:0/3263574670 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55f8021090 con 0x7f56080cb440 2023-10-08T16:04:43.355 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 msgr2=0x7ff26c06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 0x7ff26c06b5f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff270009160 tx=0x7ff270016000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 msgr2=0x7ff280155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff27800cdb0 tx=0x7ff2780077d0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 shutdown_connections 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2800c91e0 0x7ff280155050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff26c069140 0x7ff26c06b5f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff28006aa40 0x7ff28015b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 --2- 172.21.15.170:0/1886917436 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff28006bac0 0x7ff280167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 >> 172.21.15.170:0/1886917436 conn(0x7ff2800cd670 msgr2=0x7ff2800c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 shutdown_connections 2023-10-08T16:04:43.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.355+0000 7ff285538700 1 -- 172.21.15.170:0/1886917436 wait complete. 2023-10-08T16:04:43.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.357+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0024005560 con 0x7f0044162250 2023-10-08T16:04:43.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.358+0000 7f002bfff700 1 -- 172.21.15.170:0/1213447642 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f003c056f40 con 0x7f0044162250 2023-10-08T16:04:43.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-10-08T16:04:43.360 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:43.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-10-08T16:04:43.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 msgr2=0x7f55f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 0x7f55f40693e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f560000e400 tx=0x7f560000b040 comp rx=0 tx=0).stop 2023-10-08T16:04:43.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 msgr2=0x7f56080c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f55f800ec80 tx=0x7f55f80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 shutdown_connections 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5608162080 0x7f56080c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f560806b830 0x7f5608142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f55f4066f30 0x7f55f40693e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 --2- 172.21.15.170:0/3263574670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56080cb440 0x7f56080c4070 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.361+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 >> 172.21.15.170:0/3263574670 conn(0x7f56080c9870 msgr2=0x7f56080c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 msgr2=0x7f0030069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 0x7f0030069530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00380168e0 tx=0x7f003801a040 comp rx=0 tx=0).stop 2023-10-08T16:04:43.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 msgr2=0x7f0044148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044148ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f003c016000 tx=0x7f003c019000 comp rx=0 tx=0).stop 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 shutdown_connections 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067080 0x7f0030069530 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0044162250 0x7f0044148ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004406b830 0x7f004415e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 --2- 172.21.15.170:0/1213447642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00440c8f50 0x7f0044148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 >> 172.21.15.170:0/1213447642 conn(0x7f00440c9870 msgr2=0x7f0044069450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 shutdown_connections 2023-10-08T16:04:43.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.362+0000 7f004a6e7700 1 -- 172.21.15.170:0/1213447642 wait complete. 2023-10-08T16:04:43.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.368+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 shutdown_connections 2023-10-08T16:04:43.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:43.368+0000 7f560f03d700 1 -- 172.21.15.170:0/3263574670 wait complete. 2023-10-08T16:04:43.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-08T16:04:43.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-10-08T16:04:43.378 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-08T16:04:43.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-10-08T16:04:43.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-08T16:04:43.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-08T16:04:44.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T16:04:44.362 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T16:04:44.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T16:04:44.377 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T16:04:44.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T16:04:44.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T16:04:44.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T16:04:44.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T16:04:44.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.531+0000 7f78de4d6700 1 Processor -- start 2023-10-08T16:04:44.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.531+0000 7f78de4d6700 1 -- start start 2023-10-08T16:04:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.532+0000 7f78de4d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 0x7f78d80879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78de4d6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f78d8068aa0 0x7f78d8068f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78de4d6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80acdf0 con 0x7f78d80b1c50 2023-10-08T16:04:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78de4d6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80abfd0 con 0x7f78d8068aa0 2023-10-08T16:04:44.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78de4d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8069460 con 0x7f78d80875d0 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78d77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f78d80b1c50 0x7f78d80ab8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54838/0 (socket says 172.21.15.170:54838) 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78d77fe700 1 -- 172.21.15.170:0/3973459599 learned_addr learned my addr 172.21.15.170:0/3973459599 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.533+0000 7f78d7fff700 1 --2- 172.21.15.170:0/3973459599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 0x7f78d80879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78dca73700 1 --2- 172.21.15.170:0/3973459599 >> v2:172.21.15.191:3300/0 conn(0x7f78d8068aa0 0x7f78d8068f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d6ffd700 1 -- 172.21.15.170:0/3973459599 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2419005913 0 0) 0x7f78d80acdf0 con 0x7f78d80b1c50 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d6ffd700 1 -- 172.21.15.170:0/3973459599 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c0003570 con 0x7f78d80b1c50 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d7fff700 1 -- 172.21.15.170:0/3973459599 >> v2:172.21.15.191:3300/0 conn(0x7f78d8068aa0 msgr2=0x7f78d8068f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d7fff700 1 --2- 172.21.15.170:0/3973459599 >> v2:172.21.15.191:3300/0 conn(0x7f78d8068aa0 0x7f78d8068f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d7fff700 1 -- 172.21.15.170:0/3973459599 >> v1:172.21.15.170:6789/0 conn(0x7f78d80b1c50 legacy=0x7f78d80ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.534+0000 7f78d7fff700 1 -- 172.21.15.170:0/3973459599 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80ac1e0 con 0x7f78d80875d0 2023-10-08T16:04:44.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.536+0000 7f78d7fff700 1 --2- 172.21.15.170:0/3973459599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 0x7f78d80879a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d00 tx=0x7f78d00095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97ecda561e5dd64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.536+0000 7f78d6ffd700 1 -- 172.21.15.170:0/3973459599 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0009e60 con 0x7f78d80875d0 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.536+0000 7f78d6ffd700 1 -- 172.21.15.170:0/3973459599 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000bd90 con 0x7f78d80875d0 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.536+0000 7f78d6ffd700 1 -- 172.21.15.170:0/3973459599 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000e670 con 0x7f78d80875d0 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 -- 172.21.15.170:0/3973459599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 msgr2=0x7f78d80879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 --2- 172.21.15.170:0/3973459599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 0x7f78d80879a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009d00 tx=0x7f78d00095a0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 -- 172.21.15.170:0/3973459599 shutdown_connections 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 --2- 172.21.15.170:0/3973459599 >> v2:172.21.15.191:3300/0 conn(0x7f78d8068aa0 0x7f78d8068f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 --2- 172.21.15.170:0/3973459599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d80875d0 0x7f78d80879a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 -- 172.21.15.170:0/3973459599 >> 172.21.15.170:0/3973459599 conn(0x7f78d80cd000 msgr2=0x7f78d8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 -- 172.21.15.170:0/3973459599 shutdown_connections 2023-10-08T16:04:44.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.537+0000 7f78de4d6700 1 -- 172.21.15.170:0/3973459599 wait complete. 2023-10-08T16:04:44.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.539+0000 7f78de4d6700 1 Processor -- start 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.540+0000 7f78de4d6700 1 -- start start 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d8068aa0 0x7f78d81875b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78d80875d0 0x7f78d816c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d816eda0 con 0x7f78d80b1c50 2023-10-08T16:04:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80a9ad0 con 0x7f78d80875d0 2023-10-08T16:04:44.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78de4d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80a9db0 con 0x7f78d8068aa0 2023-10-08T16:04:44.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78dca73700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78dca73700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46522/0 (socket says 172.21.15.170:46522) 2023-10-08T16:04:44.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78dca73700 1 -- 172.21.15.170:0/948451996 learned_addr learned my addr 172.21.15.170:0/948451996 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.541+0000 7f78d7fff700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d8068aa0 0x7f78d81875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 -- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d8068aa0 msgr2=0x7f78d81875b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d8068aa0 0x7f78d81875b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 -- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78d80875d0 msgr2=0x7f78d816c730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78d80875d0 0x7f78d816c730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 -- 172.21.15.170:0/948451996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009840 con 0x7f78d80b1c50 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.542+0000 7f78dca73700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c880 tx=0x7f78cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.543+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011600 con 0x7f78d80b1c50 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.543+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc011760 con 0x7f78d80b1c50 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.543+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011ae0 con 0x7f78d80b1c50 2023-10-08T16:04:44.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.543+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d816ea30 con 0x7f78d80b1c50 2023-10-08T16:04:44.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.543+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d8187af0 con 0x7f78d80b1c50 2023-10-08T16:04:44.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.546+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c40052e0 con 0x7f78d80b1c50 2023-10-08T16:04:44.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.546+0000 7fca693bf700 1 Processor -- start 2023-10-08T16:04:44.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.546+0000 7fca693bf700 1 -- start start 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.547+0000 7fca693bf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fca640b0010 0x7fca6415d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.547+0000 7fca693bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415dd90 0x7fca640adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca693bf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca640b0750 con 0x7fca640afc40 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca693bf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca640aca70 con 0x7fca640b0010 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca693bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca640acd50 con 0x7fca6415dd90 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca62ffd700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fca640afc40 0x7fca640b1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55990/0 (socket says 172.21.15.170:55990) 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca62ffd700 1 -- 172.21.15.170:0/1743911931 learned_addr learned my addr 172.21.15.170:0/1743911931 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7fca637fe700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415dd90 0x7fca640adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.546+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f78cc021350 con 0x7f78d80b1c50 2023-10-08T16:04:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.546+0000 7f78bffff700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 0x7f78c00699b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.547+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f78cc0841c0 con 0x7f78d80b1c50 2023-10-08T16:04:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.548+0000 7f78d7fff700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 0x7f78c00699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.550+0000 7f78d7fff700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 0x7f78c00699b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f78d00095e0 tx=0x7f78d0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356633253 0 0) 0x7fca640b0750 con 0x7fca640afc40 2023-10-08T16:04:44.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca4c003570 con 0x7fca640afc40 2023-10-08T16:04:44.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388854098 0 0) 0x7fca4c003570 con 0x7fca640afc40 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415dd90 msgr2=0x7fca640adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415dd90 0x7fca640adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 >> v2:172.21.15.170:3300/0 conn(0x7fca640b0010 msgr2=0x7fca6415d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 --2- 172.21.15.170:0/1743911931 >> v2:172.21.15.170:3300/0 conn(0x7fca640b0010 0x7fca6415d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640ad030 con 0x7fca640afc40 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fca54001470 con 0x7fca640afc40 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 >> v1:172.21.15.191:6789/0 conn(0x7fca640afc40 legacy=0x7fca640b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca4c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c004810 0x7fca4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c005230 0x7fca4c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640aca70 con 0x7fca4c003ae0 2023-10-08T16:04:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640acd50 con 0x7fca4c005230 2023-10-08T16:04:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.552+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c00c1b0 con 0x7fca4c004810 2023-10-08T16:04:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca4c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 -- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c004810 msgr2=0x7fca4c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca4c004810 0x7fca4c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 -- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c005230 msgr2=0x7fca4c00ba60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca4c005230 0x7fca4c00ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.553+0000 7fca62ffd700 1 -- 172.21.15.170:0/1743911931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640ad030 con 0x7fca4c003ae0 2023-10-08T16:04:44.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.554+0000 7fca62ffd700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca4c003ef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c410 tx=0x7fca54007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4548be3e409e8aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.556+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400ccc0 con 0x7fca4c003ae0 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.556+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400ce20 con 0x7fca4c003ae0 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.556+0000 7fca61ffb700 1 -- 172.21.15.170:0/1743911931 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54005b30 con 0x7fca4c003ae0 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.557+0000 7fca693bf700 1 -- 172.21.15.170:0/1743911931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 msgr2=0x7fca4c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.557+0000 7fca693bf700 1 --2- 172.21.15.170:0/1743911931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca4c003ef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fca5400c410 tx=0x7fca54007940 comp rx=0 tx=0).stop 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.557+0000 7fca637fe700 1 -- 172.21.15.170:0/1743911931 reap_dead start 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.557+0000 7fca693bf700 1 -- 172.21.15.170:0/1743911931 shutdown_connections 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.557+0000 7fca693bf700 1 -- 172.21.15.170:0/1743911931 >> 172.21.15.170:0/1743911931 conn(0x7fca6406a320 msgr2=0x7fca640cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.558+0000 7fca693bf700 1 -- 172.21.15.170:0/1743911931 shutdown_connections 2023-10-08T16:04:44.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.558+0000 7fca693bf700 1 -- 172.21.15.170:0/1743911931 wait complete. 2023-10-08T16:04:44.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.560+0000 7fca693bf700 1 Processor -- start 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.560+0000 7fca693bf700 1 -- start start 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca640a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca640a7c90 0x7fca640a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640a6380 con 0x7fca4c003ae0 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6415fed0 con 0x7fca640a7c90 2023-10-08T16:04:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.561+0000 7fca693bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca64161f90 con 0x7fca64159a40 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35598/0 (socket says 172.21.15.170:35598) 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 -- 172.21.15.170:0/3608091150 learned_addr learned my addr 172.21.15.170:0/3608091150 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca637fe700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca640a7c90 0x7fca640a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 -- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca640a7c90 msgr2=0x7fca640a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca640a7c90 0x7fca640a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 -- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 msgr2=0x7fca640a7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca640a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 -- 172.21.15.170:0/3608091150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca64160730 con 0x7fca64159a40 2023-10-08T16:04:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.562+0000 7fca627fc700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bc60 tx=0x7fca5c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.563+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0092e0 con 0x7fca64159a40 2023-10-08T16:04:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.563+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00c680 con 0x7fca64159a40 2023-10-08T16:04:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.563+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00ca20 con 0x7fca64159a40 2023-10-08T16:04:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.563+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca640a60a0 con 0x7fca64159a40 2023-10-08T16:04:44.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.563+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca641603c0 con 0x7fca64159a40 2023-10-08T16:04:44.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.556+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f78cc054e20 con 0x7f78d80b1c50 2023-10-08T16:04:44.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.566+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca5c012070 con 0x7fca64159a40 2023-10-08T16:04:44.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.566+0000 7fca4affd700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 0x7fca4c06aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.566+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca5c0840c0 con 0x7fca64159a40 2023-10-08T16:04:44.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.570+0000 7fca62ffd700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 0x7fca4c06aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.571+0000 7fca62ffd700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 0x7fca4c06aff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca54015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.574+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca500052e0 con 0x7fca64159a40 2023-10-08T16:04:44.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.583+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca5c054da0 con 0x7fca64159a40 2023-10-08T16:04:44.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.605+0000 7efe3cf36700 1 Processor -- start 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.605+0000 7efe3cf36700 1 -- start start 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3cf36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3cf36700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe380b21b0 0x7efe380ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3cf36700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe38069450 con 0x7efe38068fe0 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3cf36700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe380aef20 con 0x7efe380b21b0 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3cf36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3815f260 con 0x7efe38158440 2023-10-08T16:04:44.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7fddd87a6700 1 Processor -- start 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35600/0 (socket says 172.21.15.170:35600) 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7efe3659c700 1 -- 172.21.15.170:0/213250462 learned_addr learned my addr 172.21.15.170:0/213250462 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3659c700 1 -- 172.21.15.170:0/213250462 >> v2:172.21.15.170:3300/0 conn(0x7efe380b21b0 msgr2=0x7efe380ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3659c700 1 --2- 172.21.15.170:0/213250462 >> v2:172.21.15.170:3300/0 conn(0x7efe380b21b0 0x7efe380ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3659c700 1 -- 172.21.15.170:0/213250462 >> v1:172.21.15.191:6789/0 conn(0x7efe38068fe0 legacy=0x7efe380b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3659c700 1 -- 172.21.15.170:0/213250462 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3815e380 con 0x7efe38158440 2023-10-08T16:04:44.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.607+0000 7fddd87a6700 1 -- start start 2023-10-08T16:04:44.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3659c700 1 --2- 172.21.15.170:0/213250462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe280099b0 tx=0x7efe28009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c10f48f2a0d32f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7f9043d30700 1 Processor -- start 2023-10-08T16:04:44.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3559a700 1 -- 172.21.15.170:0/213250462 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800b820 con 0x7efe38158440 2023-10-08T16:04:44.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3559a700 1 -- 172.21.15.170:0/213250462 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2800b980 con 0x7efe38158440 2023-10-08T16:04:44.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.608+0000 7efe3559a700 1 -- 172.21.15.170:0/213250462 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800bd00 con 0x7efe38158440 2023-10-08T16:04:44.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7f9043d30700 1 -- start start 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd87a6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fddd00691b0 0x7fddd00ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd87a6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd00b2490 con 0x7fddd0156050 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd87a6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd00ac4e0 con 0x7fddd00cba20 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd87a6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00aceb0 con 0x7fddd00691b0 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd6542700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fddd00cba20 0x7fddd0068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:56004/0 (socket says 172.21.15.170:56004) 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd6542700 1 -- 172.21.15.170:0/2292183475 learned_addr learned my addr 172.21.15.170:0/2292183475 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7fddd5d41700 1 --2- 172.21.15.170:0/2292183475 >> v2:172.21.15.191:3300/0 conn(0x7fddd00691b0 0x7fddd00ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7efe3cf36700 1 -- 172.21.15.170:0/213250462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 msgr2=0x7efe38068aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.609+0000 7efe3cf36700 1 --2- 172.21.15.170:0/213250462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe280099b0 tx=0x7efe28009250 comp rx=0 tx=0).stop 2023-10-08T16:04:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210312580 0 0) 0x7fddd00ac4e0 con 0x7fddd00cba20 2023-10-08T16:04:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 -- 172.21.15.170:0/213250462 shutdown_connections 2023-10-08T16:04:44.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 --2- 172.21.15.170:0/213250462 >> v2:172.21.15.170:3300/0 conn(0x7efe380b21b0 0x7efe380ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 --2- 172.21.15.170:0/213250462 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe38158440 0x7efe38068aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 -- 172.21.15.170:0/213250462 >> 172.21.15.170:0/213250462 conn(0x7efe380cd000 msgr2=0x7efe38068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddbc003490 con 0x7fddd00cba20 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027594737 0 0) 0x7fddc8001490 con 0x7fddd0156050 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddd00ac4e0 con 0x7fddd0156050 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.1 v1:172.21.15.191:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031450296 0 0) 0x7fddbc003490 con 0x7fddd00cba20 2023-10-08T16:04:44.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 >> v2:172.21.15.191:3300/0 conn(0x7fddd00691b0 msgr2=0x7fddd00ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 --2- 172.21.15.170:0/2292183475 >> v2:172.21.15.191:3300/0 conn(0x7fddd00691b0 0x7fddd00ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 >> v1:172.21.15.170:6789/0 conn(0x7fddd0156050 legacy=0x7fddd01564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> v1:172.21.15.191:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0156cd0 con 0x7fddd00cba20 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.1 v1:172.21.15.191:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fddcc001470 con 0x7fddd00cba20 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5d41700 1 --2- 172.21.15.170:0/2292183475 >> v2:172.21.15.191:3300/0 conn(0x7fddd00691b0 0x7fddd00ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 >> v1:172.21.15.191:6789/0 conn(0x7fddd00cba20 legacy=0x7fddd0068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddbc004f10 0x7fddbc0052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc005820 0x7fddbc00a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 0x7fddbc00ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00aceb0 con 0x7fddbc00a690 2023-10-08T16:04:44.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc00d0f0 con 0x7fddbc005820 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddbc00d2b0 con 0x7fddbc004f10 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 -- 172.21.15.170:0/213250462 shutdown_connections 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7efe3cf36700 1 -- 172.21.15.170:0/213250462 wait complete. 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9043d30700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f903c159890 0x7f903c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9043d30700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c15a160 con 0x7f903c069560 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9043d30700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c0d13a0 con 0x7f903c159c60 2023-10-08T16:04:44.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9043d30700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903c15e140 con 0x7f903c159890 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9041acc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f903c159890 0x7f903c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9041acc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f903c159890 0x7f903c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46564/0 (socket says 172.21.15.170:46564) 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.610+0000 7f9041acc700 1 -- 172.21.15.170:0/319004518 learned_addr learned my addr 172.21.15.170:0/319004518 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9040aca700 1 -- 172.21.15.170:0/319004518 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982175643 0 0) 0x7f903c0d13a0 con 0x7f903c159c60 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9040aca700 1 -- 172.21.15.170:0/319004518 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9028003570 con 0x7f903c159c60 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9041acc700 1 -- 172.21.15.170:0/319004518 >> v1:172.21.15.191:6789/0 conn(0x7f903c159c60 legacy=0x7f903c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9041acc700 1 -- 172.21.15.170:0/319004518 >> v1:172.21.15.170:6789/0 conn(0x7f903c069560 legacy=0x7f903c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9041acc700 1 -- 172.21.15.170:0/319004518 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c15a3d0 con 0x7f903c159890 2023-10-08T16:04:44.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7fddd6d43700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 0x7fddbc00ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9041acc700 1 --2- 172.21.15.170:0/319004518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159890 0x7f903c15abd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90380099b0 tx=0x7f9038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a9ccfa48f9c4b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7efe3cf36700 1 Processor -- start 2023-10-08T16:04:44.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.611+0000 7f9040aca700 1 -- 172.21.15.170:0/319004518 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038009ea0 con 0x7f903c159890 2023-10-08T16:04:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7f9040aca700 1 -- 172.21.15.170:0/319004518 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903800b860 con 0x7f903c159890 2023-10-08T16:04:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7f9040aca700 1 -- 172.21.15.170:0/319004518 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903800bc00 con 0x7f903c159890 2023-10-08T16:04:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 -- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddbc004f10 msgr2=0x7fddbc0052e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T16:04:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddbc004f10 0x7fddbc0052e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 -- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc005820 msgr2=0x7fddbc00a150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc005820 0x7fddbc00a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 -- 172.21.15.170:0/2292183475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0156cd0 con 0x7fddbc00a690 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 -- 172.21.15.170:0/2292183475 reap_dead start 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd6d43700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 0x7fddbc00ca70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c190 tx=0x7fddc8001960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580e3495b9055648 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc800ca80 con 0x7fddbc00a690 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc800cc00 con 0x7fddbc00a690 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7fddd5540700 1 -- 172.21.15.170:0/2292183475 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8012400 con 0x7fddbc00a690 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7f9043d30700 1 -- 172.21.15.170:0/319004518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159890 msgr2=0x7f903c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7f9043d30700 1 --2- 172.21.15.170:0/319004518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159890 0x7f903c15abd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f90380099b0 tx=0x7f9038009250 comp rx=0 tx=0).stop 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.612+0000 7efe3cf36700 1 -- start start 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe380b21b0 0x7efe38147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe38158440 0x7efe3806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe381494e0 con 0x7efe38068fe0 2023-10-08T16:04:44.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3813d1c0 con 0x7efe38158440 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3cf36700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe38143210 con 0x7efe380b21b0 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46588/0 (socket says 172.21.15.170:46588) 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 -- 172.21.15.170:0/2774858909 learned_addr learned my addr 172.21.15.170:0/2774858909 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe35d9b700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe380b21b0 0x7efe38147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe36d9d700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe38158440 0x7efe3806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7fddd87a6700 1 -- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 msgr2=0x7fddbc00ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7fddd87a6700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 0x7fddbc00ca70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c190 tx=0x7fddc8001960 comp rx=0 tx=0).stop 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 -- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe380b21b0 msgr2=0x7efe38147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe380b21b0 0x7efe38147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 -- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe38158440 msgr2=0x7efe3806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe38158440 0x7efe3806e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.613+0000 7efe3659c700 1 -- 172.21.15.170:0/2774858909 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280094f0 con 0x7efe38068fe0 2023-10-08T16:04:44.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7f9043d30700 1 -- 172.21.15.170:0/319004518 shutdown_connections 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7f9043d30700 1 --2- 172.21.15.170:0/319004518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159890 0x7f903c15abd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7f9043d30700 1 -- 172.21.15.170:0/319004518 >> 172.21.15.170:0/319004518 conn(0x7f903c068e20 msgr2=0x7f903c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7fddd87a6700 1 -- 172.21.15.170:0/2292183475 shutdown_connections 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7fddd87a6700 1 --2- 172.21.15.170:0/2292183475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddbc00a690 0x7fddbc00ca70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7fddd87a6700 1 -- 172.21.15.170:0/2292183475 >> 172.21.15.170:0/2292183475 conn(0x7fddd00cd000 msgr2=0x7fddd0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe3659c700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe28009290 tx=0x7efe28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7f9043d30700 1 -- 172.21.15.170:0/319004518 shutdown_connections 2023-10-08T16:04:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe28011bb0 con 0x7efe38068fe0 2023-10-08T16:04:44.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28020070 con 0x7efe38068fe0 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2801b450 con 0x7efe38068fe0 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe38142450 con 0x7efe38068fe0 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe38142a10 con 0x7efe38068fe0 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.614+0000 7f9043d30700 1 -- 172.21.15.170:0/319004518 wait complete. 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.616+0000 7f9043d30700 1 Processor -- start 2023-10-08T16:04:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.616+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe28022030 con 0x7efe38068fe0 2023-10-08T16:04:44.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f0c6411e700 1 Processor -- start 2023-10-08T16:04:44.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7efe267fc700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 0x7efe180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efe28086200 con 0x7efe38068fe0 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.616+0000 7f9043d30700 1 -- start start 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f903c069560 0x7f903c15cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 0x7f903c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15d820 con 0x7f903c159c60 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15db00 con 0x7f903c069560 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f9043d30700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c15dde0 con 0x7f903c159890 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f90422cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f90422cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46598/0 (socket says 172.21.15.170:46598) 2023-10-08T16:04:44.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f90422cd700 1 -- 172.21.15.170:0/105244010 learned_addr learned my addr 172.21.15.170:0/105244010 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f90412cb700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 0x7f903c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7fddd87a6700 1 -- 172.21.15.170:0/2292183475 shutdown_connections 2023-10-08T16:04:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7fddd87a6700 1 -- 172.21.15.170:0/2292183475 wait complete. 2023-10-08T16:04:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.617+0000 7f0c6411e700 1 -- start start 2023-10-08T16:04:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c6411e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0c5c06bcd0 0x7f0c5c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c6411e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0c5c0c9510 0x7f0c5c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c6411e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c15c2a0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c6411e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c0c9900 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c6411e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c15b690 con 0x7f0c5c0c9510 2023-10-08T16:04:44.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7efe35d9b700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 0x7efe180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 -- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 msgr2=0x7f903c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 0x7f903c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 -- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f903c069560 msgr2=0x7f903c15cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f903c069560 0x7f903c15cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 -- 172.21.15.170:0/105244010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380094f0 con 0x7f903c159c60 2023-10-08T16:04:44.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90412cb700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 0x7f903c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f90422cd700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f902c00c560 tx=0x7f902c007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011590 con 0x7f903c159c60 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c0116f0 con 0x7f903c159c60 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011ab0 con 0x7f903c159c60 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c0c7940 con 0x7f903c159c60 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c0cf560 con 0x7f903c159c60 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3804c3c0 con 0x7efe38068fe0 2023-10-08T16:04:44.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7efe35d9b700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 0x7efe180693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe3806f3c0 tx=0x7efe2c007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.620+0000 7fddd87a6700 1 Processor -- start 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c616b9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0c5c06c0a0 0x7f0c5c0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54852/0 (socket says 172.21.15.170:54852) 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c616b9700 1 -- 172.21.15.170:0/2680131960 learned_addr learned my addr 172.21.15.170:0/2680131960 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.618+0000 7f0c61eba700 1 --2- 172.21.15.170:0/2680131960 >> v2:172.21.15.170:3300/0 conn(0x7f0c5c06bcd0 0x7f0c5c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f0c626bb700 1 --2- 172.21.15.170:0/2680131960 >> v2:172.21.15.191:3300/0 conn(0x7f0c5c0c9510 0x7f0c5c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.619+0000 7f0c60eb8700 1 -- 172.21.15.170:0/2680131960 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547657150 0 0) 0x7f0c5c15c2a0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.620+0000 7fddd87a6700 1 -- start start 2023-10-08T16:04:44.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.620+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c0c7e60 con 0x7f903c159c60 2023-10-08T16:04:44.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7f8f7d839700 1 Processor -- start 2023-10-08T16:04:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7f8f7d839700 1 -- start start 2023-10-08T16:04:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc00a690 0x7fddd01352a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd006ef20 0x7fddd0159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01357e0 con 0x7fddd006eab0 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd006da10 con 0x7fddbc00a690 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd87a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd006dcf0 con 0x7fddd006ef20 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd5d41700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd5d41700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46622/0 (socket says 172.21.15.170:46622) 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.621+0000 7fddd5d41700 1 -- 172.21.15.170:0/1243460432 learned_addr learned my addr 172.21.15.170:0/1243460432 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 -- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd006ef20 msgr2=0x7fddd0159ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd006ef20 0x7fddd0159ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 -- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc00a690 msgr2=0x7fddd01352a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc00a690 0x7fddd01352a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 -- 172.21.15.170:0/1243460432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd006dfd0 con 0x7fddd006eab0 2023-10-08T16:04:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f7d839700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f78158c10 0x7f8f7806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f7d839700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8f7806da30 0x7f8f780c6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f7d839700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7806add0 con 0x7f8f78158840 2023-10-08T16:04:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f7d839700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7806f110 con 0x7f8f78158c10 2023-10-08T16:04:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f7d839700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7806de60 con 0x7f8f7806da30 2023-10-08T16:04:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f76ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f78158840 0x7f8f78069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54856/0 (socket says 172.21.15.170:54856) 2023-10-08T16:04:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f76ffd700 1 -- 172.21.15.170:0/349690131 learned_addr learned my addr 172.21.15.170:0/349690131 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f767fc700 1 --2- 172.21.15.170:0/349690131 >> v2:172.21.15.170:3300/0 conn(0x7f8f78158c10 0x7f8f7806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f777fe700 1 --2- 172.21.15.170:0/349690131 >> v2:172.21.15.191:3300/0 conn(0x7f8f7806da30 0x7f8f780c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7fddd5d41700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9d0 tx=0x7fddc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f767fc700 1 -- 172.21.15.170:0/349690131 >> v2:172.21.15.191:3300/0 conn(0x7f8f7806da30 msgr2=0x7f8f780c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f767fc700 1 --2- 172.21.15.170:0/349690131 >> v2:172.21.15.191:3300/0 conn(0x7f8f7806da30 0x7f8f780c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.622+0000 7f8f767fc700 1 -- 172.21.15.170:0/349690131 >> v1:172.21.15.170:6789/0 conn(0x7f8f78158840 legacy=0x7f8f78069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.623+0000 7f8f767fc700 1 -- 172.21.15.170:0/349690131 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7806a1c0 con 0x7f8f78158c10 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.623+0000 7f8f767fc700 1 --2- 172.21.15.170:0/349690131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f7806d4f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806cf70 tx=0x7f8f6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8aea848c2afc4c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.623+0000 7f8c1b8d3700 1 Processor -- start 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c60eb8700 1 -- 172.21.15.170:0/2680131960 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c48007980 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c61eba700 1 -- 172.21.15.170:0/2680131960 >> v2:172.21.15.191:3300/0 conn(0x7f0c5c0c9510 msgr2=0x7f0c5c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c61eba700 1 --2- 172.21.15.170:0/2680131960 >> v2:172.21.15.191:3300/0 conn(0x7f0c5c0c9510 0x7f0c5c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c61eba700 1 -- 172.21.15.170:0/2680131960 >> v1:172.21.15.170:6789/0 conn(0x7f0c5c06c0a0 legacy=0x7f0c5c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c61eba700 1 -- 172.21.15.170:0/2680131960 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c15b970 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f8c1b8d3700 1 -- start start 2023-10-08T16:04:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.624+0000 7f0c61eba700 1 --2- 172.21.15.170:0/2680131960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce190 tx=0x7f0c4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bc548773b41857 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1b8d3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8c14159c60 0x7f8c1415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1b8d3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1415a160 con 0x7f8c14159890 2023-10-08T16:04:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1b8d3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c140d13a0 con 0x7f8c1415db10 2023-10-08T16:04:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1b8d3700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1415df50 con 0x7f8c14159c60 2023-10-08T16:04:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f0c60eb8700 1 -- 172.21.15.170:0/2680131960 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011630 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1966f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8c14159890 0x7f8c1415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54858/0 (socket says 172.21.15.170:54858) 2023-10-08T16:04:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c18e6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8c14159c60 0x7f8c1415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7f8c1966f700 1 -- 172.21.15.170:0/4142780617 learned_addr learned my addr 172.21.15.170:0/4142780617 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7f8c18e6e700 1 -- 172.21.15.170:0/4142780617 >> v1:172.21.15.191:6789/0 conn(0x7f8c1415db10 legacy=0x7f8c14069560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7f8c18e6e700 1 -- 172.21.15.170:0/4142780617 >> v1:172.21.15.170:6789/0 conn(0x7f8c14159890 legacy=0x7f8c1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7f8c18e6e700 1 -- 172.21.15.170:0/4142780617 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1415e140 con 0x7f8c14159c60 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7f8c18e6e700 1 --2- 172.21.15.170:0/4142780617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c1415d5d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098e0 tx=0x7f8c040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdd21b2962d76c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000c850 con 0x7fddd006eab0 2023-10-08T16:04:44.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000c9b0 con 0x7fddd006eab0 2023-10-08T16:04:44.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000cd30 con 0x7fddd006eab0 2023-10-08T16:04:44.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.626+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f902c01b3b0 con 0x7f903c159c60 2023-10-08T16:04:44.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.627+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd006e2b0 con 0x7fddd006eab0 2023-10-08T16:04:44.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.627+0000 7f9031ffb700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 0x7f90200694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.627+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd014fb90 con 0x7fddd006eab0 2023-10-08T16:04:44.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.627+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f902c083da0 con 0x7f903c159c60 2023-10-08T16:04:44.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.628+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd004c3c0 con 0x7fddd006eab0 2023-10-08T16:04:44.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.625+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe28053470 con 0x7efe38068fe0 2023-10-08T16:04:44.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f9041acc700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 0x7f90200694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f8f75ffb700 1 -- 172.21.15.170:0/349690131 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c011630 con 0x7f8f78158c10 2023-10-08T16:04:44.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f8f75ffb700 1 -- 172.21.15.170:0/349690131 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c011790 con 0x7f8f78158c10 2023-10-08T16:04:44.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f8f75ffb700 1 -- 172.21.15.170:0/349690131 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c011b30 con 0x7f8f78158c10 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f9041acc700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 0x7f90200694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90380099b0 tx=0x7f9038017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f8f7d839700 1 -- 172.21.15.170:0/349690131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 msgr2=0x7f8f7806d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f8f7d839700 1 --2- 172.21.15.170:0/349690131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f7806d4f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806cf70 tx=0x7f8f6c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 -- 172.21.15.170:0/349690131 shutdown_connections 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 --2- 172.21.15.170:0/349690131 >> v2:172.21.15.191:3300/0 conn(0x7f8f7806da30 0x7f8f780c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 --2- 172.21.15.170:0/349690131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f7806d4f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 -- 172.21.15.170:0/349690131 >> 172.21.15.170:0/349690131 conn(0x7f8f780cc1e0 msgr2=0x7f8f780cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f902c051170 con 0x7f903c159c60 2023-10-08T16:04:44.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 -- 172.21.15.170:0/349690131 shutdown_connections 2023-10-08T16:04:44.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f8f7d839700 1 -- 172.21.15.170:0/349690131 wait complete. 2023-10-08T16:04:44.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f0c60eb8700 1 -- 172.21.15.170:0/2680131960 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c011790 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.630+0000 7f0c60eb8700 1 -- 172.21.15.170:0/2680131960 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011b10 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f0c6411e700 1 -- 172.21.15.170:0/2680131960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c06bcd0 msgr2=0x7f0c5c140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.631+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2680131960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce190 tx=0x7f0c4c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.633+0000 7f8f7d839700 1 Processor -- start 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.633+0000 7f8f7d839700 1 -- start start 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f0c6411e700 1 -- 172.21.15.170:0/2680131960 shutdown_connections 2023-10-08T16:04:44.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2680131960 >> v2:172.21.15.191:3300/0 conn(0x7f0c5c0c9510 0x7f0c5c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2680131960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140df0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f0c6411e700 1 -- 172.21.15.170:0/2680131960 >> 172.21.15.170:0/2680131960 conn(0x7f0c5c0ce600 msgr2=0x7f0c5c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8c0bfff700 1 -- 172.21.15.170:0/4142780617 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04009e80 con 0x7f8c14159c60 2023-10-08T16:04:44.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8c0bfff700 1 -- 172.21.15.170:0/4142780617 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0400b820 con 0x7f8c14159c60 2023-10-08T16:04:44.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8c0bfff700 1 -- 172.21.15.170:0/4142780617 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400bba0 con 0x7f8c14159c60 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.633+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddc0013680 con 0x7fddd006eab0 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7fddc67fc700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 0x7fddb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fddc0084cd0 con 0x7fddd006eab0 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f78158840 0x7f8f780c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f780b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f78177de0 con 0x7f8f78158c10 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f78176f30 con 0x7f8f78158840 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7f8f7d839700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f78157d60 con 0x7f8f7806da30 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.634+0000 7fddd6542700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 0x7fddb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.635+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/4142780617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 msgr2=0x7f8c1415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.635+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/4142780617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c1415d5d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098e0 tx=0x7f8c040091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.635+0000 7fddd6542700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 0x7fddb40693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000e10 tx=0x7fddcc000da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35628/0 (socket says 172.21.15.170:35628) 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 -- 172.21.15.170:0/533143782 learned_addr learned my addr 172.21.15.170:0/533143782 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 -- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f78158840 msgr2=0x7f8f780c42b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f78158840 0x7f8f780c42b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 -- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 msgr2=0x7f8f780b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f780b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 -- 172.21.15.170:0/533143782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c009420 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.636+0000 7f8f76ffd700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c7c0 tx=0x7f8f68007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.637+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68005b30 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.637+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68005c90 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.637+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68004a80 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.637+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f781771d0 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.637+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f78177790 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.638+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7804c3c0 con 0x7f8f7806da30 2023-10-08T16:04:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fddc0051fa0 con 0x7fddd006eab0 2023-10-08T16:04:44.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f680070f0 con 0x7f8f7806da30 2023-10-08T16:04:44.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f0c6411e700 1 -- 172.21.15.170:0/2680131960 shutdown_connections 2023-10-08T16:04:44.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f8f5effd700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 0x7f8f64069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f680838f0 con 0x7f8f7806da30 2023-10-08T16:04:44.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f8f767fc700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 0x7f8f64069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.639+0000 7f0c6411e700 1 -- 172.21.15.170:0/2680131960 wait complete. 2023-10-08T16:04:44.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.640+0000 7f8f767fc700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 0x7f8f64069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806cf70 tx=0x7f8f6c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.641+0000 7f0c6411e700 1 Processor -- start 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.641+0000 7f0c6411e700 1 -- start start 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c0c9510 0x7f0c5c15a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c16bf80 con 0x7f0c5c0c9510 2023-10-08T16:04:44.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b13a0 con 0x7f0c5c06bcd0 2023-10-08T16:04:44.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.642+0000 7f0c6411e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c140890 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.643+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/4142780617 shutdown_connections 2023-10-08T16:04:44.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.643+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/4142780617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c1415d5d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.643+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/4142780617 >> 172.21.15.170:0/4142780617 conn(0x7f8c14068e20 msgr2=0x7f8c140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.643+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f6804c5a0 con 0x7f8f7806da30 2023-10-08T16:04:44.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:35630/0 (socket says 172.21.15.170:35630) 2023-10-08T16:04:44.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 -- 172.21.15.170:0/2029130904 learned_addr learned my addr 172.21.15.170:0/2029130904 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 -- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c5c06bcd0 msgr2=0x7f0c5c140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 -- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c0c9510 msgr2=0x7f0c5c15a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c0c9510 0x7f0c5c15a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.644+0000 7f0c616b9700 1 -- 172.21.15.170:0/2029130904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c009420 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.649+0000 7f0c616b9700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400cd30 tx=0x7f0c54001b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.649+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5400ab70 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.649+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5400acf0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.649+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54007340 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.647+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/4142780617 shutdown_connections 2023-10-08T16:04:44.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.648+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/4142780617 wait complete. 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.649+0000 7f8c1b8d3700 1 Processor -- start 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.650+0000 7f8c1b8d3700 1 -- start start 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c14159890 0x7f8c1417a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 0x7f8c14157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140c64f0 con 0x7f8c14159c60 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1415bfa0 con 0x7f8c1415db10 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c1b8d3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1415b670 con 0x7f8c14159890 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c18e6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c18e6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46658/0 (socket says 172.21.15.170:46658) 2023-10-08T16:04:44.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.652+0000 7f8c18e6e700 1 -- 172.21.15.170:0/1818448292 learned_addr learned my addr 172.21.15.170:0/1818448292 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c19e70700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 0x7f8c14157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 -- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c14159890 msgr2=0x7f8c1417a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c14159890 0x7f8c1417a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 -- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 msgr2=0x7f8c14157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 0x7f8c14157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 -- 172.21.15.170:0/1818448292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04009420 con 0x7f8c14159c60 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c19e70700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 0x7f8c14157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c18e6e700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c040030c0 tx=0x7f8c04016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0400ba10 con 0x7f8c14159c60 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c04020070 con 0x7f8c14159c60 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0401b6a0 con 0x7f8c14159c60 2023-10-08T16:04:44.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1415b950 con 0x7f8c14159c60 2023-10-08T16:04:44.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.653+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c140c78a0 con 0x7f8c14159c60 2023-10-08T16:04:44.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.655+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c168a70 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.655+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c15a9e0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.656+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c04c3c0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.658+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c54007770 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.659+0000 7f0c51ffb700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 0x7f0c400694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.659+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c540855a0 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.660+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c04022030 con 0x7f8c14159c60 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.660+0000 7f8c097fa700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 0x7f8c000695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:44.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.660+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c04086610 con 0x7f8c14159c60 2023-10-08T16:04:44.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.662+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c54052870 con 0x7f0c5c06c0a0 2023-10-08T16:04:44.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.666+0000 7f8c1966f700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 0x7f8c000695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.666+0000 7f8c1966f700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 0x7f8c000695d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c100015c0 tx=0x7f8c10007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.666+0000 7f0c61eba700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 0x7f0c400694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:44.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.668+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf80052e0 con 0x7f8c14159c60 2023-10-08T16:04:44.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.677+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c040537e0 con 0x7f8c14159c60 2023-10-08T16:04:44.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.677+0000 7f0c61eba700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 0x7f0c400694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce190 tx=0x7f0c4c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:44.958 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.949+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fca50005590 con 0x7fca64159a40 2023-10-08T16:04:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.949+0000 7fca4affd700 1 -- 172.21.15.170:0/3608091150 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca5c054930 con 0x7fca64159a40 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.957+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 msgr2=0x7fca4c06aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.957+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 0x7fca4c06aff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca54015000 comp rx=0 tx=0).stop 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.957+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 msgr2=0x7fca640a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.957+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00bc60 tx=0x7fca5c009840 comp rx=0 tx=0).stop 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 shutdown_connections 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca640a7c90 0x7fca640a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca4c068b40 0x7fca4c06aff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca4c003ae0 0x7fca640a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 --2- 172.21.15.170:0/3608091150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159a40 0x7fca640a54c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 >> 172.21.15.170:0/3608091150 conn(0x7fca6406a320 msgr2=0x7fca640ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f78c4005590 con 0x7f78d80b1c50 2023-10-08T16:04:44.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.958+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 shutdown_connections 2023-10-08T16:04:44.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.959+0000 7fca693bf700 1 -- 172.21.15.170:0/3608091150 wait complete. 2023-10-08T16:04:44.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.960+0000 7f78bffff700 1 -- 172.21.15.170:0/948451996 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f78cc01e090 con 0x7f78d80b1c50 2023-10-08T16:04:44.961 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.963+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 msgr2=0x7f78c00699b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.963+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 0x7f78c00699b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f78d00095e0 tx=0x7f78d0017000 comp rx=0 tx=0).stop 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.963+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 msgr2=0x7f78d816cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.963+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f78cc00c880 tx=0x7f78cc008040 comp rx=0 tx=0).stop 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 shutdown_connections 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f78c0067500 0x7f78c00699b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f78d80b1c50 0x7f78d816cc70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f78d8068aa0 0x7f78d81875b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 --2- 172.21.15.170:0/948451996 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f78d80875d0 0x7f78d816c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:44.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.964+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 >> 172.21.15.170:0/948451996 conn(0x7f78d80cd000 msgr2=0x7f78d80ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:44.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.965+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 shutdown_connections 2023-10-08T16:04:44.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:44.965+0000 7f78de4d6700 1 -- 172.21.15.170:0/948451996 wait complete. 2023-10-08T16:04:44.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-10-08T16:04:44.991 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:44.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-10-08T16:04:44.996 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.001+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efe3804c3c0 con 0x7efe38068fe0 2023-10-08T16:04:45.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.002+0000 7efe267fc700 1 -- 172.21.15.170:0/2774858909 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe28056a90 con 0x7efe38068fe0 2023-10-08T16:04:45.003 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:45.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.005+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 msgr2=0x7efe180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.005+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 0x7efe180693e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe3806f3c0 tx=0x7efe2c007ea0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.005+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 msgr2=0x7efe3813cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.005+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe28009290 tx=0x7efe28016000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 shutdown_connections 2023-10-08T16:04:45.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe38158440 0x7efe3806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe18066f30 0x7efe180693e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe38068fe0 0x7efe3813cbe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 --2- 172.21.15.170:0/2774858909 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe380b21b0 0x7efe38147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 >> 172.21.15.170:0/2774858909 conn(0x7efe380cd000 msgr2=0x7efe380ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 shutdown_connections 2023-10-08T16:04:45.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.009+0000 7efe3cf36700 1 -- 172.21.15.170:0/2774858909 wait complete. 2023-10-08T16:04:45.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.010+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fddd00ab520 con 0x7fddd006eab0 2023-10-08T16:04:45.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.011+0000 7fddc67fc700 1 -- 172.21.15.170:0/1243460432 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fddc0017020 con 0x7fddd006eab0 2023-10-08T16:04:45.012 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:45.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.014+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 msgr2=0x7fddb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.014+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 0x7fddb40693e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddcc000e10 tx=0x7fddcc000da0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.014+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 msgr2=0x7fddd0159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.014+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fddc000e9d0 tx=0x7fddc00091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 shutdown_connections 2023-10-08T16:04:45.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fddd006ef20 0x7fddd0159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fddbc00a690 0x7fddd01352a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fddb4066f30 0x7fddb40693e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 --2- 172.21.15.170:0/1243460432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd006eab0 0x7fddd0159ab0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 >> 172.21.15.170:0/1243460432 conn(0x7fddd00cd000 msgr2=0x7fddd0154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 shutdown_connections 2023-10-08T16:04:45.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.015+0000 7fddd87a6700 1 -- 172.21.15.170:0/1243460432 wait complete. 2023-10-08T16:04:45.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.022+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f903c04c3c0 con 0x7f903c159c60 2023-10-08T16:04:45.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.023+0000 7f9031ffb700 1 -- 172.21.15.170:0/105244010 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f902c054790 con 0x7f903c159c60 2023-10-08T16:04:45.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.023+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0c5c16c0f0 con 0x7f0c5c06c0a0 2023-10-08T16:04:45.024 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:45.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.023+0000 7f0c51ffb700 1 -- 172.21.15.170:0/2029130904 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c5400ae60 con 0x7f0c5c06c0a0 2023-10-08T16:04:45.025 INFO:teuthology.orchestra.run.smithi170.stdout:55834574853 2023-10-08T16:04:45.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.026+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 msgr2=0x7f90200694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.026+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 0x7f90200694b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90380099b0 tx=0x7f9038017000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 msgr2=0x7f903c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f902c00c560 tx=0x7f902c007ef0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-08T16:04:45.029 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 shutdown_connections 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9020067000 0x7f90200694b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f903c159c60 0x7f903c0ce8d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f903c069560 0x7f903c15cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 --2- 172.21.15.170:0/105244010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903c159890 0x7f903c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 >> 172.21.15.170:0/105244010 conn(0x7f903c068e20 msgr2=0x7f903c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 shutdown_connections 2023-10-08T16:04:45.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.027+0000 7f9043d30700 1 -- 172.21.15.170:0/105244010 wait complete. 2023-10-08T16:04:45.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 msgr2=0x7f0c400694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 0x7f0c400694b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c0ce190 tx=0x7f0c4c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 msgr2=0x7f0c5c159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400cd30 tx=0x7f0c54001b10 comp rx=0 tx=0).stop 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 shutdown_connections 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c40067000 0x7f0c400694b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c5c0c9510 0x7f0c5c15a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c5c06bcd0 0x7f0c5c140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 --2- 172.21.15.170:0/2029130904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c5c06c0a0 0x7f0c5c159b40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 >> 172.21.15.170:0/2029130904 conn(0x7f0c5c0ce600 msgr2=0x7f0c5c069b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 shutdown_connections 2023-10-08T16:04:45.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.028+0000 7f0c6411e700 1 -- 172.21.15.170:0/2029130904 wait complete. 2023-10-08T16:04:45.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-08T16:04:45.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.032+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8f780ae940 con 0x7f8f7806da30 2023-10-08T16:04:45.033 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.032+0000 7f8f5effd700 1 -- 172.21.15.170:0/533143782 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f68018020 con 0x7f8f7806da30 2023-10-08T16:04:45.034 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:45.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 msgr2=0x7f8f64069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 0x7f8f64069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f7806cf70 tx=0x7f8f6c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 msgr2=0x7f8f78157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800c7c0 tx=0x7f8f68007f70 comp rx=0 tx=0).stop 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 shutdown_connections 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f64066fb0 0x7f8f64069460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f78158c10 0x7f8f780b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7806da30 0x7f8f78157820 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 --2- 172.21.15.170:0/533143782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f78158840 0x7f8f780c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 >> 172.21.15.170:0/533143782 conn(0x7f8f780cc1e0 msgr2=0x7f8f78161810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 shutdown_connections 2023-10-08T16:04:45.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.035+0000 7f8f7d839700 1 -- 172.21.15.170:0/533143782 wait complete. 2023-10-08T16:04:45.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-08T16:04:45.043 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-08T16:04:45.050 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-10-08T16:04:45.051 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.050+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8bf8005590 con 0x7f8c14159c60 2023-10-08T16:04:45.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.050+0000 7f8c097fa700 1 -- 172.21.15.170:0/1818448292 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c04056e00 con 0x7f8c14159c60 2023-10-08T16:04:45.052 INFO:teuthology.orchestra.run.smithi170.stdout:55834574854 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 msgr2=0x7f8c000695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 0x7f8c000695d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8c100015c0 tx=0x7f8c10007b50 comp rx=0 tx=0).stop 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 msgr2=0x7f8c14157510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8c040030c0 tx=0x7f8c04016000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 shutdown_connections 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c1415db10 0x7f8c14157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c14159890 0x7f8c1417a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c00067120 0x7f8c000695d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 --2- 172.21.15.170:0/1818448292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c14159c60 0x7f8c14157510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.052+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 >> 172.21.15.170:0/1818448292 conn(0x7f8c14068e20 msgr2=0x7f8c14069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.053+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 shutdown_connections 2023-10-08T16:04:45.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.053+0000 7f8c1b8d3700 1 -- 172.21.15.170:0/1818448292 wait complete. 2023-10-08T16:04:45.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-08T16:04:45.066 DEBUG:teuthology.parallel:result is None 2023-10-08T16:04:45.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T16:04:45.066 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:04:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 Processor -- start 2023-10-08T16:04:45.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 -- start start 2023-10-08T16:04:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0188159c60 0x7f018815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 0x7f0188069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f018815a160 con 0x7f0188159890 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f018815df50 con 0x7f0188159c60 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.219+0000 7f018d23a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01880d13a0 con 0x7f018815db10 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 0x7f0188069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0188159c60 0x7f018815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0188159c60 0x7f018815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:36916/0 (socket says 172.21.15.170:36916) 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 0x7f0188069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:42900/0 (socket says 172.21.15.170:42900) 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 -- 172.21.15.170:0/2982514976 learned_addr learned my addr 172.21.15.170:0/2982514976 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f0186d9d700 1 --1- 172.21.15.170:0/2982514976 >> v1:172.21.15.191:6789/0 conn(0x7f0188159890 0x7f018815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:53370/0 (socket says 172.21.15.170:53370) 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 -- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 msgr2=0x7f0188069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 --2- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 0x7f0188069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 -- 172.21.15.170:0/2982514976 >> v1:172.21.15.191:6789/0 conn(0x7f0188159890 legacy=0x7f018815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 -- 172.21.15.170:0/2982514976 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018815e140 con 0x7f0188159c60 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f018659c700 1 --2- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159c60 0x7f018815d5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f017c0099b0 tx=0x7f017c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204879b86b3cfb99 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f0185d9b700 1 -- 172.21.15.170:0/2982514976 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c009ea0 con 0x7f0188159c60 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f0185d9b700 1 -- 172.21.15.170:0/2982514976 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c00b860 con 0x7f0188159c60 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.220+0000 7f0185d9b700 1 -- 172.21.15.170:0/2982514976 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00bc00 con 0x7f0188159c60 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 -- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159c60 msgr2=0x7f018815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 --2- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159c60 0x7f018815d5d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f017c0099b0 tx=0x7f017c009250 comp rx=0 tx=0).stop 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 -- 172.21.15.170:0/2982514976 shutdown_connections 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 --2- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f018815db10 0x7f0188069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 --2- 172.21.15.170:0/2982514976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159c60 0x7f018815d5d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 -- 172.21.15.170:0/2982514976 >> 172.21.15.170:0/2982514976 conn(0x7f0188068e20 msgr2=0x7f01880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 -- 172.21.15.170:0/2982514976 shutdown_connections 2023-10-08T16:04:45.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.221+0000 7f018d23a700 1 -- 172.21.15.170:0/2982514976 wait complete. 2023-10-08T16:04:45.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.222+0000 7f018d23a700 1 Processor -- start 2023-10-08T16:04:45.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.222+0000 7f018d23a700 1 -- start start 2023-10-08T16:04:45.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 0x7f01880c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0188159c60 0x7f018814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c78a0 con 0x7f018815db10 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01880c8140 con 0x7f0188159890 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018d23a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018814a860 con 0x7f0188159c60 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58172/0 (socket says 172.21.15.170:58172) 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 -- 172.21.15.170:0/2197993266 learned_addr learned my addr 172.21.15.170:0/2197993266 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f0186d9d700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 0x7f01880c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018659c700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0188159c60 0x7f018814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 -- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0188159c60 msgr2=0x7f018814a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0188159c60 0x7f018814a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 -- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 msgr2=0x7f01880c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 0x7f01880c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f018759e700 1 -- 172.21.15.170:0/2197993266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0094f0 con 0x7f018815db10 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.223+0000 7f0186d9d700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 0x7f01880c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f018759e700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f018000e900 tx=0x7f01800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018000eea0 con 0x7f018815db10 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018000c850 con 0x7f018815db10 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01880cf010 con 0x7f018815db10 2023-10-08T16:04:45.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018000cc10 con 0x7f018815db10 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.224+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01880c7dd0 con 0x7f018815db10 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.225+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018804c3c0 con 0x7f018815db10 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.225+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0180013470 con 0x7f018815db10 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.225+0000 7f016effd700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 0x7f01740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.226+0000 7f0186d9d700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 0x7f01740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.226+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0180084750 con 0x7f018815db10 2023-10-08T16:04:45.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.226+0000 7f0186d9d700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 0x7f01740693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f017800d220 tx=0x7f0178009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.230+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01800519c0 con 0x7f018815db10 2023-10-08T16:04:45.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.410+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f018806a5b0 con 0x7f0174066f30 2023-10-08T16:04:45.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.417+0000 7f016effd700 1 -- 172.21.15.170:0/2197993266 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264507 (secure 0 0 0) 0x7f018806a5b0 con 0x7f0174066f30 2023-10-08T16:04:45.422 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 msgr2=0x7f01740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 0x7f01740693e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f017800d220 tx=0x7f0178009040 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 msgr2=0x7f01880ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f018000e900 tx=0x7f01800091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 shutdown_connections 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0174066f30 0x7f01740693e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f018815db10 0x7f01880ce6c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0188159890 0x7f01880c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 --2- 172.21.15.170:0/2197993266 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0188159c60 0x7f018814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 >> 172.21.15.170:0/2197993266 conn(0x7f0188068e20 msgr2=0x7f0188158620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 shutdown_connections 2023-10-08T16:04:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.420+0000 7f018d23a700 1 -- 172.21.15.170:0/2197993266 wait complete. 2023-10-08T16:04:45.425 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:04:45.434 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T16:04:44.225828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001635"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854867+0000","last_change":"2023-10-08T16:04:36.847335+0000","last_active":"2023-10-08T16:04:36.854867+0000","last_peered":"2023-10-08T16:04:36.854867+0000","last_clean":"2023-10-08T16:04:36.854867+0000","last_became_active":"2023-10-08T16:04:36.846900+0000","last_became_peered":"2023-10-08T16:04:36.846900+0000","last_unstale":"2023-10-08T16:04:36.854867+0000","last_undegraded":"2023-10-08T16:04:36.854867+0000","last_fullsized":"2023-10-08T16:04:36.854867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.847766+0000","last_change":"2023-10-08T16:04:36.841973+0000","last_active":"2023-10-08T16:04:36.847766+0000","last_peered":"2023-10-08T16:04:36.847766+0000","last_clean":"2023-10-08T16:04:36.847766+0000","last_became_active":"2023-10-08T16:04:36.841752+0000","last_became_peered":"2023-10-08T16:04:36.841752+0000","last_unstale":"2023-10-08T16:04:36.847766+0000","last_undegraded":"2023-10-08T16:04:36.847766+0000","last_fullsized":"2023-10-08T16:04:36.847766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844363+0000","last_change":"2023-10-08T16:04:36.841982+0000","last_active":"2023-10-08T16:04:36.844363+0000","last_peered":"2023-10-08T16:04:36.844363+0000","last_clean":"2023-10-08T16:04:36.844363+0000","last_became_active":"2023-10-08T16:04:36.841738+0000","last_became_peered":"2023-10-08T16:04:36.841738+0000","last_unstale":"2023-10-08T16:04:36.844363+0000","last_undegraded":"2023-10-08T16:04:36.844363+0000","last_fullsized":"2023-10-08T16:04:36.844363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:28.994728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861145+0000","last_change":"2023-10-08T16:04:36.857688+0000","last_active":"2023-10-08T16:04:36.861145+0000","last_peered":"2023-10-08T16:04:36.861145+0000","last_clean":"2023-10-08T16:04:36.861145+0000","last_became_active":"2023-10-08T16:04:36.857001+0000","last_became_peered":"2023-10-08T16:04:36.857001+0000","last_unstale":"2023-10-08T16:04:36.861145+0000","last_undegraded":"2023-10-08T16:04:36.861145+0000","last_fullsized":"2023-10-08T16:04:36.861145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848398+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848398+0000","last_peered":"2023-10-08T16:04:36.848398+0000","last_clean":"2023-10-08T16:04:36.848398+0000","last_became_active":"2023-10-08T16:04:36.842806+0000","last_became_peered":"2023-10-08T16:04:36.842806+0000","last_unstale":"2023-10-08T16:04:36.848398+0000","last_undegraded":"2023-10-08T16:04:36.848398+0000","last_fullsized":"2023-10-08T16:04:36.848398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854094+0000","last_change":"2023-10-08T16:04:36.845475+0000","last_active":"2023-10-08T16:04:36.854094+0000","last_peered":"2023-10-08T16:04:36.854094+0000","last_clean":"2023-10-08T16:04:36.854094+0000","last_became_active":"2023-10-08T16:04:36.845053+0000","last_became_peered":"2023-10-08T16:04:36.845053+0000","last_unstale":"2023-10-08T16:04:36.854094+0000","last_undegraded":"2023-10-08T16:04:36.854094+0000","last_fullsized":"2023-10-08T16:04:36.854094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848364+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848364+0000","last_peered":"2023-10-08T16:04:36.848364+0000","last_clean":"2023-10-08T16:04:36.848364+0000","last_became_active":"2023-10-08T16:04:36.842841+0000","last_became_peered":"2023-10-08T16:04:36.842841+0000","last_unstale":"2023-10-08T16:04:36.848364+0000","last_undegraded":"2023-10-08T16:04:36.848364+0000","last_fullsized":"2023-10-08T16:04:36.848364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855557+0000","last_change":"2023-10-08T16:04:36.850496+0000","last_active":"2023-10-08T16:04:36.855557+0000","last_peered":"2023-10-08T16:04:36.855557+0000","last_clean":"2023-10-08T16:04:36.855557+0000","last_became_active":"2023-10-08T16:04:36.850237+0000","last_became_peered":"2023-10-08T16:04:36.850237+0000","last_unstale":"2023-10-08T16:04:36.855557+0000","last_undegraded":"2023-10-08T16:04:36.855557+0000","last_fullsized":"2023-10-08T16:04:36.855557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854848+0000","last_change":"2023-10-08T16:04:36.847152+0000","last_active":"2023-10-08T16:04:36.854848+0000","last_peered":"2023-10-08T16:04:36.854848+0000","last_clean":"2023-10-08T16:04:36.854848+0000","last_became_active":"2023-10-08T16:04:36.846573+0000","last_became_peered":"2023-10-08T16:04:36.846573+0000","last_unstale":"2023-10-08T16:04:36.854848+0000","last_undegraded":"2023-10-08T16:04:36.854848+0000","last_fullsized":"2023-10-08T16:04:36.854848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853411+0000","last_change":"2023-10-08T16:04:36.847416+0000","last_active":"2023-10-08T16:04:36.853411+0000","last_peered":"2023-10-08T16:04:36.853411+0000","last_clean":"2023-10-08T16:04:36.853411+0000","last_became_active":"2023-10-08T16:04:36.847092+0000","last_became_peered":"2023-10-08T16:04:36.847092+0000","last_unstale":"2023-10-08T16:04:36.853411+0000","last_undegraded":"2023-10-08T16:04:36.853411+0000","last_fullsized":"2023-10-08T16:04:36.853411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860614+0000","last_change":"2023-10-08T16:04:36.854204+0000","last_active":"2023-10-08T16:04:36.860614+0000","last_peered":"2023-10-08T16:04:36.860614+0000","last_clean":"2023-10-08T16:04:36.860614+0000","last_became_active":"2023-10-08T16:04:36.853963+0000","last_became_peered":"2023-10-08T16:04:36.853963+0000","last_unstale":"2023-10-08T16:04:36.860614+0000","last_undegraded":"2023-10-08T16:04:36.860614+0000","last_fullsized":"2023-10-08T16:04:36.860614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859253+0000","last_change":"2023-10-08T16:04:36.855269+0000","last_active":"2023-10-08T16:04:36.859253+0000","last_peered":"2023-10-08T16:04:36.859253+0000","last_clean":"2023-10-08T16:04:36.859253+0000","last_became_active":"2023-10-08T16:04:36.854443+0000","last_became_peered":"2023-10-08T16:04:36.854443+0000","last_unstale":"2023-10-08T16:04:36.859253+0000","last_undegraded":"2023-10-08T16:04:36.859253+0000","last_fullsized":"2023-10-08T16:04:36.859253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844837+0000","last_change":"2023-10-08T16:04:36.842900+0000","last_active":"2023-10-08T16:04:36.844837+0000","last_peered":"2023-10-08T16:04:36.844837+0000","last_clean":"2023-10-08T16:04:36.844837+0000","last_became_active":"2023-10-08T16:04:36.842642+0000","last_became_peered":"2023-10-08T16:04:36.842642+0000","last_unstale":"2023-10-08T16:04:36.844837+0000","last_undegraded":"2023-10-08T16:04:36.844837+0000","last_fullsized":"2023-10-08T16:04:36.844837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844389+0000","last_change":"2023-10-08T16:04:36.842577+0000","last_active":"2023-10-08T16:04:36.844389+0000","last_peered":"2023-10-08T16:04:36.844389+0000","last_clean":"2023-10-08T16:04:36.844389+0000","last_became_active":"2023-10-08T16:04:36.841699+0000","last_became_peered":"2023-10-08T16:04:36.841699+0000","last_unstale":"2023-10-08T16:04:36.844389+0000","last_undegraded":"2023-10-08T16:04:36.844389+0000","last_fullsized":"2023-10-08T16:04:36.844389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850510+0000","last_change":"2023-10-08T16:04:36.845580+0000","last_active":"2023-10-08T16:04:36.850510+0000","last_peered":"2023-10-08T16:04:36.850510+0000","last_clean":"2023-10-08T16:04:36.850510+0000","last_became_active":"2023-10-08T16:04:36.845348+0000","last_became_peered":"2023-10-08T16:04:36.845348+0000","last_unstale":"2023-10-08T16:04:36.850510+0000","last_undegraded":"2023-10-08T16:04:36.850510+0000","last_fullsized":"2023-10-08T16:04:36.850510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854086+0000","last_change":"2023-10-08T16:04:36.845629+0000","last_active":"2023-10-08T16:04:36.854086+0000","last_peered":"2023-10-08T16:04:36.854086+0000","last_clean":"2023-10-08T16:04:36.854086+0000","last_became_active":"2023-10-08T16:04:36.845283+0000","last_became_peered":"2023-10-08T16:04:36.845283+0000","last_unstale":"2023-10-08T16:04:36.854086+0000","last_undegraded":"2023-10-08T16:04:36.854086+0000","last_fullsized":"2023-10-08T16:04:36.854086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856506+0000","last_change":"2023-10-08T16:04:36.852540+0000","last_active":"2023-10-08T16:04:36.856506+0000","last_peered":"2023-10-08T16:04:36.856506+0000","last_clean":"2023-10-08T16:04:36.856506+0000","last_became_active":"2023-10-08T16:04:36.852294+0000","last_became_peered":"2023-10-08T16:04:36.852294+0000","last_unstale":"2023-10-08T16:04:36.856506+0000","last_undegraded":"2023-10-08T16:04:36.856506+0000","last_fullsized":"2023-10-08T16:04:36.856506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850863+0000","last_change":"2023-10-08T16:04:36.845236+0000","last_active":"2023-10-08T16:04:36.850863+0000","last_peered":"2023-10-08T16:04:36.850863+0000","last_clean":"2023-10-08T16:04:36.850863+0000","last_became_active":"2023-10-08T16:04:36.844966+0000","last_became_peered":"2023-10-08T16:04:36.844966+0000","last_unstale":"2023-10-08T16:04:36.850863+0000","last_undegraded":"2023-10-08T16:04:36.850863+0000","last_fullsized":"2023-10-08T16:04:36.850863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849074+0000","last_change":"2023-10-08T16:04:36.844871+0000","last_active":"2023-10-08T16:04:36.849074+0000","last_peered":"2023-10-08T16:04:36.849074+0000","last_clean":"2023-10-08T16:04:36.849074+0000","last_became_active":"2023-10-08T16:04:36.844378+0000","last_became_peered":"2023-10-08T16:04:36.844378+0000","last_unstale":"2023-10-08T16:04:36.849074+0000","last_undegraded":"2023-10-08T16:04:36.849074+0000","last_fullsized":"2023-10-08T16:04:36.849074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861810+0000","last_change":"2023-10-08T16:04:35.842581+0000","last_active":"2023-10-08T16:04:36.861810+0000","last_peered":"2023-10-08T16:04:36.861810+0000","last_clean":"2023-10-08T16:04:36.861810+0000","last_became_active":"2023-10-08T16:04:35.842258+0000","last_became_peered":"2023-10-08T16:04:35.842258+0000","last_unstale":"2023-10-08T16:04:36.861810+0000","last_undegraded":"2023-10-08T16:04:36.861810+0000","last_fullsized":"2023-10-08T16:04:36.861810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:04.459691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844626+0000","last_change":"2023-10-08T16:04:36.842889+0000","last_active":"2023-10-08T16:04:36.844626+0000","last_peered":"2023-10-08T16:04:36.844626+0000","last_clean":"2023-10-08T16:04:36.844626+0000","last_became_active":"2023-10-08T16:04:36.842161+0000","last_became_peered":"2023-10-08T16:04:36.842161+0000","last_unstale":"2023-10-08T16:04:36.844626+0000","last_undegraded":"2023-10-08T16:04:36.844626+0000","last_fullsized":"2023-10-08T16:04:36.844626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856629+0000","last_change":"2023-10-08T16:04:35.835973+0000","last_active":"2023-10-08T16:04:36.856629+0000","last_peered":"2023-10-08T16:04:36.856629+0000","last_clean":"2023-10-08T16:04:36.856629+0000","last_became_active":"2023-10-08T16:04:35.835785+0000","last_became_peered":"2023-10-08T16:04:35.835785+0000","last_unstale":"2023-10-08T16:04:36.856629+0000","last_undegraded":"2023-10-08T16:04:36.856629+0000","last_fullsized":"2023-10-08T16:04:36.856629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849375+0000","last_change":"2023-10-08T16:04:36.845806+0000","last_active":"2023-10-08T16:04:36.849375+0000","last_peered":"2023-10-08T16:04:36.849375+0000","last_clean":"2023-10-08T16:04:36.849375+0000","last_became_active":"2023-10-08T16:04:36.845574+0000","last_became_peered":"2023-10-08T16:04:36.845574+0000","last_unstale":"2023-10-08T16:04:36.849375+0000","last_undegraded":"2023-10-08T16:04:36.849375+0000","last_fullsized":"2023-10-08T16:04:36.849375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860925+0000","last_change":"2023-10-08T16:04:35.844449+0000","last_active":"2023-10-08T16:04:36.860925+0000","last_peered":"2023-10-08T16:04:36.860925+0000","last_clean":"2023-10-08T16:04:36.860925+0000","last_became_active":"2023-10-08T16:04:35.844192+0000","last_became_peered":"2023-10-08T16:04:35.844192+0000","last_unstale":"2023-10-08T16:04:36.860925+0000","last_undegraded":"2023-10-08T16:04:36.860925+0000","last_fullsized":"2023-10-08T16:04:36.860925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856262+0000","last_change":"2023-10-08T16:04:36.853135+0000","last_active":"2023-10-08T16:04:36.856262+0000","last_peered":"2023-10-08T16:04:36.856262+0000","last_clean":"2023-10-08T16:04:36.856262+0000","last_became_active":"2023-10-08T16:04:36.852832+0000","last_became_peered":"2023-10-08T16:04:36.852832+0000","last_unstale":"2023-10-08T16:04:36.856262+0000","last_undegraded":"2023-10-08T16:04:36.856262+0000","last_fullsized":"2023-10-08T16:04:36.856262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861744+0000","last_change":"2023-10-08T16:04:35.845418+0000","last_active":"2023-10-08T16:04:36.861744+0000","last_peered":"2023-10-08T16:04:36.861744+0000","last_clean":"2023-10-08T16:04:36.861744+0000","last_became_active":"2023-10-08T16:04:35.845104+0000","last_became_peered":"2023-10-08T16:04:35.845104+0000","last_unstale":"2023-10-08T16:04:36.861744+0000","last_undegraded":"2023-10-08T16:04:36.861744+0000","last_fullsized":"2023-10-08T16:04:36.861744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856172+0000","last_change":"2023-10-08T16:04:36.851677+0000","last_active":"2023-10-08T16:04:36.856172+0000","last_peered":"2023-10-08T16:04:36.856172+0000","last_clean":"2023-10-08T16:04:36.856172+0000","last_became_active":"2023-10-08T16:04:36.847960+0000","last_became_peered":"2023-10-08T16:04:36.847960+0000","last_unstale":"2023-10-08T16:04:36.856172+0000","last_undegraded":"2023-10-08T16:04:36.856172+0000","last_fullsized":"2023-10-08T16:04:36.856172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861300+0000","last_change":"2023-10-08T16:04:35.833685+0000","last_active":"2023-10-08T16:04:36.861300+0000","last_peered":"2023-10-08T16:04:36.861300+0000","last_clean":"2023-10-08T16:04:36.861300+0000","last_became_active":"2023-10-08T16:04:35.833403+0000","last_became_peered":"2023-10-08T16:04:35.833403+0000","last_unstale":"2023-10-08T16:04:36.861300+0000","last_undegraded":"2023-10-08T16:04:36.861300+0000","last_fullsized":"2023-10-08T16:04:36.861300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851430+0000","last_change":"2023-10-08T16:04:36.845487+0000","last_active":"2023-10-08T16:04:36.851430+0000","last_peered":"2023-10-08T16:04:36.851430+0000","last_clean":"2023-10-08T16:04:36.851430+0000","last_became_active":"2023-10-08T16:04:36.845204+0000","last_became_peered":"2023-10-08T16:04:36.845204+0000","last_unstale":"2023-10-08T16:04:36.851430+0000","last_undegraded":"2023-10-08T16:04:36.851430+0000","last_fullsized":"2023-10-08T16:04:36.851430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:37.980506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861219+0000","last_change":"2023-10-08T16:04:35.833842+0000","last_active":"2023-10-08T16:04:36.861219+0000","last_peered":"2023-10-08T16:04:36.861219+0000","last_clean":"2023-10-08T16:04:36.861219+0000","last_became_active":"2023-10-08T16:04:35.833461+0000","last_became_peered":"2023-10-08T16:04:35.833461+0000","last_unstale":"2023-10-08T16:04:36.861219+0000","last_undegraded":"2023-10-08T16:04:36.861219+0000","last_fullsized":"2023-10-08T16:04:36.861219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855228+0000","last_change":"2023-10-08T16:04:36.849156+0000","last_active":"2023-10-08T16:04:36.855228+0000","last_peered":"2023-10-08T16:04:36.855228+0000","last_clean":"2023-10-08T16:04:36.855228+0000","last_became_active":"2023-10-08T16:04:36.848907+0000","last_became_peered":"2023-10-08T16:04:36.848907+0000","last_unstale":"2023-10-08T16:04:36.855228+0000","last_undegraded":"2023-10-08T16:04:36.855228+0000","last_fullsized":"2023-10-08T16:04:36.855228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:04.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834426+0000","last_change":"2023-10-08T16:04:35.838361+0000","last_active":"2023-10-08T16:04:36.834426+0000","last_peered":"2023-10-08T16:04:36.834426+0000","last_clean":"2023-10-08T16:04:36.834426+0000","last_became_active":"2023-10-08T16:04:35.838175+0000","last_became_peered":"2023-10-08T16:04:35.838175+0000","last_unstale":"2023-10-08T16:04:36.834426+0000","last_undegraded":"2023-10-08T16:04:36.834426+0000","last_fullsized":"2023-10-08T16:04:36.834426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851367+0000","last_change":"2023-10-08T16:04:36.843489+0000","last_active":"2023-10-08T16:04:36.851367+0000","last_peered":"2023-10-08T16:04:36.851367+0000","last_clean":"2023-10-08T16:04:36.851367+0000","last_became_active":"2023-10-08T16:04:36.843221+0000","last_became_peered":"2023-10-08T16:04:36.843221+0000","last_unstale":"2023-10-08T16:04:36.851367+0000","last_undegraded":"2023-10-08T16:04:36.851367+0000","last_fullsized":"2023-10-08T16:04:36.851367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840321+0000","last_change":"2023-10-08T16:04:35.839307+0000","last_active":"2023-10-08T16:04:36.840321+0000","last_peered":"2023-10-08T16:04:36.840321+0000","last_clean":"2023-10-08T16:04:36.840321+0000","last_became_active":"2023-10-08T16:04:35.838997+0000","last_became_peered":"2023-10-08T16:04:35.838997+0000","last_unstale":"2023-10-08T16:04:36.840321+0000","last_undegraded":"2023-10-08T16:04:36.840321+0000","last_fullsized":"2023-10-08T16:04:36.840321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840059+0000","last_change":"2023-10-08T16:04:36.838211+0000","last_active":"2023-10-08T16:04:36.840059+0000","last_peered":"2023-10-08T16:04:36.840059+0000","last_clean":"2023-10-08T16:04:36.840059+0000","last_became_active":"2023-10-08T16:04:36.837948+0000","last_became_peered":"2023-10-08T16:04:36.837948+0000","last_unstale":"2023-10-08T16:04:36.840059+0000","last_undegraded":"2023-10-08T16:04:36.840059+0000","last_fullsized":"2023-10-08T16:04:36.840059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838498+0000","last_change":"2023-10-08T16:04:35.841004+0000","last_active":"2023-10-08T16:04:36.838498+0000","last_peered":"2023-10-08T16:04:36.838498+0000","last_clean":"2023-10-08T16:04:36.838498+0000","last_became_active":"2023-10-08T16:04:35.840735+0000","last_became_peered":"2023-10-08T16:04:35.840735+0000","last_unstale":"2023-10-08T16:04:36.838498+0000","last_undegraded":"2023-10-08T16:04:36.838498+0000","last_fullsized":"2023-10-08T16:04:36.838498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855922+0000","last_change":"2023-10-08T16:04:36.850043+0000","last_active":"2023-10-08T16:04:36.855922+0000","last_peered":"2023-10-08T16:04:36.855922+0000","last_clean":"2023-10-08T16:04:36.855922+0000","last_became_active":"2023-10-08T16:04:36.849743+0000","last_became_peered":"2023-10-08T16:04:36.849743+0000","last_unstale":"2023-10-08T16:04:36.855922+0000","last_undegraded":"2023-10-08T16:04:36.855922+0000","last_fullsized":"2023-10-08T16:04:36.855922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838528+0000","last_change":"2023-10-08T16:04:35.844419+0000","last_active":"2023-10-08T16:04:36.838528+0000","last_peered":"2023-10-08T16:04:36.838528+0000","last_clean":"2023-10-08T16:04:36.838528+0000","last_became_active":"2023-10-08T16:04:35.844125+0000","last_became_peered":"2023-10-08T16:04:35.844125+0000","last_unstale":"2023-10-08T16:04:36.838528+0000","last_undegraded":"2023-10-08T16:04:36.838528+0000","last_fullsized":"2023-10-08T16:04:36.838528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849356+0000","last_change":"2023-10-08T16:04:36.845771+0000","last_active":"2023-10-08T16:04:36.849356+0000","last_peered":"2023-10-08T16:04:36.849356+0000","last_clean":"2023-10-08T16:04:36.849356+0000","last_became_active":"2023-10-08T16:04:36.845274+0000","last_became_peered":"2023-10-08T16:04:36.845274+0000","last_unstale":"2023-10-08T16:04:36.849356+0000","last_undegraded":"2023-10-08T16:04:36.849356+0000","last_fullsized":"2023-10-08T16:04:36.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856530+0000","last_change":"2023-10-08T16:04:35.840602+0000","last_active":"2023-10-08T16:04:36.856530+0000","last_peered":"2023-10-08T16:04:36.856530+0000","last_clean":"2023-10-08T16:04:36.856530+0000","last_became_active":"2023-10-08T16:04:35.840299+0000","last_became_peered":"2023-10-08T16:04:35.840299+0000","last_unstale":"2023-10-08T16:04:36.856530+0000","last_undegraded":"2023-10-08T16:04:36.856530+0000","last_fullsized":"2023-10-08T16:04:36.856530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853166+0000","last_change":"2023-10-08T16:04:36.845842+0000","last_active":"2023-10-08T16:04:36.853166+0000","last_peered":"2023-10-08T16:04:36.853166+0000","last_clean":"2023-10-08T16:04:36.853166+0000","last_became_active":"2023-10-08T16:04:36.844954+0000","last_became_peered":"2023-10-08T16:04:36.844954+0000","last_unstale":"2023-10-08T16:04:36.853166+0000","last_undegraded":"2023-10-08T16:04:36.853166+0000","last_fullsized":"2023-10-08T16:04:36.853166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:34.979889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.863226+0000","last_change":"2023-10-08T16:04:35.838972+0000","last_active":"2023-10-08T16:04:36.863226+0000","last_peered":"2023-10-08T16:04:36.863226+0000","last_clean":"2023-10-08T16:04:36.863226+0000","last_became_active":"2023-10-08T16:04:35.838694+0000","last_became_peered":"2023-10-08T16:04:35.838694+0000","last_unstale":"2023-10-08T16:04:36.863226+0000","last_undegraded":"2023-10-08T16:04:36.863226+0000","last_fullsized":"2023-10-08T16:04:36.863226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849056+0000","last_change":"2023-10-08T16:04:36.844679+0000","last_active":"2023-10-08T16:04:36.849056+0000","last_peered":"2023-10-08T16:04:36.849056+0000","last_clean":"2023-10-08T16:04:36.849056+0000","last_became_active":"2023-10-08T16:04:36.844115+0000","last_became_peered":"2023-10-08T16:04:36.844115+0000","last_unstale":"2023-10-08T16:04:36.849056+0000","last_undegraded":"2023-10-08T16:04:36.849056+0000","last_fullsized":"2023-10-08T16:04:36.849056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859530+0000","last_change":"2023-10-08T16:04:35.837996+0000","last_active":"2023-10-08T16:04:36.859530+0000","last_peered":"2023-10-08T16:04:36.859530+0000","last_clean":"2023-10-08T16:04:36.859530+0000","last_became_active":"2023-10-08T16:04:35.837567+0000","last_became_peered":"2023-10-08T16:04:35.837567+0000","last_unstale":"2023-10-08T16:04:36.859530+0000","last_undegraded":"2023-10-08T16:04:36.859530+0000","last_fullsized":"2023-10-08T16:04:36.859530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855385+0000","last_change":"2023-10-08T16:04:36.849839+0000","last_active":"2023-10-08T16:04:36.855385+0000","last_peered":"2023-10-08T16:04:36.855385+0000","last_clean":"2023-10-08T16:04:36.855385+0000","last_became_active":"2023-10-08T16:04:36.849577+0000","last_became_peered":"2023-10-08T16:04:36.849577+0000","last_unstale":"2023-10-08T16:04:36.855385+0000","last_undegraded":"2023-10-08T16:04:36.855385+0000","last_fullsized":"2023-10-08T16:04:36.855385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:55.787779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838421+0000","last_change":"2023-10-08T16:04:35.841168+0000","last_active":"2023-10-08T16:04:36.838421+0000","last_peered":"2023-10-08T16:04:36.838421+0000","last_clean":"2023-10-08T16:04:36.838421+0000","last_became_active":"2023-10-08T16:04:35.840667+0000","last_became_peered":"2023-10-08T16:04:35.840667+0000","last_unstale":"2023-10-08T16:04:36.838421+0000","last_undegraded":"2023-10-08T16:04:36.838421+0000","last_fullsized":"2023-10-08T16:04:36.838421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859673+0000","last_change":"2023-10-08T16:04:36.849407+0000","last_active":"2023-10-08T16:04:36.859673+0000","last_peered":"2023-10-08T16:04:36.859673+0000","last_clean":"2023-10-08T16:04:36.859673+0000","last_became_active":"2023-10-08T16:04:36.849157+0000","last_became_peered":"2023-10-08T16:04:36.849157+0000","last_unstale":"2023-10-08T16:04:36.859673+0000","last_undegraded":"2023-10-08T16:04:36.859673+0000","last_fullsized":"2023-10-08T16:04:36.859673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836282+0000","last_change":"2023-10-08T16:04:35.835032+0000","last_active":"2023-10-08T16:04:36.836282+0000","last_peered":"2023-10-08T16:04:36.836282+0000","last_clean":"2023-10-08T16:04:36.836282+0000","last_became_active":"2023-10-08T16:04:35.834550+0000","last_became_peered":"2023-10-08T16:04:35.834550+0000","last_unstale":"2023-10-08T16:04:36.836282+0000","last_undegraded":"2023-10-08T16:04:36.836282+0000","last_fullsized":"2023-10-08T16:04:36.836282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855292+0000","last_change":"2023-10-08T16:04:36.848762+0000","last_active":"2023-10-08T16:04:36.855292+0000","last_peered":"2023-10-08T16:04:36.855292+0000","last_clean":"2023-10-08T16:04:36.855292+0000","last_became_active":"2023-10-08T16:04:36.848407+0000","last_became_peered":"2023-10-08T16:04:36.848407+0000","last_unstale":"2023-10-08T16:04:36.855292+0000","last_undegraded":"2023-10-08T16:04:36.855292+0000","last_fullsized":"2023-10-08T16:04:36.855292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856015+0000","last_change":"2023-10-08T16:04:35.834021+0000","last_active":"2023-10-08T16:04:36.856015+0000","last_peered":"2023-10-08T16:04:36.856015+0000","last_clean":"2023-10-08T16:04:36.856015+0000","last_became_active":"2023-10-08T16:04:35.833739+0000","last_became_peered":"2023-10-08T16:04:35.833739+0000","last_unstale":"2023-10-08T16:04:36.856015+0000","last_undegraded":"2023-10-08T16:04:36.856015+0000","last_fullsized":"2023-10-08T16:04:36.856015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859686+0000","last_change":"2023-10-08T16:04:36.850254+0000","last_active":"2023-10-08T16:04:36.859686+0000","last_peered":"2023-10-08T16:04:36.859686+0000","last_clean":"2023-10-08T16:04:36.859686+0000","last_became_active":"2023-10-08T16:04:36.849843+0000","last_became_peered":"2023-10-08T16:04:36.849843+0000","last_unstale":"2023-10-08T16:04:36.859686+0000","last_undegraded":"2023-10-08T16:04:36.859686+0000","last_fullsized":"2023-10-08T16:04:36.859686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859275+0000","last_change":"2023-10-08T16:04:35.837065+0000","last_active":"2023-10-08T16:04:36.859275+0000","last_peered":"2023-10-08T16:04:36.859275+0000","last_clean":"2023-10-08T16:04:36.859275+0000","last_became_active":"2023-10-08T16:04:35.836764+0000","last_became_peered":"2023-10-08T16:04:35.836764+0000","last_unstale":"2023-10-08T16:04:36.859275+0000","last_undegraded":"2023-10-08T16:04:36.859275+0000","last_fullsized":"2023-10-08T16:04:36.859275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:43.449593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848948+0000","last_change":"2023-10-08T16:04:36.845465+0000","last_active":"2023-10-08T16:04:36.848948+0000","last_peered":"2023-10-08T16:04:36.848948+0000","last_clean":"2023-10-08T16:04:36.848948+0000","last_became_active":"2023-10-08T16:04:36.845207+0000","last_became_peered":"2023-10-08T16:04:36.845207+0000","last_unstale":"2023-10-08T16:04:36.848948+0000","last_undegraded":"2023-10-08T16:04:36.848948+0000","last_fullsized":"2023-10-08T16:04:36.848948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831893+0000","last_change":"2023-10-08T16:04:35.836100+0000","last_active":"2023-10-08T16:04:36.831893+0000","last_peered":"2023-10-08T16:04:36.831893+0000","last_clean":"2023-10-08T16:04:36.831893+0000","last_became_active":"2023-10-08T16:04:35.835832+0000","last_became_peered":"2023-10-08T16:04:35.835832+0000","last_unstale":"2023-10-08T16:04:36.831893+0000","last_undegraded":"2023-10-08T16:04:36.831893+0000","last_fullsized":"2023-10-08T16:04:36.831893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860666+0000","last_change":"2023-10-08T16:04:36.853664+0000","last_active":"2023-10-08T16:04:36.860666+0000","last_peered":"2023-10-08T16:04:36.860666+0000","last_clean":"2023-10-08T16:04:36.860666+0000","last_became_active":"2023-10-08T16:04:36.853412+0000","last_became_peered":"2023-10-08T16:04:36.853412+0000","last_unstale":"2023-10-08T16:04:36.860666+0000","last_undegraded":"2023-10-08T16:04:36.860666+0000","last_fullsized":"2023-10-08T16:04:36.860666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838343+0000","last_change":"2023-10-08T16:04:35.838075+0000","last_active":"2023-10-08T16:04:36.838343+0000","last_peered":"2023-10-08T16:04:36.838343+0000","last_clean":"2023-10-08T16:04:36.838343+0000","last_became_active":"2023-10-08T16:04:35.837821+0000","last_became_peered":"2023-10-08T16:04:35.837821+0000","last_unstale":"2023-10-08T16:04:36.838343+0000","last_undegraded":"2023-10-08T16:04:36.838343+0000","last_fullsized":"2023-10-08T16:04:36.838343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850970+0000","last_change":"2023-10-08T16:04:36.845413+0000","last_active":"2023-10-08T16:04:36.850970+0000","last_peered":"2023-10-08T16:04:36.850970+0000","last_clean":"2023-10-08T16:04:36.850970+0000","last_became_active":"2023-10-08T16:04:36.845152+0000","last_became_peered":"2023-10-08T16:04:36.845152+0000","last_unstale":"2023-10-08T16:04:36.850970+0000","last_undegraded":"2023-10-08T16:04:36.850970+0000","last_fullsized":"2023-10-08T16:04:36.850970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839476+0000","last_change":"2023-10-08T16:04:36.837273+0000","last_active":"2023-10-08T16:04:36.839476+0000","last_peered":"2023-10-08T16:04:36.839476+0000","last_clean":"2023-10-08T16:04:36.839476+0000","last_became_active":"2023-10-08T16:04:36.837025+0000","last_became_peered":"2023-10-08T16:04:36.837025+0000","last_unstale":"2023-10-08T16:04:36.839476+0000","last_undegraded":"2023-10-08T16:04:36.839476+0000","last_fullsized":"2023-10-08T16:04:36.839476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833690+0000","last_change":"2023-10-08T16:04:35.831326+0000","last_active":"2023-10-08T16:04:36.833690+0000","last_peered":"2023-10-08T16:04:36.833690+0000","last_clean":"2023-10-08T16:04:36.833690+0000","last_became_active":"2023-10-08T16:04:35.831088+0000","last_became_peered":"2023-10-08T16:04:35.831088+0000","last_unstale":"2023-10-08T16:04:36.833690+0000","last_undegraded":"2023-10-08T16:04:36.833690+0000","last_fullsized":"2023-10-08T16:04:36.833690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855327+0000","last_change":"2023-10-08T16:04:36.847842+0000","last_active":"2023-10-08T16:04:36.855327+0000","last_peered":"2023-10-08T16:04:36.855327+0000","last_clean":"2023-10-08T16:04:36.855327+0000","last_became_active":"2023-10-08T16:04:36.847222+0000","last_became_peered":"2023-10-08T16:04:36.847222+0000","last_unstale":"2023-10-08T16:04:36.855327+0000","last_undegraded":"2023-10-08T16:04:36.855327+0000","last_fullsized":"2023-10-08T16:04:36.855327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835188+0000","last_change":"2023-10-08T16:04:35.837295+0000","last_active":"2023-10-08T16:04:36.835188+0000","last_peered":"2023-10-08T16:04:36.835188+0000","last_clean":"2023-10-08T16:04:36.835188+0000","last_became_active":"2023-10-08T16:04:35.836609+0000","last_became_peered":"2023-10-08T16:04:35.836609+0000","last_unstale":"2023-10-08T16:04:36.835188+0000","last_undegraded":"2023-10-08T16:04:36.835188+0000","last_fullsized":"2023-10-08T16:04:36.835188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835250+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:04:36.835250+0000","last_peered":"2023-10-08T16:04:36.835250+0000","last_clean":"2023-10-08T16:04:36.835250+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:04:36.835250+0000","last_undegraded":"2023-10-08T16:04:36.835250+0000","last_fullsized":"2023-10-08T16:04:36.835250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855399+0000","last_change":"2023-10-08T16:04:36.849109+0000","last_active":"2023-10-08T16:04:36.855399+0000","last_peered":"2023-10-08T16:04:36.855399+0000","last_clean":"2023-10-08T16:04:36.855399+0000","last_became_active":"2023-10-08T16:04:36.848847+0000","last_became_peered":"2023-10-08T16:04:36.848847+0000","last_unstale":"2023-10-08T16:04:36.855399+0000","last_undegraded":"2023-10-08T16:04:36.855399+0000","last_fullsized":"2023-10-08T16:04:36.855399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:34:26.769453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835538+0000","last_change":"2023-10-08T16:04:35.840218+0000","last_active":"2023-10-08T16:04:36.835538+0000","last_peered":"2023-10-08T16:04:36.835538+0000","last_clean":"2023-10-08T16:04:36.835538+0000","last_became_active":"2023-10-08T16:04:35.839272+0000","last_became_peered":"2023-10-08T16:04:35.839272+0000","last_unstale":"2023-10-08T16:04:36.835538+0000","last_undegraded":"2023-10-08T16:04:36.835538+0000","last_fullsized":"2023-10-08T16:04:36.835538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836042+0000","last_change":"2023-10-08T16:04:35.842423+0000","last_active":"2023-10-08T16:04:36.836042+0000","last_peered":"2023-10-08T16:04:36.836042+0000","last_clean":"2023-10-08T16:04:36.836042+0000","last_became_active":"2023-10-08T16:04:35.842183+0000","last_became_peered":"2023-10-08T16:04:35.842183+0000","last_unstale":"2023-10-08T16:04:36.836042+0000","last_undegraded":"2023-10-08T16:04:36.836042+0000","last_fullsized":"2023-10-08T16:04:36.836042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839745+0000","last_change":"2023-10-08T16:04:36.837348+0000","last_active":"2023-10-08T16:04:36.839745+0000","last_peered":"2023-10-08T16:04:36.839745+0000","last_clean":"2023-10-08T16:04:36.839745+0000","last_became_active":"2023-10-08T16:04:36.837116+0000","last_became_peered":"2023-10-08T16:04:36.837116+0000","last_unstale":"2023-10-08T16:04:36.839745+0000","last_undegraded":"2023-10-08T16:04:36.839745+0000","last_fullsized":"2023-10-08T16:04:36.839745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T16:04:37.832298+0000","last_change":"2023-10-08T16:04:35.834800+0000","last_active":"2023-10-08T16:04:37.832298+0000","last_peered":"2023-10-08T16:04:37.832298+0000","last_clean":"2023-10-08T16:04:37.832298+0000","last_became_active":"2023-10-08T16:04:35.834516+0000","last_became_peered":"2023-10-08T16:04:35.834516+0000","last_unstale":"2023-10-08T16:04:37.832298+0000","last_undegraded":"2023-10-08T16:04:37.832298+0000","last_fullsized":"2023-10-08T16:04:37.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855314+0000","last_change":"2023-10-08T16:04:36.850191+0000","last_active":"2023-10-08T16:04:36.855314+0000","last_peered":"2023-10-08T16:04:36.855314+0000","last_clean":"2023-10-08T16:04:36.855314+0000","last_became_active":"2023-10-08T16:04:36.849886+0000","last_became_peered":"2023-10-08T16:04:36.849886+0000","last_unstale":"2023-10-08T16:04:36.855314+0000","last_undegraded":"2023-10-08T16:04:36.855314+0000","last_fullsized":"2023-10-08T16:04:36.855314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861589+0000","last_change":"2023-10-08T16:04:35.842733+0000","last_active":"2023-10-08T16:04:36.861589+0000","last_peered":"2023-10-08T16:04:36.861589+0000","last_clean":"2023-10-08T16:04:36.861589+0000","last_became_active":"2023-10-08T16:04:35.842431+0000","last_became_peered":"2023-10-08T16:04:35.842431+0000","last_unstale":"2023-10-08T16:04:36.861589+0000","last_undegraded":"2023-10-08T16:04:36.861589+0000","last_fullsized":"2023-10-08T16:04:36.861589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843518+0000","last_change":"2023-10-08T16:04:36.841209+0000","last_active":"2023-10-08T16:04:36.843518+0000","last_peered":"2023-10-08T16:04:36.843518+0000","last_clean":"2023-10-08T16:04:36.843518+0000","last_became_active":"2023-10-08T16:04:36.840900+0000","last_became_peered":"2023-10-08T16:04:36.840900+0000","last_unstale":"2023-10-08T16:04:36.843518+0000","last_undegraded":"2023-10-08T16:04:36.843518+0000","last_fullsized":"2023-10-08T16:04:36.843518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834548+0000","last_change":"2023-10-08T16:04:35.840034+0000","last_active":"2023-10-08T16:04:36.834548+0000","last_peered":"2023-10-08T16:04:36.834548+0000","last_clean":"2023-10-08T16:04:36.834548+0000","last_became_active":"2023-10-08T16:04:35.839696+0000","last_became_peered":"2023-10-08T16:04:35.839696+0000","last_unstale":"2023-10-08T16:04:36.834548+0000","last_undegraded":"2023-10-08T16:04:36.834548+0000","last_fullsized":"2023-10-08T16:04:36.834548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:51.871435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853454+0000","last_change":"2023-10-08T16:04:36.847014+0000","last_active":"2023-10-08T16:04:36.853454+0000","last_peered":"2023-10-08T16:04:36.853454+0000","last_clean":"2023-10-08T16:04:36.853454+0000","last_became_active":"2023-10-08T16:04:36.846715+0000","last_became_peered":"2023-10-08T16:04:36.846715+0000","last_unstale":"2023-10-08T16:04:36.853454+0000","last_undegraded":"2023-10-08T16:04:36.853454+0000","last_fullsized":"2023-10-08T16:04:36.853454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:13.602001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831854+0000","last_change":"2023-10-08T16:04:35.833211+0000","last_active":"2023-10-08T16:04:36.831854+0000","last_peered":"2023-10-08T16:04:36.831854+0000","last_clean":"2023-10-08T16:04:36.831854+0000","last_became_active":"2023-10-08T16:04:35.832926+0000","last_became_peered":"2023-10-08T16:04:35.832926+0000","last_unstale":"2023-10-08T16:04:36.831854+0000","last_undegraded":"2023-10-08T16:04:36.831854+0000","last_fullsized":"2023-10-08T16:04:36.831854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840582+0000","last_change":"2023-10-08T16:04:36.839516+0000","last_active":"2023-10-08T16:04:36.840582+0000","last_peered":"2023-10-08T16:04:36.840582+0000","last_clean":"2023-10-08T16:04:36.840582+0000","last_became_active":"2023-10-08T16:04:36.839275+0000","last_became_peered":"2023-10-08T16:04:36.839275+0000","last_unstale":"2023-10-08T16:04:36.840582+0000","last_undegraded":"2023-10-08T16:04:36.840582+0000","last_fullsized":"2023-10-08T16:04:36.840582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831607+0000","last_change":"2023-10-08T16:04:35.826206+0000","last_active":"2023-10-08T16:04:36.831607+0000","last_peered":"2023-10-08T16:04:36.831607+0000","last_clean":"2023-10-08T16:04:36.831607+0000","last_became_active":"2023-10-08T16:04:35.825925+0000","last_became_peered":"2023-10-08T16:04:35.825925+0000","last_unstale":"2023-10-08T16:04:36.831607+0000","last_undegraded":"2023-10-08T16:04:36.831607+0000","last_fullsized":"2023-10-08T16:04:36.831607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855072+0000","last_change":"2023-10-08T16:04:36.845340+0000","last_active":"2023-10-08T16:04:36.855072+0000","last_peered":"2023-10-08T16:04:36.855072+0000","last_clean":"2023-10-08T16:04:36.855072+0000","last_became_active":"2023-10-08T16:04:36.845040+0000","last_became_peered":"2023-10-08T16:04:36.845040+0000","last_unstale":"2023-10-08T16:04:36.855072+0000","last_undegraded":"2023-10-08T16:04:36.855072+0000","last_fullsized":"2023-10-08T16:04:36.855072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860563+0000","last_change":"2023-10-08T16:04:35.841423+0000","last_active":"2023-10-08T16:04:36.860563+0000","last_peered":"2023-10-08T16:04:36.860563+0000","last_clean":"2023-10-08T16:04:36.860563+0000","last_became_active":"2023-10-08T16:04:35.841174+0000","last_became_peered":"2023-10-08T16:04:35.841174+0000","last_unstale":"2023-10-08T16:04:36.860563+0000","last_undegraded":"2023-10-08T16:04:36.860563+0000","last_fullsized":"2023-10-08T16:04:36.860563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860494+0000","last_change":"2023-10-08T16:04:36.851799+0000","last_active":"2023-10-08T16:04:36.860494+0000","last_peered":"2023-10-08T16:04:36.860494+0000","last_clean":"2023-10-08T16:04:36.860494+0000","last_became_active":"2023-10-08T16:04:36.851458+0000","last_became_peered":"2023-10-08T16:04:36.851458+0000","last_unstale":"2023-10-08T16:04:36.860494+0000","last_undegraded":"2023-10-08T16:04:36.860494+0000","last_fullsized":"2023-10-08T16:04:36.860494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833797+0000","last_change":"2023-10-08T16:04:35.840122+0000","last_active":"2023-10-08T16:04:36.833797+0000","last_peered":"2023-10-08T16:04:36.833797+0000","last_clean":"2023-10-08T16:04:36.833797+0000","last_became_active":"2023-10-08T16:04:35.839817+0000","last_became_peered":"2023-10-08T16:04:35.839817+0000","last_unstale":"2023-10-08T16:04:36.833797+0000","last_undegraded":"2023-10-08T16:04:36.833797+0000","last_fullsized":"2023-10-08T16:04:36.833797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855128+0000","last_change":"2023-10-08T16:04:36.847797+0000","last_active":"2023-10-08T16:04:36.855128+0000","last_peered":"2023-10-08T16:04:36.855128+0000","last_clean":"2023-10-08T16:04:36.855128+0000","last_became_active":"2023-10-08T16:04:36.847542+0000","last_became_peered":"2023-10-08T16:04:36.847542+0000","last_unstale":"2023-10-08T16:04:36.855128+0000","last_undegraded":"2023-10-08T16:04:36.855128+0000","last_fullsized":"2023-10-08T16:04:36.855128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859329+0000","last_change":"2023-10-08T16:04:35.826935+0000","last_active":"2023-10-08T16:04:36.859329+0000","last_peered":"2023-10-08T16:04:36.859329+0000","last_clean":"2023-10-08T16:04:36.859329+0000","last_became_active":"2023-10-08T16:04:35.826668+0000","last_became_peered":"2023-10-08T16:04:35.826668+0000","last_unstale":"2023-10-08T16:04:36.859329+0000","last_undegraded":"2023-10-08T16:04:36.859329+0000","last_fullsized":"2023-10-08T16:04:36.859329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861167+0000","last_change":"2023-10-08T16:04:36.857918+0000","last_active":"2023-10-08T16:04:36.861167+0000","last_peered":"2023-10-08T16:04:36.861167+0000","last_clean":"2023-10-08T16:04:36.861167+0000","last_became_active":"2023-10-08T16:04:36.857368+0000","last_became_peered":"2023-10-08T16:04:36.857368+0000","last_unstale":"2023-10-08T16:04:36.861167+0000","last_undegraded":"2023-10-08T16:04:36.861167+0000","last_fullsized":"2023-10-08T16:04:36.861167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855827+0000","last_change":"2023-10-08T16:04:35.836755+0000","last_active":"2023-10-08T16:04:36.855827+0000","last_peered":"2023-10-08T16:04:36.855827+0000","last_clean":"2023-10-08T16:04:36.855827+0000","last_became_active":"2023-10-08T16:04:35.836467+0000","last_became_peered":"2023-10-08T16:04:35.836467+0000","last_unstale":"2023-10-08T16:04:36.855827+0000","last_undegraded":"2023-10-08T16:04:36.855827+0000","last_fullsized":"2023-10-08T16:04:36.855827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860389+0000","last_change":"2023-10-08T16:04:36.851314+0000","last_active":"2023-10-08T16:04:36.860389+0000","last_peered":"2023-10-08T16:04:36.860389+0000","last_clean":"2023-10-08T16:04:36.860389+0000","last_became_active":"2023-10-08T16:04:36.850726+0000","last_became_peered":"2023-10-08T16:04:36.850726+0000","last_unstale":"2023-10-08T16:04:36.860389+0000","last_undegraded":"2023-10-08T16:04:36.860389+0000","last_fullsized":"2023-10-08T16:04:36.860389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853164+0000","last_change":"2023-10-08T16:04:36.843832+0000","last_active":"2023-10-08T16:04:36.853164+0000","last_peered":"2023-10-08T16:04:36.853164+0000","last_clean":"2023-10-08T16:04:36.853164+0000","last_became_active":"2023-10-08T16:04:36.843552+0000","last_became_peered":"2023-10-08T16:04:36.843552+0000","last_unstale":"2023-10-08T16:04:36.853164+0000","last_undegraded":"2023-10-08T16:04:36.853164+0000","last_fullsized":"2023-10-08T16:04:36.853164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848791+0000","last_change":"2023-10-08T16:04:36.846568+0000","last_active":"2023-10-08T16:04:36.848791+0000","last_peered":"2023-10-08T16:04:36.848791+0000","last_clean":"2023-10-08T16:04:36.848791+0000","last_became_active":"2023-10-08T16:04:36.846344+0000","last_became_peered":"2023-10-08T16:04:36.846344+0000","last_unstale":"2023-10-08T16:04:36.848791+0000","last_undegraded":"2023-10-08T16:04:36.848791+0000","last_fullsized":"2023-10-08T16:04:36.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861183+0000","last_change":"2023-10-08T16:04:36.857746+0000","last_active":"2023-10-08T16:04:36.861183+0000","last_peered":"2023-10-08T16:04:36.861183+0000","last_clean":"2023-10-08T16:04:36.861183+0000","last_became_active":"2023-10-08T16:04:36.856907+0000","last_became_peered":"2023-10-08T16:04:36.856907+0000","last_unstale":"2023-10-08T16:04:36.861183+0000","last_undegraded":"2023-10-08T16:04:36.861183+0000","last_fullsized":"2023-10-08T16:04:36.861183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.846189+0000","last_change":"2023-10-08T16:04:36.843885+0000","last_active":"2023-10-08T16:04:36.846189+0000","last_peered":"2023-10-08T16:04:36.846189+0000","last_clean":"2023-10-08T16:04:36.846189+0000","last_became_active":"2023-10-08T16:04:36.843640+0000","last_became_peered":"2023-10-08T16:04:36.843640+0000","last_unstale":"2023-10-08T16:04:36.846189+0000","last_undegraded":"2023-10-08T16:04:36.846189+0000","last_fullsized":"2023-10-08T16:04:36.846189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855356+0000","last_change":"2023-10-08T16:04:36.851431+0000","last_active":"2023-10-08T16:04:36.855356+0000","last_peered":"2023-10-08T16:04:36.855356+0000","last_clean":"2023-10-08T16:04:36.855356+0000","last_became_active":"2023-10-08T16:04:36.851215+0000","last_became_peered":"2023-10-08T16:04:36.851215+0000","last_unstale":"2023-10-08T16:04:36.855356+0000","last_undegraded":"2023-10-08T16:04:36.855356+0000","last_fullsized":"2023-10-08T16:04:36.855356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840134+0000","last_change":"2023-10-08T16:04:36.837981+0000","last_active":"2023-10-08T16:04:36.840134+0000","last_peered":"2023-10-08T16:04:36.840134+0000","last_clean":"2023-10-08T16:04:36.840134+0000","last_became_active":"2023-10-08T16:04:36.837478+0000","last_became_peered":"2023-10-08T16:04:36.837478+0000","last_unstale":"2023-10-08T16:04:36.840134+0000","last_undegraded":"2023-10-08T16:04:36.840134+0000","last_fullsized":"2023-10-08T16:04:36.840134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855950+0000","last_change":"2023-10-08T16:04:36.850056+0000","last_active":"2023-10-08T16:04:36.855950+0000","last_peered":"2023-10-08T16:04:36.855950+0000","last_clean":"2023-10-08T16:04:36.855950+0000","last_became_active":"2023-10-08T16:04:36.849732+0000","last_became_peered":"2023-10-08T16:04:36.849732+0000","last_unstale":"2023-10-08T16:04:36.855950+0000","last_undegraded":"2023-10-08T16:04:36.855950+0000","last_fullsized":"2023-10-08T16:04:36.855950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843571+0000","last_change":"2023-10-08T16:04:36.842352+0000","last_active":"2023-10-08T16:04:36.843571+0000","last_peered":"2023-10-08T16:04:36.843571+0000","last_clean":"2023-10-08T16:04:36.843571+0000","last_became_active":"2023-10-08T16:04:36.842078+0000","last_became_peered":"2023-10-08T16:04:36.842078+0000","last_unstale":"2023-10-08T16:04:36.843571+0000","last_undegraded":"2023-10-08T16:04:36.843571+0000","last_fullsized":"2023-10-08T16:04:36.843571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861173+0000","last_change":"2023-10-08T16:04:36.858170+0000","last_active":"2023-10-08T16:04:36.861173+0000","last_peered":"2023-10-08T16:04:36.861173+0000","last_clean":"2023-10-08T16:04:36.861173+0000","last_became_active":"2023-10-08T16:04:36.857822+0000","last_became_peered":"2023-10-08T16:04:36.857822+0000","last_unstale":"2023-10-08T16:04:36.861173+0000","last_undegraded":"2023-10-08T16:04:36.861173+0000","last_fullsized":"2023-10-08T16:04:36.861173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855132+0000","last_change":"2023-10-08T16:04:36.847659+0000","last_active":"2023-10-08T16:04:36.855132+0000","last_peered":"2023-10-08T16:04:36.855132+0000","last_clean":"2023-10-08T16:04:36.855132+0000","last_became_active":"2023-10-08T16:04:36.847337+0000","last_became_peered":"2023-10-08T16:04:36.847337+0000","last_unstale":"2023-10-08T16:04:36.855132+0000","last_undegraded":"2023-10-08T16:04:36.855132+0000","last_fullsized":"2023-10-08T16:04:36.855132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855842+0000","last_change":"2023-10-08T16:04:36.850206+0000","last_active":"2023-10-08T16:04:36.855842+0000","last_peered":"2023-10-08T16:04:36.855842+0000","last_clean":"2023-10-08T16:04:36.855842+0000","last_became_active":"2023-10-08T16:04:36.849892+0000","last_became_peered":"2023-10-08T16:04:36.849892+0000","last_unstale":"2023-10-08T16:04:36.855842+0000","last_undegraded":"2023-10-08T16:04:36.855842+0000","last_fullsized":"2023-10-08T16:04:36.855842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848345+0000","last_change":"2023-10-08T16:04:36.846291+0000","last_active":"2023-10-08T16:04:36.848345+0000","last_peered":"2023-10-08T16:04:36.848345+0000","last_clean":"2023-10-08T16:04:36.848345+0000","last_became_active":"2023-10-08T16:04:36.846030+0000","last_became_peered":"2023-10-08T16:04:36.846030+0000","last_unstale":"2023-10-08T16:04:36.848345+0000","last_undegraded":"2023-10-08T16:04:36.848345+0000","last_fullsized":"2023-10-08T16:04:36.848345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860629+0000","last_change":"2023-10-08T16:04:36.852345+0000","last_active":"2023-10-08T16:04:36.860629+0000","last_peered":"2023-10-08T16:04:36.860629+0000","last_clean":"2023-10-08T16:04:36.860629+0000","last_became_active":"2023-10-08T16:04:36.852087+0000","last_became_peered":"2023-10-08T16:04:36.852087+0000","last_unstale":"2023-10-08T16:04:36.860629+0000","last_undegraded":"2023-10-08T16:04:36.860629+0000","last_fullsized":"2023-10-08T16:04:36.860629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":666235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":666291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":75439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:04:45.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:04:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.587+0000 7fc95fabf700 1 Processor -- start 2023-10-08T16:04:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 -- start start 2023-10-08T16:04:45.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc958159890 0x7fc95815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc958069560 0x7fc9580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc958069ee0 con 0x7fc958159c60 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95815a160 con 0x7fc958159890 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.588+0000 7fc95fabf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9580d13a0 con 0x7fc958069560 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc958159890 0x7fc95815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d05a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc958159c60 0x7fc95815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51236/0 (socket says 172.21.15.170:51236) 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d05a700 1 -- 172.21.15.170:0/4013952959 learned_addr learned my addr 172.21.15.170:0/4013952959 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95e05c700 1 --2- 172.21.15.170:0/4013952959 >> v2:172.21.15.191:3300/0 conn(0x7fc958069560 0x7fc9580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 -- 172.21.15.170:0/4013952959 >> v2:172.21.15.191:3300/0 conn(0x7fc958069560 msgr2=0x7fc9580699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 --2- 172.21.15.170:0/4013952959 >> v2:172.21.15.191:3300/0 conn(0x7fc958069560 0x7fc9580699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 -- 172.21.15.170:0/4013952959 >> v1:172.21.15.170:6789/0 conn(0x7fc958159c60 legacy=0x7fc95815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95c859700 1 -- 172.21.15.170:0/4013952959 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113187760 0 0) 0x7fc958069ee0 con 0x7fc958159c60 2023-10-08T16:04:45.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 -- 172.21.15.170:0/4013952959 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95815e140 con 0x7fc958159890 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95d85b700 1 --2- 172.21.15.170:0/4013952959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958159890 0x7fc95815abd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc95806a890 tx=0x7fc948009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea6eebbe86be97f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95c859700 1 -- 172.21.15.170:0/4013952959 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800e070 con 0x7fc958159890 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95c859700 1 -- 172.21.15.170:0/4013952959 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800bc30 con 0x7fc958159890 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.589+0000 7fc95c859700 1 -- 172.21.15.170:0/4013952959 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948012430 con 0x7fc958159890 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 -- 172.21.15.170:0/4013952959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958159890 msgr2=0x7fc95815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 --2- 172.21.15.170:0/4013952959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958159890 0x7fc95815abd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc95806a890 tx=0x7fc948009780 comp rx=0 tx=0).stop 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 -- 172.21.15.170:0/4013952959 shutdown_connections 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 --2- 172.21.15.170:0/4013952959 >> v2:172.21.15.191:3300/0 conn(0x7fc958069560 0x7fc9580699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 --2- 172.21.15.170:0/4013952959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958159890 0x7fc95815abd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 -- 172.21.15.170:0/4013952959 >> 172.21.15.170:0/4013952959 conn(0x7fc958068e20 msgr2=0x7fc9580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 -- 172.21.15.170:0/4013952959 shutdown_connections 2023-10-08T16:04:45.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.590+0000 7fc95fabf700 1 -- 172.21.15.170:0/4013952959 wait complete. 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.591+0000 7fc95fabf700 1 Processor -- start 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.591+0000 7fc95fabf700 1 -- start start 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 0x7fc9580b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc958159c60 0x7fc9580b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580ce720 con 0x7fc958069560 2023-10-08T16:04:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580b60c0 con 0x7fc958159890 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580b63a0 con 0x7fc958159c60 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58188/0 (socket says 172.21.15.170:58188) 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 -- 172.21.15.170:0/1275790042 learned_addr learned my addr 172.21.15.170:0/1275790042 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95e05c700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc958159c60 0x7fc9580b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d05a700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 0x7fc9580b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 -- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc958159c60 msgr2=0x7fc9580b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc958159c60 0x7fc9580b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 -- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 msgr2=0x7fc9580b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 0x7fc9580b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 -- 172.21.15.170:0/1275790042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948009a20 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d05a700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 0x7fc9580b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95d85b700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9480099f0 tx=0x7fc948019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948001090 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94801d630 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9580ce0d0 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.592+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94801d9f0 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.593+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9580cdcd0 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.593+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95804c3c0 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.594+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc948024070 con 0x7fc958069560 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.594+0000 7fc955ffb700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 0x7fc9440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.594+0000 7fc95d05a700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 0x7fc9440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.594+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc948086da0 con 0x7fc958069560 2023-10-08T16:04:45.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.595+0000 7fc95d05a700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 0x7fc9440693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc9580cd690 tx=0x7fc94c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.599+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc948053f70 con 0x7fc958069560 2023-10-08T16:04:45.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.775+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9580ce860 con 0x7fc944066f30 2023-10-08T16:04:45.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.783+0000 7fc955ffb700 1 -- 172.21.15.170:0/1275790042 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264507 (secure 0 0 0) 0x7fc94c001c50 con 0x7fc944066f30 2023-10-08T16:04:45.787 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.785+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 msgr2=0x7fc9440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.785+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 0x7fc9440693e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc9580cd690 tx=0x7fc94c009040 comp rx=0 tx=0).stop 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.785+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 msgr2=0x7fc9581528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.785+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc9480099f0 tx=0x7fc948019000 comp rx=0 tx=0).stop 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 shutdown_connections 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc958159c60 0x7fc9580b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc944066f30 0x7fc9440693e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc958069560 0x7fc9581528c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 --2- 172.21.15.170:0/1275790042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc958159890 0x7fc9580b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 >> 172.21.15.170:0/1275790042 conn(0x7fc958068e20 msgr2=0x7fc9580b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 shutdown_connections 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.786+0000 7fc95fabf700 1 -- 172.21.15.170:0/1275790042 wait complete. 2023-10-08T16:04:45.790 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:04:45.799 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T16:04:44.225828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381224,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593496,"statfs":{"total":773094113280,"available":770655739904,"internally_reserved":0,"allocated":2990080,"data_stored":1787852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001635"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854867+0000","last_change":"2023-10-08T16:04:36.847335+0000","last_active":"2023-10-08T16:04:36.854867+0000","last_peered":"2023-10-08T16:04:36.854867+0000","last_clean":"2023-10-08T16:04:36.854867+0000","last_became_active":"2023-10-08T16:04:36.846900+0000","last_became_peered":"2023-10-08T16:04:36.846900+0000","last_unstale":"2023-10-08T16:04:36.854867+0000","last_undegraded":"2023-10-08T16:04:36.854867+0000","last_fullsized":"2023-10-08T16:04:36.854867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.847766+0000","last_change":"2023-10-08T16:04:36.841973+0000","last_active":"2023-10-08T16:04:36.847766+0000","last_peered":"2023-10-08T16:04:36.847766+0000","last_clean":"2023-10-08T16:04:36.847766+0000","last_became_active":"2023-10-08T16:04:36.841752+0000","last_became_peered":"2023-10-08T16:04:36.841752+0000","last_unstale":"2023-10-08T16:04:36.847766+0000","last_undegraded":"2023-10-08T16:04:36.847766+0000","last_fullsized":"2023-10-08T16:04:36.847766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844363+0000","last_change":"2023-10-08T16:04:36.841982+0000","last_active":"2023-10-08T16:04:36.844363+0000","last_peered":"2023-10-08T16:04:36.844363+0000","last_clean":"2023-10-08T16:04:36.844363+0000","last_became_active":"2023-10-08T16:04:36.841738+0000","last_became_peered":"2023-10-08T16:04:36.841738+0000","last_unstale":"2023-10-08T16:04:36.844363+0000","last_undegraded":"2023-10-08T16:04:36.844363+0000","last_fullsized":"2023-10-08T16:04:36.844363+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:28.994728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861145+0000","last_change":"2023-10-08T16:04:36.857688+0000","last_active":"2023-10-08T16:04:36.861145+0000","last_peered":"2023-10-08T16:04:36.861145+0000","last_clean":"2023-10-08T16:04:36.861145+0000","last_became_active":"2023-10-08T16:04:36.857001+0000","last_became_peered":"2023-10-08T16:04:36.857001+0000","last_unstale":"2023-10-08T16:04:36.861145+0000","last_undegraded":"2023-10-08T16:04:36.861145+0000","last_fullsized":"2023-10-08T16:04:36.861145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848398+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848398+0000","last_peered":"2023-10-08T16:04:36.848398+0000","last_clean":"2023-10-08T16:04:36.848398+0000","last_became_active":"2023-10-08T16:04:36.842806+0000","last_became_peered":"2023-10-08T16:04:36.842806+0000","last_unstale":"2023-10-08T16:04:36.848398+0000","last_undegraded":"2023-10-08T16:04:36.848398+0000","last_fullsized":"2023-10-08T16:04:36.848398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854094+0000","last_change":"2023-10-08T16:04:36.845475+0000","last_active":"2023-10-08T16:04:36.854094+0000","last_peered":"2023-10-08T16:04:36.854094+0000","last_clean":"2023-10-08T16:04:36.854094+0000","last_became_active":"2023-10-08T16:04:36.845053+0000","last_became_peered":"2023-10-08T16:04:36.845053+0000","last_unstale":"2023-10-08T16:04:36.854094+0000","last_undegraded":"2023-10-08T16:04:36.854094+0000","last_fullsized":"2023-10-08T16:04:36.854094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848364+0000","last_change":"2023-10-08T16:04:36.843146+0000","last_active":"2023-10-08T16:04:36.848364+0000","last_peered":"2023-10-08T16:04:36.848364+0000","last_clean":"2023-10-08T16:04:36.848364+0000","last_became_active":"2023-10-08T16:04:36.842841+0000","last_became_peered":"2023-10-08T16:04:36.842841+0000","last_unstale":"2023-10-08T16:04:36.848364+0000","last_undegraded":"2023-10-08T16:04:36.848364+0000","last_fullsized":"2023-10-08T16:04:36.848364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855557+0000","last_change":"2023-10-08T16:04:36.850496+0000","last_active":"2023-10-08T16:04:36.855557+0000","last_peered":"2023-10-08T16:04:36.855557+0000","last_clean":"2023-10-08T16:04:36.855557+0000","last_became_active":"2023-10-08T16:04:36.850237+0000","last_became_peered":"2023-10-08T16:04:36.850237+0000","last_unstale":"2023-10-08T16:04:36.855557+0000","last_undegraded":"2023-10-08T16:04:36.855557+0000","last_fullsized":"2023-10-08T16:04:36.855557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854848+0000","last_change":"2023-10-08T16:04:36.847152+0000","last_active":"2023-10-08T16:04:36.854848+0000","last_peered":"2023-10-08T16:04:36.854848+0000","last_clean":"2023-10-08T16:04:36.854848+0000","last_became_active":"2023-10-08T16:04:36.846573+0000","last_became_peered":"2023-10-08T16:04:36.846573+0000","last_unstale":"2023-10-08T16:04:36.854848+0000","last_undegraded":"2023-10-08T16:04:36.854848+0000","last_fullsized":"2023-10-08T16:04:36.854848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853411+0000","last_change":"2023-10-08T16:04:36.847416+0000","last_active":"2023-10-08T16:04:36.853411+0000","last_peered":"2023-10-08T16:04:36.853411+0000","last_clean":"2023-10-08T16:04:36.853411+0000","last_became_active":"2023-10-08T16:04:36.847092+0000","last_became_peered":"2023-10-08T16:04:36.847092+0000","last_unstale":"2023-10-08T16:04:36.853411+0000","last_undegraded":"2023-10-08T16:04:36.853411+0000","last_fullsized":"2023-10-08T16:04:36.853411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860614+0000","last_change":"2023-10-08T16:04:36.854204+0000","last_active":"2023-10-08T16:04:36.860614+0000","last_peered":"2023-10-08T16:04:36.860614+0000","last_clean":"2023-10-08T16:04:36.860614+0000","last_became_active":"2023-10-08T16:04:36.853963+0000","last_became_peered":"2023-10-08T16:04:36.853963+0000","last_unstale":"2023-10-08T16:04:36.860614+0000","last_undegraded":"2023-10-08T16:04:36.860614+0000","last_fullsized":"2023-10-08T16:04:36.860614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859253+0000","last_change":"2023-10-08T16:04:36.855269+0000","last_active":"2023-10-08T16:04:36.859253+0000","last_peered":"2023-10-08T16:04:36.859253+0000","last_clean":"2023-10-08T16:04:36.859253+0000","last_became_active":"2023-10-08T16:04:36.854443+0000","last_became_peered":"2023-10-08T16:04:36.854443+0000","last_unstale":"2023-10-08T16:04:36.859253+0000","last_undegraded":"2023-10-08T16:04:36.859253+0000","last_fullsized":"2023-10-08T16:04:36.859253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844837+0000","last_change":"2023-10-08T16:04:36.842900+0000","last_active":"2023-10-08T16:04:36.844837+0000","last_peered":"2023-10-08T16:04:36.844837+0000","last_clean":"2023-10-08T16:04:36.844837+0000","last_became_active":"2023-10-08T16:04:36.842642+0000","last_became_peered":"2023-10-08T16:04:36.842642+0000","last_unstale":"2023-10-08T16:04:36.844837+0000","last_undegraded":"2023-10-08T16:04:36.844837+0000","last_fullsized":"2023-10-08T16:04:36.844837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844389+0000","last_change":"2023-10-08T16:04:36.842577+0000","last_active":"2023-10-08T16:04:36.844389+0000","last_peered":"2023-10-08T16:04:36.844389+0000","last_clean":"2023-10-08T16:04:36.844389+0000","last_became_active":"2023-10-08T16:04:36.841699+0000","last_became_peered":"2023-10-08T16:04:36.841699+0000","last_unstale":"2023-10-08T16:04:36.844389+0000","last_undegraded":"2023-10-08T16:04:36.844389+0000","last_fullsized":"2023-10-08T16:04:36.844389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850510+0000","last_change":"2023-10-08T16:04:36.845580+0000","last_active":"2023-10-08T16:04:36.850510+0000","last_peered":"2023-10-08T16:04:36.850510+0000","last_clean":"2023-10-08T16:04:36.850510+0000","last_became_active":"2023-10-08T16:04:36.845348+0000","last_became_peered":"2023-10-08T16:04:36.845348+0000","last_unstale":"2023-10-08T16:04:36.850510+0000","last_undegraded":"2023-10-08T16:04:36.850510+0000","last_fullsized":"2023-10-08T16:04:36.850510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.854086+0000","last_change":"2023-10-08T16:04:36.845629+0000","last_active":"2023-10-08T16:04:36.854086+0000","last_peered":"2023-10-08T16:04:36.854086+0000","last_clean":"2023-10-08T16:04:36.854086+0000","last_became_active":"2023-10-08T16:04:36.845283+0000","last_became_peered":"2023-10-08T16:04:36.845283+0000","last_unstale":"2023-10-08T16:04:36.854086+0000","last_undegraded":"2023-10-08T16:04:36.854086+0000","last_fullsized":"2023-10-08T16:04:36.854086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856506+0000","last_change":"2023-10-08T16:04:36.852540+0000","last_active":"2023-10-08T16:04:36.856506+0000","last_peered":"2023-10-08T16:04:36.856506+0000","last_clean":"2023-10-08T16:04:36.856506+0000","last_became_active":"2023-10-08T16:04:36.852294+0000","last_became_peered":"2023-10-08T16:04:36.852294+0000","last_unstale":"2023-10-08T16:04:36.856506+0000","last_undegraded":"2023-10-08T16:04:36.856506+0000","last_fullsized":"2023-10-08T16:04:36.856506+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:16.564895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850863+0000","last_change":"2023-10-08T16:04:36.845236+0000","last_active":"2023-10-08T16:04:36.850863+0000","last_peered":"2023-10-08T16:04:36.850863+0000","last_clean":"2023-10-08T16:04:36.850863+0000","last_became_active":"2023-10-08T16:04:36.844966+0000","last_became_peered":"2023-10-08T16:04:36.844966+0000","last_unstale":"2023-10-08T16:04:36.850863+0000","last_undegraded":"2023-10-08T16:04:36.850863+0000","last_fullsized":"2023-10-08T16:04:36.850863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:00.230855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849074+0000","last_change":"2023-10-08T16:04:36.844871+0000","last_active":"2023-10-08T16:04:36.849074+0000","last_peered":"2023-10-08T16:04:36.849074+0000","last_clean":"2023-10-08T16:04:36.849074+0000","last_became_active":"2023-10-08T16:04:36.844378+0000","last_became_peered":"2023-10-08T16:04:36.844378+0000","last_unstale":"2023-10-08T16:04:36.849074+0000","last_undegraded":"2023-10-08T16:04:36.849074+0000","last_fullsized":"2023-10-08T16:04:36.849074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861810+0000","last_change":"2023-10-08T16:04:35.842581+0000","last_active":"2023-10-08T16:04:36.861810+0000","last_peered":"2023-10-08T16:04:36.861810+0000","last_clean":"2023-10-08T16:04:36.861810+0000","last_became_active":"2023-10-08T16:04:35.842258+0000","last_became_peered":"2023-10-08T16:04:35.842258+0000","last_unstale":"2023-10-08T16:04:36.861810+0000","last_undegraded":"2023-10-08T16:04:36.861810+0000","last_fullsized":"2023-10-08T16:04:36.861810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:04.459691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.844626+0000","last_change":"2023-10-08T16:04:36.842889+0000","last_active":"2023-10-08T16:04:36.844626+0000","last_peered":"2023-10-08T16:04:36.844626+0000","last_clean":"2023-10-08T16:04:36.844626+0000","last_became_active":"2023-10-08T16:04:36.842161+0000","last_became_peered":"2023-10-08T16:04:36.842161+0000","last_unstale":"2023-10-08T16:04:36.844626+0000","last_undegraded":"2023-10-08T16:04:36.844626+0000","last_fullsized":"2023-10-08T16:04:36.844626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856629+0000","last_change":"2023-10-08T16:04:35.835973+0000","last_active":"2023-10-08T16:04:36.856629+0000","last_peered":"2023-10-08T16:04:36.856629+0000","last_clean":"2023-10-08T16:04:36.856629+0000","last_became_active":"2023-10-08T16:04:35.835785+0000","last_became_peered":"2023-10-08T16:04:35.835785+0000","last_unstale":"2023-10-08T16:04:36.856629+0000","last_undegraded":"2023-10-08T16:04:36.856629+0000","last_fullsized":"2023-10-08T16:04:36.856629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849375+0000","last_change":"2023-10-08T16:04:36.845806+0000","last_active":"2023-10-08T16:04:36.849375+0000","last_peered":"2023-10-08T16:04:36.849375+0000","last_clean":"2023-10-08T16:04:36.849375+0000","last_became_active":"2023-10-08T16:04:36.845574+0000","last_became_peered":"2023-10-08T16:04:36.845574+0000","last_unstale":"2023-10-08T16:04:36.849375+0000","last_undegraded":"2023-10-08T16:04:36.849375+0000","last_fullsized":"2023-10-08T16:04:36.849375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860925+0000","last_change":"2023-10-08T16:04:35.844449+0000","last_active":"2023-10-08T16:04:36.860925+0000","last_peered":"2023-10-08T16:04:36.860925+0000","last_clean":"2023-10-08T16:04:36.860925+0000","last_became_active":"2023-10-08T16:04:35.844192+0000","last_became_peered":"2023-10-08T16:04:35.844192+0000","last_unstale":"2023-10-08T16:04:36.860925+0000","last_undegraded":"2023-10-08T16:04:36.860925+0000","last_fullsized":"2023-10-08T16:04:36.860925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856262+0000","last_change":"2023-10-08T16:04:36.853135+0000","last_active":"2023-10-08T16:04:36.856262+0000","last_peered":"2023-10-08T16:04:36.856262+0000","last_clean":"2023-10-08T16:04:36.856262+0000","last_became_active":"2023-10-08T16:04:36.852832+0000","last_became_peered":"2023-10-08T16:04:36.852832+0000","last_unstale":"2023-10-08T16:04:36.856262+0000","last_undegraded":"2023-10-08T16:04:36.856262+0000","last_fullsized":"2023-10-08T16:04:36.856262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:35:45.830318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861744+0000","last_change":"2023-10-08T16:04:35.845418+0000","last_active":"2023-10-08T16:04:36.861744+0000","last_peered":"2023-10-08T16:04:36.861744+0000","last_clean":"2023-10-08T16:04:36.861744+0000","last_became_active":"2023-10-08T16:04:35.845104+0000","last_became_peered":"2023-10-08T16:04:35.845104+0000","last_unstale":"2023-10-08T16:04:36.861744+0000","last_undegraded":"2023-10-08T16:04:36.861744+0000","last_fullsized":"2023-10-08T16:04:36.861744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856172+0000","last_change":"2023-10-08T16:04:36.851677+0000","last_active":"2023-10-08T16:04:36.856172+0000","last_peered":"2023-10-08T16:04:36.856172+0000","last_clean":"2023-10-08T16:04:36.856172+0000","last_became_active":"2023-10-08T16:04:36.847960+0000","last_became_peered":"2023-10-08T16:04:36.847960+0000","last_unstale":"2023-10-08T16:04:36.856172+0000","last_undegraded":"2023-10-08T16:04:36.856172+0000","last_fullsized":"2023-10-08T16:04:36.856172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:10:51.960686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861300+0000","last_change":"2023-10-08T16:04:35.833685+0000","last_active":"2023-10-08T16:04:36.861300+0000","last_peered":"2023-10-08T16:04:36.861300+0000","last_clean":"2023-10-08T16:04:36.861300+0000","last_became_active":"2023-10-08T16:04:35.833403+0000","last_became_peered":"2023-10-08T16:04:35.833403+0000","last_unstale":"2023-10-08T16:04:36.861300+0000","last_undegraded":"2023-10-08T16:04:36.861300+0000","last_fullsized":"2023-10-08T16:04:36.861300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851430+0000","last_change":"2023-10-08T16:04:36.845487+0000","last_active":"2023-10-08T16:04:36.851430+0000","last_peered":"2023-10-08T16:04:36.851430+0000","last_clean":"2023-10-08T16:04:36.851430+0000","last_became_active":"2023-10-08T16:04:36.845204+0000","last_became_peered":"2023-10-08T16:04:36.845204+0000","last_unstale":"2023-10-08T16:04:36.851430+0000","last_undegraded":"2023-10-08T16:04:36.851430+0000","last_fullsized":"2023-10-08T16:04:36.851430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:37.980506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861219+0000","last_change":"2023-10-08T16:04:35.833842+0000","last_active":"2023-10-08T16:04:36.861219+0000","last_peered":"2023-10-08T16:04:36.861219+0000","last_clean":"2023-10-08T16:04:36.861219+0000","last_became_active":"2023-10-08T16:04:35.833461+0000","last_became_peered":"2023-10-08T16:04:35.833461+0000","last_unstale":"2023-10-08T16:04:36.861219+0000","last_undegraded":"2023-10-08T16:04:36.861219+0000","last_fullsized":"2023-10-08T16:04:36.861219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855228+0000","last_change":"2023-10-08T16:04:36.849156+0000","last_active":"2023-10-08T16:04:36.855228+0000","last_peered":"2023-10-08T16:04:36.855228+0000","last_clean":"2023-10-08T16:04:36.855228+0000","last_became_active":"2023-10-08T16:04:36.848907+0000","last_became_peered":"2023-10-08T16:04:36.848907+0000","last_unstale":"2023-10-08T16:04:36.855228+0000","last_undegraded":"2023-10-08T16:04:36.855228+0000","last_fullsized":"2023-10-08T16:04:36.855228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:04.498124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834426+0000","last_change":"2023-10-08T16:04:35.838361+0000","last_active":"2023-10-08T16:04:36.834426+0000","last_peered":"2023-10-08T16:04:36.834426+0000","last_clean":"2023-10-08T16:04:36.834426+0000","last_became_active":"2023-10-08T16:04:35.838175+0000","last_became_peered":"2023-10-08T16:04:35.838175+0000","last_unstale":"2023-10-08T16:04:36.834426+0000","last_undegraded":"2023-10-08T16:04:36.834426+0000","last_fullsized":"2023-10-08T16:04:36.834426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.851367+0000","last_change":"2023-10-08T16:04:36.843489+0000","last_active":"2023-10-08T16:04:36.851367+0000","last_peered":"2023-10-08T16:04:36.851367+0000","last_clean":"2023-10-08T16:04:36.851367+0000","last_became_active":"2023-10-08T16:04:36.843221+0000","last_became_peered":"2023-10-08T16:04:36.843221+0000","last_unstale":"2023-10-08T16:04:36.851367+0000","last_undegraded":"2023-10-08T16:04:36.851367+0000","last_fullsized":"2023-10-08T16:04:36.851367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840321+0000","last_change":"2023-10-08T16:04:35.839307+0000","last_active":"2023-10-08T16:04:36.840321+0000","last_peered":"2023-10-08T16:04:36.840321+0000","last_clean":"2023-10-08T16:04:36.840321+0000","last_became_active":"2023-10-08T16:04:35.838997+0000","last_became_peered":"2023-10-08T16:04:35.838997+0000","last_unstale":"2023-10-08T16:04:36.840321+0000","last_undegraded":"2023-10-08T16:04:36.840321+0000","last_fullsized":"2023-10-08T16:04:36.840321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840059+0000","last_change":"2023-10-08T16:04:36.838211+0000","last_active":"2023-10-08T16:04:36.840059+0000","last_peered":"2023-10-08T16:04:36.840059+0000","last_clean":"2023-10-08T16:04:36.840059+0000","last_became_active":"2023-10-08T16:04:36.837948+0000","last_became_peered":"2023-10-08T16:04:36.837948+0000","last_unstale":"2023-10-08T16:04:36.840059+0000","last_undegraded":"2023-10-08T16:04:36.840059+0000","last_fullsized":"2023-10-08T16:04:36.840059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838498+0000","last_change":"2023-10-08T16:04:35.841004+0000","last_active":"2023-10-08T16:04:36.838498+0000","last_peered":"2023-10-08T16:04:36.838498+0000","last_clean":"2023-10-08T16:04:36.838498+0000","last_became_active":"2023-10-08T16:04:35.840735+0000","last_became_peered":"2023-10-08T16:04:35.840735+0000","last_unstale":"2023-10-08T16:04:36.838498+0000","last_undegraded":"2023-10-08T16:04:36.838498+0000","last_fullsized":"2023-10-08T16:04:36.838498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855922+0000","last_change":"2023-10-08T16:04:36.850043+0000","last_active":"2023-10-08T16:04:36.855922+0000","last_peered":"2023-10-08T16:04:36.855922+0000","last_clean":"2023-10-08T16:04:36.855922+0000","last_became_active":"2023-10-08T16:04:36.849743+0000","last_became_peered":"2023-10-08T16:04:36.849743+0000","last_unstale":"2023-10-08T16:04:36.855922+0000","last_undegraded":"2023-10-08T16:04:36.855922+0000","last_fullsized":"2023-10-08T16:04:36.855922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838528+0000","last_change":"2023-10-08T16:04:35.844419+0000","last_active":"2023-10-08T16:04:36.838528+0000","last_peered":"2023-10-08T16:04:36.838528+0000","last_clean":"2023-10-08T16:04:36.838528+0000","last_became_active":"2023-10-08T16:04:35.844125+0000","last_became_peered":"2023-10-08T16:04:35.844125+0000","last_unstale":"2023-10-08T16:04:36.838528+0000","last_undegraded":"2023-10-08T16:04:36.838528+0000","last_fullsized":"2023-10-08T16:04:36.838528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849356+0000","last_change":"2023-10-08T16:04:36.845771+0000","last_active":"2023-10-08T16:04:36.849356+0000","last_peered":"2023-10-08T16:04:36.849356+0000","last_clean":"2023-10-08T16:04:36.849356+0000","last_became_active":"2023-10-08T16:04:36.845274+0000","last_became_peered":"2023-10-08T16:04:36.845274+0000","last_unstale":"2023-10-08T16:04:36.849356+0000","last_undegraded":"2023-10-08T16:04:36.849356+0000","last_fullsized":"2023-10-08T16:04:36.849356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856530+0000","last_change":"2023-10-08T16:04:35.840602+0000","last_active":"2023-10-08T16:04:36.856530+0000","last_peered":"2023-10-08T16:04:36.856530+0000","last_clean":"2023-10-08T16:04:36.856530+0000","last_became_active":"2023-10-08T16:04:35.840299+0000","last_became_peered":"2023-10-08T16:04:35.840299+0000","last_unstale":"2023-10-08T16:04:36.856530+0000","last_undegraded":"2023-10-08T16:04:36.856530+0000","last_fullsized":"2023-10-08T16:04:36.856530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853166+0000","last_change":"2023-10-08T16:04:36.845842+0000","last_active":"2023-10-08T16:04:36.853166+0000","last_peered":"2023-10-08T16:04:36.853166+0000","last_clean":"2023-10-08T16:04:36.853166+0000","last_became_active":"2023-10-08T16:04:36.844954+0000","last_became_peered":"2023-10-08T16:04:36.844954+0000","last_unstale":"2023-10-08T16:04:36.853166+0000","last_undegraded":"2023-10-08T16:04:36.853166+0000","last_fullsized":"2023-10-08T16:04:36.853166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:34.979889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.863226+0000","last_change":"2023-10-08T16:04:35.838972+0000","last_active":"2023-10-08T16:04:36.863226+0000","last_peered":"2023-10-08T16:04:36.863226+0000","last_clean":"2023-10-08T16:04:36.863226+0000","last_became_active":"2023-10-08T16:04:35.838694+0000","last_became_peered":"2023-10-08T16:04:35.838694+0000","last_unstale":"2023-10-08T16:04:36.863226+0000","last_undegraded":"2023-10-08T16:04:36.863226+0000","last_fullsized":"2023-10-08T16:04:36.863226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.849056+0000","last_change":"2023-10-08T16:04:36.844679+0000","last_active":"2023-10-08T16:04:36.849056+0000","last_peered":"2023-10-08T16:04:36.849056+0000","last_clean":"2023-10-08T16:04:36.849056+0000","last_became_active":"2023-10-08T16:04:36.844115+0000","last_became_peered":"2023-10-08T16:04:36.844115+0000","last_unstale":"2023-10-08T16:04:36.849056+0000","last_undegraded":"2023-10-08T16:04:36.849056+0000","last_fullsized":"2023-10-08T16:04:36.849056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:21.608359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859530+0000","last_change":"2023-10-08T16:04:35.837996+0000","last_active":"2023-10-08T16:04:36.859530+0000","last_peered":"2023-10-08T16:04:36.859530+0000","last_clean":"2023-10-08T16:04:36.859530+0000","last_became_active":"2023-10-08T16:04:35.837567+0000","last_became_peered":"2023-10-08T16:04:35.837567+0000","last_unstale":"2023-10-08T16:04:36.859530+0000","last_undegraded":"2023-10-08T16:04:36.859530+0000","last_fullsized":"2023-10-08T16:04:36.859530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:47:05.546574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855385+0000","last_change":"2023-10-08T16:04:36.849839+0000","last_active":"2023-10-08T16:04:36.855385+0000","last_peered":"2023-10-08T16:04:36.855385+0000","last_clean":"2023-10-08T16:04:36.855385+0000","last_became_active":"2023-10-08T16:04:36.849577+0000","last_became_peered":"2023-10-08T16:04:36.849577+0000","last_unstale":"2023-10-08T16:04:36.855385+0000","last_undegraded":"2023-10-08T16:04:36.855385+0000","last_fullsized":"2023-10-08T16:04:36.855385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:55.787779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838421+0000","last_change":"2023-10-08T16:04:35.841168+0000","last_active":"2023-10-08T16:04:36.838421+0000","last_peered":"2023-10-08T16:04:36.838421+0000","last_clean":"2023-10-08T16:04:36.838421+0000","last_became_active":"2023-10-08T16:04:35.840667+0000","last_became_peered":"2023-10-08T16:04:35.840667+0000","last_unstale":"2023-10-08T16:04:36.838421+0000","last_undegraded":"2023-10-08T16:04:36.838421+0000","last_fullsized":"2023-10-08T16:04:36.838421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859673+0000","last_change":"2023-10-08T16:04:36.849407+0000","last_active":"2023-10-08T16:04:36.859673+0000","last_peered":"2023-10-08T16:04:36.859673+0000","last_clean":"2023-10-08T16:04:36.859673+0000","last_became_active":"2023-10-08T16:04:36.849157+0000","last_became_peered":"2023-10-08T16:04:36.849157+0000","last_unstale":"2023-10-08T16:04:36.859673+0000","last_undegraded":"2023-10-08T16:04:36.859673+0000","last_fullsized":"2023-10-08T16:04:36.859673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836282+0000","last_change":"2023-10-08T16:04:35.835032+0000","last_active":"2023-10-08T16:04:36.836282+0000","last_peered":"2023-10-08T16:04:36.836282+0000","last_clean":"2023-10-08T16:04:36.836282+0000","last_became_active":"2023-10-08T16:04:35.834550+0000","last_became_peered":"2023-10-08T16:04:35.834550+0000","last_unstale":"2023-10-08T16:04:36.836282+0000","last_undegraded":"2023-10-08T16:04:36.836282+0000","last_fullsized":"2023-10-08T16:04:36.836282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855292+0000","last_change":"2023-10-08T16:04:36.848762+0000","last_active":"2023-10-08T16:04:36.855292+0000","last_peered":"2023-10-08T16:04:36.855292+0000","last_clean":"2023-10-08T16:04:36.855292+0000","last_became_active":"2023-10-08T16:04:36.848407+0000","last_became_peered":"2023-10-08T16:04:36.848407+0000","last_unstale":"2023-10-08T16:04:36.855292+0000","last_undegraded":"2023-10-08T16:04:36.855292+0000","last_fullsized":"2023-10-08T16:04:36.855292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.856015+0000","last_change":"2023-10-08T16:04:35.834021+0000","last_active":"2023-10-08T16:04:36.856015+0000","last_peered":"2023-10-08T16:04:36.856015+0000","last_clean":"2023-10-08T16:04:36.856015+0000","last_became_active":"2023-10-08T16:04:35.833739+0000","last_became_peered":"2023-10-08T16:04:35.833739+0000","last_unstale":"2023-10-08T16:04:36.856015+0000","last_undegraded":"2023-10-08T16:04:36.856015+0000","last_fullsized":"2023-10-08T16:04:36.856015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:24.598148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859686+0000","last_change":"2023-10-08T16:04:36.850254+0000","last_active":"2023-10-08T16:04:36.859686+0000","last_peered":"2023-10-08T16:04:36.859686+0000","last_clean":"2023-10-08T16:04:36.859686+0000","last_became_active":"2023-10-08T16:04:36.849843+0000","last_became_peered":"2023-10-08T16:04:36.849843+0000","last_unstale":"2023-10-08T16:04:36.859686+0000","last_undegraded":"2023-10-08T16:04:36.859686+0000","last_fullsized":"2023-10-08T16:04:36.859686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859275+0000","last_change":"2023-10-08T16:04:35.837065+0000","last_active":"2023-10-08T16:04:36.859275+0000","last_peered":"2023-10-08T16:04:36.859275+0000","last_clean":"2023-10-08T16:04:36.859275+0000","last_became_active":"2023-10-08T16:04:35.836764+0000","last_became_peered":"2023-10-08T16:04:35.836764+0000","last_unstale":"2023-10-08T16:04:36.859275+0000","last_undegraded":"2023-10-08T16:04:36.859275+0000","last_fullsized":"2023-10-08T16:04:36.859275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:43.449593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848948+0000","last_change":"2023-10-08T16:04:36.845465+0000","last_active":"2023-10-08T16:04:36.848948+0000","last_peered":"2023-10-08T16:04:36.848948+0000","last_clean":"2023-10-08T16:04:36.848948+0000","last_became_active":"2023-10-08T16:04:36.845207+0000","last_became_peered":"2023-10-08T16:04:36.845207+0000","last_unstale":"2023-10-08T16:04:36.848948+0000","last_undegraded":"2023-10-08T16:04:36.848948+0000","last_fullsized":"2023-10-08T16:04:36.848948+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:06:59.916307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831893+0000","last_change":"2023-10-08T16:04:35.836100+0000","last_active":"2023-10-08T16:04:36.831893+0000","last_peered":"2023-10-08T16:04:36.831893+0000","last_clean":"2023-10-08T16:04:36.831893+0000","last_became_active":"2023-10-08T16:04:35.835832+0000","last_became_peered":"2023-10-08T16:04:35.835832+0000","last_unstale":"2023-10-08T16:04:36.831893+0000","last_undegraded":"2023-10-08T16:04:36.831893+0000","last_fullsized":"2023-10-08T16:04:36.831893+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:47:33.973873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860666+0000","last_change":"2023-10-08T16:04:36.853664+0000","last_active":"2023-10-08T16:04:36.860666+0000","last_peered":"2023-10-08T16:04:36.860666+0000","last_clean":"2023-10-08T16:04:36.860666+0000","last_became_active":"2023-10-08T16:04:36.853412+0000","last_became_peered":"2023-10-08T16:04:36.853412+0000","last_unstale":"2023-10-08T16:04:36.860666+0000","last_undegraded":"2023-10-08T16:04:36.860666+0000","last_fullsized":"2023-10-08T16:04:36.860666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.838343+0000","last_change":"2023-10-08T16:04:35.838075+0000","last_active":"2023-10-08T16:04:36.838343+0000","last_peered":"2023-10-08T16:04:36.838343+0000","last_clean":"2023-10-08T16:04:36.838343+0000","last_became_active":"2023-10-08T16:04:35.837821+0000","last_became_peered":"2023-10-08T16:04:35.837821+0000","last_unstale":"2023-10-08T16:04:36.838343+0000","last_undegraded":"2023-10-08T16:04:36.838343+0000","last_fullsized":"2023-10-08T16:04:36.838343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.850970+0000","last_change":"2023-10-08T16:04:36.845413+0000","last_active":"2023-10-08T16:04:36.850970+0000","last_peered":"2023-10-08T16:04:36.850970+0000","last_clean":"2023-10-08T16:04:36.850970+0000","last_became_active":"2023-10-08T16:04:36.845152+0000","last_became_peered":"2023-10-08T16:04:36.845152+0000","last_unstale":"2023-10-08T16:04:36.850970+0000","last_undegraded":"2023-10-08T16:04:36.850970+0000","last_fullsized":"2023-10-08T16:04:36.850970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839476+0000","last_change":"2023-10-08T16:04:36.837273+0000","last_active":"2023-10-08T16:04:36.839476+0000","last_peered":"2023-10-08T16:04:36.839476+0000","last_clean":"2023-10-08T16:04:36.839476+0000","last_became_active":"2023-10-08T16:04:36.837025+0000","last_became_peered":"2023-10-08T16:04:36.837025+0000","last_unstale":"2023-10-08T16:04:36.839476+0000","last_undegraded":"2023-10-08T16:04:36.839476+0000","last_fullsized":"2023-10-08T16:04:36.839476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833690+0000","last_change":"2023-10-08T16:04:35.831326+0000","last_active":"2023-10-08T16:04:36.833690+0000","last_peered":"2023-10-08T16:04:36.833690+0000","last_clean":"2023-10-08T16:04:36.833690+0000","last_became_active":"2023-10-08T16:04:35.831088+0000","last_became_peered":"2023-10-08T16:04:35.831088+0000","last_unstale":"2023-10-08T16:04:36.833690+0000","last_undegraded":"2023-10-08T16:04:36.833690+0000","last_fullsized":"2023-10-08T16:04:36.833690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:38.836853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855327+0000","last_change":"2023-10-08T16:04:36.847842+0000","last_active":"2023-10-08T16:04:36.855327+0000","last_peered":"2023-10-08T16:04:36.855327+0000","last_clean":"2023-10-08T16:04:36.855327+0000","last_became_active":"2023-10-08T16:04:36.847222+0000","last_became_peered":"2023-10-08T16:04:36.847222+0000","last_unstale":"2023-10-08T16:04:36.855327+0000","last_undegraded":"2023-10-08T16:04:36.855327+0000","last_fullsized":"2023-10-08T16:04:36.855327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835188+0000","last_change":"2023-10-08T16:04:35.837295+0000","last_active":"2023-10-08T16:04:36.835188+0000","last_peered":"2023-10-08T16:04:36.835188+0000","last_clean":"2023-10-08T16:04:36.835188+0000","last_became_active":"2023-10-08T16:04:35.836609+0000","last_became_peered":"2023-10-08T16:04:35.836609+0000","last_unstale":"2023-10-08T16:04:36.835188+0000","last_undegraded":"2023-10-08T16:04:36.835188+0000","last_fullsized":"2023-10-08T16:04:36.835188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835250+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:04:36.835250+0000","last_peered":"2023-10-08T16:04:36.835250+0000","last_clean":"2023-10-08T16:04:36.835250+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:04:36.835250+0000","last_undegraded":"2023-10-08T16:04:36.835250+0000","last_fullsized":"2023-10-08T16:04:36.835250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855399+0000","last_change":"2023-10-08T16:04:36.849109+0000","last_active":"2023-10-08T16:04:36.855399+0000","last_peered":"2023-10-08T16:04:36.855399+0000","last_clean":"2023-10-08T16:04:36.855399+0000","last_became_active":"2023-10-08T16:04:36.848847+0000","last_became_peered":"2023-10-08T16:04:36.848847+0000","last_unstale":"2023-10-08T16:04:36.855399+0000","last_undegraded":"2023-10-08T16:04:36.855399+0000","last_fullsized":"2023-10-08T16:04:36.855399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:34:26.769453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.835538+0000","last_change":"2023-10-08T16:04:35.840218+0000","last_active":"2023-10-08T16:04:36.835538+0000","last_peered":"2023-10-08T16:04:36.835538+0000","last_clean":"2023-10-08T16:04:36.835538+0000","last_became_active":"2023-10-08T16:04:35.839272+0000","last_became_peered":"2023-10-08T16:04:35.839272+0000","last_unstale":"2023-10-08T16:04:36.835538+0000","last_undegraded":"2023-10-08T16:04:36.835538+0000","last_fullsized":"2023-10-08T16:04:36.835538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:25.481170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.836042+0000","last_change":"2023-10-08T16:04:35.842423+0000","last_active":"2023-10-08T16:04:36.836042+0000","last_peered":"2023-10-08T16:04:36.836042+0000","last_clean":"2023-10-08T16:04:36.836042+0000","last_became_active":"2023-10-08T16:04:35.842183+0000","last_became_peered":"2023-10-08T16:04:35.842183+0000","last_unstale":"2023-10-08T16:04:36.836042+0000","last_undegraded":"2023-10-08T16:04:36.836042+0000","last_fullsized":"2023-10-08T16:04:36.836042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.839745+0000","last_change":"2023-10-08T16:04:36.837348+0000","last_active":"2023-10-08T16:04:36.839745+0000","last_peered":"2023-10-08T16:04:36.839745+0000","last_clean":"2023-10-08T16:04:36.839745+0000","last_became_active":"2023-10-08T16:04:36.837116+0000","last_became_peered":"2023-10-08T16:04:36.837116+0000","last_unstale":"2023-10-08T16:04:36.839745+0000","last_undegraded":"2023-10-08T16:04:36.839745+0000","last_fullsized":"2023-10-08T16:04:36.839745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-08T16:04:37.832298+0000","last_change":"2023-10-08T16:04:35.834800+0000","last_active":"2023-10-08T16:04:37.832298+0000","last_peered":"2023-10-08T16:04:37.832298+0000","last_clean":"2023-10-08T16:04:37.832298+0000","last_became_active":"2023-10-08T16:04:35.834516+0000","last_became_peered":"2023-10-08T16:04:35.834516+0000","last_unstale":"2023-10-08T16:04:37.832298+0000","last_undegraded":"2023-10-08T16:04:37.832298+0000","last_fullsized":"2023-10-08T16:04:37.832298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855314+0000","last_change":"2023-10-08T16:04:36.850191+0000","last_active":"2023-10-08T16:04:36.855314+0000","last_peered":"2023-10-08T16:04:36.855314+0000","last_clean":"2023-10-08T16:04:36.855314+0000","last_became_active":"2023-10-08T16:04:36.849886+0000","last_became_peered":"2023-10-08T16:04:36.849886+0000","last_unstale":"2023-10-08T16:04:36.855314+0000","last_undegraded":"2023-10-08T16:04:36.855314+0000","last_fullsized":"2023-10-08T16:04:36.855314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861589+0000","last_change":"2023-10-08T16:04:35.842733+0000","last_active":"2023-10-08T16:04:36.861589+0000","last_peered":"2023-10-08T16:04:36.861589+0000","last_clean":"2023-10-08T16:04:36.861589+0000","last_became_active":"2023-10-08T16:04:35.842431+0000","last_became_peered":"2023-10-08T16:04:35.842431+0000","last_unstale":"2023-10-08T16:04:36.861589+0000","last_undegraded":"2023-10-08T16:04:36.861589+0000","last_fullsized":"2023-10-08T16:04:36.861589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843518+0000","last_change":"2023-10-08T16:04:36.841209+0000","last_active":"2023-10-08T16:04:36.843518+0000","last_peered":"2023-10-08T16:04:36.843518+0000","last_clean":"2023-10-08T16:04:36.843518+0000","last_became_active":"2023-10-08T16:04:36.840900+0000","last_became_peered":"2023-10-08T16:04:36.840900+0000","last_unstale":"2023-10-08T16:04:36.843518+0000","last_undegraded":"2023-10-08T16:04:36.843518+0000","last_fullsized":"2023-10-08T16:04:36.843518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.834548+0000","last_change":"2023-10-08T16:04:35.840034+0000","last_active":"2023-10-08T16:04:36.834548+0000","last_peered":"2023-10-08T16:04:36.834548+0000","last_clean":"2023-10-08T16:04:36.834548+0000","last_became_active":"2023-10-08T16:04:35.839696+0000","last_became_peered":"2023-10-08T16:04:35.839696+0000","last_unstale":"2023-10-08T16:04:36.834548+0000","last_undegraded":"2023-10-08T16:04:36.834548+0000","last_fullsized":"2023-10-08T16:04:36.834548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:51.871435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853454+0000","last_change":"2023-10-08T16:04:36.847014+0000","last_active":"2023-10-08T16:04:36.853454+0000","last_peered":"2023-10-08T16:04:36.853454+0000","last_clean":"2023-10-08T16:04:36.853454+0000","last_became_active":"2023-10-08T16:04:36.846715+0000","last_became_peered":"2023-10-08T16:04:36.846715+0000","last_unstale":"2023-10-08T16:04:36.853454+0000","last_undegraded":"2023-10-08T16:04:36.853454+0000","last_fullsized":"2023-10-08T16:04:36.853454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:13.602001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831854+0000","last_change":"2023-10-08T16:04:35.833211+0000","last_active":"2023-10-08T16:04:36.831854+0000","last_peered":"2023-10-08T16:04:36.831854+0000","last_clean":"2023-10-08T16:04:36.831854+0000","last_became_active":"2023-10-08T16:04:35.832926+0000","last_became_peered":"2023-10-08T16:04:35.832926+0000","last_unstale":"2023-10-08T16:04:36.831854+0000","last_undegraded":"2023-10-08T16:04:36.831854+0000","last_fullsized":"2023-10-08T16:04:36.831854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840582+0000","last_change":"2023-10-08T16:04:36.839516+0000","last_active":"2023-10-08T16:04:36.840582+0000","last_peered":"2023-10-08T16:04:36.840582+0000","last_clean":"2023-10-08T16:04:36.840582+0000","last_became_active":"2023-10-08T16:04:36.839275+0000","last_became_peered":"2023-10-08T16:04:36.839275+0000","last_unstale":"2023-10-08T16:04:36.840582+0000","last_undegraded":"2023-10-08T16:04:36.840582+0000","last_fullsized":"2023-10-08T16:04:36.840582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:17:45.361058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.831607+0000","last_change":"2023-10-08T16:04:35.826206+0000","last_active":"2023-10-08T16:04:36.831607+0000","last_peered":"2023-10-08T16:04:36.831607+0000","last_clean":"2023-10-08T16:04:36.831607+0000","last_became_active":"2023-10-08T16:04:35.825925+0000","last_became_peered":"2023-10-08T16:04:35.825925+0000","last_unstale":"2023-10-08T16:04:36.831607+0000","last_undegraded":"2023-10-08T16:04:36.831607+0000","last_fullsized":"2023-10-08T16:04:36.831607+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855072+0000","last_change":"2023-10-08T16:04:36.845340+0000","last_active":"2023-10-08T16:04:36.855072+0000","last_peered":"2023-10-08T16:04:36.855072+0000","last_clean":"2023-10-08T16:04:36.855072+0000","last_became_active":"2023-10-08T16:04:36.845040+0000","last_became_peered":"2023-10-08T16:04:36.845040+0000","last_unstale":"2023-10-08T16:04:36.855072+0000","last_undegraded":"2023-10-08T16:04:36.855072+0000","last_fullsized":"2023-10-08T16:04:36.855072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860563+0000","last_change":"2023-10-08T16:04:35.841423+0000","last_active":"2023-10-08T16:04:36.860563+0000","last_peered":"2023-10-08T16:04:36.860563+0000","last_clean":"2023-10-08T16:04:36.860563+0000","last_became_active":"2023-10-08T16:04:35.841174+0000","last_became_peered":"2023-10-08T16:04:35.841174+0000","last_unstale":"2023-10-08T16:04:36.860563+0000","last_undegraded":"2023-10-08T16:04:36.860563+0000","last_fullsized":"2023-10-08T16:04:36.860563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:36.974490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860494+0000","last_change":"2023-10-08T16:04:36.851799+0000","last_active":"2023-10-08T16:04:36.860494+0000","last_peered":"2023-10-08T16:04:36.860494+0000","last_clean":"2023-10-08T16:04:36.860494+0000","last_became_active":"2023-10-08T16:04:36.851458+0000","last_became_peered":"2023-10-08T16:04:36.851458+0000","last_unstale":"2023-10-08T16:04:36.860494+0000","last_undegraded":"2023-10-08T16:04:36.860494+0000","last_fullsized":"2023-10-08T16:04:36.860494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:58.179554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.833797+0000","last_change":"2023-10-08T16:04:35.840122+0000","last_active":"2023-10-08T16:04:36.833797+0000","last_peered":"2023-10-08T16:04:36.833797+0000","last_clean":"2023-10-08T16:04:36.833797+0000","last_became_active":"2023-10-08T16:04:35.839817+0000","last_became_peered":"2023-10-08T16:04:35.839817+0000","last_unstale":"2023-10-08T16:04:36.833797+0000","last_undegraded":"2023-10-08T16:04:36.833797+0000","last_fullsized":"2023-10-08T16:04:36.833797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:07:45.955717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855128+0000","last_change":"2023-10-08T16:04:36.847797+0000","last_active":"2023-10-08T16:04:36.855128+0000","last_peered":"2023-10-08T16:04:36.855128+0000","last_clean":"2023-10-08T16:04:36.855128+0000","last_became_active":"2023-10-08T16:04:36.847542+0000","last_became_peered":"2023-10-08T16:04:36.847542+0000","last_unstale":"2023-10-08T16:04:36.855128+0000","last_undegraded":"2023-10-08T16:04:36.855128+0000","last_fullsized":"2023-10-08T16:04:36.855128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:56:25.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.859329+0000","last_change":"2023-10-08T16:04:35.826935+0000","last_active":"2023-10-08T16:04:36.859329+0000","last_peered":"2023-10-08T16:04:36.859329+0000","last_clean":"2023-10-08T16:04:36.859329+0000","last_became_active":"2023-10-08T16:04:35.826668+0000","last_became_peered":"2023-10-08T16:04:35.826668+0000","last_unstale":"2023-10-08T16:04:36.859329+0000","last_undegraded":"2023-10-08T16:04:36.859329+0000","last_fullsized":"2023-10-08T16:04:36.859329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:35:12.595985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861167+0000","last_change":"2023-10-08T16:04:36.857918+0000","last_active":"2023-10-08T16:04:36.861167+0000","last_peered":"2023-10-08T16:04:36.861167+0000","last_clean":"2023-10-08T16:04:36.861167+0000","last_became_active":"2023-10-08T16:04:36.857368+0000","last_became_peered":"2023-10-08T16:04:36.857368+0000","last_unstale":"2023-10-08T16:04:36.861167+0000","last_undegraded":"2023-10-08T16:04:36.861167+0000","last_fullsized":"2023-10-08T16:04:36.861167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:14:16.925039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855827+0000","last_change":"2023-10-08T16:04:35.836755+0000","last_active":"2023-10-08T16:04:36.855827+0000","last_peered":"2023-10-08T16:04:36.855827+0000","last_clean":"2023-10-08T16:04:36.855827+0000","last_became_active":"2023-10-08T16:04:35.836467+0000","last_became_peered":"2023-10-08T16:04:35.836467+0000","last_unstale":"2023-10-08T16:04:36.855827+0000","last_undegraded":"2023-10-08T16:04:36.855827+0000","last_fullsized":"2023-10-08T16:04:36.855827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:34.811998+0000","last_clean_scrub_stamp":"2023-10-08T16:04:34.811998+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:13:23.595157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860389+0000","last_change":"2023-10-08T16:04:36.851314+0000","last_active":"2023-10-08T16:04:36.860389+0000","last_peered":"2023-10-08T16:04:36.860389+0000","last_clean":"2023-10-08T16:04:36.860389+0000","last_became_active":"2023-10-08T16:04:36.850726+0000","last_became_peered":"2023-10-08T16:04:36.850726+0000","last_unstale":"2023-10-08T16:04:36.860389+0000","last_undegraded":"2023-10-08T16:04:36.860389+0000","last_fullsized":"2023-10-08T16:04:36.860389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:34.950879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.853164+0000","last_change":"2023-10-08T16:04:36.843832+0000","last_active":"2023-10-08T16:04:36.853164+0000","last_peered":"2023-10-08T16:04:36.853164+0000","last_clean":"2023-10-08T16:04:36.853164+0000","last_became_active":"2023-10-08T16:04:36.843552+0000","last_became_peered":"2023-10-08T16:04:36.843552+0000","last_unstale":"2023-10-08T16:04:36.853164+0000","last_undegraded":"2023-10-08T16:04:36.853164+0000","last_fullsized":"2023-10-08T16:04:36.853164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848791+0000","last_change":"2023-10-08T16:04:36.846568+0000","last_active":"2023-10-08T16:04:36.848791+0000","last_peered":"2023-10-08T16:04:36.848791+0000","last_clean":"2023-10-08T16:04:36.848791+0000","last_became_active":"2023-10-08T16:04:36.846344+0000","last_became_peered":"2023-10-08T16:04:36.846344+0000","last_unstale":"2023-10-08T16:04:36.848791+0000","last_undegraded":"2023-10-08T16:04:36.848791+0000","last_fullsized":"2023-10-08T16:04:36.848791+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:31:52.877451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861183+0000","last_change":"2023-10-08T16:04:36.857746+0000","last_active":"2023-10-08T16:04:36.861183+0000","last_peered":"2023-10-08T16:04:36.861183+0000","last_clean":"2023-10-08T16:04:36.861183+0000","last_became_active":"2023-10-08T16:04:36.856907+0000","last_became_peered":"2023-10-08T16:04:36.856907+0000","last_unstale":"2023-10-08T16:04:36.861183+0000","last_undegraded":"2023-10-08T16:04:36.861183+0000","last_fullsized":"2023-10-08T16:04:36.861183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:01.148060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.846189+0000","last_change":"2023-10-08T16:04:36.843885+0000","last_active":"2023-10-08T16:04:36.846189+0000","last_peered":"2023-10-08T16:04:36.846189+0000","last_clean":"2023-10-08T16:04:36.846189+0000","last_became_active":"2023-10-08T16:04:36.843640+0000","last_became_peered":"2023-10-08T16:04:36.843640+0000","last_unstale":"2023-10-08T16:04:36.846189+0000","last_undegraded":"2023-10-08T16:04:36.846189+0000","last_fullsized":"2023-10-08T16:04:36.846189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855356+0000","last_change":"2023-10-08T16:04:36.851431+0000","last_active":"2023-10-08T16:04:36.855356+0000","last_peered":"2023-10-08T16:04:36.855356+0000","last_clean":"2023-10-08T16:04:36.855356+0000","last_became_active":"2023-10-08T16:04:36.851215+0000","last_became_peered":"2023-10-08T16:04:36.851215+0000","last_unstale":"2023-10-08T16:04:36.855356+0000","last_undegraded":"2023-10-08T16:04:36.855356+0000","last_fullsized":"2023-10-08T16:04:36.855356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:31.532873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.840134+0000","last_change":"2023-10-08T16:04:36.837981+0000","last_active":"2023-10-08T16:04:36.840134+0000","last_peered":"2023-10-08T16:04:36.840134+0000","last_clean":"2023-10-08T16:04:36.840134+0000","last_became_active":"2023-10-08T16:04:36.837478+0000","last_became_peered":"2023-10-08T16:04:36.837478+0000","last_unstale":"2023-10-08T16:04:36.840134+0000","last_undegraded":"2023-10-08T16:04:36.840134+0000","last_fullsized":"2023-10-08T16:04:36.840134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855950+0000","last_change":"2023-10-08T16:04:36.850056+0000","last_active":"2023-10-08T16:04:36.855950+0000","last_peered":"2023-10-08T16:04:36.855950+0000","last_clean":"2023-10-08T16:04:36.855950+0000","last_became_active":"2023-10-08T16:04:36.849732+0000","last_became_peered":"2023-10-08T16:04:36.849732+0000","last_unstale":"2023-10-08T16:04:36.855950+0000","last_undegraded":"2023-10-08T16:04:36.855950+0000","last_fullsized":"2023-10-08T16:04:36.855950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:56.072737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.843571+0000","last_change":"2023-10-08T16:04:36.842352+0000","last_active":"2023-10-08T16:04:36.843571+0000","last_peered":"2023-10-08T16:04:36.843571+0000","last_clean":"2023-10-08T16:04:36.843571+0000","last_became_active":"2023-10-08T16:04:36.842078+0000","last_became_peered":"2023-10-08T16:04:36.842078+0000","last_unstale":"2023-10-08T16:04:36.843571+0000","last_undegraded":"2023-10-08T16:04:36.843571+0000","last_fullsized":"2023-10-08T16:04:36.843571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.861173+0000","last_change":"2023-10-08T16:04:36.858170+0000","last_active":"2023-10-08T16:04:36.861173+0000","last_peered":"2023-10-08T16:04:36.861173+0000","last_clean":"2023-10-08T16:04:36.861173+0000","last_became_active":"2023-10-08T16:04:36.857822+0000","last_became_peered":"2023-10-08T16:04:36.857822+0000","last_unstale":"2023-10-08T16:04:36.861173+0000","last_undegraded":"2023-10-08T16:04:36.861173+0000","last_fullsized":"2023-10-08T16:04:36.861173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:25.562634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855132+0000","last_change":"2023-10-08T16:04:36.847659+0000","last_active":"2023-10-08T16:04:36.855132+0000","last_peered":"2023-10-08T16:04:36.855132+0000","last_clean":"2023-10-08T16:04:36.855132+0000","last_became_active":"2023-10-08T16:04:36.847337+0000","last_became_peered":"2023-10-08T16:04:36.847337+0000","last_unstale":"2023-10-08T16:04:36.855132+0000","last_undegraded":"2023-10-08T16:04:36.855132+0000","last_fullsized":"2023-10-08T16:04:36.855132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:19.024469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.855842+0000","last_change":"2023-10-08T16:04:36.850206+0000","last_active":"2023-10-08T16:04:36.855842+0000","last_peered":"2023-10-08T16:04:36.855842+0000","last_clean":"2023-10-08T16:04:36.855842+0000","last_became_active":"2023-10-08T16:04:36.849892+0000","last_became_peered":"2023-10-08T16:04:36.849892+0000","last_unstale":"2023-10-08T16:04:36.855842+0000","last_undegraded":"2023-10-08T16:04:36.855842+0000","last_fullsized":"2023-10-08T16:04:36.855842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:15.348765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.848345+0000","last_change":"2023-10-08T16:04:36.846291+0000","last_active":"2023-10-08T16:04:36.848345+0000","last_peered":"2023-10-08T16:04:36.848345+0000","last_clean":"2023-10-08T16:04:36.848345+0000","last_became_active":"2023-10-08T16:04:36.846030+0000","last_became_peered":"2023-10-08T16:04:36.846030+0000","last_unstale":"2023-10-08T16:04:36.848345+0000","last_undegraded":"2023-10-08T16:04:36.848345+0000","last_fullsized":"2023-10-08T16:04:36.848345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:57:10.900218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T16:04:36.860629+0000","last_change":"2023-10-08T16:04:36.852345+0000","last_active":"2023-10-08T16:04:36.860629+0000","last_peered":"2023-10-08T16:04:36.860629+0000","last_clean":"2023-10-08T16:04:36.860629+0000","last_became_active":"2023-10-08T16:04:36.852087+0000","last_became_peered":"2023-10-08T16:04:36.852087+0000","last_unstale":"2023-10-08T16:04:36.860629+0000","last_undegraded":"2023-10-08T16:04:36.860629+0000","last_fullsized":"2023-10-08T16:04:36.860629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:35.818014+0000","last_clean_scrub_stamp":"2023-10-08T16:04:35.818014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:33.244464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298084,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073756,"statfs":{"total":96636764160,"available":96331526144,"internally_reserved":0,"allocated":815104,"data_stored":666235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297508,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074332,"statfs":{"total":96636764160,"available":96332115968,"internally_reserved":0,"allocated":225280,"data_stored":75473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":666291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":75313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":76849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.238}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.27}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.825}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":75439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":75403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:04:45.804 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T16:04:45.804 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T16:04:45.804 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-08T16:04:45.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-08T16:04:45.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.950+0000 7f488c3d8700 1 Processor -- start 2023-10-08T16:04:45.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.950+0000 7f488c3d8700 1 -- start start 2023-10-08T16:04:45.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488c3d8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4884157720 0x7f48841589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488c3d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488c3d8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4884158560 con 0x7f48840acf10 2023-10-08T16:04:45.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488c3d8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f488406f140 con 0x7f4884157720 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488c3d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488406f320 con 0x7f488406e7b0 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f4889973700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.951+0000 7f488a174700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4884157720 0x7f48841589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889973700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:42920/0 (socket says 172.21.15.170:42920) 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889973700 1 -- 172.21.15.170:0/2269276536 learned_addr learned my addr 172.21.15.170:0/2269276536 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f488a174700 1 -- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 msgr2=0x7f488406ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f488a174700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f488a174700 1 -- 172.21.15.170:0/2269276536 >> v1:172.21.15.191:6789/0 conn(0x7f48840acf10 legacy=0x7f488415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f488a174700 1 -- 172.21.15.170:0/2269276536 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488406f600 con 0x7f4884157720 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889973700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f488a174700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48841589f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4874009e60 tx=0x7f4874009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b95184e248b4b44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889172700 1 -- 172.21.15.170:0/2269276536 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487400e070 con 0x7f4884157720 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889172700 1 -- 172.21.15.170:0/2269276536 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487400bb60 con 0x7f4884157720 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.952+0000 7f4889172700 1 -- 172.21.15.170:0/2269276536 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874012430 con 0x7f4884157720 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 -- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 msgr2=0x7f48841589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48841589f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4874009e60 tx=0x7f4874009780 comp rx=0 tx=0).stop 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 -- 172.21.15.170:0/2269276536 shutdown_connections 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48841589f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 --2- 172.21.15.170:0/2269276536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f488406ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 -- 172.21.15.170:0/2269276536 >> 172.21.15.170:0/2269276536 conn(0x7f48840cd640 msgr2=0x7f48840cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 -- 172.21.15.170:0/2269276536 shutdown_connections 2023-10-08T16:04:45.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.953+0000 7f488c3d8700 1 -- 172.21.15.170:0/2269276536 wait complete. 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.954+0000 7f488c3d8700 1 Processor -- start 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.954+0000 7f488c3d8700 1 -- start start 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f48840c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48840b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840c3220 con 0x7f48840acf10 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840c2610 con 0x7f4884157720 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840c28f0 con 0x7f488406e7b0 2023-10-08T16:04:45.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58196/0 (socket says 172.21.15.170:58196) 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 -- 172.21.15.170:0/3913252894 learned_addr learned my addr 172.21.15.170:0/3913252894 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488a174700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f48840c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488a975700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48840b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 -- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 msgr2=0x7f48840c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f48840c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 -- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 msgr2=0x7f48840b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48840b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 -- 172.21.15.170:0/3913252894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48780094f0 con 0x7f48840acf10 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f4889973700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f487800ec50 tx=0x7f4878009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800c850 con 0x7f48840acf10 2023-10-08T16:04:45.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487800c9b0 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4874009a20 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487800cd30 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.955+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48840c2eb0 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.956+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48840c3390 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.957+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4878013680 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.957+0000 7f48827fc700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 0x7f48700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.957+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4878084800 con 0x7f48840acf10 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.957+0000 7f488a174700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 0x7f48700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:45.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.958+0000 7f488a174700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 0x7f48700693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48740097c0 tx=0x7f4874019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:45.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:45.961+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4878051a70 con 0x7f48840acf10 2023-10-08T16:04:46.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.182+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f488404c3c0 con 0x7f48840acf10 2023-10-08T16:04:46.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.183+0000 7f48827fc700 1 -- 172.21.15.170:0/3913252894 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4878055090 con 0x7f48840acf10 2023-10-08T16:04:46.184 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:46.184 INFO:teuthology.orchestra.run.smithi170.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T16:04:46.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 msgr2=0x7f48700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 0x7f48700693e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48740097c0 tx=0x7f4874019000 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 msgr2=0x7f48840c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f487800ec50 tx=0x7f4878009250 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 shutdown_connections 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4884157720 0x7f48840b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f488406e7b0 0x7f48840c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4870066f30 0x7f48700693e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 --2- 172.21.15.170:0/3913252894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48840acf10 0x7f48840c36b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 >> 172.21.15.170:0/3913252894 conn(0x7f48840cd640 msgr2=0x7f48840cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 shutdown_connections 2023-10-08T16:04:46.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.185+0000 7f488c3d8700 1 -- 172.21.15.170:0/3913252894 wait complete. 2023-10-08T16:04:46.198 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-08T16:04:46.198 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.312+0000 7f49bffff700 1 Processor -- start 2023-10-08T16:04:46.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.312+0000 7f49bffff700 1 -- start start 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f49c0159420 0x7f49c00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 0x7f49c00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c015d230 0x7f49c015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c015dbb0 con 0x7f49c0159420 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00b11d0 con 0x7f49c01597f0 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b14b0 con 0x7f49c015d230 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f49c0159420 0x7f49c00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f49c0159420 0x7f49c00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58200/0 (socket says 172.21.15.170:58200) 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 -- 172.21.15.170:0/3812497450 learned_addr learned my addr 172.21.15.170:0/3812497450 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bf7fe700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c015d230 0x7f49c015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49be7fc700 1 --2- 172.21.15.170:0/3812497450 >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 0x7f49c00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 -- 172.21.15.170:0/3812497450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c015d230 msgr2=0x7f49c015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c015d230 0x7f49c015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 -- 172.21.15.170:0/3812497450 >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 msgr2=0x7f49c00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 --2- 172.21.15.170:0/3812497450 >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 0x7f49c00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 -- 172.21.15.170:0/3812497450 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b1790 con 0x7f49c0159420 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49be7fc700 1 --2- 172.21.15.170:0/3812497450 >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 0x7f49c00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49beffd700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c0159420 0x7f49c00b5f40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49c0089750 tx=0x7f49b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1531821ff5e654a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bdffb700 1 -- 172.21.15.170:0/3812497450 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000e070 con 0x7f49c0159420 2023-10-08T16:04:46.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bdffb700 1 -- 172.21.15.170:0/3812497450 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000ba50 con 0x7f49c0159420 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.313+0000 7f49bdffb700 1 -- 172.21.15.170:0/3812497450 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00092e0 con 0x7f49c0159420 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 -- 172.21.15.170:0/3812497450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c0159420 msgr2=0x7f49c00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c0159420 0x7f49c00b5f40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49c0089750 tx=0x7f49b0009840 comp rx=0 tx=0).stop 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 -- 172.21.15.170:0/3812497450 shutdown_connections 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c015d230 0x7f49c015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 --2- 172.21.15.170:0/3812497450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c0159420 0x7f49c00b5f40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 --2- 172.21.15.170:0/3812497450 >> v2:172.21.15.191:3300/0 conn(0x7f49c01597f0 0x7f49c00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 -- 172.21.15.170:0/3812497450 >> 172.21.15.170:0/3812497450 conn(0x7f49c00d0660 msgr2=0x7f49c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 -- 172.21.15.170:0/3812497450 shutdown_connections 2023-10-08T16:04:46.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.314+0000 7f49bffff700 1 -- 172.21.15.170:0/3812497450 wait complete. 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.315+0000 7f49bffff700 1 Processor -- start 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.315+0000 7f49bffff700 1 -- start start 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c0159420 0x7f49c00c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c015d230 0x7f49c00c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00ca480 con 0x7f49c01597f0 2023-10-08T16:04:46.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c9ec0 con 0x7f49c015d230 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00ca1a0 con 0x7f49c0159420 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58210/0 (socket says 172.21.15.170:58210) 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 -- 172.21.15.170:0/112896005 learned_addr learned my addr 172.21.15.170:0/112896005 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49beffd700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c0159420 0x7f49c00c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49bf7fe700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c015d230 0x7f49c00c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 -- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c0159420 msgr2=0x7f49c00c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c0159420 0x7f49c00c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 -- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c015d230 msgr2=0x7f49c00c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c015d230 0x7f49c00c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 -- 172.21.15.170:0/112896005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0009ae0 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.316+0000 7f49be7fc700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c880 tx=0x7f49b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.317+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011600 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.317+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4011760 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.317+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00c5530 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.317+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011ae0 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.317+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00c7500 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.318+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00c5a20 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.318+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f49b40073d0 con 0x7f49c01597f0 2023-10-08T16:04:46.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.318+0000 7f49a6ffd700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 0x7f49ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.318+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f49b40845c0 con 0x7f49c01597f0 2023-10-08T16:04:46.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.318+0000 7f49beffd700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 0x7f49ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.319+0000 7f49beffd700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 0x7f49ac0693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009880 tx=0x7f49b0018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.322+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49b4051830 con 0x7f49c01597f0 2023-10-08T16:04:46.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.507+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49c004c3c0 con 0x7f49c01597f0 2023-10-08T16:04:46.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.508+0000 7f49a6ffd700 1 -- 172.21.15.170:0/112896005 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f49b4054e50 con 0x7f49c01597f0 2023-10-08T16:04:46.510 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:46.510 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 msgr2=0x7f49ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 0x7f49ac0693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009880 tx=0x7f49b0018000 comp rx=0 tx=0).stop 2023-10-08T16:04:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 msgr2=0x7f49c01456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c880 tx=0x7f49b4008040 comp rx=0 tx=0).stop 2023-10-08T16:04:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 shutdown_connections 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49c015d230 0x7f49c00c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49c0159420 0x7f49c00c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49ac066f30 0x7f49ac0693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 --2- 172.21.15.170:0/112896005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49c01597f0 0x7f49c01456c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 >> 172.21.15.170:0/112896005 conn(0x7f49c00d0660 msgr2=0x7f49c00af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 shutdown_connections 2023-10-08T16:04:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.510+0000 7f49bffff700 1 -- 172.21.15.170:0/112896005 wait complete. 2023-10-08T16:04:46.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:04:46.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:04:46.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.639+0000 7f9f28366700 1 Processor -- start 2023-10-08T16:04:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.639+0000 7f9f28366700 1 -- start start 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f28366700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f20159c60 0x7f9f2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f28366700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9f2015db10 0x7f9f20069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f28366700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2015a160 con 0x7f9f20159890 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f28366700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2015df50 con 0x7f9f20159c60 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f28366700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f200d13a0 con 0x7f9f2015db10 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f25901700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f20159c60 0x7f9f2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f25901700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f20159c60 0x7f9f2015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58214/0 (socket says 172.21.15.170:58214) 2023-10-08T16:04:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f25901700 1 -- 172.21.15.170:0/3408037403 learned_addr learned my addr 172.21.15.170:0/3408037403 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f25901700 1 -- 172.21.15.170:0/3408037403 >> v2:172.21.15.191:3300/0 conn(0x7f9f2015db10 msgr2=0x7f9f20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.640+0000 7f9f26903700 1 --2- 172.21.15.170:0/3408037403 >> v2:172.21.15.191:3300/0 conn(0x7f9f2015db10 0x7f9f20069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25901700 1 --2- 172.21.15.170:0/3408037403 >> v2:172.21.15.191:3300/0 conn(0x7f9f2015db10 0x7f9f20069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25901700 1 -- 172.21.15.170:0/3408037403 >> v1:172.21.15.191:6789/0 conn(0x7f9f20159890 legacy=0x7f9f2015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25901700 1 -- 172.21.15.170:0/3408037403 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2015e140 con 0x7f9f20159c60 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25901700 1 --2- 172.21.15.170:0/3408037403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f20159c60 0x7f9f2015d5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006a890 tx=0x7f9f14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb1c37b0c6e2818 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25100700 1 -- 172.21.15.170:0/3408037403 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400b820 con 0x7f9f20159c60 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25100700 1 -- 172.21.15.170:0/3408037403 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400b980 con 0x7f9f20159c60 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f25100700 1 -- 172.21.15.170:0/3408037403 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400bd20 con 0x7f9f20159c60 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 -- 172.21.15.170:0/3408037403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f20159c60 msgr2=0x7f9f2015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 --2- 172.21.15.170:0/3408037403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f20159c60 0x7f9f2015d5d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9f2006a890 tx=0x7f9f14009250 comp rx=0 tx=0).stop 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 -- 172.21.15.170:0/3408037403 shutdown_connections 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 --2- 172.21.15.170:0/3408037403 >> v2:172.21.15.191:3300/0 conn(0x7f9f2015db10 0x7f9f20069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 --2- 172.21.15.170:0/3408037403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f20159c60 0x7f9f2015d5d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 -- 172.21.15.170:0/3408037403 >> 172.21.15.170:0/3408037403 conn(0x7f9f20068e20 msgr2=0x7f9f200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 -- 172.21.15.170:0/3408037403 shutdown_connections 2023-10-08T16:04:46.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.641+0000 7f9f28366700 1 -- 172.21.15.170:0/3408037403 wait complete. 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.642+0000 7f9f28366700 1 Processor -- start 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.642+0000 7f9f28366700 1 -- start start 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 0x7f9f200cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f20159c60 0x7f9f200b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200b2810 con 0x7f9f2015db10 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200ce010 con 0x7f9f20159c60 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f28366700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200b1620 con 0x7f9f20159890 2023-10-08T16:04:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58220/0 (socket says 172.21.15.170:58220) 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 -- 172.21.15.170:0/3462730350 learned_addr learned my addr 172.21.15.170:0/3462730350 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26102700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 0x7f9f200cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f25901700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f20159c60 0x7f9f200b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 -- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 msgr2=0x7f9f200cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 0x7f9f200cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 -- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f20159c60 msgr2=0x7f9f200b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.643+0000 7f9f26903700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f20159c60 0x7f9f200b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f26903700 1 -- 172.21.15.170:0/3462730350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f140094f0 con 0x7f9f2015db10 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f26102700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 0x7f9f200cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f26903700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c880 tx=0x7f9f18008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18011600 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18011760 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f200b5dc0 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18011b00 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.644+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f200b0cf0 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.645+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2004c3c0 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.645+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f18018360 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.645+0000 7f9f1e7fc700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.646+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9f180841e0 con 0x7f9f2015db10 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.646+0000 7f9f26102700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.646+0000 7f9f26102700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f10001410 tx=0x7f9f100096a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.650+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f18054ee0 con 0x7f9f2015db10 2023-10-08T16:04:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.829+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f200b1760 con 0x7f9f2015db10 2023-10-08T16:04:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.830+0000 7f9f1e7fc700 1 -- 172.21.15.170:0/3462730350 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f18054a70 con 0x7f9f2015db10 2023-10-08T16:04:46.831 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:04:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 msgr2=0x7f9f0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 0x7f9f0c0693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f10001410 tx=0x7f9f100096a0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 msgr2=0x7f9f200b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800c880 tx=0x7f9f18008040 comp rx=0 tx=0).stop 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 shutdown_connections 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f0c066f30 0x7f9f0c0693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f2015db10 0x7f9f200b1ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f20159890 0x7f9f200cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 --2- 172.21.15.170:0/3462730350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f20159c60 0x7f9f200b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 >> 172.21.15.170:0/3462730350 conn(0x7f9f20068e20 msgr2=0x7f9f2014d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 shutdown_connections 2023-10-08T16:04:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.832+0000 7f9f28366700 1 -- 172.21.15.170:0/3462730350 wait complete. 2023-10-08T16:04:46.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:04:46.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.961+0000 7f3149198700 1 Processor -- start 2023-10-08T16:04:46.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.961+0000 7f3149198700 1 -- start start 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3149198700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 0x7f31440ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3149198700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f31440b1b40 0x7f314415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3149198700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440cfb00 con 0x7f31440b16d0 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3149198700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440b5c40 con 0x7f31440b1b40 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3149198700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b5f20 con 0x7f31440b4e70 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f31440b1b40 0x7f314415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f31437fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f31440b16d0 0x7f314415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51242/0 (socket says 172.21.15.170:51242) 2023-10-08T16:04:46.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f31437fe700 1 -- 172.21.15.170:0/3119632411 learned_addr learned my addr 172.21.15.170:0/3119632411 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3143fff700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 0x7f31440ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 -- 172.21.15.170:0/3119632411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 msgr2=0x7f31440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 0x7f31440ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3142ffd700 1 -- 172.21.15.170:0/3119632411 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593925245 0 0) 0x7f31440cfb00 con 0x7f31440b16d0 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 -- 172.21.15.170:0/3119632411 >> v1:172.21.15.170:6789/0 conn(0x7f31440b16d0 legacy=0x7f314415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 -- 172.21.15.170:0/3119632411 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440b6200 con 0x7f31440b1b40 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3143fff700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 0x7f31440ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.962+0000 7f3148997700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b1b40 0x7f314415d950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f313c009e60 tx=0x7f313c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66945d5092a4dc3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3142ffd700 1 -- 172.21.15.170:0/3119632411 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c00e070 con 0x7f31440b1b40 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3142ffd700 1 -- 172.21.15.170:0/3119632411 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00bc30 con 0x7f31440b1b40 2023-10-08T16:04:46.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3142ffd700 1 -- 172.21.15.170:0/3119632411 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c012430 con 0x7f31440b1b40 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 -- 172.21.15.170:0/3119632411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b1b40 msgr2=0x7f314415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b1b40 0x7f314415d950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f313c009e60 tx=0x7f313c009780 comp rx=0 tx=0).stop 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 -- 172.21.15.170:0/3119632411 shutdown_connections 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b4e70 0x7f31440ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 --2- 172.21.15.170:0/3119632411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b1b40 0x7f314415d950 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 -- 172.21.15.170:0/3119632411 >> 172.21.15.170:0/3119632411 conn(0x7f31440d10c0 msgr2=0x7f31440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 -- 172.21.15.170:0/3119632411 shutdown_connections 2023-10-08T16:04:46.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.963+0000 7f3149198700 1 -- 172.21.15.170:0/3119632411 wait complete. 2023-10-08T16:04:46.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.964+0000 7f3149198700 1 Processor -- start 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.964+0000 7f3149198700 1 -- start start 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31440b16d0 0x7f3144158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b1b40 0x7f314416a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440c8640 con 0x7f31440b4e70 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3144158a70 con 0x7f31440b16d0 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3149198700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440c7a30 con 0x7f31440b1b40 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3148997700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3148997700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58234/0 (socket says 172.21.15.170:58234) 2023-10-08T16:04:46.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3148997700 1 -- 172.21.15.170:0/3146597862 learned_addr learned my addr 172.21.15.170:0/3146597862 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3143fff700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31440b16d0 0x7f3144158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f31437fe700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b1b40 0x7f314416a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3148997700 1 -- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b1b40 msgr2=0x7f314416a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.965+0000 7f3148997700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b1b40 0x7f314416a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3148997700 1 -- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31440b16d0 msgr2=0x7f3144158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3148997700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31440b16d0 0x7f3144158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3148997700 1 -- 172.21.15.170:0/3146597862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c009a20 con 0x7f31440b4e70 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3148997700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f313c013000 tx=0x7f313c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c012e00 con 0x7f31440b4e70 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c01d630 con 0x7f31440b4e70 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440c7d10 con 0x7f31440b4e70 2023-10-08T16:04:46.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c01d9f0 con 0x7f31440b4e70 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.966+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31440c82d0 con 0x7f31440b4e70 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.967+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31440c87f0 con 0x7f31440b4e70 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.967+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f313c024070 con 0x7f31440b4e70 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.968+0000 7f312bfff700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 0x7f31300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.968+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f313c086d50 con 0x7f31440b4e70 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.968+0000 7f3143fff700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 0x7f31300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:46.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.968+0000 7f3143fff700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 0x7f31300693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f3134008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:46.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:46.973+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31440c87f0 con 0x7f31440b4e70 2023-10-08T16:04:47.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.154+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31440b1510 con 0x7f31440b4e70 2023-10-08T16:04:47.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.155+0000 7f312bfff700 1 -- 172.21.15.170:0/3146597862 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f313c08a030 con 0x7f31440b4e70 2023-10-08T16:04:47.157 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:04:47.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 msgr2=0x7f31300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 0x7f31300693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3134006fd0 tx=0x7f3134008040 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 msgr2=0x7f31440c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f313c013000 tx=0x7f313c019000 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 shutdown_connections 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3130066f30 0x7f31300693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31440b4e70 0x7f31440c8ad0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f31440b16d0 0x7f3144158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 --2- 172.21.15.170:0/3146597862 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f31440b1b40 0x7f314416a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 >> 172.21.15.170:0/3146597862 conn(0x7f31440d10c0 msgr2=0x7f31440b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 shutdown_connections 2023-10-08T16:04:47.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.157+0000 7f3149198700 1 -- 172.21.15.170:0/3146597862 wait complete. 2023-10-08T16:04:47.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:04:47.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.286+0000 7f5154f3f700 1 Processor -- start 2023-10-08T16:04:47.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.286+0000 7f5154f3f700 1 -- start start 2023-10-08T16:04:47.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f5154f3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f51500c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f5154f3f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515015d7e0 con 0x7f515006c0a0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f5154f3f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f515015dac0 con 0x7f515006bcd0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f5154f3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51500caf30 con 0x7f51500caa90 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f514dd9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f515006c0a0 0x7f515015df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51250/0 (socket says 172.21.15.170:51250) 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f514dd9b700 1 -- 172.21.15.170:0/2634734768 learned_addr learned my addr 172.21.15.170:0/2634734768 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.287+0000 7f514ed9d700 1 --2- 172.21.15.170:0/2634734768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f51500c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840720641 0 0) 0x7f515015d7e0 con 0x7f515006c0a0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f513c003570 con 0x7f515006c0a0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611725035 0 0) 0x7f515015dac0 con 0x7f515006bcd0 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514ed9d700 1 -- 172.21.15.170:0/2634734768 >> v1:172.21.15.191:6789/0 conn(0x7f515006bcd0 legacy=0x7f515015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514ed9d700 1 -- 172.21.15.170:0/2634734768 >> v1:172.21.15.170:6789/0 conn(0x7f515006c0a0 legacy=0x7f515015df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514ed9d700 1 -- 172.21.15.170:0/2634734768 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515015aa30 con 0x7f51500caa90 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514ed9d700 1 --2- 172.21.15.170:0/2634734768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f51500c8c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e90 tx=0x7f5148009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507fc79f28176a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800e070 con 0x7f51500caa90 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514800bc30 con 0x7f51500caa90 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.288+0000 7f514d59a700 1 -- 172.21.15.170:0/2634734768 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5148012430 con 0x7f51500caa90 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 -- 172.21.15.170:0/2634734768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 msgr2=0x7f51500c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 --2- 172.21.15.170:0/2634734768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f51500c8c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5148009e90 tx=0x7f5148009780 comp rx=0 tx=0).stop 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 -- 172.21.15.170:0/2634734768 shutdown_connections 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 --2- 172.21.15.170:0/2634734768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f51500c8c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 -- 172.21.15.170:0/2634734768 >> 172.21.15.170:0/2634734768 conn(0x7f51500ce600 msgr2=0x7f51500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 -- 172.21.15.170:0/2634734768 shutdown_connections 2023-10-08T16:04:47.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.289+0000 7f5154f3f700 1 -- 172.21.15.170:0/2634734768 wait complete. 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 Processor -- start 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 -- start start 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515006bcd0 0x7f5150155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f515015b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51500c8350 con 0x7f515006c0a0 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51500c85f0 con 0x7f515006bcd0 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.290+0000 7f5154f3f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51500c88d0 con 0x7f51500caa90 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58250/0 (socket says 172.21.15.170:58250) 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 -- 172.21.15.170:0/13486530 learned_addr learned my addr 172.21.15.170:0/13486530 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514ed9d700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f515015b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514e59c700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515006bcd0 0x7f5150155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 -- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 msgr2=0x7f515015b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f515015b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 -- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515006bcd0 msgr2=0x7f5150155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515006bcd0 0x7f5150155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 -- 172.21.15.170:0/13486530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148009a20 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f514dd9b700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51380011e0 tx=0x7f5138007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138005b30 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5138005c90 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51500c8bb0 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138007300 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.291+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51500c9170 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.292+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51500c9390 con 0x7f515006c0a0 2023-10-08T16:04:47.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.293+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5138007670 con 0x7f515006c0a0 2023-10-08T16:04:47.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.293+0000 7f51467fc700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 0x7f51300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.293+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f51380839e0 con 0x7f515006c0a0 2023-10-08T16:04:47.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.293+0000 7f514e59c700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 0x7f51300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.294+0000 7f514e59c700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 0x7f51300693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51400039f0 tx=0x7f5140007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.297+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5138050bb0 con 0x7f515006c0a0 2023-10-08T16:04:47.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.478+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f515015ad10 con 0x7f515006c0a0 2023-10-08T16:04:47.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.479+0000 7f51467fc700 1 -- 172.21.15.170:0/13486530 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5138017020 con 0x7f515006c0a0 2023-10-08T16:04:47.480 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:04:47.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 msgr2=0x7f51300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 0x7f51300693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51400039f0 tx=0x7f5140007ec0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 msgr2=0x7f51500c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f51380011e0 tx=0x7f5138007df0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 shutdown_connections 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51500caa90 0x7f515015b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f515006bcd0 0x7f5150155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5130066f30 0x7f51300693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 --2- 172.21.15.170:0/13486530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f515006c0a0 0x7f51500c56b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 >> 172.21.15.170:0/13486530 conn(0x7f51500ce600 msgr2=0x7f51500ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 shutdown_connections 2023-10-08T16:04:47.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.481+0000 7f5154f3f700 1 -- 172.21.15.170:0/13486530 wait complete. 2023-10-08T16:04:47.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:47.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.608+0000 7f2ff0b2c700 1 Processor -- start 2023-10-08T16:04:47.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.608+0000 7f2ff0b2c700 1 -- start start 2023-10-08T16:04:47.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2ff0b2c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2ff0b2c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 0x7f2fec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2ff0b2c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fec15a160 con 0x7f2fec15db10 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2ff0b2c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec0d13a0 con 0x7f2fec159c60 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2ff0b2c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec15e140 con 0x7f2fec159890 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2fead9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2fec15db10 0x7f2fec069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51266/0 (socket says 172.21.15.170:51266) 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.609+0000 7f2fead9d700 1 -- 172.21.15.170:0/2622904823 learned_addr learned my addr 172.21.15.170:0/2622904823 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe9d9b700 1 --2- 172.21.15.170:0/2622904823 >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 0x7f2fec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 --2- 172.21.15.170:0/2622904823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285819687 0 0) 0x7f2fec15a160 con 0x7f2fec15db10 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd8003570 con 0x7f2fec15db10 2023-10-08T16:04:47.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618173946 0 0) 0x7f2fd8003570 con 0x7f2fec15db10 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 -- 172.21.15.170:0/2622904823 >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 msgr2=0x7f2fec15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 --2- 172.21.15.170:0/2622904823 >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 0x7f2fec15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 -- 172.21.15.170:0/2622904823 >> v1:172.21.15.170:6789/0 conn(0x7f2fec15db10 legacy=0x7f2fec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 -- 172.21.15.170:0/2622904823 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec15a3d0 con 0x7f2fec159890 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe9d9b700 1 --2- 172.21.15.170:0/2622904823 >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 0x7f2fec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fea59c700 1 --2- 172.21.15.170:0/2622904823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec15abd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e60 tx=0x7f2fdc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dbcc87e932d0cff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00e070 con 0x7f2fec159890 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00b7a0 con 0x7f2fec159890 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.610+0000 7f2fe959a700 1 -- 172.21.15.170:0/2622904823 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4001470 con 0x7f2fec159890 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/2622904823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 msgr2=0x7f2fec15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/2622904823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec15abd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e60 tx=0x7f2fdc009780 comp rx=0 tx=0).stop 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/2622904823 shutdown_connections 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/2622904823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec15abd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/2622904823 >> v2:172.21.15.191:3300/0 conn(0x7f2fec159c60 0x7f2fec15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/2622904823 >> 172.21.15.170:0/2622904823 conn(0x7f2fec068e20 msgr2=0x7f2fec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/2622904823 shutdown_connections 2023-10-08T16:04:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.611+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/2622904823 wait complete. 2023-10-08T16:04:47.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.612+0000 7f2ff0b2c700 1 Processor -- start 2023-10-08T16:04:47.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.612+0000 7f2ff0b2c700 1 -- start start 2023-10-08T16:04:47.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fec15db10 0x7f2fec0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec147830 con 0x7f2fec159c60 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b5940 con 0x7f2fec15db10 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b6430 con 0x7f2fec159890 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58258/0 (socket says 172.21.15.170:58258) 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 -- 172.21.15.170:0/1198646045 learned_addr learned my addr 172.21.15.170:0/1198646045 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fea59c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fead9d700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fec15db10 0x7f2fec0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 -- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 msgr2=0x7f2fec1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec1472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 -- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fec15db10 msgr2=0x7f2fec0c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fec15db10 0x7f2fec0c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 -- 172.21.15.170:0/1198646045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009420 con 0x7f2fec159c60 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fea59c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe9d9b700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ec80 tx=0x7f2fd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400c9d0 con 0x7f2fec159c60 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd400cb30 con 0x7f2fec159c60 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4013470 con 0x7f2fec159c60 2023-10-08T16:04:47.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc009a20 con 0x7f2fec159c60 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.613+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec0ce4d0 con 0x7f2fec159c60 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.614+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0ce660 con 0x7f2fec159c60 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.615+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fd4005a00 con 0x7f2fec159c60 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.615+0000 7f2fe27fc700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 0x7f2fcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.615+0000 7f2fea59c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 0x7f2fcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.615+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fd4084780 con 0x7f2fec159c60 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.615+0000 7f2fea59c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 0x7f2fcc0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0cffa0 tx=0x7f2fdc018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.619+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd40519f0 con 0x7f2fec159c60 2023-10-08T16:04:47.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.849+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fec04c3c0 con 0x7f2fec159c60 2023-10-08T16:04:47.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.850+0000 7f2fe27fc700 1 -- 172.21.15.170:0/1198646045 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f2fd4017090 con 0x7f2fec159c60 2023-10-08T16:04:47.852 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:47.852 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:04:47.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 msgr2=0x7f2fcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 0x7f2fcc0693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0cffa0 tx=0x7f2fdc018000 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 msgr2=0x7f2fec0ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400ec80 tx=0x7f2fd40091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 shutdown_connections 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fec15db10 0x7f2fec0c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fec159890 0x7f2fec1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fcc066f30 0x7f2fcc0693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 --2- 172.21.15.170:0/1198646045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fec159c60 0x7f2fec0ca280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 >> 172.21.15.170:0/1198646045 conn(0x7f2fec068e20 msgr2=0x7f2fec153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 shutdown_connections 2023-10-08T16:04:47.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.852+0000 7f2ff0b2c700 1 -- 172.21.15.170:0/1198646045 wait complete. 2023-10-08T16:04:47.855 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:04:47.865 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:47.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.978+0000 7fa2210a5700 1 Processor -- start 2023-10-08T16:04:47.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.978+0000 7fa2210a5700 1 -- start start 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa2210a5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa21c159c60 0x7fa21c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa2210a5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c15a160 con 0x7fa21c15db10 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa2210a5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa21c0d13a0 con 0x7fa21c159890 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa2210a5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21c15df50 con 0x7fa21c159c60 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21b59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa21c15db10 0x7fa21c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51270/0 (socket says 172.21.15.170:51270) 2023-10-08T16:04:47.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21b59e700 1 -- 172.21.15.170:0/2111982210 learned_addr learned my addr 172.21.15.170:0/2111982210 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141312763 0 0) 0x7fa21c15a160 con 0x7fa21c15db10 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21259c700 1 --2- 172.21.15.170:0/2111982210 >> v2:172.21.15.191:3300/0 conn(0x7fa21c159c60 0x7fa21c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa208003570 con 0x7fa21c15db10 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773451093 0 0) 0x7fa208003570 con 0x7fa21c15db10 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 >> v2:172.21.15.191:3300/0 conn(0x7fa21c159c60 msgr2=0x7fa21c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 --2- 172.21.15.170:0/2111982210 >> v2:172.21.15.191:3300/0 conn(0x7fa21c159c60 0x7fa21c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 >> v1:172.21.15.191:6789/0 conn(0x7fa21c159890 legacy=0x7fa21c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c15e140 con 0x7fa21c15db10 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fa2140017f0 con 0x7fa21c15db10 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.979+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 >> v1:172.21.15.170:6789/0 conn(0x7fa21c15db10 legacy=0x7fa21c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa208003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa208004810 0x7fa208004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 0x7fa20800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20800c0a0 con 0x7fa2080051c0 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20800c210 con 0x7fa208004810 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20800c3d0 con 0x7fa208003a60 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 0x7fa20800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21259c700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa208004810 0x7fa208004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21ad9d700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa208003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 -- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 msgr2=0x7fa208003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa208003a60 0x7fa208003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 -- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa208004810 msgr2=0x7fa208004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa208004810 0x7fa208004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 -- 172.21.15.170:0/2111982210 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa208003570 con 0x7fa2080051c0 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 -- 172.21.15.170:0/2111982210 reap_dead start 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21b59e700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 0x7fa20800ba60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa21400a930 tx=0x7fa2140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98acc6768bb7c0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140070d0 con 0x7fa2080051c0 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214007230 con 0x7fa2080051c0 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.980+0000 7fa21a59c700 1 -- 172.21.15.170:0/2111982210 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140075d0 con 0x7fa2080051c0 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 -- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 msgr2=0x7fa20800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 0x7fa20800ba60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa21400a930 tx=0x7fa2140021d0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 -- 172.21.15.170:0/2111982210 shutdown_connections 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 --2- 172.21.15.170:0/2111982210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2080051c0 0x7fa20800ba60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 -- 172.21.15.170:0/2111982210 >> 172.21.15.170:0/2111982210 conn(0x7fa21c068e20 msgr2=0x7fa21c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:47.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 -- 172.21.15.170:0/2111982210 shutdown_connections 2023-10-08T16:04:47.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.981+0000 7fa2210a5700 1 -- 172.21.15.170:0/2111982210 wait complete. 2023-10-08T16:04:47.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.982+0000 7fa2210a5700 1 Processor -- start 2023-10-08T16:04:47.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.982+0000 7fa2210a5700 1 -- start start 2023-10-08T16:04:47.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2080051c0 0x7fa21c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa21c0b3590 0x7fa21c0b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b3f40 con 0x7fa21c0b56a0 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0ce1d0 con 0x7fa21c0b3590 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21c0b63f0 con 0x7fa2080051c0 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58280/0 (socket says 172.21.15.170:58280) 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 -- 172.21.15.170:0/3653245727 learned_addr learned my addr 172.21.15.170:0/3653245727 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21ad9d700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2080051c0 0x7fa21c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21259c700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa21c0b3590 0x7fa21c0b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 -- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2080051c0 msgr2=0x7fa21c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2080051c0 0x7fa21c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 -- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa21c0b3590 msgr2=0x7fa21c0b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa21c0b3590 0x7fa21c0b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 -- 172.21.15.170:0/3653245727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0cd8a0 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa21b59e700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cadc0 tx=0x7fa214016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140075b0 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21401d420 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21c0cdb80 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21401d7e0 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.983+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21c151740 con 0x7fa21c0b56a0 2023-10-08T16:04:47.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.984+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21c0ce340 con 0x7fa21c0b56a0 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.985+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa21401db50 con 0x7fa21c0b56a0 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.985+0000 7fa2137fe700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.985+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa214086a00 con 0x7fa21c0b56a0 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.985+0000 7fa21ad9d700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.985+0000 7fa21ad9d700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 0x7fa1fc0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa21c16cea0 tx=0x7fa20c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:47.989+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa214053c70 con 0x7fa21c0b56a0 2023-10-08T16:04:48.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.173+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa21c04c3c0 con 0x7fa21c0b56a0 2023-10-08T16:04:48.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.174+0000 7fa2137fe700 1 -- 172.21.15.170:0/3653245727 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa214057290 con 0x7fa21c0b56a0 2023-10-08T16:04:48.175 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:48.175 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:48.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 msgr2=0x7fa1fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 0x7fa1fc0693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa21c16cea0 tx=0x7fa20c007d30 comp rx=0 tx=0).stop 2023-10-08T16:04:48.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 msgr2=0x7fa21c0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0cadc0 tx=0x7fa214016000 comp rx=0 tx=0).stop 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 shutdown_connections 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1fc066f30 0x7fa1fc0693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa21c0b56a0 0x7fa21c0b5af0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa2080051c0 0x7fa21c151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 --2- 172.21.15.170:0/3653245727 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa21c0b3590 0x7fa21c0b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 >> 172.21.15.170:0/3653245727 conn(0x7fa21c068e20 msgr2=0x7fa21c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 shutdown_connections 2023-10-08T16:04:48.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.176+0000 7fa2210a5700 1 -- 172.21.15.170:0/3653245727 wait complete. 2023-10-08T16:04:48.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:04:48.190 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:04:48.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.305+0000 7f8da964c700 1 Processor -- start 2023-10-08T16:04:48.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.305+0000 7f8da964c700 1 -- start start 2023-10-08T16:04:48.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da964c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 0x7f8da415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da964c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da415a160 con 0x7f8da415db10 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da964c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da40d13a0 con 0x7f8da4159890 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da964c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da415e140 con 0x7f8da4159c60 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da37fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8da415db10 0x7f8da4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51272/0 (socket says 172.21.15.170:51272) 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da37fe700 1 -- 172.21.15.170:0/3391997767 learned_addr learned my addr 172.21.15.170:0/3391997767 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.306+0000 7f8da27fc700 1 --2- 172.21.15.170:0/3391997767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 0x7f8da415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651184906 0 0) 0x7f8da415a160 con 0x7f8da415db10 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d90003570 con 0x7f8da415db10 2023-10-08T16:04:48.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678566238 0 0) 0x7f8da40d13a0 con 0x7f8da4159890 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da415a160 con 0x7f8da4159890 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da27fc700 1 -- 172.21.15.170:0/3391997767 >> v1:172.21.15.191:6789/0 conn(0x7f8da4159890 legacy=0x7f8da415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da27fc700 1 -- 172.21.15.170:0/3391997767 >> v1:172.21.15.170:6789/0 conn(0x7f8da415db10 legacy=0x7f8da4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da27fc700 1 -- 172.21.15.170:0/3391997767 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da415a3d0 con 0x7f8da4159c60 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da27fc700 1 --2- 172.21.15.170:0/3391997767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 0x7f8da415d5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009910 tx=0x7f8d8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24567083bc907915 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c011630 con 0x7f8da4159c60 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8c011790 con 0x7f8da4159c60 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.307+0000 7f8da1ffb700 1 -- 172.21.15.170:0/3391997767 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c011b10 con 0x7f8da4159c60 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 -- 172.21.15.170:0/3391997767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 msgr2=0x7f8da415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 --2- 172.21.15.170:0/3391997767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 0x7f8da415d5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009910 tx=0x7f8d8c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 -- 172.21.15.170:0/3391997767 shutdown_connections 2023-10-08T16:04:48.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 --2- 172.21.15.170:0/3391997767 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da4159c60 0x7f8da415d5d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 -- 172.21.15.170:0/3391997767 >> 172.21.15.170:0/3391997767 conn(0x7f8da4068e20 msgr2=0x7f8da40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 -- 172.21.15.170:0/3391997767 shutdown_connections 2023-10-08T16:04:48.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.308+0000 7f8da964c700 1 -- 172.21.15.170:0/3391997767 wait complete. 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.309+0000 7f8da964c700 1 Processor -- start 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.309+0000 7f8da964c700 1 -- start start 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8da4159c60 0x7f8da414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 0x7f8da40b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b3e20 con 0x7f8da4159890 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b4100 con 0x7f8da4159c60 2023-10-08T16:04:48.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40b43e0 con 0x7f8da415db10 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58284/0 (socket says 172.21.15.170:58284) 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 -- 172.21.15.170:0/2378676586 learned_addr learned my addr 172.21.15.170:0/2378676586 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da37fe700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 0x7f8da40b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da27fc700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8da4159c60 0x7f8da414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 -- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 msgr2=0x7f8da40b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 0x7f8da40b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 -- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8da4159c60 msgr2=0x7f8da414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8da4159c60 0x7f8da414d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 -- 172.21.15.170:0/2378676586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c009420 con 0x7f8da4159890 2023-10-08T16:04:48.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da37fe700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 0x7f8da40b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da2ffd700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400cb40 tx=0x7f8d940027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400a7e0 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9400a940 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da40b46c0 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400ad00 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.310+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da4156460 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.311+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da41566e0 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.312+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d94012630 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.312+0000 7f8d9affd700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 0x7f8d840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.312+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8d94084a10 con 0x7f8da4159890 2023-10-08T16:04:48.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.312+0000 7f8da27fc700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 0x7f8d840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.312+0000 7f8da27fc700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 0x7f8d840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8da415a090 tx=0x7f8d8c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.316+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d94051c80 con 0x7f8da4159890 2023-10-08T16:04:48.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.501+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8da404c3c0 con 0x7f8da4159890 2023-10-08T16:04:48.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.502+0000 7f8d9affd700 1 -- 172.21.15.170:0/2378676586 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d94016020 con 0x7f8da4159890 2023-10-08T16:04:48.503 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 msgr2=0x7f8d840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 0x7f8d840693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8da415a090 tx=0x7f8d8c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 msgr2=0x7f8da40b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400cb40 tx=0x7f8d940027a0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 shutdown_connections 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8da415db10 0x7f8da40b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d84066f30 0x7f8d840693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8da4159890 0x7f8da40b3590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 --2- 172.21.15.170:0/2378676586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8da4159c60 0x7f8da414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 >> 172.21.15.170:0/2378676586 conn(0x7f8da4068e20 msgr2=0x7f8da415b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 shutdown_connections 2023-10-08T16:04:48.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.504+0000 7f8da964c700 1 -- 172.21.15.170:0/2378676586 wait complete. 2023-10-08T16:04:48.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:04:48.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.633+0000 7f46fac85700 1 Processor -- start 2023-10-08T16:04:48.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.633+0000 7f46fac85700 1 -- start start 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46fac85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46fac85700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 0x7f46f40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46fac85700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40b2080 con 0x7f46f415d240 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46fac85700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40cfb00 con 0x7f46f40b16d0 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46fac85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40b5c40 con 0x7f46f40b4e70 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46f9c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46f9482700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 0x7f46f40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46f9c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:43078/0 (socket says 172.21.15.170:43078) 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.634+0000 7f46f9c83700 1 -- 172.21.15.170:0/2605115452 learned_addr learned my addr 172.21.15.170:0/2605115452 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9c83700 1 -- 172.21.15.170:0/2605115452 >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 msgr2=0x7f46f40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9c83700 1 --2- 172.21.15.170:0/2605115452 >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 0x7f46f40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9c83700 1 -- 172.21.15.170:0/2605115452 >> v1:172.21.15.191:6789/0 conn(0x7f46f415d240 legacy=0x7f46f415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9c83700 1 -- 172.21.15.170:0/2605115452 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40b5f20 con 0x7f46f40b4e70 2023-10-08T16:04:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9482700 1 --2- 172.21.15.170:0/2605115452 >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 0x7f46f40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f9c83700 1 --2- 172.21.15.170:0/2605115452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009f20 tx=0x7f46e4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca9477de88690a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f8c81700 1 -- 172.21.15.170:0/2605115452 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400e070 con 0x7f46f40b4e70 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f8c81700 1 -- 172.21.15.170:0/2605115452 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400b9d0 con 0x7f46f40b4e70 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46f8c81700 1 -- 172.21.15.170:0/2605115452 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40092e0 con 0x7f46f40b4e70 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46fac85700 1 -- 172.21.15.170:0/2605115452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 msgr2=0x7f46f40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.635+0000 7f46fac85700 1 --2- 172.21.15.170:0/2605115452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009f20 tx=0x7f46e4009840 comp rx=0 tx=0).stop 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 -- 172.21.15.170:0/2605115452 shutdown_connections 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 --2- 172.21.15.170:0/2605115452 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f40b4e70 0x7f46f40ccff0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 --2- 172.21.15.170:0/2605115452 >> v2:172.21.15.191:3300/0 conn(0x7f46f40b16d0 0x7f46f40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 -- 172.21.15.170:0/2605115452 >> 172.21.15.170:0/2605115452 conn(0x7f46f40d10c0 msgr2=0x7f46f40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 -- 172.21.15.170:0/2605115452 shutdown_connections 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.636+0000 7f46fac85700 1 -- 172.21.15.170:0/2605115452 wait complete. 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 Processor -- start 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 -- start start 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f40b4e70 0x7f46f40cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 0x7f46f41450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c7460 con 0x7f46f40b4e70 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c7fc0 con 0x7f46f40b16d0 2023-10-08T16:04:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.637+0000 7f46fac85700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c7c70 con 0x7f46f415d240 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9482700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f40b4e70 0x7f46f40cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46fa484700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 0x7f46f41450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:37096/0 (socket says 172.21.15.170:37096) 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 -- 172.21.15.170:0/2570331028 learned_addr learned my addr 172.21.15.170:0/2570331028 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 -- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 msgr2=0x7f46f41450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 0x7f46f41450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 -- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f40b4e70 msgr2=0x7f46f40cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f40b4e70 0x7f46f40cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 -- 172.21.15.170:0/2570331028 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8009420 con 0x7f46f40b16d0 2023-10-08T16:04:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46fa484700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 0x7f46f41450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f9c83700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b610 tx=0x7f46e4018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40089e0 con 0x7f46f40b16d0 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e4008b60 con 0x7f46f40b16d0 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4026460 con 0x7f46f40b16d0 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.638+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e4009ae0 con 0x7f46f40b16d0 2023-10-08T16:04:48.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.639+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f40c7990 con 0x7f46f40b16d0 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.639+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f4088990 con 0x7f46f40b16d0 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.640+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46e4003040 con 0x7f46f40b16d0 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.640+0000 7f46f1ffb700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.640+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46e40862e0 con 0x7f46f40b16d0 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.640+0000 7f46f9482700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.641+0000 7f46f9482700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 0x7f46e00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46f416d2d0 tx=0x7f46e80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.646+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46e4053600 con 0x7f46f40b16d0 2023-10-08T16:04:48.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.825+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46f404bb80 con 0x7f46f40b16d0 2023-10-08T16:04:48.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.827+0000 7f46f1ffb700 1 -- 172.21.15.170:0/2570331028 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f46e4056c20 con 0x7f46f40b16d0 2023-10-08T16:04:48.827 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 msgr2=0x7f46e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 0x7f46e00693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f46f416d2d0 tx=0x7f46e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 msgr2=0x7f46f416dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b610 tx=0x7f46e4018000 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 shutdown_connections 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f415d240 0x7f46f41450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f40b16d0 0x7f46f416dda0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 --2- 172.21.15.170:0/2570331028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f40b4e70 0x7f46f40cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 >> 172.21.15.170:0/2570331028 conn(0x7f46f40d10c0 msgr2=0x7f46f4069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 shutdown_connections 2023-10-08T16:04:48.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.828+0000 7f46fac85700 1 -- 172.21.15.170:0/2570331028 wait complete. 2023-10-08T16:04:48.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:04:48.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.956+0000 7fb608e7c700 1 Processor -- start 2023-10-08T16:04:48.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.956+0000 7fb608e7c700 1 -- start start 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb608e7c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 0x7fb604159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb608e7c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb6040cd1c0 0x7fb6040cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb608e7c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6040689e0 con 0x7fb604159930 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb608e7c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040cdb40 con 0x7fb6040cd1c0 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb608e7c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6040cbcc0 con 0x7fb6040cb710 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb602d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb6040cd1c0 0x7fb6040cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb60259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb604159930 0x7fb6040b3ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51274/0 (socket says 172.21.15.170:51274) 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb60259c700 1 -- 172.21.15.170:0/3390777988 learned_addr learned my addr 172.21.15.170:0/3390777988 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:48.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb601d9b700 1 --2- 172.21.15.170:0/3390777988 >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 0x7fb604159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb602d9d700 1 -- 172.21.15.170:0/3390777988 >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 msgr2=0x7fb604159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.957+0000 7fb602d9d700 1 --2- 172.21.15.170:0/3390777988 >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 0x7fb604159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb602d9d700 1 -- 172.21.15.170:0/3390777988 >> v1:172.21.15.170:6789/0 conn(0x7fb604159930 legacy=0x7fb6040b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb60159a700 1 -- 172.21.15.170:0/3390777988 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047581771 0 0) 0x7fb6040689e0 con 0x7fb604159930 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb602d9d700 1 -- 172.21.15.170:0/3390777988 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040cbfa0 con 0x7fb6040cd1c0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb601d9b700 1 --2- 172.21.15.170:0/3390777988 >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 0x7fb604159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb602d9d700 1 --2- 172.21.15.170:0/3390777988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb6040cd600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb60415d6b0 tx=0x7fb5fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398b2dcc5aaba0f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb60159a700 1 -- 172.21.15.170:0/3390777988 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00b740 con 0x7fb6040cd1c0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb60159a700 1 -- 172.21.15.170:0/3390777988 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5fc00b8a0 con 0x7fb6040cd1c0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb60159a700 1 -- 172.21.15.170:0/3390777988 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00bc40 con 0x7fb6040cd1c0 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 -- 172.21.15.170:0/3390777988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 msgr2=0x7fb6040cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 --2- 172.21.15.170:0/3390777988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb6040cd600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb60415d6b0 tx=0x7fb5fc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 -- 172.21.15.170:0/3390777988 shutdown_connections 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 --2- 172.21.15.170:0/3390777988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb6040cd600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 --2- 172.21.15.170:0/3390777988 >> v2:172.21.15.191:3300/0 conn(0x7fb6040cb710 0x7fb604159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 -- 172.21.15.170:0/3390777988 >> 172.21.15.170:0/3390777988 conn(0x7fb6040999d0 msgr2=0x7fb6040b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 -- 172.21.15.170:0/3390777988 shutdown_connections 2023-10-08T16:04:48.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.958+0000 7fb608e7c700 1 -- 172.21.15.170:0/3390777988 wait complete. 2023-10-08T16:04:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.959+0000 7fb608e7c700 1 Processor -- start 2023-10-08T16:04:48.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.959+0000 7fb608e7c700 1 -- start start 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6040cb710 0x7fb60409ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb604159930 0x7fb60409b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60409b6a0 con 0x7fb6040cd1c0 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60409a3e0 con 0x7fb6040cb710 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb608e7c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60409a680 con 0x7fb604159930 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58316/0 (socket says 172.21.15.170:58316) 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 -- 172.21.15.170:0/1295696856 learned_addr learned my addr 172.21.15.170:0/1295696856 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb60259c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6040cb710 0x7fb60409ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb602d9d700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb604159930 0x7fb60409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 -- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb604159930 msgr2=0x7fb60409b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb604159930 0x7fb60409b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.960+0000 7fb601d9b700 1 -- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6040cb710 msgr2=0x7fb60409ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb601d9b700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6040cb710 0x7fb60409ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb601d9b700 1 -- 172.21.15.170:0/1295696856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc009420 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb601d9b700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e8d0 tx=0x7fb5ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec013410 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5ec013570 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60409a960 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec013910 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.961+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60415d430 con 0x7fb6040cd1c0 2023-10-08T16:04:48.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.962+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60404c3c0 con 0x7fb6040cd1c0 2023-10-08T16:04:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.962+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5ec013d00 con 0x7fb6040cd1c0 2023-10-08T16:04:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.962+0000 7fb5fa7fc700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 0x7fb5e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.963+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb5ec0842c0 con 0x7fb6040cd1c0 2023-10-08T16:04:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.963+0000 7fb60259c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 0x7fb5e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.963+0000 7fb60259c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 0x7fb5e40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4003070 tx=0x7fb5f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:48.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:48.967+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5ec051530 con 0x7fb6040cd1c0 2023-10-08T16:04:49.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.148+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb604063650 con 0x7fb6040cd1c0 2023-10-08T16:04:49.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.149+0000 7fb5fa7fc700 1 -- 172.21.15.170:0/1295696856 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5ec054b50 con 0x7fb6040cd1c0 2023-10-08T16:04:49.151 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:04:49.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 msgr2=0x7fb5e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 0x7fb5e40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4003070 tx=0x7fb5f4009040 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 msgr2=0x7fb60415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e8d0 tx=0x7fb5ec008040 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 shutdown_connections 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb604159930 0x7fb60409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6040cb710 0x7fb60409ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb5e4066f30 0x7fb5e40693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 --2- 172.21.15.170:0/1295696856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6040cd1c0 0x7fb60415cd30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 >> 172.21.15.170:0/1295696856 conn(0x7fb6040999d0 msgr2=0x7fb6040c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 shutdown_connections 2023-10-08T16:04:49.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.151+0000 7fb608e7c700 1 -- 172.21.15.170:0/1295696856 wait complete. 2023-10-08T16:04:49.164 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:49.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.279+0000 7f27677f4700 1 Processor -- start 2023-10-08T16:04:49.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.279+0000 7f27677f4700 1 -- start start 2023-10-08T16:04:49.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f27677f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f276015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f27677f4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f276015a160 con 0x7f2760159c60 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f27677f4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27600d13a0 con 0x7f2760069560 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f27677f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276015e140 con 0x7f2760159890 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2760159c60 0x7f276015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51286/0 (socket says 172.21.15.170:51286) 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 -- 172.21.15.170:0/1626345976 learned_addr learned my addr 172.21.15.170:0/1626345976 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2765590700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f276015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923861004 0 0) 0x7f276015a160 con 0x7f2760159c60 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f274c003570 con 0x7f2760159c60 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609208031 0 0) 0x7f274c003570 con 0x7f2760159c60 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 msgr2=0x7f276015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f276015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 >> v1:172.21.15.191:6789/0 conn(0x7f2760069560 legacy=0x7f27600ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276015a3d0 con 0x7f2760159c60 2023-10-08T16:04:49.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f2750001470 con 0x7f2760159c60 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 >> v1:172.21.15.170:6789/0 conn(0x7f2760159c60 legacy=0x7f276015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f274c003a90 0x7f274c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f274c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f274c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276015e140 con 0x7f274c004810 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c00bb10 con 0x7f274c003a90 2023-10-08T16:04:49.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c00bcd0 con 0x7f2760159890 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f274c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2765590700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f274c003a90 0x7f274c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2765d91700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f274c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 -- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 msgr2=0x7f274c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f274c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 -- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f274c003a90 msgr2=0x7f274c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f274c003a90 0x7f274c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 -- 172.21.15.170:0/1626345976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c003570 con 0x7f274c004810 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2765d91700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2760159890 0x7f274c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f274c004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f275000c390 tx=0x7f2750001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fe821f6c2b8c1f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275000cc60 con 0x7f274c004810 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275000cdc0 con 0x7f274c004810 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.280+0000 7f2757fff700 1 -- 172.21.15.170:0/1626345976 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750005b70 con 0x7f274c004810 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 -- 172.21.15.170:0/1626345976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 msgr2=0x7f274c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 --2- 172.21.15.170:0/1626345976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f274c004c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f275000c390 tx=0x7f2750001e70 comp rx=0 tx=0).stop 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f2765d91700 1 -- 172.21.15.170:0/1626345976 reap_dead start 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 -- 172.21.15.170:0/1626345976 shutdown_connections 2023-10-08T16:04:49.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 -- 172.21.15.170:0/1626345976 >> 172.21.15.170:0/1626345976 conn(0x7f2760068e20 msgr2=0x7f27600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 -- 172.21.15.170:0/1626345976 shutdown_connections 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.281+0000 7f27677f4700 1 -- 172.21.15.170:0/1626345976 wait complete. 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.282+0000 7f27677f4700 1 Processor -- start 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.282+0000 7f27677f4700 1 -- start start 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27600cf860 0x7f2760169b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f276016a050 0x7f276016a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276016a9e0 con 0x7f274c004810 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27600ce400 con 0x7f27600cf860 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27600cfe20 con 0x7f276016a050 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58340/0 (socket says 172.21.15.170:58340) 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 -- 172.21.15.170:0/1542772665 learned_addr learned my addr 172.21.15.170:0/1542772665 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765d91700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f276016a050 0x7f276016a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 -- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f276016a050 msgr2=0x7f276016a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f276016a050 0x7f276016a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 -- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27600cf860 msgr2=0x7f2760169b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27600cf860 0x7f2760169b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 -- 172.21.15.170:0/1542772665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600cddb0 con 0x7f274c004810 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f2765590700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f275c011d70 tx=0x7f275c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c009e20 con 0x7f274c004810 2023-10-08T16:04:49.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c00fcd0 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27600ce090 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c014650 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2760148f00 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.284+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276004c3c0 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.285+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f275c014a80 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.285+0000 7f27557fa700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 0x7f2744069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.285+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f275c018030 con 0x7f274c004810 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.285+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 0x7f2744069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.285+0000 7f2764d8f700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 0x7f2744069410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27600b2dc0 tx=0x7f2750015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.289+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f275c052300 con 0x7f274c004810 2023-10-08T16:04:49.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.517+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27600c6e20 con 0x7f274c004810 2023-10-08T16:04:49.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.517+0000 7f27557fa700 1 -- 172.21.15.170:0/1542772665 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f275c022090 con 0x7f274c004810 2023-10-08T16:04:49.519 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:49.519 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 msgr2=0x7f2744069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 0x7f2744069410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f27600b2dc0 tx=0x7f2750015000 comp rx=0 tx=0).stop 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 msgr2=0x7f27601489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f275c011d70 tx=0x7f275c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 shutdown_connections 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f276016a050 0x7f276016a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2744066f60 0x7f2744069410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f274c004810 0x7f27601489c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 --2- 172.21.15.170:0/1542772665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f27600cf860 0x7f2760169b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 >> 172.21.15.170:0/1542772665 conn(0x7f2760068e20 msgr2=0x7f27600c5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 shutdown_connections 2023-10-08T16:04:49.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.519+0000 7f27677f4700 1 -- 172.21.15.170:0/1542772665 wait complete. 2023-10-08T16:04:49.522 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:04:49.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:49.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.645+0000 7f7f22e9a700 1 Processor -- start 2023-10-08T16:04:49.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.645+0000 7f7f22e9a700 1 -- start start 2023-10-08T16:04:49.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f22e9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f1c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f22e9a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c15a160 con 0x7f7f1c159c60 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f22e9a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c0d13a0 con 0x7f7f1c069560 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f22e9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c15e140 con 0x7f7f1c159890 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f1bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7f1c159c60 0x7f7f1c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51288/0 (socket says 172.21.15.170:51288) 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f1bfff700 1 -- 172.21.15.170:0/3899371047 learned_addr learned my addr 172.21.15.170:0/3899371047 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3467418691 0 0) 0x7f7f1c15a160 con 0x7f7f1c159c60 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f1c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.646+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f08003570 con 0x7f7f1c159c60 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2581714078 0 0) 0x7f7f08003570 con 0x7f7f1c159c60 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 msgr2=0x7f7f1c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f1c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 >> v1:172.21.15.191:6789/0 conn(0x7f7f1c069560 legacy=0x7f7f1c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c15a3d0 con 0x7f7f1c159c60 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7f04001470 con 0x7f7f1c159c60 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f1c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 >> v1:172.21.15.170:6789/0 conn(0x7f7f1c159c60 legacy=0x7f7f1c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f08003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f08004810 0x7f7f08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f08005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c15e140 con 0x7f7f08003a90 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0800bbf0 con 0x7f7f08004810 2023-10-08T16:04:49.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0800bdb0 con 0x7f7f1c159890 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f08003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1bfff700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f08004810 0x7f7f08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f21437700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f08005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 -- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 msgr2=0x7f7f08005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f08005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 -- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f08004810 msgr2=0x7f7f08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f08004810 0x7f7f08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 -- 172.21.15.170:0/3899371047 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08003570 con 0x7f7f08003a90 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f21437700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c159890 0x7f7f08005250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f20c36700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f08003ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c007580 tx=0x7f7f0c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c439725c48deb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c009ae0 con 0x7f7f08003a90 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c007dd0 con 0x7f7f08003a90 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.647+0000 7f7f1b7fe700 1 -- 172.21.15.170:0/3899371047 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c014630 con 0x7f7f08003a90 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 -- 172.21.15.170:0/3899371047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 msgr2=0x7f7f08003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/3899371047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f08003ea0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c007580 tx=0x7f7f0c0097d0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f21437700 1 -- 172.21.15.170:0/3899371047 reap_dead start 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 -- 172.21.15.170:0/3899371047 shutdown_connections 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 -- 172.21.15.170:0/3899371047 >> 172.21.15.170:0/3899371047 conn(0x7f7f1c068e20 msgr2=0x7f7f1c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 -- 172.21.15.170:0/3899371047 shutdown_connections 2023-10-08T16:04:49.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.648+0000 7f7f22e9a700 1 -- 172.21.15.170:0/3899371047 wait complete. 2023-10-08T16:04:49.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.649+0000 7f7f22e9a700 1 Processor -- start 2023-10-08T16:04:49.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.649+0000 7f7f22e9a700 1 -- start start 2023-10-08T16:04:49.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f1c0b3f50 0x7f7f1c169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 0x7f7f1c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0cd0c0 con 0x7f7f08003a90 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b4780 con 0x7f7f1c0b3f50 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f22e9a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0cd380 con 0x7f7f1c16c9f0 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58358/0 (socket says 172.21.15.170:58358) 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 -- 172.21.15.170:0/2227330956 learned_addr learned my addr 172.21.15.170:0/2227330956 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f1bfff700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f1c0b3f50 0x7f7f1c169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f21437700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 0x7f7f1c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 -- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 msgr2=0x7f7f1c0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 0x7f7f1c0b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 -- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f1c0b3f50 msgr2=0x7f7f1c169cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f1c0b3f50 0x7f7f1c169cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 -- 172.21.15.170:0/2227330956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0cd660 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f21437700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 0x7f7f1c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.650+0000 7f7f20c36700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f080057d0 tx=0x7f7f0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c01f630 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c01f7d0 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c0cd940 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c01fb90 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c0b43c0 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.651+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c0cdb80 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.652+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f0c003040 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.652+0000 7f7f18ff9700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 0x7f7efc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.652+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f0c0869d0 con 0x7f7f08003a90 2023-10-08T16:04:49.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.652+0000 7f7f1bfff700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 0x7f7efc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.653+0000 7f7f1bfff700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 0x7f7efc069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f040015a0 tx=0x7f7f04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.658+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f1c0cdb80 con 0x7f7f08003a90 2023-10-08T16:04:49.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.840+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f1c0cdb80 con 0x7f7f08003a90 2023-10-08T16:04:49.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.841+0000 7f7f18ff9700 1 -- 172.21.15.170:0/2227330956 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7f0c053c40 con 0x7f7f08003a90 2023-10-08T16:04:49.843 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:49.843 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:49.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 msgr2=0x7f7efc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 0x7f7efc069410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7f040015a0 tx=0x7f7f04009040 comp rx=0 tx=0).stop 2023-10-08T16:04:49.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 msgr2=0x7f7f1c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7f080057d0 tx=0x7f7f0c018000 comp rx=0 tx=0).stop 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 shutdown_connections 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f1c16c9f0 0x7f7f1c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7efc066f60 0x7f7efc069410 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f08003a90 0x7f7f1c0cfb70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 --2- 172.21.15.170:0/2227330956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f1c0b3f50 0x7f7f1c169cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 >> 172.21.15.170:0/2227330956 conn(0x7f7f1c068e20 msgr2=0x7f7f1c0c98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 shutdown_connections 2023-10-08T16:04:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.844+0000 7f7f22e9a700 1 -- 172.21.15.170:0/2227330956 wait complete. 2023-10-08T16:04:49.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:04:49.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.974+0000 7f6a4a399700 1 Processor -- start 2023-10-08T16:04:49.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.974+0000 7f6a4a399700 1 -- start start 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a4a399700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6a44156690 0x7f6a440ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a4a399700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a44157e30 con 0x7f6a44156a60 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a4a399700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a441578b0 con 0x7f6a440af480 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a4a399700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4406aa00 con 0x7f6a44156690 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a43fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6a44156690 0x7f6a440ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a48936700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6a44156a60 0x7f6a4406ae20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51302/0 (socket says 172.21.15.170:51302) 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.975+0000 7f6a48936700 1 -- 172.21.15.170:0/516586506 learned_addr learned my addr 172.21.15.170:0/516586506 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a43fff700 1 -- 172.21.15.170:0/516586506 >> v1:172.21.15.191:6789/0 conn(0x7f6a440af480 legacy=0x7f6a440c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a43fff700 1 -- 172.21.15.170:0/516586506 >> v1:172.21.15.170:6789/0 conn(0x7f6a44156a60 legacy=0x7f6a4406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a43fff700 1 -- 172.21.15.170:0/516586506 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a44157b50 con 0x7f6a44156690 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a42ffd700 1 -- 172.21.15.170:0/516586506 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288909074 0 0) 0x7f6a44157e30 con 0x7f6a44156a60 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a43fff700 1 --2- 172.21.15.170:0/516586506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156690 0x7f6a440ca9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a380098e0 tx=0x7f6a380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9e031937461193 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a42ffd700 1 -- 172.21.15.170:0/516586506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38009e80 con 0x7f6a44156690 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a42ffd700 1 -- 172.21.15.170:0/516586506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3800b860 con 0x7f6a44156690 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a42ffd700 1 -- 172.21.15.170:0/516586506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3800bbe0 con 0x7f6a44156690 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 -- 172.21.15.170:0/516586506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156690 msgr2=0x7f6a440ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 --2- 172.21.15.170:0/516586506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156690 0x7f6a440ca9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a380098e0 tx=0x7f6a380091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 -- 172.21.15.170:0/516586506 shutdown_connections 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 --2- 172.21.15.170:0/516586506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156690 0x7f6a440ca9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 -- 172.21.15.170:0/516586506 >> 172.21.15.170:0/516586506 conn(0x7f6a440c9870 msgr2=0x7f6a440c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:49.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.976+0000 7f6a4a399700 1 -- 172.21.15.170:0/516586506 shutdown_connections 2023-10-08T16:04:49.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.977+0000 7f6a4a399700 1 -- 172.21.15.170:0/516586506 wait complete. 2023-10-08T16:04:49.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 Processor -- start 2023-10-08T16:04:49.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 -- start start 2023-10-08T16:04:49.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a440af480 0x7f6a441691c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a44156690 0x7f6a440aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44069240 con 0x7f6a44156a60 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a440a7010 con 0x7f6a44156690 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.978+0000 7f6a4a399700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a44169700 con 0x7f6a440af480 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58378/0 (socket says 172.21.15.170:58378) 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 -- 172.21.15.170:0/3809860483 learned_addr learned my addr 172.21.15.170:0/3809860483 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a437fe700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a44156690 0x7f6a440aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a43fff700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a440af480 0x7f6a441691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 -- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a440af480 msgr2=0x7f6a441691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a440af480 0x7f6a441691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 -- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a44156690 msgr2=0x7f6a440aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a44156690 0x7f6a440aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 -- 172.21.15.170:0/3809860483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009420 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a48936700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00cc20 tx=0x7f6a3c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00a870 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c00a9f0 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a44068390 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00adb0 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.979+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a44068630 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.980+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4404c3c0 con 0x7f6a44156a60 2023-10-08T16:04:49.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.980+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a3c007300 con 0x7f6a44156a60 2023-10-08T16:04:49.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.981+0000 7f6a2bfff700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 0x7f6a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:49.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.981+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6a3c084740 con 0x7f6a44156a60 2023-10-08T16:04:49.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.981+0000 7f6a43fff700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 0x7f6a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:49.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.981+0000 7f6a43fff700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 0x7f6a300693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a380093f0 tx=0x7f6a38017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:49.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:49.985+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a3c051a30 con 0x7f6a44156a60 2023-10-08T16:04:50.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.208+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a440ca8f0 con 0x7f6a44156a60 2023-10-08T16:04:50.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.209+0000 7f6a2bfff700 1 -- 172.21.15.170:0/3809860483 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f6a3c055050 con 0x7f6a44156a60 2023-10-08T16:04:50.209 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:50.209 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:04:50.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 msgr2=0x7f6a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 0x7f6a300693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a380093f0 tx=0x7f6a38017000 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 msgr2=0x7f6a440c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00cc20 tx=0x7f6a3c0019b0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 shutdown_connections 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6a30066f30 0x7f6a300693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a44156a60 0x7f6a440c6650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6a440af480 0x7f6a441691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 --2- 172.21.15.170:0/3809860483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6a44156690 0x7f6a440aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 >> 172.21.15.170:0/3809860483 conn(0x7f6a440c9870 msgr2=0x7f6a44155500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 shutdown_connections 2023-10-08T16:04:50.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.211+0000 7f6a4a399700 1 -- 172.21.15.170:0/3809860483 wait complete. 2023-10-08T16:04:50.213 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:04:50.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:04:36.203197+0000', 'modified': '2023-10-08T16:04:39.853349+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/2316279916', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 2316279916}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 2316279916}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:04:50.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:04:50.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T16:04:50.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.740+0000 7f74ce1ca700 1 Processor -- start 2023-10-08T16:04:50.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.740+0000 7f74ce1ca700 1 -- start start 2023-10-08T16:04:50.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74ce1ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 0x7f74c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74ce1ca700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f74c8069560 0x7f74c80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74ce1ca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c815a160 con 0x7f74c8159c60 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74ce1ca700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c8069f20 con 0x7f74c8069560 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74ce1ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80d13a0 con 0x7f74c8159890 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74c7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f74c8069560 0x7f74c80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74c7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f74c8069560 0x7f74c80699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58388/0 (socket says 172.21.15.170:58388) 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.741+0000 7f74c7fff700 1 -- 172.21.15.170:0/3664893968 learned_addr learned my addr 172.21.15.170:0/3664893968 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c77fe700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 0x7f74c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c7fff700 1 -- 172.21.15.170:0/3664893968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 msgr2=0x7f74c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c7fff700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 0x7f74c815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c7fff700 1 -- 172.21.15.170:0/3664893968 >> v1:172.21.15.191:6789/0 conn(0x7f74c8159c60 legacy=0x7f74c815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c7fff700 1 -- 172.21.15.170:0/3664893968 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c815e140 con 0x7f74c8069560 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c77fe700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 0x7f74c815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c7fff700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c80699e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74c00099b0 tx=0x7f74c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991be60f88ccd793 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c67fc700 1 -- 172.21.15.170:0/3664893968 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000b860 con 0x7f74c8069560 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c67fc700 1 -- 172.21.15.170:0/3664893968 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000b9c0 con 0x7f74c8069560 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74c67fc700 1 -- 172.21.15.170:0/3664893968 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000bd60 con 0x7f74c8069560 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74ce1ca700 1 -- 172.21.15.170:0/3664893968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 msgr2=0x7f74c80699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c80699e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f74c00099b0 tx=0x7f74c0009250 comp rx=0 tx=0).stop 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74ce1ca700 1 -- 172.21.15.170:0/3664893968 shutdown_connections 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.742+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c80699e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.743+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/3664893968 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159890 0x7f74c815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.743+0000 7f74ce1ca700 1 -- 172.21.15.170:0/3664893968 >> 172.21.15.170:0/3664893968 conn(0x7f74c8068e20 msgr2=0x7f74c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.743+0000 7f74ce1ca700 1 -- 172.21.15.170:0/3664893968 shutdown_connections 2023-10-08T16:04:50.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.743+0000 7f74ce1ca700 1 -- 172.21.15.170:0/3664893968 wait complete. 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.744+0000 7f74ce1ca700 1 Processor -- start 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.744+0000 7f74ce1ca700 1 -- start start 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.744+0000 7f74ce1ca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.744+0000 7f74ce1ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 0x7f74c80ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.744+0000 7f74ce1ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159c60 0x7f74c80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74ce1ca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c816d9b0 con 0x7f74c8069560 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74ce1ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81497e0 con 0x7f74c8159890 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74ce1ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c816cda0 con 0x7f74c8159c60 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58390/0 (socket says 172.21.15.170:58390) 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 -- 172.21.15.170:0/407195657 learned_addr learned my addr 172.21.15.170:0/407195657 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 0x7f74c80ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c7fff700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159c60 0x7f74c80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 -- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159c60 msgr2=0x7f74c80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159c60 0x7f74c80cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 -- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 msgr2=0x7f74c80ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 0x7f74c80ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 -- 172.21.15.170:0/407195657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c00094f0 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 0x7f74c80ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74c77fe700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d1d0 tx=0x7f74b8009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74af7fe700 1 -- 172.21.15.170:0/407195657 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800de70 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74af7fe700 1 -- 172.21.15.170:0/407195657 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800f040 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c816d080 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74af7fe700 1 -- 172.21.15.170:0/407195657 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8015670 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.745+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c816d640 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.746+0000 7f74af7fe700 1 -- 172.21.15.170:0/407195657 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f74b8015a60 con 0x7f74c8069560 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.747+0000 7f74af7fe700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.747+0000 7f74af7fe700 1 -- 172.21.15.170:0/407195657 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74b8084a20 con 0x7f74c8069560 2023-10-08T16:04:50.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.747+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.747+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 0x7f74b40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cbad0 tx=0x7f74bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.749+0000 7f74d74c2b80 1 Processor -- start 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.749+0000 7f74d74c2b80 1 -- start start 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d8441a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 0x5638d8414cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d836fa70 con 0x5638d8418a50 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d836d270 con 0x5638d8462d00 2023-10-08T16:04:50.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d8369060 con 0x5638d8441f60 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58406/0 (socket says 172.21.15.170:58406) 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 -- 172.21.15.170:0/3079533480 learned_addr learned my addr 172.21.15.170:0/3079533480 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d8441a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c7fff700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 0x5638d8414cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 -- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 msgr2=0x5638d8414cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 0x5638d8414cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 -- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 msgr2=0x5638d8441a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d8441a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 -- 172.21.15.170:0/3079533480 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d8203270 con 0x5638d8418a50 2023-10-08T16:04:50.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d8441a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c7fff700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 0x5638d8414cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74c77fe700 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74b8082f40 tx=0x7f74b8082ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765a0856484c6e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74acff9700 1 -- 172.21.15.170:0/3079533480 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8053da0 con 0x5638d8418a50 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74acff9700 1 -- 172.21.15.170:0/3079533480 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8052110 con 0x5638d8418a50 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.750+0000 7f74acff9700 1 -- 172.21.15.170:0/3079533480 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8055e70 con 0x5638d8418a50 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 -- 172.21.15.170:0/3079533480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 msgr2=0x5638d8418e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74b8082f40 tx=0x7f74b8082ea0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 -- 172.21.15.170:0/3079533480 shutdown_connections 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8441f60 0x5638d8414cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8418a50 0x5638d8418e20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/3079533480 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d8441a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 -- 172.21.15.170:0/3079533480 >> 172.21.15.170:0/3079533480 conn(0x5638d841a370 msgr2=0x5638d841a740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 -- 172.21.15.170:0/3079533480 shutdown_connections 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.751+0000 7f74d74c2b80 1 -- 172.21.15.170:0/3079533480 wait complete. 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 -- start start 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 Processor -- start 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8443550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d84594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8314890 con 0x5638d8441f60 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d832fea0 con 0x5638d8462d00 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d81ebe30 con 0x5638d8418a50 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74d74c2b80 10 client.0 _ll_register_callbacks cb 0x5638d83dc110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:04:50.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74c6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74c6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58414/0 (socket says 172.21.15.170:58414) 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74c6ffd700 1 -- 172.21.15.170:0/164399880 learned_addr learned my addr 172.21.15.170:0/164399880 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.752+0000 7f74c77fe700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8443550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c7fff700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d84594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 -- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 msgr2=0x5638d8443550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8443550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 -- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 msgr2=0x5638d84594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d84594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 -- 172.21.15.170:0/164399880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c816d640 con 0x5638d8441f60 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cf4e0 tx=0x7f74bc016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72144e5960d8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:04:50.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc014da0 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74bc01f410 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638d840db70 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74bc01f7e0 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 10 client.4632 fetch_fsmap learned FSMap version 8 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5638d81a5fe0 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5638d840db70 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74bc01f940 con 0x5638d8441f60 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 10 client.4632 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 20 client.4632 populate_metadata read hostname 'smithi170' 2023-10-08T16:04:50.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d846f1d0 0x5638d846f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 4 client.4632 mds_command: new command op to 4272 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.753+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 --> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638d83eaaa0 con 0x5638d846f1d0 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74c77fe700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d846f1d0 0x5638d846f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74c77fe700 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d846f1d0 0x5638d846f5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97257d6fe042abe server_cookie=840b7fbc8db2b4fa in_seq=0 out_seq=0 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74a5ffb700 10 client.4632 ms_handle_connect on v2:172.21.15.170:6835/2316279916 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74a5ffb700 1 -- 172.21.15.170:0/164399880 <== mds.0 v2:172.21.15.170:6835/2316279916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5638d83eaaa0 con 0x5638d846f1d0 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74a5ffb700 10 client.4632 handle_command_reply: tid=0 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.754+0000 7f74d74c2b80 1 client.4632 shutdown 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 msgr2=0x5638d8444f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cf4e0 tx=0x7f74bc016040 comp rx=0 tx=0).stop 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 shutdown_connections 2023-10-08T16:04:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8462d00 0x5638d84594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8443550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d846f1d0 0x5638d846f5a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/164399880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8441f60 0x5638d8444f60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74a5ffb700 0 client.4632 ms_handle_reset on v2:172.21.15.170:6835/2316279916 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 >> 172.21.15.170:0/164399880 conn(0x5638d841a370 msgr2=0x5638d84193e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 shutdown_connections 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 1 -- 172.21.15.170:0/164399880 wait complete. 2023-10-08T16:04:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.755+0000 7f74d74c2b80 20 client.4632 trim_cache size 0 max 16384 2023-10-08T16:04:50.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.771+0000 7f74d74c2b80 1 Processor -- start 2023-10-08T16:04:50.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.771+0000 7f74d74c2b80 1 -- start start 2023-10-08T16:04:50.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8440840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d8419100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8413830 con 0x5638d8462d00 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8443f60 con 0x5638d8441f60 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d846f050 con 0x5638d8418a50 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58428/0 (socket says 172.21.15.170:58428) 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 -- 172.21.15.170:0/503919803 learned_addr learned my addr 172.21.15.170:0/503919803 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c77fe700 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8440840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d8419100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 -- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 msgr2=0x5638d8440840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8440840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 -- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 msgr2=0x5638d8419100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d8419100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.772+0000 7f74c7fff700 1 -- 172.21.15.170:0/503919803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d8337780 con 0x5638d8462d00 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74c7fff700 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b3c0 tx=0x7f74c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8598ad1bbf4990 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74ad7fa700 1 -- 172.21.15.170:0/503919803 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c001f050 con 0x5638d8462d00 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74ad7fa700 1 -- 172.21.15.170:0/503919803 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c0020040 con 0x5638d8462d00 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74ad7fa700 1 -- 172.21.15.170:0/503919803 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000bd80 con 0x5638d8462d00 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 -- 172.21.15.170:0/503919803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 msgr2=0x5638d8419640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b3c0 tx=0x7f74c0017000 comp rx=0 tx=0).stop 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 -- 172.21.15.170:0/503919803 shutdown_connections 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d8419640 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8440840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/503919803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d8419100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 -- 172.21.15.170:0/503919803 >> 172.21.15.170:0/503919803 conn(0x5638d841a370 msgr2=0x5638d841bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 -- 172.21.15.170:0/503919803 shutdown_connections 2023-10-08T16:04:50.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.773+0000 7f74d74c2b80 1 -- 172.21.15.170:0/503919803 wait complete. 2023-10-08T16:04:50.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.774+0000 7f74d74c2b80 1 -- start start 2023-10-08T16:04:50.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.774+0000 7f74d74c2b80 1 Processor -- start 2023-10-08T16:04:50.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8442330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d84c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8447e30 con 0x5638d8462d00 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d8419b30 con 0x5638d8441f60 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d8442870 con 0x5638d8418a50 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 10 client.0 _ll_register_callbacks cb 0x5638d83dc110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58434/0 (socket says 172.21.15.170:58434) 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 -- 172.21.15.170:0/549884817 learned_addr learned my addr 172.21.15.170:0/549884817 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c77fe700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8442330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c6ffd700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d84c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 -- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 msgr2=0x5638d8442330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8442330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 -- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 msgr2=0x5638d84c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d84c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 -- 172.21.15.170:0/549884817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c815e140 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74c7fff700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74c0003010 tx=0x7f74c001c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b2646198f2b7de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c000bd40 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74c0039ab0 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.775+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74b8052110 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c001f020 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 10 client.4638 fetch_fsmap learned FSMap version 8 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5638d831e470 con 0x5638d8462d00 2023-10-08T16:04:50.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8052110 con 0x5638d8462d00 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f74c0011840 con 0x5638d8462d00 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 10 client.4638 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 20 client.4638 populate_metadata read hostname 'smithi170' 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d84482c0 0x5638d8448690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 4 client.4638 mds_command: new command op to 4272 tid=0[{"prefix": "status"}] 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 --> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5638d830be00 con 0x5638d84482c0 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74c77fe700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d84482c0 0x5638d8448690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 10 client.4638 ms_handle_connect on v2:172.21.15.170:6835/2316279916 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74c77fe700 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d84482c0 0x5638d8448690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06eecd523452ceb server_cookie=1d9a644f8ff64f25 in_seq=0 out_seq=0 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 1 -- 172.21.15.170:0/549884817 <== mds.0 v2:172.21.15.170:6835/2316279916 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x5638d830be00 con 0x5638d84482c0 2023-10-08T16:04:50.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 10 client.4638 handle_command_reply: tid=0 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 client.4638 shutdown 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 msgr2=0x5638d84bf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74c0003010 tx=0x7f74c001c070 comp rx=0 tx=0).stop 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 shutdown_connections 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:6835/2316279916,v1:172.21.15.170:6837/2316279916] conn(0x5638d84482c0 0x5638d8448690 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d8462d00 0x5638d84bf040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d8418a50 0x5638d8442330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 --2- 172.21.15.170:0/549884817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d8441f60 0x5638d84c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74a5ffb700 0 client.4638 ms_handle_reset on v2:172.21.15.170:6835/2316279916 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.776+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 >> 172.21.15.170:0/549884817 conn(0x5638d841a370 msgr2=0x5638d8418e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 shutdown_connections 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74d74c2b80 1 -- 172.21.15.170:0/549884817 wait complete. 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74d74c2b80 20 client.4638 trim_cache size 0 max 16384 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4272, 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:04:50.785 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 13.936050687, 2023-10-08T16:04:50.786 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 8, 2023-10-08T16:04:50.786 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 21, 2023-10-08T16:04:50.786 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 21, 2023-10-08T16:04:50.786 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 21.220229086 2023-10-08T16:04:50.786 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 msgr2=0x7f74b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 0x7f74b40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f74c80cbad0 tx=0x7f74bc009040 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 msgr2=0x7f74c81492a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f74b800d1d0 tx=0x7f74b8009a90 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 shutdown_connections 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74c8159c60 0x7f74c80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f74b4066f30 0x7f74b40693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74c8069560 0x7f74c81492a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 --2- 172.21.15.170:0/407195657 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74c8159890 0x7f74c80ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 >> 172.21.15.170:0/407195657 conn(0x7f74c8068e20 msgr2=0x7f74c80c6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 shutdown_connections 2023-10-08T16:04:50.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:50.777+0000 7f74ce1ca700 1 -- 172.21.15.170:0/407195657 wait complete. 2023-10-08T16:04:50.789 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T16:04:50.814 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T16:04:50.814 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-08T16:04:50.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:50.814 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:04:50.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:50.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-08T16:04:50.895 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-08T16:04:50.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:04:50.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:50.897 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:04:50.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:50.952 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-08T16:04:51.017 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-08T16:04:51.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:04:51.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.019 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:04:51.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.037 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T16:04:51.107 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T16:04:51.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:04:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.109 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:04:51.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.165 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T16:04:51.233 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T16:04:51.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:04:51.235 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-08T16:04:51.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:04:51.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.349+0000 7f543628f700 1 Processor -- start 2023-10-08T16:04:51.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.349+0000 7f543628f700 1 -- start start 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543628f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5430141500 0x7f54300c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543628f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f543006c0a0 0x7f54300c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543628f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543015c2a0 con 0x7f543006bcd0 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543628f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54300c9880 con 0x7f543006c0a0 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543628f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543015b690 con 0x7f5430141500 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f543006c0a0 0x7f54300c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f543006bcd0 0x7f5430140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51308/0 (socket says 172.21.15.170:51308) 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542ffff700 1 -- 172.21.15.170:0/3186408945 learned_addr learned my addr 172.21.15.170:0/3186408945 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542f7fe700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5430141500 0x7f54300c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 -- 172.21.15.170:0/3186408945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5430141500 msgr2=0x7f54300c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5430141500 0x7f54300c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 -- 172.21.15.170:0/3186408945 >> v1:172.21.15.170:6789/0 conn(0x7f543006bcd0 legacy=0x7f5430140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 -- 172.21.15.170:0/3186408945 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543015b970 con 0x7f543006c0a0 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542effd700 1 -- 172.21.15.170:0/3186408945 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519717999 0 0) 0x7f543015c2a0 con 0x7f543006bcd0 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f543482c700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543006c0a0 0x7f54300c9340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5428009e60 tx=0x7f5428009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4bc86d937b0b36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:51.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542effd700 1 -- 172.21.15.170:0/3186408945 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800e070 con 0x7f543006c0a0 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542effd700 1 -- 172.21.15.170:0/3186408945 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542800bb60 con 0x7f543006c0a0 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.350+0000 7f542effd700 1 -- 172.21.15.170:0/3186408945 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428012430 con 0x7f543006c0a0 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 -- 172.21.15.170:0/3186408945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543006c0a0 msgr2=0x7f54300c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543006c0a0 0x7f54300c9340 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5428009e60 tx=0x7f5428009780 comp rx=0 tx=0).stop 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 -- 172.21.15.170:0/3186408945 shutdown_connections 2023-10-08T16:04:51.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543006c0a0 0x7f54300c9340 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 --2- 172.21.15.170:0/3186408945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5430141500 0x7f54300c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 -- 172.21.15.170:0/3186408945 >> 172.21.15.170:0/3186408945 conn(0x7f54300ce600 msgr2=0x7f54300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:51.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 -- 172.21.15.170:0/3186408945 shutdown_connections 2023-10-08T16:04:51.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.351+0000 7f543628f700 1 -- 172.21.15.170:0/3186408945 wait complete. 2023-10-08T16:04:51.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.352+0000 7f543628f700 1 Processor -- start 2023-10-08T16:04:51.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.352+0000 7f543628f700 1 -- start start 2023-10-08T16:04:51.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f543006bcd0 0x7f54300c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f543006c0a0 0x7f54300c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5430158470 con 0x7f5430141500 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54300b0380 con 0x7f543006bcd0 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543628f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543013ec60 con 0x7f543006c0a0 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58450/0 (socket says 172.21.15.170:58450) 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 -- 172.21.15.170:0/2217566874 learned_addr learned my addr 172.21.15.170:0/2217566874 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:51.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f542f7fe700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f543006c0a0 0x7f54300c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f542ffff700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f543006bcd0 0x7f54300c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 -- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f543006c0a0 msgr2=0x7f54300c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f543006c0a0 0x7f54300c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 -- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f543006bcd0 msgr2=0x7f54300c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f543006bcd0 0x7f54300c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.353+0000 7f543482c700 1 -- 172.21.15.170:0/2217566874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428009a20 con 0x7f5430141500 2023-10-08T16:04:51.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f543482c700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f542800b6c0 tx=0x7f5428019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428027470 con 0x7f5430141500 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54280275d0 con 0x7f5430141500 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5428027990 con 0x7f5430141500 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543013f220 con 0x7f5430141500 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.354+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543013e610 con 0x7f5430141500 2023-10-08T16:04:51.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.355+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543004c3c0 con 0x7f5430141500 2023-10-08T16:04:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.355+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5428024070 con 0x7f5430141500 2023-10-08T16:04:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.355+0000 7f5417fff700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.356+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5428086bc0 con 0x7f5430141500 2023-10-08T16:04:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.356+0000 7f542ffff700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.356+0000 7f542ffff700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 0x7f541c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54300b4050 tx=0x7f5420007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:51.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.360+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5428053e30 con 0x7f5430141500 2023-10-08T16:04:51.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.585+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f543006ca00 con 0x7f5430141500 2023-10-08T16:04:51.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.585+0000 7f5417fff700 1 -- 172.21.15.170:0/2217566874 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f542802c070 con 0x7f5430141500 2023-10-08T16:04:51.586 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 msgr2=0x7f541c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 0x7f541c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54300b4050 tx=0x7f5420007d30 comp rx=0 tx=0).stop 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 msgr2=0x7f5430157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f542800b6c0 tx=0x7f5428019000 comp rx=0 tx=0).stop 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 shutdown_connections 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f541c066f30 0x7f541c0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5430141500 0x7f5430157f30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f543006bcd0 0x7f54300c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 --2- 172.21.15.170:0/2217566874 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f543006c0a0 0x7f54300c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 >> 172.21.15.170:0/2217566874 conn(0x7f54300ce600 msgr2=0x7f54300c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 shutdown_connections 2023-10-08T16:04:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:51.587+0000 7f543628f700 1 -- 172.21.15.170:0/2217566874 wait complete. 2023-10-08T16:04:51.599 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:04:51.600 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:04:51.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.601 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:04:51.615 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:51.615 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:51.615 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:51.615 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:04:51.616 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:51.617 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:04:51.618 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42459sec preferred_lft 42459sec 2023-10-08T16:04:51.618 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:04:51.618 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:51.618 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T16:04:51.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.619 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:51.619 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:51.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add name ceph-brx type bridge 2023-10-08T16:04:51.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr flush dev ceph-brx 2023-10-08T16:04:51.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set ceph-brx up 2023-10-08T16:04:51.620 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T16:04:51.620 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:51.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.776 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:04:51.820 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:04:51.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.822 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:04:51.849 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:04:51.849 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:04:51.849 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:04:51.849 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:04:51.849 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:04:51.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:51.850 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:51.850 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:51.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:04:51.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:04:51.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:04:51.851 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:52.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.097 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:04:52.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.112 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2023-10-08T16:04:52.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:52.167 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:52.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:04:52.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T16:04:52.168 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:52.256 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T16:04:52.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T16:04:52.257 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:52.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.463 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:52.463 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:52.464 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:04:52.464 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T16:04:52.464 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:52.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:52.556 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:04:52.570 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:04:52.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:52.571 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:04:52.625 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:04:52.625 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:04:52.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:52.712 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:52.712 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:52.803 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:52.804 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:52.826 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:52.827 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:52.827 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:52.827 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:04:52.827 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:04:52.827 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:04:52.829 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:04:52.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:52.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:04:52.854 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-08T16:04:52.854 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:04:53.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.009+0000 7f3251f7b700 1 Processor -- start 2023-10-08T16:04:53.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.009+0000 7f3251f7b700 1 -- start start 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3251f7b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f324c159420 0x7f324c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3251f7b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f324c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3251f7b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324c0b2980 con 0x7f324c1597f0 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3251f7b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f324c15dbb0 con 0x7f324c159420 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3251f7b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b11d0 con 0x7f324c15d230 2023-10-08T16:04:53.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f324bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f324c1597f0 0x7f324c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51310/0 (socket says 172.21.15.170:51310) 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f324bfff700 1 -- 172.21.15.170:0/894257034 learned_addr learned my addr 172.21.15.170:0/894257034 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223493624 0 0) 0x7f324c0b2980 con 0x7f324c1597f0 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3238003570 con 0x7f324c1597f0 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f325177a700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f324c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.010+0000 7f3250f79700 1 --2- 172.21.15.170:0/894257034 >> v2:172.21.15.191:3300/0 conn(0x7f324c159420 0x7f324c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021635381 0 0) 0x7f3238003570 con 0x7f324c1597f0 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 msgr2=0x7f324c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f324c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 >> v2:172.21.15.191:3300/0 conn(0x7f324c159420 msgr2=0x7f324c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 --2- 172.21.15.170:0/894257034 >> v2:172.21.15.191:3300/0 conn(0x7f324c159420 0x7f324c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c0b14b0 con 0x7f324c1597f0 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f3234001470 con 0x7f324c1597f0 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 >> v1:172.21.15.170:6789/0 conn(0x7f324c1597f0 legacy=0x7f324c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3238003a60 0x7f3238003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f3238004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238004d50 0x7f32380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15dbb0 con 0x7f3238004d50 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0b11d0 con 0x7f3238003a60 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323800bae0 con 0x7f324c15d230 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f3250f79700 1 --2- 172.21.15.170:0/894257034 >> v2:172.21.15.191:3300/0 conn(0x7f324c159420 0x7f324c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238004d50 0x7f32380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324bfff700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f3238004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f3250f79700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3238003a60 0x7f3238003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 -- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 msgr2=0x7f3238004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c15d230 0x7f3238004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 -- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3238003a60 msgr2=0x7f3238003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3238003a60 0x7f3238003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 -- 172.21.15.170:0/894257034 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238003570 con 0x7f3238004d50 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f325177a700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238004d50 0x7f32380051c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3244009c00 tx=0x7f32440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59ae03dff87f784 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400e710 con 0x7f3238004d50 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324400e870 con 0x7f3238004d50 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.011+0000 7f324b7fe700 1 -- 172.21.15.170:0/894257034 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324400ec30 con 0x7f3238004d50 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 -- 172.21.15.170:0/894257034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238004d50 msgr2=0x7f32380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 --2- 172.21.15.170:0/894257034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3238004d50 0x7f32380051c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3244009c00 tx=0x7f32440091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f325177a700 1 -- 172.21.15.170:0/894257034 reap_dead start 2023-10-08T16:04:53.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 -- 172.21.15.170:0/894257034 shutdown_connections 2023-10-08T16:04:53.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 -- 172.21.15.170:0/894257034 >> 172.21.15.170:0/894257034 conn(0x7f324c0d0660 msgr2=0x7f324c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:53.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 -- 172.21.15.170:0/894257034 shutdown_connections 2023-10-08T16:04:53.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.012+0000 7f3251f7b700 1 -- 172.21.15.170:0/894257034 wait complete. 2023-10-08T16:04:53.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 Processor -- start 2023-10-08T16:04:53.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 -- start start 2023-10-08T16:04:53.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c1550e0 0x7f324c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f324c0c6740 0x7f324c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15c300 con 0x7f324c154d10 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15b6f0 con 0x7f324c0c6740 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3251f7b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324c15b9d0 con 0x7f324c1550e0 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3250f79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.014+0000 7f3250f79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58468/0 (socket says 172.21.15.170:58468) 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 -- 172.21.15.170:0/1695511654 learned_addr learned my addr 172.21.15.170:0/1695511654 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f325177a700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f324c0c6740 0x7f324c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f324bfff700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c1550e0 0x7f324c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 -- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c1550e0 msgr2=0x7f324c15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c1550e0 0x7f324c15c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 -- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f324c0c6740 msgr2=0x7f324c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f324c0c6740 0x7f324c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 -- 172.21.15.170:0/1695511654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c15bcb0 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3250f79700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f323c007230 tx=0x7f323c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c01b070 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c010d40 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c15bf90 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c017680 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.015+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c0c6d70 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.016+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c15c5e0 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.016+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f323c003020 con 0x7f324c154d10 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.017+0000 7f3248ff9700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 0x7f322c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:53.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.017+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f323c084c90 con 0x7f324c154d10 2023-10-08T16:04:53.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.017+0000 7f324bfff700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 0x7f322c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:53.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.017+0000 7f324bfff700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 0x7f322c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3234001d20 tx=0x7f3234001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:53.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.021+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f323c0505a0 con 0x7f324c154d10 2023-10-08T16:04:53.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.248+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f324c04c3c0 con 0x7f324c154d10 2023-10-08T16:04:53.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.248+0000 7f3248ff9700 1 -- 172.21.15.170:0/1695511654 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f323c051f00 con 0x7f324c154d10 2023-10-08T16:04:53.248 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 msgr2=0x7f322c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 0x7f322c0693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3234001d20 tx=0x7f3234001e90 comp rx=0 tx=0).stop 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 msgr2=0x7f324c15b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f323c007230 tx=0x7f323c0097d0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 shutdown_connections 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f324c0c6740 0x7f324c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f322c066f30 0x7f322c0693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f324c154d10 0x7f324c15b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 --2- 172.21.15.170:0/1695511654 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f324c1550e0 0x7f324c15c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:53.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 >> 172.21.15.170:0/1695511654 conn(0x7f324c0d0660 msgr2=0x7f324c0b4020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:53.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 shutdown_connections 2023-10-08T16:04:53.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:53.250+0000 7f3251f7b700 1 -- 172.21.15.170:0/1695511654 wait complete. 2023-10-08T16:04:53.262 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:04:53.263 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-08T16:04:53.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.263 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:04:53.278 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:53.278 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:53.279 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42458sec preferred_lft 42458sec 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:53.280 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link tentative 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.281 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:04:53.342 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:04:53.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.344 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:04:53.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:04:53.401 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:53.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.500 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:04:53.514 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:04:53.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.515 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list-id 2023-10-08T16:04:53.568 INFO:teuthology.orchestra.run.smithi170.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-08T16:04:53.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:53.569 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:53.569 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:04:53.569 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-08T16:04:53.569 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:53.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.692 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:04:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6e:70:04:fd:dd:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T16:04:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T16:04:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6c70:4ff:fefd:ddf2/64 scope link tentative 2023-10-08T16:04:53.693 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:04:53.782 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6e:70:04:fd:dd:f2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T16:04:53.783 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T16:04:53.783 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.783 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6c70:4ff:fefd:ddf2/64 scope link tentative 2023-10-08T16:04:53.783 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:53.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-08T16:04:53.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:53.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:53.795 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-08T16:04:53.796 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:54.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.020 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:04:54.020 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:04:54.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.1 up 2023-10-08T16:04:54.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T16:04:54.020 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:04:54.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:54.100 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-08T16:04:54.114 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-08T16:04:54.114 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:54.114 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-08T16:04:54.168 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-08T16:04:54.168 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:04:54.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:54.247 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:54.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:54.266 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:54.266 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:54.266 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:54.266 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 2@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:54.267 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:54.267 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:54.267 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:54.267 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 2@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:04:54.289 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:04:54.290 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:04:54.291 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:04:54.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.291 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-08T16:04:54.356 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-08T16:04:54.356 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:04:54.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.511+0000 7f3309966700 1 Processor -- start 2023-10-08T16:04:54.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.511+0000 7f3309966700 1 -- start start 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3309966700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f330415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3309966700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3304159c60 0x7f330415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3309966700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330415a160 con 0x7f330415db10 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3309966700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33040d13a0 con 0x7f3304159c60 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3309966700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330415e140 con 0x7f3304159890 2023-10-08T16:04:54.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f33037fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f330415db10 0x7f3304069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51314/0 (socket says 172.21.15.170:51314) 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f33037fe700 1 -- 172.21.15.170:0/198946322 learned_addr learned my addr 172.21.15.170:0/198946322 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.512+0000 7f3302ffd700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f330415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 --2- 172.21.15.170:0/198946322 >> v2:172.21.15.191:3300/0 conn(0x7f3304159c60 0x7f330415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073681974 0 0) 0x7f330415a160 con 0x7f330415db10 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f0003570 con 0x7f330415db10 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 875217980 0 0) 0x7f32f0003570 con 0x7f330415db10 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 msgr2=0x7f330415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:54.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f330415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 >> v2:172.21.15.191:3300/0 conn(0x7f3304159c60 msgr2=0x7f330415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 --2- 172.21.15.170:0/198946322 >> v2:172.21.15.191:3300/0 conn(0x7f3304159c60 0x7f330415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330415a3d0 con 0x7f330415db10 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f32fc001470 con 0x7f330415db10 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 >> v1:172.21.15.170:6789/0 conn(0x7f330415db10 legacy=0x7f3304069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f32f0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f0003fe0 0x7f32f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0004d50 0x7f32f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33040d13a0 con 0x7f32f0003fe0 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330415e140 con 0x7f32f0004d50 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f000ba90 con 0x7f3304159890 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f0003fe0 0x7f32f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3302ffd700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f32f0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33037fe700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0004d50 0x7f32f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 -- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 msgr2=0x7f32f0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:54.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3304159890 0x7f32f0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 -- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0004d50 msgr2=0x7f32f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0004d50 0x7f32f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 -- 172.21.15.170:0/198946322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0003570 con 0x7f32f0003fe0 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33037fe700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0004d50 0x7f32f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f33027fc700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f0003fe0 0x7f32f0004810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f330415a090 tx=0x7f32ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad67efc74fc1253b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.513+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00e6b0 con 0x7f32f0003fe0 2023-10-08T16:04:54.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec00e810 con 0x7f32f0003fe0 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3301ffb700 1 -- 172.21.15.170:0/198946322 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec00ebd0 con 0x7f32f0003fe0 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 -- 172.21.15.170:0/198946322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f0003fe0 msgr2=0x7f32f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 --2- 172.21.15.170:0/198946322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32f0003fe0 0x7f32f0004810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f330415a090 tx=0x7f32ec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f33037fe700 1 -- 172.21.15.170:0/198946322 reap_dead start 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 -- 172.21.15.170:0/198946322 shutdown_connections 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 -- 172.21.15.170:0/198946322 >> 172.21.15.170:0/198946322 conn(0x7f3304068e20 msgr2=0x7f33040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 -- 172.21.15.170:0/198946322 shutdown_connections 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.514+0000 7f3309966700 1 -- 172.21.15.170:0/198946322 wait complete. 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.515+0000 7f3309966700 1 Processor -- start 2023-10-08T16:04:54.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.515+0000 7f3309966700 1 -- start start 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0003fe0 0x7f3304151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f330416ce50 0x7f330416d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330416d830 con 0x7f33040b17b0 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33040b36c0 con 0x7f32f0003fe0 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3309966700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33040b1c20 con 0x7f330416ce50 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f33027fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f33027fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:58494/0 (socket says 172.21.15.170:58494) 2023-10-08T16:04:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f33027fc700 1 -- 172.21.15.170:0/2482173951 learned_addr learned my addr 172.21.15.170:0/2482173951 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f3302ffd700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0003fe0 0x7f3304151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.516+0000 7f33037fe700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f330416ce50 0x7f330416d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 -- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f330416ce50 msgr2=0x7f330416d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f330416ce50 0x7f330416d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 -- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0003fe0 msgr2=0x7f3304151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0003fe0 0x7f3304151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 -- 172.21.15.170:0/2482173951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304152280 con 0x7f33040b17b0 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f33027fc700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f32f0004730 tx=0x7f32ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec022070 con 0x7f33040b17b0 2023-10-08T16:04:54.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec00e980 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3304152560 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec027470 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.517+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3304152b20 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.518+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330404c3c0 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.518+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32ec01c450 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.519+0000 7f32faffd700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 0x7f32e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.519+0000 7f3302ffd700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 0x7f32e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.519+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32ec085ff0 con 0x7f33040b17b0 2023-10-08T16:04:54.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.519+0000 7f3302ffd700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 0x7f32e40693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33041534f0 tx=0x7f32f400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:54.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.523+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32ec053360 con 0x7f33040b17b0 2023-10-08T16:04:54.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.751+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33040b3800 con 0x7f33040b17b0 2023-10-08T16:04:54.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.752+0000 7f32faffd700 1 -- 172.21.15.170:0/2482173951 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32ec02c060 con 0x7f33040b17b0 2023-10-08T16:04:54.752 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:04:54.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 msgr2=0x7f32e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 0x7f32e40693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f33041534f0 tx=0x7f32f400b040 comp rx=0 tx=0).stop 2023-10-08T16:04:54.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 msgr2=0x7f330414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f32f0004730 tx=0x7f32ec01a000 comp rx=0 tx=0).stop 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 shutdown_connections 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f330416ce50 0x7f330416d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32f0003fe0 0x7f3304151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f32e4066f30 0x7f32e40693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 --2- 172.21.15.170:0/2482173951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33040b17b0 0x7f330414d1b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 >> 172.21.15.170:0/2482173951 conn(0x7f3304068e20 msgr2=0x7f33040cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 shutdown_connections 2023-10-08T16:04:54.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:54.754+0000 7f3309966700 1 -- 172.21.15.170:0/2482173951 wait complete. 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T16:04:54.766 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:04:54.767 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:04:54.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.767 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:54.785 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42457sec preferred_lft 42457sec 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:04:54.786 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:54.787 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T16:04:54.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add name ceph-brx type bridge 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr flush dev ceph-brx 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set ceph-brx up 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T16:04:54.787 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:54.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.952 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:04:54.996 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:04:54.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:54.997 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:04:55.023 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:04:55.024 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:04:55.024 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:04:55.024 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:04:55.024 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:04:55.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.024 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:55.024 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:55.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:04:55.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:04:55.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:04:55.025 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:55.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.264 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:04:55.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.278 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2023-10-08T16:04:55.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.332 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:55.332 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:55.332 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:04:55.332 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-08T16:04:55.333 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:55.422 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-08T16:04:55.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.422 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:55.422 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:55.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-08T16:04:55.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T16:04:55.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-08T16:04:55.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-08T16:04:55.423 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-08T16:04:55.423 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:55.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.601 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:55.601 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:55.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:04:55.601 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T16:04:55.601 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:55.693 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:55.693 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:04:55.708 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:04:55.708 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:55.708 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:04:55.762 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:04:55.762 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:04:55.762 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:55.860 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:55.860 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:55.950 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:55.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:55.950 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:55.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:55.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:55.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:55.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:55.951 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:55.972 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:55.973 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:55.974 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:04:55.974 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:04:55.974 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:04:55.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:04:55.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:55.976 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:04:56.000 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-08T16:04:56.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:04:56.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.126+0000 7fafd7e5b700 1 Processor -- start 2023-10-08T16:04:56.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 -- start start 2023-10-08T16:04:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fafd0159890 0x7fafd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fafd015db10 0x7fafd0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd015a160 con 0x7fafd0159890 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00d13a0 con 0x7fafd015db10 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.127+0000 7fafd7e5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd015e140 con 0x7fafd0159c60 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fafd0159890 0x7fafd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fafd0159890 0x7fafd015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53476/0 (socket says 172.21.15.170:53476) 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 -- 172.21.15.170:0/729335831 learned_addr learned my addr 172.21.15.170:0/729335831 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd63f8700 1 --2- 172.21.15.170:0/729335831 >> v2:172.21.15.191:3300/0 conn(0x7fafd015db10 0x7fafd0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd53f6700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 -- 172.21.15.170:0/729335831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 msgr2=0x7fafd015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 -- 172.21.15.170:0/729335831 >> v2:172.21.15.191:3300/0 conn(0x7fafd015db10 msgr2=0x7fafd0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/729335831 >> v2:172.21.15.191:3300/0 conn(0x7fafd015db10 0x7fafd0069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 -- 172.21.15.170:0/729335831 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd015a3d0 con 0x7fafd0159890 2023-10-08T16:04:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd0159890 0x7fafd015abd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafd0089b50 tx=0x7fafcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3713bc6305554ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd4bf5700 1 -- 172.21.15.170:0/729335831 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00e070 con 0x7fafd0159890 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd4bf5700 1 -- 172.21.15.170:0/729335831 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc00ba50 con 0x7fafd0159890 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.128+0000 7fafd4bf5700 1 -- 172.21.15.170:0/729335831 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0092e0 con 0x7fafd0159890 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 -- 172.21.15.170:0/729335831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd0159890 msgr2=0x7fafd015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd0159890 0x7fafd015abd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fafd0089b50 tx=0x7fafcc009840 comp rx=0 tx=0).stop 2023-10-08T16:04:56.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 -- 172.21.15.170:0/729335831 shutdown_connections 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/729335831 >> v2:172.21.15.191:3300/0 conn(0x7fafd015db10 0x7fafd0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd0159890 0x7fafd015abd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/729335831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 -- 172.21.15.170:0/729335831 >> 172.21.15.170:0/729335831 conn(0x7fafd0068e20 msgr2=0x7fafd00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 -- 172.21.15.170:0/729335831 shutdown_connections 2023-10-08T16:04:56.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.129+0000 7fafd7e5b700 1 -- 172.21.15.170:0/729335831 wait complete. 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.130+0000 7fafd7e5b700 1 Processor -- start 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.130+0000 7fafd7e5b700 1 -- start start 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd0159890 0x7fafd00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd00b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b3f30 con 0x7fafd015db10 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b4210 con 0x7fafd0159890 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00b46a0 con 0x7fafd0159c60 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53478/0 (socket says 172.21.15.170:53478) 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 -- 172.21.15.170:0/2179876470 learned_addr learned my addr 172.21.15.170:0/2179876470 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd53f6700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd00b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 -- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 msgr2=0x7fafd00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd00b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd0159890 0x7fafd00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 -- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd0159890 msgr2=0x7fafd00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd0159890 0x7fafd00cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 -- 172.21.15.170:0/2179876470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc009ae0 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd63f8700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c8b0 tx=0x7fafc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8011600 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc8011760 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc8011b00 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.131+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd01583e0 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.132+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd01589a0 con 0x7fafd015db10 2023-10-08T16:04:56.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.133+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd004c3c0 con 0x7fafd015db10 2023-10-08T16:04:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.133+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafc8018360 con 0x7fafd015db10 2023-10-08T16:04:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.133+0000 7fafc5ffb700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 0x7fafb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.133+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 0x7fafb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.133+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fafc8084280 con 0x7fafd015db10 2023-10-08T16:04:56.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.134+0000 7fafd5bf7700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 0x7fafb40693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009ab0 tx=0x7fafcc018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:56.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.138+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafc8054f80 con 0x7fafd015db10 2023-10-08T16:04:56.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.363+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fafd006a5b0 con 0x7fafd015db10 2023-10-08T16:04:56.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.364+0000 7fafc5ffb700 1 -- 172.21.15.170:0/2179876470 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fafc8054b10 con 0x7fafd015db10 2023-10-08T16:04:56.364 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:04:56.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 msgr2=0x7fafb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:56.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 0x7fafb40693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009ab0 tx=0x7fafcc018000 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 msgr2=0x7fafd016d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fafc800c8b0 tx=0x7fafc8008040 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 shutdown_connections 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fafb4066f30 0x7fafb40693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafd015db10 0x7fafd016d160 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fafd0159890 0x7fafd00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 --2- 172.21.15.170:0/2179876470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fafd0159c60 0x7fafd00b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 >> 172.21.15.170:0/2179876470 conn(0x7fafd0068e20 msgr2=0x7fafd0153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 shutdown_connections 2023-10-08T16:04:56.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:56.366+0000 7fafd7e5b700 1 -- 172.21.15.170:0/2179876470 wait complete. 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T16:04:56.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:04:56.379 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-08T16:04:56.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.379 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:56.395 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42455sec preferred_lft 42455sec 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.396 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link tentative 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link tentative 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.398 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:04:56.459 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:04:56.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.460 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:04:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:04:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:04:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:04:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:04:56.516 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:04:56.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:04:56.517 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:56.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.617 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:04:56.631 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:04:56.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.632 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list-id 2023-10-08T16:04:56.686 INFO:teuthology.orchestra.run.smithi191.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-08T16:04:56.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.686 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:56.687 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:56.687 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:04:56.687 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-08T16:04:56.687 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:56.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.775 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 16:db:79:70:17:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T16:04:56.819 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T16:04:56.820 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.820 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::14db:79ff:fe70:173a/64 scope link tentative 2023-10-08T16:04:56.820 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.833 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T16:04:56.912 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 16:db:79:70:17:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::14db:79ff:fe70:173a/64 scope link tentative 2023-10-08T16:04:56.913 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:04:56.922 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-08T16:04:56.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-08T16:04:56.922 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-08T16:04:56.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-08T16:04:56.923 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:57.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:57.126 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:04:57.126 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:04:57.127 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.1 up 2023-10-08T16:04:57.127 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T16:04:57.127 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:04:57.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:57.206 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-08T16:04:57.221 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-08T16:04:57.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:04:57.221 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-08T16:04:57.276 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-08T16:04:57.276 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:04:57.277 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:57.356 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:57.357 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:57.375 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:57.375 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:57.375 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:57.375 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 3@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:57.376 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:57.376 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:57.376 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:57.376 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:57.397 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:04:57.397 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:04:57.397 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:04:57.397 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 3@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:04:57.397 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:04:57.398 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:04:57.399 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:04:57.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:04:57.400 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-08T16:04:57.464 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-08T16:04:57.482 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:57.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.606+0000 7fae78c88700 1 Processor -- start 2023-10-08T16:04:57.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.606+0000 7fae78c88700 1 -- start start 2023-10-08T16:04:57.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae78c88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 0x7fae7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae78c88700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fae74069560 0x7fae740699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae78c88700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7415a160 con 0x7fae74159c60 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae78c88700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae74069f20 con 0x7fae74069560 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae78c88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740d13a0 con 0x7fae74159890 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fae74069560 0x7fae740699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae71d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fae74159c60 0x7fae7415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54160/0 (socket says 172.21.15.170:54160) 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae71d9b700 1 -- 172.21.15.170:0/2817363044 learned_addr learned my addr 172.21.15.170:0/2817363044 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 -- 172.21.15.170:0/2817363044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 msgr2=0x7fae7415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7259c700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 0x7fae7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 0x7fae7415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7159a700 1 -- 172.21.15.170:0/2817363044 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882366958 0 0) 0x7fae7415a160 con 0x7fae74159c60 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 -- 172.21.15.170:0/2817363044 >> v1:172.21.15.170:6789/0 conn(0x7fae74159c60 legacy=0x7fae7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 -- 172.21.15.170:0/2817363044 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7415e140 con 0x7fae74069560 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7259c700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 0x7fae7415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae72d9d700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74069560 0x7fae740699e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009e60 tx=0x7fae6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f0f5da2777f0df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7159a700 1 -- 172.21.15.170:0/2817363044 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00e070 con 0x7fae74069560 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7159a700 1 -- 172.21.15.170:0/2817363044 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c00bc30 con 0x7fae74069560 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.607+0000 7fae7159a700 1 -- 172.21.15.170:0/2817363044 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c012430 con 0x7fae74069560 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 -- 172.21.15.170:0/2817363044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74069560 msgr2=0x7fae740699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74069560 0x7fae740699e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009e60 tx=0x7fae6c009780 comp rx=0 tx=0).stop 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 -- 172.21.15.170:0/2817363044 shutdown_connections 2023-10-08T16:04:57.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74159890 0x7fae7415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 --2- 172.21.15.170:0/2817363044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74069560 0x7fae740699e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 -- 172.21.15.170:0/2817363044 >> 172.21.15.170:0/2817363044 conn(0x7fae74068e20 msgr2=0x7fae740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:57.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 -- 172.21.15.170:0/2817363044 shutdown_connections 2023-10-08T16:04:57.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.608+0000 7fae78c88700 1 -- 172.21.15.170:0/2817363044 wait complete. 2023-10-08T16:04:57.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.609+0000 7fae78c88700 1 Processor -- start 2023-10-08T16:04:57.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.609+0000 7fae78c88700 1 -- start start 2023-10-08T16:04:57.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74069560 0x7fae740c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae74159890 0x7fae74145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae741524a0 con 0x7fae74159c60 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae740c8760 con 0x7fae74159890 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae78c88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae740c81a0 con 0x7fae74069560 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53494/0 (socket says 172.21.15.170:53494) 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 -- 172.21.15.170:0/3593782289 learned_addr learned my addr 172.21.15.170:0/3593782289 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae7259c700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74069560 0x7fae740c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae71d9b700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae74159890 0x7fae74145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 -- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74069560 msgr2=0x7fae740c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74069560 0x7fae740c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 -- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae74159890 msgr2=0x7fae74145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae74159890 0x7fae74145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 -- 172.21.15.170:0/3593782289 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6c009a20 con 0x7fae74159c60 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.610+0000 7fae72d9d700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae6c013000 tx=0x7fae6c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c012e90 con 0x7fae74159c60 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6c01e630 con 0x7fae74159c60 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae740c8480 con 0x7fae74159c60 2023-10-08T16:04:57.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c01e9f0 con 0x7fae74159c60 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae740c7de0 con 0x7fae74159c60 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.611+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae74152610 con 0x7fae74159c60 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.612+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae6c024070 con 0x7fae74159c60 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.612+0000 7fae627fc700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.612+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae6c086e90 con 0x7fae74159c60 2023-10-08T16:04:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.612+0000 7fae7259c700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.613+0000 7fae7259c700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 0x7fae540693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae6400adc0 tx=0x7fae64006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.616+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae6c054100 con 0x7fae74159c60 2023-10-08T16:04:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.800+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae7404c3c0 con 0x7fae74159c60 2023-10-08T16:04:57.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.801+0000 7fae627fc700 1 -- 172.21.15.170:0/3593782289 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fae6c057720 con 0x7fae74159c60 2023-10-08T16:04:57.801 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:57.801 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:57.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 msgr2=0x7fae540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 0x7fae540693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae6400adc0 tx=0x7fae64006f90 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 msgr2=0x7fae740cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae6c013000 tx=0x7fae6c019000 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 shutdown_connections 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae54066f30 0x7fae540693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae74159c60 0x7fae740cd0c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae74069560 0x7fae740c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 --2- 172.21.15.170:0/3593782289 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae74159890 0x7fae74145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 >> 172.21.15.170:0/3593782289 conn(0x7fae74068e20 msgr2=0x7fae7415d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 shutdown_connections 2023-10-08T16:04:57.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.803+0000 7fae78c88700 1 -- 172.21.15.170:0/3593782289 wait complete. 2023-10-08T16:04:57.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:04:57.816 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:04:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.931+0000 7fc0043ec700 1 Processor -- start 2023-10-08T16:04:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.931+0000 7fc0043ec700 1 -- start start 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc0043ec700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbffc159890 0x7fbffc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc0043ec700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15a160 con 0x7fbffc159c60 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc0043ec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc0d13a0 con 0x7fbffc069560 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc0043ec700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc15e140 con 0x7fbffc159890 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc002188700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbffc159890 0x7fbffc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc001987700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbffc159c60 0x7fbffc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54174/0 (socket says 172.21.15.170:54174) 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.932+0000 7fc001987700 1 -- 172.21.15.170:0/1881413350 learned_addr learned my addr 172.21.15.170:0/1881413350 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc002188700 1 -- 172.21.15.170:0/1881413350 >> v1:172.21.15.191:6789/0 conn(0x7fbffc069560 legacy=0x7fbffc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc002188700 1 -- 172.21.15.170:0/1881413350 >> v1:172.21.15.170:6789/0 conn(0x7fbffc159c60 legacy=0x7fbffc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc002188700 1 -- 172.21.15.170:0/1881413350 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc15a3d0 con 0x7fbffc159890 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc002188700 1 --2- 172.21.15.170:0/1881413350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc159890 0x7fbffc15abd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009e60 tx=0x7fbfec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb39d57c16f9ed43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc001186700 1 -- 172.21.15.170:0/1881413350 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec00e070 con 0x7fbffc159890 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc001186700 1 -- 172.21.15.170:0/1881413350 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec00bb30 con 0x7fbffc159890 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc001186700 1 -- 172.21.15.170:0/1881413350 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec012430 con 0x7fbffc159890 2023-10-08T16:04:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc0043ec700 1 -- 172.21.15.170:0/1881413350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc159890 msgr2=0x7fbffc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.933+0000 7fc0043ec700 1 --2- 172.21.15.170:0/1881413350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc159890 0x7fbffc15abd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009e60 tx=0x7fbfec009780 comp rx=0 tx=0).stop 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.934+0000 7fc0043ec700 1 -- 172.21.15.170:0/1881413350 shutdown_connections 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.934+0000 7fc0043ec700 1 --2- 172.21.15.170:0/1881413350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc159890 0x7fbffc15abd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.934+0000 7fc0043ec700 1 -- 172.21.15.170:0/1881413350 >> 172.21.15.170:0/1881413350 conn(0x7fbffc068e20 msgr2=0x7fbffc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.934+0000 7fc0043ec700 1 -- 172.21.15.170:0/1881413350 shutdown_connections 2023-10-08T16:04:57.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.934+0000 7fc0043ec700 1 -- 172.21.15.170:0/1881413350 wait complete. 2023-10-08T16:04:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 Processor -- start 2023-10-08T16:04:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 -- start start 2023-10-08T16:04:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbffc159890 0x7fbffc16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbffc159c60 0x7fbffc0c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc157230 con 0x7fbffc069560 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0b5080 con 0x7fbffc159890 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.935+0000 7fc0043ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc16acc0 con 0x7fbffc159c60 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53524/0 (socket says 172.21.15.170:53524) 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 -- 172.21.15.170:0/2522664055 learned_addr learned my addr 172.21.15.170:0/2522664055 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc001987700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbffc159890 0x7fbffc16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002989700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbffc159c60 0x7fbffc0c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 -- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbffc159c60 msgr2=0x7fbffc0c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbffc159c60 0x7fbffc0c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 -- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbffc159890 msgr2=0x7fbffc16a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbffc159890 0x7fbffc16a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 -- 172.21.15.170:0/2522664055 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec009a20 con 0x7fbffc069560 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc002188700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbffc070e20 tx=0x7fbfec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec012e20 con 0x7fbffc069560 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec01d640 con 0x7fbffc069560 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc16afa0 con 0x7fbffc069560 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01da00 con 0x7fbffc069560 2023-10-08T16:04:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.936+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc0b4800 con 0x7fbffc069560 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.937+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc0b4b70 con 0x7fbffc069560 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.937+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfec024070 con 0x7fbffc069560 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.938+0000 7fbffa7fc700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 0x7fbfe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.938+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbfec086df0 con 0x7fbffc069560 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.938+0000 7fc001987700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 0x7fbfe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.938+0000 7fc001987700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 0x7fbfe80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0c7d90 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:57.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:57.942+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbffc0b4b70 con 0x7fbffc069560 2023-10-08T16:04:58.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.125+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbffc0b4b70 con 0x7fbffc069560 2023-10-08T16:04:58.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.125+0000 7fbffa7fc700 1 -- 172.21.15.170:0/2522664055 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbffc0b4b70 con 0x7fbffc069560 2023-10-08T16:04:58.126 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 msgr2=0x7fbfe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 0x7fbfe80693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0c7d90 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 msgr2=0x7fbffc156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbffc070e20 tx=0x7fbfec018000 comp rx=0 tx=0).stop 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 shutdown_connections 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbffc159c60 0x7fbffc0c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbfe8066f30 0x7fbfe80693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbffc069560 0x7fbffc156cf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 --2- 172.21.15.170:0/2522664055 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbffc159890 0x7fbffc16a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 >> 172.21.15.170:0/2522664055 conn(0x7fbffc068e20 msgr2=0x7fbffc158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 shutdown_connections 2023-10-08T16:04:58.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.127+0000 7fc0043ec700 1 -- 172.21.15.170:0/2522664055 wait complete. 2023-10-08T16:04:58.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:04:58.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.256+0000 7f8c8e8d4700 1 Processor -- start 2023-10-08T16:04:58.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.256+0000 7f8c8e8d4700 1 -- start start 2023-10-08T16:04:58.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e8d4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 0x7f8c880ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e8d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c8815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e8d4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880cfb00 con 0x7f8c880b16d0 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e8d4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880b5c40 con 0x7f8c880b4e70 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e8d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880b5f20 con 0x7f8c880b1b40 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e0d3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c8815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8d8d2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 0x7f8c880ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8d0d1700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f8c880b16d0 0x7f8c8815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58306/0 (socket says 172.21.15.170:58306) 2023-10-08T16:04:58.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8d0d1700 1 -- 172.21.15.170:0/3748113915 learned_addr learned my addr 172.21.15.170:0/3748113915 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e0d3700 1 -- 172.21.15.170:0/3748113915 >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 msgr2=0x7f8c880ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e0d3700 1 --2- 172.21.15.170:0/3748113915 >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 0x7f8c880ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e0d3700 1 -- 172.21.15.170:0/3748113915 >> v1:172.21.15.191:6789/0 conn(0x7f8c880b16d0 legacy=0x7f8c8815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8e0d3700 1 -- 172.21.15.170:0/3748113915 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880b6200 con 0x7f8c880b1b40 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.257+0000 7f8c8d8d2700 1 --2- 172.21.15.170:0/3748113915 >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 0x7f8c880ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e0d3700 1 --2- 172.21.15.170:0/3748113915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c8815d950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c840098e0 tx=0x7f8c840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4375e27ed337c34 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8c8d0700 1 -- 172.21.15.170:0/3748113915 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400b640 con 0x7f8c880b1b40 2023-10-08T16:04:58.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8c8d0700 1 -- 172.21.15.170:0/3748113915 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8400b7a0 con 0x7f8c880b1b40 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8c8d0700 1 -- 172.21.15.170:0/3748113915 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400bb40 con 0x7f8c880b1b40 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/3748113915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 msgr2=0x7f8c8815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/3748113915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c8815d950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8c840098e0 tx=0x7f8c840091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/3748113915 shutdown_connections 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/3748113915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c8815d950 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/3748113915 >> v2:172.21.15.191:3300/0 conn(0x7f8c880b4e70 0x7f8c880ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/3748113915 >> 172.21.15.170:0/3748113915 conn(0x7f8c880d10c0 msgr2=0x7f8c880affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/3748113915 shutdown_connections 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.258+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/3748113915 wait complete. 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.259+0000 7f8c8e8d4700 1 Processor -- start 2023-10-08T16:04:58.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 -- start start 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c880cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c880b4e70 0x7f8c88157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8814a660 con 0x7f8c880b16d0 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880cbba0 con 0x7f8c880b4e70 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e8d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c61e0 con 0x7f8c880b1b40 2023-10-08T16:04:58.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8d8d2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8d8d2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53536/0 (socket says 172.21.15.170:53536) 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8d8d2700 1 -- 172.21.15.170:0/596092571 learned_addr learned my addr 172.21.15.170:0/596092571 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8e0d3700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c880b4e70 0x7f8c88157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.260+0000 7f8c8d0d1700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c880cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 -- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 msgr2=0x7f8c880cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c880cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 -- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c880b4e70 msgr2=0x7f8c88157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c880b4e70 0x7f8c88157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 -- 172.21.15.170:0/596092571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009a20 con 0x7f8c880b16d0 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8d8d2700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c1a0 tx=0x7f8c8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000cce0 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8000f040 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80014690 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c84009420 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.261+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c880c5b90 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.262+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880c64c0 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.262+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c80014a00 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.263+0000 7f8c7dffb700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 0x7f8c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.263+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8c800852b0 con 0x7f8c880b16d0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.263+0000 7f8c8d0d1700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 0x7f8c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.263+0000 7f8c8d0d1700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 0x7f8c6c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c880b2af0 tx=0x7f8c78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.267+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c80052520 con 0x7f8c880b16d0 2023-10-08T16:04:58.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.450+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c8804bb80 con 0x7f8c880b16d0 2023-10-08T16:04:58.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.450+0000 7f8c7dffb700 1 -- 172.21.15.170:0/596092571 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c8005b030 con 0x7f8c880b16d0 2023-10-08T16:04:58.451 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 msgr2=0x7f8c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 0x7f8c6c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c880b2af0 tx=0x7f8c78009040 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 msgr2=0x7f8c8814a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c1a0 tx=0x7f8c8000b040 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 shutdown_connections 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8c880b4e70 0x7f8c88157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8c6c066f30 0x7f8c6c0693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8c880b16d0 0x7f8c8814a120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 --2- 172.21.15.170:0/596092571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8c880b1b40 0x7f8c880cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 >> 172.21.15.170:0/596092571 conn(0x7f8c880d10c0 msgr2=0x7f8c88158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 shutdown_connections 2023-10-08T16:04:58.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.452+0000 7f8c8e8d4700 1 -- 172.21.15.170:0/596092571 wait complete. 2023-10-08T16:04:58.464 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:04:58.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.581+0000 7fcfa8e8b700 1 Processor -- start 2023-10-08T16:04:58.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.581+0000 7fcfa8e8b700 1 -- start start 2023-10-08T16:04:58.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa8e8b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcfa406c0a0 0x7fcfa40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa8e8b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa415d7e0 con 0x7fcfa415df50 2023-10-08T16:04:58.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa8e8b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa415dac0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa8e8b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa415aa30 con 0x7fcfa406c0a0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa2d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcfa415df50 0x7fcfa40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54186/0 (socket says 172.21.15.170:54186) 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcfa406c0a0 0x7fcfa40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.582+0000 7fcfa2d9d700 1 -- 172.21.15.170:0/3873860444 learned_addr learned my addr 172.21.15.170:0/3873860444 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa1d9b700 1 -- 172.21.15.170:0/3873860444 >> v1:172.21.15.191:6789/0 conn(0x7fcfa406bcd0 legacy=0x7fcfa415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa1d9b700 1 -- 172.21.15.170:0/3873860444 >> v1:172.21.15.170:6789/0 conn(0x7fcfa415df50 legacy=0x7fcfa40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa159a700 1 -- 172.21.15.170:0/3873860444 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859981138 0 0) 0x7fcfa415d7e0 con 0x7fcfa415df50 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa1d9b700 1 -- 172.21.15.170:0/3873860444 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa415ad10 con 0x7fcfa406c0a0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa1d9b700 1 --2- 172.21.15.170:0/3873860444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406c0a0 0x7fcfa40caa90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf980098e0 tx=0x7fcf980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224036f042ad57ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa159a700 1 -- 172.21.15.170:0/3873860444 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf98009e80 con 0x7fcfa406c0a0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa159a700 1 -- 172.21.15.170:0/3873860444 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9800b860 con 0x7fcfa406c0a0 2023-10-08T16:04:58.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.583+0000 7fcfa159a700 1 -- 172.21.15.170:0/3873860444 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9800bbe0 con 0x7fcfa406c0a0 2023-10-08T16:04:58.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3873860444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406c0a0 msgr2=0x7fcfa40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3873860444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406c0a0 0x7fcfa40caa90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcf980098e0 tx=0x7fcf980091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3873860444 shutdown_connections 2023-10-08T16:04:58.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3873860444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406c0a0 0x7fcfa40caa90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3873860444 >> 172.21.15.170:0/3873860444 conn(0x7fcfa40ce600 msgr2=0x7fcfa40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3873860444 shutdown_connections 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.584+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3873860444 wait complete. 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 Processor -- start 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 -- start start 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 0x7fcfa416a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa415df50 0x7fcfa40c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.585+0000 7fcfa8e8b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa416b430 con 0x7fcfa406bcd0 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa8e8b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4158bc0 con 0x7fcfa415df50 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa8e8b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa416a910 con 0x7fcfa406c0a0 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53554/0 (socket says 172.21.15.170:53554) 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 -- 172.21.15.170:0/3517529291 learned_addr learned my addr 172.21.15.170:0/3517529291 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa2d9d700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa415df50 0x7fcfa40c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa1d9b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 0x7fcfa416a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 -- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 msgr2=0x7fcfa416a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 0x7fcfa416a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 -- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa415df50 msgr2=0x7fcfa40c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa415df50 0x7fcfa40c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 -- 172.21.15.170:0/3517529291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98009420 con 0x7fcfa406bcd0 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa1d9b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 0x7fcfa416a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa259c700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c910 tx=0x7fcf94001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400a7a0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9400a920 con 0x7fcfa406bcd0 2023-10-08T16:04:58.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa416ab90 con 0x7fcfa406bcd0 2023-10-08T16:04:58.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400ace0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.586+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa416b150 con 0x7fcfa406bcd0 2023-10-08T16:04:58.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.587+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa40c99e0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.587+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf94003030 con 0x7fcfa406bcd0 2023-10-08T16:04:58.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.588+0000 7fcf927fc700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 0x7fcf840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.588+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf94084550 con 0x7fcfa406bcd0 2023-10-08T16:04:58.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.588+0000 7fcfa1d9b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 0x7fcf840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.588+0000 7fcfa1d9b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 0x7fcf840693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b44a0 tx=0x7fcf98017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.592+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf940517c0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.773+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcfa4158990 con 0x7fcfa406bcd0 2023-10-08T16:04:58.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.774+0000 7fcf927fc700 1 -- 172.21.15.170:0/3517529291 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf94054de0 con 0x7fcfa406bcd0 2023-10-08T16:04:58.774 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:04:58.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 msgr2=0x7fcf840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 0x7fcf840693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcfa40b44a0 tx=0x7fcf98017000 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 msgr2=0x7fcfa40c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c910 tx=0x7fcf94001710 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 shutdown_connections 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcfa415df50 0x7fcfa40c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf84066f30 0x7fcf840693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfa406bcd0 0x7fcfa40c8a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 --2- 172.21.15.170:0/3517529291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcfa406c0a0 0x7fcfa416a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 >> 172.21.15.170:0/3517529291 conn(0x7fcfa40ce600 msgr2=0x7fcfa406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 shutdown_connections 2023-10-08T16:04:58.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.776+0000 7fcfa8e8b700 1 -- 172.21.15.170:0/3517529291 wait complete. 2023-10-08T16:04:58.788 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:04:58.902 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.902+0000 7f9e6a0de700 1 Processor -- start 2023-10-08T16:04:58.902 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.902+0000 7f9e6a0de700 1 -- start start 2023-10-08T16:04:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e6a0de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e6415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e6a0de700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9e64159c60 0x7f9e6415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e6a0de700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6415a160 con 0x7f9e6415db10 2023-10-08T16:04:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e6a0de700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e640d13a0 con 0x7f9e64159c60 2023-10-08T16:04:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e6a0de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6415e140 con 0x7f9e64159890 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e63fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9e6415db10 0x7f9e64069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54200/0 (socket says 172.21.15.170:54200) 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e63fff700 1 -- 172.21.15.170:0/1814679869 learned_addr learned my addr 172.21.15.170:0/1814679869 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625358216 0 0) 0x7f9e6415a160 con 0x7f9e6415db10 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> v2:172.21.15.191:3300/0 conn(0x7f9e64159c60 0x7f9e6415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e637fe700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e6415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4c003570 con 0x7f9e6415db10 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915386118 0 0) 0x7f9e4c003570 con 0x7f9e6415db10 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 msgr2=0x7f9e6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e6415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 >> v2:172.21.15.191:3300/0 conn(0x7f9e64159c60 msgr2=0x7f9e6415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 --2- 172.21.15.170:0/1814679869 >> v2:172.21.15.191:3300/0 conn(0x7f9e64159c60 0x7f9e6415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6415a3d0 con 0x7f9e6415db10 2023-10-08T16:04:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f9e5c0017f0 con 0x7f9e6415db10 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> v2:172.21.15.191:3300/0 conn(0x7f9e64159c60 0x7f9e6415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 >> v1:172.21.15.170:6789/0 conn(0x7f9e6415db10 legacy=0x7f9e64069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e4c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e4c003fe0 0x7f9e4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e4c004d50 0x7f9e4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640d13a0 con 0x7f9e4c003fe0 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6415e140 con 0x7f9e4c004d50 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00bb40 con 0x7f9e64159890 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.903+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e4c003fe0 0x7f9e4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e637fe700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e4c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e63fff700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e4c004d50 0x7f9e4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 -- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 msgr2=0x7f9e4c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e64159890 0x7f9e4c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 -- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e4c004d50 msgr2=0x7f9e4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e4c004d50 0x7f9e4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 -- 172.21.15.170:0/1814679869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c003570 con 0x7f9e4c003fe0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e4c003fe0 0x7f9e4c004810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ae709d460f3c20 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800c070 con 0x7f9e4c003fe0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5800e5b0 con 0x7f9e4c003fe0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e627fc700 1 -- 172.21.15.170:0/1814679869 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58009510 con 0x7f9e4c003fe0 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e6a0de700 1 -- 172.21.15.170:0/1814679869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e4c003fe0 msgr2=0x7f9e4c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/1814679869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e4c003fe0 0x7f9e4c004810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e58006fd0 tx=0x7f9e58008040 comp rx=0 tx=0).stop 2023-10-08T16:04:58.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e63fff700 1 -- 172.21.15.170:0/1814679869 reap_dead start 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.904+0000 7f9e6a0de700 1 -- 172.21.15.170:0/1814679869 shutdown_connections 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.905+0000 7f9e6a0de700 1 -- 172.21.15.170:0/1814679869 >> 172.21.15.170:0/1814679869 conn(0x7f9e64068e20 msgr2=0x7f9e640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.905+0000 7f9e6a0de700 1 -- 172.21.15.170:0/1814679869 shutdown_connections 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.905+0000 7f9e6a0de700 1 -- 172.21.15.170:0/1814679869 wait complete. 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 Processor -- start 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 -- start start 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e4c003fe0 0x7f9e6414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 0x7f9e64157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b4170 con 0x7f9e640b3590 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640b3f40 con 0x7f9e64156ca0 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.906+0000 7f9e6a0de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64157650 con 0x7f9e4c003fe0 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53570/0 (socket says 172.21.15.170:53570) 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 -- 172.21.15.170:0/3308268283 learned_addr learned my addr 172.21.15.170:0/3308268283 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e637fe700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e4c003fe0 0x7f9e6414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e63fff700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 0x7f9e64157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 -- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e4c003fe0 msgr2=0x7f9e6414d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e4c003fe0 0x7f9e6414d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 -- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 msgr2=0x7f9e64157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 0x7f9e64157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 -- 172.21.15.170:0/3308268283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e640b1760 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e63fff700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 0x7f9e64157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e62ffd700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e520 tx=0x7f9e5801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800eed0 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58013ca0 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5801c690 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e640b0e30 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.907+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e640b13f0 con 0x7f9e640b3590 2023-10-08T16:04:58.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.908+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e640b6420 con 0x7f9e640b3590 2023-10-08T16:04:58.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.908+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e5801cac0 con 0x7f9e640b3590 2023-10-08T16:04:58.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.909+0000 7f9e4b7fe700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 0x7f9e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:58.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.909+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9e58086080 con 0x7f9e640b3590 2023-10-08T16:04:58.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.909+0000 7f9e637fe700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 0x7f9e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:58.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.909+0000 7f9e637fe700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 0x7f9e500693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e540093f0 tx=0x7f9e54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:58.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:58.913+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e580532f0 con 0x7f9e640b3590 2023-10-08T16:04:59.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.098+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e640b18f0 con 0x7f9e640b3590 2023-10-08T16:04:59.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.099+0000 7f9e4b7fe700 1 -- 172.21.15.170:0/3308268283 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f9e58056910 con 0x7f9e640b3590 2023-10-08T16:04:59.099 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:04:59.099 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:04:59.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 msgr2=0x7f9e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 0x7f9e500693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e540093f0 tx=0x7f9e54008040 comp rx=0 tx=0).stop 2023-10-08T16:04:59.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 msgr2=0x7f9e640b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800e520 tx=0x7f9e5801a000 comp rx=0 tx=0).stop 2023-10-08T16:04:59.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 shutdown_connections 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e64156ca0 0x7f9e64157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e4c003fe0 0x7f9e6414d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e50066f30 0x7f9e500693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 --2- 172.21.15.170:0/3308268283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e640b3590 0x7f9e640b3a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 >> 172.21.15.170:0/3308268283 conn(0x7f9e64068e20 msgr2=0x7f9e640c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 shutdown_connections 2023-10-08T16:04:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.101+0000 7f9e6a0de700 1 -- 172.21.15.170:0/3308268283 wait complete. 2023-10-08T16:04:59.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:04:59.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:04:59.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.230+0000 7ffba2698700 1 Processor -- start 2023-10-08T16:04:59.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.230+0000 7ffba2698700 1 -- start start 2023-10-08T16:04:59.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffb9c159890 0x7ffb9c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c159c60 0x7ffb9c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffb9c15db10 0x7ffb9c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c15a160 con 0x7ffb9c159890 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9c0d13a0 con 0x7ffb9c15db10 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffba2698700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c15e140 con 0x7ffb9c159c60 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffb9bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffb9c159890 0x7ffb9c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffb9bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffb9c159890 0x7ffb9c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53582/0 (socket says 172.21.15.170:53582) 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.231+0000 7ffb9bfff700 1 -- 172.21.15.170:0/3621473594 learned_addr learned my addr 172.21.15.170:0/3621473594 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb93fff700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c159c60 0x7ffb9c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffba0c35700 1 --2- 172.21.15.170:0/3621473594 >> v2:172.21.15.191:3300/0 conn(0x7ffb9c15db10 0x7ffb9c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 -- 172.21.15.170:0/3621473594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c159c60 msgr2=0x7ffb9c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:59.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c159c60 0x7ffb9c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 -- 172.21.15.170:0/3621473594 >> v2:172.21.15.191:3300/0 conn(0x7ffb9c15db10 msgr2=0x7ffb9c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3621473594 >> v2:172.21.15.191:3300/0 conn(0x7ffb9c15db10 0x7ffb9c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 -- 172.21.15.170:0/3621473594 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9c15a3d0 con 0x7ffb9c159890 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159890 0x7ffb9c15abd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c089b50 tx=0x7ffb8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ba6c9fd97b0534 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9b7fe700 1 -- 172.21.15.170:0/3621473594 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00b860 con 0x7ffb9c159890 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9b7fe700 1 -- 172.21.15.170:0/3621473594 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c00b9c0 con 0x7ffb9c159890 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.232+0000 7ffb9b7fe700 1 -- 172.21.15.170:0/3621473594 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c00bd60 con 0x7ffb9c159890 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 -- 172.21.15.170:0/3621473594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159890 msgr2=0x7ffb9c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159890 0x7ffb9c15abd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c089b50 tx=0x7ffb8c009250 comp rx=0 tx=0).stop 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 -- 172.21.15.170:0/3621473594 shutdown_connections 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 --2- 172.21.15.170:0/3621473594 >> v2:172.21.15.191:3300/0 conn(0x7ffb9c15db10 0x7ffb9c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159890 0x7ffb9c15abd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 --2- 172.21.15.170:0/3621473594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c159c60 0x7ffb9c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 -- 172.21.15.170:0/3621473594 >> 172.21.15.170:0/3621473594 conn(0x7ffb9c068e20 msgr2=0x7ffb9c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 -- 172.21.15.170:0/3621473594 shutdown_connections 2023-10-08T16:04:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.233+0000 7ffba2698700 1 -- 172.21.15.170:0/3621473594 wait complete. 2023-10-08T16:04:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 Processor -- start 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 -- start start 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb9c159890 0x7ffb9c0b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 0x7ffb9c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c169c20 con 0x7ffb9c159c60 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c16e100 con 0x7ffb9c159890 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.234+0000 7ffba2698700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9c0b5940 con 0x7ffb9c15db10 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53590/0 (socket says 172.21.15.170:53590) 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 -- 172.21.15.170:0/3325075600 learned_addr learned my addr 172.21.15.170:0/3325075600 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb9c159890 0x7ffb9c0b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffba0c35700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 0x7ffb9c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 -- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 msgr2=0x7ffb9c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 0x7ffb9c15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 -- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb9c159890 msgr2=0x7ffb9c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb9c159890 0x7ffb9c0b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 -- 172.21.15.170:0/3325075600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0094f0 con 0x7ffb9c159c60 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffba0c35700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 0x7ffb9c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb93fff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d890 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb840125f0 con 0x7ffb9c159c60 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb84012750 con 0x7ffb9c159c60 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9c16d210 con 0x7ffb9c159c60 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84012af0 con 0x7ffb9c159c60 2023-10-08T16:04:59.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.235+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9c16d7d0 con 0x7ffb9c159c60 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.236+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9c04c3c0 con 0x7ffb9c159c60 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.236+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffb8401e460 con 0x7ffb9c159c60 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.237+0000 7ffb98ff9700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 0x7ffb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.237+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffb84084b20 con 0x7ffb9c159c60 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.237+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 0x7ffb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.237+0000 7ffb9bfff700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 0x7ffb7c0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c089b50 tx=0x7ffb8c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.241+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb84051d90 con 0x7ffb9c159c60 2023-10-08T16:04:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.421+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb9c06a5b0 con 0x7ffb9c159c60 2023-10-08T16:04:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.421+0000 7ffb98ff9700 1 -- 172.21.15.170:0/3325075600 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb840553b0 con 0x7ffb9c159c60 2023-10-08T16:04:59.422 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 msgr2=0x7ffb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 0x7ffb7c0693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c089b50 tx=0x7ffb8c017000 comp rx=0 tx=0).stop 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 msgr2=0x7ffb9c147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400d890 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 shutdown_connections 2023-10-08T16:04:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffb9c15db10 0x7ffb9c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffb9c159890 0x7ffb9c0b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffb7c066f30 0x7ffb7c0693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 --2- 172.21.15.170:0/3325075600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffb9c159c60 0x7ffb9c147160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 >> 172.21.15.170:0/3325075600 conn(0x7ffb9c068e20 msgr2=0x7ffb9c152ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 shutdown_connections 2023-10-08T16:04:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.423+0000 7ffba2698700 1 -- 172.21.15.170:0/3325075600 wait complete. 2023-10-08T16:04:59.436 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:04:59.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.553+0000 7f2c42f65700 1 Processor -- start 2023-10-08T16:04:59.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.553+0000 7f2c42f65700 1 -- start start 2023-10-08T16:04:59.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42f65700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 0x7f2c3c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42f65700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42f65700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c0af490 con 0x7f2c3c0cb400 2023-10-08T16:04:59.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42f65700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c0af770 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42f65700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0afa50 con 0x7f2c3c0c9770 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c41f63700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 0x7f2c3c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c41762700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42764700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f2c3c0cb400 0x7f2c3c0c7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58322/0 (socket says 172.21.15.170:58322) 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.554+0000 7f2c42764700 1 -- 172.21.15.170:0/1166253181 learned_addr learned my addr 172.21.15.170:0/1166253181 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c40f61700 1 -- 172.21.15.170:0/1166253181 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278477742 0 0) 0x7f2c3c0af490 con 0x7f2c3c0cb400 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c40f61700 1 -- 172.21.15.170:0/1166253181 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c24003570 con 0x7f2c3c0cb400 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41762700 1 -- 172.21.15.170:0/1166253181 >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 msgr2=0x7f2c3c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41762700 1 --2- 172.21.15.170:0/1166253181 >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 0x7f2c3c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41762700 1 -- 172.21.15.170:0/1166253181 >> v1:172.21.15.191:6789/0 conn(0x7f2c3c0cb400 legacy=0x7f2c3c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41762700 1 -- 172.21.15.170:0/1166253181 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0c7ab0 con 0x7f2c3c0c9770 2023-10-08T16:04:59.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41f63700 1 --2- 172.21.15.170:0/1166253181 >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 0x7f2c3c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c41762700 1 --2- 172.21.15.170:0/1166253181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0caec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c300098d0 tx=0x7f2c30009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec53aaf720b7881 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c40f61700 1 -- 172.21.15.170:0/1166253181 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30005c10 con 0x7f2c3c0c9770 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c40f61700 1 -- 172.21.15.170:0/1166253181 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30005d70 con 0x7f2c3c0c9770 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.555+0000 7f2c40f61700 1 -- 172.21.15.170:0/1166253181 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30006260 con 0x7f2c3c0c9770 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 -- 172.21.15.170:0/1166253181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 msgr2=0x7f2c3c0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 --2- 172.21.15.170:0/1166253181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0caec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2c300098d0 tx=0x7f2c30009120 comp rx=0 tx=0).stop 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 -- 172.21.15.170:0/1166253181 shutdown_connections 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 --2- 172.21.15.170:0/1166253181 >> v2:172.21.15.191:3300/0 conn(0x7f2c3c0c93a0 0x7f2c3c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 --2- 172.21.15.170:0/1166253181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0caec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 -- 172.21.15.170:0/1166253181 >> 172.21.15.170:0/1166253181 conn(0x7f2c3c0cddc0 msgr2=0x7f2c3c0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 -- 172.21.15.170:0/1166253181 shutdown_connections 2023-10-08T16:04:59.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.556+0000 7f2c42f65700 1 -- 172.21.15.170:0/1166253181 wait complete. 2023-10-08T16:04:59.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.557+0000 7f2c42f65700 1 Processor -- start 2023-10-08T16:04:59.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.557+0000 7f2c42f65700 1 -- start start 2023-10-08T16:04:59.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 0x7f2c3c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c14d8f0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0c4db0 con 0x7f2c3c0cb400 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0c5370 con 0x7f2c3c0c9770 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53602/0 (socket says 172.21.15.170:53602) 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 -- 172.21.15.170:0/122096199 learned_addr learned my addr 172.21.15.170:0/122096199 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42764700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 0x7f2c3c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41762700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 -- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 msgr2=0x7f2c3c0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 -- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 msgr2=0x7f2c3c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 0x7f2c3c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 -- 172.21.15.170:0/122096199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c009420 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42764700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 0x7f2c3c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c41f63700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00ecb0 tx=0x7f2c2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c009780 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00cbe0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c300093c0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.558+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c013470 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.559+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c0b0dd0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.560+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3c06d560 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.560+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c2c013820 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.560+0000 7f2c39ffb700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 0x7f2c280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.560+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c2c084930 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.560+0000 7f2c41762700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 0x7f2c280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.561+0000 7f2c41762700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 0x7f2c280693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009160 tx=0x7f2c30016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.564+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c2c051ba0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.746+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c3c0cc760 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.746+0000 7f2c39ffb700 1 -- 172.21.15.170:0/122096199 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c2c0551c0 con 0x7f2c3c0c93a0 2023-10-08T16:04:59.747 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:04:59.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.748+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 msgr2=0x7f2c280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.748+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 0x7f2c280693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2c30009160 tx=0x7f2c30016000 comp rx=0 tx=0).stop 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 msgr2=0x7f2c3c160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c00ecb0 tx=0x7f2c2c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 shutdown_connections 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c3c0cb400 0x7f2c3c0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c28066f30 0x7f2c280693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c3c0c93a0 0x7f2c3c160eb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 --2- 172.21.15.170:0/122096199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c3c0c9770 0x7f2c3c0b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 >> 172.21.15.170:0/122096199 conn(0x7f2c3c0cddc0 msgr2=0x7f2c3c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 shutdown_connections 2023-10-08T16:04:59.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.749+0000 7f2c42f65700 1 -- 172.21.15.170:0/122096199 wait complete. 2023-10-08T16:04:59.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:04:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.879+0000 7f4737b58700 1 Processor -- start 2023-10-08T16:04:59.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.879+0000 7f4737b58700 1 -- start start 2023-10-08T16:04:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f4737b58700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4730159890 0x7f473015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f4737b58700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4730159c60 0x7f473015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f4737b58700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473015a160 con 0x7f473015db10 2023-10-08T16:04:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f4737b58700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473015df50 con 0x7f4730159890 2023-10-08T16:04:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f4737b58700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47300d13a0 con 0x7f4730159c60 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f47358f4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4730159890 0x7f473015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f47358f4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4730159890 0x7f473015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53608/0 (socket says 172.21.15.170:53608) 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.880+0000 7f47358f4700 1 -- 172.21.15.170:0/1378728010 learned_addr learned my addr 172.21.15.170:0/1378728010 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47350f3700 1 --2- 172.21.15.170:0/1378728010 >> v2:172.21.15.191:3300/0 conn(0x7f4730159c60 0x7f473015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47358f4700 1 -- 172.21.15.170:0/1378728010 >> v2:172.21.15.191:3300/0 conn(0x7f4730159c60 msgr2=0x7f473015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47358f4700 1 --2- 172.21.15.170:0/1378728010 >> v2:172.21.15.191:3300/0 conn(0x7f4730159c60 0x7f473015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47358f4700 1 -- 172.21.15.170:0/1378728010 >> v1:172.21.15.191:6789/0 conn(0x7f473015db10 legacy=0x7f4730069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47358f4700 1 -- 172.21.15.170:0/1378728010 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473015e140 con 0x7f4730159890 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47358f4700 1 --2- 172.21.15.170:0/1378728010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f473015abd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47300b67f0 tx=0x7f4720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9768a47451d199 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47348f2700 1 -- 172.21.15.170:0/1378728010 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472000e070 con 0x7f4730159890 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47348f2700 1 -- 172.21.15.170:0/1378728010 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472000ba20 con 0x7f4730159890 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.881+0000 7f47348f2700 1 -- 172.21.15.170:0/1378728010 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47200092e0 con 0x7f4730159890 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 -- 172.21.15.170:0/1378728010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 msgr2=0x7f473015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 --2- 172.21.15.170:0/1378728010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f473015abd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f47300b67f0 tx=0x7f4720009840 comp rx=0 tx=0).stop 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 -- 172.21.15.170:0/1378728010 shutdown_connections 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 --2- 172.21.15.170:0/1378728010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f473015abd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 --2- 172.21.15.170:0/1378728010 >> v2:172.21.15.191:3300/0 conn(0x7f4730159c60 0x7f473015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 -- 172.21.15.170:0/1378728010 >> 172.21.15.170:0/1378728010 conn(0x7f4730068e20 msgr2=0x7f47300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 -- 172.21.15.170:0/1378728010 shutdown_connections 2023-10-08T16:04:59.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.882+0000 7f4737b58700 1 -- 172.21.15.170:0/1378728010 wait complete. 2023-10-08T16:04:59.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 Processor -- start 2023-10-08T16:04:59.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 -- start start 2023-10-08T16:04:59.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 0x7f473016b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f473015db10 0x7f47300b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300ceb60 con 0x7f4730159890 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300ce230 con 0x7f4730159c60 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f4737b58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300ce510 con 0x7f473015db10 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.883+0000 7f47358f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53610/0 (socket says 172.21.15.170:53610) 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 -- 172.21.15.170:0/3659647898 learned_addr learned my addr 172.21.15.170:0/3659647898 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47350f3700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 0x7f473016b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47360f5700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f473015db10 0x7f47300b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 -- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f473015db10 msgr2=0x7f47300b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f473015db10 0x7f47300b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 -- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 msgr2=0x7f473016b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 0x7f473016b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 -- 172.21.15.170:0/3659647898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4720009ae0 con 0x7f4730159890 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47350f3700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 0x7f473016b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:04:59.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f47358f4700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4730070e20 tx=0x7f4720018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720008a80 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4720008c00 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47300ce7f0 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720026460 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.884+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300b3ad0 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.885+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473004c3c0 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.885+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4720003040 con 0x7f4730159890 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.886+0000 7f472dffb700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 0x7f471c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:04:59.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.886+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4720086030 con 0x7f4730159890 2023-10-08T16:04:59.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.886+0000 7f47350f3700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 0x7f471c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:04:59.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.886+0000 7f47350f3700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 0x7f471c0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4730089650 tx=0x7f4724008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:04:59.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:04:59.890+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47200532a0 con 0x7f4730159890 2023-10-08T16:05:00.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.072+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4730152950 con 0x7f4730159890 2023-10-08T16:05:00.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.073+0000 7f472dffb700 1 -- 172.21.15.170:0/3659647898 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47200568c0 con 0x7f4730159890 2023-10-08T16:05:00.073 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:05:00.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 msgr2=0x7f471c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 0x7f471c0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4730089650 tx=0x7f4724008040 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 msgr2=0x7f47300b60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4730070e20 tx=0x7f4720018000 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 shutdown_connections 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f473015db10 0x7f47300b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f471c066f30 0x7f471c0693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4730159890 0x7f47300b60a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 --2- 172.21.15.170:0/3659647898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4730159c60 0x7f473016b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 >> 172.21.15.170:0/3659647898 conn(0x7f4730068e20 msgr2=0x7f47300b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 shutdown_connections 2023-10-08T16:05:00.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.075+0000 7f4737b58700 1 -- 172.21.15.170:0/3659647898 wait complete. 2023-10-08T16:05:00.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:00.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 Processor -- start 2023-10-08T16:05:00.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 -- start start 2023-10-08T16:05:00.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f34ac06bcd0 0x7f34ac140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac15c2a0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac15b690 con 0x7f34ac0c9510 2023-10-08T16:05:00.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.202+0000 7f34b12f1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac0c9900 con 0x7f34ac06bcd0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f34ac06bcd0 0x7f34ac140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aa7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f34ac06c0a0 0x7f34ac0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54216/0 (socket says 172.21.15.170:54216) 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aa7fc700 1 -- 172.21.15.170:0/3810486401 learned_addr learned my addr 172.21.15.170:0/3810486401 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aaffd700 1 -- 172.21.15.170:0/3810486401 >> v1:172.21.15.191:6789/0 conn(0x7f34ac0c9510 legacy=0x7f34ac0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aaffd700 1 -- 172.21.15.170:0/3810486401 >> v1:172.21.15.170:6789/0 conn(0x7f34ac06c0a0 legacy=0x7f34ac0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aaffd700 1 -- 172.21.15.170:0/3810486401 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac15b970 con 0x7f34ac06bcd0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34a9ffb700 1 -- 172.21.15.170:0/3810486401 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535136532 0 0) 0x7f34ac15c2a0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34aaffd700 1 --2- 172.21.15.170:0/3810486401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06bcd0 0x7f34ac140df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098e0 tx=0x7f349c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cd46e2769d2289 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34a9ffb700 1 -- 172.21.15.170:0/3810486401 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c009e80 con 0x7f34ac06bcd0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34a9ffb700 1 -- 172.21.15.170:0/3810486401 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c00b860 con 0x7f34ac06bcd0 2023-10-08T16:05:00.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.203+0000 7f34a9ffb700 1 -- 172.21.15.170:0/3810486401 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c00bbe0 con 0x7f34ac06bcd0 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 -- 172.21.15.170:0/3810486401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06bcd0 msgr2=0x7f34ac140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3810486401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06bcd0 0x7f34ac140df0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098e0 tx=0x7f349c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 -- 172.21.15.170:0/3810486401 shutdown_connections 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3810486401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06bcd0 0x7f34ac140df0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 -- 172.21.15.170:0/3810486401 >> 172.21.15.170:0/3810486401 conn(0x7f34ac0ce600 msgr2=0x7f34ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 -- 172.21.15.170:0/3810486401 shutdown_connections 2023-10-08T16:05:00.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.204+0000 7f34b12f1700 1 -- 172.21.15.170:0/3810486401 wait complete. 2023-10-08T16:05:00.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.205+0000 7f34b12f1700 1 Processor -- start 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.205+0000 7f34b12f1700 1 -- start start 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 0x7f34ac1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ac0c9510 0x7f34ac0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac13d270 con 0x7f34ac06c0a0 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0c6070 con 0x7f34ac0c9510 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34b12f1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0c6350 con 0x7f34ac06bcd0 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53630/0 (socket says 172.21.15.170:53630) 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 -- 172.21.15.170:0/3179524821 learned_addr learned my addr 172.21.15.170:0/3179524821 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34ab7fe700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ac0c9510 0x7f34ac0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aaffd700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 0x7f34ac1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 -- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 msgr2=0x7f34ac1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 0x7f34ac1568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 -- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ac0c9510 msgr2=0x7f34ac0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ac0c9510 0x7f34ac0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 -- 172.21.15.170:0/3179524821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c009420 con 0x7f34ac06c0a0 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aaffd700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 0x7f34ac1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.206+0000 7f34aa7fc700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c7e0 tx=0x7f34a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.207+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0005b30 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.207+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a0005c90 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.207+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac0c6630 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.207+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a0007300 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.207+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac0c74a0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.208+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac0c76f0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.208+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34a001b400 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.208+0000 7f3492ffd700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 0x7f3498069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.208+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f34a0083fd0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.208+0000 7f34aaffd700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 0x7f3498069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.209+0000 7f34aaffd700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 0x7f3498069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098e0 tx=0x7f349c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.213+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34a00512c0 con 0x7f34ac06c0a0 2023-10-08T16:05:00.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.399+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34ac13d040 con 0x7f34ac06c0a0 2023-10-08T16:05:00.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.400+0000 7f3492ffd700 1 -- 172.21.15.170:0/3179524821 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f34ac13d040 con 0x7f34ac06c0a0 2023-10-08T16:05:00.400 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:00.400 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 msgr2=0x7f3498069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 0x7f3498069410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098e0 tx=0x7f349c016000 comp rx=0 tx=0).stop 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 msgr2=0x7f34ac0c56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34a000c7e0 tx=0x7f34a0007df0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 shutdown_connections 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34ac0c9510 0x7f34ac0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34ac06bcd0 0x7f34ac1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3498066f60 0x7f3498069410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 --2- 172.21.15.170:0/3179524821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34ac06c0a0 0x7f34ac0c56b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 >> 172.21.15.170:0/3179524821 conn(0x7f34ac0ce600 msgr2=0x7f34ac1577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 shutdown_connections 2023-10-08T16:05:00.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.402+0000 7f34b12f1700 1 -- 172.21.15.170:0/3179524821 wait complete. 2023-10-08T16:05:00.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:05:00.415 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 Processor -- start 2023-10-08T16:05:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 -- start start 2023-10-08T16:05:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1004159c60 0x7f100415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100415a160 con 0x7f1004159890 2023-10-08T16:05:00.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040d13a0 con 0x7f100415db10 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.532+0000 7f100aa22700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100415df50 con 0x7f1004159c60 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f10037fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1004159c60 0x7f100415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1003fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1004159890 0x7f100415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54222/0 (socket says 172.21.15.170:54222) 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1003fff700 1 -- 172.21.15.170:0/195086655 learned_addr learned my addr 172.21.15.170:0/195086655 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f10037fe700 1 -- 172.21.15.170:0/195086655 >> v1:172.21.15.191:6789/0 conn(0x7f100415db10 legacy=0x7f1004069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f10037fe700 1 -- 172.21.15.170:0/195086655 >> v1:172.21.15.170:6789/0 conn(0x7f1004159890 legacy=0x7f100415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1002ffd700 1 -- 172.21.15.170:0/195086655 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221920268 0 0) 0x7f100415a160 con 0x7f1004159890 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f10037fe700 1 -- 172.21.15.170:0/195086655 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100415e140 con 0x7f1004159c60 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f10037fe700 1 --2- 172.21.15.170:0/195086655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159c60 0x7f100415d5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40098e0 tx=0x7f0ff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5373bcf05cee21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1002ffd700 1 -- 172.21.15.170:0/195086655 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4009e80 con 0x7f1004159c60 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1002ffd700 1 -- 172.21.15.170:0/195086655 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff400b820 con 0x7f1004159c60 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.533+0000 7f1002ffd700 1 -- 172.21.15.170:0/195086655 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff400bba0 con 0x7f1004159c60 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 -- 172.21.15.170:0/195086655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159c60 msgr2=0x7f100415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 --2- 172.21.15.170:0/195086655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159c60 0x7f100415d5d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40098e0 tx=0x7f0ff40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 -- 172.21.15.170:0/195086655 shutdown_connections 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 --2- 172.21.15.170:0/195086655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159c60 0x7f100415d5d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 -- 172.21.15.170:0/195086655 >> 172.21.15.170:0/195086655 conn(0x7f1004068e20 msgr2=0x7f10040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 -- 172.21.15.170:0/195086655 shutdown_connections 2023-10-08T16:05:00.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.534+0000 7f100aa22700 1 -- 172.21.15.170:0/195086655 wait complete. 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.535+0000 7f100aa22700 1 Processor -- start 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.535+0000 7f100aa22700 1 -- start start 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1004159c60 0x7f10040cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f100415db10 0x7f10041584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040ce610 con 0x7f1004159890 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040cda00 con 0x7f100415db10 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040cdce0 con 0x7f1004159c60 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53640/0 (socket says 172.21.15.170:53640) 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 -- 172.21.15.170:0/62705835 learned_addr learned my addr 172.21.15.170:0/62705835 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 -- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1004159c60 msgr2=0x7f10040cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1004159c60 0x7f10040cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 -- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f100415db10 msgr2=0x7f10041584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f100415db10 0x7f10041584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 -- 172.21.15.170:0/62705835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4009420 con 0x7f1004159890 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f1003fff700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c5b0 tx=0x7f0ff8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8005b30 con 0x7f1004159890 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8005c90 con 0x7f1004159890 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040cdfc0 con 0x7f1004159890 2023-10-08T16:05:00.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8007320 con 0x7f1004159890 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.536+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004150a00 con 0x7f1004159890 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.537+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100404c3c0 con 0x7f1004159890 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.538+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ff801b3b0 con 0x7f1004159890 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.538+0000 7f0febfff700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.538+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0ff8083a00 con 0x7f1004159890 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.538+0000 7f10037fe700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.539+0000 7f10037fe700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 0x7f0ff00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1004159210 tx=0x7f0ff4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.542+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ff8050c70 con 0x7f1004159890 2023-10-08T16:05:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.725+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f100406a5b0 con 0x7f1004159890 2023-10-08T16:05:00.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.725+0000 7f0febfff700 1 -- 172.21.15.170:0/62705835 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ff80170a0 con 0x7f1004159890 2023-10-08T16:05:00.726 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:00.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 msgr2=0x7f0ff00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 0x7f0ff00693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1004159210 tx=0x7f0ff4016000 comp rx=0 tx=0).stop 2023-10-08T16:05:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 msgr2=0x7f10041504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c5b0 tx=0x7f0ff8001070 comp rx=0 tx=0).stop 2023-10-08T16:05:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 shutdown_connections 2023-10-08T16:05:00.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f100415db10 0x7f10041584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ff0066f30 0x7f0ff00693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1004159890 0x7f10041504c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 --2- 172.21.15.170:0/62705835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1004159c60 0x7f10040cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 >> 172.21.15.170:0/62705835 conn(0x7f1004068e20 msgr2=0x7f1004153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 shutdown_connections 2023-10-08T16:05:00.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.727+0000 7f100aa22700 1 -- 172.21.15.170:0/62705835 wait complete. 2023-10-08T16:05:00.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:05:00.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.858+0000 7f7cc579a700 1 Processor -- start 2023-10-08T16:05:00.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.858+0000 7f7cc579a700 1 -- start start 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cc579a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7cc00ad5c0 0x7f7cc006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cc579a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc01570b0 con 0x7f7cc006e9c0 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cc579a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc015a3f0 con 0x7f7cc0156c70 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cc579a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc006ee10 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbeffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7cc00ad5c0 0x7f7cc006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbe7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7cc006e9c0 0x7f7cc0156560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54236/0 (socket says 172.21.15.170:54236) 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbe7fc700 1 -- 172.21.15.170:0/1571336474 learned_addr learned my addr 172.21.15.170:0/1571336474 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbeffd700 1 -- 172.21.15.170:0/1571336474 >> v1:172.21.15.191:6789/0 conn(0x7f7cc0156c70 legacy=0x7f7cc00c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbeffd700 1 -- 172.21.15.170:0/1571336474 >> v1:172.21.15.170:6789/0 conn(0x7f7cc006e9c0 legacy=0x7f7cc0156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbeffd700 1 -- 172.21.15.170:0/1571336474 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc0159820 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbdffb700 1 -- 172.21.15.170:0/1571336474 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300920981 0 0) 0x7f7cb4001110 con 0x7f7cc006e9c0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbeffd700 1 --2- 172.21.15.170:0/1571336474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc006e480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d0d0 tx=0x7f7cb800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5e182e6d9536ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbdffb700 1 -- 172.21.15.170:0/1571336474 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8010070 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbdffb700 1 -- 172.21.15.170:0/1571336474 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb800ddf0 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.859+0000 7f7cbdffb700 1 -- 172.21.15.170:0/1571336474 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb800b320 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 -- 172.21.15.170:0/1571336474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc00ad5c0 msgr2=0x7f7cc006e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 --2- 172.21.15.170:0/1571336474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc006e480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d0d0 tx=0x7f7cb800b990 comp rx=0 tx=0).stop 2023-10-08T16:05:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 -- 172.21.15.170:0/1571336474 shutdown_connections 2023-10-08T16:05:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 --2- 172.21.15.170:0/1571336474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc006e480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 -- 172.21.15.170:0/1571336474 >> 172.21.15.170:0/1571336474 conn(0x7f7cc00c9fb0 msgr2=0x7f7cc006dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 -- 172.21.15.170:0/1571336474 shutdown_connections 2023-10-08T16:05:00.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.860+0000 7f7cc579a700 1 -- 172.21.15.170:0/1571336474 wait complete. 2023-10-08T16:05:00.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.861+0000 7f7cc579a700 1 Processor -- start 2023-10-08T16:05:00.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.861+0000 7f7cc579a700 1 -- start start 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc0158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cc0156c70 0x7f7cc01499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0159240 con 0x7f7cc006e9c0 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0148820 con 0x7f7cc00ad5c0 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cc579a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc0148b00 con 0x7f7cc0156c70 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cbeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cbeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53660/0 (socket says 172.21.15.170:53660) 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cbeffd700 1 -- 172.21.15.170:0/3509251741 learned_addr learned my addr 172.21.15.170:0/3509251741 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:00.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbe7fc700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc0158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.862+0000 7f7cbf7fe700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cc0156c70 0x7f7cc01499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 -- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cc0156c70 msgr2=0x7f7cc01499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cc0156c70 0x7f7cc01499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 -- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cc00ad5c0 msgr2=0x7f7cc0158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc0158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 -- 172.21.15.170:0/3509251741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb800bc30 con 0x7f7cc006e9c0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cbeffd700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b9d0 tx=0x7f7cb801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8009d00 con 0x7f7cc006e9c0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb8009e60 con 0x7f7cc006e9c0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc01491d0 con 0x7f7cc006e9c0 2023-10-08T16:05:00.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80286b0 con 0x7f7cc006e9c0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.863+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc0149f10 con 0x7f7cc006e9c0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.864+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc01496f0 con 0x7f7cc006e9c0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.864+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cb801f420 con 0x7f7cc006e9c0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.865+0000 7f7ca6ffd700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 0x7f7cac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.865+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7cb8088780 con 0x7f7cc006e9c0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.865+0000 7f7cbe7fc700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 0x7f7cac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:00.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.865+0000 7f7cbe7fc700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 0x7f7cac0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00aded0 tx=0x7f7cb4007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:00.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:00.869+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cc01496f0 con 0x7f7cc006e9c0 2023-10-08T16:05:01.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.050+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7cc01496f0 con 0x7f7cc006e9c0 2023-10-08T16:05:01.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.051+0000 7f7ca6ffd700 1 -- 172.21.15.170:0/3509251741 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7cc01496f0 con 0x7f7cc006e9c0 2023-10-08T16:05:01.051 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:05:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 msgr2=0x7f7cac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 0x7f7cac0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00aded0 tx=0x7f7cb4007f30 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 msgr2=0x7f7cc0143c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b9d0 tx=0x7f7cb801a000 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 shutdown_connections 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cc0156c70 0x7f7cc01499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cac066f30 0x7f7cac0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7cc006e9c0 0x7f7cc0143c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 --2- 172.21.15.170:0/3509251741 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cc00ad5c0 0x7f7cc0158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 >> 172.21.15.170:0/3509251741 conn(0x7f7cc00c9fb0 msgr2=0x7f7cc014c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 shutdown_connections 2023-10-08T16:05:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.053+0000 7f7cc579a700 1 -- 172.21.15.170:0/3509251741 wait complete. 2023-10-08T16:05:01.065 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:05:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.182+0000 7f809acae700 1 Processor -- start 2023-10-08T16:05:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.182+0000 7f809acae700 1 -- start start 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809acae700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8094159890 0x7f809415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809acae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 0x7f80940699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809acae700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8094069ee0 con 0x7f8094159c60 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809acae700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809415a160 con 0x7f8094159890 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809acae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80940d13a0 con 0x7f8094069560 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8094159890 0x7f809415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8093fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8094159c60 0x7f809415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54248/0 (socket says 172.21.15.170:54248) 2023-10-08T16:05:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8093fff700 1 -- 172.21.15.170:0/2786878351 learned_addr learned my addr 172.21.15.170:0/2786878351 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809924b700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 0x7f80940699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 -- 172.21.15.170:0/2786878351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 msgr2=0x7f80940699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 0x7f80940699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 -- 172.21.15.170:0/2786878351 >> v1:172.21.15.170:6789/0 conn(0x7f8094159c60 legacy=0x7f809415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 -- 172.21.15.170:0/2786878351 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809415e140 con 0x7f8094159890 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f80937fe700 1 -- 172.21.15.170:0/2786878351 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213563438 0 0) 0x7f8094069ee0 con 0x7f8094159c60 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f809924b700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 0x7f80940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f8098a4a700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094159890 0x7f809415abd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8084009e60 tx=0x7f8084009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd86d449479ecff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f80937fe700 1 -- 172.21.15.170:0/2786878351 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400e070 con 0x7f8094159890 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f80937fe700 1 -- 172.21.15.170:0/2786878351 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808400bc30 con 0x7f8094159890 2023-10-08T16:05:01.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.183+0000 7f80937fe700 1 -- 172.21.15.170:0/2786878351 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084012430 con 0x7f8094159890 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 -- 172.21.15.170:0/2786878351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094159890 msgr2=0x7f809415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094159890 0x7f809415abd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8084009e60 tx=0x7f8084009780 comp rx=0 tx=0).stop 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 -- 172.21.15.170:0/2786878351 shutdown_connections 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094069560 0x7f80940699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 --2- 172.21.15.170:0/2786878351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094159890 0x7f809415abd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 -- 172.21.15.170:0/2786878351 >> 172.21.15.170:0/2786878351 conn(0x7f8094068e20 msgr2=0x7f80940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 -- 172.21.15.170:0/2786878351 shutdown_connections 2023-10-08T16:05:01.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.184+0000 7f809acae700 1 -- 172.21.15.170:0/2786878351 wait complete. 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.185+0000 7f809acae700 1 Processor -- start 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.185+0000 7f809acae700 1 -- start start 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8094159890 0x7f80940b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094159c60 0x7f80941518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b1860 con 0x7f8094069560 2023-10-08T16:05:01.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b3590 con 0x7f8094159890 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f809acae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8094169a90 con 0x7f8094159c60 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53680/0 (socket says 172.21.15.170:53680) 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 -- 172.21.15.170:0/160080098 learned_addr learned my addr 172.21.15.170:0/160080098 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 -- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094159c60 msgr2=0x7f80941518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094159c60 0x7f80941518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 -- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8094159890 msgr2=0x7f80940b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8094159890 0x7f80940b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 -- 172.21.15.170:0/160080098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084009a20 con 0x7f8094069560 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8098a4a700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80940899c0 tx=0x7f8084019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.186+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8084012e50 con 0x7f8094069560 2023-10-08T16:05:01.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.187+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808401d630 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.187+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8094169d70 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.187+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808401d9f0 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.187+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f809416a330 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.187+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f809404c3c0 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.188+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8084024070 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.188+0000 7f8090ff9700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 0x7f8080069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.188+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8084086d50 con 0x7f8094069560 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.188+0000 7f8093fff700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 0x7f8080069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.189+0000 7f8093fff700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 0x7f8080069460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80880011e0 tx=0x7f80880079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.193+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8084053f20 con 0x7f8094069560 2023-10-08T16:05:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.375+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f809406a5b0 con 0x7f8094069560 2023-10-08T16:05:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.375+0000 7f8090ff9700 1 -- 172.21.15.170:0/160080098 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f808402c020 con 0x7f8094069560 2023-10-08T16:05:01.376 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:05:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 msgr2=0x7f8080069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 0x7f8080069460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f80880011e0 tx=0x7f80880079d0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 msgr2=0x7f80940c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80940899c0 tx=0x7f8084019000 comp rx=0 tx=0).stop 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 shutdown_connections 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8094159c60 0x7f80941518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8080066fb0 0x7f8080069460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8094069560 0x7f80940c82f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 --2- 172.21.15.170:0/160080098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8094159890 0x7f80940b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.377+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 >> 172.21.15.170:0/160080098 conn(0x7f8094068e20 msgr2=0x7f80940b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.378+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 shutdown_connections 2023-10-08T16:05:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.378+0000 7f809acae700 1 -- 172.21.15.170:0/160080098 wait complete. 2023-10-08T16:05:01.390 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:01.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.504+0000 7f7ca0032700 1 Processor -- start 2023-10-08T16:05:01.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.504+0000 7f7ca0032700 1 -- start start 2023-10-08T16:05:01.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7ca0032700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7c98159c60 0x7f7c9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7ca0032700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9815a160 con 0x7f7c9815db10 2023-10-08T16:05:01.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7ca0032700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c980d13a0 con 0x7f7c98159890 2023-10-08T16:05:01.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7ca0032700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9815df50 con 0x7f7c98159c60 2023-10-08T16:05:01.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7c9e5cf700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7c9815db10 0x7f7c98069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54254/0 (socket says 172.21.15.170:54254) 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.505+0000 7f7c9e5cf700 1 -- 172.21.15.170:0/1394665799 learned_addr learned my addr 172.21.15.170:0/1394665799 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/1394665799 >> v2:172.21.15.191:3300/0 conn(0x7f7c98159c60 0x7f7c9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396027407 0 0) 0x7f7c9815a160 con 0x7f7c9815db10 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c80003570 con 0x7f7c9815db10 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562671887 0 0) 0x7f7c80003570 con 0x7f7c9815db10 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 >> v2:172.21.15.191:3300/0 conn(0x7f7c98159c60 msgr2=0x7f7c9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 --2- 172.21.15.170:0/1394665799 >> v2:172.21.15.191:3300/0 conn(0x7f7c98159c60 0x7f7c9815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 >> v1:172.21.15.191:6789/0 conn(0x7f7c98159890 legacy=0x7f7c9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9815e140 con 0x7f7c9815db10 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7c90001470 con 0x7f7c9815db10 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 >> v1:172.21.15.170:6789/0 conn(0x7f7c9815db10 legacy=0x7f7c98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80004810 0x7f7c80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c800051c0 0x7f7c8000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000c0a0 con 0x7f7c80003a60 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000c210 con 0x7f7c80004810 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8000c3d0 con 0x7f7c800051c0 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80004810 0x7f7c80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9e5cf700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c800051c0 0x7f7c8000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 -- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c800051c0 msgr2=0x7f7c8000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c800051c0 0x7f7c8000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 -- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80004810 msgr2=0x7f7c80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80004810 0x7f7c80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 -- 172.21.15.170:0/1394665799 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80003570 con 0x7f7c80003a60 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9e5cf700 1 -- 172.21.15.170:0/1394665799 reap_dead start 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80004810 0x7f7c80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.506+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001fc0 tx=0x7f7c88001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1b653ceeb9da24 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88001950 con 0x7f7c80003a60 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8800c8b0 con 0x7f7c80003a60 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7c9cdcc700 1 -- 172.21.15.170:0/1394665799 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8800cc50 con 0x7f7c80003a60 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 -- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 msgr2=0x7f7c80003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001fc0 tx=0x7f7c88001640 comp rx=0 tx=0).stop 2023-10-08T16:05:01.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 -- 172.21.15.170:0/1394665799 shutdown_connections 2023-10-08T16:05:01.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 --2- 172.21.15.170:0/1394665799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c80003a60 0x7f7c80003e70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 -- 172.21.15.170:0/1394665799 >> 172.21.15.170:0/1394665799 conn(0x7f7c98068e20 msgr2=0x7f7c980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 -- 172.21.15.170:0/1394665799 shutdown_connections 2023-10-08T16:05:01.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.507+0000 7f7ca0032700 1 -- 172.21.15.170:0/1394665799 wait complete. 2023-10-08T16:05:01.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.508+0000 7f7ca0032700 1 Processor -- start 2023-10-08T16:05:01.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.508+0000 7f7ca0032700 1 -- start start 2023-10-08T16:05:01.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 0x7f7c98149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c980c7b10 0x7f7c980c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b3930 con 0x7f7c980cf690 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b3c10 con 0x7f7c80003a60 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7ca0032700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980b3ef0 con 0x7f7c980c7b10 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53700/0 (socket says 172.21.15.170:53700) 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 -- 172.21.15.170:0/3548036263 learned_addr learned my addr 172.21.15.170:0/3548036263 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 0x7f7c98149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9e5cf700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c980c7b10 0x7f7c980c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 -- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c980c7b10 msgr2=0x7f7c980c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c980c7b10 0x7f7c980c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 -- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 msgr2=0x7f7c98149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 0x7f7c98149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9d5cd700 1 -- 172.21.15.170:0/3548036263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980b41d0 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.509+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 0x7f7c98149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7c9d5cd700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c011700 tx=0x7f7c8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c015670 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c0157d0 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c980b44b0 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c015b90 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.510+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c980b1cb0 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.511+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9804c3c0 con 0x7f7c980cf690 2023-10-08T16:05:01.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.511+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7c8c013410 con 0x7f7c980cf690 2023-10-08T16:05:01.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.512+0000 7f7c95ffb700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 0x7f7c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.512+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7c8c084b70 con 0x7f7c980cf690 2023-10-08T16:05:01.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.512+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 0x7f7c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.512+0000 7f7c9ddce700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 0x7f7c840693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001fc0 tx=0x7f7c88015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.516+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c8c051de0 con 0x7f7c980cf690 2023-10-08T16:05:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.701+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c980b4820 con 0x7f7c980cf690 2023-10-08T16:05:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.702+0000 7f7c95ffb700 1 -- 172.21.15.170:0/3548036263 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7c8c055400 con 0x7f7c980cf690 2023-10-08T16:05:01.703 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:01.703 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 msgr2=0x7f7c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 0x7f7c840693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7c88001fc0 tx=0x7f7c88015000 comp rx=0 tx=0).stop 2023-10-08T16:05:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 msgr2=0x7f7c980cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c011700 tx=0x7f7c8c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 shutdown_connections 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7c980c7b10 0x7f7c980c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c80003a60 0x7f7c98149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c84066f30 0x7f7c840693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 --2- 172.21.15.170:0/3548036263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c980cf690 0x7f7c980cfb00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 >> 172.21.15.170:0/3548036263 conn(0x7f7c98068e20 msgr2=0x7f7c9815d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 shutdown_connections 2023-10-08T16:05:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.705+0000 7f7ca0032700 1 -- 172.21.15.170:0/3548036263 wait complete. 2023-10-08T16:05:01.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:05:01.718 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:01.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.834+0000 7f57fbccf700 1 Processor -- start 2023-10-08T16:05:01.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.834+0000 7f57fbccf700 1 -- start start 2023-10-08T16:05:01.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fbccf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 0x7f57f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fbccf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 0x7f57f4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fbccf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f415a160 con 0x7f57f4159890 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fbccf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f40d13a0 con 0x7f57f415db10 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fbccf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f415e140 con 0x7f57f4159c60 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f926a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 0x7f57f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fa26c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 0x7f57f4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f9a6b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f57f4159890 0x7f57f415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58360/0 (socket says 172.21.15.170:58360) 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f9a6b700 1 -- 172.21.15.170:0/1788671805 learned_addr learned my addr 172.21.15.170:0/1788671805 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f926a700 1 -- 172.21.15.170:0/1788671805 >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 msgr2=0x7f57f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f926a700 1 --2- 172.21.15.170:0/1788671805 >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 0x7f57f4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f926a700 1 -- 172.21.15.170:0/1788671805 >> v1:172.21.15.191:6789/0 conn(0x7f57f4159890 legacy=0x7f57f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57f926a700 1 -- 172.21.15.170:0/1788671805 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f415a3d0 con 0x7f57f4159c60 2023-10-08T16:05:01.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.835+0000 7f57fa26c700 1 --2- 172.21.15.170:0/1788671805 >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 0x7f57f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57f926a700 1 --2- 172.21.15.170:0/1788671805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 0x7f57f415d5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57e40098e0 tx=0x7f57e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33ffefb7ffa72bd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57f8a69700 1 -- 172.21.15.170:0/1788671805 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400b640 con 0x7f57f4159c60 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57f8a69700 1 -- 172.21.15.170:0/1788671805 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e400b7a0 con 0x7f57f4159c60 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57f8a69700 1 -- 172.21.15.170:0/1788671805 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400bb40 con 0x7f57f4159c60 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 -- 172.21.15.170:0/1788671805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 msgr2=0x7f57f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 --2- 172.21.15.170:0/1788671805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 0x7f57f415d5d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57e40098e0 tx=0x7f57e40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 -- 172.21.15.170:0/1788671805 shutdown_connections 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 --2- 172.21.15.170:0/1788671805 >> v2:172.21.15.191:3300/0 conn(0x7f57f415db10 0x7f57f4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 --2- 172.21.15.170:0/1788671805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f4159c60 0x7f57f415d5d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 -- 172.21.15.170:0/1788671805 >> 172.21.15.170:0/1788671805 conn(0x7f57f4068e20 msgr2=0x7f57f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:01.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 -- 172.21.15.170:0/1788671805 shutdown_connections 2023-10-08T16:05:01.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.836+0000 7f57fbccf700 1 -- 172.21.15.170:0/1788671805 wait complete. 2023-10-08T16:05:01.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.837+0000 7f57fbccf700 1 Processor -- start 2023-10-08T16:05:01.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.837+0000 7f57fbccf700 1 -- start start 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f4159890 0x7f57f40ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f415db10 0x7f57f40cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40cd720 con 0x7f57f4159c60 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40ce560 con 0x7f57f4159890 2023-10-08T16:05:01.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fbccf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40cd950 con 0x7f57f415db10 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57f926a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57f926a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53706/0 (socket says 172.21.15.170:53706) 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57f926a700 1 -- 172.21.15.170:0/4165521570 learned_addr learned my addr 172.21.15.170:0/4165521570 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57f9a6b700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f4159890 0x7f57f40ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.838+0000 7f57fa26c700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f415db10 0x7f57f40cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 -- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f415db10 msgr2=0x7f57f40cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f415db10 0x7f57f40cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 -- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f4159890 msgr2=0x7f57f40ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f4159890 0x7f57f40ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:01.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 -- 172.21.15.170:0/4165521570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009a20 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57f926a700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57e4013000 tx=0x7f57e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e401bcf0 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e401be90 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e4009420 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4011670 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.839+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f40cdf10 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.840+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f40ce6f0 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.840+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57e40119e0 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.841+0000 7f57e9ffb700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.841+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f57e4085b00 con 0x7f57f4159c60 2023-10-08T16:05:01.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.841+0000 7f57f9a6b700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:01.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.841+0000 7f57f9a6b700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 0x7f57d80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57f0007130 tx=0x7f57f0007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:01.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:01.845+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57e4052e30 con 0x7f57f4159c60 2023-10-08T16:05:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.026+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57f404c3c0 con 0x7f57f4159c60 2023-10-08T16:05:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.027+0000 7f57e9ffb700 1 -- 172.21.15.170:0/4165521570 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57e4056450 con 0x7f57f4159c60 2023-10-08T16:05:02.028 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 msgr2=0x7f57d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 0x7f57d80693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f57f0007130 tx=0x7f57f0007eb0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 msgr2=0x7f57f414a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57e4013000 tx=0x7f57e4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 shutdown_connections 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f57f415db10 0x7f57f40cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f57f4159890 0x7f57f40ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f57d8066f30 0x7f57d80693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 --2- 172.21.15.170:0/4165521570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f57f4159c60 0x7f57f414a180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 >> 172.21.15.170:0/4165521570 conn(0x7f57f4068e20 msgr2=0x7f57f415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 shutdown_connections 2023-10-08T16:05:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.029+0000 7f57fbccf700 1 -- 172.21.15.170:0/4165521570 wait complete. 2023-10-08T16:05:02.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:05:02.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 Processor -- start 2023-10-08T16:05:02.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 -- start start 2023-10-08T16:05:02.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1dc8159c60 0x7f1dc815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1dc815db10 0x7f1dc8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc815a160 con 0x7f1dc8159890 2023-10-08T16:05:02.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc815df50 con 0x7f1dc8159c60 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.158+0000 7f1dce837700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80d13a0 con 0x7f1dc815db10 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1dc8159c60 0x7f1dc815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1dc8159c60 0x7f1dc815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53720/0 (socket says 172.21.15.170:53720) 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 -- 172.21.15.170:0/3115313031 learned_addr learned my addr 172.21.15.170:0/3115313031 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dccdd4700 1 --2- 172.21.15.170:0/3115313031 >> v2:172.21.15.191:3300/0 conn(0x7f1dc815db10 0x7f1dc8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 -- 172.21.15.170:0/3115313031 >> v2:172.21.15.191:3300/0 conn(0x7f1dc815db10 msgr2=0x7f1dc8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 --2- 172.21.15.170:0/3115313031 >> v2:172.21.15.191:3300/0 conn(0x7f1dc815db10 0x7f1dc8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 -- 172.21.15.170:0/3115313031 >> v1:172.21.15.191:6789/0 conn(0x7f1dc8159890 legacy=0x7f1dc815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 -- 172.21.15.170:0/3115313031 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc815e140 con 0x7f1dc8159c60 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc77fe700 1 --2- 172.21.15.170:0/3115313031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc815d5d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1dc806a890 tx=0x7f1db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40693d9078f0d82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc6ffd700 1 -- 172.21.15.170:0/3115313031 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8009e70 con 0x7f1dc8159c60 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc6ffd700 1 -- 172.21.15.170:0/3115313031 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db800b860 con 0x7f1dc8159c60 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.159+0000 7f1dc6ffd700 1 -- 172.21.15.170:0/3115313031 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db800bc00 con 0x7f1dc8159c60 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 -- 172.21.15.170:0/3115313031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 msgr2=0x7f1dc815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 --2- 172.21.15.170:0/3115313031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc815d5d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1dc806a890 tx=0x7f1db8009250 comp rx=0 tx=0).stop 2023-10-08T16:05:02.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 -- 172.21.15.170:0/3115313031 shutdown_connections 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 --2- 172.21.15.170:0/3115313031 >> v2:172.21.15.191:3300/0 conn(0x7f1dc815db10 0x7f1dc8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 --2- 172.21.15.170:0/3115313031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc815d5d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 -- 172.21.15.170:0/3115313031 >> 172.21.15.170:0/3115313031 conn(0x7f1dc8068e20 msgr2=0x7f1dc80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 -- 172.21.15.170:0/3115313031 shutdown_connections 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.160+0000 7f1dce837700 1 -- 172.21.15.170:0/3115313031 wait complete. 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.161+0000 7f1dce837700 1 Processor -- start 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.161+0000 7f1dce837700 1 -- start start 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 0x7f1dc8149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 0x7f1dc816d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc816ddf0 con 0x7f1dc8159c60 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc816e0d0 con 0x7f1dc815db10 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dce837700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b0540 con 0x7f1dc8159890 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53728/0 (socket says 172.21.15.170:53728) 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 -- 172.21.15.170:0/4281027140 learned_addr learned my addr 172.21.15.170:0/4281027140 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dccdd4700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 0x7f1dc816d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 -- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 msgr2=0x7f1dc8149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc7fff700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 0x7f1dc8149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 0x7f1dc8149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 -- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 msgr2=0x7f1dc816d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 0x7f1dc816d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 -- 172.21.15.170:0/4281027140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80094f0 con 0x7f1dc8159c60 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dccdd4700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 0x7f1dc816d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc7fff700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 0x7f1dc8149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.162+0000 7f1dc77fe700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8070e20 tx=0x7f1db8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.163+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8011420 con 0x7f1dc8159c60 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.163+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db80115a0 con 0x7f1dc8159c60 2023-10-08T16:05:02.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.163+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc80b0820 con 0x7f1dc8159c60 2023-10-08T16:05:02.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.163+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8011960 con 0x7f1dc8159c60 2023-10-08T16:05:02.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.163+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc814a430 con 0x7f1dc8159c60 2023-10-08T16:05:02.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.164+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc804c3c0 con 0x7f1dc8159c60 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.164+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1db80253f0 con 0x7f1dc8159c60 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.164+0000 7f1daffff700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.164+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1db8085ee0 con 0x7f1dc8159c60 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.164+0000 7f1dc7fff700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.165+0000 7f1dc7fff700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001840 tx=0x7f1dbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.169+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1db8053150 con 0x7f1dc8159c60 2023-10-08T16:05:02.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.349+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1dc806a5b0 con 0x7f1dc8159c60 2023-10-08T16:05:02.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.350+0000 7f1daffff700 1 -- 172.21.15.170:0/4281027140 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1db8056770 con 0x7f1dc8159c60 2023-10-08T16:05:02.350 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:05:02.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.351+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 msgr2=0x7f1db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 0x7f1db40693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001840 tx=0x7f1dbc009040 comp rx=0 tx=0).stop 2023-10-08T16:05:02.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 msgr2=0x7f1dc814afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8070e20 tx=0x7f1db8017000 comp rx=0 tx=0).stop 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 shutdown_connections 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1dc815db10 0x7f1dc816d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1dc8159890 0x7f1dc8149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1db4066f30 0x7f1db40693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 --2- 172.21.15.170:0/4281027140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc8159c60 0x7f1dc814afc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 >> 172.21.15.170:0/4281027140 conn(0x7f1dc8068e20 msgr2=0x7f1dc814d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 shutdown_connections 2023-10-08T16:05:02.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.352+0000 7f1dce837700 1 -- 172.21.15.170:0/4281027140 wait complete. 2023-10-08T16:05:02.364 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:05:02.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.483+0000 7fe2fdd9f700 1 Processor -- start 2023-10-08T16:05:02.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.483+0000 7fe2fdd9f700 1 -- start start 2023-10-08T16:05:02.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2fdd9f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe2f80af250 0x7fe2f8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2fdd9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2fdd9f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f806b030 con 0x7fe2f8157960 2023-10-08T16:05:02.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2fdd9f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f80ac760 con 0x7fe2f80af250 2023-10-08T16:05:02.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2fdd9f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80af830 con 0x7fe2f80688d0 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2effff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe2f80af250 0x7fe2f8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:41052/0 (socket says 172.21.15.170:41052) 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2f7fff700 1 -- 172.21.15.170:0/4030716755 learned_addr learned my addr 172.21.15.170:0/4030716755 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.484+0000 7fe2f77fe700 1 --1- 172.21.15.170:0/4030716755 >> v1:172.21.15.191:6789/0 conn(0x7fe2f8157960 0x7fe2f806b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58374/0 (socket says 172.21.15.170:58374) 2023-10-08T16:05:02.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f6ffd700 1 -- 172.21.15.170:0/4030716755 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182098657 0 0) 0x7fe2f806b030 con 0x7fe2f8157960 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f6ffd700 1 -- 172.21.15.170:0/4030716755 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e4003570 con 0x7fe2f8157960 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2effff700 1 -- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 msgr2=0x7fe2f8157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2effff700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2effff700 1 -- 172.21.15.170:0/4030716755 >> v1:172.21.15.191:6789/0 conn(0x7fe2f8157960 legacy=0x7fe2f806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2effff700 1 -- 172.21.15.170:0/4030716755 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f806bbe0 con 0x7fe2f80af250 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f7fff700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2effff700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f8068390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00098e0 tx=0x7fe2e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3cd405661435011 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f6ffd700 1 -- 172.21.15.170:0/4030716755 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0011630 con 0x7fe2f80af250 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f6ffd700 1 -- 172.21.15.170:0/4030716755 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0011790 con 0x7fe2f80af250 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.485+0000 7fe2f6ffd700 1 -- 172.21.15.170:0/4030716755 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0011b30 con 0x7fe2f80af250 2023-10-08T16:05:02.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 msgr2=0x7fe2f8068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f8068390 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe2e00098e0 tx=0x7fe2e00091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/4030716755 shutdown_connections 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/4030716755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f8068390 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/4030716755 >> 172.21.15.170:0/4030716755 conn(0x7fe2f80c9870 msgr2=0x7fe2f80c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/4030716755 shutdown_connections 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.486+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/4030716755 wait complete. 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.487+0000 7fe2fdd9f700 1 Processor -- start 2023-10-08T16:05:02.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.487+0000 7fe2fdd9f700 1 -- start start 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f80adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f80c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80afdd0 con 0x7fe2f8157960 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80c4dd0 con 0x7fe2f80af250 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80c5080 con 0x7fe2f80688d0 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53736/0 (socket says 172.21.15.170:53736) 2023-10-08T16:05:02.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 -- 172.21.15.170:0/2301840148 learned_addr learned my addr 172.21.15.170:0/2301840148 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2effff700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f80c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f77fe700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f80adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 -- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 msgr2=0x7fe2f80adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f80adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 -- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 msgr2=0x7fe2f80c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f80c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 -- 172.21.15.170:0/2301840148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0009ae0 con 0x7fe2f8157960 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2f7fff700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009f30 tx=0x7fe2f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0018070 con 0x7fe2f8157960 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000f040 con 0x7fe2f8157960 2023-10-08T16:05:02.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e0009420 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000cc00 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.488+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f80c5640 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.489+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f80afba0 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.490+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2f0003020 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.490+0000 7fe2ef7fe700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 0x7fe2d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.490+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe2f0084a10 con 0x7fe2f8157960 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.490+0000 7fe2f77fe700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 0x7fe2d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.491+0000 7fe2f77fe700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 0x7fe2d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8004ee0 tx=0x7fe2e8007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.494+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2f80afba0 con 0x7fe2f8157960 2023-10-08T16:05:02.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.675+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2f80afba0 con 0x7fe2f8157960 2023-10-08T16:05:02.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.676+0000 7fe2ef7fe700 1 -- 172.21.15.170:0/2301840148 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2f80afba0 con 0x7fe2f8157960 2023-10-08T16:05:02.676 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 msgr2=0x7fe2d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 0x7fe2d80693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8004ee0 tx=0x7fe2e8007d40 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 msgr2=0x7fe2f80c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009f30 tx=0x7fe2f000b040 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 shutdown_connections 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe2d8066f30 0x7fe2d80693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2f8157960 0x7fe2f80c2dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2f80688d0 0x7fe2f80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 --2- 172.21.15.170:0/2301840148 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2f80af250 0x7fe2f80c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 >> 172.21.15.170:0/2301840148 conn(0x7fe2f80c9870 msgr2=0x7fe2f81557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 shutdown_connections 2023-10-08T16:05:02.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.678+0000 7fe2fdd9f700 1 -- 172.21.15.170:0/2301840148 wait complete. 2023-10-08T16:05:02.757 INFO:tasks.cephfs_test_runner:Starting test: test_volume_create (tasks.cephfs.test_volumes.TestVolumes) That the volume can be created and then cleans up 2023-10-08T16:05:02.757 INFO:tasks.cephfs_test_runner:test_volume_create (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:05:02.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-08T16:05:02.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.872+0000 7fa01970b700 1 Processor -- start 2023-10-08T16:05:02.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.872+0000 7fa01970b700 1 -- start start 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa01970b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa014159c60 0x7fa01415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa01970b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa01415db10 0x7fa014069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa01970b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01415a160 con 0x7fa014159890 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa01970b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01415df50 con 0x7fa014159c60 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa01970b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0140d13a0 con 0x7fa01415db10 2023-10-08T16:05:02.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa0127fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa014159c60 0x7fa01415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa0127fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa014159c60 0x7fa01415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53744/0 (socket says 172.21.15.170:53744) 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa0127fc700 1 -- 172.21.15.170:0/291418402 learned_addr learned my addr 172.21.15.170:0/291418402 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa012ffd700 1 --1- 172.21.15.170:0/291418402 >> v1:172.21.15.191:6789/0 conn(0x7fa014159890 0x7fa01415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58376/0 (socket says 172.21.15.170:58376) 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.873+0000 7fa0137fe700 1 --2- 172.21.15.170:0/291418402 >> v2:172.21.15.191:3300/0 conn(0x7fa01415db10 0x7fa014069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa0127fc700 1 -- 172.21.15.170:0/291418402 >> v2:172.21.15.191:3300/0 conn(0x7fa01415db10 msgr2=0x7fa014069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa0127fc700 1 --2- 172.21.15.170:0/291418402 >> v2:172.21.15.191:3300/0 conn(0x7fa01415db10 0x7fa014069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa0127fc700 1 -- 172.21.15.170:0/291418402 >> v1:172.21.15.191:6789/0 conn(0x7fa014159890 legacy=0x7fa01415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa0127fc700 1 -- 172.21.15.170:0/291418402 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01415e140 con 0x7fa014159c60 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa011ffb700 1 -- 172.21.15.170:0/291418402 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675012904 0 0) 0x7fa01415a160 con 0x7fa014159890 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa0127fc700 1 --2- 172.21.15.170:0/291418402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa01415d5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa01406a890 tx=0x7f9ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7c18ffc8acb46b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa011ffb700 1 -- 172.21.15.170:0/291418402 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc009e70 con 0x7fa014159c60 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa011ffb700 1 -- 172.21.15.170:0/291418402 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc00b860 con 0x7fa014159c60 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa011ffb700 1 -- 172.21.15.170:0/291418402 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00bc00 con 0x7fa014159c60 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 -- 172.21.15.170:0/291418402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 msgr2=0x7fa01415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 --2- 172.21.15.170:0/291418402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa01415d5d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa01406a890 tx=0x7f9ffc009250 comp rx=0 tx=0).stop 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 -- 172.21.15.170:0/291418402 shutdown_connections 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 --2- 172.21.15.170:0/291418402 >> v2:172.21.15.191:3300/0 conn(0x7fa01415db10 0x7fa014069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 --2- 172.21.15.170:0/291418402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa01415d5d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 -- 172.21.15.170:0/291418402 >> 172.21.15.170:0/291418402 conn(0x7fa014068e20 msgr2=0x7fa0140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 -- 172.21.15.170:0/291418402 shutdown_connections 2023-10-08T16:05:02.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.874+0000 7fa01970b700 1 -- 172.21.15.170:0/291418402 wait complete. 2023-10-08T16:05:02.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 Processor -- start 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 -- start start 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa014159890 0x7fa0140cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa01415db10 0x7fa01414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140cdad0 con 0x7fa014159c60 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140ce330 con 0x7fa014159890 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.876+0000 7fa01970b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01414cb40 con 0x7fa01415db10 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53752/0 (socket says 172.21.15.170:53752) 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 -- 172.21.15.170:0/3776876359 learned_addr learned my addr 172.21.15.170:0/3776876359 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:02.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa012ffd700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa014159890 0x7fa0140cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 -- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa01415db10 msgr2=0x7fa01414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0137fe700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa01415db10 0x7fa01414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa01415db10 0x7fa01414c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 -- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa014159890 msgr2=0x7fa0140cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa014159890 0x7fa0140cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 -- 172.21.15.170:0/3776876359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc0094f0 con 0x7fa014159c60 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa0127fc700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0094c0 tx=0x7f9ffc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00b9d0 con 0x7fa014159c60 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ffc00bb50 con 0x7fa014159c60 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc025670 con 0x7fa014159c60 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140cfd50 con 0x7fa014159c60 2023-10-08T16:05:02.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.877+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0140cdfc0 con 0x7fa014159c60 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.878+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01404c3c0 con 0x7fa014159c60 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.878+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ffc023070 con 0x7fa014159c60 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.879+0000 7fa00affd700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 0x7f9ff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.879+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ffc085b20 con 0x7fa014159c60 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.879+0000 7fa012ffd700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 0x7f9ff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:02.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.879+0000 7fa012ffd700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 0x7f9ff40693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa014153030 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:02.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:02.883+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ffc052d90 con 0x7fa014159c60 2023-10-08T16:05:03.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.143+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7fa01406a5b0 con 0x7fa014159c60 2023-10-08T16:05:03.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.441+0000 7fa00affd700 1 -- 172.21.15.170:0/3776876359 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9ffc0150d0 con 0x7fa014159c60 2023-10-08T16:05:03.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 msgr2=0x7f9ff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 0x7f9ff40693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa014153030 tx=0x7fa004009040 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 msgr2=0x7fa0140cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0094c0 tx=0x7f9ffc017000 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 shutdown_connections 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa01415db10 0x7fa01414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa014159890 0x7fa0140cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ff4066f30 0x7f9ff40693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 --2- 172.21.15.170:0/3776876359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa014159c60 0x7fa0140cf720 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 >> 172.21.15.170:0/3776876359 conn(0x7fa014068e20 msgr2=0x7fa01414d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:03.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 shutdown_connections 2023-10-08T16:05:03.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.443+0000 7fa01970b700 1 -- 172.21.15.170:0/3776876359 wait complete. 2023-10-08T16:05:03.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:05:03.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.572+0000 7fb4da8b9700 1 Processor -- start 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.572+0000 7fb4da8b9700 1 -- start start 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.573+0000 7fb4da8b9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb4d4159890 0x7fb4d415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.573+0000 7fb4da8b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.573+0000 7fb4da8b9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d4069ee0 con 0x7fb4d4159c60 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.573+0000 7fb4da8b9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d415a160 con 0x7fb4d4159890 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.573+0000 7fb4da8b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40d13a0 con 0x7fb4d4069560 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d8e56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb4d4159890 0x7fb4d415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb4d4159890 0x7fb4d415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:53220/0 (socket says 172.21.15.170:53220) 2023-10-08T16:05:03.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d8e56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:41076/0 (socket says 172.21.15.170:41076) 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d37fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fb4d4159c60 0x7fb4d415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58392/0 (socket says 172.21.15.170:58392) 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 -- 172.21.15.170:0/2374284765 learned_addr learned my addr 172.21.15.170:0/2374284765 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 -- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 msgr2=0x7fb4d40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 -- 172.21.15.170:0/2374284765 >> v1:172.21.15.191:6789/0 conn(0x7fb4d4159c60 legacy=0x7fb4d415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 -- 172.21.15.170:0/2374284765 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d415e140 con 0x7fb4d4159890 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d8e56700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d3fff700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159890 0x7fb4d415abd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40099b0 tx=0x7fb4c4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729024936063cae5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d2ffd700 1 -- 172.21.15.170:0/2374284765 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4009ea0 con 0x7fb4d4159890 2023-10-08T16:05:03.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d2ffd700 1 -- 172.21.15.170:0/2374284765 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c400b860 con 0x7fb4d4159890 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.574+0000 7fb4d2ffd700 1 -- 172.21.15.170:0/2374284765 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c400bc00 con 0x7fb4d4159890 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 -- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159890 msgr2=0x7fb4d415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159890 0x7fb4d415abd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40099b0 tx=0x7fb4c4009250 comp rx=0 tx=0).stop 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 -- 172.21.15.170:0/2374284765 shutdown_connections 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/2374284765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159890 0x7fb4d415abd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 -- 172.21.15.170:0/2374284765 >> 172.21.15.170:0/2374284765 conn(0x7fb4d4068e20 msgr2=0x7fb4d40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 -- 172.21.15.170:0/2374284765 shutdown_connections 2023-10-08T16:05:03.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.575+0000 7fb4da8b9700 1 -- 172.21.15.170:0/2374284765 wait complete. 2023-10-08T16:05:03.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.576+0000 7fb4da8b9700 1 Processor -- start 2023-10-08T16:05:03.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.576+0000 7fb4da8b9700 1 -- start start 2023-10-08T16:05:03.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159c60 0x7fb4d4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d4158c50 con 0x7fb4d4159890 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d41589b0 con 0x7fb4d4159c60 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4da8b9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d40b40d0 con 0x7fb4d4069560 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53762/0 (socket says 172.21.15.170:53762) 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 -- 172.21.15.170:0/562875308 learned_addr learned my addr 172.21.15.170:0/562875308 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d8e56700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159c60 0x7fb4d4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d3fff700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 -- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 msgr2=0x7fb4d40c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 -- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159c60 msgr2=0x7fb4d4158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159c60 0x7fb4d4158470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 -- 172.21.15.170:0/562875308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc009a20 con 0x7fb4d4159890 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.577+0000 7fb4d37fe700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c7c0 tx=0x7fb4c8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.578+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8005a70 con 0x7fb4d4159890 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.578+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c8005bd0 con 0x7fb4d4159890 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.578+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c40094f0 con 0x7fb4d4159890 2023-10-08T16:05:03.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.578+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80070d0 con 0x7fb4d4159890 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.578+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d40b4690 con 0x7fb4d4159890 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.579+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d404c3c0 con 0x7fb4d4159890 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.579+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c8007480 con 0x7fb4d4159890 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.579+0000 7fb4bbfff700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 0x7fb4c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.579+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb4c80839f0 con 0x7fb4d4159890 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.579+0000 7fb4d3fff700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 0x7fb4c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:03.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.580+0000 7fb4d3fff700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 0x7fb4c00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40099b0 tx=0x7fb4c4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:03.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.584+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c8050d20 con 0x7fb4d4159890 2023-10-08T16:05:03.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.764+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4d406a5b0 con 0x7fb4d4159890 2023-10-08T16:05:03.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.768+0000 7fb4bbfff700 1 -- 172.21.15.170:0/562875308 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4c8017020 con 0x7fb4d4159890 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 msgr2=0x7fb4c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 0x7fb4c00693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40099b0 tx=0x7fb4c4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 msgr2=0x7fb4d4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c7c0 tx=0x7fb4c8007f70 comp rx=0 tx=0).stop 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 shutdown_connections 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4d4159c60 0x7fb4d4158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4d4069560 0x7fb4d40c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4c0066f30 0x7fb4c00693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 --2- 172.21.15.170:0/562875308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d4159890 0x7fb4d4153c40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 >> 172.21.15.170:0/562875308 conn(0x7fb4d4068e20 msgr2=0x7fb4d40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 shutdown_connections 2023-10-08T16:05:03.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:03.770+0000 7fb4da8b9700 1 -- 172.21.15.170:0/562875308 wait complete. 2023-10-08T16:05:03.783 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:05:03.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:03.820 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:03.865 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:05:03.866 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:05:03.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:03.866 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-08T16:05:03.902 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:05:03.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:03.903 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:05:03.957 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:05:03.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:03.957 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-08T16:05:03.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:05:03.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:03.995 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:05:04.051 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-08T16:05:04.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:04.051 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-08T16:05:04.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:05:04.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:04.091 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T16:05:04.146 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-08T16:05:04.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:04.146 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-08T16:05:04.196 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:05:04.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:04.197 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T16:05:04.213 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:05:04.213 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.327+0000 7f7a3adab700 1 Processor -- start 2023-10-08T16:05:04.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.327+0000 7f7a3adab700 1 -- start start 2023-10-08T16:05:04.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a3adab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a34155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a3adab700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340acac0 con 0x7f7a340ac230 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a3adab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a340ad930 con 0x7f7a340ac6a0 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a3adab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340acd20 con 0x7f7a34155c10 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a33fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7a340ac230 0x7f7a341568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54258/0 (socket says 172.21.15.170:54258) 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.328+0000 7f7a33fff700 1 -- 172.21.15.170:0/1695783740 learned_addr learned my addr 172.21.15.170:0/1695783740 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776755896 0 0) 0x7f7a340acac0 con 0x7f7a340ac230 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a38b47700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a34155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a1c003570 con 0x7f7a340ac230 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553698364 0 0) 0x7f7a1c003570 con 0x7f7a340ac230 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 msgr2=0x7f7a34155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a34155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 >> v1:172.21.15.191:6789/0 conn(0x7f7a340ac6a0 legacy=0x7f7a340b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340ad000 con 0x7f7a340ac230 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7a28001470 con 0x7f7a340ac230 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 >> v1:172.21.15.170:6789/0 conn(0x7f7a340ac230 legacy=0x7f7a341568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1c003a90 0x7f7a1c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a1c004810 0x7f7a1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340acd20 con 0x7f7a1c004810 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c00bb10 con 0x7f7a1c003a90 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c00bcd0 con 0x7f7a34155c10 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a1c004810 0x7f7a1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a38b47700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1c003a90 0x7f7a1c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a39348700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 -- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 msgr2=0x7f7a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 -- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1c003a90 msgr2=0x7f7a1c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a1c003a90 0x7f7a1c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 -- 172.21.15.170:0/1695783740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c003570 con 0x7f7a1c004810 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a39348700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a34155c10 0x7f7a1c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.329+0000 7f7a33fff700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a1c004810 0x7f7a1c004c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c3c0 tx=0x7f7a28001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597332734e1f123d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cc90 con 0x7f7a1c004810 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2800cdf0 con 0x7f7a1c004810 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a337fe700 1 -- 172.21.15.170:0/1695783740 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28005b70 con 0x7f7a1c004810 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 -- 172.21.15.170:0/1695783740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a1c004810 msgr2=0x7f7a1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1695783740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a1c004810 0x7f7a1c004c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800c3c0 tx=0x7f7a28001e70 comp rx=0 tx=0).stop 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a39348700 1 -- 172.21.15.170:0/1695783740 reap_dead start 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 -- 172.21.15.170:0/1695783740 shutdown_connections 2023-10-08T16:05:04.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 -- 172.21.15.170:0/1695783740 >> 172.21.15.170:0/1695783740 conn(0x7f7a3406a320 msgr2=0x7f7a340cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:04.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 -- 172.21.15.170:0/1695783740 shutdown_connections 2023-10-08T16:05:04.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.330+0000 7f7a3adab700 1 -- 172.21.15.170:0/1695783740 wait complete. 2023-10-08T16:05:04.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.331+0000 7f7a3adab700 1 Processor -- start 2023-10-08T16:05:04.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.331+0000 7f7a3adab700 1 -- start start 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1c004810 0x7f7a341460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a340cb7a0 0x7f7a340a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340cc890 con 0x7f7a341437f0 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340cbc80 con 0x7f7a340cb7a0 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a3adab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a340cbf60 con 0x7f7a1c004810 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53778/0 (socket says 172.21.15.170:53778) 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a38b47700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1c004810 0x7f7a341460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 -- 172.21.15.170:0/1598672848 learned_addr learned my addr 172.21.15.170:0/1598672848 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a33fff700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a340cb7a0 0x7f7a340a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 -- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1c004810 msgr2=0x7f7a341460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1c004810 0x7f7a341460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 -- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a340cb7a0 msgr2=0x7f7a340a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a340cb7a0 0x7f7a340a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.332+0000 7f7a39348700 1 -- 172.21.15.170:0/1598672848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340cc240 con 0x7f7a341437f0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a39348700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a34063d40 tx=0x7f7a2c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00eeb0 con 0x7f7a341437f0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c0114f0 con 0x7f7a341437f0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0118d0 con 0x7f7a341437f0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a340cc520 con 0x7f7a341437f0 2023-10-08T16:05:04.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.333+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a340ccd20 con 0x7f7a341437f0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.334+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a340cca00 con 0x7f7a341437f0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.334+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a2c011c40 con 0x7f7a341437f0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.335+0000 7f7a30ff9700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 0x7f7a200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.335+0000 7f7a38b47700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 0x7f7a200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.335+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7a2c083e60 con 0x7f7a341437f0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.335+0000 7f7a38b47700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 0x7f7a200693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400a5e0 tx=0x7f7a2400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.339+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a2c051030 con 0x7f7a341437f0 2023-10-08T16:05:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.566+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a3404c3c0 con 0x7f7a341437f0 2023-10-08T16:05:04.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.567+0000 7f7a30ff9700 1 -- 172.21.15.170:0/1598672848 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f7a2c019090 con 0x7f7a341437f0 2023-10-08T16:05:04.567 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:04.567 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:05:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 msgr2=0x7f7a200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 0x7f7a200693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400a5e0 tx=0x7f7a2400c040 comp rx=0 tx=0).stop 2023-10-08T16:05:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 msgr2=0x7f7a34143c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a34063d40 tx=0x7f7a2c009040 comp rx=0 tx=0).stop 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 shutdown_connections 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a20066f30 0x7f7a200693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a341437f0 0x7f7a34143c90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1c004810 0x7f7a341460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 --2- 172.21.15.170:0/1598672848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a340cb7a0 0x7f7a340a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 >> 172.21.15.170:0/1598672848 conn(0x7f7a3406a320 msgr2=0x7f7a340ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 shutdown_connections 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.569+0000 7f7a3adab700 1 -- 172.21.15.170:0/1598672848 wait complete. 2023-10-08T16:05:04.570 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:05:04.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:04.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.695+0000 7fceaeff7700 1 Processor -- start 2023-10-08T16:05:04.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.695+0000 7fceaeff7700 1 -- start start 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceaeff7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fcea815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceaeff7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea80b2980 con 0x7fcea81597f0 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceaeff7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea815dbb0 con 0x7fcea8159420 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceaeff7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80b11d0 con 0x7fcea815d230 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fcead7f4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcea81597f0 0x7fcea80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54260/0 (socket says 172.21.15.170:54260) 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fcead7f4700 1 -- 172.21.15.170:0/480540499 learned_addr learned my addr 172.21.15.170:0/480540499 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942944620 0 0) 0x7fcea80b2980 con 0x7fcea81597f0 2023-10-08T16:05:04.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceae7f6700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fcea815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.696+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce94003570 con 0x7fcea81597f0 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521196324 0 0) 0x7fce94003570 con 0x7fcea81597f0 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 msgr2=0x7fcea815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fcea815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 >> v1:172.21.15.191:6789/0 conn(0x7fcea8159420 legacy=0x7fcea80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80b14b0 con 0x7fcea81597f0 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fce98001470 con 0x7fcea81597f0 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 >> v1:172.21.15.170:6789/0 conn(0x7fcea81597f0 legacy=0x7fcea80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94003a90 0x7fce94003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce94004810 0x7fce94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fce940051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80b11d0 con 0x7fce94004810 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9400bb60 con 0x7fce94003a90 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9400bd20 con 0x7fcea815d230 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce94004810 0x7fce94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceadff5700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94003a90 0x7fce94003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceae7f6700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fce940051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 -- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 msgr2=0x7fce940051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea815d230 0x7fce940051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 -- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94003a90 msgr2=0x7fce94003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94003a90 0x7fce94003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 -- 172.21.15.170:0/480540499 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce94003570 con 0x7fce94004810 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceadff5700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94003a90 0x7fce94003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fcead7f4700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce94004810 0x7fce94004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c390 tx=0x7fce98001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8a741bd77242e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9800cc60 con 0x7fce94004810 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9800cdc0 con 0x7fce94004810 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.697+0000 7fceacff3700 1 -- 172.21.15.170:0/480540499 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98005b70 con 0x7fce94004810 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 -- 172.21.15.170:0/480540499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce94004810 msgr2=0x7fce94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 --2- 172.21.15.170:0/480540499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce94004810 0x7fce94004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fce9800c390 tx=0x7fce98001e50 comp rx=0 tx=0).stop 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceae7f6700 1 -- 172.21.15.170:0/480540499 reap_dead start 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 -- 172.21.15.170:0/480540499 shutdown_connections 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 -- 172.21.15.170:0/480540499 >> 172.21.15.170:0/480540499 conn(0x7fcea80d0660 msgr2=0x7fcea80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:04.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 -- 172.21.15.170:0/480540499 shutdown_connections 2023-10-08T16:05:04.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.698+0000 7fceaeff7700 1 -- 172.21.15.170:0/480540499 wait complete. 2023-10-08T16:05:04.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.699+0000 7fceaeff7700 1 Processor -- start 2023-10-08T16:05:04.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.699+0000 7fceaeff7700 1 -- start start 2023-10-08T16:05:04.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 0x7fcea80c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 0x7fcea80cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80c8730 con 0x7fcea80ce980 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80cbd00 con 0x7fce94004810 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceaeff7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea80c7b20 con 0x7fcea80cb350 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53808/0 (socket says 172.21.15.170:53808) 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 -- 172.21.15.170:0/2283733759 learned_addr learned my addr 172.21.15.170:0/2283733759 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fcead7f4700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 0x7fcea80cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceadff5700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 0x7fcea80c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 -- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 msgr2=0x7fcea80cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 0x7fcea80cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 -- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 msgr2=0x7fcea80c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 0x7fcea80c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 -- 172.21.15.170:0/2283733759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80c7e00 con 0x7fcea80ce980 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fcead7f4700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 0x7fcea80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceadff5700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 0x7fcea80c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.700+0000 7fceae7f6700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af930 tx=0x7fcea000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.701+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea000e070 con 0x7fcea80ce980 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.701+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea0011d00 con 0x7fcea80ce980 2023-10-08T16:05:04.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.701+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea80c80e0 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.701+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea00166c0 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.701+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea80c9100 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.702+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea80c8580 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.702+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcea0016ab0 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.702+0000 7fce9dffb700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 0x7fce8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.702+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcea00850e0 con 0x7fcea80ce980 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.703+0000 7fceadff5700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 0x7fce8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.703+0000 7fceadff5700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 0x7fce8c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcea4003a90 tx=0x7fcea4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:04.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.707+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcea0055de0 con 0x7fcea80ce980 2023-10-08T16:05:04.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.892+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcea804c3c0 con 0x7fcea80ce980 2023-10-08T16:05:04.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.893+0000 7fce9dffb700 1 -- 172.21.15.170:0/2283733759 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fcea0055970 con 0x7fcea80ce980 2023-10-08T16:05:04.893 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:04.893 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 msgr2=0x7fce8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 0x7fce8c0693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcea4003a90 tx=0x7fcea4009040 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 msgr2=0x7fcea80cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af930 tx=0x7fcea000b040 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 shutdown_connections 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce8c066f30 0x7fce8c0693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea80ce980 0x7fcea80cedf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce94004810 0x7fcea80c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 --2- 172.21.15.170:0/2283733759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea80cb350 0x7fcea80cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:04.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 >> 172.21.15.170:0/2283733759 conn(0x7fcea80d0660 msgr2=0x7fcea8156100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:04.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 shutdown_connections 2023-10-08T16:05:04.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:04.895+0000 7fceaeff7700 1 -- 172.21.15.170:0/2283733759 wait complete. 2023-10-08T16:05:04.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:04:33.930734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:04:35.181236+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:05:04.908 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:05.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.024+0000 7f52c95fd700 1 Processor -- start 2023-10-08T16:05:05.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.024+0000 7f52c95fd700 1 -- start start 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c95fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f52c406c0a0 0x7f52c40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c95fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52c415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c95fd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c415d7e0 con 0x7f52c406bcd0 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c95fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c415dac0 con 0x7f52c406c0a0 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c95fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415aa30 con 0x7f52c415df50 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f52c406bcd0 0x7f52c415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54272/0 (socket says 172.21.15.170:54272) 2023-10-08T16:05:05.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.025+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2568025724 learned_addr learned my addr 172.21.15.170:0/2568025724 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954688317 0 0) 0x7f52c415d7e0 con 0x7f52c406bcd0 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52ac003570 con 0x7f52c406bcd0 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2568025724 >> v2:172.21.15.191:3300/0 conn(0x7f52c406c0a0 0x7f52c40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c37fe700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52c415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2807595836 0 0) 0x7f52ac003570 con 0x7f52c406bcd0 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 msgr2=0x7f52c415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52c415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 >> v2:172.21.15.191:3300/0 conn(0x7f52c406c0a0 msgr2=0x7f52c40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 --2- 172.21.15.170:0/2568025724 >> v2:172.21.15.191:3300/0 conn(0x7f52c406c0a0 0x7f52c40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c415ad10 con 0x7f52c406bcd0 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f52b40017f0 con 0x7f52c406bcd0 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 >> v1:172.21.15.170:6789/0 conn(0x7f52c406bcd0 legacy=0x7f52c415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52ac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52ac005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415dac0 con 0x7f52ac003a60 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415aa30 con 0x7f52ac004810 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac00bc20 con 0x7f52c415df50 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52ac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c37fe700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52ac005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 msgr2=0x7f52ac005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c415df50 0x7f52ac005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ac004810 msgr2=0x7f52ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2568025724 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac003570 con 0x7f52ac003a60 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.026+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52ac004810 0x7f52ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52ac003e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52b400a970 tx=0x7f52b4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8165e94a8e6046 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4012440 con 0x7f52ac003a60 2023-10-08T16:05:05.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b40125c0 con 0x7f52ac003a60 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c1ffb700 1 -- 172.21.15.170:0/2568025724 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4012980 con 0x7f52ac003a60 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 -- 172.21.15.170:0/2568025724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 msgr2=0x7f52ac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2568025724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52ac003e70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f52b400a970 tx=0x7f52b4002210 comp rx=0 tx=0).stop 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c37fe700 1 -- 172.21.15.170:0/2568025724 reap_dead start 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 -- 172.21.15.170:0/2568025724 shutdown_connections 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 -- 172.21.15.170:0/2568025724 >> 172.21.15.170:0/2568025724 conn(0x7f52c40ce600 msgr2=0x7f52c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 -- 172.21.15.170:0/2568025724 shutdown_connections 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.027+0000 7f52c95fd700 1 -- 172.21.15.170:0/2568025724 wait complete. 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.028+0000 7f52c95fd700 1 Processor -- start 2023-10-08T16:05:05.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.028+0000 7f52c95fd700 1 -- start start 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c41541b0 0x7f52c40c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c4154620 0x7f52c40c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b36c0 con 0x7f52ac003a60 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b2ab0 con 0x7f52c41541b0 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c95fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40b2d90 con 0x7f52c4154620 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53826/0 (socket says 172.21.15.170:53826) 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2837550701 learned_addr learned my addr 172.21.15.170:0/2837550701 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c41541b0 0x7f52c40c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.029+0000 7f52c37fe700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c4154620 0x7f52c40c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c4154620 msgr2=0x7f52c40c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c4154620 0x7f52c40c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c41541b0 msgr2=0x7f52c40c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c41541b0 0x7f52c40c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 -- 172.21.15.170:0/2837550701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40b3070 con 0x7f52ac003a60 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c2ffd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52b400a430 tx=0x7f52b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4012940 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b401f410 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b401f7d0 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40b3350 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.030+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c40b26f0 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.031+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c40c9e10 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.031+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f52b401fbc0 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.031+0000 7f52aaffd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.032+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f52b4086670 con 0x7f52ac003a60 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.032+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.032+0000 7f52c27fc700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 0x7f52b00693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52c414fb70 tx=0x7f52b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.036+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52b40538e0 con 0x7f52ac003a60 2023-10-08T16:05:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.216+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52c404c3c0 con 0x7f52ac003a60 2023-10-08T16:05:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.217+0000 7f52aaffd700 1 -- 172.21.15.170:0/2837550701 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52b4056f00 con 0x7f52ac003a60 2023-10-08T16:05:05.218 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 msgr2=0x7f52b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 0x7f52b00693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52c414fb70 tx=0x7f52b8008040 comp rx=0 tx=0).stop 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 msgr2=0x7f52c414c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52b400a430 tx=0x7f52b4015000 comp rx=0 tx=0).stop 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 shutdown_connections 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f52c4154620 0x7f52c40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f52b0066f30 0x7f52b00693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52ac003a60 0x7f52c414c2a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 --2- 172.21.15.170:0/2837550701 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f52c41541b0 0x7f52c40c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 >> 172.21.15.170:0/2837550701 conn(0x7f52c40ce600 msgr2=0x7f52c41572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 shutdown_connections 2023-10-08T16:05:05.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.219+0000 7f52c95fd700 1 -- 172.21.15.170:0/2837550701 wait complete. 2023-10-08T16:05:05.232 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:05:05.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.348+0000 7fca2c811700 1 Processor -- start 2023-10-08T16:05:05.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.348+0000 7fca2c811700 1 -- start start 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca2c811700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca2806c0a0 0x7fca280caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca2c811700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fca2815df50 0x7fca2815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca2c811700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2815d7e0 con 0x7fca2806bcd0 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca2c811700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2815dac0 con 0x7fca2815df50 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca2c811700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2815aa30 con 0x7fca2806c0a0 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fca2815df50 0x7fca2815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fca2815df50 0x7fca2815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54322/0 (socket says 172.21.15.170:54322) 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 -- 172.21.15.170:0/110049383 learned_addr learned my addr 172.21.15.170:0/110049383 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca1dd9b700 1 --2- 172.21.15.170:0/110049383 >> v2:172.21.15.191:3300/0 conn(0x7fca2806c0a0 0x7fca280caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 -- 172.21.15.170:0/110049383 >> v2:172.21.15.191:3300/0 conn(0x7fca2806c0a0 msgr2=0x7fca280caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 --2- 172.21.15.170:0/110049383 >> v2:172.21.15.191:3300/0 conn(0x7fca2806c0a0 0x7fca280caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 -- 172.21.15.170:0/110049383 >> v1:172.21.15.191:6789/0 conn(0x7fca2806bcd0 legacy=0x7fca2815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.349+0000 7fca26d9d700 1 -- 172.21.15.170:0/110049383 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2815ad10 con 0x7fca2815df50 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca26d9d700 1 --2- 172.21.15.170:0/110049383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca2815e3b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca2806b4e0 tx=0x7fca20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c34fb45e3e72b79 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca25d9b700 1 -- 172.21.15.170:0/110049383 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000e070 con 0x7fca2815df50 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca25d9b700 1 -- 172.21.15.170:0/110049383 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000ba20 con 0x7fca2815df50 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca25d9b700 1 -- 172.21.15.170:0/110049383 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca200092e0 con 0x7fca2815df50 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 -- 172.21.15.170:0/110049383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 msgr2=0x7fca2815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 --2- 172.21.15.170:0/110049383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca2815e3b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca2806b4e0 tx=0x7fca20009840 comp rx=0 tx=0).stop 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 -- 172.21.15.170:0/110049383 shutdown_connections 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 --2- 172.21.15.170:0/110049383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca2815e3b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 --2- 172.21.15.170:0/110049383 >> v2:172.21.15.191:3300/0 conn(0x7fca2806c0a0 0x7fca280caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 -- 172.21.15.170:0/110049383 >> 172.21.15.170:0/110049383 conn(0x7fca280ce600 msgr2=0x7fca280cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 -- 172.21.15.170:0/110049383 shutdown_connections 2023-10-08T16:05:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.350+0000 7fca2c811700 1 -- 172.21.15.170:0/110049383 wait complete. 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 Processor -- start 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 -- start start 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2806bcd0 0x7fca281549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca2806c0a0 0x7fca28157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c7cf0 con 0x7fca2815df50 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c70e0 con 0x7fca2806c0a0 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca2c811700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280c73c0 con 0x7fca2806bcd0 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca26d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca26d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54336/0 (socket says 172.21.15.170:54336) 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.352+0000 7fca26d9d700 1 -- 172.21.15.170:0/2073283117 learned_addr learned my addr 172.21.15.170:0/2073283117 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca1dd9b700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca2806c0a0 0x7fca28157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca2659c700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2806bcd0 0x7fca281549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 -- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2806bcd0 msgr2=0x7fca281549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2806bcd0 0x7fca281549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 -- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca2806c0a0 msgr2=0x7fca28157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca2806c0a0 0x7fca28157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 -- 172.21.15.170:0/2073283117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca20009ae0 con 0x7fca2815df50 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca26d9d700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b6c0 tx=0x7fca20018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000be50 con 0x7fca2815df50 2023-10-08T16:05:05.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca20007040 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280c76a0 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2001c410 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.353+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca28154f30 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.354+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2804c3c0 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.355+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca20003050 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.355+0000 7fca1effd700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.355+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca20085de0 con 0x7fca2815df50 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.355+0000 7fca2659c700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.355+0000 7fca2659c700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 0x7fca080693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca180039e0 tx=0x7fca18007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.359+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca20053050 con 0x7fca2815df50 2023-10-08T16:05:05.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.540+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca280c7e30 con 0x7fca2815df50 2023-10-08T16:05:05.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.541+0000 7fca1effd700 1 -- 172.21.15.170:0/2073283117 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca20056670 con 0x7fca2815df50 2023-10-08T16:05:05.541 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:05:05.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.542+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 msgr2=0x7fca080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.542+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 0x7fca080693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fca180039e0 tx=0x7fca18007ec0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.542+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 msgr2=0x7fca280c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.542+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fca2000b6c0 tx=0x7fca20018000 comp rx=0 tx=0).stop 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 shutdown_connections 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca08066f30 0x7fca080693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca2815df50 0x7fca280c8180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca2806bcd0 0x7fca281549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 --2- 172.21.15.170:0/2073283117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca2806c0a0 0x7fca28157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 >> 172.21.15.170:0/2073283117 conn(0x7fca280ce600 msgr2=0x7fca280c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 shutdown_connections 2023-10-08T16:05:05.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.543+0000 7fca2c811700 1 -- 172.21.15.170:0/2073283117 wait complete. 2023-10-08T16:05:05.555 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:05:05.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.672+0000 7fb11f133700 1 Processor -- start 2023-10-08T16:05:05.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.672+0000 7fb11f133700 1 -- start start 2023-10-08T16:05:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 0x7fb11815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb118159c60 0x7fb11815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 0x7fb118069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11815df50 con 0x7fb118159c60 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11815a160 con 0x7fb118159890 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11f133700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180d13a0 con 0x7fb11815db10 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb118159c60 0x7fb11815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb118159c60 0x7fb11815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54340/0 (socket says 172.21.15.170:54340) 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 -- 172.21.15.170:0/3455297091 learned_addr learned my addr 172.21.15.170:0/3455297091 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11d6d0700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 0x7fb118069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11cecf700 1 --2- 172.21.15.170:0/3455297091 >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 0x7fb11815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 -- 172.21.15.170:0/3455297091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 msgr2=0x7fb118069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 0x7fb118069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 -- 172.21.15.170:0/3455297091 >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 msgr2=0x7fb11815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 --2- 172.21.15.170:0/3455297091 >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 0x7fb11815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 -- 172.21.15.170:0/3455297091 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11815e140 con 0x7fb118159c60 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11d6d0700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 0x7fb118069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb11cecf700 1 --2- 172.21.15.170:0/3455297091 >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 0x7fb11815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb117fff700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb118159c60 0x7fb11815d5d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1180d1090 tx=0x7fb104009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea9c54afd0b3971 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb1177fe700 1 -- 172.21.15.170:0/3455297091 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400b860 con 0x7fb118159c60 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb1177fe700 1 -- 172.21.15.170:0/3455297091 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10400b9c0 con 0x7fb118159c60 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.673+0000 7fb1177fe700 1 -- 172.21.15.170:0/3455297091 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400bd60 con 0x7fb118159c60 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 -- 172.21.15.170:0/3455297091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb118159c60 msgr2=0x7fb11815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb118159c60 0x7fb11815d5d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb1180d1090 tx=0x7fb104009250 comp rx=0 tx=0).stop 2023-10-08T16:05:05.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 -- 172.21.15.170:0/3455297091 shutdown_connections 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11815db10 0x7fb118069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 --2- 172.21.15.170:0/3455297091 >> v2:172.21.15.191:3300/0 conn(0x7fb118159890 0x7fb11815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 --2- 172.21.15.170:0/3455297091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb118159c60 0x7fb11815d5d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 -- 172.21.15.170:0/3455297091 >> 172.21.15.170:0/3455297091 conn(0x7fb118068e20 msgr2=0x7fb1180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 -- 172.21.15.170:0/3455297091 shutdown_connections 2023-10-08T16:05:05.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.674+0000 7fb11f133700 1 -- 172.21.15.170:0/3455297091 wait complete. 2023-10-08T16:05:05.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.675+0000 7fb11f133700 1 Processor -- start 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.675+0000 7fb11f133700 1 -- start start 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 0x7fb11814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 0x7fb1180b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180c6320 con 0x7fb11815db10 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180c5710 con 0x7fb118159890 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11f133700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180c59f0 con 0x7fb118159c60 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54344/0 (socket says 172.21.15.170:54344) 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 -- 172.21.15.170:0/1507951769 learned_addr learned my addr 172.21.15.170:0/1507951769 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb117fff700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 0x7fb1180b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11cecf700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 0x7fb11814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 -- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 msgr2=0x7fb1180b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 0x7fb1180b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 -- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 msgr2=0x7fb11814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 0x7fb11814ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11d6d0700 1 -- 172.21.15.170:0/1507951769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040094f0 con 0x7fb11815db10 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb11cecf700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 0x7fb11814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.676+0000 7fb117fff700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 0x7fb1180b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb11d6d0700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb11000cef0 tx=0x7fb110007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11000ae00 con 0x7fb11815db10 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb110016420 con 0x7fb11815db10 2023-10-08T16:05:05.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1180c5cd0 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1100167e0 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.677+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1180b0ec0 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.678+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1180c6490 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.678+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb110016bd0 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.678+0000 7fb114ff9700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 0x7fb0fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.678+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb110012070 con 0x7fb11815db10 2023-10-08T16:05:05.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.678+0000 7fb11cecf700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 0x7fb0fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.679+0000 7fb11cecf700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 0x7fb0fc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00ab50 tx=0x7fb10c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.683+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb11004e5a0 con 0x7fb11815db10 2023-10-08T16:05:05.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.863+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb11804c3c0 con 0x7fb11815db10 2023-10-08T16:05:05.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.864+0000 7fb114ff9700 1 -- 172.21.15.170:0/1507951769 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb110052140 con 0x7fb11815db10 2023-10-08T16:05:05.864 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 msgr2=0x7fb0fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 0x7fb0fc0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00ab50 tx=0x7fb10c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 msgr2=0x7fb1180c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb11000cef0 tx=0x7fb110007860 comp rx=0 tx=0).stop 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 shutdown_connections 2023-10-08T16:05:05.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0fc066f30 0x7fb0fc0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb11815db10 0x7fb1180c67b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb118159890 0x7fb11814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 --2- 172.21.15.170:0/1507951769 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb118159c60 0x7fb1180b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.865+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 >> 172.21.15.170:0/1507951769 conn(0x7fb118068e20 msgr2=0x7fb1181573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.866+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 shutdown_connections 2023-10-08T16:05:05.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.866+0000 7fb11f133700 1 -- 172.21.15.170:0/1507951769 wait complete. 2023-10-08T16:05:05.878 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:05.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.991+0000 7f9aebfff700 1 Processor -- start 2023-10-08T16:05:05.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.991+0000 7f9aebfff700 1 -- start start 2023-10-08T16:05:05.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aebfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 0x7f9aec0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aebfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9aec1597f0 0x7f9aec0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aebfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aec0b11d0 con 0x7f9aec15d230 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aebfff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0b14b0 con 0x7f9aec1597f0 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aebfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aec0b1790 con 0x7f9aec159420 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9aec1597f0 0x7f9aec0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9aec1597f0 0x7f9aec0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54358/0 (socket says 172.21.15.170:54358) 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 -- 172.21.15.170:0/1648736524 learned_addr learned my addr 172.21.15.170:0/1648736524 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aeaffd700 1 --2- 172.21.15.170:0/1648736524 >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 0x7f9aec0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 -- 172.21.15.170:0/1648736524 >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 msgr2=0x7f9aec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 --2- 172.21.15.170:0/1648736524 >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 0x7f9aec0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 -- 172.21.15.170:0/1648736524 >> v1:172.21.15.191:6789/0 conn(0x7f9aec15d230 legacy=0x7f9aec15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:05.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 -- 172.21.15.170:0/1648736524 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0b1a70 con 0x7f9aec1597f0 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aeaffd700 1 --2- 172.21.15.170:0/1648736524 >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 0x7f9aec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9aea7fc700 1 --2- 172.21.15.170:0/1648736524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec0b2270 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0b2f20 tx=0x7f9ae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80838bbc76b123f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9ae9ffb700 1 -- 172.21.15.170:0/1648736524 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000b820 con 0x7f9aec1597f0 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9ae9ffb700 1 -- 172.21.15.170:0/1648736524 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae000b980 con 0x7f9aec1597f0 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.992+0000 7f9ae9ffb700 1 -- 172.21.15.170:0/1648736524 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000bd20 con 0x7f9aec1597f0 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 -- 172.21.15.170:0/1648736524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 msgr2=0x7f9aec0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 --2- 172.21.15.170:0/1648736524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec0b2270 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9aec0b2f20 tx=0x7f9ae0009250 comp rx=0 tx=0).stop 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 -- 172.21.15.170:0/1648736524 shutdown_connections 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 --2- 172.21.15.170:0/1648736524 >> v2:172.21.15.191:3300/0 conn(0x7f9aec159420 0x7f9aec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 --2- 172.21.15.170:0/1648736524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec0b2270 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:05.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 -- 172.21.15.170:0/1648736524 >> 172.21.15.170:0/1648736524 conn(0x7f9aec0d0660 msgr2=0x7f9aec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 -- 172.21.15.170:0/1648736524 shutdown_connections 2023-10-08T16:05:05.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.993+0000 7f9aebfff700 1 -- 172.21.15.170:0/1648736524 wait complete. 2023-10-08T16:05:05.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.994+0000 7f9aebfff700 1 Processor -- start 2023-10-08T16:05:05.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.994+0000 7f9aebfff700 1 -- start start 2023-10-08T16:05:05.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9aec159420 0x7f9aec152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9aec15d230 0x7f9aec155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec0c57a0 con 0x7f9aec1597f0 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec0c54a0 con 0x7f9aec159420 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aebfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aec1534d0 con 0x7f9aec15d230 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54364/0 (socket says 172.21.15.170:54364) 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 -- 172.21.15.170:0/2501653458 learned_addr learned my addr 172.21.15.170:0/2501653458 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aeaffd700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9aec159420 0x7f9aec152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aeb7fe700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9aec15d230 0x7f9aec155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 -- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9aec15d230 msgr2=0x7f9aec155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:05.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9aec15d230 0x7f9aec155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 -- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9aec159420 msgr2=0x7f9aec152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9aec159420 0x7f9aec152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 -- 172.21.15.170:0/2501653458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae00094f0 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9aea7fc700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006fd0 tx=0x7f9ae0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0011d10 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.995+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae0020070 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.996+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aec15b640 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.996+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae001b6a0 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.996+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aec15bbc0 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.996+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aec0c5950 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.997+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ae0022030 con 0x7f9aec1597f0 2023-10-08T16:05:06.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.997+0000 7f9ad2ffd700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 0x7f9ad80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.997+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9ae0086520 con 0x7f9aec1597f0 2023-10-08T16:05:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.997+0000 7f9aeaffd700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 0x7f9ad80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:05.998+0000 7f9aeaffd700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 0x7f9ad80693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00ac00 tx=0x7f9adc006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.001+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9aec0c5950 con 0x7f9aec1597f0 2023-10-08T16:05:06.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.223+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aec0c5950 con 0x7f9aec1597f0 2023-10-08T16:05:06.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.224+0000 7f9ad2ffd700 1 -- 172.21.15.170:0/2501653458 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f9aec0c5950 con 0x7f9aec1597f0 2023-10-08T16:05:06.226 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:06.226 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 msgr2=0x7f9ad80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 0x7f9ad80693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00ac00 tx=0x7f9adc006f90 comp rx=0 tx=0).stop 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 msgr2=0x7f9aec15c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006fd0 tx=0x7f9ae0016000 comp rx=0 tx=0).stop 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 shutdown_connections 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9aec15d230 0x7f9aec155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9aec159420 0x7f9aec152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9ad8066f30 0x7f9ad80693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 --2- 172.21.15.170:0/2501653458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9aec1597f0 0x7f9aec15c320 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 >> 172.21.15.170:0/2501653458 conn(0x7f9aec0d0660 msgr2=0x7f9aec15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:06.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 shutdown_connections 2023-10-08T16:05:06.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.226+0000 7f9aebfff700 1 -- 172.21.15.170:0/2501653458 wait complete. 2023-10-08T16:05:06.228 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:05:06.239 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:06.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.352+0000 7f3ce5e78700 1 Processor -- start 2023-10-08T16:05:06.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.352+0000 7f3ce5e78700 1 -- start start 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5e78700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 0x7f3ce00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5e78700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3ce015d230 0x7f3ce015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5e78700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce015db70 con 0x7f3ce0159420 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5e78700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00b11d0 con 0x7f3ce015d230 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5e78700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce00b14b0 con 0x7f3ce01597f0 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3ce015d230 0x7f3ce015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3ce015d230 0x7f3ce015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54372/0 (socket says 172.21.15.170:54372) 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 -- 172.21.15.170:0/4130733973 learned_addr learned my addr 172.21.15.170:0/4130733973 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 -- 172.21.15.170:0/4130733973 >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 msgr2=0x7f3ce00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3cd7fff700 1 --2- 172.21.15.170:0/4130733973 >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 0x7f3ce00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 --2- 172.21.15.170:0/4130733973 >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 0x7f3ce00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 -- 172.21.15.170:0/4130733973 >> v1:172.21.15.191:6789/0 conn(0x7f3ce0159420 legacy=0x7f3ce00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3ce5677700 1 -- 172.21.15.170:0/4130733973 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce00b1790 con 0x7f3ce015d230 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.353+0000 7f3cd7fff700 1 --2- 172.21.15.170:0/4130733973 >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 0x7f3ce00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5677700 1 --2- 172.21.15.170:0/4130733973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce015d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0089750 tx=0x7f3cd8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670f231c09592c0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3cd77fe700 1 -- 172.21.15.170:0/4130733973 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800b820 con 0x7f3ce015d230 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3cd77fe700 1 -- 172.21.15.170:0/4130733973 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd800b980 con 0x7f3ce015d230 2023-10-08T16:05:06.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3cd77fe700 1 -- 172.21.15.170:0/4130733973 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd800bd20 con 0x7f3ce015d230 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 -- 172.21.15.170:0/4130733973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 msgr2=0x7f3ce015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/4130733973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce015d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0089750 tx=0x7f3cd8009250 comp rx=0 tx=0).stop 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 -- 172.21.15.170:0/4130733973 shutdown_connections 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/4130733973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce015d630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/4130733973 >> v2:172.21.15.191:3300/0 conn(0x7f3ce01597f0 0x7f3ce00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 -- 172.21.15.170:0/4130733973 >> 172.21.15.170:0/4130733973 conn(0x7f3ce00d0660 msgr2=0x7f3ce00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 -- 172.21.15.170:0/4130733973 shutdown_connections 2023-10-08T16:05:06.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.354+0000 7f3ce5e78700 1 -- 172.21.15.170:0/4130733973 wait complete. 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.355+0000 7f3ce5e78700 1 Processor -- start 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 -- start start 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0159420 0x7f3ce0169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce01597f0 0x7f3ce01545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015b830 con 0x7f3ce015d230 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce016a350 con 0x7f3ce01597f0 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5e78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce015abb0 con 0x7f3ce0159420 2023-10-08T16:05:06.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5677700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5677700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54384/0 (socket says 172.21.15.170:54384) 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce5677700 1 -- 172.21.15.170:0/900578178 learned_addr learned my addr 172.21.15.170:0/900578178 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3cd7fff700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce01597f0 0x7f3ce01545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.356+0000 7f3ce4e76700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0159420 0x7f3ce0169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 -- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0159420 msgr2=0x7f3ce0169e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0159420 0x7f3ce0169e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 -- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce01597f0 msgr2=0x7f3ce01545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce01597f0 0x7f3ce01545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 -- 172.21.15.170:0/900578178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80094f0 con 0x7f3ce015d230 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5677700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8006fd0 tx=0x7f3cd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8011e80 con 0x7f3ce015d230 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd8020070 con 0x7f3ce015d230 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd801b6a0 con 0x7f3ce015d230 2023-10-08T16:05:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce015ae90 con 0x7f3ce015d230 2023-10-08T16:05:06.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.357+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce015b450 con 0x7f3ce015d230 2023-10-08T16:05:06.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.358+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce004c3c0 con 0x7f3ce015d230 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.358+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cd8022030 con 0x7f3ce015d230 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.359+0000 7f3cd4ff9700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 0x7f3ccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.359+0000 7f3ce4e76700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 0x7f3ccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.359+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3cd8086490 con 0x7f3ce015d230 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.359+0000 7f3ce4e76700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 0x7f3ccc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0075c0 tx=0x7f3cdc009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.363+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cd8053700 con 0x7f3ce015d230 2023-10-08T16:05:06.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.547+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ce004c3c0 con 0x7f3ce015d230 2023-10-08T16:05:06.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.548+0000 7f3cd4ff9700 1 -- 172.21.15.170:0/900578178 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f3cd8056d20 con 0x7f3ce015d230 2023-10-08T16:05:06.548 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:06.549 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 msgr2=0x7f3ccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 0x7f3ccc0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0075c0 tx=0x7f3cdc009780 comp rx=0 tx=0).stop 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 msgr2=0x7f3ce0154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8006fd0 tx=0x7f3cd8016000 comp rx=0 tx=0).stop 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 shutdown_connections 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ccc066f30 0x7f3ccc0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ce015d230 0x7f3ce0154af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ce0159420 0x7f3ce0169e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 --2- 172.21.15.170:0/900578178 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ce01597f0 0x7f3ce01545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.550+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 >> 172.21.15.170:0/900578178 conn(0x7f3ce00d0660 msgr2=0x7f3ce0069b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:06.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.551+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 shutdown_connections 2023-10-08T16:05:06.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.551+0000 7f3ce5e78700 1 -- 172.21.15.170:0/900578178 wait complete. 2023-10-08T16:05:06.563 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:05:06.563 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:06.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.677+0000 7f592bdb8700 1 Processor -- start 2023-10-08T16:05:06.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.678+0000 7f592bdb8700 1 -- start start 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.678+0000 7f592bdb8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f59240c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.678+0000 7f592bdb8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592415d7e0 con 0x7f592406bcd0 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.678+0000 7f592bdb8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592415dac0 con 0x7f592406c0a0 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.678+0000 7f592bdb8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59240caf30 con 0x7f59240caa90 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5929b54700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f592406bcd0 0x7f592415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42310/0 (socket says 172.21.15.170:42310) 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5929b54700 1 -- 172.21.15.170:0/3077245609 learned_addr learned my addr 172.21.15.170:0/3077245609 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:06.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f592a355700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f59240c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037052359 0 0) 0x7f592415d7e0 con 0x7f592406bcd0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5910003570 con 0x7f592406bcd0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734436400 0 0) 0x7f592415dac0 con 0x7f592406c0a0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f592415d7e0 con 0x7f592406c0a0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340404781 0 0) 0x7f5910003570 con 0x7f592406bcd0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 msgr2=0x7f59240c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f59240c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 >> v1:172.21.15.191:6789/0 conn(0x7f592406c0a0 legacy=0x7f592415df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592415aa30 con 0x7f592406bcd0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f5920001470 con 0x7f592406bcd0 2023-10-08T16:05:06.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 >> v1:172.21.15.170:6789/0 conn(0x7f592406bcd0 legacy=0x7f592415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910003c20 0x7f5910004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910004810 0x7f5910004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f5910005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59240caf30 con 0x7f5910004810 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591000bbf0 con 0x7f5910003c20 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591000bdb0 con 0x7f59240caa90 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5929353700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910004810 0x7f5910004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f5929b54700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910003c20 0x7f5910004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.679+0000 7f592a355700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f5910005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 -- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 msgr2=0x7f5910005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:06.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f5910005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 -- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910003c20 msgr2=0x7f5910004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910003c20 0x7f5910004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 -- 172.21.15.170:0/3077245609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910003570 con 0x7f5910004810 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592a355700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59240caa90 0x7f5910005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5929353700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910004810 0x7f5910004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5914001a20 tx=0x7f5914007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e2a640ba99b507 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400c5d0 con 0x7f5910004810 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591400c730 con 0x7f5910004810 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f5928b52700 1 -- 172.21.15.170:0/3077245609 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591400cad0 con 0x7f5910004810 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592bdb8700 1 -- 172.21.15.170:0/3077245609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910004810 msgr2=0x7f5910004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592bdb8700 1 --2- 172.21.15.170:0/3077245609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910004810 0x7f5910004c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5914001a20 tx=0x7f5914007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:06.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592a355700 1 -- 172.21.15.170:0/3077245609 reap_dead start 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592bdb8700 1 -- 172.21.15.170:0/3077245609 shutdown_connections 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592bdb8700 1 -- 172.21.15.170:0/3077245609 >> 172.21.15.170:0/3077245609 conn(0x7f59240ce600 msgr2=0x7f59240cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.680+0000 7f592bdb8700 1 -- 172.21.15.170:0/3077245609 shutdown_connections 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.681+0000 7f592bdb8700 1 -- 172.21.15.170:0/3077245609 wait complete. 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 Processor -- start 2023-10-08T16:05:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 -- start start 2023-10-08T16:05:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910004810 0x7f592416aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 0x7f59241698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592416a8b0 con 0x7f592416c590 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5924169de0 con 0x7f5910004810 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592bdb8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59240b1540 con 0x7f5924155a30 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592a355700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592a355700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54410/0 (socket says 172.21.15.170:54410) 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f592a355700 1 -- 172.21.15.170:0/1133047617 learned_addr learned my addr 172.21.15.170:0/1133047617 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.682+0000 7f5929b54700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910004810 0x7f592416aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f5929353700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 0x7f59241698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 -- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 msgr2=0x7f59241698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 0x7f59241698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 -- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910004810 msgr2=0x7f592416aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910004810 0x7f592416aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 -- 172.21.15.170:0/1133047617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924155ea0 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f5929353700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 0x7f59241698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592a355700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f592406b640 tx=0x7f591c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00e050 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c00f040 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592416a010 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c010df0 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.683+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592416a5d0 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.684+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59240b13a0 con 0x7f592416c590 2023-10-08T16:05:06.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.685+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f591c003020 con 0x7f592416c590 2023-10-08T16:05:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.685+0000 7f5919ffb700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 0x7f59080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.685+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f591c0847c0 con 0x7f592416c590 2023-10-08T16:05:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.685+0000 7f5929b54700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 0x7f59080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.685+0000 7f5929b54700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 0x7f59080693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5920001200 tx=0x7f5920001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.689+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59240b13a0 con 0x7f592416c590 2023-10-08T16:05:06.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.914+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59240b13a0 con 0x7f592416c590 2023-10-08T16:05:06.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.914+0000 7f5919ffb700 1 -- 172.21.15.170:0/1133047617 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f59240b13a0 con 0x7f592416c590 2023-10-08T16:05:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:06.915 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 msgr2=0x7f59080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 0x7f59080693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5920001200 tx=0x7f5920001df0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 msgr2=0x7f59240c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f592406b640 tx=0x7f591c00b040 comp rx=0 tx=0).stop 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 shutdown_connections 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5908066f30 0x7f59080693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f592416c590 0x7f59240c4520 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910004810 0x7f592416aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 --2- 172.21.15.170:0/1133047617 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5924155a30 0x7f59241698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 >> 172.21.15.170:0/1133047617 conn(0x7f59240ce600 msgr2=0x7f592415c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 shutdown_connections 2023-10-08T16:05:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:06.916+0000 7f592bdb8700 1 -- 172.21.15.170:0/1133047617 wait complete. 2023-10-08T16:05:06.919 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:05:06.929 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:07.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.042+0000 7f805ac2b700 1 Processor -- start 2023-10-08T16:05:07.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.042+0000 7f805ac2b700 1 -- start start 2023-10-08T16:05:07.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f805ac2b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8054141500 0x7f80540c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f805ac2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406c0a0 0x7f80540c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f805ac2b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805415c2a0 con 0x7f805406bcd0 2023-10-08T16:05:07.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f805ac2b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80540c9880 con 0x7f8054141500 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f805ac2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805415b690 con 0x7f805406c0a0 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f8053fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8054141500 0x7f80540c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f80589c7700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f805406bcd0 0x7f8054140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42324/0 (socket says 172.21.15.170:42324) 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.043+0000 7f80589c7700 1 -- 172.21.15.170:0/281764820 learned_addr learned my addr 172.21.15.170:0/281764820 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f8053fff700 1 -- 172.21.15.170:0/281764820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406c0a0 msgr2=0x7f80540c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f80591c8700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406c0a0 0x7f80540c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f8053fff700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406c0a0 0x7f80540c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f8053fff700 1 -- 172.21.15.170:0/281764820 >> v1:172.21.15.170:6789/0 conn(0x7f805406bcd0 legacy=0x7f8054140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f80537fe700 1 -- 172.21.15.170:0/281764820 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709096011 0 0) 0x7f805415c2a0 con 0x7f805406bcd0 2023-10-08T16:05:07.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f8053fff700 1 -- 172.21.15.170:0/281764820 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805415b970 con 0x7f8054141500 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f8053fff700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8054141500 0x7f80540c8e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368f382c0f257bfb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f80537fe700 1 -- 172.21.15.170:0/281764820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c009e80 con 0x7f8054141500 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f80537fe700 1 -- 172.21.15.170:0/281764820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c00b860 con 0x7f8054141500 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f80537fe700 1 -- 172.21.15.170:0/281764820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00bbe0 con 0x7f8054141500 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f805ac2b700 1 -- 172.21.15.170:0/281764820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8054141500 msgr2=0x7f80540c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f805ac2b700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8054141500 0x7f80540c8e00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f803c0098e0 tx=0x7f803c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f805ac2b700 1 -- 172.21.15.170:0/281764820 shutdown_connections 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.044+0000 7f805ac2b700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406c0a0 0x7f80540c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.045+0000 7f805ac2b700 1 --2- 172.21.15.170:0/281764820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8054141500 0x7f80540c8e00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.045+0000 7f805ac2b700 1 -- 172.21.15.170:0/281764820 >> 172.21.15.170:0/281764820 conn(0x7f80540ce600 msgr2=0x7f80540cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.045+0000 7f805ac2b700 1 -- 172.21.15.170:0/281764820 shutdown_connections 2023-10-08T16:05:07.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.045+0000 7f805ac2b700 1 -- 172.21.15.170:0/281764820 wait complete. 2023-10-08T16:05:07.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.046+0000 7f805ac2b700 1 Processor -- start 2023-10-08T16:05:07.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.046+0000 7f805ac2b700 1 -- start start 2023-10-08T16:05:07.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406bcd0 0x7f8054069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 0x7f80541582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805406b1c0 con 0x7f805406c0a0 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805406a5b0 con 0x7f8054141500 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805406a890 con 0x7f805406bcd0 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54422/0 (socket says 172.21.15.170:54422) 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 -- 172.21.15.170:0/2765243828 learned_addr learned my addr 172.21.15.170:0/2765243828 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f80591c8700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 0x7f80541582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 -- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406bcd0 msgr2=0x7f8054069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f80589c7700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406bcd0 0x7f8054069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406bcd0 0x7f8054069eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 -- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 msgr2=0x7f80541582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 0x7f80541582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 -- 172.21.15.170:0/2765243828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c009420 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f80591c8700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 0x7f80541582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8053fff700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8054069220 tx=0x7f803c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00bad0 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c025420 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805406ab70 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0257e0 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.047+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80541587e0 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.048+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80540b19e0 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.049+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f803c023070 con 0x7f805406c0a0 2023-10-08T16:05:07.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.049+0000 7f8050ff9700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 0x7f80340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.049+0000 7f80589c7700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 0x7f80340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.049+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f803c085c00 con 0x7f805406c0a0 2023-10-08T16:05:07.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.050+0000 7f80589c7700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 0x7f80340693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8044002220 tx=0x7f8044009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.054+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80540b19e0 con 0x7f805406c0a0 2023-10-08T16:05:07.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.277+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80540b19e0 con 0x7f805406c0a0 2023-10-08T16:05:07.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.277+0000 7f8050ff9700 1 -- 172.21.15.170:0/2765243828 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f80540b19e0 con 0x7f805406c0a0 2023-10-08T16:05:07.278 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:07.278 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3646622121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3646622121},{"type":"v1","addr":"172.21.15.191:6834","nonce":3646622121}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1983012215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1983012215},{"type":"v1","addr":"172.21.15.191:6835","nonce":1983012215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3121234053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3121234053},{"type":"v1","addr":"172.21.15.170:6836","nonce":3121234053}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:04:36.203197+0000","modified":"2023-10-08T16:04:39.853349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/2316279916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2316279916},{"type":"v1","addr":"172.21.15.170:6837","nonce":2316279916}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T16:05:07.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 msgr2=0x7f80340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 0x7f80340693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8044002220 tx=0x7f8044009040 comp rx=0 tx=0).stop 2023-10-08T16:05:07.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 msgr2=0x7f80540c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8054069220 tx=0x7f803c017000 comp rx=0 tx=0).stop 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 shutdown_connections 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8054141500 0x7f80541582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f805406bcd0 0x7f8054069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8034066f30 0x7f80340693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 --2- 172.21.15.170:0/2765243828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f805406c0a0 0x7f80540c8280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 >> 172.21.15.170:0/2765243828 conn(0x7f80540ce600 msgr2=0x7f80540c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.279+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 shutdown_connections 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.280+0000 7f805ac2b700 1 -- 172.21.15.170:0/2765243828 wait complete. 2023-10-08T16:05:07.281 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 8 2023-10-08T16:05:07.292 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:07.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.405+0000 7f627e39b700 1 Processor -- start 2023-10-08T16:05:07.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.405+0000 7f627e39b700 1 -- start start 2023-10-08T16:05:07.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627e39b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6278159890 0x7f627815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627e39b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627815a160 con 0x7f6278069560 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627e39b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62780d13a0 con 0x7f6278159c60 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627e39b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627815e140 con 0x7f6278159890 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627c938700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6278069560 0x7f62780ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42340/0 (socket says 172.21.15.170:42340) 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.406+0000 7f627c938700 1 -- 172.21.15.170:0/949019376 learned_addr learned my addr 172.21.15.170:0/949019376 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6277fff700 1 --2- 172.21.15.170:0/949019376 >> v2:172.21.15.191:3300/0 conn(0x7f6278159890 0x7f627815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 252876062 0 0) 0x7f627815a160 con 0x7f6278069560 2023-10-08T16:05:07.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6260003570 con 0x7f6278069560 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3281933217 0 0) 0x7f6260003570 con 0x7f6278069560 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 >> v2:172.21.15.191:3300/0 conn(0x7f6278159890 msgr2=0x7f627815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 --2- 172.21.15.170:0/949019376 >> v2:172.21.15.191:3300/0 conn(0x7f6278159890 0x7f627815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 >> v1:172.21.15.191:6789/0 conn(0x7f6278159c60 legacy=0x7f627815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627815a3d0 con 0x7f6278069560 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f62700017f0 con 0x7f6278069560 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 >> v1:172.21.15.170:6789/0 conn(0x7f6278069560 legacy=0x7f62780ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6260003a60 0x7f6260003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6260004810 0x7f6260004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 0x7f626000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815e140 con 0x7f62600051c0 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626000c0d0 con 0x7f6260004810 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626000c290 con 0x7f6260003a60 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 0x7f626000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f6277fff700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6260003a60 0x7f6260003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f62777fe700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6260004810 0x7f6260004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 -- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6260003a60 msgr2=0x7f6260003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6260003a60 0x7f6260003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 -- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6260004810 msgr2=0x7f6260004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6260004810 0x7f6260004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 -- 172.21.15.170:0/949019376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260003570 con 0x7f62600051c0 2023-10-08T16:05:07.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f62777fe700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6260004810 0x7f6260004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 -- 172.21.15.170:0/949019376 reap_dead start 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.407+0000 7f627c938700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 0x7f626000ba60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f627000a930 tx=0x7f62700021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d02b53d65ad41 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62700070d0 con 0x7f62600051c0 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6270007230 con 0x7f62600051c0 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f6276ffd700 1 -- 172.21.15.170:0/949019376 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62700075d0 con 0x7f62600051c0 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 -- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 msgr2=0x7f626000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 0x7f626000ba60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f627000a930 tx=0x7f62700021d0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 -- 172.21.15.170:0/949019376 shutdown_connections 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 --2- 172.21.15.170:0/949019376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62600051c0 0x7f626000ba60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 -- 172.21.15.170:0/949019376 >> 172.21.15.170:0/949019376 conn(0x7f6278068e20 msgr2=0x7f62780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 -- 172.21.15.170:0/949019376 shutdown_connections 2023-10-08T16:05:07.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.408+0000 7f627e39b700 1 -- 172.21.15.170:0/949019376 wait complete. 2023-10-08T16:05:07.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.409+0000 7f627e39b700 1 Processor -- start 2023-10-08T16:05:07.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.409+0000 7f627e39b700 1 -- start start 2023-10-08T16:05:07.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62600051c0 0x7f62781493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62780b4440 0x7f62780b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62780b24e0 con 0x7f62780b1b30 2023-10-08T16:05:07.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62780cde00 con 0x7f62780b4440 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627e39b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62780cd1f0 con 0x7f62600051c0 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54444/0 (socket says 172.21.15.170:54444) 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 -- 172.21.15.170:0/129172945 learned_addr learned my addr 172.21.15.170:0/129172945 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f6277fff700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62600051c0 0x7f62781493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f627c938700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62780b4440 0x7f62780b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 -- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62600051c0 msgr2=0x7f62781493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62600051c0 0x7f62781493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 -- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62780b4440 msgr2=0x7f62780b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62780b4440 0x7f62780b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 -- 172.21.15.170:0/129172945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62780cd4d0 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.410+0000 7f62777fe700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e550 tx=0x7f626c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.411+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c011460 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.411+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c010040 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.411+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62780cd7b0 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.411+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c0117f0 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.411+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62780b4df0 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.412+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62780cdf70 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.412+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f626c011b60 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.412+0000 7f625ffff700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 0x7f62640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.412+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f626c083730 con 0x7f62780b1b30 2023-10-08T16:05:07.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.412+0000 7f6277fff700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 0x7f62640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.413+0000 7f6277fff700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 0x7f62640693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62780b4380 tx=0x7f62680091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.417+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f626c04d5a0 con 0x7f62780b1b30 2023-10-08T16:05:07.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.600+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f627804c3c0 con 0x7f62780b1b30 2023-10-08T16:05:07.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.601+0000 7f625ffff700 1 -- 172.21.15.170:0/129172945 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f626c050a50 con 0x7f62780b1b30 2023-10-08T16:05:07.602 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:07.602 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":21,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:04:37.829475+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:04:33.930734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T16:04:35.181236+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:07.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 msgr2=0x7f62640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 0x7f62640693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62780b4380 tx=0x7f62680091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 msgr2=0x7f62780b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f626c00e550 tx=0x7f626c009040 comp rx=0 tx=0).stop 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 shutdown_connections 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62780b4440 0x7f62780b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62600051c0 0x7f62781493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6264066f30 0x7f62640693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 --2- 172.21.15.170:0/129172945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62780b1b30 0x7f62780b1fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 >> 172.21.15.170:0/129172945 conn(0x7f6278068e20 msgr2=0x7f627815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 shutdown_connections 2023-10-08T16:05:07.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.603+0000 7f627e39b700 1 -- 172.21.15.170:0/129172945 wait complete. 2023-10-08T16:05:07.616 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:05:07.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.731+0000 7fc4f69a2700 1 Processor -- start 2023-10-08T16:05:07.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.731+0000 7fc4f69a2700 1 -- start start 2023-10-08T16:05:07.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f69a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4f00cc3c0 0x7fc4f006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f69a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 0x7fc4f00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f69a2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f00b16a0 con 0x7fc4f01574d0 2023-10-08T16:05:07.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f69a2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f00cd980 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f69a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00b1c60 con 0x7fc4f006bf20 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f59a0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc4f01574d0 0x7fc4f00cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42342/0 (socket says 172.21.15.170:42342) 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f59a0700 1 -- 172.21.15.170:0/4032725754 learned_addr learned my addr 172.21.15.170:0/4032725754 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f519f700 1 --2- 172.21.15.170:0/4032725754 >> v2:172.21.15.191:3300/0 conn(0x7fc4f00cc3c0 0x7fc4f006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.732+0000 7fc4f61a1700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 0x7fc4f00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060596753 0 0) 0x7fc4f00b16a0 con 0x7fc4f01574d0 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4dc003570 con 0x7fc4f01574d0 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709710692 0 0) 0x7fc4dc003570 con 0x7fc4f01574d0 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f519f700 1 -- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 msgr2=0x7fc4f00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f519f700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 0x7fc4f00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f519f700 1 -- 172.21.15.170:0/4032725754 >> v1:172.21.15.170:6789/0 conn(0x7fc4f01574d0 legacy=0x7fc4f00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f519f700 1 -- 172.21.15.170:0/4032725754 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f006c360 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f61a1700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 0x7fc4f00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f519f700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f006b9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00098e0 tx=0x7fc4e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a88eaf573cfaee3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec001970 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0011420 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.733+0000 7fc4f499e700 1 -- 172.21.15.170:0/4032725754 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00117c0 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 -- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 msgr2=0x7fc4f006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f006b9e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00098e0 tx=0x7fc4e00091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 -- 172.21.15.170:0/4032725754 shutdown_connections 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f006bf20 0x7fc4f00b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/4032725754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f006b9e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 -- 172.21.15.170:0/4032725754 >> 172.21.15.170:0/4032725754 conn(0x7fc4f00ce430 msgr2=0x7fc4f00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 -- 172.21.15.170:0/4032725754 shutdown_connections 2023-10-08T16:05:07.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.734+0000 7fc4f69a2700 1 -- 172.21.15.170:0/4032725754 wait complete. 2023-10-08T16:05:07.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.735+0000 7fc4f69a2700 1 Processor -- start 2023-10-08T16:05:07.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.735+0000 7fc4f69a2700 1 -- start start 2023-10-08T16:05:07.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.735+0000 7fc4f69a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f00c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 0x7fc4f015c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f015b350 con 0x7fc4f006bf20 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f015b630 con 0x7fc4f00cc3c0 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f015b910 con 0x7fc4f01574d0 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54456/0 (socket says 172.21.15.170:54456) 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 -- 172.21.15.170:0/545611913 learned_addr learned my addr 172.21.15.170:0/545611913 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f519f700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f00c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f61a1700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 0x7fc4f015c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 -- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 msgr2=0x7fc4f015c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 0x7fc4f015c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 -- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 msgr2=0x7fc4f00c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f00c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 -- 172.21.15.170:0/545611913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e8009420 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f61a1700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 0x7fc4f015c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f59a0700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007fd0 tx=0x7fc4ec001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec005d80 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec00a750 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e0009420 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.736+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00ab10 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.737+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f015bf50 con 0x7fc4f006bf20 2023-10-08T16:05:07.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.738+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f015c240 con 0x7fc4f006bf20 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.738+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4ec0070f0 con 0x7fc4f006bf20 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.738+0000 7fc4e5ffb700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.738+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4ec0846d0 con 0x7fc4f006bf20 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.738+0000 7fc4f519f700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.739+0000 7fc4f519f700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 0x7fc4d40693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00cc800 tx=0x7fc4e0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:07.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.742+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4ec051940 con 0x7fc4f006bf20 2023-10-08T16:05:07.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.970+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc4f004c3c0 con 0x7fc4f006bf20 2023-10-08T16:05:07.971 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:05:07.971+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:05:07.974 INFO:tasks.ceph.mds.a.smithi170.stderr: -2764> 2023-10-08T16:04:29.555+0000 7fa8c4d1d700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:05:07.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.975+0000 7fc4e5ffb700 1 -- 172.21.15.170:0/545611913 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc4ec054f60 con 0x7fc4f006bf20 2023-10-08T16:05:07.976 INFO:tasks.ceph.mds.a.smithi170.stderr: -1566> 2023-10-08T16:04:40.570+0000 7fa8bed11700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 msgr2=0x7fc4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 0x7fc4d40693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00cc800 tx=0x7fc4e0016000 comp rx=0 tx=0).stop 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 msgr2=0x7fc4f00c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec007fd0 tx=0x7fc4ec001730 comp rx=0 tx=0).stop 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 shutdown_connections 2023-10-08T16:05:07.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4f01574d0 0x7fc4f015c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4f006bf20 0x7fc4f00c4050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 --2- 172.21.15.170:0/545611913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4f00cc3c0 0x7fc4f00c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 >> 172.21.15.170:0/545611913 conn(0x7fc4f00ce430 msgr2=0x7fc4f0155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 shutdown_connections 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:07.977+0000 7fc4f69a2700 1 -- 172.21.15.170:0/545611913 wait complete. 2023-10-08T16:05:07.979 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:05:07.991 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:05:07.998 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:05:08.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.106+0000 7f4a6a097700 1 Processor -- start 2023-10-08T16:05:08.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 -- start start 2023-10-08T16:05:08.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a64141500 0x7f4a640c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a6406c0a0 0x7f4a640c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6415c2a0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a640c9880 con 0x7f4a64141500 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a6a097700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6415b690 con 0x7f4a6406c0a0 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.107+0000 7f4a62ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a64141500 0x7f4a640c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a637fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4a6406bcd0 0x7f4a64140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42354/0 (socket says 172.21.15.170:42354) 2023-10-08T16:05:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a64141500 0x7f4a640c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54480/0 (socket says 172.21.15.170:54480) 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a637fe700 1 -- 172.21.15.170:0/1578445505 learned_addr learned my addr 172.21.15.170:0/1578445505 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a63fff700 1 --2- 172.21.15.170:0/1578445505 >> v2:172.21.15.191:3300/0 conn(0x7f4a6406c0a0 0x7f4a640c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 -- 172.21.15.170:0/1578445505 >> v2:172.21.15.191:3300/0 conn(0x7f4a6406c0a0 msgr2=0x7f4a640c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 --2- 172.21.15.170:0/1578445505 >> v2:172.21.15.191:3300/0 conn(0x7f4a6406c0a0 0x7f4a640c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 -- 172.21.15.170:0/1578445505 >> v1:172.21.15.170:6789/0 conn(0x7f4a6406bcd0 legacy=0x7f4a64140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 -- 172.21.15.170:0/1578445505 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6415b970 con 0x7f4a64141500 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a627fc700 1 -- 172.21.15.170:0/1578445505 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029999437 0 0) 0x7f4a6415c2a0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a62ffd700 1 --2- 172.21.15.170:0/1578445505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a64141500 0x7f4a640c8e00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cf210 tx=0x7f4a580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3a172e0eaa0982 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:08.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a627fc700 1 -- 172.21.15.170:0/1578445505 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800b6b0 con 0x7f4a64141500 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a627fc700 1 -- 172.21.15.170:0/1578445505 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5800b810 con 0x7f4a64141500 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.108+0000 7f4a627fc700 1 -- 172.21.15.170:0/1578445505 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5800bbb0 con 0x7f4a64141500 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 -- 172.21.15.170:0/1578445505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a64141500 msgr2=0x7f4a640c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 --2- 172.21.15.170:0/1578445505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a64141500 0x7f4a640c8e00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4a640cf210 tx=0x7f4a580091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 -- 172.21.15.170:0/1578445505 shutdown_connections 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 --2- 172.21.15.170:0/1578445505 >> v2:172.21.15.191:3300/0 conn(0x7f4a6406c0a0 0x7f4a640c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 --2- 172.21.15.170:0/1578445505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a64141500 0x7f4a640c8e00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 -- 172.21.15.170:0/1578445505 >> 172.21.15.170:0/1578445505 conn(0x7f4a640ce600 msgr2=0x7f4a640cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 -- 172.21.15.170:0/1578445505 shutdown_connections 2023-10-08T16:05:08.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.109+0000 7f4a6a097700 1 -- 172.21.15.170:0/1578445505 wait complete. 2023-10-08T16:05:08.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.110+0000 7f4a6a097700 1 Processor -- start 2023-10-08T16:05:08.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.110+0000 7f4a6a097700 1 -- start start 2023-10-08T16:05:08.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a6406c0a0 0x7f4a6413bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 0x7f4a64156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640b19b0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640b2ad0 con 0x7f4a64141500 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6413c520 con 0x7f4a6406c0a0 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54490/0 (socket says 172.21.15.170:54490) 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 -- 172.21.15.170:0/242660888 learned_addr learned my addr 172.21.15.170:0/242660888 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a63fff700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 0x7f4a64156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a62ffd700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a6406c0a0 0x7f4a6413bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 -- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a6406c0a0 msgr2=0x7f4a6413bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a6406c0a0 0x7f4a6413bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 -- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 msgr2=0x7f4a64156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 0x7f4a64156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 -- 172.21.15.170:0/242660888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a58009420 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a63fff700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 0x7f4a64156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a637fe700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400c810 tx=0x7f4a54007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54005b30 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a54005c90 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6416ded0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54007300 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.111+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6416d880 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.112+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6404c3c0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.113+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a5401b400 con 0x7f4a6406bcd0 2023-10-08T16:05:08.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.113+0000 7f4a4b7fe700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 0x7f4a500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:08.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.113+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f4a54084050 con 0x7f4a6406bcd0 2023-10-08T16:05:08.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.113+0000 7f4a62ffd700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 0x7f4a500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:08.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.114+0000 7f4a62ffd700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 0x7f4a500693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a580091e0 tx=0x7f4a58017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:08.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.117+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a540512f0 con 0x7f4a6406bcd0 2023-10-08T16:05:08.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.343+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4a6415bc50 con 0x7f4a6406bcd0 2023-10-08T16:05:08.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.980+0000 7f4a4b7fe700 1 -- 172.21.15.170:0/242660888 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4a54054910 con 0x7f4a6406bcd0 2023-10-08T16:05:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.982+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 msgr2=0x7f4a500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.982+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 0x7f4a500693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a580091e0 tx=0x7f4a58017000 comp rx=0 tx=0).stop 2023-10-08T16:05:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.982+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 msgr2=0x7f4a640b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:08.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.982+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400c810 tx=0x7f4a54007df0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 shutdown_connections 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a64141500 0x7f4a64156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4a50066f30 0x7f4a500693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a6406bcd0 0x7f4a640b2590 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 --2- 172.21.15.170:0/242660888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a6406c0a0 0x7f4a6413bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 >> 172.21.15.170:0/242660888 conn(0x7f4a640ce600 msgr2=0x7f4a640c49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 shutdown_connections 2023-10-08T16:05:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:08.983+0000 7f4a6a097700 1 -- 172.21.15.170:0/242660888 wait complete. 2023-10-08T16:05:08.996 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:05:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.156+0000 7fc11dfb9700 1 Processor -- start 2023-10-08T16:05:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.156+0000 7fc11dfb9700 1 -- start start 2023-10-08T16:05:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11dfb9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 0x7fc1180cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11dfb9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc1181591e0 0x7fc118159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11dfb9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1180cfa00 con 0x7fc1180b03c0 2023-10-08T16:05:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11dfb9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1180cb580 con 0x7fc1181591e0 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11dfb9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1180cf160 con 0x7fc1180b0790 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11d7b8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc1181591e0 0x7fc118159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc117fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 0x7fc1180cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11d7b8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc1181591e0 0x7fc118159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:40632/0 (socket says 172.21.15.170:40632) 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.157+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2123494189 learned_addr learned my addr 172.21.15.170:0/2123494189 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 msgr2=0x7fc1180cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc11d7b8700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 0x7fc1180cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2123494189 >> v1:172.21.15.191:6789/0 conn(0x7fc1180b03c0 legacy=0x7fc1180b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2123494189 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1180cf440 con 0x7fc1181591e0 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc117fff700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 0x7fc1180cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc11d7b8700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1181591e0 0x7fc118159620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1100099b0 tx=0x7fc110009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8cabb57288e778 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc1177fe700 1 -- 172.21.15.170:0/2123494189 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110009ea0 con 0x7fc1181591e0 2023-10-08T16:05:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc1177fe700 1 -- 172.21.15.170:0/2123494189 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b860 con 0x7fc1181591e0 2023-10-08T16:05:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.158+0000 7fc1177fe700 1 -- 172.21.15.170:0/2123494189 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bc00 con 0x7fc1181591e0 2023-10-08T16:05:09.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1181591e0 msgr2=0x7fc118159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1181591e0 0x7fc118159620 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc1100099b0 tx=0x7fc110009250 comp rx=0 tx=0).stop 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2123494189 shutdown_connections 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1181591e0 0x7fc118159620 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2123494189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b0790 0x7fc1180cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2123494189 >> 172.21.15.170:0/2123494189 conn(0x7fc118063230 msgr2=0x7fc118063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2123494189 shutdown_connections 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.159+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2123494189 wait complete. 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.160+0000 7fc11dfb9700 1 Processor -- start 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.160+0000 7fc11dfb9700 1 -- start start 2023-10-08T16:05:09.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b03c0 0x7fc118169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1180b0790 0x7fc11815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11815d810 con 0x7fc1181591e0 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118169f00 con 0x7fc1180b0790 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11dfb9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11816a1e0 con 0x7fc1180b03c0 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54502/0 (socket says 172.21.15.170:54502) 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2845402638 learned_addr learned my addr 172.21.15.170:0/2845402638 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc117fff700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1180b0790 0x7fc11815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11cfb7700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b03c0 0x7fc118169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b03c0 msgr2=0x7fc118169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b03c0 0x7fc118169870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1180b0790 msgr2=0x7fc11815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1180b0790 0x7fc11815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 -- 172.21.15.170:0/2845402638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009420 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.161+0000 7fc11d7b8700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1180cfe40 tx=0x7fc110017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.162+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110011420 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.162+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc110011580 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.162+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1100094f0 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.162+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110011920 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.162+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11816a570 con 0x7fc1181591e0 2023-10-08T16:05:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.163+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11804c3c0 con 0x7fc1181591e0 2023-10-08T16:05:09.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.163+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc110011cd0 con 0x7fc1181591e0 2023-10-08T16:05:09.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.163+0000 7fc114ff9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 0x7fc1040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:09.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.163+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fc110085f60 con 0x7fc1181591e0 2023-10-08T16:05:09.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.163+0000 7fc11cfb7700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 0x7fc1040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:09.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.164+0000 7fc11cfb7700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 0x7fc1040693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.168+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc110053180 con 0x7fc1181591e0 2023-10-08T16:05:09.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.350+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1180c7360 con 0x7fc1181591e0 2023-10-08T16:05:09.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.983+0000 7fc114ff9700 1 -- 172.21.15.170:0/2845402638 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc1100567a0 con 0x7fc1181591e0 2023-10-08T16:05:09.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 msgr2=0x7fc1040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 0x7fc1040693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108009040 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 msgr2=0x7fc11814c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc1180cfe40 tx=0x7fc110017000 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 shutdown_connections 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc104066f30 0x7fc1040693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1181591e0 0x7fc11814c980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc1180b03c0 0x7fc118169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 --2- 172.21.15.170:0/2845402638 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc1180b0790 0x7fc11815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 >> 172.21.15.170:0/2845402638 conn(0x7fc118063230 msgr2=0x7fc118158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 shutdown_connections 2023-10-08T16:05:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:09.986+0000 7fc11dfb9700 1 -- 172.21.15.170:0/2845402638 wait complete. 2023-10-08T16:05:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:05:09.999 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:05:10.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.124+0000 7fa9b0bcc700 1 Processor -- start 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.124+0000 7fa9b0bcc700 1 -- start start 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9b0bcc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa9ac155360 0x7fa9ac0b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9b0bcc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9ac0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9b0bcc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9ac0b4560 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9b0bcc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9ac15c320 con 0x7fa9ac155360 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9b0bcc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0cef90 con 0x7fa9ac0b40f0 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9aad9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa9ac0ceb60 0x7fa9ac06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42356/0 (socket says 172.21.15.170:42356) 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9aad9d700 1 -- 172.21.15.170:0/899625636 learned_addr learned my addr 172.21.15.170:0/899625636 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206174990 0 0) 0x7fa9ac0b4560 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa994003570 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9aa59c700 1 --2- 172.21.15.170:0/899625636 >> v2:172.21.15.191:3300/0 conn(0x7fa9ac155360 0x7fa9ac0b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.125+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9ac0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1725438072 0 0) 0x7fa994003570 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 msgr2=0x7fa9ac0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9ac0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 >> v2:172.21.15.191:3300/0 conn(0x7fa9ac155360 msgr2=0x7fa9ac0b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 --2- 172.21.15.170:0/899625636 >> v2:172.21.15.191:3300/0 conn(0x7fa9ac155360 0x7fa9ac0b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac15b710 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fa9a4001470 con 0x7fa9ac0ceb60 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 >> v1:172.21.15.170:6789/0 conn(0x7fa9ac0ceb60 legacy=0x7fa9ac06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994003a60 0x7fa994003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004810 0x7fa994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac15c320 con 0x7fa994004810 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0cef90 con 0x7fa994003a60 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99400bbc0 con 0x7fa9ac0b40f0 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004810 0x7fa994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9aa59c700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994003a60 0x7fa994003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9aad9d700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 -- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 msgr2=0x7fa9940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9ac0b40f0 0x7fa9940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 -- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994003a60 msgr2=0x7fa994003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994003a60 0x7fa994003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 -- 172.21.15.170:0/899625636 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa994003570 con 0x7fa994004810 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004810 0x7fa994004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000bb90 tx=0x7fa9a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a291bbb691315a33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a000d5d0 con 0x7fa994004810 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a000d730 con 0x7fa994004810 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.126+0000 7fa9a959a700 1 -- 172.21.15.170:0/899625636 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a000daf0 con 0x7fa994004810 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/899625636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004810 msgr2=0x7fa994004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/899625636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994004810 0x7fa994004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000bb90 tx=0x7fa9a0008040 comp rx=0 tx=0).stop 2023-10-08T16:05:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9aad9d700 1 -- 172.21.15.170:0/899625636 reap_dead start 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/899625636 shutdown_connections 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/899625636 >> 172.21.15.170:0/899625636 conn(0x7fa9ac06baf0 msgr2=0x7fa9ac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/899625636 shutdown_connections 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.127+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/899625636 wait complete. 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.128+0000 7fa9b0bcc700 1 Processor -- start 2023-10-08T16:05:10.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.128+0000 7fa9b0bcc700 1 -- start start 2023-10-08T16:05:10.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994004810 0x7fa9ac0c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9ac0b2860 0x7fa9ac0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c9120 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c8df0 con 0x7fa9ac0b2860 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9b0bcc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ac0c3f90 con 0x7fa994004810 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54514/0 (socket says 172.21.15.170:54514) 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 -- 172.21.15.170:0/4270568578 learned_addr learned my addr 172.21.15.170:0/4270568578 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aa59c700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994004810 0x7fa9ac0c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9a9d9b700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9ac0b2860 0x7fa9ac0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 -- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994004810 msgr2=0x7fa9ac0c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994004810 0x7fa9ac0c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 -- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9ac0b2860 msgr2=0x7fa9ac0c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9ac0b2860 0x7fa9ac0c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 -- 172.21.15.170:0/4270568578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0c4240 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa9aad9d700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4018030 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.129+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a400f690 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.130+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ac0c4520 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.130+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400fa30 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.130+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ac0c95b0 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.130+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ac0c8c50 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.131+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9a401b070 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.131+0000 7fa99a7fc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 0x7fa98c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.131+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa9a4019070 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.131+0000 7fa9aa59c700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 0x7fa98c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.132+0000 7fa9aa59c700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 0x7fa98c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0078b0 tx=0x7fa99c009710 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:10.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.135+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9ac0c8c50 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.317+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9ac0c8c50 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.987+0000 7fa99a7fc700 1 -- 172.21.15.170:0/4270568578 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa9ac0c8c50 con 0x7fa9ac0b2cd0 2023-10-08T16:05:10.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 msgr2=0x7fa98c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 0x7fa98c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa99c0078b0 tx=0x7fa99c009710 comp rx=0 tx=0).stop 2023-10-08T16:05:10.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 msgr2=0x7fa9ac0c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:10.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4000c00 tx=0x7fa9a4001e90 comp rx=0 tx=0).stop 2023-10-08T16:05:10.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 shutdown_connections 2023-10-08T16:05:10.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa98c066f30 0x7fa98c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9ac0b2cd0 0x7fa9ac0c3a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994004810 0x7fa9ac0c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 --2- 172.21.15.170:0/4270568578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9ac0b2860 0x7fa9ac0c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:10.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 >> 172.21.15.170:0/4270568578 conn(0x7fa9ac06baf0 msgr2=0x7fa9ac154590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:10.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.992+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 shutdown_connections 2023-10-08T16:05:10.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:10.993+0000 7fa9b0bcc700 1 -- 172.21.15.170:0/4270568578 wait complete. 2023-10-08T16:05:10.994 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:05:11.006 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:05:11.007 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:05:11.007 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:05:11.007 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:11.007 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:05:11.007 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:05:11.007 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:05:11.008 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:11.008 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:05:11.008 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:05:11.008 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:05:11.008 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:11.008 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:05:11.009 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:05:11.009 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:05:11.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:11.016 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:05:11.017 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:05:11.017 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:05:11.018 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:05:11.019 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:05:11.020 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:05:11.020 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:05:11.021 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:05:11.059 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:05:11.060 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:11.060 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:05:11.060 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:11.062 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:05:11.062 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:11.063 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:05:11.063 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:11.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:05:11.134 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:05:11.134 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:05:11.147 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:05:11.147 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:05:11.148 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T16:05:11.147+0000 7f3b94320700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T16:05:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.184+0000 7fb8028c3700 1 Processor -- start 2023-10-08T16:05:11.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.184+0000 7fb8028c3700 1 -- start start 2023-10-08T16:05:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb8028c3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb8028c3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0b1420 0x7fb7fc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb8028c3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc0cc300 con 0x7fb7fc0ccf90 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb8028c3700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc156bf0 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb8028c3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0ae950 con 0x7fb7fc0b1420 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb7fbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb7fbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54558/0 (socket says 172.21.15.170:54558) 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb7fbfff700 1 -- 172.21.15.170:0/590209437 learned_addr learned my addr 172.21.15.170:0/590209437 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:11.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.185+0000 7fb800e60700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0b1420 0x7fb7fc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7fbfff700 1 -- 172.21.15.170:0/590209437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0b1420 msgr2=0x7fb7fc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7fbfff700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0b1420 0x7fb7fc0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7fbfff700 1 -- 172.21.15.170:0/590209437 >> v1:172.21.15.191:6789/0 conn(0x7fb7fc0ccf90 legacy=0x7fb7fc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7fbfff700 1 -- 172.21.15.170:0/590209437 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0ad370 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7fbfff700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0099e0 tx=0x7fb7ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3207c1706e6fc4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7faffd700 1 -- 172.21.15.170:0/590209437 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00b7e0 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7faffd700 1 -- 172.21.15.170:0/590209437 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec00b940 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb7faffd700 1 -- 172.21.15.170:0/590209437 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00bce0 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 -- 172.21.15.170:0/590209437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0cbf30 msgr2=0x7fb7fc0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0099e0 tx=0x7fb7ec009250 comp rx=0 tx=0).stop 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 -- 172.21.15.170:0/590209437 shutdown_connections 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0b1420 0x7fb7fc0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 --2- 172.21.15.170:0/590209437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0cbf30 0x7fb7fc0ae410 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:11.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 -- 172.21.15.170:0/590209437 >> 172.21.15.170:0/590209437 conn(0x7fb7fc06a320 msgr2=0x7fb7fc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:11.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 -- 172.21.15.170:0/590209437 shutdown_connections 2023-10-08T16:05:11.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.186+0000 7fb8028c3700 1 -- 172.21.15.170:0/590209437 wait complete. 2023-10-08T16:05:11.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 Processor -- start 2023-10-08T16:05:11.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 -- start start 2023-10-08T16:05:11.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0cbf30 0x7fb7fc16a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7fc0ccf90 0x7fb7fc0c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc16d780 con 0x7fb7fc0b1420 2023-10-08T16:05:11.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc15d1a0 con 0x7fb7fc0ccf90 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.188+0000 7fb8028c3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc15d480 con 0x7fb7fc0cbf30 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54560/0 (socket says 172.21.15.170:54560) 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb800e60700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7fc0ccf90 0x7fb7fc0c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 -- 172.21.15.170:0/2533058454 learned_addr learned my addr 172.21.15.170:0/2533058454 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fb7fe700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0cbf30 0x7fb7fc16a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 -- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0cbf30 msgr2=0x7fb7fc16a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0cbf30 0x7fb7fc16a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 -- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7fc0ccf90 msgr2=0x7fb7fc0c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7fc0ccf90 0x7fb7fc0c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 -- 172.21.15.170:0/2533058454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0094f0 con 0x7fb7fc0b1420 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7fbfff700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009290 tx=0x7fb7ec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec011c30 con 0x7fb7fc0b1420 2023-10-08T16:05:11.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec020070 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec01b450 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc15d760 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.189+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc15cde0 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.190+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc16d8f0 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.191+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7ec022030 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.191+0000 7fb7e3fff700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 0x7fb7e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.191+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb7ec085fa0 con 0x7fb7fc0b1420 2023-10-08T16:05:11.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.191+0000 7fb7fb7fe700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 0x7fb7e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:11.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.191+0000 7fb7fb7fe700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 0x7fb7e80693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc15de50 tx=0x7fb7f0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:11.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.195+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7ec0b75a0 con 0x7fb7fc0b1420 2023-10-08T16:05:11.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.392+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb7fc15dc80 con 0x7fb7fc0b1420 2023-10-08T16:05:11.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.996+0000 7fb7e3fff700 1 -- 172.21.15.170:0/2533058454 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb7fc16d8f0 con 0x7fb7fc0b1420 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 msgr2=0x7fb7e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 0x7fb7e80693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc15de50 tx=0x7fb7f0009060 comp rx=0 tx=0).stop 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 msgr2=0x7fb7fc155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009290 tx=0x7fb7ec016000 comp rx=0 tx=0).stop 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 shutdown_connections 2023-10-08T16:05:12.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7fc0ccf90 0x7fb7fc0c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb7e8066f30 0x7fb7e80693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7fc0b1420 0x7fb7fc155c90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 --2- 172.21.15.170:0/2533058454 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb7fc0cbf30 0x7fb7fc16a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 >> 172.21.15.170:0/2533058454 conn(0x7fb7fc06a320 msgr2=0x7fb7fc0ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 shutdown_connections 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:11.999+0000 7fb8028c3700 1 -- 172.21.15.170:0/2533058454 wait complete. 2023-10-08T16:05:12.001 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:05:12.012 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:05:12.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.127+0000 7f2a389fd700 1 Processor -- start 2023-10-08T16:05:12.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.127+0000 7f2a389fd700 1 -- start start 2023-10-08T16:05:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a389fd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a34159890 0x7f2a3415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a389fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a34069560 0x7f2a340699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a389fd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a34069ee0 con 0x7f2a34159c60 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a389fd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3415a160 con 0x7f2a34159890 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a389fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a340d13a0 con 0x7f2a34069560 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a3259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a34159890 0x7f2a3415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2a34159890 0x7f2a3415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54566/0 (socket says 172.21.15.170:54566) 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.128+0000 7f2a31d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2a34159c60 0x7f2a3415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42386/0 (socket says 172.21.15.170:42386) 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 -- 172.21.15.170:0/1709519869 learned_addr learned my addr 172.21.15.170:0/1709519869 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a32d9d700 1 --2- 172.21.15.170:0/1709519869 >> v2:172.21.15.191:3300/0 conn(0x7f2a34069560 0x7f2a340699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 -- 172.21.15.170:0/1709519869 >> v2:172.21.15.191:3300/0 conn(0x7f2a34069560 msgr2=0x7f2a340699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:12.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 --2- 172.21.15.170:0/1709519869 >> v2:172.21.15.191:3300/0 conn(0x7f2a34069560 0x7f2a340699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 -- 172.21.15.170:0/1709519869 >> v1:172.21.15.170:6789/0 conn(0x7f2a34159c60 legacy=0x7f2a3415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 -- 172.21.15.170:0/1709519869 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3415e140 con 0x7f2a34159890 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3159a700 1 -- 172.21.15.170:0/1709519869 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262943200 0 0) 0x7f2a34069ee0 con 0x7f2a34159c60 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3259c700 1 --2- 172.21.15.170:0/1709519869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34159890 0x7f2a3415abd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406a890 tx=0x7f2a24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b0f858a03f65b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3159a700 1 -- 172.21.15.170:0/1709519869 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400e070 con 0x7f2a34159890 2023-10-08T16:05:12.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3159a700 1 -- 172.21.15.170:0/1709519869 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400ba50 con 0x7f2a34159890 2023-10-08T16:05:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.129+0000 7f2a3159a700 1 -- 172.21.15.170:0/1709519869 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240092e0 con 0x7f2a34159890 2023-10-08T16:05:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 -- 172.21.15.170:0/1709519869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34159890 msgr2=0x7f2a3415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1709519869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34159890 0x7f2a3415abd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2a3406a890 tx=0x7f2a24009840 comp rx=0 tx=0).stop 2023-10-08T16:05:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 -- 172.21.15.170:0/1709519869 shutdown_connections 2023-10-08T16:05:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1709519869 >> v2:172.21.15.191:3300/0 conn(0x7f2a34069560 0x7f2a340699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1709519869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34159890 0x7f2a3415abd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 -- 172.21.15.170:0/1709519869 >> 172.21.15.170:0/1709519869 conn(0x7f2a34068e20 msgr2=0x7f2a340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 -- 172.21.15.170:0/1709519869 shutdown_connections 2023-10-08T16:05:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.130+0000 7f2a389fd700 1 -- 172.21.15.170:0/1709519869 wait complete. 2023-10-08T16:05:12.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.131+0000 7f2a389fd700 1 Processor -- start 2023-10-08T16:05:12.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.131+0000 7f2a389fd700 1 -- start start 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 0x7f2a340b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a34159c60 0x7f2a340b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3414dbd0 con 0x7f2a34069560 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340ceac0 con 0x7f2a34159890 2023-10-08T16:05:12.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340cf980 con 0x7f2a34159c60 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54574/0 (socket says 172.21.15.170:54574) 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 -- 172.21.15.170:0/1891770028 learned_addr learned my addr 172.21.15.170:0/1891770028 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a31d9b700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 0x7f2a340b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a32d9d700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a34159c60 0x7f2a340b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 -- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a34159c60 msgr2=0x7f2a340b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a34159c60 0x7f2a340b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 -- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 msgr2=0x7f2a340b4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 0x7f2a340b4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 -- 172.21.15.170:0/1891770028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24009ae0 con 0x7f2a34069560 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a31d9b700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 0x7f2a340b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:12.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a3259c700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a34070e20 tx=0x7f2a24018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24008be0 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a24008d60 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340cabe0 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240266b0 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.132+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340ce190 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.133+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3404c3c0 con 0x7f2a34069560 2023-10-08T16:05:12.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.134+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a24003040 con 0x7f2a34069560 2023-10-08T16:05:12.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.134+0000 7f2a2a7fc700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.134+0000 7f2a31d9b700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.134+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2a240860f0 con 0x7f2a34069560 2023-10-08T16:05:12.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.134+0000 7f2a31d9b700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 0x7f2a140693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a3415b6c0 tx=0x7f2a1c007da0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.138+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a240536b0 con 0x7f2a34069560 2023-10-08T16:05:12.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.393+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2a3406a5b0 con 0x7f2a34069560 2023-10-08T16:05:12.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.394+0000 7f2a2a7fc700 1 -- 172.21.15.170:0/1891770028 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2a24056cd0 con 0x7f2a34069560 2023-10-08T16:05:12.394 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:12.395 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.396 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:05:12.397 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:05:12.398 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.400 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:05:12.401 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.402 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.403 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.404 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:05:12.405 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.406 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:12.407 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.408 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-mds", 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJlGIqeJxAAKsHhTWn3yfcK6hO/VmshRg==", 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-mds" 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-mgr", 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJlQZWeJxAArqMsMdPPpjalutEC1UtHEQ==", 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-08T16:05:12.409 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-osd", 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJljZyeJxAAqQS+SLfpCZVdGQDHiW4ogQ==", 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-osd" 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rbd", 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJlm6SeJxAALZvNHfqQADZx9NbPpyrxgQ==", 2023-10-08T16:05:12.410 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJl4KueJxAAt6kOvZjaIUmYEJuT0Mag8Q==", 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.411 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.bootstrap-rgw", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQAC0yJlbrOeJxAA0YjYA4f1vDsUZ5gU2Zixyw==", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:05:12.412 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:12.413 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:05:12.414 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 msgr2=0x7f2a140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 0x7f2a140693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2a3415b6c0 tx=0x7f2a1c007da0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 msgr2=0x7f2a340ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2a34070e20 tx=0x7f2a24018000 comp rx=0 tx=0).stop 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 shutdown_connections 2023-10-08T16:05:12.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a34159c60 0x7f2a340b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a14066f30 0x7f2a140693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a34069560 0x7f2a340ca6a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 --2- 172.21.15.170:0/1891770028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a34159890 0x7f2a340b4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 >> 172.21.15.170:0/1891770028 conn(0x7f2a34068e20 msgr2=0x7f2a340c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 shutdown_connections 2023-10-08T16:05:12.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.396+0000 7f2a389fd700 1 -- 172.21.15.170:0/1891770028 wait complete. 2023-10-08T16:05:12.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-08T16:05:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.532+0000 7f223a7d8700 1 Processor -- start 2023-10-08T16:05:12.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.532+0000 7f223a7d8700 1 -- start start 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2234159890 0x7f223415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2234159c60 0x7f223415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f2234069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223415df50 con 0x7f2234159c60 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223415a160 con 0x7f2234159890 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f223a7d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340d13a0 con 0x7f223415db10 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f22337fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2234159c60 0x7f223415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f22337fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2234159c60 0x7f223415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54582/0 (socket says 172.21.15.170:54582) 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.533+0000 7f22337fe700 1 -- 172.21.15.170:0/3739650130 learned_addr learned my addr 172.21.15.170:0/3739650130 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2238d75700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f2234069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2233fff700 1 --2- 172.21.15.170:0/3739650130 >> v2:172.21.15.191:3300/0 conn(0x7f2234159890 0x7f223415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 -- 172.21.15.170:0/3739650130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 msgr2=0x7f2234069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f2234069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 -- 172.21.15.170:0/3739650130 >> v2:172.21.15.191:3300/0 conn(0x7f2234159890 msgr2=0x7f223415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 --2- 172.21.15.170:0/3739650130 >> v2:172.21.15.191:3300/0 conn(0x7f2234159890 0x7f223415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 -- 172.21.15.170:0/3739650130 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223415e140 con 0x7f2234159c60 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2238d75700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f2234069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f22337fe700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159c60 0x7f223415d5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22340d1090 tx=0x7f2228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d16afa3a498e77 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2232ffd700 1 -- 172.21.15.170:0/3739650130 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800b860 con 0x7f2234159c60 2023-10-08T16:05:12.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2232ffd700 1 -- 172.21.15.170:0/3739650130 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222800b9c0 con 0x7f2234159c60 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f2232ffd700 1 -- 172.21.15.170:0/3739650130 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222800bd60 con 0x7f2234159c60 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 -- 172.21.15.170:0/3739650130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159c60 msgr2=0x7f223415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159c60 0x7f223415d5d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f22340d1090 tx=0x7f2228009250 comp rx=0 tx=0).stop 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 -- 172.21.15.170:0/3739650130 shutdown_connections 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f2234069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 --2- 172.21.15.170:0/3739650130 >> v2:172.21.15.191:3300/0 conn(0x7f2234159890 0x7f223415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 --2- 172.21.15.170:0/3739650130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159c60 0x7f223415d5d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.534+0000 7f223a7d8700 1 -- 172.21.15.170:0/3739650130 >> 172.21.15.170:0/3739650130 conn(0x7f2234068e20 msgr2=0x7f22340d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.535+0000 7f223a7d8700 1 -- 172.21.15.170:0/3739650130 shutdown_connections 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.535+0000 7f223a7d8700 1 -- 172.21.15.170:0/3739650130 wait complete. 2023-10-08T16:05:12.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 Processor -- start 2023-10-08T16:05:12.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 -- start start 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 0x7f22341473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f22340b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234144b90 con 0x7f2234159890 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22340b4e60 con 0x7f2234159c60 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.536+0000 7f223a7d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22340b48a0 con 0x7f223415db10 2023-10-08T16:05:12.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54598/0 (socket says 172.21.15.170:54598) 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 -- 172.21.15.170:0/1973083984 learned_addr learned my addr 172.21.15.170:0/1973083984 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2238d75700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f22340b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f22337fe700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 0x7f22341473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 -- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 msgr2=0x7f22340b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f22340b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 -- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 msgr2=0x7f22341473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 0x7f22341473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 -- 172.21.15.170:0/1973083984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22280094f0 con 0x7f2234159890 2023-10-08T16:05:12.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f22337fe700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 0x7f22341473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f2233fff700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f222400c880 tx=0x7f2224008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011600 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224011780 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22340b4b80 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224011b40 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.537+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22340c7c20 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.538+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22340c7e40 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.539+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2224018360 con 0x7f2234159890 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.539+0000 7f221bfff700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.539+0000 7f22337fe700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.539+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2224083f90 con 0x7f2234159890 2023-10-08T16:05:12.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.539+0000 7f22337fe700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22340d1090 tx=0x7f2228017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.543+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22240b55a0 con 0x7f2234159890 2023-10-08T16:05:12.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.800+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f223404c3c0 con 0x7f2234159890 2023-10-08T16:05:12.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.804+0000 7f221bfff700 1 -- 172.21.15.170:0/1973083984 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f22240150a0 con 0x7f2234159890 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 msgr2=0x7f22200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 0x7f22200693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22340d1090 tx=0x7f2228017000 comp rx=0 tx=0).stop 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 msgr2=0x7f22340b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f222400c880 tx=0x7f2224008040 comp rx=0 tx=0).stop 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 shutdown_connections 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223415db10 0x7f22340b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2220066f30 0x7f22200693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2234159890 0x7f22340b3f50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 --2- 172.21.15.170:0/1973083984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2234159c60 0x7f22341473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 >> 172.21.15.170:0/1973083984 conn(0x7f2234068e20 msgr2=0x7f223415d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 shutdown_connections 2023-10-08T16:05:12.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.806+0000 7f223a7d8700 1 -- 172.21.15.170:0/1973083984 wait complete. 2023-10-08T16:05:12.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-08T16:05:12.830 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:05:12.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:05:12.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.937+0000 7fc9174f2700 1 Processor -- start 2023-10-08T16:05:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.937+0000 7fc9174f2700 1 -- start start 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc9174f2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc91006aa40 0x7fc9100cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc9174f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910160390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc9174f2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9100cc210 con 0x7fc91006bac0 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc9174f2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9100cc4f0 con 0x7fc91006aa40 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc9174f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100cc7d0 con 0x7fc9100c91e0 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc91528e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc91006aa40 0x7fc9100cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc91528e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc91006aa40 0x7fc9100cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54610/0 (socket says 172.21.15.170:54610) 2023-10-08T16:05:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.938+0000 7fc91528e700 1 -- 172.21.15.170:0/3254258501 learned_addr learned my addr 172.21.15.170:0/3254258501 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc915a8f700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc91528e700 1 -- 172.21.15.170:0/3254258501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 msgr2=0x7fc910160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc91528e700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc91528e700 1 -- 172.21.15.170:0/3254258501 >> v1:172.21.15.191:6789/0 conn(0x7fc91006bac0 legacy=0x7fc91006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc91528e700 1 -- 172.21.15.170:0/3254258501 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100ccab0 con 0x7fc91006aa40 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc91528e700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100cbc00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc900009f70 tx=0x7fc900009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164f59d60de8d265 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc90ffff700 1 -- 172.21.15.170:0/3254258501 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000e070 con 0x7fc91006aa40 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc90ffff700 1 -- 172.21.15.170:0/3254258501 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90000ba90 con 0x7fc91006aa40 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc90ffff700 1 -- 172.21.15.170:0/3254258501 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000092e0 con 0x7fc91006aa40 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 -- 172.21.15.170:0/3254258501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 msgr2=0x7fc9100cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100cbc00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc900009f70 tx=0x7fc900009840 comp rx=0 tx=0).stop 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 -- 172.21.15.170:0/3254258501 shutdown_connections 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 --2- 172.21.15.170:0/3254258501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100cbc00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.939+0000 7fc9174f2700 1 -- 172.21.15.170:0/3254258501 >> 172.21.15.170:0/3254258501 conn(0x7fc9100cd670 msgr2=0x7fc9100cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.940+0000 7fc9174f2700 1 -- 172.21.15.170:0/3254258501 shutdown_connections 2023-10-08T16:05:12.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.940+0000 7fc9174f2700 1 -- 172.21.15.170:0/3254258501 wait complete. 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 Processor -- start 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 -- start start 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91006bac0 0x7fc9100c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100c7a70 con 0x7fc91006aa40 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100c4aa0 con 0x7fc91006bac0 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.941+0000 7fc9174f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100b2460 con 0x7fc9100c91e0 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54618/0 (socket says 172.21.15.170:54618) 2023-10-08T16:05:12.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 -- 172.21.15.170:0/1164789368 learned_addr learned my addr 172.21.15.170:0/1164789368 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc914a8d700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91006bac0 0x7fc9100c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc915a8f700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 -- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 msgr2=0x7fc910148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 -- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91006bac0 msgr2=0x7fc9100c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91006bac0 0x7fc9100c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 -- 172.21.15.170:0/1164789368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009ae0 con 0x7fc91006aa40 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc91528e700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc91006e540 tx=0x7fc900017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900012950 con 0x7fc91006aa40 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc900012ad0 con 0x7fc91006aa40 2023-10-08T16:05:12.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9100b2740 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900012e90 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.942+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9100b2d00 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.943+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91004c3c0 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.944+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc90001d630 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.944+0000 7fc90d7fa700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 0x7fc8fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.944+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc900085c50 con 0x7fc91006aa40 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.944+0000 7fc914a8d700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 0x7fc8fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:12.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.944+0000 7fc914a8d700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 0x7fc8fc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc910098850 tx=0x7fc9040090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:12.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:12.948+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc90008b020 con 0x7fc91006aa40 2023-10-08T16:05:13.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.208+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fc9100c7cb0 con 0x7fc91006aa40 2023-10-08T16:05:13.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.211+0000 7fc90d7fa700 1 -- 172.21.15.170:0/1164789368 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc90005ae60 con 0x7fc91006aa40 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 msgr2=0x7fc8fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 0x7fc8fc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc910098850 tx=0x7fc9040090f0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 msgr2=0x7fc9100b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc91006e540 tx=0x7fc900017000 comp rx=0 tx=0).stop 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 shutdown_connections 2023-10-08T16:05:13.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9100c91e0 0x7fc910148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8fc066f30 0x7fc8fc0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc91006aa40 0x7fc9100b1da0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 --2- 172.21.15.170:0/1164789368 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc91006bac0 0x7fc9100c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.213+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 >> 172.21.15.170:0/1164789368 conn(0x7fc9100cd670 msgr2=0x7fc91006b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.214+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 shutdown_connections 2023-10-08T16:05:13.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.214+0000 7fc9174f2700 1 -- 172.21.15.170:0/1164789368 wait complete. 2023-10-08T16:05:13.226 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-08T16:05:13.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.381+0000 7f4e9c120700 1 Processor -- start 2023-10-08T16:05:13.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.382+0000 7f4e9c120700 1 -- start start 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.382+0000 7f4e9c120700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.382+0000 7f4e9c120700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9415a160 con 0x7f4e94159890 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.382+0000 7f4e9c120700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940d13a0 con 0x7f4e9415db10 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.382+0000 7f4e9c120700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9415df50 con 0x7f4e94159c60 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e99ebc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4e94159890 0x7f4e9415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42396/0 (socket says 172.21.15.170:42396) 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54632/0 (socket says 172.21.15.170:54632) 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e99ebc700 1 -- 172.21.15.170:0/3577169632 learned_addr learned my addr 172.21.15.170:0/3577169632 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 -- 172.21.15.170:0/3577169632 >> v1:172.21.15.191:6789/0 conn(0x7f4e9415db10 legacy=0x7f4e94069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 -- 172.21.15.170:0/3577169632 >> v1:172.21.15.170:6789/0 conn(0x7f4e94159890 legacy=0x7f4e9415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e98eba700 1 -- 172.21.15.170:0/3577169632 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025699936 0 0) 0x7f4e9415a160 con 0x7f4e94159890 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 -- 172.21.15.170:0/3577169632 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9415e140 con 0x7f4e94159c60 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e996bb700 1 --2- 172.21.15.170:0/3577169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159c60 0x7f4e9415d5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098e0 tx=0x7f4e840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d93b52fc27ea29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e98eba700 1 -- 172.21.15.170:0/3577169632 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84009e80 con 0x7f4e94159c60 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e98eba700 1 -- 172.21.15.170:0/3577169632 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8400b820 con 0x7f4e94159c60 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.383+0000 7f4e98eba700 1 -- 172.21.15.170:0/3577169632 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8400bba0 con 0x7f4e94159c60 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 -- 172.21.15.170:0/3577169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159c60 msgr2=0x7f4e9415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 --2- 172.21.15.170:0/3577169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159c60 0x7f4e9415d5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e840098e0 tx=0x7f4e840091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 -- 172.21.15.170:0/3577169632 shutdown_connections 2023-10-08T16:05:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 --2- 172.21.15.170:0/3577169632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159c60 0x7f4e9415d5d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 -- 172.21.15.170:0/3577169632 >> 172.21.15.170:0/3577169632 conn(0x7f4e94068e20 msgr2=0x7f4e940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 -- 172.21.15.170:0/3577169632 shutdown_connections 2023-10-08T16:05:13.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.384+0000 7f4e9c120700 1 -- 172.21.15.170:0/3577169632 wait complete. 2023-10-08T16:05:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.385+0000 7f4e9c120700 1 Processor -- start 2023-10-08T16:05:13.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.385+0000 7f4e9c120700 1 -- start start 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e94159c60 0x7f4e940b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e9415db10 0x7f4e940b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940cd410 con 0x7f4e94159890 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940b6230 con 0x7f4e9415db10 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940cd6d0 con 0x7f4e94159c60 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54636/0 (socket says 172.21.15.170:54636) 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 -- 172.21.15.170:0/95372310 learned_addr learned my addr 172.21.15.170:0/95372310 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 -- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e94159c60 msgr2=0x7f4e940b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e94159c60 0x7f4e940b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 -- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e9415db10 msgr2=0x7f4e940b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e9415db10 0x7f4e940b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 -- 172.21.15.170:0/95372310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e84009420 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e99ebc700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000cc70 tx=0x7f4e90001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000ce80 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9000ab70 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e940cd9b0 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.386+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e900070f0 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.387+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e940b5ec0 con 0x7f4e94159890 2023-10-08T16:05:13.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e900074e0 con 0x7f4e94159890 2023-10-08T16:05:13.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9404c3c0 con 0x7f4e94159890 2023-10-08T16:05:13.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e89ffb700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4e90084680 con 0x7f4e94159890 2023-10-08T16:05:13.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e996bb700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.388+0000 7f4e996bb700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e940cf7d0 tx=0x7f4e84016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.392+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e90051cc0 con 0x7f4e94159890 2023-10-08T16:05:13.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.654+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f4e940b0120 con 0x7f4e94159890 2023-10-08T16:05:13.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.657+0000 7f4e89ffb700 1 -- 172.21.15.170:0/95372310 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4e900552e0 con 0x7f4e94159890 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 msgr2=0x7f4e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 0x7f4e780693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4e940cf7d0 tx=0x7f4e84016000 comp rx=0 tx=0).stop 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 msgr2=0x7f4e940b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000cc70 tx=0x7f4e90001a30 comp rx=0 tx=0).stop 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 shutdown_connections 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e9415db10 0x7f4e940b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e78066f30 0x7f4e780693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e94159890 0x7f4e940b3590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 --2- 172.21.15.170:0/95372310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e94159c60 0x7f4e940b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 >> 172.21.15.170:0/95372310 conn(0x7f4e94068e20 msgr2=0x7f4e941532f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:13.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 shutdown_connections 2023-10-08T16:05:13.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.659+0000 7f4e9c120700 1 -- 172.21.15.170:0/95372310 wait complete. 2023-10-08T16:05:13.671 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-08T16:05:13.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.786+0000 7f7cb5288700 1 Processor -- start 2023-10-08T16:05:13.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.786+0000 7f7cb5288700 1 -- start start 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb5288700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7cb0159420 0x7f7cb00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb5288700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7cb015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb5288700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb00b2980 con 0x7f7cb01597f0 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb5288700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb015dbb0 con 0x7f7cb0159420 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb5288700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00b11d0 con 0x7f7cb015d230 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7caf7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7cb01597f0 0x7f7cb00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42408/0 (socket says 172.21.15.170:42408) 2023-10-08T16:05:13.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7caf7fe700 1 -- 172.21.15.170:0/1336012143 learned_addr learned my addr 172.21.15.170:0/1336012143 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7cb4a87700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7cb015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7caffff700 1 --2- 172.21.15.170:0/1336012143 >> v2:172.21.15.191:3300/0 conn(0x7f7cb0159420 0x7f7cb00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595171432 0 0) 0x7f7cb00b2980 con 0x7f7cb01597f0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.787+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c98003570 con 0x7f7cb01597f0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044920002 0 0) 0x7f7c98003570 con 0x7f7cb01597f0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 msgr2=0x7f7cb015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7cb015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 >> v2:172.21.15.191:3300/0 conn(0x7f7cb0159420 msgr2=0x7f7cb00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 --2- 172.21.15.170:0/1336012143 >> v2:172.21.15.191:3300/0 conn(0x7f7cb0159420 0x7f7cb00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00b14b0 con 0x7f7cb01597f0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7ca4001470 con 0x7f7cb01597f0 2023-10-08T16:05:13.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 >> v1:172.21.15.170:6789/0 conn(0x7f7cb01597f0 legacy=0x7f7cb00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7c98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c98004810 0x7f7c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7c980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb015dbb0 con 0x7f7c98003a60 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00b11d0 con 0x7f7c98004810 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9800ba60 con 0x7f7cb015d230 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7c98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caf7fe700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c98004810 0x7f7c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7cb4a87700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7c980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 -- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 msgr2=0x7f7c980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:13.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb015d230 0x7f7c980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 -- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c98004810 msgr2=0x7f7c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c98004810 0x7f7c98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 -- 172.21.15.170:0/1336012143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98003570 con 0x7f7c98003a60 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caf7fe700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7c98004810 0x7f7c98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caffff700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7c98003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009c30 tx=0x7f7ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d6b01a622dd4c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000e710 con 0x7f7c98003a60 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca000e870 con 0x7f7c98003a60 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.788+0000 7f7caeffd700 1 -- 172.21.15.170:0/1336012143 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000ec30 con 0x7f7c98003a60 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 -- 172.21.15.170:0/1336012143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 msgr2=0x7f7c98003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 --2- 172.21.15.170:0/1336012143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7c98003e70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009c30 tx=0x7f7ca00091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb4a87700 1 -- 172.21.15.170:0/1336012143 reap_dead start 2023-10-08T16:05:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 -- 172.21.15.170:0/1336012143 shutdown_connections 2023-10-08T16:05:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 -- 172.21.15.170:0/1336012143 >> 172.21.15.170:0/1336012143 conn(0x7f7cb00d0660 msgr2=0x7f7cb00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 -- 172.21.15.170:0/1336012143 shutdown_connections 2023-10-08T16:05:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.789+0000 7f7cb5288700 1 -- 172.21.15.170:0/1336012143 wait complete. 2023-10-08T16:05:13.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.790+0000 7f7cb5288700 1 Processor -- start 2023-10-08T16:05:13.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.790+0000 7f7cb5288700 1 -- start start 2023-10-08T16:05:13.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb0155460 0x7f7cb016d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 0x7f7cb016d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb016e090 con 0x7f7c98003a60 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00cb020 con 0x7f7cb01558d0 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb5288700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00cb300 con 0x7f7cb0155460 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54650/0 (socket says 172.21.15.170:54650) 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 -- 172.21.15.170:0/2366369383 learned_addr learned my addr 172.21.15.170:0/2366369383 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caf7fe700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb0155460 0x7f7cb016d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb4a87700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 0x7f7cb016d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 -- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb0155460 msgr2=0x7f7cb016d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb0155460 0x7f7cb016d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 -- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 msgr2=0x7f7cb016d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 0x7f7cb016d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 -- 172.21.15.170:0/2366369383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00cb5e0 con 0x7f7c98003a60 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7cb4a87700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 0x7f7cb016d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7caffff700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c980046d0 tx=0x7f7ca001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.791+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0022070 con 0x7f7c98003a60 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.792+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca000e9e0 con 0x7f7c98003a60 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.792+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb00cb8c0 con 0x7f7c98003a60 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.792+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca001c450 con 0x7f7c98003a60 2023-10-08T16:05:13.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.792+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb0146f60 con 0x7f7c98003a60 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.793+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb016e090 con 0x7f7c98003a60 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.793+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ca001c800 con 0x7f7c98003a60 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.793+0000 7f7c97fff700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.793+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7ca0085840 con 0x7f7c98003a60 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.793+0000 7f7caf7fe700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.794+0000 7f7caf7fe700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000e10 tx=0x7f7ca4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:13.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:13.798+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ca0052e00 con 0x7f7c98003a60 2023-10-08T16:05:14.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.055+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7cb00cbf10 con 0x7f7c98003a60 2023-10-08T16:05:14.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.058+0000 7f7c97fff700 1 -- 172.21.15.170:0/2366369383 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7ca002b020 con 0x7f7c98003a60 2023-10-08T16:05:14.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 msgr2=0x7f7c9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 0x7f7c9c0693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4000e10 tx=0x7f7ca4009040 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 msgr2=0x7f7cb0146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c980046d0 tx=0x7f7ca001a000 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 shutdown_connections 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7cb01558d0 0x7f7cb016d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7c9c066f30 0x7f7c9c0693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c98003a60 0x7f7cb0146a20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 --2- 172.21.15.170:0/2366369383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7cb0155460 0x7f7cb016d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 >> 172.21.15.170:0/2366369383 conn(0x7f7cb00d0660 msgr2=0x7f7cb0155e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 shutdown_connections 2023-10-08T16:05:14.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.060+0000 7f7cb5288700 1 -- 172.21.15.170:0/2366369383 wait complete. 2023-10-08T16:05:14.072 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-08T16:05:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.186+0000 7f87ac945700 1 Processor -- start 2023-10-08T16:05:14.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.186+0000 7f87ac945700 1 -- start start 2023-10-08T16:05:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87ac945700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87a8159890 0x7f87a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87ac945700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8069560 0x7f87a80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87ac945700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a8069ee0 con 0x7f87a8159c60 2023-10-08T16:05:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87ac945700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a815a160 con 0x7f87a8069560 2023-10-08T16:05:14.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87ac945700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80d13a0 con 0x7f87a8159890 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.187+0000 7f87a6d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a8069560 0x7f87a80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a5d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f87a8159c60 0x7f87a815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42420/0 (socket says 172.21.15.170:42420) 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a5d9b700 1 -- 172.21.15.170:0/865448683 learned_addr learned my addr 172.21.15.170:0/865448683 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a659c700 1 --2- 172.21.15.170:0/865448683 >> v2:172.21.15.191:3300/0 conn(0x7f87a8159890 0x7f87a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a6d9d700 1 -- 172.21.15.170:0/865448683 >> v2:172.21.15.191:3300/0 conn(0x7f87a8159890 msgr2=0x7f87a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a6d9d700 1 --2- 172.21.15.170:0/865448683 >> v2:172.21.15.191:3300/0 conn(0x7f87a8159890 0x7f87a815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a6d9d700 1 -- 172.21.15.170:0/865448683 >> v1:172.21.15.170:6789/0 conn(0x7f87a8159c60 legacy=0x7f87a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a559a700 1 -- 172.21.15.170:0/865448683 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604816628 0 0) 0x7f87a8069ee0 con 0x7f87a8159c60 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a6d9d700 1 -- 172.21.15.170:0/865448683 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a815e140 con 0x7f87a8069560 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a6d9d700 1 --2- 172.21.15.170:0/865448683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80699a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d1090 tx=0x7f87a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac3a9d1dc78fd97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a559a700 1 -- 172.21.15.170:0/865448683 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000b6b0 con 0x7f87a8069560 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a559a700 1 -- 172.21.15.170:0/865448683 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000b810 con 0x7f87a8069560 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.188+0000 7f87a559a700 1 -- 172.21.15.170:0/865448683 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000bbb0 con 0x7f87a8069560 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 -- 172.21.15.170:0/865448683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 msgr2=0x7f87a80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 --2- 172.21.15.170:0/865448683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80699a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87a80d1090 tx=0x7f87a00091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 -- 172.21.15.170:0/865448683 shutdown_connections 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 --2- 172.21.15.170:0/865448683 >> v2:172.21.15.191:3300/0 conn(0x7f87a8159890 0x7f87a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 --2- 172.21.15.170:0/865448683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80699a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 -- 172.21.15.170:0/865448683 >> 172.21.15.170:0/865448683 conn(0x7f87a8068e20 msgr2=0x7f87a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 -- 172.21.15.170:0/865448683 shutdown_connections 2023-10-08T16:05:14.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.189+0000 7f87ac945700 1 -- 172.21.15.170:0/865448683 wait complete. 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.190+0000 7f87ac945700 1 Processor -- start 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.190+0000 7f87ac945700 1 -- start start 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a8159890 0x7f87a80cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a8159c60 0x7f87a80b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a816dfd0 con 0x7f87a8069560 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a816d6a0 con 0x7f87a8159c60 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a816d980 con 0x7f87a8159890 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54666/0 (socket says 172.21.15.170:54666) 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 -- 172.21.15.170:0/870652062 learned_addr learned my addr 172.21.15.170:0/870652062 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a5d9b700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a8159890 0x7f87a80cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 -- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a8159890 msgr2=0x7f87a80cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a8159890 0x7f87a80cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 -- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a8159c60 msgr2=0x7f87a80b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a8159c60 0x7f87a80b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 -- 172.21.15.170:0/870652062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0009420 con 0x7f87a8069560 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87a659c700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f879800e190 tx=0x7f8798009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879800ee30 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798014420 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a816dc60 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980147e0 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.191+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80cfc10 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.192+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a816e2b0 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.193+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8798014b90 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.193+0000 7f87967fc700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.193+0000 7f87a5d9b700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.193+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8798084a30 con 0x7f87a8069560 2023-10-08T16:05:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.193+0000 7f87a5d9b700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 0x7f87880693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd800 tx=0x7f879c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.197+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8798051ff0 con 0x7f87a8069560 2023-10-08T16:05:14.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.454+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f87a80cc600 con 0x7f87a8069560 2023-10-08T16:05:14.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.458+0000 7f87967fc700 1 -- 172.21.15.170:0/870652062 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8798009a60 con 0x7f87a8069560 2023-10-08T16:05:14.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 msgr2=0x7f87880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 0x7f87880693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f87a80cd800 tx=0x7f879c009040 comp rx=0 tx=0).stop 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 msgr2=0x7f87a80b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f879800e190 tx=0x7f8798009920 comp rx=0 tx=0).stop 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 shutdown_connections 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a8159c60 0x7f87a80b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8788066f30 0x7f87880693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a8069560 0x7f87a80b5ad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 --2- 172.21.15.170:0/870652062 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a8159890 0x7f87a80cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 >> 172.21.15.170:0/870652062 conn(0x7f87a8068e20 msgr2=0x7f87a8157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 shutdown_connections 2023-10-08T16:05:14.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.460+0000 7f87ac945700 1 -- 172.21.15.170:0/870652062 wait complete. 2023-10-08T16:05:14.472 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-08T16:05:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.587+0000 7f7382cf0700 1 Processor -- start 2023-10-08T16:05:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.587+0000 7f7382cf0700 1 -- start start 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7382cf0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159890 0x7f737c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7382cf0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f737c069560 0x7f737c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7382cf0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737c15a160 con 0x7f737c159c60 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7382cf0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737c069f20 con 0x7f737c069560 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7382cf0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0d13a0 con 0x7f737c159890 2023-10-08T16:05:14.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f738128d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f737c069560 0x7f737c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f738128d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f737c069560 0x7f737c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54678/0 (socket says 172.21.15.170:54678) 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f738128d700 1 -- 172.21.15.170:0/3262135651 learned_addr learned my addr 172.21.15.170:0/3262135651 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.588+0000 7f7380a8c700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159890 0x7f737c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f738128d700 1 -- 172.21.15.170:0/3262135651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159890 msgr2=0x7f737c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f738128d700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159890 0x7f737c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f738128d700 1 -- 172.21.15.170:0/3262135651 >> v1:172.21.15.191:6789/0 conn(0x7f737c159c60 legacy=0x7f737c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f738128d700 1 -- 172.21.15.170:0/3262135651 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c15e140 con 0x7f737c069560 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f738128d700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c0699e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73740099b0 tx=0x7f7374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed29a72cdec3ec2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f737b7fe700 1 -- 172.21.15.170:0/3262135651 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374009ea0 con 0x7f737c069560 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f737b7fe700 1 -- 172.21.15.170:0/3262135651 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737400b860 con 0x7f737c069560 2023-10-08T16:05:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f737b7fe700 1 -- 172.21.15.170:0/3262135651 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737400bc00 con 0x7f737c069560 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f7382cf0700 1 -- 172.21.15.170:0/3262135651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 msgr2=0x7f737c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.589+0000 7f7382cf0700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c0699e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73740099b0 tx=0x7f7374009250 comp rx=0 tx=0).stop 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 -- 172.21.15.170:0/3262135651 shutdown_connections 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c0699e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 --2- 172.21.15.170:0/3262135651 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159890 0x7f737c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 -- 172.21.15.170:0/3262135651 >> 172.21.15.170:0/3262135651 conn(0x7f737c068e20 msgr2=0x7f737c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 -- 172.21.15.170:0/3262135651 shutdown_connections 2023-10-08T16:05:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.590+0000 7f7382cf0700 1 -- 172.21.15.170:0/3262135651 wait complete. 2023-10-08T16:05:14.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.591+0000 7f7382cf0700 1 Processor -- start 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.591+0000 7f7382cf0700 1 -- start start 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.591+0000 7f7382cf0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c159890 0x7f737c152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159c60 0x7f737c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0ce080 con 0x7f737c069560 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c1534e0 con 0x7f737c159890 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c0cd470 con 0x7f737c159c60 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54680/0 (socket says 172.21.15.170:54680) 2023-10-08T16:05:14.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 -- 172.21.15.170:0/2627225217 learned_addr learned my addr 172.21.15.170:0/2627225217 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f737bfff700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c159890 0x7f737c152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f738128d700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159c60 0x7f737c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 -- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159c60 msgr2=0x7f737c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159c60 0x7f737c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 -- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c159890 msgr2=0x7f737c152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c159890 0x7f737c152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 -- 172.21.15.170:0/2627225217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73740094f0 con 0x7f737c069560 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7380a8c700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f736c009ef0 tx=0x7f736c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c00cc50 con 0x7f737c069560 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736c00f040 con 0x7f737c069560 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737c0cd750 con 0x7f737c069560 2023-10-08T16:05:14.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c014480 con 0x7f737c069560 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.592+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737c0cdd10 con 0x7f737c069560 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.593+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f737c04c3c0 con 0x7f737c069560 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.594+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f736c0147f0 con 0x7f737c069560 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.594+0000 7f7378ff9700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.594+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f736c084c90 con 0x7f737c069560 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.594+0000 7f737bfff700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.595+0000 7f737bfff700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 0x7f73680693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f737c15cb90 tx=0x7f7370009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.598+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f736c052250 con 0x7f737c069560 2023-10-08T16:05:14.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.856+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f737c15de80 con 0x7f737c069560 2023-10-08T16:05:14.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.860+0000 7f7378ff9700 1 -- 172.21.15.170:0/2627225217 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f736c055870 con 0x7f737c069560 2023-10-08T16:05:14.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 msgr2=0x7f73680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 0x7f73680693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f737c15cb90 tx=0x7f7370009040 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 msgr2=0x7f737c14d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f736c009ef0 tx=0x7f736c00b040 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 shutdown_connections 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f737c159c60 0x7f737c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7368066f30 0x7f73680693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f737c069560 0x7f737c14d7d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 --2- 172.21.15.170:0/2627225217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f737c159890 0x7f737c152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 >> 172.21.15.170:0/2627225217 conn(0x7f737c068e20 msgr2=0x7f737c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 shutdown_connections 2023-10-08T16:05:14.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.862+0000 7f7382cf0700 1 -- 172.21.15.170:0/2627225217 wait complete. 2023-10-08T16:05:14.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:14.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.988+0000 7f67b960a700 1 Processor -- start 2023-10-08T16:05:14.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 -- start start 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f67b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f67b415db10 0x7f67b4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b415a160 con 0x7f67b4159890 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b40d13a0 con 0x7f67b415db10 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.989+0000 7f67b960a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b415e140 con 0x7f67b4159c60 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f67b4159890 0x7f67b415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:42434/0 (socket says 172.21.15.170:42434) 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b2ffd700 1 -- 172.21.15.170:0/2387357891 learned_addr learned my addr 172.21.15.170:0/2387357891 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b37fe700 1 --2- 172.21.15.170:0/2387357891 >> v2:172.21.15.191:3300/0 conn(0x7f67b415db10 0x7f67b4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b27fc700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f67b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165284540 0 0) 0x7f67b415a160 con 0x7f67b4159890 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f679c003570 con 0x7f67b4159890 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464305738 0 0) 0x7f679c003570 con 0x7f67b4159890 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 msgr2=0x7f67b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f67b415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 >> v2:172.21.15.191:3300/0 conn(0x7f67b415db10 msgr2=0x7f67b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 --2- 172.21.15.170:0/2387357891 >> v2:172.21.15.191:3300/0 conn(0x7f67b415db10 0x7f67b4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b415a3d0 con 0x7f67b4159890 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f67a40017f0 con 0x7f67b4159890 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 >> v1:172.21.15.170:6789/0 conn(0x7f67b4159890 legacy=0x7f67b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f679c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f679c003fe0 0x7f679c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f679c004d50 0x7f679c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40d13a0 con 0x7f679c003fe0 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b415e140 con 0x7f679c004d50 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679c00ba90 con 0x7f67b4159c60 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b27fc700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f679c003fe0 0x7f679c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b2ffd700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f679c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b37fe700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f679c004d50 0x7f679c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b27fc700 1 -- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 msgr2=0x7f679c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.990+0000 7f67b27fc700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67b4159c60 0x7f679c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b27fc700 1 -- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f679c004d50 msgr2=0x7f679c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:14.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b27fc700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f679c004d50 0x7f679c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b27fc700 1 -- 172.21.15.170:0/2387357891 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c003570 con 0x7f679c003fe0 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b27fc700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f679c003fe0 0x7f679c004810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67b415a090 tx=0x7f67a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7701a1243c17824 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800e6b0 con 0x7f679c003fe0 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800e810 con 0x7f679c003fe0 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b1ffb700 1 -- 172.21.15.170:0/2387357891 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800ebd0 con 0x7f679c003fe0 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 -- 172.21.15.170:0/2387357891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f679c003fe0 msgr2=0x7f679c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 --2- 172.21.15.170:0/2387357891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f679c003fe0 0x7f679c004810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67b415a090 tx=0x7f67a80091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b37fe700 1 -- 172.21.15.170:0/2387357891 reap_dead start 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 -- 172.21.15.170:0/2387357891 shutdown_connections 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 -- 172.21.15.170:0/2387357891 >> 172.21.15.170:0/2387357891 conn(0x7f67b4068e20 msgr2=0x7f67b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 -- 172.21.15.170:0/2387357891 shutdown_connections 2023-10-08T16:05:14.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.991+0000 7f67b960a700 1 -- 172.21.15.170:0/2387357891 wait complete. 2023-10-08T16:05:14.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 Processor -- start 2023-10-08T16:05:14.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 -- start start 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679c003fe0 0x7f67b416a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b40b3590 0x7f67b40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40b3ce0 con 0x7f67b4147380 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40cd640 con 0x7f67b40b3590 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b960a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b40c7030 con 0x7f679c003fe0 2023-10-08T16:05:14.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54712/0 (socket says 172.21.15.170:54712) 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b37fe700 1 -- 172.21.15.170:0/1960006985 learned_addr learned my addr 172.21.15.170:0/1960006985 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b2ffd700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679c003fe0 0x7f67b416a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.993+0000 7f67b27fc700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b40b3590 0x7f67b40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 -- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679c003fe0 msgr2=0x7f67b416a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679c003fe0 0x7f67b416a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 -- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b40b3590 msgr2=0x7f67b40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b40b3590 0x7f67b40cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 -- 172.21.15.170:0/1960006985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40c7310 con 0x7f67b4147380 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b37fe700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f67b40d03b0 tx=0x7f67ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac013680 con 0x7f67b4147380 2023-10-08T16:05:15.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac0137e0 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b40c75f0 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac013b60 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.994+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b40b3a00 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.995+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b40b51e0 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.996+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67ac014440 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.996+0000 7f679affd700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 0x7f67a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.996+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f67ac084620 con 0x7f67b4147380 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.996+0000 7f67b2ffd700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 0x7f67a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:14.996+0000 7f67b2ffd700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 0x7f67a00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67a40020a0 tx=0x7f67a4006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.000+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b40b51e0 con 0x7f67b4147380 2023-10-08T16:05:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.186+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67b40b55c0 con 0x7f67b4147380 2023-10-08T16:05:15.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.187+0000 7f679affd700 1 -- 172.21.15.170:0/1960006985 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f67b40b55c0 con 0x7f67b4147380 2023-10-08T16:05:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:15.188 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":28,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:12.006040+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 msgr2=0x7f67a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 0x7f67a00693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67a40020a0 tx=0x7f67a4006040 comp rx=0 tx=0).stop 2023-10-08T16:05:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 msgr2=0x7f67b4157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f67b40d03b0 tx=0x7f67ac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 shutdown_connections 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a0066f30 0x7f67a00693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67b4147380 0x7f67b4157170 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679c003fe0 0x7f67b416a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 --2- 172.21.15.170:0/1960006985 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67b40b3590 0x7f67b40cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 >> 172.21.15.170:0/1960006985 conn(0x7f67b4068e20 msgr2=0x7f67b4155540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 shutdown_connections 2023-10-08T16:05:15.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.189+0000 7f67b960a700 1 -- 172.21.15.170:0/1960006985 wait complete. 2023-10-08T16:05:15.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:05:15.202 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.319+0000 7ffa622b3700 1 Processor -- start 2023-10-08T16:05:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.319+0000 7ffa622b3700 1 -- start start 2023-10-08T16:05:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa622b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa622b3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa622b3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa5c15a160 con 0x7ffa5c159c60 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa622b3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5c069f20 con 0x7ffa5c069560 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa622b3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c0d13a0 con 0x7ffa5c159890 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa60850700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa5bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa5bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52704/0 (socket says 172.21.15.170:52704) 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa60850700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:32942/0 (socket says 172.21.15.170:32942) 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.320+0000 7ffa5bfff700 1 -- 172.21.15.170:0/3869758119 learned_addr learned my addr 172.21.15.170:0/3869758119 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5bfff700 1 -- 172.21.15.170:0/3869758119 >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 msgr2=0x7ffa5c0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5bfff700 1 --2- 172.21.15.170:0/3869758119 >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5bfff700 1 -- 172.21.15.170:0/3869758119 >> v1:172.21.15.191:6789/0 conn(0x7ffa5c159c60 legacy=0x7ffa5c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5bfff700 1 -- 172.21.15.170:0/3869758119 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5c15e140 con 0x7ffa5c159890 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa60850700 1 --2- 172.21.15.170:0/3869758119 >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5bfff700 1 --2- 172.21.15.170:0/3869758119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0099b0 tx=0x7ffa4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b656ea042ee2bb07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5affd700 1 -- 172.21.15.170:0/3869758119 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c009ea0 con 0x7ffa5c159890 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5affd700 1 -- 172.21.15.170:0/3869758119 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c00b860 con 0x7ffa5c159890 2023-10-08T16:05:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.321+0000 7ffa5affd700 1 -- 172.21.15.170:0/3869758119 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00bc00 con 0x7ffa5c159890 2023-10-08T16:05:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 -- 172.21.15.170:0/3869758119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 msgr2=0x7ffa5c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 --2- 172.21.15.170:0/3869758119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0099b0 tx=0x7ffa4c009250 comp rx=0 tx=0).stop 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 -- 172.21.15.170:0/3869758119 shutdown_connections 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 --2- 172.21.15.170:0/3869758119 >> v2:172.21.15.191:3300/0 conn(0x7ffa5c069560 0x7ffa5c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 --2- 172.21.15.170:0/3869758119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c159890 0x7ffa5c15abd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 -- 172.21.15.170:0/3869758119 >> 172.21.15.170:0/3869758119 conn(0x7ffa5c068e20 msgr2=0x7ffa5c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 -- 172.21.15.170:0/3869758119 shutdown_connections 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.322+0000 7ffa622b3700 1 -- 172.21.15.170:0/3869758119 wait complete. 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.323+0000 7ffa622b3700 1 Processor -- start 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.323+0000 7ffa622b3700 1 -- start start 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c069560 0x7ffa5c16b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c159890 0x7ffa5c16ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c155bd0 con 0x7ffa5c159c60 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c16b7a0 con 0x7ffa5c159890 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa622b3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c16e460 con 0x7ffa5c069560 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54608/0 (socket says 172.21.15.170:54608) 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 -- 172.21.15.170:0/647293332 learned_addr learned my addr 172.21.15.170:0/647293332 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa5bfff700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c069560 0x7ffa5c16b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa5b7fe700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c159890 0x7ffa5c16ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 -- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c069560 msgr2=0x7ffa5c16b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c069560 0x7ffa5c16b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 -- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c159890 msgr2=0x7ffa5c16ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c159890 0x7ffa5c16ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 -- 172.21.15.170:0/647293332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009a20 con 0x7ffa5c159c60 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.324+0000 7ffa60850700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009f50 tx=0x7ffa5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.325+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5400ccb0 con 0x7ffa5c159c60 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.325+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5400f040 con 0x7ffa5c159c60 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.325+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0094f0 con 0x7ffa5c159c60 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.325+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54014480 con 0x7ffa5c159c60 2023-10-08T16:05:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.325+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c156140 con 0x7ffa5c159c60 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.326+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0699e0 con 0x7ffa5c159c60 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.326+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffa54003020 con 0x7ffa5c159c60 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.326+0000 7ffa43fff700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 0x7ffa480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.326+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ffa54084bb0 con 0x7ffa5c159c60 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.326+0000 7ffa5bfff700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 0x7ffa480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.327+0000 7ffa5bfff700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 0x7ffa480693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0099b0 tx=0x7ffa4c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.330+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa54052170 con 0x7ffa5c159c60 2023-10-08T16:05:15.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.516+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa5c0b3110 con 0x7ffa5c159c60 2023-10-08T16:05:15.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.516+0000 7ffa43fff700 1 -- 172.21.15.170:0/647293332 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa54055790 con 0x7ffa5c159c60 2023-10-08T16:05:15.517 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:15.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 msgr2=0x7ffa480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 0x7ffa480693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0099b0 tx=0x7ffa4c017000 comp rx=0 tx=0).stop 2023-10-08T16:05:15.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 msgr2=0x7ffa5c0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009f50 tx=0x7ffa5400b040 comp rx=0 tx=0).stop 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 shutdown_connections 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffa48066f30 0x7ffa480693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa5c159c60 0x7ffa5c0b3590 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffa5c069560 0x7ffa5c16b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 --2- 172.21.15.170:0/647293332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffa5c159890 0x7ffa5c16ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 >> 172.21.15.170:0/647293332 conn(0x7ffa5c068e20 msgr2=0x7ffa5c1586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 shutdown_connections 2023-10-08T16:05:15.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.519+0000 7ffa622b3700 1 -- 172.21.15.170:0/647293332 wait complete. 2023-10-08T16:05:15.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:05:15.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.645+0000 7f5f085fd700 1 Processor -- start 2023-10-08T16:05:15.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.645+0000 7f5f085fd700 1 -- start start 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f085fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f00159c60 0x7f5f0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f085fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 0x7f5f00069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f085fd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0015a160 con 0x7f5f00159890 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f085fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0015df50 con 0x7f5f00159c60 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f085fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000d13a0 con 0x7f5f0015db10 2023-10-08T16:05:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f00159c60 0x7f5f0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f06b9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 0x7f5f00069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5f00159c60 0x7f5f0015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:32966/0 (socket says 172.21.15.170:32966) 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f06b9a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 0x7f5f00069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:52728/0 (socket says 172.21.15.170:52728) 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f06399700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f5f00159890 0x7f5f0015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57104/0 (socket says 172.21.15.170:57104) 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 -- 172.21.15.170:0/1371707304 learned_addr learned my addr 172.21.15.170:0/1371707304 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 -- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 msgr2=0x7f5f00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 --2- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 0x7f5f00069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 -- 172.21.15.170:0/1371707304 >> v1:172.21.15.191:6789/0 conn(0x7f5f00159890 legacy=0x7f5f0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.646+0000 7f5f05b98700 1 -- 172.21.15.170:0/1371707304 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0015e140 con 0x7f5f00159c60 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f05b98700 1 --2- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159c60 0x7f5f0015d5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40099b0 tx=0x7f5ef4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb37511dec83a07e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f05397700 1 -- 172.21.15.170:0/1371707304 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4009ea0 con 0x7f5f00159c60 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f05397700 1 -- 172.21.15.170:0/1371707304 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400b860 con 0x7f5f00159c60 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f05397700 1 -- 172.21.15.170:0/1371707304 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400bc00 con 0x7f5f00159c60 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 -- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159c60 msgr2=0x7f5f0015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 --2- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159c60 0x7f5f0015d5d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40099b0 tx=0x7f5ef4009250 comp rx=0 tx=0).stop 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 -- 172.21.15.170:0/1371707304 shutdown_connections 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 --2- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f0015db10 0x7f5f00069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 --2- 172.21.15.170:0/1371707304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159c60 0x7f5f0015d5d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 -- 172.21.15.170:0/1371707304 >> 172.21.15.170:0/1371707304 conn(0x7f5f00068e20 msgr2=0x7f5f000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 -- 172.21.15.170:0/1371707304 shutdown_connections 2023-10-08T16:05:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.647+0000 7f5f085fd700 1 -- 172.21.15.170:0/1371707304 wait complete. 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.648+0000 7f5f085fd700 1 Processor -- start 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.648+0000 7f5f085fd700 1 -- start start 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159890 0x7f5f000b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f00159c60 0x7f5f000cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b0eb0 con 0x7f5f0015db10 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000cdcb0 con 0x7f5f00159890 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f085fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00150b00 con 0x7f5f00159c60 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f06b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f06b9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54612/0 (socket says 172.21.15.170:54612) 2023-10-08T16:05:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f06b9a700 1 -- 172.21.15.170:0/3903536604 learned_addr learned my addr 172.21.15.170:0/3903536604 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f06399700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159890 0x7f5f000b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.649+0000 7f5f05b98700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f00159c60 0x7f5f000cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 -- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f00159c60 msgr2=0x7f5f000cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f00159c60 0x7f5f000cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 -- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159890 msgr2=0x7f5f000b6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159890 0x7f5f000b6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 -- 172.21.15.170:0/3903536604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40094f0 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f06b9a700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009df0 tx=0x7f5ef000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000caa0 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef000f040 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000ce70 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f001507b0 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.650+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0016aa40 con 0x7f5f0015db10 2023-10-08T16:05:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.651+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0004c3c0 con 0x7f5f0015db10 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.651+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ef0014690 con 0x7f5f0015db10 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.652+0000 7f5efe7fc700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 0x7f5eec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.652+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5ef0084a70 con 0x7f5f0015db10 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.652+0000 7f5f06399700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 0x7f5eec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.652+0000 7f5f06399700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 0x7f5eec0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b06d0 tx=0x7f5ef8007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.656+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ef0055ac0 con 0x7f5f0015db10 2023-10-08T16:05:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.839+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f00150c40 con 0x7f5f0015db10 2023-10-08T16:05:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.840+0000 7f5efe7fc700 1 -- 172.21.15.170:0/3903536604 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5ef0018060 con 0x7f5f0015db10 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:05:15.841 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:15.842 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:05:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 msgr2=0x7f5eec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 0x7f5eec0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b06d0 tx=0x7f5ef8007eb0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 msgr2=0x7f5f00169ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009df0 tx=0x7f5ef000b040 comp rx=0 tx=0).stop 2023-10-08T16:05:15.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 shutdown_connections 2023-10-08T16:05:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5eec066f30 0x7f5eec0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f0015db10 0x7f5f00169ed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f00159890 0x7f5f000b6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 --2- 172.21.15.170:0/3903536604 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f00159c60 0x7f5f000cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 >> 172.21.15.170:0/3903536604 conn(0x7f5f00068e20 msgr2=0x7f5f00153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 shutdown_connections 2023-10-08T16:05:15.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.842+0000 7f5f085fd700 1 -- 172.21.15.170:0/3903536604 wait complete. 2023-10-08T16:05:15.854 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:05:15.854 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:05:15.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.972+0000 7ff8d8dd6700 1 Processor -- start 2023-10-08T16:05:15.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.972+0000 7ff8d8dd6700 1 -- start start 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff8d4159ec0 0x7ff8d40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d40cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8d40cce40 0x7ff8d40cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d40b5b00 con 0x7ff8d40cce40 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d40b5850 con 0x7ff8d4159ec0 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d8dd6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d40b5da0 con 0x7ff8d40b53d0 2023-10-08T16:05:15.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8d40cce40 0x7ff8d40cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8d40cce40 0x7ff8d40cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54620/0 (socket says 172.21.15.170:54620) 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d2d9d700 1 -- 172.21.15.170:0/736070231 learned_addr learned my addr 172.21.15.170:0/736070231 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d259c700 1 --2- 172.21.15.170:0/736070231 >> v2:172.21.15.191:3300/0 conn(0x7ff8d4159ec0 0x7ff8d40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.973+0000 7ff8d1d9b700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d40cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 -- 172.21.15.170:0/736070231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 msgr2=0x7ff8d40cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d40cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 -- 172.21.15.170:0/736070231 >> v2:172.21.15.191:3300/0 conn(0x7ff8d4159ec0 msgr2=0x7ff8d40b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 --2- 172.21.15.170:0/736070231 >> v2:172.21.15.191:3300/0 conn(0x7ff8d4159ec0 0x7ff8d40b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 -- 172.21.15.170:0/736070231 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d40b6080 con 0x7ff8d40cce40 2023-10-08T16:05:15.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d2d9d700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d40cd2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4089120 tx=0x7ff8cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a51a168e132c698 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d159a700 1 -- 172.21.15.170:0/736070231 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00e070 con 0x7ff8d40cce40 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d159a700 1 -- 172.21.15.170:0/736070231 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc00b970 con 0x7ff8d40cce40 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d159a700 1 -- 172.21.15.170:0/736070231 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0092e0 con 0x7ff8d40cce40 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/736070231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 msgr2=0x7ff8d40cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d40cd2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4089120 tx=0x7ff8cc009840 comp rx=0 tx=0).stop 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/736070231 shutdown_connections 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d40cd2a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/736070231 >> v2:172.21.15.191:3300/0 conn(0x7ff8d4159ec0 0x7ff8d40b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/736070231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d40cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/736070231 >> 172.21.15.170:0/736070231 conn(0x7ff8d40999f0 msgr2=0x7ff8d40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:15.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/736070231 shutdown_connections 2023-10-08T16:05:15.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.974+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/736070231 wait complete. 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.975+0000 7ff8d8dd6700 1 Processor -- start 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 -- start start 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d4069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4159ec0 0x7ff8d40b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41546b0 con 0x7ff8d40cce40 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4154990 con 0x7ff8d4159ec0 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d8dd6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4154c70 con 0x7ff8d40b53d0 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54624/0 (socket says 172.21.15.170:54624) 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d1d9b700 1 -- 172.21.15.170:0/3166363773 learned_addr learned my addr 172.21.15.170:0/3166363773 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d259c700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d4069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.976+0000 7ff8d2d9d700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4159ec0 0x7ff8d40b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 -- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 msgr2=0x7ff8d4069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d4069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 -- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4159ec0 msgr2=0x7ff8d40b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4159ec0 0x7ff8d40b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 -- 172.21.15.170:0/3166363773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc009ae0 con 0x7ff8d40cce40 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d1d9b700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e930 tx=0x7ff8c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8013680 con 0x7ff8d40cce40 2023-10-08T16:05:15.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80137e0 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d4154f50 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8013b80 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.977+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d406a260 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.978+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d4063c90 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.978+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff8c800c3e0 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.979+0000 7ff8c27fc700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 0x7ff8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.979+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff8c80842d0 con 0x7ff8d40cce40 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.979+0000 7ff8d259c700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 0x7ff8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:15.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.979+0000 7ff8d259c700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 0x7ff8b40693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009da0 tx=0x7ff8c4009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:15.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:15.983+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8c8051890 con 0x7ff8d40cce40 2023-10-08T16:05:16.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:16.166+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff8d41567e0 con 0x7ff8d40cce40 2023-10-08T16:05:17.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.066+0000 7ff8c27fc700 1 -- 172.21.15.170:0/3166363773 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff8c8054eb0 con 0x7ff8d40cce40 2023-10-08T16:05:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.070+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 msgr2=0x7ff8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.070+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 0x7ff8b40693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009da0 tx=0x7ff8c4009120 comp rx=0 tx=0).stop 2023-10-08T16:05:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.070+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 msgr2=0x7ff8d4150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.070+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e930 tx=0x7ff8c80091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 shutdown_connections 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8d4159ec0 0x7ff8d40b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8d40b53d0 0x7ff8d4069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff8b4066f30 0x7ff8b40693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 --2- 172.21.15.170:0/3166363773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8d40cce40 0x7ff8d4150160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 >> 172.21.15.170:0/3166363773 conn(0x7ff8d40999f0 msgr2=0x7ff8d41520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 shutdown_connections 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.071+0000 7ff8d8dd6700 1 -- 172.21.15.170:0/3166363773 wait complete. 2023-10-08T16:05:17.072 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:05:17.084 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:05:17.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.206+0000 7fa14aa65700 1 Processor -- start 2023-10-08T16:05:17.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 -- start start 2023-10-08T16:05:17.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa14406bcd0 0x7fa1440c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa1441414c0 0x7fa1440cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14415b790 con 0x7fa144158d40 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14415ba70 con 0x7fa1441414c0 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.207+0000 7fa14aa65700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14415bd50 con 0x7fa14406bcd0 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa144158d40 0x7fa144140db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60562/0 (socket says 172.21.15.170:60562) 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13bfff700 1 -- 172.21.15.170:0/4114808726 learned_addr learned my addr 172.21.15.170:0/4114808726 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 --2- 172.21.15.170:0/4114808726 >> v2:172.21.15.170:3300/0 conn(0x7fa1441414c0 0x7fa1440cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa148801700 1 --2- 172.21.15.170:0/4114808726 >> v2:172.21.15.191:3300/0 conn(0x7fa14406bcd0 0x7fa1440c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13b7fe700 1 -- 172.21.15.170:0/4114808726 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283777508 0 0) 0x7fa14415b790 con 0x7fa144158d40 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13b7fe700 1 -- 172.21.15.170:0/4114808726 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa12c003570 con 0x7fa144158d40 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 -- 172.21.15.170:0/4114808726 >> v2:172.21.15.191:3300/0 conn(0x7fa14406bcd0 msgr2=0x7fa1440c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:17.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 --2- 172.21.15.170:0/4114808726 >> v2:172.21.15.191:3300/0 conn(0x7fa14406bcd0 0x7fa1440c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 -- 172.21.15.170:0/4114808726 >> v1:172.21.15.170:6789/0 conn(0x7fa144158d40 legacy=0x7fa144140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 -- 172.21.15.170:0/4114808726 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14415c030 con 0x7fa1441414c0 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa149002700 1 --2- 172.21.15.170:0/4114808726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1441414c0 0x7fa1440cc290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1440caf90 tx=0x7fa13c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ceb0c0b43501de server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13b7fe700 1 -- 172.21.15.170:0/4114808726 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00e070 con 0x7fa1441414c0 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13b7fe700 1 -- 172.21.15.170:0/4114808726 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c00bab0 con 0x7fa1441414c0 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.208+0000 7fa13b7fe700 1 -- 172.21.15.170:0/4114808726 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c00be70 con 0x7fa1441414c0 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 -- 172.21.15.170:0/4114808726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1441414c0 msgr2=0x7fa1440cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 --2- 172.21.15.170:0/4114808726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1441414c0 0x7fa1440cc290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa1440caf90 tx=0x7fa13c009780 comp rx=0 tx=0).stop 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 -- 172.21.15.170:0/4114808726 shutdown_connections 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 --2- 172.21.15.170:0/4114808726 >> v2:172.21.15.191:3300/0 conn(0x7fa14406bcd0 0x7fa1440c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 --2- 172.21.15.170:0/4114808726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1441414c0 0x7fa1440cc290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 -- 172.21.15.170:0/4114808726 >> 172.21.15.170:0/4114808726 conn(0x7fa1440ce5c0 msgr2=0x7fa1440ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:17.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 -- 172.21.15.170:0/4114808726 shutdown_connections 2023-10-08T16:05:17.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.209+0000 7fa14aa65700 1 -- 172.21.15.170:0/4114808726 wait complete. 2023-10-08T16:05:17.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.210+0000 7fa14aa65700 1 Processor -- start 2023-10-08T16:05:17.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.210+0000 7fa14aa65700 1 -- start start 2023-10-08T16:05:17.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1441414c0 0x7fa144156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa144158d40 0x7fa1440c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14416ad90 con 0x7fa14406bcd0 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440c6870 con 0x7fa144158d40 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa14aa65700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1440b2440 con 0x7fa1441414c0 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54646/0 (socket says 172.21.15.170:54646) 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 -- 172.21.15.170:0/1865101693 learned_addr learned my addr 172.21.15.170:0/1865101693 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa13bfff700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1441414c0 0x7fa144156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa149002700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa144158d40 0x7fa1440c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 -- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1441414c0 msgr2=0x7fa144156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1441414c0 0x7fa144156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 -- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa144158d40 msgr2=0x7fa1440c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa144158d40 0x7fa1440c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 -- 172.21.15.170:0/1865101693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c009a20 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.211+0000 7fa148801700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa14000e930 tx=0x7fa1400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.212+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000eed0 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.212+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14000c850 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.212+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000cbf0 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.212+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1440c6220 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.212+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1441576d0 con 0x7fa14406bcd0 2023-10-08T16:05:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.213+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1440718b0 con 0x7fa14406bcd0 2023-10-08T16:05:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.213+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa140013470 con 0x7fa14406bcd0 2023-10-08T16:05:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.213+0000 7fa138ff9700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 0x7fa1300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.213+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fa1400845c0 con 0x7fa14406bcd0 2023-10-08T16:05:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.213+0000 7fa13bfff700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 0x7fa1300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:17.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.214+0000 7fa13bfff700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 0x7fa1300693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1440b1eb0 tx=0x7fa134009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:17.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.218+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa140051a10 con 0x7fa14406bcd0 2023-10-08T16:05:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:17.399+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa1440c66c0 con 0x7fa14406bcd0 2023-10-08T16:05:18.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.070+0000 7fa138ff9700 1 -- 172.21.15.170:0/1865101693 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa140055030 con 0x7fa14406bcd0 2023-10-08T16:05:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.077+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 msgr2=0x7fa1300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.077+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 0x7fa1300693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1440b1eb0 tx=0x7fa134009040 comp rx=0 tx=0).stop 2023-10-08T16:05:18.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.077+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 msgr2=0x7fa14416a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.077+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa14000e930 tx=0x7fa1400091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 shutdown_connections 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa144158d40 0x7fa1440c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa130066f30 0x7fa1300693e0 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa1440b1eb0 tx=0x7fa134009040 comp rx=0 tx=0).stop 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa14406bcd0 0x7fa14416a850 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa14000e930 tx=0x7fa1400091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 --2- 172.21.15.170:0/1865101693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1441414c0 0x7fa144156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 >> 172.21.15.170:0/1865101693 conn(0x7fa1440ce5c0 msgr2=0x7fa144158400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 shutdown_connections 2023-10-08T16:05:18.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.078+0000 7fa14aa65700 1 -- 172.21.15.170:0/1865101693 wait complete. 2023-10-08T16:05:18.085 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:05:18.096 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:05:18.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.212+0000 7ff96fa3c700 1 Processor -- start 2023-10-08T16:05:18.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.213+0000 7ff96fa3c700 1 -- start start 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.213+0000 7ff96fa3c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff96806c0a0 0x7ff9680caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.213+0000 7ff96fa3c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96815d7e0 con 0x7ff96815df50 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.213+0000 7ff96fa3c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96815dac0 con 0x7ff96806bcd0 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.213+0000 7ff96fa3c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96815aa30 con 0x7ff96806c0a0 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96dfd9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff96815df50 0x7ff9680c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60576/0 (socket says 172.21.15.170:60576) 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96dfd9700 1 -- 172.21.15.170:0/1095439228 learned_addr learned my addr 172.21.15.170:0/1095439228 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96cfd7700 1 --2- 172.21.15.170:0/1095439228 >> v2:172.21.15.191:3300/0 conn(0x7ff96806c0a0 0x7ff9680caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116486951 0 0) 0x7ff96815d7e0 con 0x7ff96815df50 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff950003570 con 0x7ff96815df50 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010171358 0 0) 0x7ff950003570 con 0x7ff96815df50 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 >> v2:172.21.15.191:3300/0 conn(0x7ff96806c0a0 msgr2=0x7ff9680caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 --2- 172.21.15.170:0/1095439228 >> v2:172.21.15.191:3300/0 conn(0x7ff96806c0a0 0x7ff9680caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 >> v1:172.21.15.191:6789/0 conn(0x7ff96806bcd0 legacy=0x7ff96815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96815ad10 con 0x7ff96815df50 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7ff9600017f0 con 0x7ff96815df50 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 >> v1:172.21.15.170:6789/0 conn(0x7ff96815df50 legacy=0x7ff9680c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 0x7ff950003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950004810 0x7ff950004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9500052e0 0x7ff95000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96815aa30 con 0x7ff950003a60 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000c2a0 con 0x7ff950004810 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95000c460 con 0x7ff9500052e0 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 0x7ff950003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96cfd7700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950004810 0x7ff950004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.214+0000 7ff96dfd9700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9500052e0 0x7ff95000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 -- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9500052e0 msgr2=0x7ff95000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9500052e0 0x7ff95000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 -- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950004810 msgr2=0x7ff950004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950004810 0x7ff950004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96dfd9700 1 -- 172.21.15.170:0/1095439228 reap_dead start 2023-10-08T16:05:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 -- 172.21.15.170:0/1095439228 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950003570 con 0x7ff950003a60 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 0x7ff950003e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff9580018b0 tx=0x7ff958007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5844664f042160f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958012420 con 0x7ff950003a60 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9580125a0 con 0x7ff950003a60 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff967fff700 1 -- 172.21.15.170:0/1095439228 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958012960 con 0x7ff950003a60 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96fa3c700 1 -- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 msgr2=0x7ff950003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 0x7ff950003e70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff9580018b0 tx=0x7ff958007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96fa3c700 1 -- 172.21.15.170:0/1095439228 shutdown_connections 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/1095439228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff950003a60 0x7ff950003e70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.215+0000 7ff96fa3c700 1 -- 172.21.15.170:0/1095439228 >> 172.21.15.170:0/1095439228 conn(0x7ff9680ce600 msgr2=0x7ff9680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:18.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.216+0000 7ff96fa3c700 1 -- 172.21.15.170:0/1095439228 shutdown_connections 2023-10-08T16:05:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.216+0000 7ff96fa3c700 1 -- 172.21.15.170:0/1095439228 wait complete. 2023-10-08T16:05:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 Processor -- start 2023-10-08T16:05:18.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 -- start start 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950003a60 0x7ff9680b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9681415a0 0x7ff968169440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968153960 con 0x7ff968141130 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968152dc0 con 0x7ff950003a60 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.217+0000 7ff96fa3c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff968169ce0 con 0x7ff9681415a0 2023-10-08T16:05:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54656/0 (socket says 172.21.15.170:54656) 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 -- 172.21.15.170:0/134031207 learned_addr learned my addr 172.21.15.170:0/134031207 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950003a60 0x7ff9680b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96dfd9700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9681415a0 0x7ff968169440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 -- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9681415a0 msgr2=0x7ff968169440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9681415a0 0x7ff968169440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 -- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950003a60 msgr2=0x7ff9680b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950003a60 0x7ff9680b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 -- 172.21.15.170:0/134031207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968153030 con 0x7ff968141130 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96cfd7700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011d70 tx=0x7ff95c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:18.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00fa10 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00fb70 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c014440 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968153310 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.218+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9680b18e0 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.219+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96804c3c0 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.220+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff95c014870 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.220+0000 7ff9657fa700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 0x7ff9540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.220+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5605+0+0 (secure 0 0 0) 0x7ff95c018030 con 0x7ff968141130 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.220+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 0x7ff9540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:18.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.221+0000 7ff96d7d8700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 0x7ff9540693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff958001010 tx=0x7ff958015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:18.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.225+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff95c0520f0 con 0x7ff968141130 2023-10-08T16:05:18.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:18.451+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff96815aff0 con 0x7ff968141130 2023-10-08T16:05:19.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.432+0000 7ff9657fa700 1 -- 172.21.15.170:0/134031207 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff95c0220c0 con 0x7ff968141130 2023-10-08T16:05:19.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 msgr2=0x7ff9540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 0x7ff9540693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff958001010 tx=0x7ff958015000 comp rx=0 tx=0).stop 2023-10-08T16:05:19.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 msgr2=0x7ff968168f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011d70 tx=0x7ff95c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 shutdown_connections 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9681415a0 0x7ff968169440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff950003a60 0x7ff9680b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff954066f30 0x7ff9540693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 --2- 172.21.15.170:0/134031207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff968141130 0x7ff968168f00 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff95c011d70 tx=0x7ff95c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.437+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 >> 172.21.15.170:0/134031207 conn(0x7ff9680ce600 msgr2=0x7ff968154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.439+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 shutdown_connections 2023-10-08T16:05:19.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.440+0000 7ff96fa3c700 1 -- 172.21.15.170:0/134031207 wait complete. 2023-10-08T16:05:19.441 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 4 and data pool 5 2023-10-08T16:05:19.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.569+0000 7fc3588d6700 1 Processor -- start 2023-10-08T16:05:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.569+0000 7fc3588d6700 1 -- start start 2023-10-08T16:05:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc3588d6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc3540c83c0 0x7fc354164690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc3588d6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc35406bdb0 con 0x7fc354069d60 2023-10-08T16:05:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc3588d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3540cb370 con 0x7fc3540aca80 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc3588d6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc354159f60 con 0x7fc3540c83c0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc352d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc3540c83c0 0x7fc354164690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc349d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc354069d60 0x7fc3540c7cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60592/0 (socket says 172.21.15.170:60592) 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.570+0000 7fc349d9b700 1 -- 172.21.15.170:0/3663446531 learned_addr learned my addr 172.21.15.170:0/3663446531 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc352d9d700 1 -- 172.21.15.170:0/3663446531 >> v1:172.21.15.191:6789/0 conn(0x7fc3540aca80 legacy=0x7fc35406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc352d9d700 1 -- 172.21.15.170:0/3663446531 >> v1:172.21.15.170:6789/0 conn(0x7fc354069d60 legacy=0x7fc3540c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc352d9d700 1 -- 172.21.15.170:0/3663446531 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35406b2b0 con 0x7fc3540c83c0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc351d9b700 1 -- 172.21.15.170:0/3663446531 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175217402 0 0) 0x7fc35406bdb0 con 0x7fc354069d60 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc352d9d700 1 --2- 172.21.15.170:0/3663446531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc354164690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009e60 tx=0x7fc34c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52c017ec37ee0ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc351d9b700 1 -- 172.21.15.170:0/3663446531 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c00e070 con 0x7fc3540c83c0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc351d9b700 1 -- 172.21.15.170:0/3663446531 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c00bc30 con 0x7fc3540c83c0 2023-10-08T16:05:19.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc351d9b700 1 -- 172.21.15.170:0/3663446531 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c012430 con 0x7fc3540c83c0 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc3588d6700 1 -- 172.21.15.170:0/3663446531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 msgr2=0x7fc354164690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.571+0000 7fc3588d6700 1 --2- 172.21.15.170:0/3663446531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc354164690 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc34c009e60 tx=0x7fc34c009780 comp rx=0 tx=0).stop 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.572+0000 7fc3588d6700 1 -- 172.21.15.170:0/3663446531 shutdown_connections 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.572+0000 7fc3588d6700 1 --2- 172.21.15.170:0/3663446531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc354164690 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.572+0000 7fc3588d6700 1 -- 172.21.15.170:0/3663446531 >> 172.21.15.170:0/3663446531 conn(0x7fc3540cd670 msgr2=0x7fc3540cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:19.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.572+0000 7fc3588d6700 1 -- 172.21.15.170:0/3663446531 shutdown_connections 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.572+0000 7fc3588d6700 1 -- 172.21.15.170:0/3663446531 wait complete. 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.573+0000 7fc3588d6700 1 Processor -- start 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.573+0000 7fc3588d6700 1 -- start start 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.573+0000 7fc3588d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc354069d60 0x7fc3541573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3540aca80 0x7fc35415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540b1630 con 0x7fc3540c83c0 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415a100 con 0x7fc3540aca80 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540afb70 con 0x7fc354069d60 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54670/0 (socket says 172.21.15.170:54670) 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 -- 172.21.15.170:0/763836762 learned_addr learned my addr 172.21.15.170:0/763836762 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc35259c700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc354069d60 0x7fc3541573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc349d9b700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3540aca80 0x7fc35415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 -- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc354069d60 msgr2=0x7fc3541573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc354069d60 0x7fc3541573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 -- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3540aca80 msgr2=0x7fc35415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3540aca80 0x7fc35415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 -- 172.21.15.170:0/763836762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c009a20 con 0x7fc3540c83c0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc352d9d700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc35406c6a0 tx=0x7fc34c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01b630 con 0x7fc3540c83c0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34c01b790 con 0x7fc3540c83c0 2023-10-08T16:05:19.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3540b1880 con 0x7fc3540c83c0 2023-10-08T16:05:19.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c01bb50 con 0x7fc3540c83c0 2023-10-08T16:05:19.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.574+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3540b1e40 con 0x7fc3540c83c0 2023-10-08T16:05:19.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.575+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35404c3c0 con 0x7fc3540c83c0 2023-10-08T16:05:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.576+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc34c01c400 con 0x7fc3540c83c0 2023-10-08T16:05:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.576+0000 7fc34affd700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 0x7fc3340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.576+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc34c086570 con 0x7fc3540c83c0 2023-10-08T16:05:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.576+0000 7fc35259c700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 0x7fc3340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.577+0000 7fc35259c700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 0x7fc3340693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b05c0 tx=0x7fc344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.581+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc34c0537e0 con 0x7fc3540c83c0 2023-10-08T16:05:19.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.764+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc35406b510 con 0x7fc3540c83c0 2023-10-08T16:05:19.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.765+0000 7fc34affd700 1 -- 172.21.15.170:0/763836762 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fc34c056e00 con 0x7fc3540c83c0 2023-10-08T16:05:19.765 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:19.765 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":31,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:18.452900+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 msgr2=0x7fc3340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 0x7fc3340693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc3540b05c0 tx=0x7fc344009040 comp rx=0 tx=0).stop 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 msgr2=0x7fc35415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc35406c6a0 tx=0x7fc34c019000 comp rx=0 tx=0).stop 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 shutdown_connections 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc334066f30 0x7fc3340693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3540c83c0 0x7fc35415e530 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc354069d60 0x7fc3541573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 --2- 172.21.15.170:0/763836762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3540aca80 0x7fc35415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 >> 172.21.15.170:0/763836762 conn(0x7fc3540cd670 msgr2=0x7fc354068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 shutdown_connections 2023-10-08T16:05:19.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.767+0000 7fc3588d6700 1 -- 172.21.15.170:0/763836762 wait complete. 2023-10-08T16:05:19.780 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.894+0000 7f6e221b6700 1 Processor -- start 2023-10-08T16:05:19.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 -- start start 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06bcd0 0x7f6e1c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6e1c06c0a0 0x7f6e1c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e1c15df50 0x7f6e1c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c15d7e0 con 0x7f6e1c15df50 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e1c15dac0 con 0x7f6e1c06c0a0 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e221b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c15aa30 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.895+0000 7f6e1bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e1c15df50 0x7f6e1c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e1c15df50 0x7f6e1c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54686/0 (socket says 172.21.15.170:54686) 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 -- 172.21.15.170:0/660215424 learned_addr learned my addr 172.21.15.170:0/660215424 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1b7fe700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06bcd0 0x7f6e1c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1affd700 1 --2- 172.21.15.170:0/660215424 >> v2:172.21.15.191:3300/0 conn(0x7f6e1c06c0a0 0x7f6e1c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 -- 172.21.15.170:0/660215424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06bcd0 msgr2=0x7f6e1c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06bcd0 0x7f6e1c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 -- 172.21.15.170:0/660215424 >> v2:172.21.15.191:3300/0 conn(0x7f6e1c06c0a0 msgr2=0x7f6e1c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 --2- 172.21.15.170:0/660215424 >> v2:172.21.15.191:3300/0 conn(0x7f6e1c06c0a0 0x7f6e1c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 -- 172.21.15.170:0/660215424 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1c15ad10 con 0x7f6e1c15df50 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1bfff700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15e3b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0c5660 tx=0x7f6e14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525b343cfa19abb5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1a7fc700 1 -- 172.21.15.170:0/660215424 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400e070 con 0x7f6e1c15df50 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1a7fc700 1 -- 172.21.15.170:0/660215424 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1400b970 con 0x7f6e1c15df50 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.896+0000 7f6e1a7fc700 1 -- 172.21.15.170:0/660215424 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e140092e0 con 0x7f6e1c15df50 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 -- 172.21.15.170:0/660215424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c15df50 msgr2=0x7f6e1c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15e3b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0c5660 tx=0x7f6e14009840 comp rx=0 tx=0).stop 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 -- 172.21.15.170:0/660215424 shutdown_connections 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15e3b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 --2- 172.21.15.170:0/660215424 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06bcd0 0x7f6e1c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 --2- 172.21.15.170:0/660215424 >> v2:172.21.15.191:3300/0 conn(0x7f6e1c06c0a0 0x7f6e1c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 -- 172.21.15.170:0/660215424 >> 172.21.15.170:0/660215424 conn(0x7f6e1c0ce600 msgr2=0x7f6e1c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 -- 172.21.15.170:0/660215424 shutdown_connections 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.897+0000 7f6e221b6700 1 -- 172.21.15.170:0/660215424 wait complete. 2023-10-08T16:05:19.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.898+0000 7f6e221b6700 1 Processor -- start 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.898+0000 7f6e221b6700 1 -- start start 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06c0a0 0x7f6e1c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c1571f0 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16a7b0 con 0x7f6e1c15df50 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c16aa90 con 0x7f6e1c06c0a0 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54698/0 (socket says 172.21.15.170:54698) 2023-10-08T16:05:19.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 -- 172.21.15.170:0/1269540722 learned_addr learned my addr 172.21.15.170:0/1269540722 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1affd700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06c0a0 0x7f6e1c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1bfff700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 -- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06c0a0 msgr2=0x7f6e1c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06c0a0 0x7f6e1c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 -- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e1c15df50 msgr2=0x7f6e1c15f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 -- 172.21.15.170:0/1269540722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e14009ae0 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e1b7fe700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e930 tx=0x7f6e0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c013680 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c0137e0 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c16ad70 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c013b80 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.899+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c16b330 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.900+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c04c3c0 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.901+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e0c00c460 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.901+0000 7f6e037fe700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 0x7f6e080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.901+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e0c0846a0 con 0x7f6e1c06bcd0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.901+0000 7f6e1affd700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 0x7f6e080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.901+0000 7f6e1affd700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 0x7f6e080693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c15a6f0 tx=0x7f6e100091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:19.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:19.905+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e0c051910 con 0x7f6e1c06bcd0 2023-10-08T16:05:20.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.090+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e1c15aff0 con 0x7f6e1c06bcd0 2023-10-08T16:05:20.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.091+0000 7f6e037fe700 1 -- 172.21.15.170:0/1269540722 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f6e0c054f30 con 0x7f6e1c06bcd0 2023-10-08T16:05:20.091 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:20.092 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":32,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:20.079709+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 msgr2=0x7f6e080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 0x7f6e080693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c15a6f0 tx=0x7f6e100091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 msgr2=0x7f6e1c156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e930 tx=0x7f6e0c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 shutdown_connections 2023-10-08T16:05:20.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e1c15df50 0x7f6e1c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e08066f30 0x7f6e080693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e1c06bcd0 0x7f6e1c156cb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 --2- 172.21.15.170:0/1269540722 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e1c06c0a0 0x7f6e1c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.093+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 >> 172.21.15.170:0/1269540722 conn(0x7f6e1c0ce600 msgr2=0x7f6e1c069480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.094+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 shutdown_connections 2023-10-08T16:05:20.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.094+0000 7f6e221b6700 1 -- 172.21.15.170:0/1269540722 wait complete. 2023-10-08T16:05:20.106 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:05:20.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.223+0000 7f418679d700 1 Processor -- start 2023-10-08T16:05:20.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 -- start start 2023-10-08T16:05:20.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4180159a70 0x7f41800c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f418006bfa0 0x7f41800c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800ca920 con 0x7f4180068390 2023-10-08T16:05:20.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418006af00 con 0x7f418006bfa0 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.224+0000 7f418679d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f418006b1e0 con 0x7f4180159a70 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f418006bfa0 0x7f41800c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4185f9c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4180068390 0x7f41800caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60604/0 (socket says 172.21.15.170:60604) 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f418006bfa0 0x7f41800c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54706/0 (socket says 172.21.15.170:54706) 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4185f9c700 1 -- 172.21.15.170:0/3118195365 learned_addr learned my addr 172.21.15.170:0/3118195365 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f418579b700 1 --2- 172.21.15.170:0/3118195365 >> v2:172.21.15.191:3300/0 conn(0x7f4180159a70 0x7f41800c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 -- 172.21.15.170:0/3118195365 >> v2:172.21.15.191:3300/0 conn(0x7f4180159a70 msgr2=0x7f41800c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3118195365 >> v2:172.21.15.191:3300/0 conn(0x7f4180159a70 0x7f41800c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 -- 172.21.15.170:0/3118195365 >> v1:172.21.15.170:6789/0 conn(0x7f4180068390 legacy=0x7f41800caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 -- 172.21.15.170:0/3118195365 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418006b4c0 con 0x7f418006bfa0 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4177fff700 1 -- 172.21.15.170:0/3118195365 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47966813 0 0) 0x7f41800ca920 con 0x7f4180068390 2023-10-08T16:05:20.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3118195365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f418006bfa0 0x7f41800c8830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41800640b0 tx=0x7f4170009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf8b89d466072ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4177fff700 1 -- 172.21.15.170:0/3118195365 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170005c10 con 0x7f418006bfa0 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4177fff700 1 -- 172.21.15.170:0/3118195365 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170005d70 con 0x7f418006bfa0 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.225+0000 7f4177fff700 1 -- 172.21.15.170:0/3118195365 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170006260 con 0x7f418006bfa0 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 -- 172.21.15.170:0/3118195365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f418006bfa0 msgr2=0x7f41800c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 --2- 172.21.15.170:0/3118195365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f418006bfa0 0x7f41800c8830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41800640b0 tx=0x7f4170009120 comp rx=0 tx=0).stop 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 -- 172.21.15.170:0/3118195365 shutdown_connections 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 --2- 172.21.15.170:0/3118195365 >> v2:172.21.15.191:3300/0 conn(0x7f4180159a70 0x7f41800c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 --2- 172.21.15.170:0/3118195365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f418006bfa0 0x7f41800c8830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 -- 172.21.15.170:0/3118195365 >> 172.21.15.170:0/3118195365 conn(0x7f41800cddc0 msgr2=0x7f41800c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 -- 172.21.15.170:0/3118195365 shutdown_connections 2023-10-08T16:05:20.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.226+0000 7f418679d700 1 -- 172.21.15.170:0/3118195365 wait complete. 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.227+0000 7f418679d700 1 Processor -- start 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.227+0000 7f418679d700 1 -- start start 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 0x7f4180150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4180159a70 0x7f41800b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4180155b80 con 0x7f4180068390 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800b14b0 con 0x7f4180159a70 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418679d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4180155530 con 0x7f418006bfa0 2023-10-08T16:05:20.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54708/0 (socket says 172.21.15.170:54708) 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 -- 172.21.15.170:0/3039683532 learned_addr learned my addr 172.21.15.170:0/3039683532 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f4185f9c700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4180159a70 0x7f41800b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 0x7f4180150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 -- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 msgr2=0x7f4180150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 0x7f4180150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 -- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4180159a70 msgr2=0x7f41800b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4180159a70 0x7f41800b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 -- 172.21.15.170:0/3039683532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700093c0 con 0x7f4180068390 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 0x7f4180150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.228+0000 7f418579b700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f417c00d860 tx=0x7f417c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.229+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0125f0 con 0x7f4180068390 2023-10-08T16:05:20.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.229+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c012750 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.229+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4180155810 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.229+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c012b10 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.229+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800c58b0 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.230+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4180155e60 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.230+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f417c01e460 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.230+0000 7f41757fa700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 0x7f41640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.230+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f417c084ad0 con 0x7f4180068390 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.230+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 0x7f41640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:20.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.231+0000 7f4184f9a700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 0x7f41640693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4170009160 tx=0x7f4170016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:20.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.235+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f417c051d40 con 0x7f4180068390 2023-10-08T16:05:20.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:20.459+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f418006b7a0 con 0x7f4180068390 2023-10-08T16:05:21.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.438+0000 7f41757fa700 1 -- 172.21.15.170:0/3039683532 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f417c055360 con 0x7f4180068390 2023-10-08T16:05:21.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 msgr2=0x7f41640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 0x7f41640693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4170009160 tx=0x7f4170016000 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 msgr2=0x7f41800c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f417c00d860 tx=0x7f417c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 shutdown_connections 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4180159a70 0x7f41800b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4164066f30 0x7f41640693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4180068390 0x7f41800c5370 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 --2- 172.21.15.170:0/3039683532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f418006bfa0 0x7f4180150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 >> 172.21.15.170:0/3039683532 conn(0x7f41800cddc0 msgr2=0x7f4180150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 shutdown_connections 2023-10-08T16:05:21.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.441+0000 7f418679d700 1 -- 172.21.15.170:0/3039683532 wait complete. 2023-10-08T16:05:21.457 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:05:21.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.579+0000 7f18babc7700 1 Processor -- start 2023-10-08T16:05:21.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.579+0000 7f18babc7700 1 -- start start 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18babc7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f18b4069560 0x7f18b40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18babc7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b4069ee0 con 0x7f18b4159c60 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18babc7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b415a160 con 0x7f18b4159890 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18babc7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b40d13a0 con 0x7f18b4069560 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b3fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f18b4159c60 0x7f18b415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60614/0 (socket says 172.21.15.170:60614) 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b3fff700 1 -- 172.21.15.170:0/1761761260 learned_addr learned my addr 172.21.15.170:0/1761761260 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b9164700 1 --2- 172.21.15.170:0/1761761260 >> v2:172.21.15.191:3300/0 conn(0x7f18b4069560 0x7f18b40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743518154 0 0) 0x7f18b4069ee0 con 0x7f18b4159c60 2023-10-08T16:05:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003570 con 0x7f18b4159c60 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.580+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300093121 0 0) 0x7f189c003570 con 0x7f18b4159c60 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 >> v2:172.21.15.191:3300/0 conn(0x7f18b4069560 msgr2=0x7f18b40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 --2- 172.21.15.170:0/1761761260 >> v2:172.21.15.191:3300/0 conn(0x7f18b4069560 0x7f18b40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 >> v1:172.21.15.191:6789/0 conn(0x7f18b4159890 legacy=0x7f18b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b415e140 con 0x7f18b4159c60 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f18a8001470 con 0x7f18b4159c60 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 >> v1:172.21.15.170:6789/0 conn(0x7f18b4159c60 legacy=0x7f18b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189c003a60 0x7f189c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 0x7f189c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c0051c0 0x7f189c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40d13a0 con 0x7f189c004810 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c050 con 0x7f189c003a60 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c210 con 0x7f189c0051c0 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 0x7f189c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b8963700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189c003a60 0x7f189c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b9164700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c0051c0 0x7f189c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 -- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c0051c0 msgr2=0x7f189c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c0051c0 0x7f189c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 -- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189c003a60 msgr2=0x7f189c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f189c003a60 0x7f189c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 -- 172.21.15.170:0/1761761260 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c003570 con 0x7f189c004810 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b9164700 1 -- 172.21.15.170:0/1761761260 reap_dead start 2023-10-08T16:05:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b3fff700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 0x7f189c004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c3c0 tx=0x7f18a8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a7e1397ec84382 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800cc10 con 0x7f189c004810 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a800cd70 con 0x7f189c004810 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.581+0000 7f18b37fe700 1 -- 172.21.15.170:0/1761761260 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8005b70 con 0x7f189c004810 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 -- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 msgr2=0x7f189c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 0x7f189c004c80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18a800c3c0 tx=0x7f18a8001e70 comp rx=0 tx=0).stop 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 -- 172.21.15.170:0/1761761260 shutdown_connections 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 --2- 172.21.15.170:0/1761761260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f189c004810 0x7f189c004c80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 -- 172.21.15.170:0/1761761260 >> 172.21.15.170:0/1761761260 conn(0x7f18b4068e20 msgr2=0x7f18b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 -- 172.21.15.170:0/1761761260 shutdown_connections 2023-10-08T16:05:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.582+0000 7f18babc7700 1 -- 172.21.15.170:0/1761761260 wait complete. 2023-10-08T16:05:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.583+0000 7f18babc7700 1 Processor -- start 2023-10-08T16:05:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.583+0000 7f18babc7700 1 -- start start 2023-10-08T16:05:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c004810 0x7f18b414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 0x7f18b40ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40c5f40 con 0x7f18b40c5590 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4151030 con 0x7f18b40cdc90 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18babc7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b40c6b30 con 0x7f189c004810 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54724/0 (socket says 172.21.15.170:54724) 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 -- 172.21.15.170:0/485786016 learned_addr learned my addr 172.21.15.170:0/485786016 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b9164700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 0x7f18b40ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b8963700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c004810 0x7f18b414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 -- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c004810 msgr2=0x7f18b414d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c004810 0x7f18b414d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 -- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 msgr2=0x7f18b40ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 0x7f18b40ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 -- 172.21.15.170:0/485786016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40ce640 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b9164700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 0x7f18b40ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b3fff700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f18a8004b40 tx=0x7f18a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.584+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8022070 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.585+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8004e80 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.585+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b40ce8f0 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.585+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a800cc10 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.585+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b4151560 con 0x7f18b40c5590 2023-10-08T16:05:21.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.586+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b404c3c0 con 0x7f18b40c5590 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.586+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18a801e430 con 0x7f18b40c5590 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.586+0000 7f18b0ff9700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.586+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18a8085910 con 0x7f18b40c5590 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.586+0000 7f18b8963700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.587+0000 7f18b8963700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18a4010300 tx=0x7f18a40016d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:21.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.591+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a80b65a0 con 0x7f18b40c5590 2023-10-08T16:05:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:21.815+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f18b406a5b0 con 0x7f18b40c5590 2023-10-08T16:05:21.992 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:05:21.991+0000 7fe8dee1b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:05:22.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.438+0000 7f18b0ff9700 1 -- 172.21.15.170:0/485786016 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f18a8017070 con 0x7f18b40c5590 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 msgr2=0x7f18a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 0x7f18a00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f18a4010300 tx=0x7f18a40016d0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 msgr2=0x7f18b40c5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f18a8004b40 tx=0x7f18a8015000 comp rx=0 tx=0).stop 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 shutdown_connections 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b40cdc90 0x7f18b40ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f189c004810 0x7f18b414d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f18a0066f30 0x7f18a00693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 --2- 172.21.15.170:0/485786016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b40c5590 0x7f18b40c5a00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 >> 172.21.15.170:0/485786016 conn(0x7f18b4068e20 msgr2=0x7f18b41573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 shutdown_connections 2023-10-08T16:05:22.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.440+0000 7f18babc7700 1 -- 172.21.15.170:0/485786016 wait complete. 2023-10-08T16:05:22.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:22.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.568+0000 7f686ec8d700 1 Processor -- start 2023-10-08T16:05:22.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.568+0000 7f686ec8d700 1 -- start start 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686ec8d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 0x7f68680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686ec8d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f686815d230 0x7f686815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686ec8d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686815dbd0 con 0x7f6868159420 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686ec8d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68680b11d0 con 0x7f686815d230 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686ec8d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68680b14b0 con 0x7f68681597f0 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f686815d230 0x7f686815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f686815d230 0x7f686815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54736/0 (socket says 172.21.15.170:54736) 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 -- 172.21.15.170:0/2860804827 learned_addr learned my addr 172.21.15.170:0/2860804827 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686d48a700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 0x7f68680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 -- 172.21.15.170:0/2860804827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 msgr2=0x7f68680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 0x7f68680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 -- 172.21.15.170:0/2860804827 >> v1:172.21.15.191:6789/0 conn(0x7f6868159420 legacy=0x7f68680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686e48c700 1 -- 172.21.15.170:0/2860804827 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68680b1790 con 0x7f686815d230 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.569+0000 7f686d48a700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 0x7f68680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686e48c700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f686815d690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f68600099b0 tx=0x7f6860009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50555d45e769213c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686cc89700 1 -- 172.21.15.170:0/2860804827 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000b860 con 0x7f686815d230 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686cc89700 1 -- 172.21.15.170:0/2860804827 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686000b9c0 con 0x7f686815d230 2023-10-08T16:05:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686cc89700 1 -- 172.21.15.170:0/2860804827 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000bd60 con 0x7f686815d230 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 -- 172.21.15.170:0/2860804827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 msgr2=0x7f686815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f686815d690 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f68600099b0 tx=0x7f6860009250 comp rx=0 tx=0).stop 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 -- 172.21.15.170:0/2860804827 shutdown_connections 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f686815d690 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 --2- 172.21.15.170:0/2860804827 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68681597f0 0x7f68680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 -- 172.21.15.170:0/2860804827 >> 172.21.15.170:0/2860804827 conn(0x7f68680d0660 msgr2=0x7f68680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 -- 172.21.15.170:0/2860804827 shutdown_connections 2023-10-08T16:05:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.570+0000 7f686ec8d700 1 -- 172.21.15.170:0/2860804827 wait complete. 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.571+0000 7f686ec8d700 1 Processor -- start 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.571+0000 7f686ec8d700 1 -- start start 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6868159420 0x7f6868157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68681597f0 0x7f68680c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68680c9280 con 0x7f686815d230 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68680c9410 con 0x7f68681597f0 2023-10-08T16:05:22.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686ec8d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68681576c0 con 0x7f6868159420 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54738/0 (socket says 172.21.15.170:54738) 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 -- 172.21.15.170:0/1560829846 learned_addr learned my addr 172.21.15.170:0/1560829846 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686dc8b700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6868159420 0x7f6868157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686d48a700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68681597f0 0x7f68680c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 -- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6868159420 msgr2=0x7f6868157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6868159420 0x7f6868157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 -- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68681597f0 msgr2=0x7f68680c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68681597f0 0x7f68680c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.572+0000 7f686e48c700 1 -- 172.21.15.170:0/1560829846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600094f0 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f686e48c700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6860009290 tx=0x7f6860017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000b860 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686001f070 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68680c73d0 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860011ea0 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.573+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68680c7a10 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.574+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68680c7560 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.574+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f686001ab40 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.575+0000 7f685dffb700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.575+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6860021070 con 0x7f686815d230 2023-10-08T16:05:22.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.575+0000 7f686dc8b700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.575+0000 7f686dc8b700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6864007fd0 tx=0x7f68640016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.579+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6860053650 con 0x7f686815d230 2023-10-08T16:05:22.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.804+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68680b1a70 con 0x7f686815d230 2023-10-08T16:05:22.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.804+0000 7f685dffb700 1 -- 172.21.15.170:0/1560829846 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f6860015090 con 0x7f686815d230 2023-10-08T16:05:22.806 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:22.807 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2746556501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2746556501},{"type":"v1","addr":"172.21.15.170:6836","nonce":2746556501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 msgr2=0x7f684c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 0x7f684c0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6864007fd0 tx=0x7f68640016f0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 msgr2=0x7f68680b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6860009290 tx=0x7f6860017000 comp rx=0 tx=0).stop 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 shutdown_connections 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f684c066f30 0x7f684c0693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f686815d230 0x7f68680b5a90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6868159420 0x7f6868157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 --2- 172.21.15.170:0/1560829846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68681597f0 0x7f68680c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 >> 172.21.15.170:0/1560829846 conn(0x7f68680d0660 msgr2=0x7f68680ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:22.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 shutdown_connections 2023-10-08T16:05:22.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.807+0000 7f686ec8d700 1 -- 172.21.15.170:0/1560829846 wait complete. 2023-10-08T16:05:22.810 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 17 2023-10-08T16:05:22.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.932+0000 7f75259ee700 1 Processor -- start 2023-10-08T16:05:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.932+0000 7f75259ee700 1 -- start start 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75259ee700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7520159420 0x7f75200b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75259ee700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f75201597f0 0x7f75200b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75259ee700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75200b11d0 con 0x7f752015d230 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75259ee700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75200b14b0 con 0x7f7520159420 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75259ee700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75200b1790 con 0x7f75201597f0 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75249ec700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7520159420 0x7f75200b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75251ed700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f752015d230 0x7f752015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60624/0 (socket says 172.21.15.170:60624) 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75251ed700 1 -- 172.21.15.170:0/1111152134 learned_addr learned my addr 172.21.15.170:0/1111152134 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f7517fff700 1 --2- 172.21.15.170:0/1111152134 >> v2:172.21.15.191:3300/0 conn(0x7f75201597f0 0x7f75200b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75249ec700 1 -- 172.21.15.170:0/1111152134 >> v2:172.21.15.191:3300/0 conn(0x7f75201597f0 msgr2=0x7f75200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75249ec700 1 --2- 172.21.15.170:0/1111152134 >> v2:172.21.15.191:3300/0 conn(0x7f75201597f0 0x7f75200b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75249ec700 1 -- 172.21.15.170:0/1111152134 >> v1:172.21.15.170:6789/0 conn(0x7f752015d230 legacy=0x7f752015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75249ec700 1 -- 172.21.15.170:0/1111152134 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200b1a70 con 0x7f7520159420 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.933+0000 7f75177fe700 1 -- 172.21.15.170:0/1111152134 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693452511 0 0) 0x7f75200b11d0 con 0x7f752015d230 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75249ec700 1 --2- 172.21.15.170:0/1111152134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7520159420 0x7f75200b5f40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7520089750 tx=0x7f751c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ea7df1283dc627 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75177fe700 1 -- 172.21.15.170:0/1111152134 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c00e070 con 0x7f7520159420 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75177fe700 1 -- 172.21.15.170:0/1111152134 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c00bb60 con 0x7f7520159420 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75177fe700 1 -- 172.21.15.170:0/1111152134 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c012430 con 0x7f7520159420 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 -- 172.21.15.170:0/1111152134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7520159420 msgr2=0x7f75200b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 --2- 172.21.15.170:0/1111152134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7520159420 0x7f75200b5f40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7520089750 tx=0x7f751c009780 comp rx=0 tx=0).stop 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 -- 172.21.15.170:0/1111152134 shutdown_connections 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 --2- 172.21.15.170:0/1111152134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7520159420 0x7f75200b5f40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 --2- 172.21.15.170:0/1111152134 >> v2:172.21.15.191:3300/0 conn(0x7f75201597f0 0x7f75200b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 -- 172.21.15.170:0/1111152134 >> 172.21.15.170:0/1111152134 conn(0x7f75200d0660 msgr2=0x7f75200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 -- 172.21.15.170:0/1111152134 shutdown_connections 2023-10-08T16:05:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.934+0000 7f75259ee700 1 -- 172.21.15.170:0/1111152134 wait complete. 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.935+0000 7f75259ee700 1 Processor -- start 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 -- start start 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7520159420 0x7f752015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75201597f0 0x7f75200c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7520157700 con 0x7f752015d230 2023-10-08T16:05:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752015abe0 con 0x7f7520159420 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75259ee700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752015aec0 con 0x7f75201597f0 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75251ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75251ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54762/0 (socket says 172.21.15.170:54762) 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.936+0000 7f75251ed700 1 -- 172.21.15.170:0/850187815 learned_addr learned my addr 172.21.15.170:0/850187815 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75249ec700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7520159420 0x7f752015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f7517fff700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75201597f0 0x7f75200c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 -- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75201597f0 msgr2=0x7f75200c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75201597f0 0x7f75200c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 -- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7520159420 msgr2=0x7f752015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7520159420 0x7f752015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 -- 172.21.15.170:0/850187815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009a20 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75251ed700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f751800c4c0 tx=0x7f7518007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751800ce70 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518005b30 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752015b1a0 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180070d0 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.937+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75200c7f30 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.938+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75200c8150 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.938+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7518007480 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.939+0000 7f7514ff9700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.939+0000 7f75249ec700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.939+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7518083910 con 0x7f752015d230 2023-10-08T16:05:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.939+0000 7f75249ec700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f751c009fd0 tx=0x7f751c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:22.943+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7518050af0 con 0x7f752015d230 2023-10-08T16:05:23.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.128+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f752004c3c0 con 0x7f752015d230 2023-10-08T16:05:23.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.128+0000 7f7514ff9700 1 -- 172.21.15.170:0/850187815 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7518017020 con 0x7f752015d230 2023-10-08T16:05:23.130 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:23.130 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":32,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:20.079709+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 msgr2=0x7f750c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f751c009fd0 tx=0x7f751c019000 comp rx=0 tx=0).stop 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 msgr2=0x7f75200cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f751800c4c0 tx=0x7f7518007f20 comp rx=0 tx=0).stop 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 shutdown_connections 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f752015d230 0x7f75200cba80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7520159420 0x7f752015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 --2- 172.21.15.170:0/850187815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75201597f0 0x7f75200c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 >> 172.21.15.170:0/850187815 conn(0x7f75200d0660 msgr2=0x7f7520069810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:23.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.130+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 shutdown_connections 2023-10-08T16:05:23.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.131+0000 7f75259ee700 1 -- 172.21.15.170:0/850187815 wait complete. 2023-10-08T16:05:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:05:23.145 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:05:23.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.258+0000 7fd991a44700 1 Processor -- start 2023-10-08T16:05:23.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.258+0000 7fd991a44700 1 -- start start 2023-10-08T16:05:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd991a44700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd98c06e520 0x7fd98c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd991a44700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c0c7af0 con 0x7fd98c159760 2023-10-08T16:05:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd991a44700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd98c0c74e0 con 0x7fd98c159b30 2023-10-08T16:05:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd991a44700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98c06eee0 con 0x7fd98c06e520 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd98affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd98c159760 0x7fd98c0c7f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60634/0 (socket says 172.21.15.170:60634) 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd98affd700 1 -- 172.21.15.170:0/3069927367 learned_addr learned my addr 172.21.15.170:0/3069927367 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd98b7fe700 1 --2- 172.21.15.170:0/3069927367 >> v2:172.21.15.191:3300/0 conn(0x7fd98c06e520 0x7fd98c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd98a7fc700 1 -- 172.21.15.170:0/3069927367 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285145013 0 0) 0x7fd98c0c7af0 con 0x7fd98c159760 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.259+0000 7fd98a7fc700 1 -- 172.21.15.170:0/3069927367 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd978003570 con 0x7fd98c159760 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98b7fe700 1 -- 172.21.15.170:0/3069927367 >> v1:172.21.15.191:6789/0 conn(0x7fd98c159b30 legacy=0x7fd98c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98b7fe700 1 -- 172.21.15.170:0/3069927367 >> v1:172.21.15.170:6789/0 conn(0x7fd98c159760 legacy=0x7fd98c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98b7fe700 1 -- 172.21.15.170:0/3069927367 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0cc870 con 0x7fd98c06e520 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98b7fe700 1 --2- 172.21.15.170:0/3069927367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c06e520 0x7fd98c06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd984009e60 tx=0x7fd984009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4e62ecf1481422d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98a7fc700 1 -- 172.21.15.170:0/3069927367 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98400e070 con 0x7fd98c06e520 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98a7fc700 1 -- 172.21.15.170:0/3069927367 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98400bb60 con 0x7fd98c06e520 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd98a7fc700 1 -- 172.21.15.170:0/3069927367 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984012430 con 0x7fd98c06e520 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.260+0000 7fd991a44700 1 -- 172.21.15.170:0/3069927367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c06e520 msgr2=0x7fd98c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 --2- 172.21.15.170:0/3069927367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c06e520 0x7fd98c06e9a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd984009e60 tx=0x7fd984009780 comp rx=0 tx=0).stop 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 -- 172.21.15.170:0/3069927367 shutdown_connections 2023-10-08T16:05:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 --2- 172.21.15.170:0/3069927367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c06e520 0x7fd98c06e9a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 -- 172.21.15.170:0/3069927367 >> 172.21.15.170:0/3069927367 conn(0x7fd98c0ca060 msgr2=0x7fd98c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 -- 172.21.15.170:0/3069927367 shutdown_connections 2023-10-08T16:05:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.261+0000 7fd991a44700 1 -- 172.21.15.170:0/3069927367 wait complete. 2023-10-08T16:05:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.262+0000 7fd991a44700 1 Processor -- start 2023-10-08T16:05:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.262+0000 7fd991a44700 1 -- start start 2023-10-08T16:05:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 0x7fd98c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c159b30 0x7fd98c0c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c169e00 con 0x7fd98c159760 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c1576a0 con 0x7fd98c159b30 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c0b1090 con 0x7fd98c06e520 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54770/0 (socket says 172.21.15.170:54770) 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 -- 172.21.15.170:0/1506098561 learned_addr learned my addr 172.21.15.170:0/1506098561 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd98affd700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 0x7fd98c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd98b7fe700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c159b30 0x7fd98c0c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 -- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 msgr2=0x7fd98c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 0x7fd98c153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 -- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c159b30 msgr2=0x7fd98c0c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c159b30 0x7fd98c0c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 -- 172.21.15.170:0/1506098561 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd984009a20 con 0x7fd98c159760 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd98affd700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 0x7fd98c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9827fc700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c5c0 tx=0x7fd974007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011590 con 0x7fd98c159760 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd974011710 con 0x7fd98c159760 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd98c1555b0 con 0x7fd98c159760 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974011ad0 con 0x7fd98c159760 2023-10-08T16:05:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.263+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98c0b0d20 con 0x7fd98c159760 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.264+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd98c04c3c0 con 0x7fd98c159760 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.265+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd97401c440 con 0x7fd98c159760 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.265+0000 7fd9837fe700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 0x7fd96c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.265+0000 7fd98affd700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 0x7fd96c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.265+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd974083940 con 0x7fd98c159760 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.266+0000 7fd98affd700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 0x7fd96c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0058d0 tx=0x7fd97c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.269+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd974050bb0 con 0x7fd98c159760 2023-10-08T16:05:23.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.451+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd98c0b11d0 con 0x7fd96c066f30 2023-10-08T16:05:23.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.458+0000 7fd9837fe700 1 -- 172.21.15.170:0/1506098561 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264641 (secure 0 0 0) 0x7fd98c0b11d0 con 0x7fd96c066f30 2023-10-08T16:05:23.463 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:23.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 msgr2=0x7fd96c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 0x7fd96c0693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0058d0 tx=0x7fd97c009040 comp rx=0 tx=0).stop 2023-10-08T16:05:23.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 msgr2=0x7fd98c157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd97400c5c0 tx=0x7fd974007f50 comp rx=0 tx=0).stop 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 shutdown_connections 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd98c159b30 0x7fd98c0c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd98c06e520 0x7fd98c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd96c066f30 0x7fd96c0693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 --2- 172.21.15.170:0/1506098561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd98c159760 0x7fd98c157160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 >> 172.21.15.170:0/1506098561 conn(0x7fd98c0ca060 msgr2=0x7fd98c15a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 shutdown_connections 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.462+0000 7fd991a44700 1 -- 172.21.15.170:0/1506098561 wait complete. 2023-10-08T16:05:23.466 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:05:23.475 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-08T16:05:22.233144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591782,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382136,"kb_used_data":3832,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592584,"statfs":{"total":773094113280,"available":770654806016,"internally_reserved":0,"allocated":3923968,"data_stored":2210364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1414,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.168249"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.098175+0000","last_change":"2023-10-08T16:05:19.098175+0000","last_active":"2023-10-08T16:05:19.098175+0000","last_peered":"2023-10-08T16:05:19.098175+0000","last_clean":"2023-10-08T16:05:19.098175+0000","last_became_active":"2023-10-08T16:05:19.097629+0000","last_became_peered":"2023-10-08T16:05:19.097629+0000","last_unstale":"2023-10-08T16:05:19.098175+0000","last_undegraded":"2023-10-08T16:05:19.098175+0000","last_fullsized":"2023-10-08T16:05:19.098175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:00.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089604+0000","last_change":"2023-10-08T16:05:19.089604+0000","last_active":"2023-10-08T16:05:19.089604+0000","last_peered":"2023-10-08T16:05:19.089604+0000","last_clean":"2023-10-08T16:05:19.089604+0000","last_became_active":"2023-10-08T16:05:19.088983+0000","last_became_peered":"2023-10-08T16:05:19.088983+0000","last_unstale":"2023-10-08T16:05:19.089604+0000","last_undegraded":"2023-10-08T16:05:19.089604+0000","last_fullsized":"2023-10-08T16:05:19.089604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:58:03.535208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096896+0000","last_change":"2023-10-08T16:05:19.096896+0000","last_active":"2023-10-08T16:05:19.096896+0000","last_peered":"2023-10-08T16:05:19.096896+0000","last_clean":"2023-10-08T16:05:19.096896+0000","last_became_active":"2023-10-08T16:05:19.096619+0000","last_became_peered":"2023-10-08T16:05:19.096619+0000","last_unstale":"2023-10-08T16:05:19.096896+0000","last_undegraded":"2023-10-08T16:05:19.096896+0000","last_fullsized":"2023-10-08T16:05:19.096896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101732+0000","last_change":"2023-10-08T16:05:19.101732+0000","last_active":"2023-10-08T16:05:19.101732+0000","last_peered":"2023-10-08T16:05:19.101732+0000","last_clean":"2023-10-08T16:05:19.101732+0000","last_became_active":"2023-10-08T16:05:19.101477+0000","last_became_peered":"2023-10-08T16:05:19.101477+0000","last_unstale":"2023-10-08T16:05:19.101732+0000","last_undegraded":"2023-10-08T16:05:19.101732+0000","last_fullsized":"2023-10-08T16:05:19.101732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092438+0000","last_change":"2023-10-08T16:05:19.092438+0000","last_active":"2023-10-08T16:05:19.092438+0000","last_peered":"2023-10-08T16:05:19.092438+0000","last_clean":"2023-10-08T16:05:19.092438+0000","last_became_active":"2023-10-08T16:05:19.091865+0000","last_became_peered":"2023-10-08T16:05:19.091865+0000","last_unstale":"2023-10-08T16:05:19.092438+0000","last_undegraded":"2023-10-08T16:05:19.092438+0000","last_fullsized":"2023-10-08T16:05:19.092438+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:55.238889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097684+0000","last_change":"2023-10-08T16:05:19.097684+0000","last_active":"2023-10-08T16:05:19.097684+0000","last_peered":"2023-10-08T16:05:19.097684+0000","last_clean":"2023-10-08T16:05:19.097684+0000","last_became_active":"2023-10-08T16:05:19.097271+0000","last_became_peered":"2023-10-08T16:05:19.097271+0000","last_unstale":"2023-10-08T16:05:19.097684+0000","last_undegraded":"2023-10-08T16:05:19.097684+0000","last_fullsized":"2023-10-08T16:05:19.097684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:38.851839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096733+0000","last_change":"2023-10-08T16:05:19.096733+0000","last_active":"2023-10-08T16:05:19.096733+0000","last_peered":"2023-10-08T16:05:19.096733+0000","last_clean":"2023-10-08T16:05:19.096733+0000","last_became_active":"2023-10-08T16:05:19.096437+0000","last_became_peered":"2023-10-08T16:05:19.096437+0000","last_unstale":"2023-10-08T16:05:19.096733+0000","last_undegraded":"2023-10-08T16:05:19.096733+0000","last_fullsized":"2023-10-08T16:05:19.096733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090809+0000","last_change":"2023-10-08T16:05:19.090809+0000","last_active":"2023-10-08T16:05:19.090809+0000","last_peered":"2023-10-08T16:05:19.090809+0000","last_clean":"2023-10-08T16:05:19.090809+0000","last_became_active":"2023-10-08T16:05:19.090194+0000","last_became_peered":"2023-10-08T16:05:19.090194+0000","last_unstale":"2023-10-08T16:05:19.090809+0000","last_undegraded":"2023-10-08T16:05:19.090809+0000","last_fullsized":"2023-10-08T16:05:19.090809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:54.597317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.099726+0000","last_change":"2023-10-08T16:05:19.099726+0000","last_active":"2023-10-08T16:05:19.099726+0000","last_peered":"2023-10-08T16:05:19.099726+0000","last_clean":"2023-10-08T16:05:19.099726+0000","last_became_active":"2023-10-08T16:05:19.099363+0000","last_became_peered":"2023-10-08T16:05:19.099363+0000","last_unstale":"2023-10-08T16:05:19.099726+0000","last_undegraded":"2023-10-08T16:05:19.099726+0000","last_fullsized":"2023-10-08T16:05:19.099726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092185+0000","last_change":"2023-10-08T16:05:19.092185+0000","last_active":"2023-10-08T16:05:19.092185+0000","last_peered":"2023-10-08T16:05:19.092185+0000","last_clean":"2023-10-08T16:05:19.092185+0000","last_became_active":"2023-10-08T16:05:19.091943+0000","last_became_peered":"2023-10-08T16:05:19.091943+0000","last_unstale":"2023-10-08T16:05:19.092185+0000","last_undegraded":"2023-10-08T16:05:19.092185+0000","last_fullsized":"2023-10-08T16:05:19.092185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:54.597317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088808+0000","last_change":"2023-10-08T16:05:19.088808+0000","last_active":"2023-10-08T16:05:19.088808+0000","last_peered":"2023-10-08T16:05:19.088808+0000","last_clean":"2023-10-08T16:05:19.088808+0000","last_became_active":"2023-10-08T16:05:19.088511+0000","last_became_peered":"2023-10-08T16:05:19.088511+0000","last_unstale":"2023-10-08T16:05:19.088808+0000","last_undegraded":"2023-10-08T16:05:19.088808+0000","last_fullsized":"2023-10-08T16:05:19.088808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093662+0000","last_change":"2023-10-08T16:05:19.093662+0000","last_active":"2023-10-08T16:05:19.093662+0000","last_peered":"2023-10-08T16:05:19.093662+0000","last_clean":"2023-10-08T16:05:19.093662+0000","last_became_active":"2023-10-08T16:05:19.093288+0000","last_became_peered":"2023-10-08T16:05:19.093288+0000","last_unstale":"2023-10-08T16:05:19.093662+0000","last_undegraded":"2023-10-08T16:05:19.093662+0000","last_fullsized":"2023-10-08T16:05:19.093662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093605+0000","last_change":"2023-10-08T16:05:19.093605+0000","last_active":"2023-10-08T16:05:19.093605+0000","last_peered":"2023-10-08T16:05:19.093605+0000","last_clean":"2023-10-08T16:05:19.093605+0000","last_became_active":"2023-10-08T16:05:19.093369+0000","last_became_peered":"2023-10-08T16:05:19.093369+0000","last_unstale":"2023-10-08T16:05:19.093605+0000","last_undegraded":"2023-10-08T16:05:19.093605+0000","last_fullsized":"2023-10-08T16:05:19.093605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:40.590305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091886+0000","last_change":"2023-10-08T16:05:19.091886+0000","last_active":"2023-10-08T16:05:19.091886+0000","last_peered":"2023-10-08T16:05:19.091886+0000","last_clean":"2023-10-08T16:05:19.091886+0000","last_became_active":"2023-10-08T16:05:19.091635+0000","last_became_peered":"2023-10-08T16:05:19.091635+0000","last_unstale":"2023-10-08T16:05:19.091886+0000","last_undegraded":"2023-10-08T16:05:19.091886+0000","last_fullsized":"2023-10-08T16:05:19.091886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097244+0000","last_change":"2023-10-08T16:05:19.097244+0000","last_active":"2023-10-08T16:05:19.097244+0000","last_peered":"2023-10-08T16:05:19.097244+0000","last_clean":"2023-10-08T16:05:19.097244+0000","last_became_active":"2023-10-08T16:05:19.096480+0000","last_became_peered":"2023-10-08T16:05:19.096480+0000","last_unstale":"2023-10-08T16:05:19.097244+0000","last_undegraded":"2023-10-08T16:05:19.097244+0000","last_fullsized":"2023-10-08T16:05:19.097244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:22.416946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088393+0000","last_change":"2023-10-08T16:05:19.088393+0000","last_active":"2023-10-08T16:05:19.088393+0000","last_peered":"2023-10-08T16:05:19.088393+0000","last_clean":"2023-10-08T16:05:19.088393+0000","last_became_active":"2023-10-08T16:05:19.088099+0000","last_became_peered":"2023-10-08T16:05:19.088099+0000","last_unstale":"2023-10-08T16:05:19.088393+0000","last_undegraded":"2023-10-08T16:05:19.088393+0000","last_fullsized":"2023-10-08T16:05:19.088393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095984+0000","last_change":"2023-10-08T16:05:19.095984+0000","last_active":"2023-10-08T16:05:19.095984+0000","last_peered":"2023-10-08T16:05:19.095984+0000","last_clean":"2023-10-08T16:05:19.095984+0000","last_became_active":"2023-10-08T16:05:19.095723+0000","last_became_peered":"2023-10-08T16:05:19.095723+0000","last_unstale":"2023-10-08T16:05:19.095984+0000","last_undegraded":"2023-10-08T16:05:19.095984+0000","last_fullsized":"2023-10-08T16:05:19.095984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091048+0000","last_change":"2023-10-08T16:05:19.091048+0000","last_active":"2023-10-08T16:05:19.091048+0000","last_peered":"2023-10-08T16:05:19.091048+0000","last_clean":"2023-10-08T16:05:19.091048+0000","last_became_active":"2023-10-08T16:05:19.090800+0000","last_became_peered":"2023-10-08T16:05:19.090800+0000","last_unstale":"2023-10-08T16:05:19.091048+0000","last_undegraded":"2023-10-08T16:05:19.091048+0000","last_fullsized":"2023-10-08T16:05:19.091048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:22.416946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089847+0000","last_change":"2023-10-08T16:05:19.089847+0000","last_active":"2023-10-08T16:05:19.089847+0000","last_peered":"2023-10-08T16:05:19.089847+0000","last_clean":"2023-10-08T16:05:19.089847+0000","last_became_active":"2023-10-08T16:05:19.089603+0000","last_became_peered":"2023-10-08T16:05:19.089603+0000","last_unstale":"2023-10-08T16:05:19.089847+0000","last_undegraded":"2023-10-08T16:05:19.089847+0000","last_fullsized":"2023-10-08T16:05:19.089847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096510+0000","last_change":"2023-10-08T16:05:19.096510+0000","last_active":"2023-10-08T16:05:19.096510+0000","last_peered":"2023-10-08T16:05:19.096510+0000","last_clean":"2023-10-08T16:05:19.096510+0000","last_became_active":"2023-10-08T16:05:19.096275+0000","last_became_peered":"2023-10-08T16:05:19.096275+0000","last_unstale":"2023-10-08T16:05:19.096510+0000","last_undegraded":"2023-10-08T16:05:19.096510+0000","last_fullsized":"2023-10-08T16:05:19.096510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:38.495220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103512+0000","last_change":"2023-10-08T16:05:19.103512+0000","last_active":"2023-10-08T16:05:19.103512+0000","last_peered":"2023-10-08T16:05:19.103512+0000","last_clean":"2023-10-08T16:05:19.103512+0000","last_became_active":"2023-10-08T16:05:19.100387+0000","last_became_peered":"2023-10-08T16:05:19.100387+0000","last_unstale":"2023-10-08T16:05:19.103512+0000","last_undegraded":"2023-10-08T16:05:19.103512+0000","last_fullsized":"2023-10-08T16:05:19.103512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.100519+0000","last_change":"2023-10-08T16:05:19.100519+0000","last_active":"2023-10-08T16:05:19.100519+0000","last_peered":"2023-10-08T16:05:19.100519+0000","last_clean":"2023-10-08T16:05:19.100519+0000","last_became_active":"2023-10-08T16:05:19.099718+0000","last_became_peered":"2023-10-08T16:05:19.099718+0000","last_unstale":"2023-10-08T16:05:19.100519+0000","last_undegraded":"2023-10-08T16:05:19.100519+0000","last_fullsized":"2023-10-08T16:05:19.100519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:35.848197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091446+0000","last_change":"2023-10-08T16:05:19.091446+0000","last_active":"2023-10-08T16:05:19.091446+0000","last_peered":"2023-10-08T16:05:19.091446+0000","last_clean":"2023-10-08T16:05:19.091446+0000","last_became_active":"2023-10-08T16:05:19.091180+0000","last_became_peered":"2023-10-08T16:05:19.091180+0000","last_unstale":"2023-10-08T16:05:19.091446+0000","last_undegraded":"2023-10-08T16:05:19.091446+0000","last_fullsized":"2023-10-08T16:05:19.091446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:51.196559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091479+0000","last_change":"2023-10-08T16:05:19.091479+0000","last_active":"2023-10-08T16:05:19.091479+0000","last_peered":"2023-10-08T16:05:19.091479+0000","last_clean":"2023-10-08T16:05:19.091479+0000","last_became_active":"2023-10-08T16:05:19.091197+0000","last_became_peered":"2023-10-08T16:05:19.091197+0000","last_unstale":"2023-10-08T16:05:19.091479+0000","last_undegraded":"2023-10-08T16:05:19.091479+0000","last_fullsized":"2023-10-08T16:05:19.091479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088317+0000","last_change":"2023-10-08T16:05:19.088317+0000","last_active":"2023-10-08T16:05:19.088317+0000","last_peered":"2023-10-08T16:05:19.088317+0000","last_clean":"2023-10-08T16:05:19.088317+0000","last_became_active":"2023-10-08T16:05:19.088058+0000","last_became_peered":"2023-10-08T16:05:19.088058+0000","last_unstale":"2023-10-08T16:05:19.088317+0000","last_undegraded":"2023-10-08T16:05:19.088317+0000","last_fullsized":"2023-10-08T16:05:19.088317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089030+0000","last_change":"2023-10-08T16:05:19.089030+0000","last_active":"2023-10-08T16:05:19.089030+0000","last_peered":"2023-10-08T16:05:19.089030+0000","last_clean":"2023-10-08T16:05:19.089030+0000","last_became_active":"2023-10-08T16:05:19.088776+0000","last_became_peered":"2023-10-08T16:05:19.088776+0000","last_unstale":"2023-10-08T16:05:19.089030+0000","last_undegraded":"2023-10-08T16:05:19.089030+0000","last_fullsized":"2023-10-08T16:05:19.089030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101864+0000","last_change":"2023-10-08T16:05:19.101864+0000","last_active":"2023-10-08T16:05:19.101864+0000","last_peered":"2023-10-08T16:05:19.101864+0000","last_clean":"2023-10-08T16:05:19.101864+0000","last_became_active":"2023-10-08T16:05:19.101610+0000","last_became_peered":"2023-10-08T16:05:19.101610+0000","last_unstale":"2023-10-08T16:05:19.101864+0000","last_undegraded":"2023-10-08T16:05:19.101864+0000","last_fullsized":"2023-10-08T16:05:19.101864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:48.888237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095086+0000","last_change":"2023-10-08T16:05:19.095086+0000","last_active":"2023-10-08T16:05:19.095086+0000","last_peered":"2023-10-08T16:05:19.095086+0000","last_clean":"2023-10-08T16:05:19.095086+0000","last_became_active":"2023-10-08T16:05:19.094767+0000","last_became_peered":"2023-10-08T16:05:19.094767+0000","last_unstale":"2023-10-08T16:05:19.095086+0000","last_undegraded":"2023-10-08T16:05:19.095086+0000","last_fullsized":"2023-10-08T16:05:19.095086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101769+0000","last_change":"2023-10-08T16:05:19.101769+0000","last_active":"2023-10-08T16:05:19.101769+0000","last_peered":"2023-10-08T16:05:19.101769+0000","last_clean":"2023-10-08T16:05:19.101769+0000","last_became_active":"2023-10-08T16:05:19.100111+0000","last_became_peered":"2023-10-08T16:05:19.100111+0000","last_unstale":"2023-10-08T16:05:19.101769+0000","last_undegraded":"2023-10-08T16:05:19.101769+0000","last_fullsized":"2023-10-08T16:05:19.101769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:21.023580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091112+0000","last_change":"2023-10-08T16:05:19.091112+0000","last_active":"2023-10-08T16:05:19.091112+0000","last_peered":"2023-10-08T16:05:19.091112+0000","last_clean":"2023-10-08T16:05:19.091112+0000","last_became_active":"2023-10-08T16:05:19.090790+0000","last_became_peered":"2023-10-08T16:05:19.090790+0000","last_unstale":"2023-10-08T16:05:19.091112+0000","last_undegraded":"2023-10-08T16:05:19.091112+0000","last_fullsized":"2023-10-08T16:05:19.091112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091654+0000","last_change":"2023-10-08T16:05:19.091654+0000","last_active":"2023-10-08T16:05:19.091654+0000","last_peered":"2023-10-08T16:05:19.091654+0000","last_clean":"2023-10-08T16:05:19.091654+0000","last_became_active":"2023-10-08T16:05:19.091350+0000","last_became_peered":"2023-10-08T16:05:19.091350+0000","last_unstale":"2023-10-08T16:05:19.091654+0000","last_undegraded":"2023-10-08T16:05:19.091654+0000","last_fullsized":"2023-10-08T16:05:19.091654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:48.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.105015+0000","last_change":"2023-10-08T16:05:19.105015+0000","last_active":"2023-10-08T16:05:19.105015+0000","last_peered":"2023-10-08T16:05:19.105015+0000","last_clean":"2023-10-08T16:05:19.105015+0000","last_became_active":"2023-10-08T16:05:19.104757+0000","last_became_peered":"2023-10-08T16:05:19.104757+0000","last_unstale":"2023-10-08T16:05:19.105015+0000","last_undegraded":"2023-10-08T16:05:19.105015+0000","last_fullsized":"2023-10-08T16:05:19.105015+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:30.509876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091007+0000","last_change":"2023-10-08T16:05:19.091007+0000","last_active":"2023-10-08T16:05:19.091007+0000","last_peered":"2023-10-08T16:05:19.091007+0000","last_clean":"2023-10-08T16:05:19.091007+0000","last_became_active":"2023-10-08T16:05:19.090705+0000","last_became_peered":"2023-10-08T16:05:19.090705+0000","last_unstale":"2023-10-08T16:05:19.091007+0000","last_undegraded":"2023-10-08T16:05:19.091007+0000","last_fullsized":"2023-10-08T16:05:19.091007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:00.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.438299+0000","last_change":"2023-10-08T16:05:18.085551+0000","last_active":"2023-10-08T16:05:19.438299+0000","last_peered":"2023-10-08T16:05:19.438299+0000","last_clean":"2023-10-08T16:05:19.438299+0000","last_became_active":"2023-10-08T16:05:18.085240+0000","last_became_peered":"2023-10-08T16:05:18.085240+0000","last_unstale":"2023-10-08T16:05:19.438299+0000","last_undegraded":"2023-10-08T16:05:19.438299+0000","last_fullsized":"2023-10-08T16:05:19.438299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.104242+0000","last_change":"2023-10-08T16:05:19.104242+0000","last_active":"2023-10-08T16:05:19.104242+0000","last_peered":"2023-10-08T16:05:19.104242+0000","last_clean":"2023-10-08T16:05:19.104242+0000","last_became_active":"2023-10-08T16:05:19.103918+0000","last_became_peered":"2023-10-08T16:05:19.103918+0000","last_unstale":"2023-10-08T16:05:19.104242+0000","last_undegraded":"2023-10-08T16:05:19.104242+0000","last_fullsized":"2023-10-08T16:05:19.104242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:26.751269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.084343+0000","last_change":"2023-10-08T16:05:18.085176+0000","last_active":"2023-10-08T16:05:19.084343+0000","last_peered":"2023-10-08T16:05:19.084343+0000","last_clean":"2023-10-08T16:05:19.084343+0000","last_became_active":"2023-10-08T16:05:18.084941+0000","last_became_peered":"2023-10-08T16:05:18.084941+0000","last_unstale":"2023-10-08T16:05:19.084343+0000","last_undegraded":"2023-10-08T16:05:19.084343+0000","last_fullsized":"2023-10-08T16:05:19.084343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:07.761736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101237+0000","last_change":"2023-10-08T16:05:19.101237+0000","last_active":"2023-10-08T16:05:19.101237+0000","last_peered":"2023-10-08T16:05:19.101237+0000","last_clean":"2023-10-08T16:05:19.101237+0000","last_became_active":"2023-10-08T16:05:19.100930+0000","last_became_peered":"2023-10-08T16:05:19.100930+0000","last_unstale":"2023-10-08T16:05:19.101237+0000","last_undegraded":"2023-10-08T16:05:19.101237+0000","last_fullsized":"2023-10-08T16:05:19.101237+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:27.486604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439382+0000","last_change":"2023-10-08T16:05:18.082308+0000","last_active":"2023-10-08T16:05:19.439382+0000","last_peered":"2023-10-08T16:05:19.439382+0000","last_clean":"2023-10-08T16:05:19.439382+0000","last_became_active":"2023-10-08T16:05:18.082061+0000","last_became_peered":"2023-10-08T16:05:18.082061+0000","last_unstale":"2023-10-08T16:05:19.439382+0000","last_undegraded":"2023-10-08T16:05:19.439382+0000","last_fullsized":"2023-10-08T16:05:19.439382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.686570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102108+0000","last_change":"2023-10-08T16:05:19.102108+0000","last_active":"2023-10-08T16:05:19.102108+0000","last_peered":"2023-10-08T16:05:19.102108+0000","last_clean":"2023-10-08T16:05:19.102108+0000","last_became_active":"2023-10-08T16:05:19.101841+0000","last_became_peered":"2023-10-08T16:05:19.101841+0000","last_unstale":"2023-10-08T16:05:19.102108+0000","last_undegraded":"2023-10-08T16:05:19.102108+0000","last_fullsized":"2023-10-08T16:05:19.102108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:21.023580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439797+0000","last_change":"2023-10-08T16:05:18.083258+0000","last_active":"2023-10-08T16:05:19.439797+0000","last_peered":"2023-10-08T16:05:19.439797+0000","last_clean":"2023-10-08T16:05:19.439797+0000","last_became_active":"2023-10-08T16:05:18.082808+0000","last_became_peered":"2023-10-08T16:05:18.082808+0000","last_unstale":"2023-10-08T16:05:19.439797+0000","last_undegraded":"2023-10-08T16:05:19.439797+0000","last_fullsized":"2023-10-08T16:05:19.439797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:30.112796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102214+0000","last_change":"2023-10-08T16:05:19.102214+0000","last_active":"2023-10-08T16:05:19.102214+0000","last_peered":"2023-10-08T16:05:19.102214+0000","last_clean":"2023-10-08T16:05:19.102214+0000","last_became_active":"2023-10-08T16:05:19.101716+0000","last_became_peered":"2023-10-08T16:05:19.101716+0000","last_unstale":"2023-10-08T16:05:19.102214+0000","last_undegraded":"2023-10-08T16:05:19.102214+0000","last_fullsized":"2023-10-08T16:05:19.102214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103776+0000","last_change":"2023-10-08T16:05:18.083026+0000","last_active":"2023-10-08T16:05:19.103776+0000","last_peered":"2023-10-08T16:05:19.103776+0000","last_clean":"2023-10-08T16:05:19.103776+0000","last_became_active":"2023-10-08T16:05:18.082522+0000","last_became_peered":"2023-10-08T16:05:18.082522+0000","last_unstale":"2023-10-08T16:05:19.103776+0000","last_undegraded":"2023-10-08T16:05:19.103776+0000","last_fullsized":"2023-10-08T16:05:19.103776+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094558+0000","last_change":"2023-10-08T16:05:19.094558+0000","last_active":"2023-10-08T16:05:19.094558+0000","last_peered":"2023-10-08T16:05:19.094558+0000","last_clean":"2023-10-08T16:05:19.094558+0000","last_became_active":"2023-10-08T16:05:19.094330+0000","last_became_peered":"2023-10-08T16:05:19.094330+0000","last_unstale":"2023-10-08T16:05:19.094558+0000","last_undegraded":"2023-10-08T16:05:19.094558+0000","last_fullsized":"2023-10-08T16:05:19.094558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:30.509876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101639+0000","last_change":"2023-10-08T16:05:18.081112+0000","last_active":"2023-10-08T16:05:19.101639+0000","last_peered":"2023-10-08T16:05:19.101639+0000","last_clean":"2023-10-08T16:05:19.101639+0000","last_became_active":"2023-10-08T16:05:18.080818+0000","last_became_peered":"2023-10-08T16:05:18.080818+0000","last_unstale":"2023-10-08T16:05:19.101639+0000","last_undegraded":"2023-10-08T16:05:19.101639+0000","last_fullsized":"2023-10-08T16:05:19.101639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:32.348080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.099914+0000","last_change":"2023-10-08T16:05:19.099914+0000","last_active":"2023-10-08T16:05:19.099914+0000","last_peered":"2023-10-08T16:05:19.099914+0000","last_clean":"2023-10-08T16:05:19.099914+0000","last_became_active":"2023-10-08T16:05:19.099587+0000","last_became_peered":"2023-10-08T16:05:19.099587+0000","last_unstale":"2023-10-08T16:05:19.099914+0000","last_undegraded":"2023-10-08T16:05:19.099914+0000","last_fullsized":"2023-10-08T16:05:19.099914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:39.319258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084442+0000","last_change":"2023-10-08T16:05:18.080126+0000","last_active":"2023-10-08T16:05:18.084442+0000","last_peered":"2023-10-08T16:05:18.084442+0000","last_clean":"2023-10-08T16:05:18.084442+0000","last_became_active":"2023-10-08T16:05:18.079862+0000","last_became_peered":"2023-10-08T16:05:18.079862+0000","last_unstale":"2023-10-08T16:05:18.084442+0000","last_undegraded":"2023-10-08T16:05:18.084442+0000","last_fullsized":"2023-10-08T16:05:18.084442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:47.307711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096205+0000","last_change":"2023-10-08T16:05:19.096205+0000","last_active":"2023-10-08T16:05:19.096205+0000","last_peered":"2023-10-08T16:05:19.096205+0000","last_clean":"2023-10-08T16:05:19.096205+0000","last_became_active":"2023-10-08T16:05:19.095949+0000","last_became_peered":"2023-10-08T16:05:19.095949+0000","last_unstale":"2023-10-08T16:05:19.096205+0000","last_undegraded":"2023-10-08T16:05:19.096205+0000","last_fullsized":"2023-10-08T16:05:19.096205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:56.958992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088874+0000","last_change":"2023-10-08T16:05:18.084486+0000","last_active":"2023-10-08T16:05:18.088874+0000","last_peered":"2023-10-08T16:05:18.088874+0000","last_clean":"2023-10-08T16:05:18.088874+0000","last_became_active":"2023-10-08T16:05:18.084201+0000","last_became_peered":"2023-10-08T16:05:18.084201+0000","last_unstale":"2023-10-08T16:05:18.088874+0000","last_undegraded":"2023-10-08T16:05:18.088874+0000","last_fullsized":"2023-10-08T16:05:18.088874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094142+0000","last_change":"2023-10-08T16:05:19.094142+0000","last_active":"2023-10-08T16:05:19.094142+0000","last_peered":"2023-10-08T16:05:19.094142+0000","last_clean":"2023-10-08T16:05:19.094142+0000","last_became_active":"2023-10-08T16:05:19.093878+0000","last_became_peered":"2023-10-08T16:05:19.093878+0000","last_unstale":"2023-10-08T16:05:19.094142+0000","last_undegraded":"2023-10-08T16:05:19.094142+0000","last_fullsized":"2023-10-08T16:05:19.094142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086512+0000","last_change":"2023-10-08T16:05:18.085255+0000","last_active":"2023-10-08T16:05:18.086512+0000","last_peered":"2023-10-08T16:05:18.086512+0000","last_clean":"2023-10-08T16:05:18.086512+0000","last_became_active":"2023-10-08T16:05:18.085021+0000","last_became_peered":"2023-10-08T16:05:18.085021+0000","last_unstale":"2023-10-08T16:05:18.086512+0000","last_undegraded":"2023-10-08T16:05:18.086512+0000","last_fullsized":"2023-10-08T16:05:18.086512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:25.746772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090069+0000","last_change":"2023-10-08T16:05:19.090069+0000","last_active":"2023-10-08T16:05:19.090069+0000","last_peered":"2023-10-08T16:05:19.090069+0000","last_clean":"2023-10-08T16:05:19.090069+0000","last_became_active":"2023-10-08T16:05:19.089797+0000","last_became_peered":"2023-10-08T16:05:19.089797+0000","last_unstale":"2023-10-08T16:05:19.090069+0000","last_undegraded":"2023-10-08T16:05:19.090069+0000","last_fullsized":"2023-10-08T16:05:19.090069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086705+0000","last_change":"2023-10-08T16:05:18.085718+0000","last_active":"2023-10-08T16:05:18.086705+0000","last_peered":"2023-10-08T16:05:18.086705+0000","last_clean":"2023-10-08T16:05:18.086705+0000","last_became_active":"2023-10-08T16:05:18.085500+0000","last_became_peered":"2023-10-08T16:05:18.085500+0000","last_unstale":"2023-10-08T16:05:18.086705+0000","last_undegraded":"2023-10-08T16:05:18.086705+0000","last_fullsized":"2023-10-08T16:05:18.086705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090833+0000","last_change":"2023-10-08T16:05:19.090833+0000","last_active":"2023-10-08T16:05:19.090833+0000","last_peered":"2023-10-08T16:05:19.090833+0000","last_clean":"2023-10-08T16:05:19.090833+0000","last_became_active":"2023-10-08T16:05:19.090593+0000","last_became_peered":"2023-10-08T16:05:19.090593+0000","last_unstale":"2023-10-08T16:05:19.090833+0000","last_undegraded":"2023-10-08T16:05:19.090833+0000","last_fullsized":"2023-10-08T16:05:19.090833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:58.341584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439917+0000","last_change":"2023-10-08T16:05:18.085311+0000","last_active":"2023-10-08T16:05:19.439917+0000","last_peered":"2023-10-08T16:05:19.439917+0000","last_clean":"2023-10-08T16:05:19.439917+0000","last_became_active":"2023-10-08T16:05:18.085040+0000","last_became_peered":"2023-10-08T16:05:18.085040+0000","last_unstale":"2023-10-08T16:05:19.439917+0000","last_undegraded":"2023-10-08T16:05:19.439917+0000","last_fullsized":"2023-10-08T16:05:19.439917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.686570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096331+0000","last_change":"2023-10-08T16:05:19.096331+0000","last_active":"2023-10-08T16:05:19.096331+0000","last_peered":"2023-10-08T16:05:19.096331+0000","last_clean":"2023-10-08T16:05:19.096331+0000","last_became_active":"2023-10-08T16:05:19.095972+0000","last_became_peered":"2023-10-08T16:05:19.095972+0000","last_unstale":"2023-10-08T16:05:19.096331+0000","last_undegraded":"2023-10-08T16:05:19.096331+0000","last_fullsized":"2023-10-08T16:05:19.096331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'5","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.441148+0000","last_change":"2023-10-08T16:05:18.082879+0000","last_active":"2023-10-08T16:05:19.441148+0000","last_peered":"2023-10-08T16:05:19.441148+0000","last_clean":"2023-10-08T16:05:19.441148+0000","last_became_active":"2023-10-08T16:05:18.082599+0000","last_became_peered":"2023-10-08T16:05:18.082599+0000","last_unstale":"2023-10-08T16:05:19.441148+0000","last_undegraded":"2023-10-08T16:05:19.441148+0000","last_fullsized":"2023-10-08T16:05:19.441148+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:04.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097582+0000","last_change":"2023-10-08T16:05:19.097582+0000","last_active":"2023-10-08T16:05:19.097582+0000","last_peered":"2023-10-08T16:05:19.097582+0000","last_clean":"2023-10-08T16:05:19.097582+0000","last_became_active":"2023-10-08T16:05:19.097315+0000","last_became_peered":"2023-10-08T16:05:19.097315+0000","last_unstale":"2023-10-08T16:05:19.097582+0000","last_undegraded":"2023-10-08T16:05:19.097582+0000","last_fullsized":"2023-10-08T16:05:19.097582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:54.206804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.440249+0000","last_change":"2023-10-08T16:05:18.083384+0000","last_active":"2023-10-08T16:05:19.440249+0000","last_peered":"2023-10-08T16:05:19.440249+0000","last_clean":"2023-10-08T16:05:19.440249+0000","last_became_active":"2023-10-08T16:05:18.082938+0000","last_became_peered":"2023-10-08T16:05:18.082938+0000","last_unstale":"2023-10-08T16:05:19.440249+0000","last_undegraded":"2023-10-08T16:05:19.440249+0000","last_fullsized":"2023-10-08T16:05:19.440249+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:37.490723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093782+0000","last_change":"2023-10-08T16:05:19.093782+0000","last_active":"2023-10-08T16:05:19.093782+0000","last_peered":"2023-10-08T16:05:19.093782+0000","last_clean":"2023-10-08T16:05:19.093782+0000","last_became_active":"2023-10-08T16:05:19.093528+0000","last_became_peered":"2023-10-08T16:05:19.093528+0000","last_unstale":"2023-10-08T16:05:19.093782+0000","last_undegraded":"2023-10-08T16:05:19.093782+0000","last_fullsized":"2023-10-08T16:05:19.093782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086807+0000","last_change":"2023-10-08T16:05:18.086321+0000","last_active":"2023-10-08T16:05:18.086807+0000","last_peered":"2023-10-08T16:05:18.086807+0000","last_clean":"2023-10-08T16:05:18.086807+0000","last_became_active":"2023-10-08T16:05:18.085850+0000","last_became_peered":"2023-10-08T16:05:18.085850+0000","last_unstale":"2023-10-08T16:05:18.086807+0000","last_undegraded":"2023-10-08T16:05:18.086807+0000","last_fullsized":"2023-10-08T16:05:18.086807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:49.615532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102042+0000","last_change":"2023-10-08T16:05:19.102042+0000","last_active":"2023-10-08T16:05:19.102042+0000","last_peered":"2023-10-08T16:05:19.102042+0000","last_clean":"2023-10-08T16:05:19.102042+0000","last_became_active":"2023-10-08T16:05:19.101735+0000","last_became_peered":"2023-10-08T16:05:19.101735+0000","last_unstale":"2023-10-08T16:05:19.102042+0000","last_undegraded":"2023-10-08T16:05:19.102042+0000","last_fullsized":"2023-10-08T16:05:19.102042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085173+0000","last_change":"2023-10-08T16:05:18.084189+0000","last_active":"2023-10-08T16:05:18.085173+0000","last_peered":"2023-10-08T16:05:18.085173+0000","last_clean":"2023-10-08T16:05:18.085173+0000","last_became_active":"2023-10-08T16:05:18.083938+0000","last_became_peered":"2023-10-08T16:05:18.083938+0000","last_unstale":"2023-10-08T16:05:18.085173+0000","last_undegraded":"2023-10-08T16:05:18.085173+0000","last_fullsized":"2023-10-08T16:05:18.085173+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:26.482107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101815+0000","last_change":"2023-10-08T16:05:19.101815+0000","last_active":"2023-10-08T16:05:19.101815+0000","last_peered":"2023-10-08T16:05:19.101815+0000","last_clean":"2023-10-08T16:05:19.101815+0000","last_became_active":"2023-10-08T16:05:19.101561+0000","last_became_peered":"2023-10-08T16:05:19.101561+0000","last_unstale":"2023-10-08T16:05:19.101815+0000","last_undegraded":"2023-10-08T16:05:19.101815+0000","last_fullsized":"2023-10-08T16:05:19.101815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.441364+0000","last_change":"2023-10-08T16:05:18.077345+0000","last_active":"2023-10-08T16:05:19.441364+0000","last_peered":"2023-10-08T16:05:19.441364+0000","last_clean":"2023-10-08T16:05:19.441364+0000","last_became_active":"2023-10-08T16:05:18.077120+0000","last_became_peered":"2023-10-08T16:05:18.077120+0000","last_unstale":"2023-10-08T16:05:19.441364+0000","last_undegraded":"2023-10-08T16:05:19.441364+0000","last_fullsized":"2023-10-08T16:05:19.441364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.429484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101881+0000","last_change":"2023-10-08T16:05:19.101881+0000","last_active":"2023-10-08T16:05:19.101881+0000","last_peered":"2023-10-08T16:05:19.101881+0000","last_clean":"2023-10-08T16:05:19.101881+0000","last_became_active":"2023-10-08T16:05:19.101548+0000","last_became_peered":"2023-10-08T16:05:19.101548+0000","last_unstale":"2023-10-08T16:05:19.101881+0000","last_undegraded":"2023-10-08T16:05:19.101881+0000","last_fullsized":"2023-10-08T16:05:19.101881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:26.751269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.082879+0000","last_change":"2023-10-08T16:05:18.081860+0000","last_active":"2023-10-08T16:05:19.082879+0000","last_peered":"2023-10-08T16:05:19.082879+0000","last_clean":"2023-10-08T16:05:19.082879+0000","last_became_active":"2023-10-08T16:05:18.081615+0000","last_became_peered":"2023-10-08T16:05:18.081615+0000","last_unstale":"2023-10-08T16:05:19.082879+0000","last_undegraded":"2023-10-08T16:05:19.082879+0000","last_fullsized":"2023-10-08T16:05:19.082879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:18:01.935637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102020+0000","last_change":"2023-10-08T16:05:19.102020+0000","last_active":"2023-10-08T16:05:19.102020+0000","last_peered":"2023-10-08T16:05:19.102020+0000","last_clean":"2023-10-08T16:05:19.102020+0000","last_became_active":"2023-10-08T16:05:19.101774+0000","last_became_peered":"2023-10-08T16:05:19.101774+0000","last_unstale":"2023-10-08T16:05:19.102020+0000","last_undegraded":"2023-10-08T16:05:19.102020+0000","last_fullsized":"2023-10-08T16:05:19.102020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439947+0000","last_change":"2023-10-08T16:05:18.083140+0000","last_active":"2023-10-08T16:05:19.439947+0000","last_peered":"2023-10-08T16:05:19.439947+0000","last_clean":"2023-10-08T16:05:19.439947+0000","last_became_active":"2023-10-08T16:05:18.082853+0000","last_became_peered":"2023-10-08T16:05:18.082853+0000","last_unstale":"2023-10-08T16:05:19.439947+0000","last_undegraded":"2023-10-08T16:05:19.439947+0000","last_fullsized":"2023-10-08T16:05:19.439947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:32.110466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.104911+0000","last_change":"2023-10-08T16:05:19.104911+0000","last_active":"2023-10-08T16:05:19.104911+0000","last_peered":"2023-10-08T16:05:19.104911+0000","last_clean":"2023-10-08T16:05:19.104911+0000","last_became_active":"2023-10-08T16:05:19.104695+0000","last_became_peered":"2023-10-08T16:05:19.104695+0000","last_unstale":"2023-10-08T16:05:19.104911+0000","last_undegraded":"2023-10-08T16:05:19.104911+0000","last_fullsized":"2023-10-08T16:05:19.104911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103413+0000","last_change":"2023-10-08T16:05:18.082063+0000","last_active":"2023-10-08T16:05:19.103413+0000","last_peered":"2023-10-08T16:05:19.103413+0000","last_clean":"2023-10-08T16:05:19.103413+0000","last_became_active":"2023-10-08T16:05:18.081757+0000","last_became_peered":"2023-10-08T16:05:18.081757+0000","last_unstale":"2023-10-08T16:05:19.103413+0000","last_undegraded":"2023-10-08T16:05:19.103413+0000","last_fullsized":"2023-10-08T16:05:19.103413+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:37.834689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095559+0000","last_change":"2023-10-08T16:05:19.095559+0000","last_active":"2023-10-08T16:05:19.095559+0000","last_peered":"2023-10-08T16:05:19.095559+0000","last_clean":"2023-10-08T16:05:19.095559+0000","last_became_active":"2023-10-08T16:05:19.095312+0000","last_became_peered":"2023-10-08T16:05:19.095312+0000","last_unstale":"2023-10-08T16:05:19.095559+0000","last_undegraded":"2023-10-08T16:05:19.095559+0000","last_fullsized":"2023-10-08T16:05:19.095559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:33.114963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088844+0000","last_change":"2023-10-08T16:05:18.084363+0000","last_active":"2023-10-08T16:05:18.088844+0000","last_peered":"2023-10-08T16:05:18.088844+0000","last_clean":"2023-10-08T16:05:18.088844+0000","last_became_active":"2023-10-08T16:05:18.084086+0000","last_became_peered":"2023-10-08T16:05:18.084086+0000","last_unstale":"2023-10-08T16:05:18.088844+0000","last_undegraded":"2023-10-08T16:05:18.088844+0000","last_fullsized":"2023-10-08T16:05:18.088844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:18.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101888+0000","last_change":"2023-10-08T16:05:19.101888+0000","last_active":"2023-10-08T16:05:19.101888+0000","last_peered":"2023-10-08T16:05:19.101888+0000","last_clean":"2023-10-08T16:05:19.101888+0000","last_became_active":"2023-10-08T16:05:19.101584+0000","last_became_peered":"2023-10-08T16:05:19.101584+0000","last_unstale":"2023-10-08T16:05:19.101888+0000","last_undegraded":"2023-10-08T16:05:19.101888+0000","last_fullsized":"2023-10-08T16:05:19.101888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:34.408160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.440646+0000","last_change":"2023-10-08T16:05:18.084837+0000","last_active":"2023-10-08T16:05:19.440646+0000","last_peered":"2023-10-08T16:05:19.440646+0000","last_clean":"2023-10-08T16:05:19.440646+0000","last_became_active":"2023-10-08T16:05:18.084581+0000","last_became_peered":"2023-10-08T16:05:18.084581+0000","last_unstale":"2023-10-08T16:05:19.440646+0000","last_undegraded":"2023-10-08T16:05:19.440646+0000","last_fullsized":"2023-10-08T16:05:19.440646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:51.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092611+0000","last_change":"2023-10-08T16:05:19.092611+0000","last_active":"2023-10-08T16:05:19.092611+0000","last_peered":"2023-10-08T16:05:19.092611+0000","last_clean":"2023-10-08T16:05:19.092611+0000","last_became_active":"2023-10-08T16:05:19.092184+0000","last_became_peered":"2023-10-08T16:05:19.092184+0000","last_unstale":"2023-10-08T16:05:19.092611+0000","last_undegraded":"2023-10-08T16:05:19.092611+0000","last_fullsized":"2023-10-08T16:05:19.092611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:38.839186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.087028+0000","last_change":"2023-10-08T16:05:18.084953+0000","last_active":"2023-10-08T16:05:18.087028+0000","last_peered":"2023-10-08T16:05:18.087028+0000","last_clean":"2023-10-08T16:05:18.087028+0000","last_became_active":"2023-10-08T16:05:18.084672+0000","last_became_peered":"2023-10-08T16:05:18.084672+0000","last_unstale":"2023-10-08T16:05:18.087028+0000","last_undegraded":"2023-10-08T16:05:18.087028+0000","last_fullsized":"2023-10-08T16:05:18.087028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094119+0000","last_change":"2023-10-08T16:05:19.094119+0000","last_active":"2023-10-08T16:05:19.094119+0000","last_peered":"2023-10-08T16:05:19.094119+0000","last_clean":"2023-10-08T16:05:19.094119+0000","last_became_active":"2023-10-08T16:05:19.093848+0000","last_became_peered":"2023-10-08T16:05:19.093848+0000","last_unstale":"2023-10-08T16:05:19.094119+0000","last_undegraded":"2023-10-08T16:05:19.094119+0000","last_fullsized":"2023-10-08T16:05:19.094119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:19.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088721+0000","last_change":"2023-10-08T16:05:18.082002+0000","last_active":"2023-10-08T16:05:18.088721+0000","last_peered":"2023-10-08T16:05:18.088721+0000","last_clean":"2023-10-08T16:05:18.088721+0000","last_became_active":"2023-10-08T16:05:18.080550+0000","last_became_peered":"2023-10-08T16:05:18.080550+0000","last_unstale":"2023-10-08T16:05:18.088721+0000","last_undegraded":"2023-10-08T16:05:18.088721+0000","last_fullsized":"2023-10-08T16:05:18.088721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091130+0000","last_change":"2023-10-08T16:05:19.091130+0000","last_active":"2023-10-08T16:05:19.091130+0000","last_peered":"2023-10-08T16:05:19.091130+0000","last_clean":"2023-10-08T16:05:19.091130+0000","last_became_active":"2023-10-08T16:05:19.090796+0000","last_became_peered":"2023-10-08T16:05:19.090796+0000","last_unstale":"2023-10-08T16:05:19.091130+0000","last_undegraded":"2023-10-08T16:05:19.091130+0000","last_fullsized":"2023-10-08T16:05:19.091130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086870+0000","last_change":"2023-10-08T16:05:18.084760+0000","last_active":"2023-10-08T16:05:18.086870+0000","last_peered":"2023-10-08T16:05:18.086870+0000","last_clean":"2023-10-08T16:05:18.086870+0000","last_became_active":"2023-10-08T16:05:18.084511+0000","last_became_peered":"2023-10-08T16:05:18.084511+0000","last_unstale":"2023-10-08T16:05:18.086870+0000","last_undegraded":"2023-10-08T16:05:18.086870+0000","last_fullsized":"2023-10-08T16:05:18.086870+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.429484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097278+0000","last_change":"2023-10-08T16:05:19.097278+0000","last_active":"2023-10-08T16:05:19.097278+0000","last_peered":"2023-10-08T16:05:19.097278+0000","last_clean":"2023-10-08T16:05:19.097278+0000","last_became_active":"2023-10-08T16:05:19.096939+0000","last_became_peered":"2023-10-08T16:05:19.096939+0000","last_unstale":"2023-10-08T16:05:19.097278+0000","last_undegraded":"2023-10-08T16:05:19.097278+0000","last_fullsized":"2023-10-08T16:05:19.097278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:08.766233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.438801+0000","last_change":"2023-10-08T16:05:18.081498+0000","last_active":"2023-10-08T16:05:19.438801+0000","last_peered":"2023-10-08T16:05:19.438801+0000","last_clean":"2023-10-08T16:05:19.438801+0000","last_became_active":"2023-10-08T16:05:18.081246+0000","last_became_peered":"2023-10-08T16:05:18.081246+0000","last_unstale":"2023-10-08T16:05:19.438801+0000","last_undegraded":"2023-10-08T16:05:19.438801+0000","last_fullsized":"2023-10-08T16:05:19.438801+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:27:12.744966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093558+0000","last_change":"2023-10-08T16:05:19.093558+0000","last_active":"2023-10-08T16:05:19.093558+0000","last_peered":"2023-10-08T16:05:19.093558+0000","last_clean":"2023-10-08T16:05:19.093558+0000","last_became_active":"2023-10-08T16:05:19.093237+0000","last_became_peered":"2023-10-08T16:05:19.093237+0000","last_unstale":"2023-10-08T16:05:19.093558+0000","last_undegraded":"2023-10-08T16:05:19.093558+0000","last_fullsized":"2023-10-08T16:05:19.093558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085936+0000","last_change":"2023-10-08T16:05:18.084693+0000","last_active":"2023-10-08T16:05:18.085936+0000","last_peered":"2023-10-08T16:05:18.085936+0000","last_clean":"2023-10-08T16:05:18.085936+0000","last_became_active":"2023-10-08T16:05:18.084447+0000","last_became_peered":"2023-10-08T16:05:18.084447+0000","last_unstale":"2023-10-08T16:05:18.085936+0000","last_undegraded":"2023-10-08T16:05:18.085936+0000","last_fullsized":"2023-10-08T16:05:18.085936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090027+0000","last_change":"2023-10-08T16:05:19.090027+0000","last_active":"2023-10-08T16:05:19.090027+0000","last_peered":"2023-10-08T16:05:19.090027+0000","last_clean":"2023-10-08T16:05:19.090027+0000","last_became_active":"2023-10-08T16:05:19.089799+0000","last_became_peered":"2023-10-08T16:05:19.089799+0000","last_unstale":"2023-10-08T16:05:19.090027+0000","last_undegraded":"2023-10-08T16:05:19.090027+0000","last_fullsized":"2023-10-08T16:05:19.090027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:48.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085964+0000","last_change":"2023-10-08T16:05:18.081813+0000","last_active":"2023-10-08T16:05:18.085964+0000","last_peered":"2023-10-08T16:05:18.085964+0000","last_clean":"2023-10-08T16:05:18.085964+0000","last_became_active":"2023-10-08T16:05:18.081096+0000","last_became_peered":"2023-10-08T16:05:18.081096+0000","last_unstale":"2023-10-08T16:05:18.085964+0000","last_undegraded":"2023-10-08T16:05:18.085964+0000","last_fullsized":"2023-10-08T16:05:18.085964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093655+0000","last_change":"2023-10-08T16:05:19.093655+0000","last_active":"2023-10-08T16:05:19.093655+0000","last_peered":"2023-10-08T16:05:19.093655+0000","last_clean":"2023-10-08T16:05:19.093655+0000","last_became_active":"2023-10-08T16:05:19.093359+0000","last_became_peered":"2023-10-08T16:05:19.093359+0000","last_unstale":"2023-10-08T16:05:19.093655+0000","last_undegraded":"2023-10-08T16:05:19.093655+0000","last_fullsized":"2023-10-08T16:05:19.093655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:51.196559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084982+0000","last_change":"2023-10-08T16:05:18.082084+0000","last_active":"2023-10-08T16:05:18.084982+0000","last_peered":"2023-10-08T16:05:18.084982+0000","last_clean":"2023-10-08T16:05:18.084982+0000","last_became_active":"2023-10-08T16:05:18.081851+0000","last_became_peered":"2023-10-08T16:05:18.081851+0000","last_unstale":"2023-10-08T16:05:18.084982+0000","last_undegraded":"2023-10-08T16:05:18.084982+0000","last_fullsized":"2023-10-08T16:05:18.084982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093449+0000","last_change":"2023-10-08T16:05:19.093449+0000","last_active":"2023-10-08T16:05:19.093449+0000","last_peered":"2023-10-08T16:05:19.093449+0000","last_clean":"2023-10-08T16:05:19.093449+0000","last_became_active":"2023-10-08T16:05:19.093127+0000","last_became_peered":"2023-10-08T16:05:19.093127+0000","last_unstale":"2023-10-08T16:05:19.093449+0000","last_undegraded":"2023-10-08T16:05:19.093449+0000","last_fullsized":"2023-10-08T16:05:19.093449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084495+0000","last_change":"2023-10-08T16:05:18.081569+0000","last_active":"2023-10-08T16:05:18.084495+0000","last_peered":"2023-10-08T16:05:18.084495+0000","last_clean":"2023-10-08T16:05:18.084495+0000","last_became_active":"2023-10-08T16:05:18.081316+0000","last_became_peered":"2023-10-08T16:05:18.081316+0000","last_unstale":"2023-10-08T16:05:18.084495+0000","last_undegraded":"2023-10-08T16:05:18.084495+0000","last_fullsized":"2023-10-08T16:05:18.084495+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:58.178415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090128+0000","last_change":"2023-10-08T16:05:19.090128+0000","last_active":"2023-10-08T16:05:19.090128+0000","last_peered":"2023-10-08T16:05:19.090128+0000","last_clean":"2023-10-08T16:05:19.090128+0000","last_became_active":"2023-10-08T16:05:19.089876+0000","last_became_peered":"2023-10-08T16:05:19.089876+0000","last_unstale":"2023-10-08T16:05:19.090128+0000","last_undegraded":"2023-10-08T16:05:19.090128+0000","last_fullsized":"2023-10-08T16:05:19.090128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.087643+0000","last_change":"2023-10-08T16:05:18.079687+0000","last_active":"2023-10-08T16:05:18.087643+0000","last_peered":"2023-10-08T16:05:18.087643+0000","last_clean":"2023-10-08T16:05:18.087643+0000","last_became_active":"2023-10-08T16:05:18.079403+0000","last_became_peered":"2023-10-08T16:05:18.079403+0000","last_unstale":"2023-10-08T16:05:18.087643+0000","last_undegraded":"2023-10-08T16:05:18.087643+0000","last_fullsized":"2023-10-08T16:05:18.087643+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:07.761736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.105762+0000","last_change":"2023-10-08T16:05:19.105762+0000","last_active":"2023-10-08T16:05:19.105762+0000","last_peered":"2023-10-08T16:05:19.105762+0000","last_clean":"2023-10-08T16:05:19.105762+0000","last_became_active":"2023-10-08T16:05:19.102247+0000","last_became_peered":"2023-10-08T16:05:19.102247+0000","last_unstale":"2023-10-08T16:05:19.105762+0000","last_undegraded":"2023-10-08T16:05:19.105762+0000","last_fullsized":"2023-10-08T16:05:19.105762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:57.080931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084815+0000","last_change":"2023-10-08T16:05:18.084080+0000","last_active":"2023-10-08T16:05:18.084815+0000","last_peered":"2023-10-08T16:05:18.084815+0000","last_clean":"2023-10-08T16:05:18.084815+0000","last_became_active":"2023-10-08T16:05:18.083817+0000","last_became_peered":"2023-10-08T16:05:18.083817+0000","last_unstale":"2023-10-08T16:05:18.084815+0000","last_undegraded":"2023-10-08T16:05:18.084815+0000","last_fullsized":"2023-10-08T16:05:18.084815+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:34.843700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090851+0000","last_change":"2023-10-08T16:05:19.090851+0000","last_active":"2023-10-08T16:05:19.090851+0000","last_peered":"2023-10-08T16:05:19.090851+0000","last_clean":"2023-10-08T16:05:19.090851+0000","last_became_active":"2023-10-08T16:05:19.090514+0000","last_became_peered":"2023-10-08T16:05:19.090514+0000","last_unstale":"2023-10-08T16:05:19.090851+0000","last_undegraded":"2023-10-08T16:05:19.090851+0000","last_fullsized":"2023-10-08T16:05:19.090851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.090095+0000","last_change":"2023-10-08T16:05:18.089445+0000","last_active":"2023-10-08T16:05:18.090095+0000","last_peered":"2023-10-08T16:05:18.090095+0000","last_clean":"2023-10-08T16:05:18.090095+0000","last_became_active":"2023-10-08T16:05:18.089123+0000","last_became_peered":"2023-10-08T16:05:18.089123+0000","last_unstale":"2023-10-08T16:05:18.090095+0000","last_undegraded":"2023-10-08T16:05:18.090095+0000","last_fullsized":"2023-10-08T16:05:18.090095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.077013+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:05:18.077013+0000","last_peered":"2023-10-08T16:05:18.077013+0000","last_clean":"2023-10-08T16:05:18.077013+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:05:18.077013+0000","last_undegraded":"2023-10-08T16:05:18.077013+0000","last_fullsized":"2023-10-08T16:05:18.077013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1414,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":722453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":348160,"data_stored":132085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":348160,"data_stored":132085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":722453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:05:23.481 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:05:23.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.629+0000 7fd571964700 1 Processor -- start 2023-10-08T16:05:23.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.629+0000 7fd571964700 1 -- start start 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd571964700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd56c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd571964700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c15a160 con 0x7fd56c159890 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd571964700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c0d13a0 con 0x7fd56c15db10 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd571964700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c15e140 con 0x7fd56c159c60 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd56affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd56c159890 0x7fd56c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60638/0 (socket says 172.21.15.170:60638) 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd56affd700 1 -- 172.21.15.170:0/2718989613 learned_addr learned my addr 172.21.15.170:0/2718989613 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd56a7fc700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd56c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.630+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020653114 0 0) 0x7fd56c15a160 con 0x7fd56c159890 2023-10-08T16:05:23.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd558003570 con 0x7fd56c159890 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082310296 0 0) 0x7fd558003570 con 0x7fd56c159890 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 msgr2=0x7fd56c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd56c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 >> v1:172.21.15.191:6789/0 conn(0x7fd56c15db10 legacy=0x7fd56c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c15a3d0 con 0x7fd56c159890 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fd55c0017f0 con 0x7fd56c159890 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 >> v1:172.21.15.170:6789/0 conn(0x7fd56c159890 legacy=0x7fd56c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd558003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd558004010 0x7fd558004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd5580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c15e140 con 0x7fd558004d50 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55800bbf0 con 0x7fd558004010 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55800bdb0 con 0x7fd56c159c60 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56affd700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd558003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd5580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56a7fc700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd558004010 0x7fd558004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 -- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 msgr2=0x7fd558003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c159c60 0x7fd558003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 -- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd558004010 msgr2=0x7fd558004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd558004010 0x7fd558004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 -- 172.21.15.170:0/2718989613 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd558003570 con 0x7fd558004d50 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd56b7fe700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd5580051c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd564001880 tx=0x7fd564007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad60d7c8ac6cb87d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.631+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564012420 con 0x7fd558004d50 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd564012580 con 0x7fd558004d50 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd569ffb700 1 -- 172.21.15.170:0/2718989613 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564012920 con 0x7fd558004d50 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 -- 172.21.15.170:0/2718989613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 msgr2=0x7fd5580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 --2- 172.21.15.170:0/2718989613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd5580051c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd564001880 tx=0x7fd564007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd56b7fe700 1 -- 172.21.15.170:0/2718989613 reap_dead start 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 -- 172.21.15.170:0/2718989613 shutdown_connections 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 -- 172.21.15.170:0/2718989613 >> 172.21.15.170:0/2718989613 conn(0x7fd56c068e20 msgr2=0x7fd56c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 -- 172.21.15.170:0/2718989613 shutdown_connections 2023-10-08T16:05:23.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.632+0000 7fd571964700 1 -- 172.21.15.170:0/2718989613 wait complete. 2023-10-08T16:05:23.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.633+0000 7fd571964700 1 Processor -- start 2023-10-08T16:05:23.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.633+0000 7fd571964700 1 -- start start 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c14ce80 0x7fd56c0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd56c0cf020 0x7fd56c15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0cf470 con 0x7fd558004d50 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0c6ca0 con 0x7fd56c0cf020 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd571964700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0c6090 con 0x7fd56c14ce80 2023-10-08T16:05:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54798/0 (socket says 172.21.15.170:54798) 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 -- 172.21.15.170:0/4289253261 learned_addr learned my addr 172.21.15.170:0/4289253261 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56a7fc700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c14ce80 0x7fd56c0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 -- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c14ce80 msgr2=0x7fd56c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56b7fe700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd56c0cf020 0x7fd56c15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c14ce80 0x7fd56c0ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 -- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd56c0cf020 msgr2=0x7fd56c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd56c0cf020 0x7fd56c15f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.634+0000 7fd56affd700 1 -- 172.21.15.170:0/4289253261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c0c6370 con 0x7fd558004d50 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd56affd700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd55c007a90 tx=0x7fd55c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00ae40 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c0133f0 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c0c6650 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0137b0 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.635+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c0c5d40 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.636+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c04c3c0 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.636+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd55c013b20 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.636+0000 7fd562ffd700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 0x7fd54c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.637+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd55c085370 con 0x7fd558004d50 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.637+0000 7fd56a7fc700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 0x7fd54c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:23.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.637+0000 7fd56a7fc700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 0x7fd54c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0b6630 tx=0x7fd554008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.641+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd55c0525e0 con 0x7fd558004d50 2023-10-08T16:05:23.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.821+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd56c0c6de0 con 0x7fd54c066f30 2023-10-08T16:05:23.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.829+0000 7fd562ffd700 1 -- 172.21.15.170:0/4289253261 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264641 (secure 0 0 0) 0x7fd554009e30 con 0x7fd54c066f30 2023-10-08T16:05:23.833 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:23.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 msgr2=0x7fd54c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 0x7fd54c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0b6630 tx=0x7fd554008040 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 msgr2=0x7fd56c0c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd55c007a90 tx=0x7fd55c0021d0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 shutdown_connections 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd56c0cf020 0x7fd56c15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c066f30 0x7fd54c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd558004d50 0x7fd56c0c7130 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 --2- 172.21.15.170:0/4289253261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56c14ce80 0x7fd56c0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 >> 172.21.15.170:0/4289253261 conn(0x7fd56c068e20 msgr2=0x7fd56c1577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 shutdown_connections 2023-10-08T16:05:23.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:23.832+0000 7fd571964700 1 -- 172.21.15.170:0/4289253261 wait complete. 2023-10-08T16:05:23.837 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:05:23.845 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-08T16:05:22.233144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591782,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382136,"kb_used_data":3832,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592584,"statfs":{"total":773094113280,"available":770654806016,"internally_reserved":0,"allocated":3923968,"data_stored":2210364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1414,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.168249"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.098175+0000","last_change":"2023-10-08T16:05:19.098175+0000","last_active":"2023-10-08T16:05:19.098175+0000","last_peered":"2023-10-08T16:05:19.098175+0000","last_clean":"2023-10-08T16:05:19.098175+0000","last_became_active":"2023-10-08T16:05:19.097629+0000","last_became_peered":"2023-10-08T16:05:19.097629+0000","last_unstale":"2023-10-08T16:05:19.098175+0000","last_undegraded":"2023-10-08T16:05:19.098175+0000","last_fullsized":"2023-10-08T16:05:19.098175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:00.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089604+0000","last_change":"2023-10-08T16:05:19.089604+0000","last_active":"2023-10-08T16:05:19.089604+0000","last_peered":"2023-10-08T16:05:19.089604+0000","last_clean":"2023-10-08T16:05:19.089604+0000","last_became_active":"2023-10-08T16:05:19.088983+0000","last_became_peered":"2023-10-08T16:05:19.088983+0000","last_unstale":"2023-10-08T16:05:19.089604+0000","last_undegraded":"2023-10-08T16:05:19.089604+0000","last_fullsized":"2023-10-08T16:05:19.089604+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:58:03.535208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096896+0000","last_change":"2023-10-08T16:05:19.096896+0000","last_active":"2023-10-08T16:05:19.096896+0000","last_peered":"2023-10-08T16:05:19.096896+0000","last_clean":"2023-10-08T16:05:19.096896+0000","last_became_active":"2023-10-08T16:05:19.096619+0000","last_became_peered":"2023-10-08T16:05:19.096619+0000","last_unstale":"2023-10-08T16:05:19.096896+0000","last_undegraded":"2023-10-08T16:05:19.096896+0000","last_fullsized":"2023-10-08T16:05:19.096896+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101732+0000","last_change":"2023-10-08T16:05:19.101732+0000","last_active":"2023-10-08T16:05:19.101732+0000","last_peered":"2023-10-08T16:05:19.101732+0000","last_clean":"2023-10-08T16:05:19.101732+0000","last_became_active":"2023-10-08T16:05:19.101477+0000","last_became_peered":"2023-10-08T16:05:19.101477+0000","last_unstale":"2023-10-08T16:05:19.101732+0000","last_undegraded":"2023-10-08T16:05:19.101732+0000","last_fullsized":"2023-10-08T16:05:19.101732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092438+0000","last_change":"2023-10-08T16:05:19.092438+0000","last_active":"2023-10-08T16:05:19.092438+0000","last_peered":"2023-10-08T16:05:19.092438+0000","last_clean":"2023-10-08T16:05:19.092438+0000","last_became_active":"2023-10-08T16:05:19.091865+0000","last_became_peered":"2023-10-08T16:05:19.091865+0000","last_unstale":"2023-10-08T16:05:19.092438+0000","last_undegraded":"2023-10-08T16:05:19.092438+0000","last_fullsized":"2023-10-08T16:05:19.092438+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:55.238889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097684+0000","last_change":"2023-10-08T16:05:19.097684+0000","last_active":"2023-10-08T16:05:19.097684+0000","last_peered":"2023-10-08T16:05:19.097684+0000","last_clean":"2023-10-08T16:05:19.097684+0000","last_became_active":"2023-10-08T16:05:19.097271+0000","last_became_peered":"2023-10-08T16:05:19.097271+0000","last_unstale":"2023-10-08T16:05:19.097684+0000","last_undegraded":"2023-10-08T16:05:19.097684+0000","last_fullsized":"2023-10-08T16:05:19.097684+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:38.851839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096733+0000","last_change":"2023-10-08T16:05:19.096733+0000","last_active":"2023-10-08T16:05:19.096733+0000","last_peered":"2023-10-08T16:05:19.096733+0000","last_clean":"2023-10-08T16:05:19.096733+0000","last_became_active":"2023-10-08T16:05:19.096437+0000","last_became_peered":"2023-10-08T16:05:19.096437+0000","last_unstale":"2023-10-08T16:05:19.096733+0000","last_undegraded":"2023-10-08T16:05:19.096733+0000","last_fullsized":"2023-10-08T16:05:19.096733+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090809+0000","last_change":"2023-10-08T16:05:19.090809+0000","last_active":"2023-10-08T16:05:19.090809+0000","last_peered":"2023-10-08T16:05:19.090809+0000","last_clean":"2023-10-08T16:05:19.090809+0000","last_became_active":"2023-10-08T16:05:19.090194+0000","last_became_peered":"2023-10-08T16:05:19.090194+0000","last_unstale":"2023-10-08T16:05:19.090809+0000","last_undegraded":"2023-10-08T16:05:19.090809+0000","last_fullsized":"2023-10-08T16:05:19.090809+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:54.597317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.099726+0000","last_change":"2023-10-08T16:05:19.099726+0000","last_active":"2023-10-08T16:05:19.099726+0000","last_peered":"2023-10-08T16:05:19.099726+0000","last_clean":"2023-10-08T16:05:19.099726+0000","last_became_active":"2023-10-08T16:05:19.099363+0000","last_became_peered":"2023-10-08T16:05:19.099363+0000","last_unstale":"2023-10-08T16:05:19.099726+0000","last_undegraded":"2023-10-08T16:05:19.099726+0000","last_fullsized":"2023-10-08T16:05:19.099726+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092185+0000","last_change":"2023-10-08T16:05:19.092185+0000","last_active":"2023-10-08T16:05:19.092185+0000","last_peered":"2023-10-08T16:05:19.092185+0000","last_clean":"2023-10-08T16:05:19.092185+0000","last_became_active":"2023-10-08T16:05:19.091943+0000","last_became_peered":"2023-10-08T16:05:19.091943+0000","last_unstale":"2023-10-08T16:05:19.092185+0000","last_undegraded":"2023-10-08T16:05:19.092185+0000","last_fullsized":"2023-10-08T16:05:19.092185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:54.597317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088808+0000","last_change":"2023-10-08T16:05:19.088808+0000","last_active":"2023-10-08T16:05:19.088808+0000","last_peered":"2023-10-08T16:05:19.088808+0000","last_clean":"2023-10-08T16:05:19.088808+0000","last_became_active":"2023-10-08T16:05:19.088511+0000","last_became_peered":"2023-10-08T16:05:19.088511+0000","last_unstale":"2023-10-08T16:05:19.088808+0000","last_undegraded":"2023-10-08T16:05:19.088808+0000","last_fullsized":"2023-10-08T16:05:19.088808+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093662+0000","last_change":"2023-10-08T16:05:19.093662+0000","last_active":"2023-10-08T16:05:19.093662+0000","last_peered":"2023-10-08T16:05:19.093662+0000","last_clean":"2023-10-08T16:05:19.093662+0000","last_became_active":"2023-10-08T16:05:19.093288+0000","last_became_peered":"2023-10-08T16:05:19.093288+0000","last_unstale":"2023-10-08T16:05:19.093662+0000","last_undegraded":"2023-10-08T16:05:19.093662+0000","last_fullsized":"2023-10-08T16:05:19.093662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093605+0000","last_change":"2023-10-08T16:05:19.093605+0000","last_active":"2023-10-08T16:05:19.093605+0000","last_peered":"2023-10-08T16:05:19.093605+0000","last_clean":"2023-10-08T16:05:19.093605+0000","last_became_active":"2023-10-08T16:05:19.093369+0000","last_became_peered":"2023-10-08T16:05:19.093369+0000","last_unstale":"2023-10-08T16:05:19.093605+0000","last_undegraded":"2023-10-08T16:05:19.093605+0000","last_fullsized":"2023-10-08T16:05:19.093605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:40.590305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091886+0000","last_change":"2023-10-08T16:05:19.091886+0000","last_active":"2023-10-08T16:05:19.091886+0000","last_peered":"2023-10-08T16:05:19.091886+0000","last_clean":"2023-10-08T16:05:19.091886+0000","last_became_active":"2023-10-08T16:05:19.091635+0000","last_became_peered":"2023-10-08T16:05:19.091635+0000","last_unstale":"2023-10-08T16:05:19.091886+0000","last_undegraded":"2023-10-08T16:05:19.091886+0000","last_fullsized":"2023-10-08T16:05:19.091886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097244+0000","last_change":"2023-10-08T16:05:19.097244+0000","last_active":"2023-10-08T16:05:19.097244+0000","last_peered":"2023-10-08T16:05:19.097244+0000","last_clean":"2023-10-08T16:05:19.097244+0000","last_became_active":"2023-10-08T16:05:19.096480+0000","last_became_peered":"2023-10-08T16:05:19.096480+0000","last_unstale":"2023-10-08T16:05:19.097244+0000","last_undegraded":"2023-10-08T16:05:19.097244+0000","last_fullsized":"2023-10-08T16:05:19.097244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:22.416946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088393+0000","last_change":"2023-10-08T16:05:19.088393+0000","last_active":"2023-10-08T16:05:19.088393+0000","last_peered":"2023-10-08T16:05:19.088393+0000","last_clean":"2023-10-08T16:05:19.088393+0000","last_became_active":"2023-10-08T16:05:19.088099+0000","last_became_peered":"2023-10-08T16:05:19.088099+0000","last_unstale":"2023-10-08T16:05:19.088393+0000","last_undegraded":"2023-10-08T16:05:19.088393+0000","last_fullsized":"2023-10-08T16:05:19.088393+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095984+0000","last_change":"2023-10-08T16:05:19.095984+0000","last_active":"2023-10-08T16:05:19.095984+0000","last_peered":"2023-10-08T16:05:19.095984+0000","last_clean":"2023-10-08T16:05:19.095984+0000","last_became_active":"2023-10-08T16:05:19.095723+0000","last_became_peered":"2023-10-08T16:05:19.095723+0000","last_unstale":"2023-10-08T16:05:19.095984+0000","last_undegraded":"2023-10-08T16:05:19.095984+0000","last_fullsized":"2023-10-08T16:05:19.095984+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091048+0000","last_change":"2023-10-08T16:05:19.091048+0000","last_active":"2023-10-08T16:05:19.091048+0000","last_peered":"2023-10-08T16:05:19.091048+0000","last_clean":"2023-10-08T16:05:19.091048+0000","last_became_active":"2023-10-08T16:05:19.090800+0000","last_became_peered":"2023-10-08T16:05:19.090800+0000","last_unstale":"2023-10-08T16:05:19.091048+0000","last_undegraded":"2023-10-08T16:05:19.091048+0000","last_fullsized":"2023-10-08T16:05:19.091048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:22.416946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089847+0000","last_change":"2023-10-08T16:05:19.089847+0000","last_active":"2023-10-08T16:05:19.089847+0000","last_peered":"2023-10-08T16:05:19.089847+0000","last_clean":"2023-10-08T16:05:19.089847+0000","last_became_active":"2023-10-08T16:05:19.089603+0000","last_became_peered":"2023-10-08T16:05:19.089603+0000","last_unstale":"2023-10-08T16:05:19.089847+0000","last_undegraded":"2023-10-08T16:05:19.089847+0000","last_fullsized":"2023-10-08T16:05:19.089847+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096510+0000","last_change":"2023-10-08T16:05:19.096510+0000","last_active":"2023-10-08T16:05:19.096510+0000","last_peered":"2023-10-08T16:05:19.096510+0000","last_clean":"2023-10-08T16:05:19.096510+0000","last_became_active":"2023-10-08T16:05:19.096275+0000","last_became_peered":"2023-10-08T16:05:19.096275+0000","last_unstale":"2023-10-08T16:05:19.096510+0000","last_undegraded":"2023-10-08T16:05:19.096510+0000","last_fullsized":"2023-10-08T16:05:19.096510+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:38.495220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103512+0000","last_change":"2023-10-08T16:05:19.103512+0000","last_active":"2023-10-08T16:05:19.103512+0000","last_peered":"2023-10-08T16:05:19.103512+0000","last_clean":"2023-10-08T16:05:19.103512+0000","last_became_active":"2023-10-08T16:05:19.100387+0000","last_became_peered":"2023-10-08T16:05:19.100387+0000","last_unstale":"2023-10-08T16:05:19.103512+0000","last_undegraded":"2023-10-08T16:05:19.103512+0000","last_fullsized":"2023-10-08T16:05:19.103512+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.100519+0000","last_change":"2023-10-08T16:05:19.100519+0000","last_active":"2023-10-08T16:05:19.100519+0000","last_peered":"2023-10-08T16:05:19.100519+0000","last_clean":"2023-10-08T16:05:19.100519+0000","last_became_active":"2023-10-08T16:05:19.099718+0000","last_became_peered":"2023-10-08T16:05:19.099718+0000","last_unstale":"2023-10-08T16:05:19.100519+0000","last_undegraded":"2023-10-08T16:05:19.100519+0000","last_fullsized":"2023-10-08T16:05:19.100519+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:35.848197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091446+0000","last_change":"2023-10-08T16:05:19.091446+0000","last_active":"2023-10-08T16:05:19.091446+0000","last_peered":"2023-10-08T16:05:19.091446+0000","last_clean":"2023-10-08T16:05:19.091446+0000","last_became_active":"2023-10-08T16:05:19.091180+0000","last_became_peered":"2023-10-08T16:05:19.091180+0000","last_unstale":"2023-10-08T16:05:19.091446+0000","last_undegraded":"2023-10-08T16:05:19.091446+0000","last_fullsized":"2023-10-08T16:05:19.091446+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:51.196559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091479+0000","last_change":"2023-10-08T16:05:19.091479+0000","last_active":"2023-10-08T16:05:19.091479+0000","last_peered":"2023-10-08T16:05:19.091479+0000","last_clean":"2023-10-08T16:05:19.091479+0000","last_became_active":"2023-10-08T16:05:19.091197+0000","last_became_peered":"2023-10-08T16:05:19.091197+0000","last_unstale":"2023-10-08T16:05:19.091479+0000","last_undegraded":"2023-10-08T16:05:19.091479+0000","last_fullsized":"2023-10-08T16:05:19.091479+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.088317+0000","last_change":"2023-10-08T16:05:19.088317+0000","last_active":"2023-10-08T16:05:19.088317+0000","last_peered":"2023-10-08T16:05:19.088317+0000","last_clean":"2023-10-08T16:05:19.088317+0000","last_became_active":"2023-10-08T16:05:19.088058+0000","last_became_peered":"2023-10-08T16:05:19.088058+0000","last_unstale":"2023-10-08T16:05:19.088317+0000","last_undegraded":"2023-10-08T16:05:19.088317+0000","last_fullsized":"2023-10-08T16:05:19.088317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.089030+0000","last_change":"2023-10-08T16:05:19.089030+0000","last_active":"2023-10-08T16:05:19.089030+0000","last_peered":"2023-10-08T16:05:19.089030+0000","last_clean":"2023-10-08T16:05:19.089030+0000","last_became_active":"2023-10-08T16:05:19.088776+0000","last_became_peered":"2023-10-08T16:05:19.088776+0000","last_unstale":"2023-10-08T16:05:19.089030+0000","last_undegraded":"2023-10-08T16:05:19.089030+0000","last_fullsized":"2023-10-08T16:05:19.089030+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101864+0000","last_change":"2023-10-08T16:05:19.101864+0000","last_active":"2023-10-08T16:05:19.101864+0000","last_peered":"2023-10-08T16:05:19.101864+0000","last_clean":"2023-10-08T16:05:19.101864+0000","last_became_active":"2023-10-08T16:05:19.101610+0000","last_became_peered":"2023-10-08T16:05:19.101610+0000","last_unstale":"2023-10-08T16:05:19.101864+0000","last_undegraded":"2023-10-08T16:05:19.101864+0000","last_fullsized":"2023-10-08T16:05:19.101864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:48.888237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095086+0000","last_change":"2023-10-08T16:05:19.095086+0000","last_active":"2023-10-08T16:05:19.095086+0000","last_peered":"2023-10-08T16:05:19.095086+0000","last_clean":"2023-10-08T16:05:19.095086+0000","last_became_active":"2023-10-08T16:05:19.094767+0000","last_became_peered":"2023-10-08T16:05:19.094767+0000","last_unstale":"2023-10-08T16:05:19.095086+0000","last_undegraded":"2023-10-08T16:05:19.095086+0000","last_fullsized":"2023-10-08T16:05:19.095086+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101769+0000","last_change":"2023-10-08T16:05:19.101769+0000","last_active":"2023-10-08T16:05:19.101769+0000","last_peered":"2023-10-08T16:05:19.101769+0000","last_clean":"2023-10-08T16:05:19.101769+0000","last_became_active":"2023-10-08T16:05:19.100111+0000","last_became_peered":"2023-10-08T16:05:19.100111+0000","last_unstale":"2023-10-08T16:05:19.101769+0000","last_undegraded":"2023-10-08T16:05:19.101769+0000","last_fullsized":"2023-10-08T16:05:19.101769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:21.023580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091112+0000","last_change":"2023-10-08T16:05:19.091112+0000","last_active":"2023-10-08T16:05:19.091112+0000","last_peered":"2023-10-08T16:05:19.091112+0000","last_clean":"2023-10-08T16:05:19.091112+0000","last_became_active":"2023-10-08T16:05:19.090790+0000","last_became_peered":"2023-10-08T16:05:19.090790+0000","last_unstale":"2023-10-08T16:05:19.091112+0000","last_undegraded":"2023-10-08T16:05:19.091112+0000","last_fullsized":"2023-10-08T16:05:19.091112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091654+0000","last_change":"2023-10-08T16:05:19.091654+0000","last_active":"2023-10-08T16:05:19.091654+0000","last_peered":"2023-10-08T16:05:19.091654+0000","last_clean":"2023-10-08T16:05:19.091654+0000","last_became_active":"2023-10-08T16:05:19.091350+0000","last_became_peered":"2023-10-08T16:05:19.091350+0000","last_unstale":"2023-10-08T16:05:19.091654+0000","last_undegraded":"2023-10-08T16:05:19.091654+0000","last_fullsized":"2023-10-08T16:05:19.091654+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:48.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.105015+0000","last_change":"2023-10-08T16:05:19.105015+0000","last_active":"2023-10-08T16:05:19.105015+0000","last_peered":"2023-10-08T16:05:19.105015+0000","last_clean":"2023-10-08T16:05:19.105015+0000","last_became_active":"2023-10-08T16:05:19.104757+0000","last_became_peered":"2023-10-08T16:05:19.104757+0000","last_unstale":"2023-10-08T16:05:19.105015+0000","last_undegraded":"2023-10-08T16:05:19.105015+0000","last_fullsized":"2023-10-08T16:05:19.105015+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:30.509876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091007+0000","last_change":"2023-10-08T16:05:19.091007+0000","last_active":"2023-10-08T16:05:19.091007+0000","last_peered":"2023-10-08T16:05:19.091007+0000","last_clean":"2023-10-08T16:05:19.091007+0000","last_became_active":"2023-10-08T16:05:19.090705+0000","last_became_peered":"2023-10-08T16:05:19.090705+0000","last_unstale":"2023-10-08T16:05:19.091007+0000","last_undegraded":"2023-10-08T16:05:19.091007+0000","last_fullsized":"2023-10-08T16:05:19.091007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:00.691067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.438299+0000","last_change":"2023-10-08T16:05:18.085551+0000","last_active":"2023-10-08T16:05:19.438299+0000","last_peered":"2023-10-08T16:05:19.438299+0000","last_clean":"2023-10-08T16:05:19.438299+0000","last_became_active":"2023-10-08T16:05:18.085240+0000","last_became_peered":"2023-10-08T16:05:18.085240+0000","last_unstale":"2023-10-08T16:05:19.438299+0000","last_undegraded":"2023-10-08T16:05:19.438299+0000","last_fullsized":"2023-10-08T16:05:19.438299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.104242+0000","last_change":"2023-10-08T16:05:19.104242+0000","last_active":"2023-10-08T16:05:19.104242+0000","last_peered":"2023-10-08T16:05:19.104242+0000","last_clean":"2023-10-08T16:05:19.104242+0000","last_became_active":"2023-10-08T16:05:19.103918+0000","last_became_peered":"2023-10-08T16:05:19.103918+0000","last_unstale":"2023-10-08T16:05:19.104242+0000","last_undegraded":"2023-10-08T16:05:19.104242+0000","last_fullsized":"2023-10-08T16:05:19.104242+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:26.751269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.084343+0000","last_change":"2023-10-08T16:05:18.085176+0000","last_active":"2023-10-08T16:05:19.084343+0000","last_peered":"2023-10-08T16:05:19.084343+0000","last_clean":"2023-10-08T16:05:19.084343+0000","last_became_active":"2023-10-08T16:05:18.084941+0000","last_became_peered":"2023-10-08T16:05:18.084941+0000","last_unstale":"2023-10-08T16:05:19.084343+0000","last_undegraded":"2023-10-08T16:05:19.084343+0000","last_fullsized":"2023-10-08T16:05:19.084343+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:07.761736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101237+0000","last_change":"2023-10-08T16:05:19.101237+0000","last_active":"2023-10-08T16:05:19.101237+0000","last_peered":"2023-10-08T16:05:19.101237+0000","last_clean":"2023-10-08T16:05:19.101237+0000","last_became_active":"2023-10-08T16:05:19.100930+0000","last_became_peered":"2023-10-08T16:05:19.100930+0000","last_unstale":"2023-10-08T16:05:19.101237+0000","last_undegraded":"2023-10-08T16:05:19.101237+0000","last_fullsized":"2023-10-08T16:05:19.101237+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:27.486604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439382+0000","last_change":"2023-10-08T16:05:18.082308+0000","last_active":"2023-10-08T16:05:19.439382+0000","last_peered":"2023-10-08T16:05:19.439382+0000","last_clean":"2023-10-08T16:05:19.439382+0000","last_became_active":"2023-10-08T16:05:18.082061+0000","last_became_peered":"2023-10-08T16:05:18.082061+0000","last_unstale":"2023-10-08T16:05:19.439382+0000","last_undegraded":"2023-10-08T16:05:19.439382+0000","last_fullsized":"2023-10-08T16:05:19.439382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.686570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102108+0000","last_change":"2023-10-08T16:05:19.102108+0000","last_active":"2023-10-08T16:05:19.102108+0000","last_peered":"2023-10-08T16:05:19.102108+0000","last_clean":"2023-10-08T16:05:19.102108+0000","last_became_active":"2023-10-08T16:05:19.101841+0000","last_became_peered":"2023-10-08T16:05:19.101841+0000","last_unstale":"2023-10-08T16:05:19.102108+0000","last_undegraded":"2023-10-08T16:05:19.102108+0000","last_fullsized":"2023-10-08T16:05:19.102108+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:37:21.023580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439797+0000","last_change":"2023-10-08T16:05:18.083258+0000","last_active":"2023-10-08T16:05:19.439797+0000","last_peered":"2023-10-08T16:05:19.439797+0000","last_clean":"2023-10-08T16:05:19.439797+0000","last_became_active":"2023-10-08T16:05:18.082808+0000","last_became_peered":"2023-10-08T16:05:18.082808+0000","last_unstale":"2023-10-08T16:05:19.439797+0000","last_undegraded":"2023-10-08T16:05:19.439797+0000","last_fullsized":"2023-10-08T16:05:19.439797+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:30.112796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102214+0000","last_change":"2023-10-08T16:05:19.102214+0000","last_active":"2023-10-08T16:05:19.102214+0000","last_peered":"2023-10-08T16:05:19.102214+0000","last_clean":"2023-10-08T16:05:19.102214+0000","last_became_active":"2023-10-08T16:05:19.101716+0000","last_became_peered":"2023-10-08T16:05:19.101716+0000","last_unstale":"2023-10-08T16:05:19.102214+0000","last_undegraded":"2023-10-08T16:05:19.102214+0000","last_fullsized":"2023-10-08T16:05:19.102214+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:12.246939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103776+0000","last_change":"2023-10-08T16:05:18.083026+0000","last_active":"2023-10-08T16:05:19.103776+0000","last_peered":"2023-10-08T16:05:19.103776+0000","last_clean":"2023-10-08T16:05:19.103776+0000","last_became_active":"2023-10-08T16:05:18.082522+0000","last_became_peered":"2023-10-08T16:05:18.082522+0000","last_unstale":"2023-10-08T16:05:19.103776+0000","last_undegraded":"2023-10-08T16:05:19.103776+0000","last_fullsized":"2023-10-08T16:05:19.103776+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094558+0000","last_change":"2023-10-08T16:05:19.094558+0000","last_active":"2023-10-08T16:05:19.094558+0000","last_peered":"2023-10-08T16:05:19.094558+0000","last_clean":"2023-10-08T16:05:19.094558+0000","last_became_active":"2023-10-08T16:05:19.094330+0000","last_became_peered":"2023-10-08T16:05:19.094330+0000","last_unstale":"2023-10-08T16:05:19.094558+0000","last_undegraded":"2023-10-08T16:05:19.094558+0000","last_fullsized":"2023-10-08T16:05:19.094558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:30.509876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101639+0000","last_change":"2023-10-08T16:05:18.081112+0000","last_active":"2023-10-08T16:05:19.101639+0000","last_peered":"2023-10-08T16:05:19.101639+0000","last_clean":"2023-10-08T16:05:19.101639+0000","last_became_active":"2023-10-08T16:05:18.080818+0000","last_became_peered":"2023-10-08T16:05:18.080818+0000","last_unstale":"2023-10-08T16:05:19.101639+0000","last_undegraded":"2023-10-08T16:05:19.101639+0000","last_fullsized":"2023-10-08T16:05:19.101639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:32.348080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.099914+0000","last_change":"2023-10-08T16:05:19.099914+0000","last_active":"2023-10-08T16:05:19.099914+0000","last_peered":"2023-10-08T16:05:19.099914+0000","last_clean":"2023-10-08T16:05:19.099914+0000","last_became_active":"2023-10-08T16:05:19.099587+0000","last_became_peered":"2023-10-08T16:05:19.099587+0000","last_unstale":"2023-10-08T16:05:19.099914+0000","last_undegraded":"2023-10-08T16:05:19.099914+0000","last_fullsized":"2023-10-08T16:05:19.099914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:39.319258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084442+0000","last_change":"2023-10-08T16:05:18.080126+0000","last_active":"2023-10-08T16:05:18.084442+0000","last_peered":"2023-10-08T16:05:18.084442+0000","last_clean":"2023-10-08T16:05:18.084442+0000","last_became_active":"2023-10-08T16:05:18.079862+0000","last_became_peered":"2023-10-08T16:05:18.079862+0000","last_unstale":"2023-10-08T16:05:18.084442+0000","last_undegraded":"2023-10-08T16:05:18.084442+0000","last_fullsized":"2023-10-08T16:05:18.084442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:47.307711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096205+0000","last_change":"2023-10-08T16:05:19.096205+0000","last_active":"2023-10-08T16:05:19.096205+0000","last_peered":"2023-10-08T16:05:19.096205+0000","last_clean":"2023-10-08T16:05:19.096205+0000","last_became_active":"2023-10-08T16:05:19.095949+0000","last_became_peered":"2023-10-08T16:05:19.095949+0000","last_unstale":"2023-10-08T16:05:19.096205+0000","last_undegraded":"2023-10-08T16:05:19.096205+0000","last_fullsized":"2023-10-08T16:05:19.096205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:56.958992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088874+0000","last_change":"2023-10-08T16:05:18.084486+0000","last_active":"2023-10-08T16:05:18.088874+0000","last_peered":"2023-10-08T16:05:18.088874+0000","last_clean":"2023-10-08T16:05:18.088874+0000","last_became_active":"2023-10-08T16:05:18.084201+0000","last_became_peered":"2023-10-08T16:05:18.084201+0000","last_unstale":"2023-10-08T16:05:18.088874+0000","last_undegraded":"2023-10-08T16:05:18.088874+0000","last_fullsized":"2023-10-08T16:05:18.088874+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094142+0000","last_change":"2023-10-08T16:05:19.094142+0000","last_active":"2023-10-08T16:05:19.094142+0000","last_peered":"2023-10-08T16:05:19.094142+0000","last_clean":"2023-10-08T16:05:19.094142+0000","last_became_active":"2023-10-08T16:05:19.093878+0000","last_became_peered":"2023-10-08T16:05:19.093878+0000","last_unstale":"2023-10-08T16:05:19.094142+0000","last_undegraded":"2023-10-08T16:05:19.094142+0000","last_fullsized":"2023-10-08T16:05:19.094142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:23:27.297231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086512+0000","last_change":"2023-10-08T16:05:18.085255+0000","last_active":"2023-10-08T16:05:18.086512+0000","last_peered":"2023-10-08T16:05:18.086512+0000","last_clean":"2023-10-08T16:05:18.086512+0000","last_became_active":"2023-10-08T16:05:18.085021+0000","last_became_peered":"2023-10-08T16:05:18.085021+0000","last_unstale":"2023-10-08T16:05:18.086512+0000","last_undegraded":"2023-10-08T16:05:18.086512+0000","last_fullsized":"2023-10-08T16:05:18.086512+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:25.746772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090069+0000","last_change":"2023-10-08T16:05:19.090069+0000","last_active":"2023-10-08T16:05:19.090069+0000","last_peered":"2023-10-08T16:05:19.090069+0000","last_clean":"2023-10-08T16:05:19.090069+0000","last_became_active":"2023-10-08T16:05:19.089797+0000","last_became_peered":"2023-10-08T16:05:19.089797+0000","last_unstale":"2023-10-08T16:05:19.090069+0000","last_undegraded":"2023-10-08T16:05:19.090069+0000","last_fullsized":"2023-10-08T16:05:19.090069+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086705+0000","last_change":"2023-10-08T16:05:18.085718+0000","last_active":"2023-10-08T16:05:18.086705+0000","last_peered":"2023-10-08T16:05:18.086705+0000","last_clean":"2023-10-08T16:05:18.086705+0000","last_became_active":"2023-10-08T16:05:18.085500+0000","last_became_peered":"2023-10-08T16:05:18.085500+0000","last_unstale":"2023-10-08T16:05:18.086705+0000","last_undegraded":"2023-10-08T16:05:18.086705+0000","last_fullsized":"2023-10-08T16:05:18.086705+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:39.585808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090833+0000","last_change":"2023-10-08T16:05:19.090833+0000","last_active":"2023-10-08T16:05:19.090833+0000","last_peered":"2023-10-08T16:05:19.090833+0000","last_clean":"2023-10-08T16:05:19.090833+0000","last_became_active":"2023-10-08T16:05:19.090593+0000","last_became_peered":"2023-10-08T16:05:19.090593+0000","last_unstale":"2023-10-08T16:05:19.090833+0000","last_undegraded":"2023-10-08T16:05:19.090833+0000","last_fullsized":"2023-10-08T16:05:19.090833+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:58.341584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439917+0000","last_change":"2023-10-08T16:05:18.085311+0000","last_active":"2023-10-08T16:05:19.439917+0000","last_peered":"2023-10-08T16:05:19.439917+0000","last_clean":"2023-10-08T16:05:19.439917+0000","last_became_active":"2023-10-08T16:05:18.085040+0000","last_became_peered":"2023-10-08T16:05:18.085040+0000","last_unstale":"2023-10-08T16:05:19.439917+0000","last_undegraded":"2023-10-08T16:05:19.439917+0000","last_fullsized":"2023-10-08T16:05:19.439917+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.686570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.096331+0000","last_change":"2023-10-08T16:05:19.096331+0000","last_active":"2023-10-08T16:05:19.096331+0000","last_peered":"2023-10-08T16:05:19.096331+0000","last_clean":"2023-10-08T16:05:19.096331+0000","last_became_active":"2023-10-08T16:05:19.095972+0000","last_became_peered":"2023-10-08T16:05:19.095972+0000","last_unstale":"2023-10-08T16:05:19.096331+0000","last_undegraded":"2023-10-08T16:05:19.096331+0000","last_fullsized":"2023-10-08T16:05:19.096331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'5","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.441148+0000","last_change":"2023-10-08T16:05:18.082879+0000","last_active":"2023-10-08T16:05:19.441148+0000","last_peered":"2023-10-08T16:05:19.441148+0000","last_clean":"2023-10-08T16:05:19.441148+0000","last_became_active":"2023-10-08T16:05:18.082599+0000","last_became_peered":"2023-10-08T16:05:18.082599+0000","last_unstale":"2023-10-08T16:05:19.441148+0000","last_undegraded":"2023-10-08T16:05:19.441148+0000","last_fullsized":"2023-10-08T16:05:19.441148+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:04.060192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":644,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097582+0000","last_change":"2023-10-08T16:05:19.097582+0000","last_active":"2023-10-08T16:05:19.097582+0000","last_peered":"2023-10-08T16:05:19.097582+0000","last_clean":"2023-10-08T16:05:19.097582+0000","last_became_active":"2023-10-08T16:05:19.097315+0000","last_became_peered":"2023-10-08T16:05:19.097315+0000","last_unstale":"2023-10-08T16:05:19.097582+0000","last_undegraded":"2023-10-08T16:05:19.097582+0000","last_fullsized":"2023-10-08T16:05:19.097582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:54.206804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.440249+0000","last_change":"2023-10-08T16:05:18.083384+0000","last_active":"2023-10-08T16:05:19.440249+0000","last_peered":"2023-10-08T16:05:19.440249+0000","last_clean":"2023-10-08T16:05:19.440249+0000","last_became_active":"2023-10-08T16:05:18.082938+0000","last_became_peered":"2023-10-08T16:05:18.082938+0000","last_unstale":"2023-10-08T16:05:19.440249+0000","last_undegraded":"2023-10-08T16:05:19.440249+0000","last_fullsized":"2023-10-08T16:05:19.440249+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:37.490723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093782+0000","last_change":"2023-10-08T16:05:19.093782+0000","last_active":"2023-10-08T16:05:19.093782+0000","last_peered":"2023-10-08T16:05:19.093782+0000","last_clean":"2023-10-08T16:05:19.093782+0000","last_became_active":"2023-10-08T16:05:19.093528+0000","last_became_peered":"2023-10-08T16:05:19.093528+0000","last_unstale":"2023-10-08T16:05:19.093782+0000","last_undegraded":"2023-10-08T16:05:19.093782+0000","last_fullsized":"2023-10-08T16:05:19.093782+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086807+0000","last_change":"2023-10-08T16:05:18.086321+0000","last_active":"2023-10-08T16:05:18.086807+0000","last_peered":"2023-10-08T16:05:18.086807+0000","last_clean":"2023-10-08T16:05:18.086807+0000","last_became_active":"2023-10-08T16:05:18.085850+0000","last_became_peered":"2023-10-08T16:05:18.085850+0000","last_unstale":"2023-10-08T16:05:18.086807+0000","last_undegraded":"2023-10-08T16:05:18.086807+0000","last_fullsized":"2023-10-08T16:05:18.086807+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:49.615532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102042+0000","last_change":"2023-10-08T16:05:19.102042+0000","last_active":"2023-10-08T16:05:19.102042+0000","last_peered":"2023-10-08T16:05:19.102042+0000","last_clean":"2023-10-08T16:05:19.102042+0000","last_became_active":"2023-10-08T16:05:19.101735+0000","last_became_peered":"2023-10-08T16:05:19.101735+0000","last_unstale":"2023-10-08T16:05:19.102042+0000","last_undegraded":"2023-10-08T16:05:19.102042+0000","last_fullsized":"2023-10-08T16:05:19.102042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:15.371874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085173+0000","last_change":"2023-10-08T16:05:18.084189+0000","last_active":"2023-10-08T16:05:18.085173+0000","last_peered":"2023-10-08T16:05:18.085173+0000","last_clean":"2023-10-08T16:05:18.085173+0000","last_became_active":"2023-10-08T16:05:18.083938+0000","last_became_peered":"2023-10-08T16:05:18.083938+0000","last_unstale":"2023-10-08T16:05:18.085173+0000","last_undegraded":"2023-10-08T16:05:18.085173+0000","last_fullsized":"2023-10-08T16:05:18.085173+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:26.482107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101815+0000","last_change":"2023-10-08T16:05:19.101815+0000","last_active":"2023-10-08T16:05:19.101815+0000","last_peered":"2023-10-08T16:05:19.101815+0000","last_clean":"2023-10-08T16:05:19.101815+0000","last_became_active":"2023-10-08T16:05:19.101561+0000","last_became_peered":"2023-10-08T16:05:19.101561+0000","last_unstale":"2023-10-08T16:05:19.101815+0000","last_undegraded":"2023-10-08T16:05:19.101815+0000","last_fullsized":"2023-10-08T16:05:19.101815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.441364+0000","last_change":"2023-10-08T16:05:18.077345+0000","last_active":"2023-10-08T16:05:19.441364+0000","last_peered":"2023-10-08T16:05:19.441364+0000","last_clean":"2023-10-08T16:05:19.441364+0000","last_became_active":"2023-10-08T16:05:18.077120+0000","last_became_peered":"2023-10-08T16:05:18.077120+0000","last_unstale":"2023-10-08T16:05:19.441364+0000","last_undegraded":"2023-10-08T16:05:19.441364+0000","last_fullsized":"2023-10-08T16:05:19.441364+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.429484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101881+0000","last_change":"2023-10-08T16:05:19.101881+0000","last_active":"2023-10-08T16:05:19.101881+0000","last_peered":"2023-10-08T16:05:19.101881+0000","last_clean":"2023-10-08T16:05:19.101881+0000","last_became_active":"2023-10-08T16:05:19.101548+0000","last_became_peered":"2023-10-08T16:05:19.101548+0000","last_unstale":"2023-10-08T16:05:19.101881+0000","last_undegraded":"2023-10-08T16:05:19.101881+0000","last_fullsized":"2023-10-08T16:05:19.101881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:48:26.751269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.082879+0000","last_change":"2023-10-08T16:05:18.081860+0000","last_active":"2023-10-08T16:05:19.082879+0000","last_peered":"2023-10-08T16:05:19.082879+0000","last_clean":"2023-10-08T16:05:19.082879+0000","last_became_active":"2023-10-08T16:05:18.081615+0000","last_became_peered":"2023-10-08T16:05:18.081615+0000","last_unstale":"2023-10-08T16:05:19.082879+0000","last_undegraded":"2023-10-08T16:05:19.082879+0000","last_fullsized":"2023-10-08T16:05:19.082879+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:18:01.935637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.102020+0000","last_change":"2023-10-08T16:05:19.102020+0000","last_active":"2023-10-08T16:05:19.102020+0000","last_peered":"2023-10-08T16:05:19.102020+0000","last_clean":"2023-10-08T16:05:19.102020+0000","last_became_active":"2023-10-08T16:05:19.101774+0000","last_became_peered":"2023-10-08T16:05:19.101774+0000","last_unstale":"2023-10-08T16:05:19.102020+0000","last_undegraded":"2023-10-08T16:05:19.102020+0000","last_fullsized":"2023-10-08T16:05:19.102020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:51:50.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.439947+0000","last_change":"2023-10-08T16:05:18.083140+0000","last_active":"2023-10-08T16:05:19.439947+0000","last_peered":"2023-10-08T16:05:19.439947+0000","last_clean":"2023-10-08T16:05:19.439947+0000","last_became_active":"2023-10-08T16:05:18.082853+0000","last_became_peered":"2023-10-08T16:05:18.082853+0000","last_unstale":"2023-10-08T16:05:19.439947+0000","last_undegraded":"2023-10-08T16:05:19.439947+0000","last_fullsized":"2023-10-08T16:05:19.439947+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:32.110466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.104911+0000","last_change":"2023-10-08T16:05:19.104911+0000","last_active":"2023-10-08T16:05:19.104911+0000","last_peered":"2023-10-08T16:05:19.104911+0000","last_clean":"2023-10-08T16:05:19.104911+0000","last_became_active":"2023-10-08T16:05:19.104695+0000","last_became_peered":"2023-10-08T16:05:19.104695+0000","last_unstale":"2023-10-08T16:05:19.104911+0000","last_undegraded":"2023-10-08T16:05:19.104911+0000","last_fullsized":"2023-10-08T16:05:19.104911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:21.463706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.103413+0000","last_change":"2023-10-08T16:05:18.082063+0000","last_active":"2023-10-08T16:05:19.103413+0000","last_peered":"2023-10-08T16:05:19.103413+0000","last_clean":"2023-10-08T16:05:19.103413+0000","last_became_active":"2023-10-08T16:05:18.081757+0000","last_became_peered":"2023-10-08T16:05:18.081757+0000","last_unstale":"2023-10-08T16:05:19.103413+0000","last_undegraded":"2023-10-08T16:05:19.103413+0000","last_fullsized":"2023-10-08T16:05:19.103413+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:37.834689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.095559+0000","last_change":"2023-10-08T16:05:19.095559+0000","last_active":"2023-10-08T16:05:19.095559+0000","last_peered":"2023-10-08T16:05:19.095559+0000","last_clean":"2023-10-08T16:05:19.095559+0000","last_became_active":"2023-10-08T16:05:19.095312+0000","last_became_peered":"2023-10-08T16:05:19.095312+0000","last_unstale":"2023-10-08T16:05:19.095559+0000","last_undegraded":"2023-10-08T16:05:19.095559+0000","last_fullsized":"2023-10-08T16:05:19.095559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:05:33.114963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088844+0000","last_change":"2023-10-08T16:05:18.084363+0000","last_active":"2023-10-08T16:05:18.088844+0000","last_peered":"2023-10-08T16:05:18.088844+0000","last_clean":"2023-10-08T16:05:18.088844+0000","last_became_active":"2023-10-08T16:05:18.084086+0000","last_became_peered":"2023-10-08T16:05:18.084086+0000","last_unstale":"2023-10-08T16:05:18.088844+0000","last_undegraded":"2023-10-08T16:05:18.088844+0000","last_fullsized":"2023-10-08T16:05:18.088844+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:18.602093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.101888+0000","last_change":"2023-10-08T16:05:19.101888+0000","last_active":"2023-10-08T16:05:19.101888+0000","last_peered":"2023-10-08T16:05:19.101888+0000","last_clean":"2023-10-08T16:05:19.101888+0000","last_became_active":"2023-10-08T16:05:19.101584+0000","last_became_peered":"2023-10-08T16:05:19.101584+0000","last_unstale":"2023-10-08T16:05:19.101888+0000","last_undegraded":"2023-10-08T16:05:19.101888+0000","last_fullsized":"2023-10-08T16:05:19.101888+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:34.408160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.440646+0000","last_change":"2023-10-08T16:05:18.084837+0000","last_active":"2023-10-08T16:05:19.440646+0000","last_peered":"2023-10-08T16:05:19.440646+0000","last_clean":"2023-10-08T16:05:19.440646+0000","last_became_active":"2023-10-08T16:05:18.084581+0000","last_became_peered":"2023-10-08T16:05:18.084581+0000","last_unstale":"2023-10-08T16:05:19.440646+0000","last_undegraded":"2023-10-08T16:05:19.440646+0000","last_fullsized":"2023-10-08T16:05:19.440646+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:51.440542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.092611+0000","last_change":"2023-10-08T16:05:19.092611+0000","last_active":"2023-10-08T16:05:19.092611+0000","last_peered":"2023-10-08T16:05:19.092611+0000","last_clean":"2023-10-08T16:05:19.092611+0000","last_became_active":"2023-10-08T16:05:19.092184+0000","last_became_peered":"2023-10-08T16:05:19.092184+0000","last_unstale":"2023-10-08T16:05:19.092611+0000","last_undegraded":"2023-10-08T16:05:19.092611+0000","last_fullsized":"2023-10-08T16:05:19.092611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:38.839186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.087028+0000","last_change":"2023-10-08T16:05:18.084953+0000","last_active":"2023-10-08T16:05:18.087028+0000","last_peered":"2023-10-08T16:05:18.087028+0000","last_clean":"2023-10-08T16:05:18.087028+0000","last_became_active":"2023-10-08T16:05:18.084672+0000","last_became_peered":"2023-10-08T16:05:18.084672+0000","last_unstale":"2023-10-08T16:05:18.087028+0000","last_undegraded":"2023-10-08T16:05:18.087028+0000","last_fullsized":"2023-10-08T16:05:18.087028+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.094119+0000","last_change":"2023-10-08T16:05:19.094119+0000","last_active":"2023-10-08T16:05:19.094119+0000","last_peered":"2023-10-08T16:05:19.094119+0000","last_clean":"2023-10-08T16:05:19.094119+0000","last_became_active":"2023-10-08T16:05:19.093848+0000","last_became_peered":"2023-10-08T16:05:19.093848+0000","last_unstale":"2023-10-08T16:05:19.094119+0000","last_undegraded":"2023-10-08T16:05:19.094119+0000","last_fullsized":"2023-10-08T16:05:19.094119+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:19.606590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.088721+0000","last_change":"2023-10-08T16:05:18.082002+0000","last_active":"2023-10-08T16:05:18.088721+0000","last_peered":"2023-10-08T16:05:18.088721+0000","last_clean":"2023-10-08T16:05:18.088721+0000","last_became_active":"2023-10-08T16:05:18.080550+0000","last_became_peered":"2023-10-08T16:05:18.080550+0000","last_unstale":"2023-10-08T16:05:18.088721+0000","last_undegraded":"2023-10-08T16:05:18.088721+0000","last_fullsized":"2023-10-08T16:05:18.088721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.091130+0000","last_change":"2023-10-08T16:05:19.091130+0000","last_active":"2023-10-08T16:05:19.091130+0000","last_peered":"2023-10-08T16:05:19.091130+0000","last_clean":"2023-10-08T16:05:19.091130+0000","last_became_active":"2023-10-08T16:05:19.090796+0000","last_became_peered":"2023-10-08T16:05:19.090796+0000","last_unstale":"2023-10-08T16:05:19.091130+0000","last_undegraded":"2023-10-08T16:05:19.091130+0000","last_fullsized":"2023-10-08T16:05:19.091130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:54:06.073457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.086870+0000","last_change":"2023-10-08T16:05:18.084760+0000","last_active":"2023-10-08T16:05:18.086870+0000","last_peered":"2023-10-08T16:05:18.086870+0000","last_clean":"2023-10-08T16:05:18.086870+0000","last_became_active":"2023-10-08T16:05:18.084511+0000","last_became_peered":"2023-10-08T16:05:18.084511+0000","last_unstale":"2023-10-08T16:05:18.086870+0000","last_undegraded":"2023-10-08T16:05:18.086870+0000","last_fullsized":"2023-10-08T16:05:18.086870+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:13.429484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.097278+0000","last_change":"2023-10-08T16:05:19.097278+0000","last_active":"2023-10-08T16:05:19.097278+0000","last_peered":"2023-10-08T16:05:19.097278+0000","last_clean":"2023-10-08T16:05:19.097278+0000","last_became_active":"2023-10-08T16:05:19.096939+0000","last_became_peered":"2023-10-08T16:05:19.096939+0000","last_unstale":"2023-10-08T16:05:19.097278+0000","last_undegraded":"2023-10-08T16:05:19.097278+0000","last_fullsized":"2023-10-08T16:05:19.097278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:08.766233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.438801+0000","last_change":"2023-10-08T16:05:18.081498+0000","last_active":"2023-10-08T16:05:19.438801+0000","last_peered":"2023-10-08T16:05:19.438801+0000","last_clean":"2023-10-08T16:05:19.438801+0000","last_became_active":"2023-10-08T16:05:18.081246+0000","last_became_peered":"2023-10-08T16:05:18.081246+0000","last_unstale":"2023-10-08T16:05:19.438801+0000","last_undegraded":"2023-10-08T16:05:19.438801+0000","last_fullsized":"2023-10-08T16:05:19.438801+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:27:12.744966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093558+0000","last_change":"2023-10-08T16:05:19.093558+0000","last_active":"2023-10-08T16:05:19.093558+0000","last_peered":"2023-10-08T16:05:19.093558+0000","last_clean":"2023-10-08T16:05:19.093558+0000","last_became_active":"2023-10-08T16:05:19.093237+0000","last_became_peered":"2023-10-08T16:05:19.093237+0000","last_unstale":"2023-10-08T16:05:19.093558+0000","last_undegraded":"2023-10-08T16:05:19.093558+0000","last_fullsized":"2023-10-08T16:05:19.093558+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:31:13.466100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085936+0000","last_change":"2023-10-08T16:05:18.084693+0000","last_active":"2023-10-08T16:05:18.085936+0000","last_peered":"2023-10-08T16:05:18.085936+0000","last_clean":"2023-10-08T16:05:18.085936+0000","last_became_active":"2023-10-08T16:05:18.084447+0000","last_became_peered":"2023-10-08T16:05:18.084447+0000","last_unstale":"2023-10-08T16:05:18.085936+0000","last_undegraded":"2023-10-08T16:05:18.085936+0000","last_fullsized":"2023-10-08T16:05:18.085936+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:54.234392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090027+0000","last_change":"2023-10-08T16:05:19.090027+0000","last_active":"2023-10-08T16:05:19.090027+0000","last_peered":"2023-10-08T16:05:19.090027+0000","last_clean":"2023-10-08T16:05:19.090027+0000","last_became_active":"2023-10-08T16:05:19.089799+0000","last_became_peered":"2023-10-08T16:05:19.089799+0000","last_unstale":"2023-10-08T16:05:19.090027+0000","last_undegraded":"2023-10-08T16:05:19.090027+0000","last_fullsized":"2023-10-08T16:05:19.090027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:48.312208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.085964+0000","last_change":"2023-10-08T16:05:18.081813+0000","last_active":"2023-10-08T16:05:18.085964+0000","last_peered":"2023-10-08T16:05:18.085964+0000","last_clean":"2023-10-08T16:05:18.085964+0000","last_became_active":"2023-10-08T16:05:18.081096+0000","last_became_peered":"2023-10-08T16:05:18.081096+0000","last_unstale":"2023-10-08T16:05:18.085964+0000","last_undegraded":"2023-10-08T16:05:18.085964+0000","last_fullsized":"2023-10-08T16:05:18.085964+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093655+0000","last_change":"2023-10-08T16:05:19.093655+0000","last_active":"2023-10-08T16:05:19.093655+0000","last_peered":"2023-10-08T16:05:19.093655+0000","last_clean":"2023-10-08T16:05:19.093655+0000","last_became_active":"2023-10-08T16:05:19.093359+0000","last_became_peered":"2023-10-08T16:05:19.093359+0000","last_unstale":"2023-10-08T16:05:19.093655+0000","last_undegraded":"2023-10-08T16:05:19.093655+0000","last_fullsized":"2023-10-08T16:05:19.093655+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:19:51.196559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084982+0000","last_change":"2023-10-08T16:05:18.082084+0000","last_active":"2023-10-08T16:05:18.084982+0000","last_peered":"2023-10-08T16:05:18.084982+0000","last_clean":"2023-10-08T16:05:18.084982+0000","last_became_active":"2023-10-08T16:05:18.081851+0000","last_became_peered":"2023-10-08T16:05:18.081851+0000","last_unstale":"2023-10-08T16:05:18.084982+0000","last_undegraded":"2023-10-08T16:05:18.084982+0000","last_fullsized":"2023-10-08T16:05:18.084982+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.093449+0000","last_change":"2023-10-08T16:05:19.093449+0000","last_active":"2023-10-08T16:05:19.093449+0000","last_peered":"2023-10-08T16:05:19.093449+0000","last_clean":"2023-10-08T16:05:19.093449+0000","last_became_active":"2023-10-08T16:05:19.093127+0000","last_became_peered":"2023-10-08T16:05:19.093127+0000","last_unstale":"2023-10-08T16:05:19.093449+0000","last_undegraded":"2023-10-08T16:05:19.093449+0000","last_fullsized":"2023-10-08T16:05:19.093449+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:14.433981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084495+0000","last_change":"2023-10-08T16:05:18.081569+0000","last_active":"2023-10-08T16:05:18.084495+0000","last_peered":"2023-10-08T16:05:18.084495+0000","last_clean":"2023-10-08T16:05:18.084495+0000","last_became_active":"2023-10-08T16:05:18.081316+0000","last_became_peered":"2023-10-08T16:05:18.081316+0000","last_unstale":"2023-10-08T16:05:18.084495+0000","last_undegraded":"2023-10-08T16:05:18.084495+0000","last_fullsized":"2023-10-08T16:05:18.084495+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:19:58.178415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090128+0000","last_change":"2023-10-08T16:05:19.090128+0000","last_active":"2023-10-08T16:05:19.090128+0000","last_peered":"2023-10-08T16:05:19.090128+0000","last_clean":"2023-10-08T16:05:19.090128+0000","last_became_active":"2023-10-08T16:05:19.089876+0000","last_became_peered":"2023-10-08T16:05:19.089876+0000","last_unstale":"2023-10-08T16:05:19.090128+0000","last_undegraded":"2023-10-08T16:05:19.090128+0000","last_fullsized":"2023-10-08T16:05:19.090128+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:04.471470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.087643+0000","last_change":"2023-10-08T16:05:18.079687+0000","last_active":"2023-10-08T16:05:18.087643+0000","last_peered":"2023-10-08T16:05:18.087643+0000","last_clean":"2023-10-08T16:05:18.087643+0000","last_became_active":"2023-10-08T16:05:18.079403+0000","last_became_peered":"2023-10-08T16:05:18.079403+0000","last_unstale":"2023-10-08T16:05:18.087643+0000","last_undegraded":"2023-10-08T16:05:18.087643+0000","last_fullsized":"2023-10-08T16:05:18.087643+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:07.761736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.105762+0000","last_change":"2023-10-08T16:05:19.105762+0000","last_active":"2023-10-08T16:05:19.105762+0000","last_peered":"2023-10-08T16:05:19.105762+0000","last_clean":"2023-10-08T16:05:19.105762+0000","last_became_active":"2023-10-08T16:05:19.102247+0000","last_became_peered":"2023-10-08T16:05:19.102247+0000","last_unstale":"2023-10-08T16:05:19.105762+0000","last_undegraded":"2023-10-08T16:05:19.105762+0000","last_fullsized":"2023-10-08T16:05:19.105762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:57.080931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.084815+0000","last_change":"2023-10-08T16:05:18.084080+0000","last_active":"2023-10-08T16:05:18.084815+0000","last_peered":"2023-10-08T16:05:18.084815+0000","last_clean":"2023-10-08T16:05:18.084815+0000","last_became_active":"2023-10-08T16:05:18.083817+0000","last_became_peered":"2023-10-08T16:05:18.083817+0000","last_unstale":"2023-10-08T16:05:18.084815+0000","last_undegraded":"2023-10-08T16:05:18.084815+0000","last_fullsized":"2023-10-08T16:05:18.084815+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:34.843700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-08T16:05:19.090851+0000","last_change":"2023-10-08T16:05:19.090851+0000","last_active":"2023-10-08T16:05:19.090851+0000","last_peered":"2023-10-08T16:05:19.090851+0000","last_clean":"2023-10-08T16:05:19.090851+0000","last_became_active":"2023-10-08T16:05:19.090514+0000","last_became_peered":"2023-10-08T16:05:19.090514+0000","last_unstale":"2023-10-08T16:05:19.090851+0000","last_undegraded":"2023-10-08T16:05:19.090851+0000","last_fullsized":"2023-10-08T16:05:19.090851+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:18.064650+0000","last_clean_scrub_stamp":"2023-10-08T16:05:18.064650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:45:28.967956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.090095+0000","last_change":"2023-10-08T16:05:18.089445+0000","last_active":"2023-10-08T16:05:18.090095+0000","last_peered":"2023-10-08T16:05:18.090095+0000","last_clean":"2023-10-08T16:05:18.090095+0000","last_became_active":"2023-10-08T16:05:18.089123+0000","last_became_peered":"2023-10-08T16:05:18.089123+0000","last_unstale":"2023-10-08T16:05:18.090095+0000","last_undegraded":"2023-10-08T16:05:18.090095+0000","last_fullsized":"2023-10-08T16:05:18.090095+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:17.060153+0000","last_clean_scrub_stamp":"2023-10-08T16:05:17.060153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:33.403663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T16:05:18.077013+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:05:18.077013+0000","last_peered":"2023-10-08T16:05:18.077013+0000","last_clean":"2023-10-08T16:05:18.077013+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:05:18.077013+0000","last_undegraded":"2023-10-08T16:05:18.077013+0000","last_fullsized":"2023-10-08T16:05:18.077013+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1414,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":722453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":348160,"data_stored":132085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297628,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074212,"statfs":{"total":96636764160,"available":96331993088,"internally_reserved":0,"allocated":348160,"data_stored":132085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298208,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073632,"statfs":{"total":96636764160,"available":96331399168,"internally_reserved":0,"allocated":942080,"data_stored":722453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":328,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":335872,"data_stored":125322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:05:23.851 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:05:23.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:05:24.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.007+0000 7f0b18db7700 1 Processor -- start 2023-10-08T16:05:24.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.007+0000 7f0b18db7700 1 -- start start 2023-10-08T16:05:24.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b18db7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0b140cbb90 0x7f0b140af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b18db7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140af860 con 0x7f0b14159950 2023-10-08T16:05:24.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b18db7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140ce380 con 0x7f0b14069c70 2023-10-08T16:05:24.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b18db7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b140cec20 con 0x7f0b140cbb90 2023-10-08T16:05:24.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b12d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0b140cbb90 0x7f0b140af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b1259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0b14159950 0x7f0b14069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60642/0 (socket says 172.21.15.170:60642) 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b1259c700 1 -- 172.21.15.170:0/1144379364 learned_addr learned my addr 172.21.15.170:0/1144379364 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1144379364 >> v1:172.21.15.191:6789/0 conn(0x7f0b14069c70 legacy=0x7f0b140cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1144379364 >> v1:172.21.15.170:6789/0 conn(0x7f0b14159950 legacy=0x7f0b14069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1144379364 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140cbfb0 con 0x7f0b140cbb90 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.008+0000 7f0b11d9b700 1 -- 172.21.15.170:0/1144379364 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413088979 0 0) 0x7f0b140af860 con 0x7f0b14159950 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b12d9d700 1 --2- 172.21.15.170:0/1144379364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b140cbb90 0x7f0b140af320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009e60 tx=0x7f0b0c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e13720105209ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b11d9b700 1 -- 172.21.15.170:0/1144379364 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00e070 con 0x7f0b140cbb90 2023-10-08T16:05:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b11d9b700 1 -- 172.21.15.170:0/1144379364 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00bbf0 con 0x7f0b140cbb90 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b11d9b700 1 -- 172.21.15.170:0/1144379364 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c012430 con 0x7f0b140cbb90 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 -- 172.21.15.170:0/1144379364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b140cbb90 msgr2=0x7f0b140af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1144379364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b140cbb90 0x7f0b140af320 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009e60 tx=0x7f0b0c009780 comp rx=0 tx=0).stop 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 -- 172.21.15.170:0/1144379364 shutdown_connections 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1144379364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b140cbb90 0x7f0b140af320 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 -- 172.21.15.170:0/1144379364 >> 172.21.15.170:0/1144379364 conn(0x7f0b14068e00 msgr2=0x7f0b140d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 -- 172.21.15.170:0/1144379364 shutdown_connections 2023-10-08T16:05:24.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.009+0000 7f0b18db7700 1 -- 172.21.15.170:0/1144379364 wait complete. 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.010+0000 7f0b18db7700 1 Processor -- start 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 -- start start 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b14069c70 0x7f0b141521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b140cbb90 0x7f0b140b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14157230 con 0x7f0b14159950 2023-10-08T16:05:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14156190 con 0x7f0b140cbb90 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b18db7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14156470 con 0x7f0b14069c70 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54818/0 (socket says 172.21.15.170:54818) 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1057665746 learned_addr learned my addr 172.21.15.170:0/1057665746 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b1259c700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b14069c70 0x7f0b141521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.011+0000 7f0b09d9b700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b140cbb90 0x7f0b140b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b14069c70 msgr2=0x7f0b141521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b14069c70 0x7f0b141521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b140cbb90 msgr2=0x7f0b140b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b140cbb90 0x7f0b140b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 -- 172.21.15.170:0/1057665746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c009a20 con 0x7f0b14159950 2023-10-08T16:05:24.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b12d9d700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b14070e20 tx=0x7f0b0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00bde0 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c01e640 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b14156750 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c01e9e0 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.012+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b14156d10 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.013+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b14156f00 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.013+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b0c024070 con 0x7f0b14159950 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.014+0000 7f0b0affd700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.014+0000 7f0b1259c700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.014+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b0c087250 con 0x7f0b14159950 2023-10-08T16:05:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.014+0000 7f0b1259c700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 0x7f0af40693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b04007160 tx=0x7f0b04007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.018+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b0c0544c0 con 0x7f0b14159950 2023-10-08T16:05:24.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.278+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0b140718b0 con 0x7f0b14159950 2023-10-08T16:05:24.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.282+0000 7f0b0affd700 1 -- 172.21.15.170:0/1057665746 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0b0c057ae0 con 0x7f0b14159950 2023-10-08T16:05:24.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 msgr2=0x7f0af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 0x7f0af40693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0b04007160 tx=0x7f0b04007f50 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 msgr2=0x7f0b140c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0b14070e20 tx=0x7f0b0c018000 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 shutdown_connections 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0af4066f30 0x7f0af40693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b14159950 0x7f0b140c99b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b14069c70 0x7f0b141521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 --2- 172.21.15.170:0/1057665746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b140cbb90 0x7f0b140b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 >> 172.21.15.170:0/1057665746 conn(0x7f0b14068e00 msgr2=0x7f0b140b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:24.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 shutdown_connections 2023-10-08T16:05:24.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.284+0000 7f0b18db7700 1 -- 172.21.15.170:0/1057665746 wait complete. 2023-10-08T16:05:24.287 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:05:24.297 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:24.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.410+0000 7eff83d11700 1 Processor -- start 2023-10-08T16:05:24.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.410+0000 7eff83d11700 1 -- start start 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff83d11700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 0x7eff7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff83d11700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff7c069560 0x7eff7c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff83d11700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7c069ee0 con 0x7eff7c159c60 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff83d11700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7c15a160 con 0x7eff7c069560 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff83d11700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff7c0d13a0 con 0x7eff7c159890 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff7c069560 0x7eff7c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff7c069560 0x7eff7c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54834/0 (socket says 172.21.15.170:54834) 2023-10-08T16:05:24.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 -- 172.21.15.170:0/991822357 learned_addr learned my addr 172.21.15.170:0/991822357 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff81aad700 1 --2- 172.21.15.170:0/991822357 >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 0x7eff7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 -- 172.21.15.170:0/991822357 >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 msgr2=0x7eff7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 --2- 172.21.15.170:0/991822357 >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 0x7eff7c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.411+0000 7eff822ae700 1 -- 172.21.15.170:0/991822357 >> v1:172.21.15.191:6789/0 conn(0x7eff7c159c60 legacy=0x7eff7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff822ae700 1 -- 172.21.15.170:0/991822357 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c15e140 con 0x7eff7c069560 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff81aad700 1 --2- 172.21.15.170:0/991822357 >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 0x7eff7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff822ae700 1 --2- 172.21.15.170:0/991822357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c0699a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0d1090 tx=0x7eff74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06123feb182652d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff80aab700 1 -- 172.21.15.170:0/991822357 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400b820 con 0x7eff7c069560 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff80aab700 1 -- 172.21.15.170:0/991822357 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7400b980 con 0x7eff7c069560 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff80aab700 1 -- 172.21.15.170:0/991822357 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400bd20 con 0x7eff7c069560 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 -- 172.21.15.170:0/991822357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 msgr2=0x7eff7c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 --2- 172.21.15.170:0/991822357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c0699a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0d1090 tx=0x7eff74009250 comp rx=0 tx=0).stop 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 -- 172.21.15.170:0/991822357 shutdown_connections 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 --2- 172.21.15.170:0/991822357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c0699a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 --2- 172.21.15.170:0/991822357 >> v2:172.21.15.191:3300/0 conn(0x7eff7c159890 0x7eff7c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 -- 172.21.15.170:0/991822357 >> 172.21.15.170:0/991822357 conn(0x7eff7c068e20 msgr2=0x7eff7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.412+0000 7eff83d11700 1 -- 172.21.15.170:0/991822357 shutdown_connections 2023-10-08T16:05:24.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.413+0000 7eff83d11700 1 -- 172.21.15.170:0/991822357 wait complete. 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 Processor -- start 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 -- start start 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff7c159890 0x7eff7c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff7c159c60 0x7eff7c15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0cd0c0 con 0x7eff7c069560 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c0cd960 con 0x7eff7c159c60 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff83d11700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c140e80 con 0x7eff7c159890 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff81aad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff81aad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54836/0 (socket says 172.21.15.170:54836) 2023-10-08T16:05:24.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.414+0000 7eff81aad700 1 -- 172.21.15.170:0/2908565397 learned_addr learned my addr 172.21.15.170:0/2908565397 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff822ae700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff7c159c60 0x7eff7c15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff812ac700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff7c159890 0x7eff7c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 -- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff7c159890 msgr2=0x7eff7c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff7c159890 0x7eff7c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 -- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff7c159c60 msgr2=0x7eff7c15ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff7c159c60 0x7eff7c15ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 -- 172.21.15.170:0/2908565397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff740094f0 con 0x7eff7c069560 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff81aad700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff7800a400 tx=0x7eff78007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800ae20 con 0x7eff7c069560 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800d650 con 0x7eff7c069560 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0ca580 con 0x7eff7c069560 2023-10-08T16:05:24.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800d9f0 con 0x7eff7c069560 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.415+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c0cd5f0 con 0x7eff7c069560 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.416+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7c04c3c0 con 0x7eff7c069560 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.416+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff78015440 con 0x7eff7c069560 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.417+0000 7eff71ffb700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.417+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff78084d10 con 0x7eff7c069560 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.417+0000 7eff812ac700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:24.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.417+0000 7eff812ac700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0b2050 tx=0x7eff6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:24.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.421+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff7804e5a0 con 0x7eff7c069560 2023-10-08T16:05:24.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.649+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff7c06a5b0 con 0x7eff7c069560 2023-10-08T16:05:24.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.649+0000 7eff71ffb700 1 -- 172.21.15.170:0/2908565397 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4606 (secure 0 0 0) 0x7eff7801a090 con 0x7eff7c069560 2023-10-08T16:05:24.651 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:24.651 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4881,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2746556501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2746556501},{"type":"v1","addr":"172.21.15.170:6836","nonce":2746556501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:24.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 msgr2=0x7eff600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 0x7eff600693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0b2050 tx=0x7eff6c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:24.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 msgr2=0x7eff7c140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:24.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff7800a400 tx=0x7eff78007910 comp rx=0 tx=0).stop 2023-10-08T16:05:24.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 shutdown_connections 2023-10-08T16:05:24.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff7c159c60 0x7eff7c15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff60066f30 0x7eff600693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff7c069560 0x7eff7c140940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 --2- 172.21.15.170:0/2908565397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff7c159890 0x7eff7c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:24.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 >> 172.21.15.170:0/2908565397 conn(0x7eff7c068e20 msgr2=0x7eff7c0c6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:24.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 shutdown_connections 2023-10-08T16:05:24.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:24.651+0000 7eff83d11700 1 -- 172.21.15.170:0/2908565397 wait complete. 2023-10-08T16:05:24.656 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 17 2023-10-08T16:05:25.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:05:18.453208+0000', 'modified': '2023-10-08T16:05:22.437568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4884}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4884': {'gid': 4884, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.170:6837/4215721719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 4215721719}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 4215721719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:05:25.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:05:25.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T16:05:25.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.184+0000 7f5871311700 1 Processor -- start 2023-10-08T16:05:25.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.184+0000 7f5871311700 1 -- start start 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f5871311700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f586c0cbf30 0x7f586c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f5871311700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c0ccf90 0x7f586c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f5871311700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c0cc300 con 0x7f586c15a270 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f5871311700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586c156bf0 con 0x7f586c0cbf30 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f5871311700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0ae950 con 0x7f586c0ccf90 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f586c15a270 0x7f586c0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54620/0 (socket says 172.21.15.170:54620) 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586b7fe700 1 -- 172.21.15.170:0/947554354 learned_addr learned my addr 172.21.15.170:0/947554354 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 --2- 172.21.15.170:0/947554354 >> v2:172.21.15.170:3300/0 conn(0x7f586c0cbf30 0x7f586c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f58627fc700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c0ccf90 0x7f586c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586a7fc700 1 -- 172.21.15.170:0/947554354 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859887261 0 0) 0x7f586c0cc300 con 0x7f586c15a270 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586a7fc700 1 -- 172.21.15.170:0/947554354 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5858003570 con 0x7f586c15a270 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 -- 172.21.15.170:0/947554354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c0ccf90 msgr2=0x7f586c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c0ccf90 0x7f586c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 -- 172.21.15.170:0/947554354 >> v1:172.21.15.170:6789/0 conn(0x7f586c15a270 legacy=0x7f586c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 -- 172.21.15.170:0/947554354 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c0ad370 con 0x7f586c0cbf30 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.185+0000 7f586affd700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0ae410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f585c009910 tx=0x7f585c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c29985aacfe62e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f586a7fc700 1 -- 172.21.15.170:0/947554354 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c011630 con 0x7f586c0cbf30 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f586a7fc700 1 -- 172.21.15.170:0/947554354 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c011790 con 0x7f586c0cbf30 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f586a7fc700 1 -- 172.21.15.170:0/947554354 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c011b30 con 0x7f586c0cbf30 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 -- 172.21.15.170:0/947554354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 msgr2=0x7f586c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0ae410 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f585c009910 tx=0x7f585c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 -- 172.21.15.170:0/947554354 shutdown_connections 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0ae410 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 --2- 172.21.15.170:0/947554354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c0ccf90 0x7f586c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 -- 172.21.15.170:0/947554354 >> 172.21.15.170:0/947554354 conn(0x7f586c06a320 msgr2=0x7f586c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 -- 172.21.15.170:0/947554354 shutdown_connections 2023-10-08T16:05:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.186+0000 7f5871311700 1 -- 172.21.15.170:0/947554354 wait complete. 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.187+0000 7f5871311700 1 Processor -- start 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.187+0000 7f5871311700 1 -- start start 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 0x7f586c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 0x7f586c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0aa980 con 0x7f586c0cbf30 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0c6e90 con 0x7f586c0ccf90 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f5871311700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c0a7960 con 0x7f586c15a270 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f586affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f586affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47080/0 (socket says 172.21.15.170:47080) 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f586affd700 1 -- 172.21.15.170:0/3617781104 learned_addr learned my addr 172.21.15.170:0/3617781104 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f58627fc700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 0x7f586c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f586b7fe700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 0x7f586c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.188+0000 7f586affd700 1 -- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 msgr2=0x7f586c0ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586affd700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 0x7f586c0ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586affd700 1 -- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 msgr2=0x7f586c14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586affd700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 0x7f586c14dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586affd700 1 -- 172.21.15.170:0/3617781104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009420 con 0x7f586c0cbf30 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f58627fc700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 0x7f586c14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586b7fe700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 0x7f586c0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f586affd700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f586c064290 tx=0x7f585c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f58637fe700 1 -- 172.21.15.170:0/3617781104 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c011900 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f58637fe700 1 -- 172.21.15.170:0/3617781104 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c011a80 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c16dc50 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f58637fe700 1 -- 172.21.15.170:0/3617781104 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c025670 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.189+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c0c6b20 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.190+0000 7f58637fe700 1 -- 172.21.15.170:0/3617781104 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f585c023070 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.190+0000 7f58637fe700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 0x7f584c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.191+0000 7f58637fe700 1 -- 172.21.15.170:0/3617781104 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f585c085ae0 con 0x7f586c0cbf30 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.191+0000 7f58627fc700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 0x7f584c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.191+0000 7f58627fc700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 0x7f584c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f586c0a8370 tx=0x7f5854008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.192+0000 7f587a609b80 1 Processor -- start 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 -- start start 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b091b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b09108a70 con 0x563b091daf60 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b09106270 con 0x563b091b1a50 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b09102060 con 0x563b091fbd00 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f586b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f586b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47088/0 (socket says 172.21.15.170:47088) 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.193+0000 7f586b7fe700 1 -- 172.21.15.170:0/258064726 learned_addr learned my addr 172.21.15.170:0/258064726 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586affd700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b091b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f58627fc700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 -- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 msgr2=0x563b091daa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091daa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 -- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 msgr2=0x563b091b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b091b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 -- 172.21.15.170:0/258064726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b08f9c270 con 0x563b091daf60 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586affd700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b091b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f586b7fe700 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5864002220 tx=0x7f5864009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cb59f67bd4e29a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f584bfff700 1 -- 172.21.15.170:0/258064726 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586400eda0 con 0x563b091daf60 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f584bfff700 1 -- 172.21.15.170:0/258064726 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864014480 con 0x563b091daf60 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.194+0000 7f584bfff700 1 -- 172.21.15.170:0/258064726 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864014800 con 0x563b091daf60 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- 172.21.15.170:0/258064726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 msgr2=0x563b091adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5864002220 tx=0x7f5864009040 comp rx=0 tx=0).stop 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- 172.21.15.170:0/258064726 shutdown_connections 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091adcf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b091b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 --2- 172.21.15.170:0/258064726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091daa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- 172.21.15.170:0/258064726 >> 172.21.15.170:0/258064726 conn(0x563b091b3370 msgr2=0x563b091b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- 172.21.15.170:0/258064726 shutdown_connections 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- 172.21.15.170:0/258064726 wait complete. 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 -- start start 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.195+0000 7f587a609b80 1 Processor -- start 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b09208200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091f1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b090ad890 con 0x563b091daf60 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b090c8ea0 con 0x563b091b1a50 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b08f84e30 con 0x563b091fbd00 2023-10-08T16:05:25.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f587a609b80 10 client.0 _ll_register_callbacks cb 0x563b09175110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f58627fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f58627fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47102/0 (socket says 172.21.15.170:47102) 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f58627fc700 1 -- 172.21.15.170:0/1842387336 learned_addr learned my addr 172.21.15.170:0/1842387336 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f586affd700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b09208200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.196+0000 7f586b7fe700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091f1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 -- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 msgr2=0x563b091f1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091f1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 -- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 msgr2=0x563b09208200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b09208200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 -- 172.21.15.170:0/1842387336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586400f040 con 0x563b091daf60 2023-10-08T16:05:25.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58627fc700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f586c0a7dd0 tx=0x7f5854018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f150fb96348df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854014670 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f58540147d0 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b091a6b70 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5854014ba0 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 10 client.5070 fetch_fsmap learned FSMap version 17 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563b08f3efe0 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b091a6b70 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5854014d00 con 0x563b091daf60 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 10 client.5070 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 10 client.5070 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:05:25.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 20 client.5070 populate_metadata read hostname 'smithi170' 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091f3f70 0x563b091f4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 4 client.5070 mds_command: new command op to 4884 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.197+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 --> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b09183aa0 con 0x563b091f3f70 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f586affd700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091f3f70 0x563b091f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f58497fa700 10 client.5070 ms_handle_connect on v2:172.21.15.170:6835/4215721719 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f586affd700 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091f3f70 0x563b091f4340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff4621f8ade95ef server_cookie=d478c10735b3d8b in_seq=0 out_seq=0 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f58497fa700 1 -- 172.21.15.170:0/1842387336 <== mds.0 v2:172.21.15.170:6835/4215721719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563b09183aa0 con 0x563b091f3f70 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f58497fa700 10 client.5070 handle_command_reply: tid=0 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.198+0000 7f587a609b80 1 client.5070 shutdown 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 msgr2=0x563b091ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f586c0a7dd0 tx=0x7f5854018000 comp rx=0 tx=0).stop 2023-10-08T16:05:25.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 shutdown_connections 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091f3f70 0x563b091f4340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091f1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091b1a50 0x563b09208200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 --2- 172.21.15.170:0/1842387336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091daf60 0x563b091ad080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f58497fa700 0 client.5070 ms_handle_reset on v2:172.21.15.170:6835/4215721719 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 >> 172.21.15.170:0/1842387336 conn(0x563b091b3370 msgr2=0x563b091ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 shutdown_connections 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 1 -- 172.21.15.170:0/1842387336 wait complete. 2023-10-08T16:05:25.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.199+0000 7f587a609b80 20 client.5070 trim_cache size 0 max 16384 2023-10-08T16:05:25.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 Processor -- start 2023-10-08T16:05:25.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 -- start start 2023-10-08T16:05:25.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 0x563b091b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b09207370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b091ef9b0 con 0x563b091fbd00 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b09191510 con 0x563b091daf60 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b091aead0 con 0x563b091b1a50 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f586b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f586b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47110/0 (socket says 172.21.15.170:47110) 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.216+0000 7f586b7fe700 1 -- 172.21.15.170:0/2919531894 learned_addr learned my addr 172.21.15.170:0/2919531894 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f58627fc700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b09207370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586affd700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 0x563b091b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 -- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 msgr2=0x563b091b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 0x563b091b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 -- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 msgr2=0x563b09207370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b09207370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 -- 172.21.15.170:0/2919531894 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b090d0780 con 0x563b091fbd00 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f58627fc700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b09207370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586affd700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 0x563b091b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f586b7fe700 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f58640021c0 tx=0x7f5864017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf376f17e41ccef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f5860ff9700 1 -- 172.21.15.170:0/2919531894 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864000fb0 con 0x563b091fbd00 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f5860ff9700 1 -- 172.21.15.170:0/2919531894 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586400eda0 con 0x563b091fbd00 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f5860ff9700 1 -- 172.21.15.170:0/2919531894 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864014840 con 0x563b091fbd00 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 -- 172.21.15.170:0/2919531894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 msgr2=0x563b092078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f58640021c0 tx=0x7f5864017000 comp rx=0 tx=0).stop 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 -- 172.21.15.170:0/2919531894 shutdown_connections 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091fbd00 0x563b092078b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091b1a50 0x563b091b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 --2- 172.21.15.170:0/2919531894 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b09207370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.217+0000 7f587a609b80 1 -- 172.21.15.170:0/2919531894 >> 172.21.15.170:0/2919531894 conn(0x563b091b3370 msgr2=0x563b091f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.218+0000 7f587a609b80 1 -- 172.21.15.170:0/2919531894 shutdown_connections 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.218+0000 7f587a609b80 1 -- 172.21.15.170:0/2919531894 wait complete. 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.218+0000 7f587a609b80 1 -- start start 2023-10-08T16:05:25.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.218+0000 7f587a609b80 1 Processor -- start 2023-10-08T16:05:25.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b091f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b092580b0 con 0x563b091b1a50 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b091f75e0 con 0x563b091daf60 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b091ddce0 con 0x563b091fbd00 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f587a609b80 10 client.0 _ll_register_callbacks cb 0x563b09175110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47120/0 (socket says 172.21.15.170:47120) 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 -- 172.21.15.170:0/719107332 learned_addr learned my addr 172.21.15.170:0/719107332 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f58627fc700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b091f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586b7fe700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 -- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 msgr2=0x563b091dd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091dd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 -- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 msgr2=0x563b091f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b091f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 -- 172.21.15.170:0/719107332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b08f9c270 con 0x563b091b1a50 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f586affd700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f00 tx=0x7f585c027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3a63d51df98a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.219+0000 7f58497fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c070070 con 0x563b091b1a50 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f585c059c60 con 0x563b091b1a50 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5864014480 con 0x563b091b1a50 2023-10-08T16:05:25.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f585c05d9b0 con 0x563b091b1a50 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 10 client.5076 fetch_fsmap learned FSMap version 17 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563b090b7470 con 0x563b091b1a50 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864014480 con 0x563b091b1a50 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f585c055f00 con 0x563b091b1a50 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 10 client.5076 fetch_fsmap finished waiting for FSMap version 17 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 10 client.5076 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 20 client.5076 populate_metadata read hostname 'smithi170' 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091d9840 0x563b091d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 4 client.5076 mds_command: new command op to 4884 tid=0[{"prefix": "status"}] 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 --> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b090a4e00 con 0x563b091d9840 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58627fc700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091d9840 0x563b091d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58497fa700 10 client.5076 ms_handle_connect on v2:172.21.15.170:6835/4215721719 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.220+0000 7f58627fc700 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091d9840 0x563b091d9c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b927f09cf5be95c server_cookie=9df0dda9e9107eb1 in_seq=0 out_seq=0 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f58497fa700 1 -- 172.21.15.170:0/719107332 <== mds.0 v2:172.21.15.170:6835/4215721719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563b090a4e00 con 0x563b091d9840 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f58497fa700 10 client.5076 handle_command_reply: tid=0 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 client.5076 shutdown 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 msgr2=0x563b091ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f585c009f00 tx=0x7f585c027040 comp rx=0 tx=0).stop 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 shutdown_connections 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:6835/4215721719,v1:172.21.15.170:6837/4215721719] conn(0x563b091d9840 0x563b091d9c10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x563b091fbd00 0x563b091dd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563b091b1a50 0x563b091ef1c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 --2- 172.21.15.170:0/719107332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x563b091daf60 0x563b091f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f58497fa700 0 client.5076 ms_handle_reset on v2:172.21.15.170:6835/4215721719 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 >> 172.21.15.170:0/719107332 conn(0x563b091b3370 msgr2=0x563b091adf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 shutdown_connections 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 1 -- 172.21.15.170:0/719107332 wait complete. 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f587a609b80 20 client.5076 trim_cache size 0 max 16384 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 4884, 2023-10-08T16:05:25.230 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.7910236859999999, 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 17, 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 32, 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 32, 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 14.086028541999999 2023-10-08T16:05:25.231 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:05:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 msgr2=0x7f584c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 0x7f584c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f586c0a8370 tx=0x7f5854008040 comp rx=0 tx=0).stop 2023-10-08T16:05:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 msgr2=0x7f586c0aa440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f586c064290 tx=0x7f585c017000 comp rx=0 tx=0).stop 2023-10-08T16:05:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 shutdown_connections 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f586c15a270 0x7f586c0ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f584c066f30 0x7f584c0693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f586c0cbf30 0x7f586c0aa440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 --2- 172.21.15.170:0/3617781104 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f586c0ccf90 0x7f586c14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.221+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 >> 172.21.15.170:0/3617781104 conn(0x7f586c06a320 msgr2=0x7f586c158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.222+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 shutdown_connections 2023-10-08T16:05:25.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.222+0000 7f5871311700 1 -- 172.21.15.170:0/3617781104 wait complete. 2023-10-08T16:05:25.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:05:25.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.348+0000 7fe59fb2c700 1 Processor -- start 2023-10-08T16:05:25.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.348+0000 7fe59fb2c700 1 -- start start 2023-10-08T16:05:25.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59fb2c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe59806bcd0 0x7fe598140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59fb2c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 0x7fe598141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59fb2c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59815c2a0 con 0x7fe5980c8e00 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59fb2c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980c9900 con 0x7fe59806bcd0 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59fb2c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59815b690 con 0x7fe59806c0a0 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59d8c8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe59806bcd0 0x7fe598140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59d8c8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe59806bcd0 0x7fe598140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47122/0 (socket says 172.21.15.170:47122) 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59e0c9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe5980c8e00 0x7fe5980c91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54624/0 (socket says 172.21.15.170:54624) 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.349+0000 7fe59d8c8700 1 -- 172.21.15.170:0/1947714214 learned_addr learned my addr 172.21.15.170:0/1947714214 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d0c7700 1 --2- 172.21.15.170:0/1947714214 >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 0x7fe598141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d8c8700 1 -- 172.21.15.170:0/1947714214 >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 msgr2=0x7fe598141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:25.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d8c8700 1 --2- 172.21.15.170:0/1947714214 >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 0x7fe598141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d8c8700 1 -- 172.21.15.170:0/1947714214 >> v1:172.21.15.170:6789/0 conn(0x7fe5980c8e00 legacy=0x7fe5980c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d8c8700 1 -- 172.21.15.170:0/1947714214 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59815b970 con 0x7fe59806bcd0 2023-10-08T16:05:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d0c7700 1 --2- 172.21.15.170:0/1947714214 >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 0x7fe598141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59d8c8700 1 --2- 172.21.15.170:0/1947714214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806bcd0 0x7fe598140df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe59806c6e0 tx=0x7fe594009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682321fb148cf69e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59c8c6700 1 -- 172.21.15.170:0/1947714214 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59400e070 con 0x7fe59806bcd0 2023-10-08T16:05:25.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59c8c6700 1 -- 172.21.15.170:0/1947714214 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59400ba50 con 0x7fe59806bcd0 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59c8c6700 1 -- 172.21.15.170:0/1947714214 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940092e0 con 0x7fe59806bcd0 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1947714214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806bcd0 msgr2=0x7fe598140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.350+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1947714214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806bcd0 0x7fe598140df0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe59806c6e0 tx=0x7fe594009840 comp rx=0 tx=0).stop 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1947714214 shutdown_connections 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1947714214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806bcd0 0x7fe598140df0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1947714214 >> v2:172.21.15.191:3300/0 conn(0x7fe59806c0a0 0x7fe598141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1947714214 >> 172.21.15.170:0/1947714214 conn(0x7fe5980ce600 msgr2=0x7fe5980cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1947714214 shutdown_connections 2023-10-08T16:05:25.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.351+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1947714214 wait complete. 2023-10-08T16:05:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 Processor -- start 2023-10-08T16:05:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 -- start start 2023-10-08T16:05:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59806bcd0 0x7fe59815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5980c8e00 0x7fe5980ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980b3790 con 0x7fe59806c0a0 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59806a940 con 0x7fe59806bcd0 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.352+0000 7fe59fb2c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980b2b80 con 0x7fe5980c8e00 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47128/0 (socket says 172.21.15.170:47128) 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 -- 172.21.15.170:0/1363311498 learned_addr learned my addr 172.21.15.170:0/1363311498 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d8c8700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59806bcd0 0x7fe59815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59e0c9700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5980c8e00 0x7fe5980ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 -- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5980c8e00 msgr2=0x7fe5980ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5980c8e00 0x7fe5980ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 -- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59806bcd0 msgr2=0x7fe59815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59806bcd0 0x7fe59815ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 -- 172.21.15.170:0/1363311498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe594009ae0 con 0x7fe59806c0a0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59d0c7700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c830 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588011600 con 0x7fe59806c0a0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe588011760 con 0x7fe59806c0a0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5980b2e60 con 0x7fe59806c0a0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe588011b20 con 0x7fe59806c0a0 2023-10-08T16:05:25.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.353+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5980b3420 con 0x7fe59806c0a0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.354+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59804c3c0 con 0x7fe59806c0a0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.355+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe588018360 con 0x7fe59806c0a0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.355+0000 7fe58dffb700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 0x7fe57c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.355+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe5880844c0 con 0x7fe59806c0a0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.355+0000 7fe59d8c8700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 0x7fe57c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.355+0000 7fe59d8c8700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 0x7fe57c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe594009880 tx=0x7fe594018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:25.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.359+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe588051730 con 0x7fe59806c0a0 2023-10-08T16:05:25.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.589+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe59806c9d0 con 0x7fe59806c0a0 2023-10-08T16:05:25.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.589+0000 7fe58dffb700 1 -- 172.21.15.170:0/1363311498 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe588054d50 con 0x7fe59806c0a0 2023-10-08T16:05:25.591 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:05:25.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 msgr2=0x7fe57c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 0x7fe57c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe594009880 tx=0x7fe594018000 comp rx=0 tx=0).stop 2023-10-08T16:05:25.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 msgr2=0x7fe5980b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:25.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe58800c830 tx=0x7fe588008040 comp rx=0 tx=0).stop 2023-10-08T16:05:25.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 shutdown_connections 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5980c8e00 0x7fe5980ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe59806bcd0 0x7fe59815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe57c066f30 0x7fe57c0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 --2- 172.21.15.170:0/1363311498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe59806c0a0 0x7fe5980b0050 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.591+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 >> 172.21.15.170:0/1363311498 conn(0x7fe5980ce600 msgr2=0x7fe5980c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.592+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 shutdown_connections 2023-10-08T16:05:25.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:25.592+0000 7fe59fb2c700 1 -- 172.21.15.170:0/1363311498 wait complete. 2023-10-08T16:05:25.605 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:05:25.605 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:05:25.605 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:05:25.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:05:25.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:25.606 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:05:25.606 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:05:25.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:05:25.606 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.607 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:05:25.622 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:05:25.622 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:05:25.622 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:25.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42425sec preferred_lft 42425sec 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:25.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.626 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:05:25.686 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:05:25.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.690 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:05:25.746 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:05:25.746 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:05:25.746 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:05:25.746 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:05:25.746 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:05:25.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:05:25.747 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:05:25.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.846 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:05:25.860 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:05:25.860 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:05:25.860 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:25.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:25.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:05:25.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:25.924 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:25.979 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:25.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:25.979 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:26.033 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:05:26.033 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:05:26.033 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:26.113 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:26.113 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:26.132 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:26.133 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:26.133 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:05:26.155 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:05:26.156 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:05:26.157 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:05:26.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.157 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:26.222 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:05:26.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.374+0000 7fa5f799b700 1 Processor -- start 2023-10-08T16:05:26.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 -- start start 2023-10-08T16:05:26.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 0x7fa5f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa5f015db10 0x7fa5f0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f015a160 con 0x7fa5f0159890 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f015df50 con 0x7fa5f015db10 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.375+0000 7fa5f799b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f00d13a0 con 0x7fa5f0159c60 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa5f015db10 0x7fa5f0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5737700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa5f0159890 0x7fa5f015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54636/0 (socket says 172.21.15.170:54636) 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5737700 1 -- 172.21.15.170:0/1829234471 learned_addr learned my addr 172.21.15.170:0/1829234471 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f4f36700 1 --2- 172.21.15.170:0/1829234471 >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 0x7fa5f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 -- 172.21.15.170:0/1829234471 >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 msgr2=0x7fa5f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 --2- 172.21.15.170:0/1829234471 >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 0x7fa5f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 -- 172.21.15.170:0/1829234471 >> v1:172.21.15.170:6789/0 conn(0x7fa5f0159890 legacy=0x7fa5f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5e7fff700 1 -- 172.21.15.170:0/1829234471 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429882832 0 0) 0x7fa5f015a160 con 0x7fa5f0159890 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 -- 172.21.15.170:0/1829234471 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f015e140 con 0x7fa5f015db10 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f4f36700 1 --2- 172.21.15.170:0/1829234471 >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 0x7fa5f015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5f5f38700 1 --2- 172.21.15.170:0/1829234471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f015db10 0x7fa5f0069560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00b67f0 tx=0x7fa5e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756bbd6913133c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5e7fff700 1 -- 172.21.15.170:0/1829234471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800b6b0 con 0x7fa5f015db10 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5e7fff700 1 -- 172.21.15.170:0/1829234471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e800b810 con 0x7fa5f015db10 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.376+0000 7fa5e7fff700 1 -- 172.21.15.170:0/1829234471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e800bbb0 con 0x7fa5f015db10 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 -- 172.21.15.170:0/1829234471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f015db10 msgr2=0x7fa5f0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 --2- 172.21.15.170:0/1829234471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f015db10 0x7fa5f0069560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00b67f0 tx=0x7fa5e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 -- 172.21.15.170:0/1829234471 shutdown_connections 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 --2- 172.21.15.170:0/1829234471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f015db10 0x7fa5f0069560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 --2- 172.21.15.170:0/1829234471 >> v2:172.21.15.191:3300/0 conn(0x7fa5f0159c60 0x7fa5f015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 -- 172.21.15.170:0/1829234471 >> 172.21.15.170:0/1829234471 conn(0x7fa5f0068e20 msgr2=0x7fa5f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:26.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 -- 172.21.15.170:0/1829234471 shutdown_connections 2023-10-08T16:05:26.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.377+0000 7fa5f799b700 1 -- 172.21.15.170:0/1829234471 wait complete. 2023-10-08T16:05:26.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.378+0000 7fa5f799b700 1 Processor -- start 2023-10-08T16:05:26.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.378+0000 7fa5f799b700 1 -- start start 2023-10-08T16:05:26.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 0x7fa5f00cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5f015db10 0x7fa5f0151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00cd8e0 con 0x7fa5f0159c60 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00cff00 con 0x7fa5f0159890 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f799b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f00cf630 con 0x7fa5f015db10 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47148/0 (socket says 172.21.15.170:47148) 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 -- 172.21.15.170:0/160225565 learned_addr learned my addr 172.21.15.170:0/160225565 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f5737700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 0x7fa5f00cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f5f38700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5f015db10 0x7fa5f0151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 -- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5f015db10 msgr2=0x7fa5f0151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5f015db10 0x7fa5f0151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 -- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 msgr2=0x7fa5f00cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 0x7fa5f00cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 -- 172.21.15.170:0/160225565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009420 con 0x7fa5f0159c60 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f5737700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 0x7fa5f00cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5f4f36700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000e930 tx=0x7fa5e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.379+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000eed0 con 0x7fa5f0159c60 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.380+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e000c5e0 con 0x7fa5f0159c60 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.380+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f00b2800 con 0x7fa5f0159c60 2023-10-08T16:05:26.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.380+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000c980 con 0x7fa5f0159c60 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.380+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f00b1be0 con 0x7fa5f0159c60 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.380+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f00cd0c0 con 0x7fa5f0159c60 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.381+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5e000cd70 con 0x7fa5f0159c60 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.381+0000 7fa5e57fa700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 0x7fa5d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.381+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5e00843e0 con 0x7fa5f0159c60 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.381+0000 7fa5f5737700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 0x7fa5d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:26.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.382+0000 7fa5f5737700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 0x7fa5d40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec001560 tx=0x7fa5ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:26.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.386+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5f00cd0c0 con 0x7fa5f0159c60 2023-10-08T16:05:26.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.617+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa5f00cd0c0 con 0x7fa5f0159c60 2023-10-08T16:05:26.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.617+0000 7fa5e57fa700 1 -- 172.21.15.170:0/160225565 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa5f00cd0c0 con 0x7fa5f0159c60 2023-10-08T16:05:26.619 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 msgr2=0x7fa5d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 0x7fa5d40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec001560 tx=0x7fa5ec009040 comp rx=0 tx=0).stop 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 msgr2=0x7fa5f00b22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa5e000e930 tx=0x7fa5e0008040 comp rx=0 tx=0).stop 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 shutdown_connections 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5f015db10 0x7fa5f0151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5f0159890 0x7fa5f00cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa5d4066f30 0x7fa5d40693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 --2- 172.21.15.170:0/160225565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5f0159c60 0x7fa5f00b22c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:26.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 >> 172.21.15.170:0/160225565 conn(0x7fa5f0068e20 msgr2=0x7fa5f0149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:26.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 shutdown_connections 2023-10-08T16:05:26.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:26.619+0000 7fa5f799b700 1 -- 172.21.15.170:0/160225565 wait complete. 2023-10-08T16:05:26.632 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:05:26.632 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:05:26.632 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:05:26.633 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:26.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.633 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.653 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:26.654 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42425sec preferred_lft 42425sec 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:05:26.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:26.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.656 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:05:26.720 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:05:26.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.723 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:05:26.781 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:05:26.781 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:05:26.782 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:05:26.782 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:05:26.782 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:05:26.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.782 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:05:26.782 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:05:26.782 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:05:26.783 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:05:26.783 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:05:26.783 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:05:26.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.888 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:05:26.903 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:05:26.903 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:05:26.903 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:26.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:26.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:05:26.968 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:26.968 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:27.023 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:27.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:27.023 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:27.077 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:05:27.077 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:05:27.078 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:27.158 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:27.158 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:27.177 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:27.198 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:05:27.199 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:05:27.201 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:05:27.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:27.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:27.265 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:05:27.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.383+0000 7f9454c5f700 1 Processor -- start 2023-10-08T16:05:27.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.383+0000 7f9454c5f700 1 -- start start 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f94500b03c0 0x7f94500cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b0790 0x7f94500b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f94501591e0 0x7f9450159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cfa00 con 0x7f94500b03c0 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cb580 con 0x7f94501591e0 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f9454c5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b2de0 con 0x7f94500b0790 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f94500b03c0 0x7f94500cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f94500b03c0 0x7f94500cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47158/0 (socket says 172.21.15.170:47158) 2023-10-08T16:05:27.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 -- 172.21.15.170:0/2804846533 learned_addr learned my addr 172.21.15.170:0/2804846533 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944ffff700 1 --2- 172.21.15.170:0/2804846533 >> v2:172.21.15.191:3300/0 conn(0x7f94501591e0 0x7f9450159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944effd700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b0790 0x7f94500b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 -- 172.21.15.170:0/2804846533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b0790 msgr2=0x7f94500b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b0790 0x7f94500b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 -- 172.21.15.170:0/2804846533 >> v2:172.21.15.191:3300/0 conn(0x7f94501591e0 msgr2=0x7f9450159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 --2- 172.21.15.170:0/2804846533 >> v2:172.21.15.191:3300/0 conn(0x7f94501591e0 0x7f9450159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 -- 172.21.15.170:0/2804846533 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500cf160 con 0x7f94500b03c0 2023-10-08T16:05:27.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944f7fe700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b03c0 0x7f94500cea70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f94500cb4f0 tx=0x7f9440009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff6c6a804fe08a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944e7fc700 1 -- 172.21.15.170:0/2804846533 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944000e070 con 0x7f94500b03c0 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944e7fc700 1 -- 172.21.15.170:0/2804846533 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944000b970 con 0x7f94500b03c0 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.384+0000 7f944e7fc700 1 -- 172.21.15.170:0/2804846533 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400092e0 con 0x7f94500b03c0 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 -- 172.21.15.170:0/2804846533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b03c0 msgr2=0x7f94500cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b03c0 0x7f94500cea70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f94500cb4f0 tx=0x7f9440009840 comp rx=0 tx=0).stop 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 -- 172.21.15.170:0/2804846533 shutdown_connections 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 --2- 172.21.15.170:0/2804846533 >> v2:172.21.15.191:3300/0 conn(0x7f94501591e0 0x7f9450159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b03c0 0x7f94500cea70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 --2- 172.21.15.170:0/2804846533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b0790 0x7f94500b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 -- 172.21.15.170:0/2804846533 >> 172.21.15.170:0/2804846533 conn(0x7f9450063230 msgr2=0x7f9450063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:27.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 -- 172.21.15.170:0/2804846533 shutdown_connections 2023-10-08T16:05:27.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.385+0000 7f9454c5f700 1 -- 172.21.15.170:0/2804846533 wait complete. 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.386+0000 7f9454c5f700 1 Processor -- start 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.386+0000 7f9454c5f700 1 -- start start 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b03c0 0x7f94500c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94501591e0 0x7f9450159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015c810 con 0x7f94500b0790 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015b960 con 0x7f94501591e0 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f9454c5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945015bc00 con 0x7f94500b03c0 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47170/0 (socket says 172.21.15.170:47170) 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 -- 172.21.15.170:0/542595269 learned_addr learned my addr 172.21.15.170:0/542595269 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944f7fe700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b03c0 0x7f94500c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944ffff700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94501591e0 0x7f9450159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 -- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b03c0 msgr2=0x7f94500c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b03c0 0x7f94500c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 -- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94501591e0 msgr2=0x7f9450159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:27.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94501591e0 0x7f9450159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.387+0000 7f944effd700 1 -- 172.21.15.170:0/542595269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009ae0 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f944effd700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f944400ec80 tx=0x7f94440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400c850 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400c9b0 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945015bee0 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400cd50 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.388+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945015c4a0 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.389+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945015c980 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.389+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9444013680 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.389+0000 7f94377fe700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.390+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9444084760 con 0x7f94500b0790 2023-10-08T16:05:27.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.390+0000 7f944f7fe700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:27.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.390+0000 7f944f7fe700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 0x7f943c0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9440009880 tx=0x7f9440018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:27.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.394+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94440519d0 con 0x7f94500b0790 2023-10-08T16:05:27.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:27.621+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f945004c3c0 con 0x7f94500b0790 2023-10-08T16:05:28.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.454+0000 7f94377fe700 1 -- 172.21.15.170:0/542595269 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7f9444054ff0 con 0x7f94500b0790 2023-10-08T16:05:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 msgr2=0x7f943c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 0x7f943c0693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9440009880 tx=0x7f9440018000 comp rx=0 tx=0).stop 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 msgr2=0x7f94500c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f944400ec80 tx=0x7f94440091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 shutdown_connections 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94501591e0 0x7f9450159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f94500b03c0 0x7f94500c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c066f30 0x7f943c0693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 --2- 172.21.15.170:0/542595269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94500b0790 0x7f94500c4e80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 >> 172.21.15.170:0/542595269 conn(0x7f9450063230 msgr2=0x7f94500b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 shutdown_connections 2023-10-08T16:05:28.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.457+0000 7f9454c5f700 1 -- 172.21.15.170:0/542595269 wait complete. 2023-10-08T16:05:28.474 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:05:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.593+0000 7fd51b8ae700 1 Processor -- start 2023-10-08T16:05:28.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.593+0000 7fd51b8ae700 1 -- start start 2023-10-08T16:05:28.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51b8ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd514159c60 0x7fd51415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51b8ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd514069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51b8ae700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51415a160 con 0x7fd514159890 2023-10-08T16:05:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51b8ae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51415df50 con 0x7fd514159c60 2023-10-08T16:05:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51b8ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140d13a0 con 0x7fd51415db10 2023-10-08T16:05:28.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51964a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd514159890 0x7fd51415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54640/0 (socket says 172.21.15.170:54640) 2023-10-08T16:05:28.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.594+0000 7fd51964a700 1 -- 172.21.15.170:0/1325174388 learned_addr learned my addr 172.21.15.170:0/1325174388 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:28.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 --2- 172.21.15.170:0/1325174388 >> v2:172.21.15.191:3300/0 conn(0x7fd514159c60 0x7fd51415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd519e4b700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd514069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664282602 0 0) 0x7fd51415a160 con 0x7fd514159890 2023-10-08T16:05:28.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd500003570 con 0x7fd514159890 2023-10-08T16:05:28.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2591163006 0 0) 0x7fd500003570 con 0x7fd514159890 2023-10-08T16:05:28.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 msgr2=0x7fd514069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:28.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd514069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 >> v2:172.21.15.191:3300/0 conn(0x7fd514159c60 msgr2=0x7fd51415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 --2- 172.21.15.170:0/1325174388 >> v2:172.21.15.191:3300/0 conn(0x7fd514159c60 0x7fd51415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51415e140 con 0x7fd514159890 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fd5100017f0 con 0x7fd514159890 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 >> v1:172.21.15.170:6789/0 conn(0x7fd514159890 legacy=0x7fd51415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd500003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd500003fe0 0x7fd500004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd500004d50 0x7fd5000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140d13a0 con 0x7fd500003fe0 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50000ba60 con 0x7fd500004d50 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50000bbd0 con 0x7fd51415db10 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd500003fe0 0x7fd500004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd51964a700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd500003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd519e4b700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd500004d50 0x7fd5000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 -- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 msgr2=0x7fd500003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd51415db10 0x7fd500003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 -- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd500004d50 msgr2=0x7fd5000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd500004d50 0x7fd5000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd518e49700 1 -- 172.21.15.170:0/1325174388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500003570 con 0x7fd500003fe0 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.595+0000 7fd519e4b700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd500004d50 0x7fd5000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd518e49700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd500003fe0 0x7fd500004810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd504009c00 tx=0x7fd5040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fc29ecfa5ab976 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400e6b0 con 0x7fd500003fe0 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400e810 con 0x7fd500003fe0 2023-10-08T16:05:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd50bfff700 1 -- 172.21.15.170:0/1325174388 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400ebd0 con 0x7fd500003fe0 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 -- 172.21.15.170:0/1325174388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd500003fe0 msgr2=0x7fd500004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/1325174388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd500003fe0 0x7fd500004810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd504009c00 tx=0x7fd5040091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd519e4b700 1 -- 172.21.15.170:0/1325174388 reap_dead start 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 -- 172.21.15.170:0/1325174388 shutdown_connections 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 -- 172.21.15.170:0/1325174388 >> 172.21.15.170:0/1325174388 conn(0x7fd514068e20 msgr2=0x7fd5140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 -- 172.21.15.170:0/1325174388 shutdown_connections 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.596+0000 7fd51b8ae700 1 -- 172.21.15.170:0/1325174388 wait complete. 2023-10-08T16:05:28.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.597+0000 7fd51b8ae700 1 Processor -- start 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.597+0000 7fd51b8ae700 1 -- start start 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd500003fe0 0x7fd514152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 0x7fd5140cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140c8e40 con 0x7fd5140b5380 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140b58f0 con 0x7fd5140cd600 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51b8ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140b5bd0 con 0x7fd500003fe0 2023-10-08T16:05:28.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd518e49700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd518e49700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47196/0 (socket says 172.21.15.170:47196) 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd518e49700 1 -- 172.21.15.170:0/2405415157 learned_addr learned my addr 172.21.15.170:0/2405415157 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd51964a700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd500003fe0 0x7fd514152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.598+0000 7fd519e4b700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 0x7fd5140cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 -- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd500003fe0 msgr2=0x7fd514152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd500003fe0 0x7fd514152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 -- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 msgr2=0x7fd5140cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 0x7fd5140cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 -- 172.21.15.170:0/2405415157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5140b5eb0 con 0x7fd5140b5380 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd519e4b700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 0x7fd5140cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd518e49700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd500004730 tx=0x7fd50401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504022070 con 0x7fd5140b5380 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400e980 con 0x7fd5140b5380 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504027470 con 0x7fd5140b5380 2023-10-08T16:05:28.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5140b6190 con 0x7fd5140b5380 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.599+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5140b3870 con 0x7fd5140b5380 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.600+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5140c8fd0 con 0x7fd5140b5380 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.600+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd50401c450 con 0x7fd5140b5380 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.601+0000 7fd5097fa700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 0x7fd4f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.601+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd504086040 con 0x7fd5140b5380 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.601+0000 7fd51964a700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 0x7fd4f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.601+0000 7fd51964a700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 0x7fd4f80693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5100019a0 tx=0x7fd510006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.605+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5140c8fd0 con 0x7fd5140b5380 2023-10-08T16:05:28.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.812+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5140c8fd0 con 0x7fd4f8066f30 2023-10-08T16:05:28.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.813+0000 7fd5097fa700 1 -- 172.21.15.170:0/2405415157 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd5140c8fd0 con 0x7fd4f8066f30 2023-10-08T16:05:28.815 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:05:28.815 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:28.815 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:05:28.815 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:28.815 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 msgr2=0x7fd4f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 0x7fd4f80693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5100019a0 tx=0x7fd510006040 comp rx=0 tx=0).stop 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 msgr2=0x7fd5140cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd500004730 tx=0x7fd50401a000 comp rx=0 tx=0).stop 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 shutdown_connections 2023-10-08T16:05:28.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5140cd600 0x7fd5140cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd500003fe0 0x7fd514152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4f8066f30 0x7fd4f80693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 --2- 172.21.15.170:0/2405415157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5140b5380 0x7fd5140cd0c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 >> 172.21.15.170:0/2405415157 conn(0x7fd514068e20 msgr2=0x7fd514152610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 shutdown_connections 2023-10-08T16:05:28.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.815+0000 7fd51b8ae700 1 -- 172.21.15.170:0/2405415157 wait complete. 2023-10-08T16:05:28.828 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:05:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.944+0000 7fb647fff700 1 Processor -- start 2023-10-08T16:05:28.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.944+0000 7fb647fff700 1 -- start start 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb647fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb6480cc600 0x7fb6480b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb647fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb6480cc9d0 0x7fb64815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb647fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb648150b50 con 0x7fb6480cd0f0 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb647fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6480cede0 con 0x7fb6480cc9d0 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb647fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6480b2260 con 0x7fb6480cc600 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb6480cc9d0 0x7fb64815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb6480cc9d0 0x7fb64815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47204/0 (socket says 172.21.15.170:47204) 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 -- 172.21.15.170:0/684215716 learned_addr learned my addr 172.21.15.170:0/684215716 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb646ffd700 1 --2- 172.21.15.170:0/684215716 >> v2:172.21.15.191:3300/0 conn(0x7fb6480cc600 0x7fb6480b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 -- 172.21.15.170:0/684215716 >> v2:172.21.15.191:3300/0 conn(0x7fb6480cc600 msgr2=0x7fb6480b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 --2- 172.21.15.170:0/684215716 >> v2:172.21.15.191:3300/0 conn(0x7fb6480cc600 0x7fb6480b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 -- 172.21.15.170:0/684215716 >> v1:172.21.15.191:6789/0 conn(0x7fb6480cd0f0 legacy=0x7fb648159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.945+0000 7fb63e7fc700 1 -- 172.21.15.170:0/684215716 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6480b2540 con 0x7fb6480cc9d0 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb63e7fc700 1 --2- 172.21.15.170:0/684215716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cc9d0 0x7fb64815d280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6480891a0 tx=0x7fb630009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e84722718cf8a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb6467fc700 1 -- 172.21.15.170:0/684215716 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000b860 con 0x7fb6480cc9d0 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb6467fc700 1 -- 172.21.15.170:0/684215716 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63000b9c0 con 0x7fb6480cc9d0 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb6467fc700 1 -- 172.21.15.170:0/684215716 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63000bd60 con 0x7fb6480cc9d0 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 -- 172.21.15.170:0/684215716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cc9d0 msgr2=0x7fb64815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 --2- 172.21.15.170:0/684215716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cc9d0 0x7fb64815d280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb6480891a0 tx=0x7fb630009250 comp rx=0 tx=0).stop 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 -- 172.21.15.170:0/684215716 shutdown_connections 2023-10-08T16:05:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 --2- 172.21.15.170:0/684215716 >> v2:172.21.15.191:3300/0 conn(0x7fb6480cc600 0x7fb6480b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 --2- 172.21.15.170:0/684215716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cc9d0 0x7fb64815d280 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 -- 172.21.15.170:0/684215716 >> 172.21.15.170:0/684215716 conn(0x7fb6480d0630 msgr2=0x7fb6480d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 -- 172.21.15.170:0/684215716 shutdown_connections 2023-10-08T16:05:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.946+0000 7fb647fff700 1 -- 172.21.15.170:0/684215716 wait complete. 2023-10-08T16:05:28.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.947+0000 7fb647fff700 1 Processor -- start 2023-10-08T16:05:28.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 -- start start 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6480cc600 0x7fb6481539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6480cc9d0 0x7fb648068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64816df10 con 0x7fb6480cd0f0 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64816e1f0 con 0x7fb6480cc600 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb647fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb648152f20 con 0x7fb6480cc9d0 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb6477fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47212/0 (socket says 172.21.15.170:47212) 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb6477fe700 1 -- 172.21.15.170:0/488395356 learned_addr learned my addr 172.21.15.170:0/488395356 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:28.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb646ffd700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6480cc600 0x7fb6481539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.948+0000 7fb63e7fc700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6480cc9d0 0x7fb648068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 -- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6480cc9d0 msgr2=0x7fb648068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6480cc9d0 0x7fb648068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 -- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6480cc600 msgr2=0x7fb6481539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6480cc600 0x7fb6481539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 -- 172.21.15.170:0/488395356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300094f0 con 0x7fb6480cd0f0 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb6477fe700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c8e0 tx=0x7fb640001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640012630 con 0x7fb6480cd0f0 2023-10-08T16:05:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb640012790 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb648153200 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb640012b50 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.949+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64814eaa0 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.950+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb648153850 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.950+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb64001a3a0 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.951+0000 7fb63f7fe700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 0x7fb6280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.951+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb640084740 con 0x7fb6480cd0f0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.951+0000 7fb646ffd700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 0x7fb6280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.951+0000 7fb646ffd700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 0x7fb6280693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb638006fd0 tx=0x7fb638008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:28.955+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6400519b0 con 0x7fb6480cd0f0 2023-10-08T16:05:29.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.137+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb64804bb80 con 0x7fb6480cd0f0 2023-10-08T16:05:29.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.137+0000 7fb63f7fe700 1 -- 172.21.15.170:0/488395356 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb640054fd0 con 0x7fb6480cd0f0 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 msgr2=0x7fb6280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 0x7fb6280693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb638006fd0 tx=0x7fb638008040 comp rx=0 tx=0).stop 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 msgr2=0x7fb64816d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb64000c8e0 tx=0x7fb640001710 comp rx=0 tx=0).stop 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 shutdown_connections 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628066f30 0x7fb6280693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb6480cd0f0 0x7fb64816d470 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb6480cc600 0x7fb6481539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 --2- 172.21.15.170:0/488395356 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb6480cc9d0 0x7fb648068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 >> 172.21.15.170:0/488395356 conn(0x7fb6480d0630 msgr2=0x7fb64814f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 shutdown_connections 2023-10-08T16:05:29.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.139+0000 7fb647fff700 1 -- 172.21.15.170:0/488395356 wait complete. 2023-10-08T16:05:29.153 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000160911 2023-10-08T16:05:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.268+0000 7f00af27a700 1 Processor -- start 2023-10-08T16:05:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.268+0000 7f00af27a700 1 -- start start 2023-10-08T16:05:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00af27a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 0x7f00a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00af27a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a815a160 con 0x7f00a8069560 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00af27a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a80d13a0 con 0x7f00a8159c60 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00af27a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a815e140 con 0x7f00a8159890 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00ad817700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f00a8069560 0x7f00a80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54648/0 (socket says 172.21.15.170:54648) 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.269+0000 7f00ad817700 1 -- 172.21.15.170:0/1061736295 learned_addr learned my addr 172.21.15.170:0/1061736295 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f00ad016700 1 --2- 172.21.15.170:0/1061736295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 0x7f00a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761054289 0 0) 0x7f00a815a160 con 0x7f00a8069560 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0094003570 con 0x7f00a8069560 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641903062 0 0) 0x7f00a80d13a0 con 0x7f00a8159c60 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a815a160 con 0x7f00a8159c60 2023-10-08T16:05:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2386049646 0 0) 0x7f0094003570 con 0x7f00a8069560 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f00ad016700 1 -- 172.21.15.170:0/1061736295 >> v1:172.21.15.191:6789/0 conn(0x7f00a8159c60 legacy=0x7f00a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f00ad016700 1 -- 172.21.15.170:0/1061736295 >> v1:172.21.15.170:6789/0 conn(0x7f00a8069560 legacy=0x7f00a80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f00ad016700 1 -- 172.21.15.170:0/1061736295 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a815a3d0 con 0x7f00a8159890 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f00ad016700 1 --2- 172.21.15.170:0/1061736295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 0x7f00a815abd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009e90 tx=0x7f00a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8932e0fb31d7d1d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400e070 con 0x7f00a8159890 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a400b870 con 0x7f00a8159890 2023-10-08T16:05:29.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.270+0000 7f009ffff700 1 -- 172.21.15.170:0/1061736295 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0001470 con 0x7f00a8159890 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 -- 172.21.15.170:0/1061736295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 msgr2=0x7f00a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 --2- 172.21.15.170:0/1061736295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 0x7f00a815abd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009e90 tx=0x7f00a4009780 comp rx=0 tx=0).stop 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 -- 172.21.15.170:0/1061736295 shutdown_connections 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 --2- 172.21.15.170:0/1061736295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8159890 0x7f00a815abd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 -- 172.21.15.170:0/1061736295 >> 172.21.15.170:0/1061736295 conn(0x7f00a8068e20 msgr2=0x7f00a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 -- 172.21.15.170:0/1061736295 shutdown_connections 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.271+0000 7f00af27a700 1 -- 172.21.15.170:0/1061736295 wait complete. 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.272+0000 7f00af27a700 1 Processor -- start 2023-10-08T16:05:29.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.272+0000 7f00af27a700 1 -- start start 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8069560 0x7f00a80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 0x7f00a80ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80b62e0 con 0x7f00a8159890 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80cd180 con 0x7f00a8159c60 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00af27a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a816d9e0 con 0x7f00a8069560 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47214/0 (socket says 172.21.15.170:47214) 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 -- 172.21.15.170:0/2105389955 learned_addr learned my addr 172.21.15.170:0/2105389955 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ad016700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8069560 0x7f00a80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:29.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ad817700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 0x7f00a80ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 -- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8069560 msgr2=0x7f00a80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8069560 0x7f00a80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 -- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 msgr2=0x7f00a80ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 0x7f00a80ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 -- 172.21.15.170:0/2105389955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a4009a20 con 0x7f00a8159890 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ad817700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 0x7f00a80ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.273+0000 7f00ac815700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f009800c590 tx=0x7f0098007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.274+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800f540 con 0x7f00a8159890 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.274+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800f6c0 con 0x7f00a8159890 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.274+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a816dcc0 con 0x7f00a8159890 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.274+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800fa80 con 0x7f00a8159890 2023-10-08T16:05:29.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.274+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a80cf900 con 0x7f00a8159890 2023-10-08T16:05:29.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.275+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a80cd330 con 0x7f00a8159890 2023-10-08T16:05:29.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.275+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f009801b3b0 con 0x7f00a8159890 2023-10-08T16:05:29.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.275+0000 7f009d7fa700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 0x7f008c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:29.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.275+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0098083d40 con 0x7f00a8159890 2023-10-08T16:05:29.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.276+0000 7f00ad016700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 0x7f008c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:29.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.276+0000 7f00ad016700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 0x7f008c0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097c0 tx=0x7f00a4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:29.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.280+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00a80cd330 con 0x7f00a8159890 2023-10-08T16:05:29.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:29.491+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000160911", "target": ["mon-mgr", ""]}) v1 -- 0x7f00a80cd2f0 con 0x7f008c066f30 2023-10-08T16:05:32.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.479+0000 7f009d7fa700 1 -- 172.21.15.170:0/2105389955 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f00a80cd2f0 con 0x7f008c066f30 2023-10-08T16:05:32.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.484+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 msgr2=0x7f008c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.484+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 0x7f008c0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00a40097c0 tx=0x7f00a4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:32.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.484+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 msgr2=0x7f00a80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.484+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f009800c590 tx=0x7f0098007de0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 shutdown_connections 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a8159c60 0x7f00a80ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a8069560 0x7f00a80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f008c066f30 0x7f008c0693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 --2- 172.21.15.170:0/2105389955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00a8159890 0x7f00a80cf3c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.486+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 >> 172.21.15.170:0/2105389955 conn(0x7f00a8068e20 msgr2=0x7f00a80b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.487+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 shutdown_connections 2023-10-08T16:05:32.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.487+0000 7f00af27a700 1 -- 172.21.15.170:0/2105389955 wait complete. 2023-10-08T16:05:32.489 INFO:teuthology.orchestra.run.smithi170.stderr:Volume created successfully (no MDS daemons created) 2023-10-08T16:05:32.504 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:05:32.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.623+0000 7f68d27bc700 1 Processor -- start 2023-10-08T16:05:32.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.623+0000 7f68d27bc700 1 -- start start 2023-10-08T16:05:32.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d27bc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d27bc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f68cc069560 0x7f68cc0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d27bc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc15a160 con 0x7f68cc159c60 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d27bc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc069f20 con 0x7f68cc069560 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d27bc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0d13a0 con 0x7f68cc159890 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f68cc069560 0x7f68cc0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68cb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f68cc159c60 0x7f68cc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54662/0 (socket says 172.21.15.170:54662) 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68cb7fe700 1 -- 172.21.15.170:0/184744363 learned_addr learned my addr 172.21.15.170:0/184744363 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68cbfff700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 -- 172.21.15.170:0/184744363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 msgr2=0x7f68cc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 -- 172.21.15.170:0/184744363 >> v1:172.21.15.170:6789/0 conn(0x7f68cc159c60 legacy=0x7f68cc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68caffd700 1 -- 172.21.15.170:0/184744363 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321334349 0 0) 0x7f68cc15a160 con 0x7f68cc159c60 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 -- 172.21.15.170:0/184744363 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc15e140 con 0x7f68cc069560 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68d0d59700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc069560 0x7f68cc0699e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009e60 tx=0x7f68c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc599faa5233fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68caffd700 1 -- 172.21.15.170:0/184744363 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400e070 con 0x7f68cc069560 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68caffd700 1 -- 172.21.15.170:0/184744363 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400bc30 con 0x7f68cc069560 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.624+0000 7f68caffd700 1 -- 172.21.15.170:0/184744363 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4012430 con 0x7f68cc069560 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 -- 172.21.15.170:0/184744363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc069560 msgr2=0x7f68cc0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc069560 0x7f68cc0699e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009e60 tx=0x7f68c4009780 comp rx=0 tx=0).stop 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 -- 172.21.15.170:0/184744363 shutdown_connections 2023-10-08T16:05:32.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 --2- 172.21.15.170:0/184744363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc069560 0x7f68cc0699e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 -- 172.21.15.170:0/184744363 >> 172.21.15.170:0/184744363 conn(0x7f68cc068e20 msgr2=0x7f68cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:32.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 -- 172.21.15.170:0/184744363 shutdown_connections 2023-10-08T16:05:32.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.625+0000 7f68d27bc700 1 -- 172.21.15.170:0/184744363 wait complete. 2023-10-08T16:05:32.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.626+0000 7f68d27bc700 1 Processor -- start 2023-10-08T16:05:32.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.626+0000 7f68d27bc700 1 -- start start 2023-10-08T16:05:32.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68cc069560 0x7f68cc15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc155770 con 0x7f68cc159c60 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc15efc0 con 0x7f68cc069560 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d27bc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc154cc0 con 0x7f68cc159890 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47226/0 (socket says 172.21.15.170:47226) 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 -- 172.21.15.170:0/2373961546 learned_addr learned my addr 172.21.15.170:0/2373961546 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68cb7fe700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68cbfff700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68cc069560 0x7f68cc15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 -- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 msgr2=0x7f68cc0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 -- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68cc069560 msgr2=0x7f68cc15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68cc069560 0x7f68cc15d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 -- 172.21.15.170:0/2373961546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009a20 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68d0d59700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0899c0 tx=0x7f68c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.627+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4012e10 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.628+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c401d630 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.628+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc0b3fe0 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.628+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c401d9f0 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.628+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc15b8a0 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.629+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc15ed90 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.629+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68c4024070 con 0x7f68cc159c60 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.629+0000 7f68b3fff700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.629+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f68c4087210 con 0x7f68cc159c60 2023-10-08T16:05:32.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.629+0000 7f68cbfff700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.630+0000 7f68cbfff700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 0x7f68b80693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.634+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68cc15ed90 con 0x7f68cc159c60 2023-10-08T16:05:32.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.846+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f68cc15f170 con 0x7f68b8066f30 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.847+0000 7f68b3fff700 1 -- 172.21.15.170:0/2373961546 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+96 (secure 0 0 0) 0x7f68cc15f170 con 0x7f68b8066f30 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:32.849 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000160911" 2023-10-08T16:05:32.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:32.850 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 msgr2=0x7f68b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 0x7f68b80693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc008040 comp rx=0 tx=0).stop 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 msgr2=0x7f68cc154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0899c0 tx=0x7f68c4019000 comp rx=0 tx=0).stop 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 shutdown_connections 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68b8066f30 0x7f68b80693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68cc159c60 0x7f68cc154780 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68cc069560 0x7f68cc15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 --2- 172.21.15.170:0/2373961546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68cc159890 0x7f68cc0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.849+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 >> 172.21.15.170:0/2373961546 conn(0x7f68cc068e20 msgr2=0x7f68cc1580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:32.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.850+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 shutdown_connections 2023-10-08T16:05:32.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.850+0000 7f68d27bc700 1 -- 172.21.15.170:0/2373961546 wait complete. 2023-10-08T16:05:32.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool ls detail --format=json 2023-10-08T16:05:32.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.978+0000 7f2df8111700 1 Processor -- start 2023-10-08T16:05:32.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.978+0000 7f2df8111700 1 -- start start 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df8111700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2df0069560 0x7f2df00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df8111700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df0069ee0 con 0x7f2df0159890 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df8111700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df015a160 con 0x7f2df0159c60 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df8111700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df00d13a0 con 0x7f2df0069560 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df5ead700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2df0159890 0x7f2df015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54676/0 (socket says 172.21.15.170:54676) 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df5ead700 1 -- 172.21.15.170:0/2072625068 learned_addr learned my addr 172.21.15.170:0/2072625068 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686010447 0 0) 0x7f2df0069ee0 con 0x7f2df0159890 2023-10-08T16:05:32.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.979+0000 7f2df66ae700 1 --2- 172.21.15.170:0/2072625068 >> v2:172.21.15.191:3300/0 conn(0x7f2df0069560 0x7f2df00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ddc003570 con 0x7f2df0159890 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468962144 0 0) 0x7f2ddc003570 con 0x7f2df0159890 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 >> v2:172.21.15.191:3300/0 conn(0x7f2df0069560 msgr2=0x7f2df00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 --2- 172.21.15.170:0/2072625068 >> v2:172.21.15.191:3300/0 conn(0x7f2df0069560 0x7f2df00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 >> v1:172.21.15.191:6789/0 conn(0x7f2df0159c60 legacy=0x7f2df015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df015e140 con 0x7f2df0159890 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f2dec0017f0 con 0x7f2df0159890 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 >> v1:172.21.15.170:6789/0 conn(0x7f2df0159890 legacy=0x7f2df015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00d13a0 con 0x7f2ddc003a60 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc00c0d0 con 0x7f2ddc0051c0 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc00c290 con 0x7f2ddc004810 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df56ac700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df66ae700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 -- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc004810 msgr2=0x7f2ddc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc004810 0x7f2ddc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 -- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ddc0051c0 msgr2=0x7f2ddc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 -- 172.21.15.170:0/2072625068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc003570 con 0x7f2ddc003a60 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df66ae700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2ddc0051c0 0x7f2ddc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df66ae700 1 -- 172.21.15.170:0/2072625068 reap_dead start 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00a930 tx=0x7f2dec0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba68ee8494658c33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0070d0 con 0x7f2ddc003a60 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec007230 con 0x7f2ddc003a60 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.980+0000 7f2df4eab700 1 -- 172.21.15.170:0/2072625068 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0075d0 con 0x7f2ddc003a60 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 -- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 msgr2=0x7f2ddc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00a930 tx=0x7f2dec0021d0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 -- 172.21.15.170:0/2072625068 shutdown_connections 2023-10-08T16:05:32.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 --2- 172.21.15.170:0/2072625068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ddc003a60 0x7f2ddc003e70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 -- 172.21.15.170:0/2072625068 >> 172.21.15.170:0/2072625068 conn(0x7f2df0068e20 msgr2=0x7f2df00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:32.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 -- 172.21.15.170:0/2072625068 shutdown_connections 2023-10-08T16:05:32.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.981+0000 7f2df8111700 1 -- 172.21.15.170:0/2072625068 wait complete. 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.982+0000 7f2df8111700 1 Processor -- start 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.982+0000 7f2df8111700 1 -- start start 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc003a60 0x7f2df0147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df0169fe0 0x7f2df016a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00c84f0 con 0x7f2df00b3590 2023-10-08T16:05:32.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00c7ee0 con 0x7f2df0169fe0 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df8111700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df016ad10 con 0x7f2ddc003a60 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47244/0 (socket says 172.21.15.170:47244) 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 -- 172.21.15.170:0/2064940431 learned_addr learned my addr 172.21.15.170:0/2064940431 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc003a60 0x7f2df0147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df66ae700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df0169fe0 0x7f2df016a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 -- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc003a60 msgr2=0x7f2df0147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc003a60 0x7f2df0147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 -- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df0169fe0 msgr2=0x7f2df016a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:32.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df0169fe0 0x7f2df016a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 -- 172.21.15.170:0/2064940431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00c8180 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.983+0000 7f2df56ac700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2de0002950 tx=0x7f2de0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.984+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de000f8d0 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.984+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de000fa30 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.984+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df00b3f40 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.984+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de000fdd0 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.984+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df016a990 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.985+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df004c3c0 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.985+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2de0013600 con 0x7f2df00b3590 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.985+0000 7f2de5ffb700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:32.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.986+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:32.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.986+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f2de0084ea0 con 0x7f2df00b3590 2023-10-08T16:05:32.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.986+0000 7f2df5ead700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00afd0 tx=0x7f2dec016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:32.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:32.990+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2de0051cf0 con 0x7f2df00b3590 2023-10-08T16:05:33.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.173+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool ls", "detail": "detail", "format": "json"} v 0) v1 -- 0x7f2df015de80 con 0x7f2df00b3590 2023-10-08T16:05:33.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.174+0000 7f2de5ffb700 1 -- 172.21.15.170:0/2064940431 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool ls", "detail": "detail", "format": "json"}]=0 v35) v1 ==== 97+0+8050 (secure 0 0 0) 0x7f2de0055310 con 0x7f2df00b3590 2023-10-08T16:05:33.175 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:33.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 msgr2=0x7f2dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 0x7f2dd40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00afd0 tx=0x7f2dec016000 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 msgr2=0x7f2df00b3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2de0002950 tx=0x7f2de0007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 shutdown_connections 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df0169fe0 0x7f2df016a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2ddc003a60 0x7f2df0147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2dd4066f30 0x7f2dd40693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 --2- 172.21.15.170:0/2064940431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df00b3590 0x7f2df00b3a00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 >> 172.21.15.170:0/2064940431 conn(0x7f2df0068e20 msgr2=0x7f2df01586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 shutdown_connections 2023-10-08T16:05:33.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.176+0000 7f2df8111700 1 -- 172.21.15.170:0/2064940431 wait complete. 2023-10-08T16:05:33.189 INFO:teuthology.orchestra.run.smithi170.stdout:[{"pool_id":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool_id":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool_id":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool_id":6,"pool_name":"cephfs.volume_0000000000160911.meta","create_time":"2023-10-08T16:05:29.510880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000160911"}}},{"pool_id":7,"pool_name":"cephfs.volume_0000000000160911.data","create_time":"2023-10-08T16:05:30.485608+0000","flags":163841,"flags_names":"hashpspool,creating,bulk","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"volume_0000000000160911"}}}] 2023-10-08T16:05:33.190 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs get volume_0000000000160911 --format=json 2023-10-08T16:05:33.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.303+0000 7f146d87a700 1 Processor -- start 2023-10-08T16:05:33.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.303+0000 7f146d87a700 1 -- start start 2023-10-08T16:05:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f146d87a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f146815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f146d87a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1468069560 0x7f14680699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f146d87a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146815a160 con 0x7f1468159c60 2023-10-08T16:05:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f146d87a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1468069f20 con 0x7f1468069560 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f146d87a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14680d13a0 con 0x7f1468159890 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f145e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1468159c60 0x7f146815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:54686/0 (socket says 172.21.15.170:54686) 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.304+0000 7f145e7fc700 1 -- 172.21.15.170:0/494762828 learned_addr learned my addr 172.21.15.170:0/494762828 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f146815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14677fe700 1 --2- 172.21.15.170:0/494762828 >> v2:172.21.15.191:3300/0 conn(0x7f1468069560 0x7f14680699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001911800 0 0) 0x7f146815a160 con 0x7f1468159c60 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1454003570 con 0x7f1468159c60 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434343869 0 0) 0x7f1454003570 con 0x7f1468159c60 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 msgr2=0x7f146815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f146815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 >> v2:172.21.15.191:3300/0 conn(0x7f1468069560 msgr2=0x7f14680699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 --2- 172.21.15.170:0/494762828 >> v2:172.21.15.191:3300/0 conn(0x7f1468069560 0x7f14680699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146815e140 con 0x7f1468159c60 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f1450001470 con 0x7f1468159c60 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 >> v1:172.21.15.170:6789/0 conn(0x7f1468159c60 legacy=0x7f146815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1454003a60 0x7f1454003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f14540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1468069f20 con 0x7f1454003a60 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680d13a0 con 0x7f1454004810 2023-10-08T16:05:33.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145400ba60 con 0x7f1468159890 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1454003a60 0x7f1454003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f145e7fc700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f14677fe700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f14540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 -- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 msgr2=0x7f14540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468159890 0x7f14540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 -- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454004810 msgr2=0x7f1454004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454004810 0x7f1454004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.305+0000 7f1466ffd700 1 -- 172.21.15.170:0/494762828 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454003570 con 0x7f1454003a60 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f1466ffd700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1454003a60 0x7f1454003e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1458009c30 tx=0x7f14580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a266ec0c1c8d133e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145800e710 con 0x7f1454003a60 2023-10-08T16:05:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145800e870 con 0x7f1454003a60 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f14667fc700 1 -- 172.21.15.170:0/494762828 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145800ec30 con 0x7f1454003a60 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 -- 172.21.15.170:0/494762828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1454003a60 msgr2=0x7f1454003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 --2- 172.21.15.170:0/494762828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1454003a60 0x7f1454003e70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1458009c30 tx=0x7f14580091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f14677fe700 1 -- 172.21.15.170:0/494762828 reap_dead start 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 -- 172.21.15.170:0/494762828 shutdown_connections 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 -- 172.21.15.170:0/494762828 >> 172.21.15.170:0/494762828 conn(0x7f1468068e20 msgr2=0x7f14680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 -- 172.21.15.170:0/494762828 shutdown_connections 2023-10-08T16:05:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.306+0000 7f146d87a700 1 -- 172.21.15.170:0/494762828 wait complete. 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 Processor -- start 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 -- start start 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454003a60 0x7f14680cfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468169c80 0x7f146816a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680cdb90 con 0x7f14680b3fa0 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680cde30 con 0x7f1454003a60 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f146d87a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14680b3630 con 0x7f1468169c80 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f145e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f145e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47268/0 (socket says 172.21.15.170:47268) 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f145e7fc700 1 -- 172.21.15.170:0/511523350 learned_addr learned my addr 172.21.15.170:0/511523350 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.308+0000 7f1466ffd700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454003a60 0x7f14680cfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f14677fe700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468169c80 0x7f146816a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 -- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468169c80 msgr2=0x7f146816a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468169c80 0x7f146816a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 -- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454003a60 msgr2=0x7f14680cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454003a60 0x7f14680cfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 -- 172.21.15.170:0/511523350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680b3910 con 0x7f14680b3fa0 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145e7fc700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1450007510 tx=0x7f1450001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000f850 con 0x7f14680b3fa0 2023-10-08T16:05:33.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145000f9b0 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145000fd50 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14680b3bf0 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.309+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14680b4410 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.310+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14680cdfc0 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.310+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1450013600 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.311+0000 7f145f7fe700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.311+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6603+0+0 (secure 0 0 0) 0x7f1450084b40 con 0x7f14680b3fa0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.311+0000 7f1466ffd700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.311+0000 7f1466ffd700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 0x7f14480693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14580091e0 tx=0x7f145801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.316+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1450051a30 con 0x7f14680b3fa0 2023-10-08T16:05:33.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.550+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs get", "fs_name": "volume_0000000000160911", "format": "json"} v 0) v1 -- 0x7f14680b3de0 con 0x7f14680b3fa0 2023-10-08T16:05:33.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.551+0000 7f145f7fe700 1 -- 172.21.15.170:0/511523350 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs get", "fs_name": "volume_0000000000160911", "format": "json"}]=0 v20) v1 ==== 110+0+1847 (secure 0 0 0) 0x7f14680b3de0 con 0x7f14680b3fa0 2023-10-08T16:05:33.552 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:33.552 INFO:teuthology.orchestra.run.smithi170.stdout:{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:31.472912+0000","modified":"2023-10-08T16:05:33.431307+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4881":{"gid":4881,"name":"a","rank":0,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.170:6836/2746556501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2746556501},{"type":"v1","addr":"172.21.15.170:6836","nonce":2746556501}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"volume_0000000000160911","balancer":"","standby_count_wanted":1},"id":3} 2023-10-08T16:05:33.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.552+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 msgr2=0x7f14480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.552+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 0x7f14480693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14580091e0 tx=0x7f145801a000 comp rx=0 tx=0).stop 2023-10-08T16:05:33.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.552+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 msgr2=0x7f14681526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.552+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1450007510 tx=0x7f1450001e90 comp rx=0 tx=0).stop 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 shutdown_connections 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1468169c80 0x7f146816a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1454003a60 0x7f14680cfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1448066f30 0x7f14480693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 --2- 172.21.15.170:0/511523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14680b3fa0 0x7f14681526f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 >> 172.21.15.170:0/511523350 conn(0x7f1468068e20 msgr2=0x7f14680c5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 shutdown_connections 2023-10-08T16:05:33.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.553+0000 7f146d87a700 1 -- 172.21.15.170:0/511523350 wait complete. 2023-10-08T16:05:33.566 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000160911 --yes-i-really-mean-it 2023-10-08T16:05:33.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 Processor -- start 2023-10-08T16:05:33.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 -- start start 2023-10-08T16:05:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 0x7fea8006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 0x7fea80069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fea80069cb0 0x7fea80159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8006a130 con 0x7fea80069cb0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8006eb90 con 0x7fea80155eb0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea8770a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8006ee70 con 0x7fea800c70b0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.682+0000 7fea85ca7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fea80069cb0 0x7fea80159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fea80069cb0 0x7fea80159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47274/0 (socket says 172.21.15.170:47274) 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 -- 172.21.15.170:0/1248981720 learned_addr learned my addr 172.21.15.170:0/1248981720 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea854a6700 1 --2- 172.21.15.170:0/1248981720 >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 0x7fea8006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea84ca5700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 0x7fea80069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 -- 172.21.15.170:0/1248981720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 msgr2=0x7fea80069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 0x7fea80069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 -- 172.21.15.170:0/1248981720 >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 msgr2=0x7fea8006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 --2- 172.21.15.170:0/1248981720 >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 0x7fea8006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 -- 172.21.15.170:0/1248981720 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8006f150 con 0x7fea80069cb0 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea84ca5700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 0x7fea80069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea854a6700 1 --2- 172.21.15.170:0/1248981720 >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 0x7fea8006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea85ca7700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80069cb0 0x7fea80159a60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea80065160 tx=0x7fea7800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f992620a8c7f88ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea77fff700 1 -- 172.21.15.170:0/1248981720 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7800be60 con 0x7fea80069cb0 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea77fff700 1 -- 172.21.15.170:0/1248981720 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7800ddc0 con 0x7fea80069cb0 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.683+0000 7fea77fff700 1 -- 172.21.15.170:0/1248981720 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780196b0 con 0x7fea80069cb0 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 -- 172.21.15.170:0/1248981720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80069cb0 msgr2=0x7fea80159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80069cb0 0x7fea80159a60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea80065160 tx=0x7fea7800b4f0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 -- 172.21.15.170:0/1248981720 shutdown_connections 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80069cb0 0x7fea80159a60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 --2- 172.21.15.170:0/1248981720 >> v2:172.21.15.191:3300/0 conn(0x7fea80155eb0 0x7fea8006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 --2- 172.21.15.170:0/1248981720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea800c70b0 0x7fea80069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 -- 172.21.15.170:0/1248981720 >> 172.21.15.170:0/1248981720 conn(0x7fea80068e10 msgr2=0x7fea800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 -- 172.21.15.170:0/1248981720 shutdown_connections 2023-10-08T16:05:33.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.684+0000 7fea8770a700 1 -- 172.21.15.170:0/1248981720 wait complete. 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.685+0000 7fea8770a700 1 Processor -- start 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.685+0000 7fea8770a700 1 -- start start 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 0x7fea80151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea80155eb0 0x7fea800ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800c5810 con 0x7fea800c70b0 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea80151d70 con 0x7fea80155eb0 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800c4ee0 con 0x7fea80069cb0 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47276/0 (socket says 172.21.15.170:47276) 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 -- 172.21.15.170:0/2488542695 learned_addr learned my addr 172.21.15.170:0/2488542695 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea85ca7700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea80155eb0 0x7fea800ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea854a6700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 0x7fea80151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 -- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 msgr2=0x7fea80151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 0x7fea80151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 -- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea80155eb0 msgr2=0x7fea800ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea80155eb0 0x7fea800ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 -- 172.21.15.170:0/2488542695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7800b790 con 0x7fea800c70b0 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea854a6700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 0x7fea80151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:33.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea84ca5700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c880 tx=0x7fea70008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70011600 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea70011760 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea70011ae0 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea800c51c0 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.686+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800abdf0 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.687+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea800c8ca0 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.688+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea70010350 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.688+0000 7fea757fa700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 0x7fea640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.688+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6603+0+0 (secure 0 0 0) 0x7fea70084860 con 0x7fea800c70b0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.688+0000 7fea854a6700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 0x7fea640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:33.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.688+0000 7fea854a6700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 0x7fea640693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d4a0 tx=0x7fea7c0075d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:33.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.693+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea700551e0 con 0x7fea800c70b0 2023-10-08T16:05:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:33.900+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000160911", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fea8004c3c0 con 0x7fea64066f30 2023-10-08T16:05:33.904 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:05:33.903+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:05:33.928 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:05:36.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.917+0000 7fea757fa700 1 -- 172.21.15.170:0/2488542695 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7fea8004c3c0 con 0x7fea64066f30 2023-10-08T16:05:36.918 INFO:teuthology.orchestra.run.smithi170.stdout:metadata pool: cephfs.volume_0000000000160911.meta data pool: ['cephfs.volume_0000000000160911.data'] removed 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 msgr2=0x7fea640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 0x7fea640693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00d4a0 tx=0x7fea7c0075d0 comp rx=0 tx=0).stop 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 msgr2=0x7fea80165910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea7000c880 tx=0x7fea70008040 comp rx=0 tx=0).stop 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 shutdown_connections 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea80155eb0 0x7fea800ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea80069cb0 0x7fea80151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea64066f30 0x7fea640693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:36.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 --2- 172.21.15.170:0/2488542695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea800c70b0 0x7fea80165910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:36.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 >> 172.21.15.170:0/2488542695 conn(0x7fea80068e10 msgr2=0x7fea801545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:36.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 shutdown_connections 2023-10-08T16:05:36.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:36.921+0000 7fea8770a700 1 -- 172.21.15.170:0/2488542695 wait complete. 2023-10-08T16:05:36.936 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:05:36.936 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:36.975 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:37.019 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:05:37.020 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:05:37.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:37.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:37.062 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:05:37.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:37.062 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:05:37.117 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:05:37.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:05:37.118 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:37.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:05:37.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:37.165 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:05:37.181 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:37.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.297+0000 7f8458550700 1 Processor -- start 2023-10-08T16:05:37.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.297+0000 7f8458550700 1 -- start start 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8458550700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8458550700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8450159c60 0x7f845015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8458550700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845015a160 con 0x7f845015db10 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8458550700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845015df50 con 0x7f8450159c60 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8458550700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84500d13a0 con 0x7f8450159890 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8455aeb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8450159c60 0x7f845015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8455aeb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8450159c60 0x7f845015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45454/0 (socket says 172.21.15.170:45454) 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.298+0000 7f8455aeb700 1 -- 172.21.15.170:0/1563479439 learned_addr learned my addr 172.21.15.170:0/1563479439 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8455aeb700 1 -- 172.21.15.170:0/1563479439 >> v2:172.21.15.191:3300/0 conn(0x7f8450159890 msgr2=0x7f845015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8455aeb700 1 --2- 172.21.15.170:0/1563479439 >> v2:172.21.15.191:3300/0 conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8455aeb700 1 -- 172.21.15.170:0/1563479439 >> v1:172.21.15.191:6789/0 conn(0x7f845015db10 legacy=0x7f8450069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8455aeb700 1 -- 172.21.15.170:0/1563479439 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845015e140 con 0x7f8450159c60 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8455aeb700 1 --2- 172.21.15.170:0/1563479439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f845015d5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84500d1090 tx=0x7f8444009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c183d01e63c03d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f84552ea700 1 -- 172.21.15.170:0/1563479439 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400b820 con 0x7f8450159c60 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f84552ea700 1 -- 172.21.15.170:0/1563479439 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844400b980 con 0x7f8450159c60 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f84552ea700 1 -- 172.21.15.170:0/1563479439 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844400bd20 con 0x7f8450159c60 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 -- 172.21.15.170:0/1563479439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 msgr2=0x7f845015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 --2- 172.21.15.170:0/1563479439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f845015d5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f84500d1090 tx=0x7f8444009250 comp rx=0 tx=0).stop 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 -- 172.21.15.170:0/1563479439 shutdown_connections 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 --2- 172.21.15.170:0/1563479439 >> v2:172.21.15.191:3300/0 conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 --2- 172.21.15.170:0/1563479439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f845015d5d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.299+0000 7f8458550700 1 -- 172.21.15.170:0/1563479439 >> 172.21.15.170:0/1563479439 conn(0x7f8450068e20 msgr2=0x7f84500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:37.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.300+0000 7f8458550700 1 -- 172.21.15.170:0/1563479439 shutdown_connections 2023-10-08T16:05:37.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.300+0000 7f8458550700 1 -- 172.21.15.170:0/1563479439 wait complete. 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 Processor -- start 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 -- start start 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159890 0x7f8450169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f845015db10 0x7f8450158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450149b90 con 0x7f8450159c60 2023-10-08T16:05:37.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845016a490 con 0x7f845015db10 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8458550700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845016d780 con 0x7f8450159890 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8455aeb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8455aeb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45470/0 (socket says 172.21.15.170:45470) 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.301+0000 7f8455aeb700 1 -- 172.21.15.170:0/4116350284 learned_addr learned my addr 172.21.15.170:0/4116350284 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8456aed700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f845015db10 0x7f8450158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f84562ec700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159890 0x7f8450169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 -- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159890 msgr2=0x7f8450169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159890 0x7f8450169f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 -- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f845015db10 msgr2=0x7f8450158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f845015db10 0x7f8450158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 -- 172.21.15.170:0/4116350284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440094f0 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8455aeb700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8444003080 tx=0x7f8444016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444011e20 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844401f070 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844401a8a0 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84500cdc00 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.302+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845016a9a0 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.303+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845004c3c0 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.303+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f844401ac90 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.304+0000 7f844e7fc700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 0x7f843c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.304+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8444021070 con 0x7f8450159c60 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.304+0000 7f84562ec700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 0x7f843c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.304+0000 7f84562ec700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 0x7f843c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8450159210 tx=0x7f8440009620 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.308+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8444053b30 con 0x7f8450159c60 2023-10-08T16:05:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.535+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8450156e90 con 0x7f8450159c60 2023-10-08T16:05:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.536+0000 7f844e7fc700 1 -- 172.21.15.170:0/4116350284 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f8444057150 con 0x7f8450159c60 2023-10-08T16:05:37.536 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:37.536 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/931871174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":931871174},{"type":"v1","addr":"172.21.15.170:6836","nonce":931871174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 msgr2=0x7f843c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 0x7f843c0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8450159210 tx=0x7f8440009620 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 msgr2=0x7f84500cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8444003080 tx=0x7f8444016000 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 shutdown_connections 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f845015db10 0x7f8450158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159890 0x7f8450169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f843c066f30 0x7f843c0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 --2- 172.21.15.170:0/4116350284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159c60 0x7f84500cd6c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 >> 172.21.15.170:0/4116350284 conn(0x7f8450068e20 msgr2=0x7f84501570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 shutdown_connections 2023-10-08T16:05:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.538+0000 7f8458550700 1 -- 172.21.15.170:0/4116350284 wait complete. 2023-10-08T16:05:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 22 2023-10-08T16:05:37.550 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.664+0000 7f20622f7700 1 Processor -- start 2023-10-08T16:05:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.664+0000 7f20622f7700 1 -- start start 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20622f7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f205c159420 0x7f205c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20622f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20622f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205c0b11d0 con 0x7f205c15d230 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20622f7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205c0b14b0 con 0x7f205c159420 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20622f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c0b1790 con 0x7f205c1597f0 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f205c159420 0x7f205c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f205c159420 0x7f205c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45474/0 (socket says 172.21.15.170:45474) 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 -- 172.21.15.170:0/4186133652 learned_addr learned my addr 172.21.15.170:0/4186133652 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f2060af4700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 -- 172.21.15.170:0/4186133652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 msgr2=0x7f205c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.665+0000 7f20612f5700 1 -- 172.21.15.170:0/4186133652 >> v1:172.21.15.191:6789/0 conn(0x7f205c15d230 legacy=0x7f205c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20612f5700 1 -- 172.21.15.170:0/4186133652 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0b1a70 con 0x7f205c159420 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20612f5700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c159420 0x7f205c0b5f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20580099b0 tx=0x7f2058009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8669a97f6c4992c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f2053fff700 1 -- 172.21.15.170:0/4186133652 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800b860 con 0x7f205c159420 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f2053fff700 1 -- 172.21.15.170:0/4186133652 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800b9c0 con 0x7f205c159420 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f2053fff700 1 -- 172.21.15.170:0/4186133652 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800bd60 con 0x7f205c159420 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 -- 172.21.15.170:0/4186133652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c159420 msgr2=0x7f205c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c159420 0x7f205c0b5f40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f20580099b0 tx=0x7f2058009250 comp rx=0 tx=0).stop 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 -- 172.21.15.170:0/4186133652 shutdown_connections 2023-10-08T16:05:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c159420 0x7f205c0b5f40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 --2- 172.21.15.170:0/4186133652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 -- 172.21.15.170:0/4186133652 >> 172.21.15.170:0/4186133652 conn(0x7f205c0d0660 msgr2=0x7f205c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 -- 172.21.15.170:0/4186133652 shutdown_connections 2023-10-08T16:05:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.666+0000 7f20622f7700 1 -- 172.21.15.170:0/4186133652 wait complete. 2023-10-08T16:05:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.667+0000 7f20622f7700 1 Processor -- start 2023-10-08T16:05:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 -- start start 2023-10-08T16:05:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f205c159420 0x7f205c154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c169750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c154770 con 0x7f205c15d230 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c068e40 con 0x7f205c159420 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f20622f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c068510 con 0x7f205c1597f0 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.668+0000 7f2061af6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45484/0 (socket says 172.21.15.170:45484) 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 -- 172.21.15.170:0/3396661671 learned_addr learned my addr 172.21.15.170:0/3396661671 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20612f5700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f205c159420 0x7f205c154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2060af4700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c169750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 -- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 msgr2=0x7f205c169750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c169750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 -- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f205c159420 msgr2=0x7f205c154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f205c159420 0x7f205c154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 -- 172.21.15.170:0/3396661671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580094f0 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f2061af6700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f205400c840 tx=0x7f2054001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054012630 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20540127b0 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205c0687f0 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054012b70 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.669+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205c15aa90 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.670+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f205c068ff0 con 0x7f205c15d230 2023-10-08T16:05:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.670+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f205401a3a0 con 0x7f205c15d230 2023-10-08T16:05:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.671+0000 7f20517fa700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 0x7f20400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.671+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f2054084680 con 0x7f205c15d230 2023-10-08T16:05:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.671+0000 7f20612f5700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 0x7f20400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.671+0000 7f20612f5700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 0x7f20400693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20580099b0 tx=0x7f2058017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.675+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20540518a0 con 0x7f205c15d230 2023-10-08T16:05:37.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.858+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f205c04c3c0 con 0x7f205c15d230 2023-10-08T16:05:37.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.859+0000 7f20517fa700 1 -- 172.21.15.170:0/3396661671 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7f2054054ec0 con 0x7f205c15d230 2023-10-08T16:05:37.859 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:37.859 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:36.909591+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/2746556501":"2023-10-09T16:05:33.905200+0000","172.21.15.170:6834/2746556501":"2023-10-09T16:05:33.905200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.860+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 msgr2=0x7f20400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.860+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 0x7f20400693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20580099b0 tx=0x7f2058017000 comp rx=0 tx=0).stop 2023-10-08T16:05:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.860+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 msgr2=0x7f205c15b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.860+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f205400c840 tx=0x7f2054001670 comp rx=0 tx=0).stop 2023-10-08T16:05:37.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 shutdown_connections 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2040066f30 0x7f20400693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f205c15d230 0x7f205c15b780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f205c159420 0x7f205c154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 --2- 172.21.15.170:0/3396661671 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205c1597f0 0x7f205c169750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 >> 172.21.15.170:0/3396661671 conn(0x7f205c0d0660 msgr2=0x7f205c0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 shutdown_connections 2023-10-08T16:05:37.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.861+0000 7f20622f7700 1 -- 172.21.15.170:0/3396661671 wait complete. 2023-10-08T16:05:37.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:05:16.184130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:05:17.417856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:05:37.873 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:37.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 Processor -- start 2023-10-08T16:05:37.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 -- start start 2023-10-08T16:05:37.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc415a160 con 0x7fcdc415db10 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40d13a0 con 0x7fcdc4159c60 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.991+0000 7fcdc951b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415e140 con 0x7fcdc4159890 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45494/0 (socket says 172.21.15.170:45494) 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 -- 172.21.15.170:0/2205837288 learned_addr learned my addr 172.21.15.170:0/2205837288 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc37fe700 1 --1- 172.21.15.170:0/2205837288 >> v1:172.21.15.170:6789/0 conn(0x7fcdc415db10 0x7fcdc4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35734/0 (socket says 172.21.15.170:35734) 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc2ffd700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 -- 172.21.15.170:0/2205837288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc4159890 msgr2=0x7fcdc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 -- 172.21.15.170:0/2205837288 >> v1:172.21.15.170:6789/0 conn(0x7fcdc415db10 legacy=0x7fcdc4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 -- 172.21.15.170:0/2205837288 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc415a3d0 con 0x7fcdc4159c60 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc1ffb700 1 -- 172.21.15.170:0/2205837288 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18689922 0 0) 0x7fcdc415a160 con 0x7fcdc415db10 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc27fc700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80099b0 tx=0x7fcdb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bab277af4b59b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc1ffb700 1 -- 172.21.15.170:0/2205837288 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8009ea0 con 0x7fcdc4159c60 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc1ffb700 1 -- 172.21.15.170:0/2205837288 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb800b860 con 0x7fcdc4159c60 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.992+0000 7fcdc1ffb700 1 -- 172.21.15.170:0/2205837288 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb800bc00 con 0x7fcdc4159c60 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 -- 172.21.15.170:0/2205837288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159c60 msgr2=0x7fcdc415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80099b0 tx=0x7fcdb8009250 comp rx=0 tx=0).stop 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 -- 172.21.15.170:0/2205837288 shutdown_connections 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc4159890 0x7fcdc415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2205837288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 -- 172.21.15.170:0/2205837288 >> 172.21.15.170:0/2205837288 conn(0x7fcdc4068e20 msgr2=0x7fcdc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:37.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 -- 172.21.15.170:0/2205837288 shutdown_connections 2023-10-08T16:05:37.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.993+0000 7fcdc951b700 1 -- 172.21.15.170:0/2205837288 wait complete. 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.994+0000 7fcdc951b700 1 Processor -- start 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.994+0000 7fcdc951b700 1 -- start start 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc4159c60 0x7fcdc40cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc415db10 0x7fcdc414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc416df60 con 0x7fcdc4159890 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40ce270 con 0x7fcdc4159c60 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc416d390 con 0x7fcdc415db10 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:37.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45498/0 (socket says 172.21.15.170:45498) 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 -- 172.21.15.170:0/2524208913 learned_addr learned my addr 172.21.15.170:0/2524208913 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 -- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc415db10 msgr2=0x7fcdc414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc415db10 0x7fcdc414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 -- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc4159c60 msgr2=0x7fcdc40cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc4159c60 0x7fcdc40cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 -- 172.21.15.170:0/2524208913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb80094f0 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc2ffd700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400ec80 tx=0x7fcdb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400c9d0 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb400cb30 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc4152510 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4013470 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.995+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc416d910 con 0x7fcdc4159890 2023-10-08T16:05:37.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.996+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc404c3c0 con 0x7fcdc4159890 2023-10-08T16:05:38.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.997+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdb4013820 con 0x7fcdc4159890 2023-10-08T16:05:38.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.997+0000 7fcdaaffd700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.997+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fcdb4084970 con 0x7fcdc4159890 2023-10-08T16:05:38.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.997+0000 7fcdc27fc700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:37.997+0000 7fcdc27fc700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 0x7fcdb00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80099b0 tx=0x7fcdb8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.001+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdb4051b90 con 0x7fcdc4159890 2023-10-08T16:05:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.179+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcdc416dcb0 con 0x7fcdc4159890 2023-10-08T16:05:38.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.180+0000 7fcdaaffd700 1 -- 172.21.15.170:0/2524208913 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcdb40551b0 con 0x7fcdc4159890 2023-10-08T16:05:38.180 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 msgr2=0x7fcdb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 0x7fcdb00693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80099b0 tx=0x7fcdb8017000 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 msgr2=0x7fcdc40cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400ec80 tx=0x7fcdb40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 shutdown_connections 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcdc415db10 0x7fcdc414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcdc4159890 0x7fcdc40cf6d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 --2- 172.21.15.170:0/2524208913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcdc4159c60 0x7fcdc40cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 >> 172.21.15.170:0/2524208913 conn(0x7fcdc4068e20 msgr2=0x7fcdc4158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 shutdown_connections 2023-10-08T16:05:38.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.182+0000 7fcdc951b700 1 -- 172.21.15.170:0/2524208913 wait complete. 2023-10-08T16:05:38.194 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:05:38.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.310+0000 7f03ac8d8700 1 Processor -- start 2023-10-08T16:05:38.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.310+0000 7f03ac8d8700 1 -- start start 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03a8159c60 0x7f03a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f03a815db10 0x7f03a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a815a160 con 0x7f03a8159c60 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a80d13a0 con 0x7f03a815db10 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03ac8d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a815e140 con 0x7f03a8159890 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03a8159c60 0x7f03a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03a8159c60 0x7f03a815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45504/0 (socket says 172.21.15.170:45504) 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 -- 172.21.15.170:0/1592039656 learned_addr learned my addr 172.21.15.170:0/1592039656 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03a6d9d700 1 --2- 172.21.15.170:0/1592039656 >> v2:172.21.15.191:3300/0 conn(0x7f03a815db10 0x7f03a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f03a659c700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 -- 172.21.15.170:0/1592039656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 msgr2=0x7f03a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 -- 172.21.15.170:0/1592039656 >> v2:172.21.15.191:3300/0 conn(0x7f03a815db10 msgr2=0x7f03a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 --2- 172.21.15.170:0/1592039656 >> v2:172.21.15.191:3300/0 conn(0x7f03a815db10 0x7f03a8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.311+0000 7f039dd9b700 1 -- 172.21.15.170:0/1592039656 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a815a3d0 con 0x7f03a8159c60 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f039dd9b700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a8159c60 0x7f03a815d5d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03a8089b50 tx=0x7f0390009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34896d1382c3646 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03a5d9b700 1 -- 172.21.15.170:0/1592039656 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000b860 con 0x7f03a8159c60 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03a5d9b700 1 -- 172.21.15.170:0/1592039656 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000b9c0 con 0x7f03a8159c60 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03a5d9b700 1 -- 172.21.15.170:0/1592039656 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000bd60 con 0x7f03a8159c60 2023-10-08T16:05:38.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 -- 172.21.15.170:0/1592039656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a8159c60 msgr2=0x7f03a815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a8159c60 0x7f03a815d5d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f03a8089b50 tx=0x7f0390009250 comp rx=0 tx=0).stop 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 -- 172.21.15.170:0/1592039656 shutdown_connections 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/1592039656 >> v2:172.21.15.191:3300/0 conn(0x7f03a815db10 0x7f03a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/1592039656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a8159c60 0x7f03a815d5d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 -- 172.21.15.170:0/1592039656 >> 172.21.15.170:0/1592039656 conn(0x7f03a8068e20 msgr2=0x7f03a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 -- 172.21.15.170:0/1592039656 shutdown_connections 2023-10-08T16:05:38.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.312+0000 7f03ac8d8700 1 -- 172.21.15.170:0/1592039656 wait complete. 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.313+0000 7f03ac8d8700 1 Processor -- start 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 -- start start 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a8151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 0x7f03a80ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8151f60 con 0x7f03a815db10 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80caac0 con 0x7f03a8159c60 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03ac8d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a80b4340 con 0x7f03a8159890 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03a6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03a6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45518/0 (socket says 172.21.15.170:45518) 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f03a6d9d700 1 -- 172.21.15.170:0/3667771558 learned_addr learned my addr 172.21.15.170:0/3667771558 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.314+0000 7f039dd9b700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 0x7f03a80ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a659c700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a8151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 -- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 msgr2=0x7f03a8151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a8151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 -- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 msgr2=0x7f03a80ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 0x7f03a80ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 -- 172.21.15.170:0/3667771558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03900094f0 con 0x7f03a815db10 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f039dd9b700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 0x7f03a80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03a6d9d700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f03a000cde0 tx=0x7f03a00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a000aaf0 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a000ac50 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a80cf170 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0017430 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.315+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a80cf7b0 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.316+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a80cfb30 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.316+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03a0003020 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.317+0000 7f039effd700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 0x7f03880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.317+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f03a0013070 con 0x7f03a815db10 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.317+0000 7f03a659c700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 0x7f03880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.317+0000 7f03a659c700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 0x7f03880693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03a8152c50 tx=0x7f0398008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.321+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03a0051a10 con 0x7f03a815db10 2023-10-08T16:05:38.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.502+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03a804c3c0 con 0x7f03a815db10 2023-10-08T16:05:38.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.503+0000 7f039effd700 1 -- 172.21.15.170:0/3667771558 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03a001c020 con 0x7f03a815db10 2023-10-08T16:05:38.503 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:05:38.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 msgr2=0x7f03880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 0x7f03880693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f03a8152c50 tx=0x7f0398008040 comp rx=0 tx=0).stop 2023-10-08T16:05:38.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 msgr2=0x7f03a80ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f03a000cde0 tx=0x7f03a00077d0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 shutdown_connections 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0388066f30 0x7f03880693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03a815db10 0x7f03a80ca580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03a8159890 0x7f03a8151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 --2- 172.21.15.170:0/3667771558 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f03a8159c60 0x7f03a80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 >> 172.21.15.170:0/3667771558 conn(0x7f03a8068e20 msgr2=0x7f03a8154690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 shutdown_connections 2023-10-08T16:05:38.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.505+0000 7f03ac8d8700 1 -- 172.21.15.170:0/3667771558 wait complete. 2023-10-08T16:05:38.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:05:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.634+0000 7feef2a86700 1 Processor -- start 2023-10-08T16:05:38.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.634+0000 7feef2a86700 1 -- start start 2023-10-08T16:05:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2a86700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feeec0b16d0 0x7feeec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2a86700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feeec0b33b0 0x7feeec0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2a86700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeec0b2080 con 0x7feeec0b4e70 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2a86700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0cfb00 con 0x7feeec0b33b0 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2a86700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeec0b5c40 con 0x7feeec0b16d0 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feeec0b33b0 0x7feeec0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feeec0b33b0 0x7feeec0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45526/0 (socket says 172.21.15.170:45526) 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef1a84700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7feeec0b4e70 0x7feeec0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35740/0 (socket says 172.21.15.170:35740) 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 -- 172.21.15.170:0/620083066 learned_addr learned my addr 172.21.15.170:0/620083066 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef1283700 1 --2- 172.21.15.170:0/620083066 >> v2:172.21.15.191:3300/0 conn(0x7feeec0b16d0 0x7feeec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 -- 172.21.15.170:0/620083066 >> v2:172.21.15.191:3300/0 conn(0x7feeec0b16d0 msgr2=0x7feeec0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 --2- 172.21.15.170:0/620083066 >> v2:172.21.15.191:3300/0 conn(0x7feeec0b16d0 0x7feeec0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 -- 172.21.15.170:0/620083066 >> v1:172.21.15.170:6789/0 conn(0x7feeec0b4e70 legacy=0x7feeec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 -- 172.21.15.170:0/620083066 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0b5f20 con 0x7feeec0b33b0 2023-10-08T16:05:38.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef0a82700 1 -- 172.21.15.170:0/620083066 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437510314 0 0) 0x7feeec0b2080 con 0x7feeec0b4e70 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.635+0000 7feef2285700 1 --2- 172.21.15.170:0/620083066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b33b0 0x7feeec0b37d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feeec089830 tx=0x7feee8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e018d33cfb48bcfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef0a82700 1 -- 172.21.15.170:0/620083066 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8009e70 con 0x7feeec0b33b0 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef0a82700 1 -- 172.21.15.170:0/620083066 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee800b860 con 0x7feeec0b33b0 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef0a82700 1 -- 172.21.15.170:0/620083066 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800bc20 con 0x7feeec0b33b0 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 -- 172.21.15.170:0/620083066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b33b0 msgr2=0x7feeec0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 --2- 172.21.15.170:0/620083066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b33b0 0x7feeec0b37d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feeec089830 tx=0x7feee8009250 comp rx=0 tx=0).stop 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 -- 172.21.15.170:0/620083066 shutdown_connections 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 --2- 172.21.15.170:0/620083066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b33b0 0x7feeec0b37d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 --2- 172.21.15.170:0/620083066 >> v2:172.21.15.191:3300/0 conn(0x7feeec0b16d0 0x7feeec0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 -- 172.21.15.170:0/620083066 >> 172.21.15.170:0/620083066 conn(0x7feeec0d10c0 msgr2=0x7feeec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 -- 172.21.15.170:0/620083066 shutdown_connections 2023-10-08T16:05:38.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.636+0000 7feef2a86700 1 -- 172.21.15.170:0/620083066 wait complete. 2023-10-08T16:05:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.637+0000 7feef2a86700 1 Processor -- start 2023-10-08T16:05:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.637+0000 7feef2a86700 1 -- start start 2023-10-08T16:05:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0b33b0 0x7feeec0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec0b4e70 0x7feeec0b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec0cc410 con 0x7feeec0b16d0 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec16a320 con 0x7feeec0b33b0 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef2a86700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec169a60 con 0x7feeec0b4e70 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45530/0 (socket says 172.21.15.170:45530) 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 -- 172.21.15.170:0/2779143512 learned_addr learned my addr 172.21.15.170:0/2779143512 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 -- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec0b4e70 msgr2=0x7feeec0b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec0b4e70 0x7feeec0b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 -- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0b33b0 msgr2=0x7feeec0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0b33b0 0x7feeec0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 -- 172.21.15.170:0/2779143512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee80094f0 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.638+0000 7feef1a84700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feee400ce00 tx=0x7feee4001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.639+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4005df0 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.639+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400a9f0 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.639+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400adb0 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.639+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec0cc130 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.639+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeec169fb0 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.640+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeec088990 con 0x7feeec0b16d0 2023-10-08T16:05:38.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.640+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feee4007300 con 0x7feeec0b16d0 2023-10-08T16:05:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.641+0000 7feee1ffb700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 0x7feed00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.641+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7feee4084800 con 0x7feeec0b16d0 2023-10-08T16:05:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.641+0000 7feef1283700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 0x7feed00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.641+0000 7feef1283700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 0x7feed00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feeec0c8bf0 tx=0x7feedc0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.645+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feee4051aa0 con 0x7feeec0b16d0 2023-10-08T16:05:38.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.826+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feeec0b48c0 con 0x7feeec0b16d0 2023-10-08T16:05:38.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.826+0000 7feee1ffb700 1 -- 172.21.15.170:0/2779143512 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7feee40550c0 con 0x7feeec0b16d0 2023-10-08T16:05:38.826 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 msgr2=0x7feed00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 0x7feed00693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feeec0c8bf0 tx=0x7feedc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 msgr2=0x7feeec157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feee400ce00 tx=0x7feee4001bc0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 shutdown_connections 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feeec0b4e70 0x7feeec0b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feed0066f30 0x7feed00693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeec0b16d0 0x7feeec157530 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 --2- 172.21.15.170:0/2779143512 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feeec0b33b0 0x7feeec0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 >> 172.21.15.170:0/2779143512 conn(0x7feeec0d10c0 msgr2=0x7feeec069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 shutdown_connections 2023-10-08T16:05:38.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.828+0000 7feef2a86700 1 -- 172.21.15.170:0/2779143512 wait complete. 2023-10-08T16:05:38.840 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:38.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.955+0000 7f6042922700 1 Processor -- start 2023-10-08T16:05:38.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f6042922700 1 -- start start 2023-10-08T16:05:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f6042922700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f603c159c60 0x7f603c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f6042922700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c15a160 con 0x7f603c15db10 2023-10-08T16:05:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f6042922700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0d13a0 con 0x7f603c159890 2023-10-08T16:05:38.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f6042922700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c15df50 con 0x7f603c159c60 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.956+0000 7f603b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f603c159c60 0x7f603c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f6040ebf700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f603c15db10 0x7f603c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35750/0 (socket says 172.21.15.170:35750) 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f6040ebf700 1 -- 172.21.15.170:0/1141934493 learned_addr learned my addr 172.21.15.170:0/1141934493 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603b7fe700 1 -- 172.21.15.170:0/1141934493 >> v1:172.21.15.191:6789/0 conn(0x7f603c159890 legacy=0x7f603c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603b7fe700 1 -- 172.21.15.170:0/1141934493 >> v1:172.21.15.170:6789/0 conn(0x7f603c15db10 legacy=0x7f603c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603b7fe700 1 -- 172.21.15.170:0/1141934493 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c15e140 con 0x7f603c159c60 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603affd700 1 -- 172.21.15.170:0/1141934493 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045171710 0 0) 0x7f603c15a160 con 0x7f603c15db10 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603b7fe700 1 --2- 172.21.15.170:0/1141934493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c159c60 0x7f603c15d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f60300098e0 tx=0x7f60300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4a3a8607682021 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603affd700 1 -- 172.21.15.170:0/1141934493 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030009e80 con 0x7f603c159c60 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603affd700 1 -- 172.21.15.170:0/1141934493 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603000b820 con 0x7f603c159c60 2023-10-08T16:05:38.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.957+0000 7f603affd700 1 -- 172.21.15.170:0/1141934493 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603000bba0 con 0x7f603c159c60 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 -- 172.21.15.170:0/1141934493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c159c60 msgr2=0x7f603c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 --2- 172.21.15.170:0/1141934493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c159c60 0x7f603c15d5d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f60300098e0 tx=0x7f60300091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 -- 172.21.15.170:0/1141934493 shutdown_connections 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 --2- 172.21.15.170:0/1141934493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c159c60 0x7f603c15d5d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 -- 172.21.15.170:0/1141934493 >> 172.21.15.170:0/1141934493 conn(0x7f603c068e20 msgr2=0x7f603c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 -- 172.21.15.170:0/1141934493 shutdown_connections 2023-10-08T16:05:38.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.958+0000 7f6042922700 1 -- 172.21.15.170:0/1141934493 wait complete. 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.959+0000 7f6042922700 1 Processor -- start 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.959+0000 7f6042922700 1 -- start start 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 0x7f603c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 0x7f603c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0ceca0 con 0x7f603c15db10 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0ce690 con 0x7f603c159c60 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0c8f20 con 0x7f603c159890 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45544/0 (socket says 172.21.15.170:45544) 2023-10-08T16:05:38.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 -- 172.21.15.170:0/1617659910 learned_addr learned my addr 172.21.15.170:0/1617659910 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f603b7fe700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 0x7f603c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f603bfff700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 0x7f603c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 -- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 msgr2=0x7f603c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 0x7f603c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 -- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 msgr2=0x7f603c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 0x7f603c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 -- 172.21.15.170:0/1617659910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009420 con 0x7f603c15db10 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f603b7fe700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 0x7f603c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f603bfff700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 0x7f603c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6040ebf700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f603400cc90 tx=0x7f6034001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603400cea0 con 0x7f603c15db10 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603400ab70 con 0x7f603c15db10 2023-10-08T16:05:38.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c0ce930 con 0x7f603c15db10 2023-10-08T16:05:38.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340070f0 con 0x7f603c15db10 2023-10-08T16:05:38.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.960+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c0c9200 con 0x7f603c15db10 2023-10-08T16:05:38.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.961+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c0cee10 con 0x7f603c15db10 2023-10-08T16:05:38.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.962+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60340074a0 con 0x7f603c15db10 2023-10-08T16:05:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.962+0000 7f6023fff700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 0x7f60280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.962+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f6034084a00 con 0x7f603c15db10 2023-10-08T16:05:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.962+0000 7f603bfff700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 0x7f60280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:38.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.963+0000 7f603bfff700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 0x7f60280693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f602c006fd0 tx=0x7f602c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:38.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:38.966+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6034051ca0 con 0x7f603c15db10 2023-10-08T16:05:39.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.190+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603c04c3c0 con 0x7f603c15db10 2023-10-08T16:05:39.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.190+0000 7f6023fff700 1 -- 172.21.15.170:0/1617659910 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f60340552c0 con 0x7f603c15db10 2023-10-08T16:05:39.191 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:39.191 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/931871174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":931871174},{"type":"v1","addr":"172.21.15.170:6836","nonce":931871174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 msgr2=0x7f60280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 0x7f60280693e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f602c006fd0 tx=0x7f602c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 msgr2=0x7f603c0b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f603400cc90 tx=0x7f6034001a80 comp rx=0 tx=0).stop 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 shutdown_connections 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6028066f30 0x7f60280693e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f603c15db10 0x7f603c0b1740 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f603c159890 0x7f603c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 --2- 172.21.15.170:0/1617659910 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f603c159c60 0x7f603c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.192+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 >> 172.21.15.170:0/1617659910 conn(0x7f603c068e20 msgr2=0x7f603c1546f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.193+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 shutdown_connections 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.193+0000 7f6042922700 1 -- 172.21.15.170:0/1617659910 wait complete. 2023-10-08T16:05:39.194 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 22 2023-10-08T16:05:39.205 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:39.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.319+0000 7fcddb59e700 1 Processor -- start 2023-10-08T16:05:39.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.319+0000 7fcddb59e700 1 -- start start 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddb59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcddc159420 0x7fcddc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddb59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcddc15d230 0x7fcddc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0b2980 con 0x7fcddc1597f0 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddb59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc15db70 con 0x7fcddc159420 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddb59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc0b11d0 con 0x7fcddc15d230 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcddc159420 0x7fcddc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcddc159420 0x7fcddc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45550/0 (socket says 172.21.15.170:45550) 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 -- 172.21.15.170:0/3908729973 learned_addr learned my addr 172.21.15.170:0/3908729973 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcddad9d700 1 --2- 172.21.15.170:0/3908729973 >> v2:172.21.15.191:3300/0 conn(0x7fcddc15d230 0x7fcddc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 -- 172.21.15.170:0/3908729973 >> v2:172.21.15.191:3300/0 conn(0x7fcddc15d230 msgr2=0x7fcddc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 --2- 172.21.15.170:0/3908729973 >> v2:172.21.15.191:3300/0 conn(0x7fcddc15d230 0x7fcddc15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 -- 172.21.15.170:0/3908729973 >> v1:172.21.15.191:6789/0 conn(0x7fcddc1597f0 legacy=0x7fcddc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.320+0000 7fcdda59c700 1 -- 172.21.15.170:0/3908729973 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0b14b0 con 0x7fcddc159420 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcdda59c700 1 --2- 172.21.15.170:0/3908729973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc0b5f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcddc15d170 tx=0x7fcdcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2841981233c7d7cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcdd9d9b700 1 -- 172.21.15.170:0/3908729973 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00b860 con 0x7fcddc159420 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcdd9d9b700 1 -- 172.21.15.170:0/3908729973 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00b9c0 con 0x7fcddc159420 2023-10-08T16:05:39.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcdd9d9b700 1 -- 172.21.15.170:0/3908729973 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bd60 con 0x7fcddc159420 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 -- 172.21.15.170:0/3908729973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 msgr2=0x7fcddc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 --2- 172.21.15.170:0/3908729973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc0b5f40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcddc15d170 tx=0x7fcdcc009250 comp rx=0 tx=0).stop 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 -- 172.21.15.170:0/3908729973 shutdown_connections 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 --2- 172.21.15.170:0/3908729973 >> v2:172.21.15.191:3300/0 conn(0x7fcddc15d230 0x7fcddc15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 --2- 172.21.15.170:0/3908729973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc0b5f40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 -- 172.21.15.170:0/3908729973 >> 172.21.15.170:0/3908729973 conn(0x7fcddc0d0660 msgr2=0x7fcddc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 -- 172.21.15.170:0/3908729973 shutdown_connections 2023-10-08T16:05:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.321+0000 7fcddb59e700 1 -- 172.21.15.170:0/3908729973 wait complete. 2023-10-08T16:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.322+0000 7fcddb59e700 1 Processor -- start 2023-10-08T16:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 -- start start 2023-10-08T16:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 0x7fcddc169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15d230 0x7fcddc16a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1566b0 con 0x7fcddc159420 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc16a7b0 con 0x7fcddc1597f0 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcddb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc16aa90 con 0x7fcddc15d230 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcdda59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcdda59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45558/0 (socket says 172.21.15.170:45558) 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.323+0000 7fcdda59c700 1 -- 172.21.15.170:0/2241058136 learned_addr learned my addr 172.21.15.170:0/2241058136 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcddad9d700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15d230 0x7fcddc16a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdd1d9b700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 0x7fcddc169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 -- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15d230 msgr2=0x7fcddc16a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15d230 0x7fcddc16a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 -- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 msgr2=0x7fcddc169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 0x7fcddc169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 -- 172.21.15.170:0/2241058136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc0094f0 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdd1d9b700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 0x7fcddc169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdda59c700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcddc089340 tx=0x7fcdcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc011e60 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc01a630 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc159ca0 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc01a9f0 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.324+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc15a260 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.325+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc15a480 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.325+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdcc02d440 con 0x7fcddc159420 2023-10-08T16:05:39.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.326+0000 7fcdd2ffd700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.326+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fcdcc086110 con 0x7fcddc159420 2023-10-08T16:05:39.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.326+0000 7fcdd1d9b700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.326+0000 7fcdd1d9b700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 0x7fcdbc0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4007fd0 tx=0x7fcdc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.330+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdcc053330 con 0x7fcddc159420 2023-10-08T16:05:39.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.519+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcddc04c3c0 con 0x7fcddc159420 2023-10-08T16:05:39.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.520+0000 7fcdd2ffd700 1 -- 172.21.15.170:0/2241058136 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7fcdcc056950 con 0x7fcddc159420 2023-10-08T16:05:39.520 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:39.521 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:36.909591+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/2746556501":"2023-10-09T16:05:33.905200+0000","172.21.15.170:6834/2746556501":"2023-10-09T16:05:33.905200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:39.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 msgr2=0x7fcdbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 0x7fcdbc0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4007fd0 tx=0x7fcdc4009040 comp rx=0 tx=0).stop 2023-10-08T16:05:39.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 msgr2=0x7fcddc155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcddc089340 tx=0x7fcdcc017000 comp rx=0 tx=0).stop 2023-10-08T16:05:39.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 shutdown_connections 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15d230 0x7fcddc16a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc159420 0x7fcddc155c20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 --2- 172.21.15.170:0/2241058136 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc1597f0 0x7fcddc169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 >> 172.21.15.170:0/2241058136 conn(0x7fcddc0d0660 msgr2=0x7fcddc069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.522+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 shutdown_connections 2023-10-08T16:05:39.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.523+0000 7fcddb59e700 1 -- 172.21.15.170:0/2241058136 wait complete. 2023-10-08T16:05:39.535 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:05:39.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:39.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.649+0000 7f138ca87700 1 Processor -- start 2023-10-08T16:05:39.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.649+0000 7f138ca87700 1 -- start start 2023-10-08T16:05:39.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f138ca87700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f13880cb5a0 0x7f13880cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f138ca87700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f13880c85a0 0x7f13880c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f138ca87700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1388069d30 con 0x7f1388156fb0 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f138ca87700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880c8110 con 0x7f13880c85a0 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f138ca87700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880cbf30 con 0x7f13880cb5a0 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.650+0000 7f1386d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f13880c85a0 0x7f13880c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f13880c85a0 0x7f13880c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45560/0 (socket says 172.21.15.170:45560) 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 -- 172.21.15.170:0/2670946575 learned_addr learned my addr 172.21.15.170:0/2670946575 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f137ffff700 1 --2- 172.21.15.170:0/2670946575 >> v2:172.21.15.191:3300/0 conn(0x7f13880cb5a0 0x7f13880cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 -- 172.21.15.170:0/2670946575 >> v2:172.21.15.191:3300/0 conn(0x7f13880cb5a0 msgr2=0x7f13880cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 --2- 172.21.15.170:0/2670946575 >> v2:172.21.15.191:3300/0 conn(0x7f13880cb5a0 0x7f13880cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 -- 172.21.15.170:0/2670946575 >> v1:172.21.15.191:6789/0 conn(0x7f1388156fb0 legacy=0x7f138806a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 -- 172.21.15.170:0/2670946575 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13881568e0 con 0x7f13880c85a0 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1386d9d700 1 --2- 172.21.15.170:0/2670946575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f13880c89e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f138000d1a0 tx=0x7f138000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9b9f2a5d729602 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1385d9b700 1 -- 172.21.15.170:0/2670946575 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380010070 con 0x7f13880c85a0 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1385d9b700 1 -- 172.21.15.170:0/2670946575 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000ddd0 con 0x7f13880c85a0 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.651+0000 7f1385d9b700 1 -- 172.21.15.170:0/2670946575 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138000b320 con 0x7f13880c85a0 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 -- 172.21.15.170:0/2670946575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 msgr2=0x7f13880c89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 --2- 172.21.15.170:0/2670946575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f13880c89e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f138000d1a0 tx=0x7f138000ba50 comp rx=0 tx=0).stop 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 -- 172.21.15.170:0/2670946575 shutdown_connections 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 --2- 172.21.15.170:0/2670946575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f13880c89e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 --2- 172.21.15.170:0/2670946575 >> v2:172.21.15.191:3300/0 conn(0x7f13880cb5a0 0x7f13880cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 -- 172.21.15.170:0/2670946575 >> 172.21.15.170:0/2670946575 conn(0x7f13880c9870 msgr2=0x7f13880c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 -- 172.21.15.170:0/2670946575 shutdown_connections 2023-10-08T16:05:39.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.652+0000 7f138ca87700 1 -- 172.21.15.170:0/2670946575 wait complete. 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.653+0000 7f138ca87700 1 Processor -- start 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.653+0000 7f138ca87700 1 -- start start 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.653+0000 7f138ca87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.653+0000 7f138ca87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 0x7f1388155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 0x7f1388166c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c3eb0 con 0x7f13880c85a0 2023-10-08T16:05:39.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c4170 con 0x7f1388156fb0 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880c4450 con 0x7f13880cb5a0 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45570/0 (socket says 172.21.15.170:45570) 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 -- 172.21.15.170:0/944771541 learned_addr learned my addr 172.21.15.170:0/944771541 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f137ffff700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 0x7f1388155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f1386d9d700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 0x7f1388166c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 -- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 msgr2=0x7f1388155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 0x7f1388155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 -- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 msgr2=0x7f1388166c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 0x7f1388166c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 -- 172.21.15.170:0/944771541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138000bcf0 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f1386d9d700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 0x7f1388166c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f137ffff700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 0x7f1388155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138659c700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f137800c590 tx=0x7f1378007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011590 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378011730 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880c4730 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011af0 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.654+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13881587a0 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.655+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138804c3c0 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.656+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1378012420 con 0x7f13880c85a0 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.656+0000 7f137e7fc700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 0x7f13680693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:39.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.656+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f13780838c0 con 0x7f13880c85a0 2023-10-08T16:05:39.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.656+0000 7f137ffff700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 0x7f13680693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:39.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.656+0000 7f137ffff700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 0x7f13680693b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f138815acb0 tx=0x7f1370008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:39.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.660+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f137804b5a0 con 0x7f13880c85a0 2023-10-08T16:05:39.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.886+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f138806a3c0 con 0x7f13880c85a0 2023-10-08T16:05:39.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.887+0000 7f137e7fc700 1 -- 172.21.15.170:0/944771541 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f1378017070 con 0x7f13880c85a0 2023-10-08T16:05:39.887 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:39.888 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/931871174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":931871174},{"type":"v1","addr":"172.21.15.170:6836","nonce":931871174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 msgr2=0x7f13680693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 0x7f13680693b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f138815acb0 tx=0x7f1370008040 comp rx=0 tx=0).stop 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 msgr2=0x7f1388158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f137800c590 tx=0x7f1378007f30 comp rx=0 tx=0).stop 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 shutdown_connections 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1388156fb0 0x7f1388166c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1368066f00 0x7f13680693b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f13880c85a0 0x7f1388158260 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 --2- 172.21.15.170:0/944771541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f13880cb5a0 0x7f1388155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 >> 172.21.15.170:0/944771541 conn(0x7f13880c9870 msgr2=0x7f1388068420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 shutdown_connections 2023-10-08T16:05:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:39.889+0000 7f138ca87700 1 -- 172.21.15.170:0/944771541 wait complete. 2023-10-08T16:05:39.891 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 22 2023-10-08T16:05:39.901 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:40.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.015+0000 7f00ae5bc700 1 Processor -- start 2023-10-08T16:05:40.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.016+0000 7f00ae5bc700 1 -- start start 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.016+0000 7f00ae5bc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f00a8159c60 0x7f00a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.016+0000 7f00ae5bc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a815a160 con 0x7f00a815db10 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.016+0000 7f00ae5bc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a80d13a0 con 0x7f00a8159890 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.016+0000 7f00ae5bc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a815df50 con 0x7f00a8159c60 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00acb59700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f00a815db10 0x7f00a8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35764/0 (socket says 172.21.15.170:35764) 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00acb59700 1 -- 172.21.15.170:0/3829518186 learned_addr learned my addr 172.21.15.170:0/3829518186 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a77fe700 1 --2- 172.21.15.170:0/3829518186 >> v2:172.21.15.191:3300/0 conn(0x7f00a8159c60 0x7f00a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766419349 0 0) 0x7f00a815a160 con 0x7f00a815db10 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0090003570 con 0x7f00a815db10 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886080396 0 0) 0x7f00a80d13a0 con 0x7f00a8159890 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a815a160 con 0x7f00a8159890 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709749376 0 0) 0x7f0090003570 con 0x7f00a815db10 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 >> v2:172.21.15.191:3300/0 conn(0x7f00a8159c60 msgr2=0x7f00a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 --2- 172.21.15.170:0/3829518186 >> v2:172.21.15.191:3300/0 conn(0x7f00a8159c60 0x7f00a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 >> v1:172.21.15.191:6789/0 conn(0x7f00a8159890 legacy=0x7f00a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a815e140 con 0x7f00a815db10 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f00a00017f0 con 0x7f00a815db10 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 >> v1:172.21.15.170:6789/0 conn(0x7f00a815db10 legacy=0x7f00a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f0090003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00900051c0 0x7f009000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009000c080 con 0x7f0090003bf0 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009000c1f0 con 0x7f0090004810 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009000c3b0 con 0x7f00900051c0 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f0090003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00a77fe700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.017+0000 7f00acb59700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00900051c0 0x7f009000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 -- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00900051c0 msgr2=0x7f009000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00900051c0 0x7f009000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 -- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 msgr2=0x7f0090004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0090004810 0x7f0090004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 -- 172.21.15.170:0/3829518186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090003570 con 0x7f0090003bf0 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00acb59700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00900051c0 0x7f009000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00acb59700 1 -- 172.21.15.170:0/3829518186 reap_dead start 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f0090003fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0098001a10 tx=0x7f0098007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9b771828e36b8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800c5d0 con 0x7f0090003bf0 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800c730 con 0x7f0090003bf0 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00a6ffd700 1 -- 172.21.15.170:0/3829518186 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800cad0 con 0x7f0090003bf0 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 msgr2=0x7f0090003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f0090003fe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0098001a10 tx=0x7f0098007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3829518186 shutdown_connections 2023-10-08T16:05:40.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3829518186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f0090003fe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3829518186 >> 172.21.15.170:0/3829518186 conn(0x7f00a8068e20 msgr2=0x7f00a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3829518186 shutdown_connections 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.018+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3829518186 wait complete. 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.019+0000 7f00ae5bc700 1 Processor -- start 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 -- start start 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a80cf6d0 0x7f00a80b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a80cfb40 0x7f00a80b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80b5430 con 0x7f0090003bf0 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80b44a0 con 0x7f00a80cfb40 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00ae5bc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80cebf0 con 0x7f00a80cf6d0 2023-10-08T16:05:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45590/0 (socket says 172.21.15.170:45590) 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.020+0000 7f00a7fff700 1 -- 172.21.15.170:0/3783539546 learned_addr learned my addr 172.21.15.170:0/3783539546 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00acb59700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a80cfb40 0x7f00a80b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a77fe700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a80cf6d0 0x7f00a80b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 -- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a80cf6d0 msgr2=0x7f00a80b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a80cf6d0 0x7f00a80b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 -- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a80cfb40 msgr2=0x7f00a80b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a80cfb40 0x7f00a80b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 -- 172.21.15.170:0/3783539546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80ceed0 con 0x7f0090003bf0 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00a7fff700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f009800c3c0 tx=0x7f0098015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009800c5d0 con 0x7f0090003bf0 2023-10-08T16:05:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009800ce80 con 0x7f0090003bf0 2023-10-08T16:05:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a80cf1b0 con 0x7f0090003bf0 2023-10-08T16:05:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098012e50 con 0x7f0090003bf0 2023-10-08T16:05:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.021+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a80c7b80 con 0x7f0090003bf0 2023-10-08T16:05:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.022+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a80cea50 con 0x7f0090003bf0 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.022+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0098019070 con 0x7f0090003bf0 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.023+0000 7f008ffff700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 0x7f00940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.023+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f0098084fc0 con 0x7f0090003bf0 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.023+0000 7f00a77fe700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 0x7f00940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.023+0000 7f00a77fe700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 0x7f00940693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f009c006f00 tx=0x7f009c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.027+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00980b65a0 con 0x7f0090003bf0 2023-10-08T16:05:40.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.252+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00a804c3c0 con 0x7f0090003bf0 2023-10-08T16:05:40.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.252+0000 7f008ffff700 1 -- 172.21.15.170:0/3783539546 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f0098017090 con 0x7f0090003bf0 2023-10-08T16:05:40.253 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:40.253 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/931871174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":931871174},{"type":"v1","addr":"172.21.15.170:6836","nonce":931871174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:18.453208+0000","modified":"2023-10-08T16:05:22.437568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4884},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4884":{"gid":4884,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.170:6837/4215721719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":4215721719},{"type":"v1","addr":"172.21.15.170:6837","nonce":4215721719}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T16:05:40.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 msgr2=0x7f00940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 0x7f00940693e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f009c006f00 tx=0x7f009c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 msgr2=0x7f00a8149c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f009800c3c0 tx=0x7f0098015000 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 shutdown_connections 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00a80cfb40 0x7f00a80b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0094066f30 0x7f00940693e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0090003bf0 0x7f00a8149c20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 --2- 172.21.15.170:0/3783539546 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00a80cf6d0 0x7f00a80b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 >> 172.21.15.170:0/3783539546 conn(0x7f00a8068e20 msgr2=0x7f00a815bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 shutdown_connections 2023-10-08T16:05:40.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.255+0000 7f00ae5bc700 1 -- 172.21.15.170:0/3783539546 wait complete. 2023-10-08T16:05:40.257 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 22 2023-10-08T16:05:40.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.381+0000 7ff76db1f700 1 Processor -- start 2023-10-08T16:05:40.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.381+0000 7ff76db1f700 1 -- start start 2023-10-08T16:05:40.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff76db1f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff768159c60 0x7ff76815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff76db1f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff76815db10 0x7ff768069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff76db1f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76815a160 con 0x7ff768159890 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff76db1f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76815df50 con 0x7ff76815db10 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff76db1f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680d13a0 con 0x7ff768159c60 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff76815db10 0x7ff768069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff7677fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff768159890 0x7ff76815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35780/0 (socket says 172.21.15.170:35780) 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff7677fe700 1 -- 172.21.15.170:0/45095401 learned_addr learned my addr 172.21.15.170:0/45095401 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff766ffd700 1 --2- 172.21.15.170:0/45095401 >> v2:172.21.15.191:3300/0 conn(0x7ff768159c60 0x7ff76815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 -- 172.21.15.170:0/45095401 >> v2:172.21.15.191:3300/0 conn(0x7ff768159c60 msgr2=0x7ff76815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:40.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 --2- 172.21.15.170:0/45095401 >> v2:172.21.15.191:3300/0 conn(0x7ff768159c60 0x7ff76815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 -- 172.21.15.170:0/45095401 >> v1:172.21.15.170:6789/0 conn(0x7ff768159890 legacy=0x7ff76815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 -- 172.21.15.170:0/45095401 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76815e140 con 0x7ff76815db10 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff767fff700 1 --2- 172.21.15.170:0/45095401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76815db10 0x7ff768069560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff7680b67f0 tx=0x7ff760009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4500b0bd4ca51b02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff7667fc700 1 -- 172.21.15.170:0/45095401 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000e070 con 0x7ff76815db10 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff7667fc700 1 -- 172.21.15.170:0/45095401 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76000ba60 con 0x7ff76815db10 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.382+0000 7ff7667fc700 1 -- 172.21.15.170:0/45095401 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76000be20 con 0x7ff76815db10 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 -- 172.21.15.170:0/45095401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76815db10 msgr2=0x7ff768069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 --2- 172.21.15.170:0/45095401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76815db10 0x7ff768069560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff7680b67f0 tx=0x7ff760009780 comp rx=0 tx=0).stop 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 -- 172.21.15.170:0/45095401 shutdown_connections 2023-10-08T16:05:40.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 --2- 172.21.15.170:0/45095401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76815db10 0x7ff768069560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 --2- 172.21.15.170:0/45095401 >> v2:172.21.15.191:3300/0 conn(0x7ff768159c60 0x7ff76815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 -- 172.21.15.170:0/45095401 >> 172.21.15.170:0/45095401 conn(0x7ff768068e20 msgr2=0x7ff7680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 -- 172.21.15.170:0/45095401 shutdown_connections 2023-10-08T16:05:40.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.383+0000 7ff76db1f700 1 -- 172.21.15.170:0/45095401 wait complete. 2023-10-08T16:05:40.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.384+0000 7ff76db1f700 1 Processor -- start 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.384+0000 7ff76db1f700 1 -- start start 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff768159c60 0x7ff7680cece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 0x7ff768152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680cf220 con 0x7ff768159890 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff768152ba0 con 0x7ff768159c60 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff76db1f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680cf450 con 0x7ff76815db10 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45620/0 (socket says 172.21.15.170:45620) 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 -- 172.21.15.170:0/2580569532 learned_addr learned my addr 172.21.15.170:0/2580569532 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff767fff700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 0x7ff768152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff766ffd700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff768159c60 0x7ff7680cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 -- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 msgr2=0x7ff768152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 0x7ff768152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 -- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff768159c60 msgr2=0x7ff7680cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff768159c60 0x7ff7680cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 -- 172.21.15.170:0/2580569532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff760009a20 con 0x7ff768159890 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff767fff700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 0x7ff768152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:40.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.385+0000 7ff7677fe700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c810 tx=0x7ff758007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.386+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758005b30 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.386+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758005c90 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.386+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7680cf730 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.386+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758007340 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.386+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7680cfcf0 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.387+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76815d950 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.387+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff75801b400 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.387+0000 7ff74f7fe700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.387+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7ff7580841a0 con 0x7ff768159890 2023-10-08T16:05:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.387+0000 7ff766ffd700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.388+0000 7ff766ffd700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 0x7ff7540693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0093f0 tx=0x7ff75c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.391+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff758054ed0 con 0x7ff768159890 2023-10-08T16:05:40.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.576+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7680cff10 con 0x7ff768159890 2023-10-08T16:05:40.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.576+0000 7ff74f7fe700 1 -- 172.21.15.170:0/2580569532 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+15529 (secure 0 0 0) 0x7ff7580076f0 con 0x7ff768159890 2023-10-08T16:05:40.577 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:40.577 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:36.909591+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:16.184130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:17.417856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/2746556501":"2023-10-09T16:05:33.905200+0000","172.21.15.170:6834/2746556501":"2023-10-09T16:05:33.905200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:40.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 msgr2=0x7ff7540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 0x7ff7540693e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0093f0 tx=0x7ff75c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:40.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 msgr2=0x7ff7680cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c810 tx=0x7ff758007f30 comp rx=0 tx=0).stop 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 shutdown_connections 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76815db10 0x7ff768152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff754066f30 0x7ff7540693e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff768159890 0x7ff7680cdfc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 --2- 172.21.15.170:0/2580569532 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff768159c60 0x7ff7680cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 >> 172.21.15.170:0/2580569532 conn(0x7ff768068e20 msgr2=0x7ff768152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 shutdown_connections 2023-10-08T16:05:40.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.579+0000 7ff76db1f700 1 -- 172.21.15.170:0/2580569532 wait complete. 2023-10-08T16:05:40.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:05:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.707+0000 7f730c662700 1 Processor -- start 2023-10-08T16:05:40.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.707+0000 7f730c662700 1 -- start start 2023-10-08T16:05:40.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730c662700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f730415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730c662700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7304159c60 0x7f730415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730c662700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730415a160 con 0x7f730415db10 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730c662700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73040d13a0 con 0x7f7304159c60 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730c662700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730415e140 con 0x7f7304159890 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f7309bfd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7304159c60 0x7f730415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730a3fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f730415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730abff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f730415db10 0x7f7304069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36152/0 (socket says 172.21.15.170:36152) 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.708+0000 7f730abff700 1 -- 172.21.15.170:0/2327802811 learned_addr learned my addr 172.21.15.170:0/2327802811 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f73093fc700 1 -- 172.21.15.170:0/2327802811 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600901 0 0) 0x7f730415a160 con 0x7f730415db10 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f73093fc700 1 -- 172.21.15.170:0/2327802811 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72f0003570 con 0x7f730415db10 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f7309bfd700 1 -- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 msgr2=0x7f730415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:40.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f7309bfd700 1 --2- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f730415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f7309bfd700 1 -- 172.21.15.170:0/2327802811 >> v1:172.21.15.191:6789/0 conn(0x7f730415db10 legacy=0x7f7304069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f7309bfd700 1 -- 172.21.15.170:0/2327802811 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730415a3d0 con 0x7f7304159c60 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f7309bfd700 1 --2- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f730415d5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72f40098e0 tx=0x7f72f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23edb67907175221 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f73093fc700 1 -- 172.21.15.170:0/2327802811 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4011630 con 0x7f7304159c60 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f73093fc700 1 -- 172.21.15.170:0/2327802811 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f4011790 con 0x7f7304159c60 2023-10-08T16:05:40.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.709+0000 7f73093fc700 1 -- 172.21.15.170:0/2327802811 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4011b10 con 0x7f7304159c60 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 -- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 msgr2=0x7f730415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 --2- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f730415d5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f72f40098e0 tx=0x7f72f40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 -- 172.21.15.170:0/2327802811 shutdown_connections 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 --2- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f730415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 --2- 172.21.15.170:0/2327802811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f730415d5d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 -- 172.21.15.170:0/2327802811 >> 172.21.15.170:0/2327802811 conn(0x7f7304068e20 msgr2=0x7f73040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 -- 172.21.15.170:0/2327802811 shutdown_connections 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.710+0000 7f730c662700 1 -- 172.21.15.170:0/2327802811 wait complete. 2023-10-08T16:05:40.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.711+0000 7f730c662700 1 Processor -- start 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.711+0000 7f730c662700 1 -- start start 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f73040b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f7304143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304157bc0 con 0x7f730415db10 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304156fb0 con 0x7f7304159c60 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7304157290 con 0x7f7304159890 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45626/0 (socket says 172.21.15.170:45626) 2023-10-08T16:05:40.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 -- 172.21.15.170:0/555017514 learned_addr learned my addr 172.21.15.170:0/555017514 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730a3fe700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f73040b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f7309bfd700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f7304143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 -- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 msgr2=0x7f73040b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f73040b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 -- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 msgr2=0x7f7304143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f7304143d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 -- 172.21.15.170:0/555017514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4009420 con 0x7f730415db10 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730abff700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72fc001460 tx=0x7f72fc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00a870 con 0x7f730415db10 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00a9d0 con 0x7f730415db10 2023-10-08T16:05:40.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7304157570 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00ad50 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.712+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73040b3c80 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.713+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730404c3c0 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.714+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72fc007300 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.714+0000 7f72fa7fc700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 0x7f72e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.714+0000 7f730a3fe700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 0x7f72e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.714+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f72fc0848b0 con 0x7f730415db10 2023-10-08T16:05:40.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.715+0000 7f730a3fe700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 0x7f72e80693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7304167730 tx=0x7f73000091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:40.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.719+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72fc051ac0 con 0x7f730415db10 2023-10-08T16:05:40.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.948+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f730406a5b0 con 0x7f730415db10 2023-10-08T16:05:40.948 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:05:40.948+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:05:40.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.952+0000 7f72fa7fc700 1 -- 172.21.15.170:0/555017514 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72fc0550e0 con 0x7f730415db10 2023-10-08T16:05:40.953 INFO:tasks.ceph.mds.c.smithi170.stderr: -1131> 2023-10-08T16:05:21.991+0000 7fe8dee1b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:05:40.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 msgr2=0x7f72e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 0x7f72e80693e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7304167730 tx=0x7f73000091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 msgr2=0x7f73040b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f72fc001460 tx=0x7f72fc001a40 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 shutdown_connections 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066f30 0x7f72e80693e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415db10 0x7f73040b0c70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7304159890 0x7f73040b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 --2- 172.21.15.170:0/555017514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159c60 0x7f7304143d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 >> 172.21.15.170:0/555017514 conn(0x7f7304068e20 msgr2=0x7f730415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:40.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 shutdown_connections 2023-10-08T16:05:40.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:40.954+0000 7f730c662700 1 -- 172.21.15.170:0/555017514 wait complete. 2023-10-08T16:05:40.956 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:05:40.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:05:40.975 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:05:41.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.083+0000 7fea518e0700 1 Processor -- start 2023-10-08T16:05:41.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.083+0000 7fea518e0700 1 -- start start 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea518e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea518e0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea4c06c0a0 0x7fea4c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea518e0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c15c2a0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea518e0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0c9880 con 0x7fea4c06c0a0 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea518e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c15b690 con 0x7fea4c141500 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fea4c06bcd0 0x7fea4c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35792/0 (socket says 172.21.15.170:35792) 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4affd700 1 -- 172.21.15.170:0/1850529742 learned_addr learned my addr 172.21.15.170:0/1850529742 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4a7fc700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 --2- 172.21.15.170:0/1850529742 >> v2:172.21.15.191:3300/0 conn(0x7fea4c06c0a0 0x7fea4c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea49ffb700 1 -- 172.21.15.170:0/1850529742 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041025904 0 0) 0x7fea4c15c2a0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea49ffb700 1 -- 172.21.15.170:0/1850529742 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003570 con 0x7fea4c06bcd0 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 -- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 msgr2=0x7fea4c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 -- 172.21.15.170:0/1850529742 >> v1:172.21.15.170:6789/0 conn(0x7fea4c06bcd0 legacy=0x7fea4c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 -- 172.21.15.170:0/1850529742 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c15b970 con 0x7fea4c06c0a0 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4a7fc700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.084+0000 7fea4b7fe700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0c9340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea440098e0 tx=0x7fea440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b57b4bea8da150a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:41.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.085+0000 7fea49ffb700 1 -- 172.21.15.170:0/1850529742 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011630 con 0x7fea4c06c0a0 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea49ffb700 1 -- 172.21.15.170:0/1850529742 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea44011790 con 0x7fea4c06c0a0 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea49ffb700 1 -- 172.21.15.170:0/1850529742 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea44011b10 con 0x7fea4c06c0a0 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 -- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 msgr2=0x7fea4c0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0c9340 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea440098e0 tx=0x7fea440091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 -- 172.21.15.170:0/1850529742 shutdown_connections 2023-10-08T16:05:41.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0c9340 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 --2- 172.21.15.170:0/1850529742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 -- 172.21.15.170:0/1850529742 >> 172.21.15.170:0/1850529742 conn(0x7fea4c0ce600 msgr2=0x7fea4c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 -- 172.21.15.170:0/1850529742 shutdown_connections 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.086+0000 7fea518e0700 1 -- 172.21.15.170:0/1850529742 wait complete. 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.087+0000 7fea518e0700 1 Processor -- start 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 -- start start 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c56b0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c5970 con 0x7fea4c06c0a0 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea518e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c8130 con 0x7fea4c141500 2023-10-08T16:05:41.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea4affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea4affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45660/0 (socket says 172.21.15.170:45660) 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea4affd700 1 -- 172.21.15.170:0/2913978977 learned_addr learned my addr 172.21.15.170:0/2913978977 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea4a7fc700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.088+0000 7fea4b7fe700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 -- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 msgr2=0x7fea4c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 -- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 msgr2=0x7fea4c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 -- 172.21.15.170:0/2913978977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34009420 con 0x7fea4c06bcd0 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea4affd700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00cd00 tx=0x7fea3c001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:41.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00ce60 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c00aa60 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea44009420 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00ae00 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.089+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0c7d70 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.090+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c0b1db0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.090+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea3c007300 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.091+0000 7fea42ffd700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.091+0000 7fea4a7fc700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.091+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fea3c084940 con 0x7fea4c06bcd0 2023-10-08T16:05:41.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.091+0000 7fea4a7fc700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 0x7fea2c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea4c13b980 tx=0x7fea340091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:41.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.095+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea3c055610 con 0x7fea4c06bcd0 2023-10-08T16:05:41.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.318+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fea4c04c3c0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.952+0000 7fea42ffd700 1 -- 172.21.15.170:0/2913978977 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea3c0551a0 con 0x7fea4c06bcd0 2023-10-08T16:05:41.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 msgr2=0x7fea2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 0x7fea2c0693e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea4c13b980 tx=0x7fea340091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 msgr2=0x7fea4c155290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00cd00 tx=0x7fea3c001b00 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 shutdown_connections 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea4c141500 0x7fea4c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea2c066f30 0x7fea2c0693e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea4c06bcd0 0x7fea4c155290 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 --2- 172.21.15.170:0/2913978977 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea4c06c0a0 0x7fea4c0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 >> 172.21.15.170:0/2913978977 conn(0x7fea4c0ce600 msgr2=0x7fea4c0c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 shutdown_connections 2023-10-08T16:05:41.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:41.954+0000 7fea518e0700 1 -- 172.21.15.170:0/2913978977 wait complete. 2023-10-08T16:05:41.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:05:42.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.129+0000 7f965a0e5700 1 Processor -- start 2023-10-08T16:05:42.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.129+0000 7f965a0e5700 1 -- start start 2023-10-08T16:05:42.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f965a0e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b0790 0x7f96540cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f965a0e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96541591e0 0x7f9654159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f965a0e5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540cfa00 con 0x7f96540b03c0 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f965a0e5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540cb580 con 0x7f96541591e0 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f965a0e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540cf160 con 0x7f96540b0790 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96541591e0 0x7f9654159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96541591e0 0x7f9654159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45666/0 (socket says 172.21.15.170:45666) 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 -- 172.21.15.170:0/3208443924 learned_addr learned my addr 172.21.15.170:0/3208443924 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96590e3700 1 --1- 172.21.15.170:0/3208443924 >> v1:172.21.15.170:6789/0 conn(0x7f96540b03c0 0x7f96540b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35796/0 (socket says 172.21.15.170:35796) 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96588e2700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b0790 0x7f96540cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 -- 172.21.15.170:0/3208443924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b0790 msgr2=0x7f96540cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b0790 0x7f96540cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 -- 172.21.15.170:0/3208443924 >> v1:172.21.15.170:6789/0 conn(0x7f96540b03c0 legacy=0x7f96540b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f964bfff700 1 -- 172.21.15.170:0/3208443924 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419577021 0 0) 0x7f96540cfa00 con 0x7f96540b03c0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 -- 172.21.15.170:0/3208443924 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540cf440 con 0x7f96541591e0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.130+0000 7f96598e4700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96541591e0 0x7f9654159620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964c009f20 tx=0x7f964c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241cedaa3dfb597b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f964bfff700 1 -- 172.21.15.170:0/3208443924 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c00e070 con 0x7f96541591e0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f964bfff700 1 -- 172.21.15.170:0/3208443924 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964c00ba80 con 0x7f96541591e0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f964bfff700 1 -- 172.21.15.170:0/3208443924 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c0092e0 con 0x7f96541591e0 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 -- 172.21.15.170:0/3208443924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96541591e0 msgr2=0x7f9654159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96541591e0 0x7f9654159620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f964c009f20 tx=0x7f964c009840 comp rx=0 tx=0).stop 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 -- 172.21.15.170:0/3208443924 shutdown_connections 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96541591e0 0x7f9654159620 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3208443924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b0790 0x7f96540cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 -- 172.21.15.170:0/3208443924 >> 172.21.15.170:0/3208443924 conn(0x7f9654063230 msgr2=0x7f9654063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:42.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 -- 172.21.15.170:0/3208443924 shutdown_connections 2023-10-08T16:05:42.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.131+0000 7f965a0e5700 1 -- 172.21.15.170:0/3208443924 wait complete. 2023-10-08T16:05:42.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.132+0000 7f965a0e5700 1 Processor -- start 2023-10-08T16:05:42.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.132+0000 7f965a0e5700 1 -- start start 2023-10-08T16:05:42.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b03c0 0x7f9654149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96541591e0 0x7f96540c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540ca600 con 0x7f96540b0790 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c5560 con 0x7f96541591e0 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f965a0e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96540c5280 con 0x7f96540b03c0 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f96588e2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f96588e2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45678/0 (socket says 172.21.15.170:45678) 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.133+0000 7f96588e2700 1 -- 172.21.15.170:0/3017396569 learned_addr learned my addr 172.21.15.170:0/3017396569 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 -- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b03c0 msgr2=0x7f9654149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b03c0 0x7f9654149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 -- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96541591e0 msgr2=0x7f96540c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96541591e0 0x7f96540c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 -- 172.21.15.170:0/3017396569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c009ae0 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96588e2700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f964400d890 tx=0x7f9644008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400b600 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964400b760 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96540c86a0 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964400bae0 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.134+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96540c8c60 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.135+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965404c3c0 con 0x7f96540b0790 2023-10-08T16:05:42.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.135+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f964401e460 con 0x7f96540b0790 2023-10-08T16:05:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.136+0000 7f96497fa700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.136+0000 7f96590e3700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.136+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f9644084cc0 con 0x7f96540b0790 2023-10-08T16:05:42.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.136+0000 7f96590e3700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 0x7f96380693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9650001660 tx=0x7f9650007b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:42.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.140+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9644051e80 con 0x7f96540b0790 2023-10-08T16:05:42.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.323+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f96540cad50 con 0x7f96540b0790 2023-10-08T16:05:42.922 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:05:42.924 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:05:42.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.957+0000 7f96497fa700 1 -- 172.21.15.170:0/3017396569 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f96440554a0 con 0x7f96540b0790 2023-10-08T16:05:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 msgr2=0x7f96380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 0x7f96380693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9650001660 tx=0x7f9650007b70 comp rx=0 tx=0).stop 2023-10-08T16:05:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 msgr2=0x7f9654169580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:42.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f964400d890 tx=0x7f9644008040 comp rx=0 tx=0).stop 2023-10-08T16:05:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 shutdown_connections 2023-10-08T16:05:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96541591e0 0x7f96540c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96540b03c0 0x7f9654149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9638066f30 0x7f96380693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 --2- 172.21.15.170:0/3017396569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96540b0790 0x7f9654169580 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 >> 172.21.15.170:0/3017396569 conn(0x7f9654063230 msgr2=0x7f96540d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.960+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 shutdown_connections 2023-10-08T16:05:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:42.961+0000 7f965a0e5700 1 -- 172.21.15.170:0/3017396569 wait complete. 2023-10-08T16:05:42.962 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:05:42.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:05:43.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.140+0000 7fab6359e700 1 Processor -- start 2023-10-08T16:05:43.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.140+0000 7fab6359e700 1 -- start start 2023-10-08T16:05:43.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fab640b03c0 0x7fab640cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 0x7fab640b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 0x7fab64159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab640cfa00 con 0x7fab640b03c0 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab640cb580 con 0x7fab640b0790 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab640b2de0 con 0x7fab641591e0 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fab640b03c0 0x7fab640cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fab640b03c0 0x7fab640cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45684/0 (socket says 172.21.15.170:45684) 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 -- 172.21.15.170:0/440981315 learned_addr learned my addr 172.21.15.170:0/440981315 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab62d9d700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 0x7fab64159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab59d9b700 1 --2- 172.21.15.170:0/440981315 >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 0x7fab640b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 -- 172.21.15.170:0/440981315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 msgr2=0x7fab64159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 0x7fab64159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 -- 172.21.15.170:0/440981315 >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 msgr2=0x7fab640b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 --2- 172.21.15.170:0/440981315 >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 0x7fab640b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 -- 172.21.15.170:0/440981315 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab640cf160 con 0x7fab640b03c0 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab59d9b700 1 --2- 172.21.15.170:0/440981315 >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 0x7fab640b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab62d9d700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 0x7fab64159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.141+0000 7fab6259c700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab640b03c0 0x7fab640cea70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab540099b0 tx=0x7fab54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7d83072c551485 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab61d9b700 1 -- 172.21.15.170:0/440981315 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400b820 con 0x7fab640b03c0 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab61d9b700 1 -- 172.21.15.170:0/440981315 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5400b980 con 0x7fab640b03c0 2023-10-08T16:05:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab61d9b700 1 -- 172.21.15.170:0/440981315 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400bd20 con 0x7fab640b03c0 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 -- 172.21.15.170:0/440981315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab640b03c0 msgr2=0x7fab640cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab640b03c0 0x7fab640cea70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fab540099b0 tx=0x7fab54009250 comp rx=0 tx=0).stop 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 -- 172.21.15.170:0/440981315 shutdown_connections 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab641591e0 0x7fab64159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 --2- 172.21.15.170:0/440981315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab640b03c0 0x7fab640cea70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 --2- 172.21.15.170:0/440981315 >> v2:172.21.15.191:3300/0 conn(0x7fab640b0790 0x7fab640b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 -- 172.21.15.170:0/440981315 >> 172.21.15.170:0/440981315 conn(0x7fab64063230 msgr2=0x7fab64063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 -- 172.21.15.170:0/440981315 shutdown_connections 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.142+0000 7fab6359e700 1 -- 172.21.15.170:0/440981315 wait complete. 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.143+0000 7fab6359e700 1 Processor -- start 2023-10-08T16:05:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.143+0000 7fab6359e700 1 -- start start 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab640b03c0 0x7fab640c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 0x7fab6415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6415b140 con 0x7fab641591e0 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6415ae60 con 0x7fab640b0790 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab640c5990 con 0x7fab640b03c0 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45700/0 (socket says 172.21.15.170:45700) 2023-10-08T16:05:43.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 -- 172.21.15.170:0/2295515090 learned_addr learned my addr 172.21.15.170:0/2295515090 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab6259c700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab640b03c0 0x7fab640c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab59d9b700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 0x7fab6415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 -- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab640b03c0 msgr2=0x7fab640c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab640b03c0 0x7fab640c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 -- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 msgr2=0x7fab6415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 0x7fab6415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab62d9d700 1 -- 172.21.15.170:0/2295515090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab540094f0 con 0x7fab641591e0 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.144+0000 7fab59d9b700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 0x7fab6415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab62d9d700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007f30 tx=0x7fab5c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:43.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00dce0 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00f040 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab640c5c70 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c015690 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.145+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab640c4cb0 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.146+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab640c57f0 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.146+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab5c01e3b0 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.146+0000 7fab5affd700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.146+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5439+0+0 (secure 0 0 0) 0x7fab5c084900 con 0x7fab641591e0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.146+0000 7fab6259c700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:43.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.147+0000 7fab6259c700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab540099b0 tx=0x7fab54017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:43.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.151+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab640c57f0 con 0x7fab641591e0 2023-10-08T16:05:43.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.334+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab640c6190 con 0x7fab641591e0 2023-10-08T16:05:43.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.964+0000 7fab5affd700 1 -- 172.21.15.170:0/2295515090 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7fab640c6190 con 0x7fab641591e0 2023-10-08T16:05:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.969+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 msgr2=0x7fab440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.969+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 0x7fab440693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fab540099b0 tx=0x7fab54017000 comp rx=0 tx=0).stop 2023-10-08T16:05:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.969+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 msgr2=0x7fab640c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:43.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.969+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007f30 tx=0x7fab5c009930 comp rx=0 tx=0).stop 2023-10-08T16:05:43.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 shutdown_connections 2023-10-08T16:05:43.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44066f30 0x7fab440693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab641591e0 0x7fab640c6470 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab640b03c0 0x7fab640c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 --2- 172.21.15.170:0/2295515090 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab640b0790 0x7fab6415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:43.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 >> 172.21.15.170:0/2295515090 conn(0x7fab64063230 msgr2=0x7fab640d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:43.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 shutdown_connections 2023-10-08T16:05:43.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:43.970+0000 7fab6359e700 1 -- 172.21.15.170:0/2295515090 wait complete. 2023-10-08T16:05:43.977 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:05:43.988 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:05:44.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.103+0000 7f71bce08700 1 Processor -- start 2023-10-08T16:05:44.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.103+0000 7f71bce08700 1 -- start start 2023-10-08T16:05:44.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71bce08700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71bce08700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b815a160 con 0x7f71b8159890 2023-10-08T16:05:44.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71bce08700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b80d13a0 con 0x7f71b815db10 2023-10-08T16:05:44.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71bce08700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b815e140 con 0x7f71b8159c60 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71b659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f71b8159890 0x7f71b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35800/0 (socket says 172.21.15.170:35800) 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71b659c700 1 -- 172.21.15.170:0/3349663704 learned_addr learned my addr 172.21.15.170:0/3349663704 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565191339 0 0) 0x7f71b815a160 con 0x7f71b8159890 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71a0003570 con 0x7f71b8159890 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.104+0000 7f71b5d9b700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1832013730 0 0) 0x7f71a0003570 con 0x7f71b8159890 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 msgr2=0x7f71b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 >> v1:172.21.15.191:6789/0 conn(0x7f71b815db10 legacy=0x7f71b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b815a3d0 con 0x7f71b8159890 2023-10-08T16:05:44.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f71a80017f0 con 0x7f71b8159890 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 >> v1:172.21.15.170:6789/0 conn(0x7f71b8159890 legacy=0x7f71b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a0003a90 0x7f71a0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a0004d50 0x7f71a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b815e140 con 0x7f71a0003a90 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a000bb70 con 0x7f71a0004d50 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a000bd30 con 0x7f71b8159c60 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a0003a90 0x7f71a0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b5d9b700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b6d9d700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a0004d50 0x7f71a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 -- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 msgr2=0x7f71a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:44.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71b8159c60 0x7f71a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 -- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a0004d50 msgr2=0x7f71a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71a0004d50 0x7f71a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 -- 172.21.15.170:0/3349663704 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0003570 con 0x7f71a0003a90 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b659c700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a0003a90 0x7f71a0003ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f71a800a930 tx=0x7f71a80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bbb756a115a211 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a80070d0 con 0x7f71a0003a90 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8007230 con 0x7f71a0003a90 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.105+0000 7f71b559a700 1 -- 172.21.15.170:0/3349663704 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a80075d0 con 0x7f71a0003a90 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 -- 172.21.15.170:0/3349663704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a0003a90 msgr2=0x7f71a0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 --2- 172.21.15.170:0/3349663704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71a0003a90 0x7f71a0003ea0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f71a800a930 tx=0x7f71a80021d0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71b6d9d700 1 -- 172.21.15.170:0/3349663704 reap_dead start 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 -- 172.21.15.170:0/3349663704 shutdown_connections 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 -- 172.21.15.170:0/3349663704 >> 172.21.15.170:0/3349663704 conn(0x7f71b8068e20 msgr2=0x7f71b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 -- 172.21.15.170:0/3349663704 shutdown_connections 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.106+0000 7f71bce08700 1 -- 172.21.15.170:0/3349663704 wait complete. 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.107+0000 7f71bce08700 1 Processor -- start 2023-10-08T16:05:44.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.107+0000 7f71bce08700 1 -- start start 2023-10-08T16:05:44.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a0003a90 0x7f71b814d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71b81528e0 0x7f71b8152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b5730 con 0x7f71b80b4b70 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b80b5520 con 0x7f71b81528e0 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b816e270 con 0x7f71a0003a90 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45712/0 (socket says 172.21.15.170:45712) 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 -- 172.21.15.170:0/4234512586 learned_addr learned my addr 172.21.15.170:0/4234512586 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b659c700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a0003a90 0x7f71b814d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b6d9d700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71b81528e0 0x7f71b8152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 -- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a0003a90 msgr2=0x7f71b814d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a0003a90 0x7f71b814d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 -- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71b81528e0 msgr2=0x7f71b8152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71b81528e0 0x7f71b8152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 -- 172.21.15.170:0/4234512586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b816dde0 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71b5d9b700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f71ac011b70 tx=0x7f71ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00f940 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac00faa0 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b8153270 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.108+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00fe20 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.109+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b816d4b0 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.109+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b816df70 con 0x7f71b80b4b70 2023-10-08T16:05:44.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.110+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71ac014650 con 0x7f71b80b4b70 2023-10-08T16:05:44.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.110+0000 7f71a67fc700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 0x7f7198069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.110+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f71ac018030 con 0x7f71b80b4b70 2023-10-08T16:05:44.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.110+0000 7f71b659c700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 0x7f7198069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.111+0000 7f71b659c700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 0x7f7198069410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71a800afd0 tx=0x7f71a8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.114+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71ac052040 con 0x7f71b80b4b70 2023-10-08T16:05:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.299+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f71b804c3c0 con 0x7f71b80b4b70 2023-10-08T16:05:44.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.302+0000 7f71a67fc700 1 -- 172.21.15.170:0/4234512586 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71ac055660 con 0x7f71b80b4b70 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 msgr2=0x7f7198069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 0x7f7198069410 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f71a800afd0 tx=0x7f71a8016000 comp rx=0 tx=0).stop 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 msgr2=0x7f71b80b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f71ac011b70 tx=0x7f71ac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 shutdown_connections 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71b81528e0 0x7f71b8152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71a0003a90 0x7f71b814d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7198066f60 0x7f7198069410 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 --2- 172.21.15.170:0/4234512586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b80b4b70 0x7f71b80b4fe0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 >> 172.21.15.170:0/4234512586 conn(0x7f71b8068e20 msgr2=0x7f71b815aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 shutdown_connections 2023-10-08T16:05:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.304+0000 7f71bce08700 1 -- 172.21.15.170:0/4234512586 wait complete. 2023-10-08T16:05:44.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create' 2023-10-08T16:05:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 Processor -- start 2023-10-08T16:05:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 -- start start 2023-10-08T16:05:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f916c159890 0x7f916c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c069560 0x7f916c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c069ee0 con 0x7f916c159c60 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916c15a160 con 0x7f916c159890 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.432+0000 7f9174609700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0d13a0 con 0x7f916c069560 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f916c159890 0x7f916c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f9171ba4700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f916c159c60 0x7f916c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36176/0 (socket says 172.21.15.170:36176) 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f9171ba4700 1 -- 172.21.15.170:0/2674114611 learned_addr learned my addr 172.21.15.170:0/2674114611 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f9172ba6700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c069560 0x7f916c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 -- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c069560 msgr2=0x7f916c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c069560 0x7f916c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 -- 172.21.15.170:0/2674114611 >> v1:172.21.15.191:6789/0 conn(0x7f916c159c60 legacy=0x7f916c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 -- 172.21.15.170:0/2674114611 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c15e140 con 0x7f916c159890 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91713a3700 1 -- 172.21.15.170:0/2674114611 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506901843 0 0) 0x7f916c069ee0 con 0x7f916c159c60 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91723a5700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159890 0x7f916c15abd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f91680098e0 tx=0x7f91680091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82a48a47f688adc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.433+0000 7f91713a3700 1 -- 172.21.15.170:0/2674114611 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800b640 con 0x7f916c159890 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f91713a3700 1 -- 172.21.15.170:0/2674114611 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916800b7a0 con 0x7f916c159890 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f91713a3700 1 -- 172.21.15.170:0/2674114611 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800bb40 con 0x7f916c159890 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 -- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159890 msgr2=0x7f916c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159890 0x7f916c15abd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f91680098e0 tx=0x7f91680091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 -- 172.21.15.170:0/2674114611 shutdown_connections 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c069560 0x7f916c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 --2- 172.21.15.170:0/2674114611 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159890 0x7f916c15abd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 -- 172.21.15.170:0/2674114611 >> 172.21.15.170:0/2674114611 conn(0x7f916c068e20 msgr2=0x7f916c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:44.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 -- 172.21.15.170:0/2674114611 shutdown_connections 2023-10-08T16:05:44.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.434+0000 7f9174609700 1 -- 172.21.15.170:0/2674114611 wait complete. 2023-10-08T16:05:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.435+0000 7f9174609700 1 Processor -- start 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.435+0000 7f9174609700 1 -- start start 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c159890 0x7f916c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159c60 0x7f916c149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0b6520 con 0x7f916c069560 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c7ec0 con 0x7f916c159c60 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9174609700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cd880 con 0x7f916c159890 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45720/0 (socket says 172.21.15.170:45720) 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 -- 172.21.15.170:0/1497803306 learned_addr learned my addr 172.21.15.170:0/1497803306 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9172ba6700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159c60 0x7f916c149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f9171ba4700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c159890 0x7f916c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 -- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c159890 msgr2=0x7f916c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c159890 0x7f916c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 -- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159c60 msgr2=0x7f916c149390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159c60 0x7f916c149390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.436+0000 7f91723a5700 1 -- 172.21.15.170:0/1497803306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168009420 con 0x7f916c069560 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f91723a5700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9168009f80 tx=0x7f9168017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916800bb20 con 0x7f916c069560 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168000c90 con 0x7f916c069560 2023-10-08T16:05:44.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0c7940 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168011650 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.437+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0c9600 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.438+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c04c3c0 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.438+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9168011a00 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.438+0000 7f91627fc700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 0x7f91500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.439+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f9168085960 con 0x7f916c069560 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.439+0000 7f9171ba4700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 0x7f91500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:44.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.439+0000 7f9171ba4700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 0x7f91500693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f916c0cef50 tx=0x7f915c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:44.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.443+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9168052dd0 con 0x7f916c069560 2023-10-08T16:05:44.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.701+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]} v 0) v1 -- 0x7f916c06a5b0 con 0x7f916c069560 2023-10-08T16:05:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.965+0000 7f91627fc700 1 -- 172.21.15.170:0/1497803306 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_create"]}]=0 v83) v1 ==== 135+0+0 (secure 0 0 0) 0x7f91680150c0 con 0x7f916c069560 2023-10-08T16:05:44.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 msgr2=0x7f91500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 0x7f91500693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f916c0cef50 tx=0x7f915c007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 msgr2=0x7f916c0b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9168009f80 tx=0x7f9168017000 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 shutdown_connections 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f916c159c60 0x7f916c149390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9150066f30 0x7f91500693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f916c069560 0x7f916c0b5fe0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 --2- 172.21.15.170:0/1497803306 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f916c159890 0x7f916c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 >> 172.21.15.170:0/1497803306 conn(0x7f916c068e20 msgr2=0x7f916c153300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:44.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 shutdown_connections 2023-10-08T16:05:44.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:44.967+0000 7f9174609700 1 -- 172.21.15.170:0/1497803306 wait complete. 2023-10-08T16:05:44.980 INFO:tasks.cephfs_test_runner:That the volume can be created and then cleans up ... ok 2023-10-08T16:05:44.980 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command 2023-10-08T16:05:44.980 INFO:tasks.cephfs_test_runner:test_volume_info (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:05:44.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-08T16:05:45.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 Processor -- start 2023-10-08T16:05:45.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 -- start start 2023-10-08T16:05:45.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a10159c60 0x7f7a1015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:45.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1015a160 con 0x7f7a10159890 2023-10-08T16:05:45.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a100d13a0 con 0x7f7a1015db10 2023-10-08T16:05:45.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.094+0000 7f7a1722f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1015df50 con 0x7f7a10159c60 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a10159c60 0x7f7a1015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a14fcb700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7a10159890 0x7f7a1015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35806/0 (socket says 172.21.15.170:35806) 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a10159c60 0x7f7a1015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45724/0 (socket says 172.21.15.170:45724) 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a14fcb700 1 -- 172.21.15.170:0/2559187440 learned_addr learned my addr 172.21.15.170:0/2559187440 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 -- 172.21.15.170:0/2559187440 >> v1:172.21.15.191:6789/0 conn(0x7f7a1015db10 legacy=0x7f7a10069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 -- 172.21.15.170:0/2559187440 >> v1:172.21.15.170:6789/0 conn(0x7f7a10159890 legacy=0x7f7a1015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a077fe700 1 -- 172.21.15.170:0/2559187440 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058959046 0 0) 0x7f7a1015a160 con 0x7f7a10159890 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 -- 172.21.15.170:0/2559187440 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1015e140 con 0x7f7a10159c60 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a07fff700 1 --2- 172.21.15.170:0/2559187440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159c60 0x7f7a1015d5d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a000098e0 tx=0x7f7a000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519275ca069dc4a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a077fe700 1 -- 172.21.15.170:0/2559187440 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00009e80 con 0x7f7a10159c60 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a077fe700 1 -- 172.21.15.170:0/2559187440 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0000b860 con 0x7f7a10159c60 2023-10-08T16:05:45.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.095+0000 7f7a077fe700 1 -- 172.21.15.170:0/2559187440 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000bbe0 con 0x7f7a10159c60 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 -- 172.21.15.170:0/2559187440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159c60 msgr2=0x7f7a1015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 --2- 172.21.15.170:0/2559187440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159c60 0x7f7a1015d5d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7a000098e0 tx=0x7f7a000091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 -- 172.21.15.170:0/2559187440 shutdown_connections 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 --2- 172.21.15.170:0/2559187440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159c60 0x7f7a1015d5d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 -- 172.21.15.170:0/2559187440 >> 172.21.15.170:0/2559187440 conn(0x7f7a10068e20 msgr2=0x7f7a100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 -- 172.21.15.170:0/2559187440 shutdown_connections 2023-10-08T16:05:45.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.096+0000 7f7a1722f700 1 -- 172.21.15.170:0/2559187440 wait complete. 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.097+0000 7f7a1722f700 1 Processor -- start 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.097+0000 7f7a1722f700 1 -- start start 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 0x7f7a10152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 0x7f7a100ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1015bfa0 con 0x7f7a10159890 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a100ca7a0 con 0x7f7a10159c60 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a1722f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1015b670 con 0x7f7a1015db10 2023-10-08T16:05:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:45730/0 (socket says 172.21.15.170:45730) 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 -- 172.21.15.170:0/1770820241 learned_addr learned my addr 172.21.15.170:0/1770820241 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a07fff700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 0x7f7a10152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 -- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 msgr2=0x7f7a100ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a157cc700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 0x7f7a100ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 0x7f7a100ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 -- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 msgr2=0x7f7a10152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 0x7f7a10152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 -- 172.21.15.170:0/1770820241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00009420 con 0x7f7a10159890 2023-10-08T16:05:45.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a07fff700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 0x7f7a10152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a157cc700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 0x7f7a100ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a14fcb700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00ccc0 tx=0x7f7a0c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.098+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00ab70 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.099+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00acd0 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.099+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1015b950 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.099+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c007340 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.099+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a100b0770 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.099+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1004c3c0 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.100+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a0c007770 con 0x7f7a10159890 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.100+0000 7f7a04ff9700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 0x7f79fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.100+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f7a0c084750 con 0x7f7a10159890 2023-10-08T16:05:45.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.100+0000 7f7a07fff700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 0x7f79fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:45.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.101+0000 7f7a07fff700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 0x7f79fc0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a1016d040 tx=0x7f7a00017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:45.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.104+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a0c051ce0 con 0x7f7a10159890 2023-10-08T16:05:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.363+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f7a10156e90 con 0x7f7a10159890 2023-10-08T16:05:45.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.966+0000 7f7a04ff9700 1 -- 172.21.15.170:0/1770820241 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v84) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7a0c055300 con 0x7f7a10159890 2023-10-08T16:05:45.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 msgr2=0x7f79fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 0x7f79fc0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a1016d040 tx=0x7f7a00017000 comp rx=0 tx=0).stop 2023-10-08T16:05:45.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 msgr2=0x7f7a1016dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00ccc0 tx=0x7f7a0c001a80 comp rx=0 tx=0).stop 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 shutdown_connections 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a1015db10 0x7f7a100ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79fc066f30 0x7f79fc0693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a10159890 0x7f7a1016dcb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 --2- 172.21.15.170:0/1770820241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a10159c60 0x7f7a10152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 >> 172.21.15.170:0/1770820241 conn(0x7f7a10068e20 msgr2=0x7f7a10153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 shutdown_connections 2023-10-08T16:05:45.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:45.969+0000 7f7a1722f700 1 -- 172.21.15.170:0/1770820241 wait complete. 2023-10-08T16:05:45.982 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:05:46.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.100+0000 7f3b8b8be700 1 Processor -- start 2023-10-08T16:05:46.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.100+0000 7f3b8b8be700 1 -- start start 2023-10-08T16:05:46.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8b8be700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3b84159890 0x7f3b8415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8b8be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b84069560 0x7f3b840699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8b8be700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b84069ee0 con 0x7f3b84159c60 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8b8be700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8415a160 con 0x7f3b84069560 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8b8be700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840d13a0 con 0x7f3b84159890 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b84069560 0x7f3b840699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b84069560 0x7f3b840699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38554/0 (socket says 172.21.15.170:38554) 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 -- 172.21.15.170:0/68039828 learned_addr learned my addr 172.21.15.170:0/68039828 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b8965a700 1 --2- 172.21.15.170:0/68039828 >> v2:172.21.15.191:3300/0 conn(0x7f3b84159890 0x7f3b8415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 -- 172.21.15.170:0/68039828 >> v2:172.21.15.191:3300/0 conn(0x7f3b84159890 msgr2=0x7f3b8415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 --2- 172.21.15.170:0/68039828 >> v2:172.21.15.191:3300/0 conn(0x7f3b84159890 0x7f3b8415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 -- 172.21.15.170:0/68039828 >> v1:172.21.15.191:6789/0 conn(0x7f3b84159c60 legacy=0x7f3b8415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 -- 172.21.15.170:0/68039828 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8415e140 con 0x7f3b84069560 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b89e5b700 1 --2- 172.21.15.170:0/68039828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840699a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3b840d1090 tx=0x7f3b7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2abff80c979f4d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b7bfff700 1 -- 172.21.15.170:0/68039828 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00e070 con 0x7f3b84069560 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b7bfff700 1 -- 172.21.15.170:0/68039828 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00ba50 con 0x7f3b84069560 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.101+0000 7f3b7bfff700 1 -- 172.21.15.170:0/68039828 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c0092e0 con 0x7f3b84069560 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 -- 172.21.15.170:0/68039828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 msgr2=0x7f3b840699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/68039828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840699a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3b840d1090 tx=0x7f3b7c009840 comp rx=0 tx=0).stop 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 -- 172.21.15.170:0/68039828 shutdown_connections 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/68039828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840699a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/68039828 >> v2:172.21.15.191:3300/0 conn(0x7f3b84159890 0x7f3b8415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 -- 172.21.15.170:0/68039828 >> 172.21.15.170:0/68039828 conn(0x7f3b84068e20 msgr2=0x7f3b840d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 -- 172.21.15.170:0/68039828 shutdown_connections 2023-10-08T16:05:46.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.102+0000 7f3b8b8be700 1 -- 172.21.15.170:0/68039828 wait complete. 2023-10-08T16:05:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.103+0000 7f3b8b8be700 1 Processor -- start 2023-10-08T16:05:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.103+0000 7f3b8b8be700 1 -- start start 2023-10-08T16:05:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 0x7f3b840cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 0x7f3b841543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840c81c0 con 0x7f3b84069560 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840cefb0 con 0x7f3b84159c60 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8b8be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840cf240 con 0x7f3b84159890 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38566/0 (socket says 172.21.15.170:38566) 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 -- 172.21.15.170:0/2963544689 learned_addr learned my addr 172.21.15.170:0/2963544689 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b88e59700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 0x7f3b840cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b89e5b700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 0x7f3b841543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 -- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 msgr2=0x7f3b840cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 0x7f3b840cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 -- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 msgr2=0x7f3b841543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 0x7f3b841543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b8965a700 1 -- 172.21.15.170:0/2963544689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c009ae0 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b88e59700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 0x7f3b840cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.104+0000 7f3b89e5b700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 0x7f3b841543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b8965a700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000d890 tx=0x7f3b80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000b600 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000b760 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840cf520 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000bae0 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.105+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b840cfae0 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.106+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b840c8000 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.106+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b8001e460 con 0x7f3b84069560 2023-10-08T16:05:46.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.106+0000 7f3b797fa700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 0x7f3b680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.106+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f3b80084940 con 0x7f3b84069560 2023-10-08T16:05:46.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.106+0000 7f3b88e59700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 0x7f3b680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.107+0000 7f3b88e59700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 0x7f3b680693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b74007fd0 tx=0x7f3b74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.111+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b840c8000 con 0x7f3b84069560 2023-10-08T16:05:46.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.291+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3b840c8000 con 0x7f3b84069560 2023-10-08T16:05:46.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.294+0000 7f3b797fa700 1 -- 172.21.15.170:0/2963544689 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3b840c8000 con 0x7f3b84069560 2023-10-08T16:05:46.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 msgr2=0x7f3b680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 0x7f3b680693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b74007fd0 tx=0x7f3b74009040 comp rx=0 tx=0).stop 2023-10-08T16:05:46.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 msgr2=0x7f3b840b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3b8000d890 tx=0x7f3b80008040 comp rx=0 tx=0).stop 2023-10-08T16:05:46.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 shutdown_connections 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b84159c60 0x7f3b841543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b68066f30 0x7f3b680693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b84069560 0x7f3b840b54e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 --2- 172.21.15.170:0/2963544689 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b84159890 0x7f3b840cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 >> 172.21.15.170:0/2963544689 conn(0x7f3b84068e20 msgr2=0x7f3b84157510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 shutdown_connections 2023-10-08T16:05:46.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.297+0000 7f3b8b8be700 1 -- 172.21.15.170:0/2963544689 wait complete. 2023-10-08T16:05:46.309 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:05:46.309 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:46.345 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:05:46.387 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:05:46.387 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:05:46.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:46.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.501+0000 7f9f1f037700 1 Processor -- start 2023-10-08T16:05:46.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.501+0000 7f9f1f037700 1 -- start start 2023-10-08T16:05:46.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.502+0000 7f9f1f037700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9f180cbf30 0x7f9f180ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.502+0000 7f9f1f037700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f180cc300 con 0x7f9f180b1420 2023-10-08T16:05:46.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.502+0000 7f9f1f037700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f180ad370 con 0x7f9f180ccf90 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.502+0000 7f9f1f037700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f18156bf0 con 0x7f9f180cbf30 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.502+0000 7f9f1d5d4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9f180b1420 0x7f9f180c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58844/0 (socket says 172.21.15.170:58844) 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1d5d4700 1 -- 172.21.15.170:0/2074055876 learned_addr learned my addr 172.21.15.170:0/2074055876 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1cdd3700 1 --2- 172.21.15.170:0/2074055876 >> v2:172.21.15.191:3300/0 conn(0x7f9f180cbf30 0x7f9f180ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724248190 0 0) 0x7f9f180cc300 con 0x7f9f180b1420 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f00003570 con 0x7f9f180b1420 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1cdd3700 1 -- 172.21.15.170:0/2074055876 >> v1:172.21.15.191:6789/0 conn(0x7f9f180ccf90 legacy=0x7f9f180b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1cdd3700 1 -- 172.21.15.170:0/2074055876 >> v1:172.21.15.170:6789/0 conn(0x7f9f180b1420 legacy=0x7f9f180c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714214534 0 0) 0x7f9f00003570 con 0x7f9f180b1420 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1cdd3700 1 -- 172.21.15.170:0/2074055876 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180ad650 con 0x7f9f180cbf30 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f1cdd3700 1 --2- 172.21.15.170:0/2074055876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180cbf30 0x7f9f180ae410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f140098e0 tx=0x7f9f140091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e298a5019caa45e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f100017f0 con 0x7f9f180cbf30 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14011420 con 0x7f9f180cbf30 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.503+0000 7f9f0f7fe700 1 -- 172.21.15.170:0/2074055876 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f140117a0 con 0x7f9f180cbf30 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 -- 172.21.15.170:0/2074055876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180cbf30 msgr2=0x7f9f180ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 --2- 172.21.15.170:0/2074055876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180cbf30 0x7f9f180ae410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9f140098e0 tx=0x7f9f140091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 -- 172.21.15.170:0/2074055876 shutdown_connections 2023-10-08T16:05:46.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 --2- 172.21.15.170:0/2074055876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180cbf30 0x7f9f180ae410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 -- 172.21.15.170:0/2074055876 >> 172.21.15.170:0/2074055876 conn(0x7f9f1806a320 msgr2=0x7f9f180cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:46.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 -- 172.21.15.170:0/2074055876 shutdown_connections 2023-10-08T16:05:46.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.504+0000 7f9f1f037700 1 -- 172.21.15.170:0/2074055876 wait complete. 2023-10-08T16:05:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.505+0000 7f9f1f037700 1 Processor -- start 2023-10-08T16:05:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.505+0000 7f9f1f037700 1 -- start start 2023-10-08T16:05:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f180b1420 0x7f9f18168760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180ccf90 0x7f9f180a9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f181565c0 con 0x7f9f180cbf30 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180ac4d0 con 0x7f9f180ccf90 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1f037700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f180c1470 con 0x7f9f180b1420 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f0ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f0ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38572/0 (socket says 172.21.15.170:38572) 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f0ffff700 1 -- 172.21.15.170:0/265908605 learned_addr learned my addr 172.21.15.170:0/265908605 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1cdd3700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f180b1420 0x7f9f18168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.506+0000 7f9f1d5d4700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180ccf90 0x7f9f180a9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 -- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f180b1420 msgr2=0x7f9f18168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f180b1420 0x7f9f18168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 -- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180ccf90 msgr2=0x7f9f180a9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180ccf90 0x7f9f180a9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 -- 172.21.15.170:0/265908605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009420 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0ffff700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c4e0 tx=0x7f9f0800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08011590 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f08011710 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f180c1750 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08011ad0 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.507+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f180acc40 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.508+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f180c12d0 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.508+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f0801b3b0 con 0x7f9f180cbf30 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.509+0000 7f9f0cff9700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 0x7f9f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.509+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f9f08083690 con 0x7f9f180cbf30 2023-10-08T16:05:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.509+0000 7f9f1cdd3700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 0x7f9f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.509+0000 7f9f1cdd3700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 0x7f9f040693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009fd0 tx=0x7f9f14016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.513+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f180c12d0 con 0x7f9f180cbf30 2023-10-08T16:05:46.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.737+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f180c12d0 con 0x7f9f180cbf30 2023-10-08T16:05:46.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.737+0000 7f9f0cff9700 1 -- 172.21.15.170:0/265908605 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f9f180c12d0 con 0x7f9f180cbf30 2023-10-08T16:05:46.738 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:46.738 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/236989080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":236989080},{"type":"v1","addr":"172.21.15.191:6834","nonce":236989080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4792,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1055524674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1055524674},{"type":"v1","addr":"172.21.15.191:6835","nonce":1055524674}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/931871174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":931871174},{"type":"v1","addr":"172.21.15.170:6836","nonce":931871174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":5235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/849552020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":849552020},{"type":"v1","addr":"172.21.15.170:6837","nonce":849552020}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 msgr2=0x7f9f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 0x7f9f040693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009fd0 tx=0x7f9f14016000 comp rx=0 tx=0).stop 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 msgr2=0x7f9f18156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c4e0 tx=0x7f9f0800c7f0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 shutdown_connections 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f180ccf90 0x7f9f180a9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f180b1420 0x7f9f18168760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f04066f30 0x7f9f040693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 --2- 172.21.15.170:0/265908605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f180cbf30 0x7f9f18156080 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 >> 172.21.15.170:0/265908605 conn(0x7f9f1806a320 msgr2=0x7f9f180afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 shutdown_connections 2023-10-08T16:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.739+0000 7f9f1f037700 1 -- 172.21.15.170:0/265908605 wait complete. 2023-10-08T16:05:46.741 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 24 2023-10-08T16:05:46.752 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:05:46.752 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:05:46.752 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:05:46.752 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:46.753 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:05:46.753 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:05:46.753 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:05:46.753 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:46.753 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:05:46.753 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:05:46.753 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:05:46.754 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:46.754 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:05:46.754 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:05:46.754 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:05:46.754 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:05:46.761 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:05:46.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:05:46.762 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:05:46.762 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:05:46.765 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:05:46.766 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:05:46.766 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:05:46.767 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:05:46.803 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:05:46.803 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:46.804 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:05:46.804 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:46.808 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:05:46.808 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:46.809 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:05:46.809 DEBUG:teuthology.parallel:result is None 2023-10-08T16:05:46.809 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:05:46.876 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:05:46.876 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:05:46.892 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:05:46.892 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:05:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.930+0000 7f2534daf700 1 Processor -- start 2023-10-08T16:05:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.930+0000 7f2534daf700 1 -- start start 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2530156950 0x7f25300cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f253006e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 0x7f25300c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253006cbf0 con 0x7f2530156950 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25300aceb0 con 0x7f253006eab0 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f2534daf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300adcf0 con 0x7f253006bbe0 2023-10-08T16:05:46.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2530156950 0x7f25300cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2530156950 0x7f25300cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38628/0 (socket says 172.21.15.170:38628) 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 -- 172.21.15.170:0/2012370577 learned_addr learned my addr 172.21.15.170:0/2012370577 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 -- 172.21.15.170:0/2012370577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 msgr2=0x7f253006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 --2- 172.21.15.170:0/2012370577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f253006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 -- 172.21.15.170:0/2012370577 >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 msgr2=0x7f25300c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252ffff700 1 --2- 172.21.15.170:0/2012370577 >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 0x7f25300c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 --2- 172.21.15.170:0/2012370577 >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 0x7f25300c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.931+0000 7f252f7fe700 1 -- 172.21.15.170:0/2012370577 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300adfb0 con 0x7f2530156950 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f252ffff700 1 --2- 172.21.15.170:0/2012370577 >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 0x7f25300c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:46.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f252f7fe700 1 --2- 172.21.15.170:0/2012370577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f25300cda10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f253006f160 tx=0x7f2520009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08e1bf9216732db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f252e7fc700 1 -- 172.21.15.170:0/2012370577 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252000e070 con 0x7f2530156950 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f252e7fc700 1 -- 172.21.15.170:0/2012370577 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252000bad0 con 0x7f2530156950 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f252e7fc700 1 -- 172.21.15.170:0/2012370577 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200092e0 con 0x7f2530156950 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 -- 172.21.15.170:0/2012370577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 msgr2=0x7f25300cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 --2- 172.21.15.170:0/2012370577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f25300cda10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f253006f160 tx=0x7f2520009840 comp rx=0 tx=0).stop 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 -- 172.21.15.170:0/2012370577 shutdown_connections 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 --2- 172.21.15.170:0/2012370577 >> v2:172.21.15.191:3300/0 conn(0x7f253006eab0 0x7f25300c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 --2- 172.21.15.170:0/2012370577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f25300cda10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 --2- 172.21.15.170:0/2012370577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f253006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 -- 172.21.15.170:0/2012370577 >> 172.21.15.170:0/2012370577 conn(0x7f253006b490 msgr2=0x7f25300cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 -- 172.21.15.170:0/2012370577 shutdown_connections 2023-10-08T16:05:46.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.932+0000 7f2534daf700 1 -- 172.21.15.170:0/2012370577 wait complete. 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 Processor -- start 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 -- start start 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f2530167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f253006eab0 0x7f25301570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25300ccce0 con 0x7f2530156950 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2530167ad0 con 0x7f253006eab0 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f2534daf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2530157bf0 con 0x7f253006bbe0 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f252ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f252ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38638/0 (socket says 172.21.15.170:38638) 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.934+0000 7f252ffff700 1 -- 172.21.15.170:0/1928937205 learned_addr learned my addr 172.21.15.170:0/1928937205 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252f7fe700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f2530167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252effd700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f253006eab0 0x7f25301570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 -- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 msgr2=0x7f2530167590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f2530167590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 -- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f253006eab0 msgr2=0x7f25301570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f253006eab0 0x7f25301570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 -- 172.21.15.170:0/1928937205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520009ae0 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252f7fe700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f2530167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f252ffff700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f252800c880 tx=0x7f2528008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528011600 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2528011760 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25300cc3b0 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2528011b20 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.935+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25300cc970 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.936+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253004c3c0 con 0x7f2530156950 2023-10-08T16:05:46.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.937+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f252801a400 con 0x7f2530156950 2023-10-08T16:05:46.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.937+0000 7f25177fe700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 0x7f251c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:46.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.937+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5025+0+0 (secure 0 0 0) 0x7f2528084440 con 0x7f2530156950 2023-10-08T16:05:46.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.937+0000 7f252f7fe700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 0x7f251c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:46.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.937+0000 7f252f7fe700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 0x7f251c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2520009f20 tx=0x7f2520018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:46.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:46.941+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25280553e0 con 0x7f2530156950 2023-10-08T16:05:47.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.142+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f25300c9750 con 0x7f2530156950 2023-10-08T16:05:47.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.985+0000 7f25177fe700 1 -- 172.21.15.170:0/1928937205 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2528054f70 con 0x7f2530156950 2023-10-08T16:05:47.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 msgr2=0x7f251c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 0x7f251c0693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2520009f20 tx=0x7f2520018000 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 msgr2=0x7f2530159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f252800c880 tx=0x7f2528008040 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 shutdown_connections 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f251c066f30 0x7f251c0693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2530156950 0x7f2530159180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f253006bbe0 0x7f2530167590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 --2- 172.21.15.170:0/1928937205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f253006eab0 0x7f25301570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 >> 172.21.15.170:0/1928937205 conn(0x7f253006b490 msgr2=0x7f25300688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:47.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 shutdown_connections 2023-10-08T16:05:47.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:47.987+0000 7f2534daf700 1 -- 172.21.15.170:0/1928937205 wait complete. 2023-10-08T16:05:47.989 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:05:48.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:05:48.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.112+0000 7f33d1fa0700 1 Processor -- start 2023-10-08T16:05:48.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.112+0000 7f33d1fa0700 1 -- start start 2023-10-08T16:05:48.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d1fa0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f33cc06c0f0 0x7f33cc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d1fa0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33cc0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d1fa0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc143420 con 0x7f33cc143050 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d1fa0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc06a1e0 con 0x7f33cc06c0f0 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d1fa0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0695d0 con 0x7f33cc0b3810 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d0f9e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f33cc143050 0x7f33cc06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58876/0 (socket says 172.21.15.170:58876) 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.113+0000 7f33d0f9e700 1 -- 172.21.15.170:0/776281286 learned_addr learned my addr 172.21.15.170:0/776281286 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890967677 0 0) 0x7f33cc143420 con 0x7f33cc143050 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cbfff700 1 --2- 172.21.15.170:0/776281286 >> v2:172.21.15.191:3300/0 conn(0x7f33cc06c0f0 0x7f33cc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b4003570 con 0x7f33cc143050 2023-10-08T16:05:48.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d179f700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33cc0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3137594507 0 0) 0x7f33b4003570 con 0x7f33cc143050 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 msgr2=0x7f33cc0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33cc0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 >> v2:172.21.15.191:3300/0 conn(0x7f33cc06c0f0 msgr2=0x7f33cc159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 --2- 172.21.15.170:0/776281286 >> v2:172.21.15.191:3300/0 conn(0x7f33cc06c0f0 0x7f33cc159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc0698b0 con 0x7f33cc143050 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f33bc001970 con 0x7f33cc143050 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 >> v1:172.21.15.170:6789/0 conn(0x7f33cc143050 legacy=0x7f33cc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33b4004d50 0x7f33b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc06a1e0 con 0x7f33b4003a60 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0695d0 con 0x7f33b4004d50 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b400bae0 con 0x7f33cc0b3810 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33cbfff700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d179f700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33b4004d50 0x7f33b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 -- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 msgr2=0x7f33b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0b3810 0x7f33b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 -- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33b4004d50 msgr2=0x7f33b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33b4004d50 0x7f33b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d0f9e700 1 -- 172.21.15.170:0/776281286 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4003570 con 0x7f33b4003a60 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.114+0000 7f33d179f700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33b4004d50 0x7f33b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33b4003e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c690 tx=0x7f33cc06a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dae941e0a43e36c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc005c40 con 0x7f33b4003a60 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc005dc0 con 0x7f33b4003a60 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33cb7fe700 1 -- 172.21.15.170:0/776281286 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc007360 con 0x7f33b4003a60 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 -- 172.21.15.170:0/776281286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 msgr2=0x7f33b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/776281286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33b4003e70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00c690 tx=0x7f33cc06a150 comp rx=0 tx=0).stop 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d179f700 1 -- 172.21.15.170:0/776281286 reap_dead start 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 -- 172.21.15.170:0/776281286 shutdown_connections 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 -- 172.21.15.170:0/776281286 >> 172.21.15.170:0/776281286 conn(0x7f33cc0ce430 msgr2=0x7f33cc0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 -- 172.21.15.170:0/776281286 shutdown_connections 2023-10-08T16:05:48.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.115+0000 7f33d1fa0700 1 -- 172.21.15.170:0/776281286 wait complete. 2023-10-08T16:05:48.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.116+0000 7f33d1fa0700 1 Processor -- start 2023-10-08T16:05:48.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 -- start start 2023-10-08T16:05:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc139f80 0x7f33cc158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 0x7f33cc0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0b0410 con 0x7f33b4003a60 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc13a760 con 0x7f33cc139f80 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d1fa0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc13aa40 con 0x7f33cc0afa30 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d0f9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d0f9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38658/0 (socket says 172.21.15.170:38658) 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d0f9e700 1 -- 172.21.15.170:0/857137077 learned_addr learned my addr 172.21.15.170:0/857137077 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33cbfff700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc139f80 0x7f33cc158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.117+0000 7f33d179f700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 0x7f33cc0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 -- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 msgr2=0x7f33cc0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 0x7f33cc0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 -- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc139f80 msgr2=0x7f33cc158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc139f80 0x7f33cc158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 -- 172.21.15.170:0/857137077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc13ad20 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d179f700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 0x7f33cc0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d0f9e700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f33bc001270 tx=0x7f33bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc001450 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc001b40 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc13b000 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc014670 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.118+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc13a3f0 con 0x7f33b4003a60 2023-10-08T16:05:48.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.119+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc04c3c0 con 0x7f33b4003a60 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.119+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33bc014a20 con 0x7f33b4003a60 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.120+0000 7f33c8ff9700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 0x7f33b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.120+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f33bc085ea0 con 0x7f33b4003a60 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.120+0000 7f33cbfff700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 0x7f33b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.120+0000 7f33cbfff700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 0x7f33b80693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0b16d0 tx=0x7f33c00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.124+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33bc053480 con 0x7f33b4003a60 2023-10-08T16:05:48.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.379+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f33cc0adad0 con 0x7f33b4003a60 2023-10-08T16:05:48.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.380+0000 7f33c8ff9700 1 -- 172.21.15.170:0/857137077 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f33bc026070 con 0x7f33b4003a60 2023-10-08T16:05:48.380 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:48.381 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:48.382 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:48.383 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.384 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.385 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.386 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.387 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.388 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.389 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:05:48.390 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:05:48.391 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:05:48.392 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:05:48.393 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.394 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.395 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:05:48.396 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.397 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:05:48.398 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:05:48.399 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 msgr2=0x7f33b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 0x7f33b80693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0b16d0 tx=0x7f33c00091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 msgr2=0x7f33cc142360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f33bc001270 tx=0x7f33bc015000 comp rx=0 tx=0).stop 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 shutdown_connections 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33cc0afa30 0x7f33cc0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33b8066f30 0x7f33b80693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33b4003a60 0x7f33cc142360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 --2- 172.21.15.170:0/857137077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33cc139f80 0x7f33cc158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 >> 172.21.15.170:0/857137077 conn(0x7f33cc0ce430 msgr2=0x7f33cc06cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 shutdown_connections 2023-10-08T16:05:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.382+0000 7f33d1fa0700 1 -- 172.21.15.170:0/857137077 wait complete. 2023-10-08T16:05:48.402 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:48.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.517+0000 7f7f6759e700 1 Processor -- start 2023-10-08T16:05:48.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.517+0000 7f7f6759e700 1 -- start start 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7f68159420 0x7f7f680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6759e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680b2980 con 0x7f7f6815d230 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680b11d0 con 0x7f7f681597f0 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680b14b0 con 0x7f7f68159420 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f66d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7f6815d230 0x7f7f6815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58884/0 (socket says 172.21.15.170:58884) 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f66d9d700 1 -- 172.21.15.170:0/401320842 learned_addr learned my addr 172.21.15.170:0/401320842 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698505909 0 0) 0x7f7f680b2980 con 0x7f7f6815d230 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f54003570 con 0x7f7f6815d230 2023-10-08T16:05:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6659c700 1 --2- 172.21.15.170:0/401320842 >> v2:172.21.15.191:3300/0 conn(0x7f7f68159420 0x7f7f680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926349431 0 0) 0x7f7f54003570 con 0x7f7f6815d230 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 >> v2:172.21.15.191:3300/0 conn(0x7f7f68159420 msgr2=0x7f7f680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 --2- 172.21.15.170:0/401320842 >> v2:172.21.15.191:3300/0 conn(0x7f7f68159420 0x7f7f680b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 >> v1:172.21.15.191:6789/0 conn(0x7f7f681597f0 legacy=0x7f7f680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680b1790 con 0x7f7f6815d230 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7f600017f0 con 0x7f7f6815d230 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f6659c700 1 --2- 172.21.15.170:0/401320842 >> v2:172.21.15.191:3300/0 conn(0x7f7f68159420 0x7f7f680b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.518+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 >> v1:172.21.15.170:6789/0 conn(0x7f7f6815d230 legacy=0x7f7f6815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f54003a60 0x7f7f54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f540052e0 0x7f7f5400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b14b0 con 0x7f7f54004810 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400c190 con 0x7f7f540052e0 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5400c350 con 0x7f7f54003a60 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f6659c700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f54003a60 0x7f7f54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f66d9d700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f540052e0 0x7f7f5400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 -- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f54003a60 msgr2=0x7f7f54003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f54003a60 0x7f7f54003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 -- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f540052e0 msgr2=0x7f7f5400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f540052e0 0x7f7f5400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f66d9d700 1 -- 172.21.15.170:0/401320842 reap_dead start 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 -- 172.21.15.170:0/401320842 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54003570 con 0x7f7f54004810 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f54004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001880 tx=0x7f7f50007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ba8b1e491d389e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50012420 con 0x7f7f54004810 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f500125a0 con 0x7f7f54004810 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.519+0000 7f7f65d9b700 1 -- 172.21.15.170:0/401320842 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50012960 con 0x7f7f54004810 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 -- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 msgr2=0x7f7f54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f54004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001880 tx=0x7f7f50007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 -- 172.21.15.170:0/401320842 shutdown_connections 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 --2- 172.21.15.170:0/401320842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f54004c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 -- 172.21.15.170:0/401320842 >> 172.21.15.170:0/401320842 conn(0x7f7f680d0660 msgr2=0x7f7f680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:48.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 -- 172.21.15.170:0/401320842 shutdown_connections 2023-10-08T16:05:48.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.520+0000 7f7f6759e700 1 -- 172.21.15.170:0/401320842 wait complete. 2023-10-08T16:05:48.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.521+0000 7f7f6759e700 1 Processor -- start 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.521+0000 7f7f6759e700 1 -- start start 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6814b4d0 0x7f7f68153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 0x7f7f680cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6814ee00 con 0x7f7f54004810 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68152410 con 0x7f7f680ce9c0 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6816d8e0 con 0x7f7f6814b4d0 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38684/0 (socket says 172.21.15.170:38684) 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 -- 172.21.15.170:0/680230679 learned_addr learned my addr 172.21.15.170:0/680230679 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f66d9d700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 0x7f7f680cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6814b4d0 0x7f7f68153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 -- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6814b4d0 msgr2=0x7f7f68153840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6814b4d0 0x7f7f68153840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 -- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 msgr2=0x7f7f680cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 0x7f7f680cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 -- 172.21.15.170:0/680230679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680cf350 con 0x7f7f54004810 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f66d9d700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 0x7f7f680cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6659c700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f58011990 tx=0x7f7f58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800c070 con 0x7f7f54004810 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f58011ed0 con 0x7f7f54004810 2023-10-08T16:05:48.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6816cd10 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.522+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800fb30 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.523+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6816d570 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.523+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6816dbc0 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.524+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f58015460 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.524+0000 7f7f5effd700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 0x7f7f480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.524+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7f58084530 con 0x7f7f54004810 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.524+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 0x7f7f480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.530 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.525+0000 7f7f5dd9b700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 0x7f7f480693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f680cf520 tx=0x7f7f50015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.529+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f58051af0 con 0x7f7f54004810 2023-10-08T16:05:48.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.714+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f6804c3c0 con 0x7f7f54004810 2023-10-08T16:05:48.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.714+0000 7f7f5effd700 1 -- 172.21.15.170:0/680230679 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f7f58055110 con 0x7f7f54004810 2023-10-08T16:05:48.715 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:48.715 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":47,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:47.982976+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 msgr2=0x7f7f480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 0x7f7f480693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7f680cf520 tx=0x7f7f50015000 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 msgr2=0x7f7f6814c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7f58011990 tx=0x7f7f58008040 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 shutdown_connections 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680ce9c0 0x7f7f680cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f48066f30 0x7f7f480693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f54004810 0x7f7f6814c1f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 --2- 172.21.15.170:0/680230679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6814b4d0 0x7f7f68153840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 >> 172.21.15.170:0/680230679 conn(0x7f7f680d0660 msgr2=0x7f7f68068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 shutdown_connections 2023-10-08T16:05:48.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.717+0000 7f7f6759e700 1 -- 172.21.15.170:0/680230679 wait complete. 2023-10-08T16:05:48.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:05:48.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:05:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.847+0000 7f87704db700 1 Processor -- start 2023-10-08T16:05:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.847+0000 7f87704db700 1 -- start start 2023-10-08T16:05:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f87704db700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8768159890 0x7f876815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f87704db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 0x7f876815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f87704db700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876815a160 con 0x7f876815db10 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f87704db700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87680d13a0 con 0x7f8768159890 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f87704db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876815e140 con 0x7f8768159c60 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f876e277700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8768159890 0x7f876815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f876ea78700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f876815db10 0x7f8768069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58900/0 (socket says 172.21.15.170:58900) 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.848+0000 7f876ea78700 1 -- 172.21.15.170:0/2670474841 learned_addr learned my addr 172.21.15.170:0/2670474841 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876da76700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 0x7f876815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876e277700 1 -- 172.21.15.170:0/2670474841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 msgr2=0x7f876815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876e277700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 0x7f876815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876e277700 1 -- 172.21.15.170:0/2670474841 >> v1:172.21.15.170:6789/0 conn(0x7f876815db10 legacy=0x7f8768069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876e277700 1 -- 172.21.15.170:0/2670474841 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876815a3d0 con 0x7f8768159890 2023-10-08T16:05:48.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876d275700 1 -- 172.21.15.170:0/2670474841 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517093520 0 0) 0x7f876815a160 con 0x7f876815db10 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876da76700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 0x7f876815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876e277700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768159890 0x7f876815abd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f87640098e0 tx=0x7f87640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66f3a5f67092a8d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876d275700 1 -- 172.21.15.170:0/2670474841 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764009e80 con 0x7f8768159890 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876d275700 1 -- 172.21.15.170:0/2670474841 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876400b860 con 0x7f8768159890 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f876d275700 1 -- 172.21.15.170:0/2670474841 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400bbe0 con 0x7f8768159890 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f87704db700 1 -- 172.21.15.170:0/2670474841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768159890 msgr2=0x7f876815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.849+0000 7f87704db700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768159890 0x7f876815abd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f87640098e0 tx=0x7f87640091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 -- 172.21.15.170:0/2670474841 shutdown_connections 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8768159890 0x7f876815abd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 --2- 172.21.15.170:0/2670474841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159c60 0x7f876815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 -- 172.21.15.170:0/2670474841 >> 172.21.15.170:0/2670474841 conn(0x7f8768068e20 msgr2=0x7f87680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 -- 172.21.15.170:0/2670474841 shutdown_connections 2023-10-08T16:05:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.850+0000 7f87704db700 1 -- 172.21.15.170:0/2670474841 wait complete. 2023-10-08T16:05:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.851+0000 7f87704db700 1 Processor -- start 2023-10-08T16:05:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.851+0000 7f87704db700 1 -- start start 2023-10-08T16:05:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.851+0000 7f87704db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159890 0x7f87680b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.851+0000 7f87704db700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8768159c60 0x7f876815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87680ce880 con 0x7f876815db10 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876815c590 con 0x7f8768159c60 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87680ceb40 con 0x7f8768159890 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38700/0 (socket says 172.21.15.170:38700) 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 -- 172.21.15.170:0/2916922474 learned_addr learned my addr 172.21.15.170:0/2916922474 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876da76700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8768159c60 0x7f876815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876e277700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159890 0x7f87680b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 -- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159890 msgr2=0x7f87680b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159890 0x7f87680b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 -- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8768159c60 msgr2=0x7f876815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8768159c60 0x7f876815c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 -- 172.21.15.170:0/2916922474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764009420 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f876ea78700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f876000cc20 tx=0x7f87600019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000a870 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000a9d0 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876815d5b0 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000ad90 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.852+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876815bfd0 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.853+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876804c3c0 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.854+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8760007300 con 0x7f876815db10 2023-10-08T16:05:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.854+0000 7f875e7fc700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 0x7f874c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.854+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f87600844a0 con 0x7f876815db10 2023-10-08T16:05:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.854+0000 7f876e277700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 0x7f874c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.855+0000 7f876e277700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 0x7f874c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87640091e0 tx=0x7f8764017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:48.858+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8760051ae0 con 0x7f876815db10 2023-10-08T16:05:49.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.045+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87680cec80 con 0x7f876815db10 2023-10-08T16:05:49.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.045+0000 7f875e7fc700 1 -- 172.21.15.170:0/2916922474 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8760055100 con 0x7f876815db10 2023-10-08T16:05:49.046 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:05:49.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 msgr2=0x7f874c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 0x7f874c0693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87640091e0 tx=0x7f8764017000 comp rx=0 tx=0).stop 2023-10-08T16:05:49.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 msgr2=0x7f876815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f876000cc20 tx=0x7f87600019b0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 shutdown_connections 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f874c066f30 0x7f874c0693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f876815db10 0x7f876815b680 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8768159890 0x7f87680b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 --2- 172.21.15.170:0/2916922474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8768159c60 0x7f876815c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 >> 172.21.15.170:0/2916922474 conn(0x7f8768068e20 msgr2=0x7f87680b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 shutdown_connections 2023-10-08T16:05:49.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.048+0000 7f87704db700 1 -- 172.21.15.170:0/2916922474 wait complete. 2023-10-08T16:05:49.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:05:49.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.173+0000 7f08fbe75700 1 Processor -- start 2023-10-08T16:05:49.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.173+0000 7f08fbe75700 1 -- start start 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08fbe75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08f4159c60 0x7f08f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08fbe75700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f415a160 con 0x7f08f4159890 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08fbe75700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f40d13a0 con 0x7f08f415db10 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08fbe75700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f415df50 con 0x7f08f4159c60 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08f9410700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08f4159c60 0x7f08f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08f9c11700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f08f4159890 0x7f08f415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58906/0 (socket says 172.21.15.170:58906) 2023-10-08T16:05:49.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.174+0000 7f08f9c11700 1 -- 172.21.15.170:0/645710221 learned_addr learned my addr 172.21.15.170:0/645710221 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f9410700 1 -- 172.21.15.170:0/645710221 >> v1:172.21.15.191:6789/0 conn(0x7f08f415db10 legacy=0x7f08f4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f9410700 1 -- 172.21.15.170:0/645710221 >> v1:172.21.15.170:6789/0 conn(0x7f08f4159890 legacy=0x7f08f415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f9410700 1 -- 172.21.15.170:0/645710221 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f415e140 con 0x7f08f4159c60 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f8c0f700 1 -- 172.21.15.170:0/645710221 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376028816 0 0) 0x7f08f415a160 con 0x7f08f4159890 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f9410700 1 --2- 172.21.15.170:0/645710221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159c60 0x7f08f415d5d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f08e40098e0 tx=0x7f08e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6861c9e836cbd71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f8c0f700 1 -- 172.21.15.170:0/645710221 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4009e80 con 0x7f08f4159c60 2023-10-08T16:05:49.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f8c0f700 1 -- 172.21.15.170:0/645710221 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400b860 con 0x7f08f4159c60 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08f8c0f700 1 -- 172.21.15.170:0/645710221 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400bbe0 con 0x7f08f4159c60 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08fbe75700 1 -- 172.21.15.170:0/645710221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159c60 msgr2=0x7f08f415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08fbe75700 1 --2- 172.21.15.170:0/645710221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159c60 0x7f08f415d5d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f08e40098e0 tx=0x7f08e40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08fbe75700 1 -- 172.21.15.170:0/645710221 shutdown_connections 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08fbe75700 1 --2- 172.21.15.170:0/645710221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159c60 0x7f08f415d5d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.175+0000 7f08fbe75700 1 -- 172.21.15.170:0/645710221 >> 172.21.15.170:0/645710221 conn(0x7f08f4068e20 msgr2=0x7f08f40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.176+0000 7f08fbe75700 1 -- 172.21.15.170:0/645710221 shutdown_connections 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.176+0000 7f08fbe75700 1 -- 172.21.15.170:0/645710221 wait complete. 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 Processor -- start 2023-10-08T16:05:49.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 -- start start 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f4159c60 0x7f08f4152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f415db10 0x7f08f40ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415bfa0 con 0x7f08f4159890 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40ca7a0 con 0x7f08f415db10 2023-10-08T16:05:49.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08fbe75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f415b670 con 0x7f08f4159c60 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08f9c11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08f9c11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38714/0 (socket says 172.21.15.170:38714) 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.177+0000 7f08f9c11700 1 -- 172.21.15.170:0/3925011252 learned_addr learned my addr 172.21.15.170:0/3925011252 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9410700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f4159c60 0x7f08f4152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08fa412700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f415db10 0x7f08f40ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 -- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f4159c60 msgr2=0x7f08f4152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f4159c60 0x7f08f4152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 -- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f415db10 msgr2=0x7f08f40ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f415db10 0x7f08f40ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 -- 172.21.15.170:0/3925011252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009420 con 0x7f08f4159890 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08f9c11700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ccc0 tx=0x7f08f0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000a8e0 con 0x7f08f4159890 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f000aa40 con 0x7f08f4159890 2023-10-08T16:05:49.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f415b950 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000ae40 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.178+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40b0770 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.179+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f404c3c0 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.180+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08f0007300 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.180+0000 7f08e9ffb700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.180+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f08f0084510 con 0x7f08f4159890 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.180+0000 7f08f9410700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.180+0000 7f08f9410700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08e40093f0 tx=0x7f08e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.184+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08f0051b50 con 0x7f08f4159890 2023-10-08T16:05:49.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.368+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f08f4156e90 con 0x7f08f4159890 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.368+0000 7f08e9ffb700 1 -- 172.21.15.170:0/3925011252 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+65 (secure 0 0 0) 0x7f08f0055170 con 0x7f08f4159890 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:05:49.369 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:05:49.370 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:05:49.370 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:05:49.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 msgr2=0x7f08d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 0x7f08d80693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f08e40093f0 tx=0x7f08e4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:49.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 msgr2=0x7f08f416dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08f000ccc0 tx=0x7f08f0001a80 comp rx=0 tx=0).stop 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 shutdown_connections 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08f415db10 0x7f08f40ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d8066f30 0x7f08d80693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08f4159890 0x7f08f416dcb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 --2- 172.21.15.170:0/3925011252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08f4159c60 0x7f08f4152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 >> 172.21.15.170:0/3925011252 conn(0x7f08f4068e20 msgr2=0x7f08f4153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 shutdown_connections 2023-10-08T16:05:49.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.371+0000 7f08fbe75700 1 -- 172.21.15.170:0/3925011252 wait complete. 2023-10-08T16:05:49.383 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:05:49.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:05:49.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.501+0000 7f3d04054700 1 Processor -- start 2023-10-08T16:05:49.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d04054700 1 -- start start 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d04054700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3cfc154320 0x7f3cfc06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d04054700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0b4570 con 0x7f3cfc15a190 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d04054700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc0b2710 con 0x7f3cfc153f50 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d04054700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc15a730 con 0x7f3cfc154320 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.502+0000 7f3d015ef700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3cfc154320 0x7f3cfc06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d025f1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3cfc15a190 0x7f3cfc069e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58908/0 (socket says 172.21.15.170:58908) 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d015ef700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3cfc154320 0x7f3cfc06d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38716/0 (socket says 172.21.15.170:38716) 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d025f1700 1 -- 172.21.15.170:0/567779477 learned_addr learned my addr 172.21.15.170:0/567779477 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d015ef700 1 -- 172.21.15.170:0/567779477 >> v1:172.21.15.191:6789/0 conn(0x7f3cfc153f50 legacy=0x7f3cfc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d015ef700 1 -- 172.21.15.170:0/567779477 >> v1:172.21.15.170:6789/0 conn(0x7f3cfc15a190 legacy=0x7f3cfc069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d015ef700 1 -- 172.21.15.170:0/567779477 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc15aa10 con 0x7f3cfc154320 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d00dee700 1 -- 172.21.15.170:0/567779477 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291543940 0 0) 0x7f3cfc0b4570 con 0x7f3cfc15a190 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d015ef700 1 --2- 172.21.15.170:0/567779477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc154320 0x7f3cfc06d130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0098e0 tx=0x7f3cec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1564b908b9d0c4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d00dee700 1 -- 172.21.15.170:0/567779477 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec009e80 con 0x7f3cfc154320 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d00dee700 1 -- 172.21.15.170:0/567779477 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cec00b860 con 0x7f3cfc154320 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.503+0000 7f3d00dee700 1 -- 172.21.15.170:0/567779477 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cec00bbe0 con 0x7f3cfc154320 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 -- 172.21.15.170:0/567779477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc154320 msgr2=0x7f3cfc06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 --2- 172.21.15.170:0/567779477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc154320 0x7f3cfc06d130 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3cec0098e0 tx=0x7f3cec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 -- 172.21.15.170:0/567779477 shutdown_connections 2023-10-08T16:05:49.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 --2- 172.21.15.170:0/567779477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc154320 0x7f3cfc06d130 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 -- 172.21.15.170:0/567779477 >> 172.21.15.170:0/567779477 conn(0x7f3cfc06baf0 msgr2=0x7f3cfc06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:49.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 -- 172.21.15.170:0/567779477 shutdown_connections 2023-10-08T16:05:49.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.504+0000 7f3d04054700 1 -- 172.21.15.170:0/567779477 wait complete. 2023-10-08T16:05:49.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.505+0000 7f3d04054700 1 Processor -- start 2023-10-08T16:05:49.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.505+0000 7f3d04054700 1 -- start start 2023-10-08T16:05:49.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.505+0000 7f3d04054700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc153f50 0x7f3cfc16cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.505+0000 7f3d04054700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 0x7f3cfc16a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.505+0000 7f3d04054700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d04054700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0cc2f0 con 0x7f3cfc15a190 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d04054700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc16d530 con 0x7f3cfc153f50 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d04054700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0cb9e0 con 0x7f3cfc154320 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38720/0 (socket says 172.21.15.170:38720) 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 -- 172.21.15.170:0/4077442531 learned_addr learned my addr 172.21.15.170:0/4077442531 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d01df0700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc153f50 0x7f3cfc16cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d015ef700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 0x7f3cfc16a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 -- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 msgr2=0x7f3cfc16a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 0x7f3cfc16a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 -- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc153f50 msgr2=0x7f3cfc16cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc153f50 0x7f3cfc16cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 -- 172.21.15.170:0/4077442531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec009420 con 0x7f3cfc15a190 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d015ef700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 0x7f3cfc16a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d025f1700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400cbf0 tx=0x7f3cf40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400a910 con 0x7f3cfc15a190 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf400aa70 con 0x7f3cfc15a190 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc1570d0 con 0x7f3cfc15a190 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400ae30 con 0x7f3cfc15a190 2023-10-08T16:05:49.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.506+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc0cbf80 con 0x7f3cfc15a190 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.507+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc0718b0 con 0x7f3cfc15a190 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.508+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cf4007300 con 0x7f3cfc15a190 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.508+0000 7f3cf1ffb700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.508+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3cf4084520 con 0x7f3cfc15a190 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.508+0000 7f3d01df0700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.509+0000 7f3d01df0700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc14ae30 tx=0x7f3cf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:49.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.512+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf4051b60 con 0x7f3cfc15a190 2023-10-08T16:05:49.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.698+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3cfc0cc430 con 0x7f3cfc15a190 2023-10-08T16:05:49.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.994+0000 7f3cf1ffb700 1 -- 172.21.15.170:0/4077442531 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3cf4055180 con 0x7f3cfc15a190 2023-10-08T16:05:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 msgr2=0x7f3ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc14ae30 tx=0x7f3cf8009040 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 msgr2=0x7f3cfc156700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400cbf0 tx=0x7f3cf40019b0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 shutdown_connections 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3cfc15a190 0x7f3cfc156700 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3cfc153f50 0x7f3cfc16cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 --2- 172.21.15.170:0/4077442531 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3cfc154320 0x7f3cfc16a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 >> 172.21.15.170:0/4077442531 conn(0x7f3cfc06baf0 msgr2=0x7f3cfc0c8170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 shutdown_connections 2023-10-08T16:05:50.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:49.998+0000 7f3d04054700 1 -- 172.21.15.170:0/4077442531 wait complete. 2023-10-08T16:05:50.000 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:05:50.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:05:50.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.132+0000 7f9cac4fd700 1 Processor -- start 2023-10-08T16:05:50.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.132+0000 7f9cac4fd700 1 -- start start 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9cac4fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9ca4159950 0x7f9ca40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9cac4fd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9ca40af320 0x7f9ca4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9cac4fd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca40ce380 con 0x7f9ca4069aa0 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9cac4fd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca40cec20 con 0x7f9ca40af320 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9cac4fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca4069ee0 con 0x7f9ca4159950 2023-10-08T16:05:50.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9ca9a98700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9ca40af320 0x7f9ca4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9ca9a98700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9ca40af320 0x7f9ca4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38728/0 (socket says 172.21.15.170:38728) 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9ca9a98700 1 -- 172.21.15.170:0/3284633038 learned_addr learned my addr 172.21.15.170:0/3284633038 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.133+0000 7f9caa299700 1 --2- 172.21.15.170:0/3284633038 >> v2:172.21.15.191:3300/0 conn(0x7f9ca4159950 0x7f9ca40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9a98700 1 -- 172.21.15.170:0/3284633038 >> v2:172.21.15.191:3300/0 conn(0x7f9ca4159950 msgr2=0x7f9ca40cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9a98700 1 --2- 172.21.15.170:0/3284633038 >> v2:172.21.15.191:3300/0 conn(0x7f9ca4159950 0x7f9ca40cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9a98700 1 -- 172.21.15.170:0/3284633038 >> v1:172.21.15.191:6789/0 conn(0x7f9ca4069aa0 legacy=0x7f9ca415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9a98700 1 -- 172.21.15.170:0/3284633038 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40afa90 con 0x7f9ca40af320 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9a98700 1 --2- 172.21.15.170:0/3284633038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca4069560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40896e0 tx=0x7f9c98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9045dbeb7682c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9297700 1 -- 172.21.15.170:0/3284633038 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800b860 con 0x7f9ca40af320 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9297700 1 -- 172.21.15.170:0/3284633038 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800b9c0 con 0x7f9ca40af320 2023-10-08T16:05:50.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9ca9297700 1 -- 172.21.15.170:0/3284633038 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800bd60 con 0x7f9ca40af320 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 -- 172.21.15.170:0/3284633038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 msgr2=0x7f9ca4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/3284633038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca4069560 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40896e0 tx=0x7f9c98009250 comp rx=0 tx=0).stop 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 -- 172.21.15.170:0/3284633038 shutdown_connections 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/3284633038 >> v2:172.21.15.191:3300/0 conn(0x7f9ca4159950 0x7f9ca40cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/3284633038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca4069560 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 -- 172.21.15.170:0/3284633038 >> 172.21.15.170:0/3284633038 conn(0x7f9ca4068e00 msgr2=0x7f9ca40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.134+0000 7f9cac4fd700 1 -- 172.21.15.170:0/3284633038 shutdown_connections 2023-10-08T16:05:50.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.135+0000 7f9cac4fd700 1 -- 172.21.15.170:0/3284633038 wait complete. 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 Processor -- start 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 -- start start 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ca4069aa0 0x7f9ca414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 0x7f9ca40ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca414a890 con 0x7f9ca40af320 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca414d320 con 0x7f9ca4159950 2023-10-08T16:05:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9cac4fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca414dcf0 con 0x7f9ca4069aa0 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9ca9a98700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9ca9a98700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38734/0 (socket says 172.21.15.170:38734) 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9ca9a98700 1 -- 172.21.15.170:0/1939557740 learned_addr learned my addr 172.21.15.170:0/1939557740 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9caa299700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ca4069aa0 0x7f9ca414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.136+0000 7f9caaa9a700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 0x7f9ca40ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 -- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ca4069aa0 msgr2=0x7f9ca414a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ca4069aa0 0x7f9ca414a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 -- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 msgr2=0x7f9ca40ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 0x7f9ca40ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 -- 172.21.15.170:0/1939557740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980094f0 con 0x7f9ca40af320 2023-10-08T16:05:50.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9caaa9a700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 0x7f9ca40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca9a98700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4070e20 tx=0x7f9c98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98024440 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c980245a0 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca414c710 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98024960 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.137+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca414ccd0 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.138+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca40718b0 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.139+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c98024d50 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.139+0000 7f9ca27fc700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 0x7f9c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.139+0000 7f9caa299700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 0x7f9c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.139+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f9c98085fb0 con 0x7f9ca40af320 2023-10-08T16:05:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.140+0000 7f9caa299700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 0x7f9c900693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c940075a0 tx=0x7f9c940097e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:50.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.144+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c98053400 con 0x7f9ca40af320 2023-10-08T16:05:50.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.327+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ca40ce660 con 0x7f9ca40af320 2023-10-08T16:05:50.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:50.999+0000 7f9ca27fc700 1 -- 172.21.15.170:0/1939557740 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9c980150d0 con 0x7f9ca40af320 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 msgr2=0x7f9c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 0x7f9c900693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9c940075a0 tx=0x7f9c940097e0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 msgr2=0x7f9ca414d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4070e20 tx=0x7f9c98017000 comp rx=0 tx=0).stop 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 shutdown_connections 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9ca4159950 0x7f9ca40ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9ca4069aa0 0x7f9ca414a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c90066f30 0x7f9c900693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 --2- 172.21.15.170:0/1939557740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ca40af320 0x7f9ca414d7b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.002+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 >> 172.21.15.170:0/1939557740 conn(0x7f9ca4068e00 msgr2=0x7f9ca40b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.003+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 shutdown_connections 2023-10-08T16:05:51.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.004+0000 7f9cac4fd700 1 -- 172.21.15.170:0/1939557740 wait complete. 2023-10-08T16:05:51.005 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:05:51.026 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:05:51.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.144+0000 7f625369b700 1 Processor -- start 2023-10-08T16:05:51.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.144+0000 7f625369b700 1 -- start start 2023-10-08T16:05:51.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f625369b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f624c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f625369b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624c15a160 con 0x7f624c15db10 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f625369b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624c0d13a0 con 0x7f624c159890 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f625369b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624c15e140 con 0x7f624c159c60 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f6251c38700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f624c15db10 0x7f624c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58920/0 (socket says 172.21.15.170:58920) 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f6251c38700 1 -- 172.21.15.170:0/2151604966 learned_addr learned my addr 172.21.15.170:0/2151604966 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467311303 0 0) 0x7f624c15a160 con 0x7f624c15db10 2023-10-08T16:05:51.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6238003570 con 0x7f624c15db10 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.145+0000 7f6250c36700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f624c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 8534349 0 0) 0x7f6238003570 con 0x7f624c15db10 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 msgr2=0x7f624c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f624c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 >> v1:172.21.15.191:6789/0 conn(0x7f624c159890 legacy=0x7f624c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c15a3d0 con 0x7f624c15db10 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6244001470 con 0x7f624c15db10 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 >> v1:172.21.15.170:6789/0 conn(0x7f624c15db10 legacy=0x7f624c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f6238003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62380040c0 0x7f6238004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6238004da0 0x7f62380051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c15e140 con 0x7f62380040c0 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623800bc70 con 0x7f6238004da0 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623800be30 con 0x7f624c159c60 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62380040c0 0x7f6238004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6251437700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f6238003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6251c38700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6238004da0 0x7f62380051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 -- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 msgr2=0x7f6238003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f624c159c60 0x7f6238003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 -- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6238004da0 msgr2=0x7f62380051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:51.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6238004da0 0x7f62380051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 -- 172.21.15.170:0/2151604966 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238003570 con 0x7f62380040c0 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6250c36700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62380040c0 0x7f6238004810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f623c00bb60 tx=0x7f623c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff533c28e5dc36e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00d5d0 con 0x7f62380040c0 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00d730 con 0x7f62380040c0 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.146+0000 7f6243fff700 1 -- 172.21.15.170:0/2151604966 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00daf0 con 0x7f62380040c0 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 -- 172.21.15.170:0/2151604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62380040c0 msgr2=0x7f6238004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 --2- 172.21.15.170:0/2151604966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62380040c0 0x7f6238004810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f623c00bb60 tx=0x7f623c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f6251c38700 1 -- 172.21.15.170:0/2151604966 reap_dead start 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 -- 172.21.15.170:0/2151604966 shutdown_connections 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 -- 172.21.15.170:0/2151604966 >> 172.21.15.170:0/2151604966 conn(0x7f624c068e20 msgr2=0x7f624c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:51.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 -- 172.21.15.170:0/2151604966 shutdown_connections 2023-10-08T16:05:51.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.147+0000 7f625369b700 1 -- 172.21.15.170:0/2151604966 wait complete. 2023-10-08T16:05:51.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.148+0000 7f625369b700 1 Processor -- start 2023-10-08T16:05:51.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.148+0000 7f625369b700 1 -- start start 2023-10-08T16:05:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62380040c0 0x7f624c0c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 0x7f624c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c0cfcd0 con 0x7f624c0ce600 2023-10-08T16:05:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c0c9bf0 con 0x7f624c0c9260 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f625369b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624c0cf0c0 con 0x7f62380040c0 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38756/0 (socket says 172.21.15.170:38756) 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 -- 172.21.15.170:0/2227437857 learned_addr learned my addr 172.21.15.170:0/2227437857 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6251437700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62380040c0 0x7f624c0c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6251c38700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 0x7f624c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 -- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62380040c0 msgr2=0x7f624c0c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62380040c0 0x7f624c0c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 -- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 msgr2=0x7f624c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 0x7f624c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6250c36700 1 -- 172.21.15.170:0/2227437857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c0cf3a0 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.149+0000 7f6251c38700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 0x7f624c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f6250c36700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f623800bc40 tx=0x7f623c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c014640 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c0147c0 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c0cf680 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c014b80 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.150+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624c14b610 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.151+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c0cfe40 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.151+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f623c028410 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.151+0000 7f62417fa700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 0x7f6230069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.151+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f623c085df0 con 0x7f624c0ce600 2023-10-08T16:05:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.152+0000 7f6251437700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 0x7f6230069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.152+0000 7f6251437700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 0x7f6230069410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6248004290 tx=0x7f6248001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.156+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f623c0530c0 con 0x7f624c0ce600 2023-10-08T16:05:51.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:51.382+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f624c04c3c0 con 0x7f624c0ce600 2023-10-08T16:05:52.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.001+0000 7f62417fa700 1 -- 172.21.15.170:0/2227437857 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7f623c0566e0 con 0x7f624c0ce600 2023-10-08T16:05:52.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.006+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 msgr2=0x7f6230069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.006+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 0x7f6230069410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6248004290 tx=0x7f6248001640 comp rx=0 tx=0).stop 2023-10-08T16:05:52.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.006+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 msgr2=0x7f624c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.006+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f623800bc40 tx=0x7f623c01b000 comp rx=0 tx=0).stop 2023-10-08T16:05:52.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 shutdown_connections 2023-10-08T16:05:52.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f624c0c9260 0x7f624c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62380040c0 0x7f624c0c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6230066f60 0x7f6230069410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 --2- 172.21.15.170:0/2227437857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f624c0ce600 0x7f624c0cea70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.010+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 >> 172.21.15.170:0/2227437857 conn(0x7f624c068e20 msgr2=0x7f624c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.012+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 shutdown_connections 2023-10-08T16:05:52.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.013+0000 7f625369b700 1 -- 172.21.15.170:0/2227437857 wait complete. 2023-10-08T16:05:52.027 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 8 and data pool 9 2023-10-08T16:05:52.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:52.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.149+0000 7f746d097700 1 Processor -- start 2023-10-08T16:05:52.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.149+0000 7f746d097700 1 -- start start 2023-10-08T16:05:52.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f746806d950 0x7f74680c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f746806a0a0 0x7f74680c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806dd20 0x7f74680c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746806ae70 con 0x7f746806d950 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746806a540 con 0x7f746806a0a0 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746d097700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746806a820 con 0x7f746806dd20 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f746806d950 0x7f74680c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f746806d950 0x7f74680c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38770/0 (socket says 172.21.15.170:38770) 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 -- 172.21.15.170:0/1200402760 learned_addr learned my addr 172.21.15.170:0/1200402760 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f74677fe700 1 --2- 172.21.15.170:0/1200402760 >> v2:172.21.15.191:3300/0 conn(0x7f746806a0a0 0x7f74680c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f746c896700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806dd20 0x7f74680c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 -- 172.21.15.170:0/1200402760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806dd20 msgr2=0x7f74680c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806dd20 0x7f74680c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 -- 172.21.15.170:0/1200402760 >> v2:172.21.15.191:3300/0 conn(0x7f746806a0a0 msgr2=0x7f74680c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 --2- 172.21.15.170:0/1200402760 >> v2:172.21.15.191:3300/0 conn(0x7f746806a0a0 0x7f74680c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 -- 172.21.15.170:0/1200402760 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746806ab00 con 0x7f746806d950 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.150+0000 7f7467fff700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806d950 0x7f74680c8350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f746806b0c0 tx=0x7f7458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a10fcbb9a3e5345 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f7466ffd700 1 -- 172.21.15.170:0/1200402760 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800b860 con 0x7f746806d950 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f7466ffd700 1 -- 172.21.15.170:0/1200402760 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745800b9c0 con 0x7f746806d950 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f7466ffd700 1 -- 172.21.15.170:0/1200402760 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800bd60 con 0x7f746806d950 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 -- 172.21.15.170:0/1200402760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806d950 msgr2=0x7f74680c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806d950 0x7f74680c8350 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f746806b0c0 tx=0x7f7458009250 comp rx=0 tx=0).stop 2023-10-08T16:05:52.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 -- 172.21.15.170:0/1200402760 shutdown_connections 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806dd20 0x7f74680c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 --2- 172.21.15.170:0/1200402760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806d950 0x7f74680c8350 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 --2- 172.21.15.170:0/1200402760 >> v2:172.21.15.191:3300/0 conn(0x7f746806a0a0 0x7f74680c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 -- 172.21.15.170:0/1200402760 >> 172.21.15.170:0/1200402760 conn(0x7f74680cc1f0 msgr2=0x7f74680cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 -- 172.21.15.170:0/1200402760 shutdown_connections 2023-10-08T16:05:52.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.151+0000 7f746d097700 1 -- 172.21.15.170:0/1200402760 wait complete. 2023-10-08T16:05:52.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.152+0000 7f746d097700 1 Processor -- start 2023-10-08T16:05:52.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.152+0000 7f746d097700 1 -- start start 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806d950 0x7f7468152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746806dd20 0x7f74681587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74680c3ec0 con 0x7f746806a0a0 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74680c5950 con 0x7f746806dd20 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746d097700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74680c4110 con 0x7f746806d950 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f7467fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f7467fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38780/0 (socket says 172.21.15.170:38780) 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f7467fff700 1 -- 172.21.15.170:0/1406886623 learned_addr learned my addr 172.21.15.170:0/1406886623 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f746c896700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746806dd20 0x7f74681587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f74677fe700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806d950 0x7f7468152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.153+0000 7f7467fff700 1 -- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806d950 msgr2=0x7f7468152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f7467fff700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806d950 0x7f7468152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f7467fff700 1 -- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746806dd20 msgr2=0x7f74681587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f7467fff700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746806dd20 0x7f74681587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f7467fff700 1 -- 172.21.15.170:0/1406886623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580094f0 con 0x7f746806a0a0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f7467fff700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7458006fd0 tx=0x7f7458017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458011d80 con 0x7f746806a0a0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7458024440 con 0x7f746806a0a0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74680c43f0 con 0x7f746806a0a0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458024800 con 0x7f746806a0a0 2023-10-08T16:05:52.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.154+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74680c55e0 con 0x7f746806a0a0 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.155+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74680c5c30 con 0x7f746806a0a0 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.155+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f745801a420 con 0x7f746806a0a0 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.155+0000 7f744ffff700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.156+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f74580862e0 con 0x7f746806a0a0 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.156+0000 7f74677fe700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.156+0000 7f74677fe700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 0x7f74540693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f746816adf0 tx=0x7f745c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.160+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7458053550 con 0x7f746806a0a0 2023-10-08T16:05:52.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.344+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f746804c3c0 con 0x7f746806a0a0 2023-10-08T16:05:52.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.345+0000 7f744ffff700 1 -- 172.21.15.170:0/1406886623 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f7458056b70 con 0x7f746806a0a0 2023-10-08T16:05:52.346 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:52.346 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":50,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:51.383137+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 msgr2=0x7f74540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 0x7f74540693e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f746816adf0 tx=0x7f745c008040 comp rx=0 tx=0).stop 2023-10-08T16:05:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 msgr2=0x7f74680c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7458006fd0 tx=0x7f7458017000 comp rx=0 tx=0).stop 2023-10-08T16:05:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 shutdown_connections 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746806dd20 0x7f74681587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7454066f30 0x7f74540693e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746806a0a0 0x7f74680c5de0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 --2- 172.21.15.170:0/1406886623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746806d950 0x7f7468152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 >> 172.21.15.170:0/1406886623 conn(0x7f74680cc1f0 msgr2=0x7f74680c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 shutdown_connections 2023-10-08T16:05:52.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.347+0000 7f746d097700 1 -- 172.21.15.170:0/1406886623 wait complete. 2023-10-08T16:05:52.360 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.474+0000 7f96bded4700 1 Processor -- start 2023-10-08T16:05:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.474+0000 7f96bded4700 1 -- start start 2023-10-08T16:05:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96bded4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96b8159c60 0x7f96b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96bded4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96b815db10 0x7f96b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96bded4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b815a160 con 0x7f96b8159890 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96bded4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b815df50 con 0x7f96b815db10 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96bded4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80d13a0 con 0x7f96b8159c60 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96b815db10 0x7f96b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96b815db10 0x7f96b8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38786/0 (socket says 172.21.15.170:38786) 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 -- 172.21.15.170:0/4164964323 learned_addr learned my addr 172.21.15.170:0/4164964323 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b6ffd700 1 --2- 172.21.15.170:0/4164964323 >> v2:172.21.15.191:3300/0 conn(0x7f96b8159c60 0x7f96b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 -- 172.21.15.170:0/4164964323 >> v2:172.21.15.191:3300/0 conn(0x7f96b8159c60 msgr2=0x7f96b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 --2- 172.21.15.170:0/4164964323 >> v2:172.21.15.191:3300/0 conn(0x7f96b8159c60 0x7f96b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 -- 172.21.15.170:0/4164964323 >> v1:172.21.15.191:6789/0 conn(0x7f96b8159890 legacy=0x7f96b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.475+0000 7f96b7fff700 1 -- 172.21.15.170:0/4164964323 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b815e140 con 0x7f96b815db10 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96b7fff700 1 --2- 172.21.15.170:0/4164964323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b8069560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96b80b67f0 tx=0x7f96b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da01819fc9eb2be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96b67fc700 1 -- 172.21.15.170:0/4164964323 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0009e70 con 0x7f96b815db10 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96b67fc700 1 -- 172.21.15.170:0/4164964323 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000b860 con 0x7f96b815db10 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96b67fc700 1 -- 172.21.15.170:0/4164964323 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000bc00 con 0x7f96b815db10 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 -- 172.21.15.170:0/4164964323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 msgr2=0x7f96b8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 --2- 172.21.15.170:0/4164964323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b8069560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f96b80b67f0 tx=0x7f96b0009250 comp rx=0 tx=0).stop 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 -- 172.21.15.170:0/4164964323 shutdown_connections 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 --2- 172.21.15.170:0/4164964323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b8069560 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 --2- 172.21.15.170:0/4164964323 >> v2:172.21.15.191:3300/0 conn(0x7f96b8159c60 0x7f96b815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 -- 172.21.15.170:0/4164964323 >> 172.21.15.170:0/4164964323 conn(0x7f96b8068e20 msgr2=0x7f96b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 -- 172.21.15.170:0/4164964323 shutdown_connections 2023-10-08T16:05:52.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.476+0000 7f96bded4700 1 -- 172.21.15.170:0/4164964323 wait complete. 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.477+0000 7f96bded4700 1 Processor -- start 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.477+0000 7f96bded4700 1 -- start start 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b8159890 0x7f96b8152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b8159c60 0x7f96b80ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80cf730 con 0x7f96b815db10 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80cd560 con 0x7f96b8159890 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96bded4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80cf170 con 0x7f96b8159c60 2023-10-08T16:05:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38798/0 (socket says 172.21.15.170:38798) 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96b7fff700 1 -- 172.21.15.170:0/309172316 learned_addr learned my addr 172.21.15.170:0/309172316 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96b77fe700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b8159890 0x7f96b8152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.478+0000 7f96b6ffd700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b8159c60 0x7f96b80ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 -- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b8159c60 msgr2=0x7f96b80ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b8159c60 0x7f96b80ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 -- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b8159890 msgr2=0x7f96b8152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b8159890 0x7f96b8152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 -- 172.21.15.170:0/309172316 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00094f0 con 0x7f96b815db10 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96b7fff700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009290 tx=0x7f96b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000baa0 con 0x7f96b815db10 2023-10-08T16:05:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0025420 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b80cf450 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b000bc00 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.479+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b80cd9f0 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.480+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b80cd3b0 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.480+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96b0023070 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.481+0000 7f969f7fe700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.481+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96b0085ac0 con 0x7f96b815db10 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.481+0000 7f96b77fe700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.481+0000 7f96b77fe700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 0x7f96a40693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007fd0 tx=0x7f96a80016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.485+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96b0052d30 con 0x7f96b815db10 2023-10-08T16:05:52.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.668+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96b80cd6d0 con 0x7f96b815db10 2023-10-08T16:05:52.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.669+0000 7f969f7fe700 1 -- 172.21.15.170:0/309172316 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f96b0085e60 con 0x7f96b815db10 2023-10-08T16:05:52.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:52.670 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":50,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:51.383137+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:52.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 msgr2=0x7f96a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 0x7f96a40693e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007fd0 tx=0x7f96a80016f0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 msgr2=0x7f96b815c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009290 tx=0x7f96b0017000 comp rx=0 tx=0).stop 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 shutdown_connections 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96a4066f30 0x7f96a40693e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b815db10 0x7f96b815c0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b8159890 0x7f96b8152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 --2- 172.21.15.170:0/309172316 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b8159c60 0x7f96b80ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 >> 172.21.15.170:0/309172316 conn(0x7f96b8068e20 msgr2=0x7f96b81583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 shutdown_connections 2023-10-08T16:05:52.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.671+0000 7f96bded4700 1 -- 172.21.15.170:0/309172316 wait complete. 2023-10-08T16:05:52.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:05:52.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.802+0000 7f008300f700 1 Processor -- start 2023-10-08T16:05:52.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.802+0000 7f008300f700 1 -- start start 2023-10-08T16:05:52.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f008300f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f008300f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f007c0c9510 0x7f007c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f008300f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f007c15c2a0 con 0x7f007c06c0a0 2023-10-08T16:05:52.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f008300f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007c15b690 con 0x7f007c0c9510 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f008300f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c15b970 con 0x7f007c06bcd0 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f00815ac700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f007c0c9510 0x7f007c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f00815ac700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f007c0c9510 0x7f007c141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38808/0 (socket says 172.21.15.170:38808) 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.803+0000 7f00815ac700 1 -- 172.21.15.170:0/751638392 learned_addr learned my addr 172.21.15.170:0/751638392 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00815ac700 1 -- 172.21.15.170:0/751638392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 msgr2=0x7f007c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:52.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00815ac700 1 --2- 172.21.15.170:0/751638392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00815ac700 1 -- 172.21.15.170:0/751638392 >> v1:172.21.15.191:6789/0 conn(0x7f007c06c0a0 legacy=0x7f007c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00815ac700 1 -- 172.21.15.170:0/751638392 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007c15bc50 con 0x7f007c0c9510 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00815ac700 1 --2- 172.21.15.170:0/751638392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c0c9510 0x7f007c141330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00740099b0 tx=0x7f0074009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631ea9d557db8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00737fe700 1 -- 172.21.15.170:0/751638392 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400b820 con 0x7f007c0c9510 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00737fe700 1 -- 172.21.15.170:0/751638392 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f007400b980 con 0x7f007c0c9510 2023-10-08T16:05:52.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f00737fe700 1 -- 172.21.15.170:0/751638392 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400bd20 con 0x7f007c0c9510 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 -- 172.21.15.170:0/751638392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c0c9510 msgr2=0x7f007c141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 --2- 172.21.15.170:0/751638392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c0c9510 0x7f007c141330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00740099b0 tx=0x7f0074009250 comp rx=0 tx=0).stop 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 -- 172.21.15.170:0/751638392 shutdown_connections 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 --2- 172.21.15.170:0/751638392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c0c9510 0x7f007c141330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 --2- 172.21.15.170:0/751638392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.804+0000 7f008300f700 1 -- 172.21.15.170:0/751638392 >> 172.21.15.170:0/751638392 conn(0x7f007c0ce600 msgr2=0x7f007c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:52.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.805+0000 7f008300f700 1 -- 172.21.15.170:0/751638392 shutdown_connections 2023-10-08T16:05:52.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.805+0000 7f008300f700 1 -- 172.21.15.170:0/751638392 wait complete. 2023-10-08T16:05:52.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 Processor -- start 2023-10-08T16:05:52.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 -- start start 2023-10-08T16:05:52.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c13bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f007c0c9510 0x7f007c0c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0c6380 con 0x7f007c06c0a0 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.806+0000 7f008300f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c0c7e40 con 0x7f007c0c9510 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f008300f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007c13c2f0 con 0x7f007c06bcd0 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38822/0 (socket says 172.21.15.170:38822) 2023-10-08T16:05:52.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 -- 172.21.15.170:0/106640060 learned_addr learned my addr 172.21.15.170:0/106640060 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0080dab700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c13bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f00815ac700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f007c0c9510 0x7f007c0c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 -- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 msgr2=0x7f007c13bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c13bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 -- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f007c0c9510 msgr2=0x7f007c0c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f007c0c9510 0x7f007c0c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:52.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 -- 172.21.15.170:0/106640060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740094f0 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0073fff700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c600 tx=0x7f006c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c011590 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c011710 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c0c7830 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c011ad0 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.807+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c0c5dc0 con 0x7f007c06c0a0 2023-10-08T16:05:52.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.808+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007c04c3c0 con 0x7f007c06c0a0 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.808+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f006c01c460 con 0x7f007c06c0a0 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.809+0000 7f0070ff9700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 0x7f00680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.809+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f006c0841b0 con 0x7f007c06c0a0 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.809+0000 7f0080dab700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 0x7f00680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.809+0000 7f0080dab700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 0x7f00680693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0078007580 tx=0x7f0078009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:52.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:52.813+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f006c051420 con 0x7f007c06c0a0 2023-10-08T16:05:53.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:53.040+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f007c06c9d0 con 0x7f007c06c0a0 2023-10-08T16:05:54.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.019+0000 7f0070ff9700 1 -- 172.21.15.170:0/106640060 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7f006c054a40 con 0x7f007c06c0a0 2023-10-08T16:05:54.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 msgr2=0x7f00680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 0x7f00680693e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0078007580 tx=0x7f0078009780 comp rx=0 tx=0).stop 2023-10-08T16:05:54.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 msgr2=0x7f007c16bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f006c00c600 tx=0x7f006c007f60 comp rx=0 tx=0).stop 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 shutdown_connections 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f007c0c9510 0x7f007c0c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f007c06bcd0 0x7f007c13bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0068066f30 0x7f00680693e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 --2- 172.21.15.170:0/106640060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007c06c0a0 0x7f007c16bf60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.021+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 >> 172.21.15.170:0/106640060 conn(0x7f007c0ce600 msgr2=0x7f007c0704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:54.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.022+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 shutdown_connections 2023-10-08T16:05:54.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.022+0000 7f008300f700 1 -- 172.21.15.170:0/106640060 wait complete. 2023-10-08T16:05:54.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:05:54.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.152+0000 7f01e1c06700 1 Processor -- start 2023-10-08T16:05:54.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 -- start start 2023-10-08T16:05:54.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01dc141500 0x7f01dc0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f01dc06c0a0 0x7f01dc0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01dc15c2a0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc0c9880 con 0x7f01dc141500 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01e1c06700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc15b690 con 0x7f01dc06c0a0 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.153+0000 7f01d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01dc141500 0x7f01dc0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01dc141500 0x7f01dc0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38834/0 (socket says 172.21.15.170:38834) 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 -- 172.21.15.170:0/4148057152 learned_addr learned my addr 172.21.15.170:0/4148057152 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01db7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f01dc06bcd0 0x7f01dc140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58932/0 (socket says 172.21.15.170:58932) 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 -- 172.21.15.170:0/4148057152 >> v2:172.21.15.191:3300/0 conn(0x7f01dc06c0a0 msgr2=0x7f01dc0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:54.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 --2- 172.21.15.170:0/4148057152 >> v2:172.21.15.191:3300/0 conn(0x7f01dc06c0a0 0x7f01dc0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 -- 172.21.15.170:0/4148057152 >> v1:172.21.15.170:6789/0 conn(0x7f01dc06bcd0 legacy=0x7f01dc140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01daffd700 1 -- 172.21.15.170:0/4148057152 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032081332 0 0) 0x7f01dc15c2a0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 -- 172.21.15.170:0/4148057152 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc15b970 con 0x7f01dc141500 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01d2ffd700 1 --2- 172.21.15.170:0/4148057152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc141500 0x7f01dc0c8e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0cf210 tx=0x7f01c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fefd4cdd53b4cbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01daffd700 1 -- 172.21.15.170:0/4148057152 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400b860 con 0x7f01dc141500 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01daffd700 1 -- 172.21.15.170:0/4148057152 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400b9c0 con 0x7f01dc141500 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.154+0000 7f01daffd700 1 -- 172.21.15.170:0/4148057152 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400bd60 con 0x7f01dc141500 2023-10-08T16:05:54.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 -- 172.21.15.170:0/4148057152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc141500 msgr2=0x7f01dc0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 --2- 172.21.15.170:0/4148057152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc141500 0x7f01dc0c8e00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0cf210 tx=0x7f01c4009250 comp rx=0 tx=0).stop 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 -- 172.21.15.170:0/4148057152 shutdown_connections 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 --2- 172.21.15.170:0/4148057152 >> v2:172.21.15.191:3300/0 conn(0x7f01dc06c0a0 0x7f01dc0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 --2- 172.21.15.170:0/4148057152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc141500 0x7f01dc0c8e00 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 -- 172.21.15.170:0/4148057152 >> 172.21.15.170:0/4148057152 conn(0x7f01dc0ce600 msgr2=0x7f01dc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 -- 172.21.15.170:0/4148057152 shutdown_connections 2023-10-08T16:05:54.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.155+0000 7f01e1c06700 1 -- 172.21.15.170:0/4148057152 wait complete. 2023-10-08T16:05:54.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.156+0000 7f01e1c06700 1 Processor -- start 2023-10-08T16:05:54.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.156+0000 7f01e1c06700 1 -- start start 2023-10-08T16:05:54.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01dc06c0a0 0x7f01dc13c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 0x7f01dc0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc0b24b0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc13ca20 con 0x7f01dc06c0a0 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc0b18a0 con 0x7f01dc141500 2023-10-08T16:05:54.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38840/0 (socket says 172.21.15.170:38840) 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 -- 172.21.15.170:0/1665436580 learned_addr learned my addr 172.21.15.170:0/1665436580 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01d2ffd700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01dc06c0a0 0x7f01dc13c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01dbfff700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 0x7f01dc0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 -- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 msgr2=0x7f01dc0c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 0x7f01dc0c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 -- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01dc06c0a0 msgr2=0x7f01dc13c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01dc06c0a0 0x7f01dc13c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 -- 172.21.15.170:0/1665436580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40094f0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01dbfff700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 0x7f01dc0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01db7fe700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0011e0 tx=0x7f01cc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc005a70 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01cc005bd0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0070d0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.157+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc0b1b80 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.158+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc0b2140 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.158+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc04c3c0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.159+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01cc007480 con 0x7f01dc06bcd0 2023-10-08T16:05:54.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.159+0000 7f01d3fff700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 0x7f01bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:54.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.159+0000 7f01d2ffd700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 0x7f01bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:54.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.159+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f01cc083980 con 0x7f01dc06bcd0 2023-10-08T16:05:54.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.160+0000 7f01d2ffd700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 0x7f01bc0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0cf210 tx=0x7f01c4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:54.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.164+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01cc04b5a0 con 0x7f01dc06bcd0 2023-10-08T16:05:54.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:54.391+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f01dc15bc50 con 0x7f01dc06bcd0 2023-10-08T16:05:55.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.023+0000 7f01d3fff700 1 -- 172.21.15.170:0/1665436580 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01dc06bcd0 2023-10-08T16:05:55.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 msgr2=0x7f01bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 0x7f01bc0693e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0cf210 tx=0x7f01c4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 msgr2=0x7f01dc0b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0011e0 tx=0x7f01cc007e10 comp rx=0 tx=0).stop 2023-10-08T16:05:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 shutdown_connections 2023-10-08T16:05:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01dc141500 0x7f01dc0c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc066f30 0x7f01bc0693e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01dc06bcd0 0x7f01dc0b2940 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 --2- 172.21.15.170:0/1665436580 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01dc06c0a0 0x7f01dc13c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 >> 172.21.15.170:0/1665436580 conn(0x7f01dc0ce600 msgr2=0x7f01dc0cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 shutdown_connections 2023-10-08T16:05:55.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.026+0000 7f01e1c06700 1 -- 172.21.15.170:0/1665436580 wait complete. 2023-10-08T16:05:55.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:55.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.155+0000 7fc8cd851700 1 Processor -- start 2023-10-08T16:05:55.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.155+0000 7fc8cd851700 1 -- start start 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd851700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc8c8159420 0x7fc8c80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd851700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 0x7fc8c815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd851700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c80b2980 con 0x7fc8c81597f0 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd851700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c815dbb0 con 0x7fc8c8159420 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd851700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80b11d0 con 0x7fc8c815d230 2023-10-08T16:05:55.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc8c8159420 0x7fc8c80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc8c8159420 0x7fc8c80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38842/0 (socket says 172.21.15.170:38842) 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 -- 172.21.15.170:0/2632632813 learned_addr learned my addr 172.21.15.170:0/2632632813 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd050700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 0x7fc8c815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 -- 172.21.15.170:0/2632632813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 msgr2=0x7fc8c815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 0x7fc8c815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 -- 172.21.15.170:0/2632632813 >> v1:172.21.15.191:6789/0 conn(0x7fc8c81597f0 legacy=0x7fc8c80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cc84f700 1 -- 172.21.15.170:0/2632632813 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c80b14b0 con 0x7fc8c8159420 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.156+0000 7fc8cd050700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 0x7fc8c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c8159420 0x7fc8c80b5f40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009f20 tx=0x7fc8c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433b667ad49f661 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8bf7fe700 1 -- 172.21.15.170:0/2632632813 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c400e070 con 0x7fc8c8159420 2023-10-08T16:05:55.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8bf7fe700 1 -- 172.21.15.170:0/2632632813 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c400ba40 con 0x7fc8c8159420 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8bf7fe700 1 -- 172.21.15.170:0/2632632813 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40092e0 con 0x7fc8c8159420 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 -- 172.21.15.170:0/2632632813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c8159420 msgr2=0x7fc8c80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c8159420 0x7fc8c80b5f40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009f20 tx=0x7fc8c4009840 comp rx=0 tx=0).stop 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 -- 172.21.15.170:0/2632632813 shutdown_connections 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c815d230 0x7fc8c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 --2- 172.21.15.170:0/2632632813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c8159420 0x7fc8c80b5f40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 -- 172.21.15.170:0/2632632813 >> 172.21.15.170:0/2632632813 conn(0x7fc8c80d0660 msgr2=0x7fc8c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 -- 172.21.15.170:0/2632632813 shutdown_connections 2023-10-08T16:05:55.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.157+0000 7fc8cd851700 1 -- 172.21.15.170:0/2632632813 wait complete. 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.158+0000 7fc8cd851700 1 Processor -- start 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 -- start start 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 0x7fc8c8143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 0x7fc8c8068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c81555d0 con 0x7fc8c81597f0 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c7110 con 0x7fc8c815d230 2023-10-08T16:05:55.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd851700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c80c6500 con 0x7fc8c8159420 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38846/0 (socket says 172.21.15.170:38846) 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8bffff700 1 -- 172.21.15.170:0/3337193520 learned_addr learned my addr 172.21.15.170:0/3337193520 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cd050700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 0x7fc8c8068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.159+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 0x7fc8c8143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 -- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 msgr2=0x7fc8c8143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 0x7fc8c8143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 -- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 msgr2=0x7fc8c8068c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 0x7fc8c8068c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 -- 172.21.15.170:0/3337193520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009ae0 con 0x7fc8c81597f0 2023-10-08T16:05:55.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 0x7fc8c8143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8cd050700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 0x7fc8c8068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bffff700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c590 tx=0x7fc8b8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011590 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b8011730 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b8011af0 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c80c67e0 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.160+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c80c6da0 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.161+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c80c7280 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.161+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc8b801c460 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.162+0000 7fc8bcff9700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 0x7fc8b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.162+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc8b80842e0 con 0x7fc8c81597f0 2023-10-08T16:05:55.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.162+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 0x7fc8b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.162+0000 7fc8cc84f700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 0x7fc8b40693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009880 tx=0x7fc8c4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.166+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc8b8051550 con 0x7fc8c81597f0 2023-10-08T16:05:55.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.393+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8c804c3c0 con 0x7fc8c81597f0 2023-10-08T16:05:55.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.394+0000 7fc8bcff9700 1 -- 172.21.15.170:0/3337193520 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fc8b8054b70 con 0x7fc8c81597f0 2023-10-08T16:05:55.394 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:55.394 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:05:55.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 msgr2=0x7fc8b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 0x7fc8b40693e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009880 tx=0x7fc8c4017000 comp rx=0 tx=0).stop 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 msgr2=0x7fc8c8068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8b800c590 tx=0x7fc8b8007f30 comp rx=0 tx=0).stop 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 shutdown_connections 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc8c815d230 0x7fc8c8068c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc8c8159420 0x7fc8c8143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc8b4066f30 0x7fc8b40693e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 --2- 172.21.15.170:0/3337193520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c81597f0 0x7fc8c8068710 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 >> 172.21.15.170:0/3337193520 conn(0x7fc8c80d0660 msgr2=0x7fc8c81561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 shutdown_connections 2023-10-08T16:05:55.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.396+0000 7fc8cd851700 1 -- 172.21.15.170:0/3337193520 wait complete. 2023-10-08T16:05:55.398 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 30 2023-10-08T16:05:55.408 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:05:55.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.522+0000 7f5f032c1700 1 Processor -- start 2023-10-08T16:05:55.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.522+0000 7f5f032c1700 1 -- start start 2023-10-08T16:05:55.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f032c1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5efc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f032c1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5efc0ac6a0 0x7f5efc0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f032c1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5efc0acac0 con 0x7f5efc0ac230 2023-10-08T16:05:55.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f032c1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5efc0ad930 con 0x7f5efc0ac6a0 2023-10-08T16:05:55.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f032c1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0acd20 con 0x7f5efc155c10 2023-10-08T16:05:55.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f0085c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5efc0ac230 0x7f5efc1568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60480/0 (socket says 172.21.15.170:60480) 2023-10-08T16:05:55.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.523+0000 7f5f0085c700 1 -- 172.21.15.170:0/2528621211 learned_addr learned my addr 172.21.15.170:0/2528621211 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:55.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0105d700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5efc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 --2- 172.21.15.170:0/2528621211 >> v2:172.21.15.191:3300/0 conn(0x7f5efc0ac6a0 0x7f5efc0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809083003 0 0) 0x7f5efc0acac0 con 0x7f5efc0ac230 2023-10-08T16:05:55.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee8003570 con 0x7f5efc0ac230 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1028781780 0 0) 0x7f5ee8003570 con 0x7f5efc0ac230 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 msgr2=0x7f5efc155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5efc155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 >> v2:172.21.15.191:3300/0 conn(0x7f5efc0ac6a0 msgr2=0x7f5efc0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 --2- 172.21.15.170:0/2528621211 >> v2:172.21.15.191:3300/0 conn(0x7f5efc0ac6a0 0x7f5efc0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0ad000 con 0x7f5efc0ac230 2023-10-08T16:05:55.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f5eec001470 con 0x7f5efc0ac230 2023-10-08T16:05:55.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 >> v1:172.21.15.170:6789/0 conn(0x7f5efc0ac230 legacy=0x7f5efc1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5ee8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee8003fe0 0x7f5ee8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0ad930 con 0x7f5ee8004d50 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc0acd20 con 0x7f5ee8003fe0 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee800ba90 con 0x7f5efc155c10 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0105d700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5ee8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0085c700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee8003fe0 0x7f5ee8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 -- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 msgr2=0x7f5ee8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc155c10 0x7f5ee8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 -- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee8003fe0 msgr2=0x7f5ee8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ee8003fe0 0x7f5ee8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.524+0000 7f5f0185e700 1 -- 172.21.15.170:0/2528621211 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8003570 con 0x7f5ee8004d50 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f0185e700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5ee80051c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009c00 tx=0x7f5ef40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1faf4003d66aef4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400e6b0 con 0x7f5ee8004d50 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400e810 con 0x7f5ee8004d50 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5ef3fff700 1 -- 172.21.15.170:0/2528621211 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400ebd0 con 0x7f5ee8004d50 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 -- 172.21.15.170:0/2528621211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 msgr2=0x7f5ee80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 --2- 172.21.15.170:0/2528621211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5ee80051c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009c00 tx=0x7f5ef40091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f0185e700 1 -- 172.21.15.170:0/2528621211 reap_dead start 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 -- 172.21.15.170:0/2528621211 shutdown_connections 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 -- 172.21.15.170:0/2528621211 >> 172.21.15.170:0/2528621211 conn(0x7f5efc06a320 msgr2=0x7f5efc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 -- 172.21.15.170:0/2528621211 shutdown_connections 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.525+0000 7f5f032c1700 1 -- 172.21.15.170:0/2528621211 wait complete. 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.526+0000 7f5f032c1700 1 Processor -- start 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 -- start start 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 0x7f5efc169130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 0x7f5efc06e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc16a7d0 con 0x7f5ee8004d50 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc154bc0 con 0x7f5efc0a98d0 2023-10-08T16:05:55.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.527+0000 7f5f032c1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5efc154ea0 con 0x7f5efc0c9b70 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36878/0 (socket says 172.21.15.170:36878) 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 -- 172.21.15.170:0/3409221600 learned_addr learned my addr 172.21.15.170:0/3409221600 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0085c700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 0x7f5efc169130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0185e700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 0x7f5efc06e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 -- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 msgr2=0x7f5efc169130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 0x7f5efc169130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 -- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 msgr2=0x7f5efc06e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 0x7f5efc06e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 -- 172.21.15.170:0/3409221600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc155180 con 0x7f5ee8004d50 2023-10-08T16:05:55.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0185e700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 0x7f5efc06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0085c700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 0x7f5efc169130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f0105d700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80032c0 tx=0x7f5ef800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef800e050 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef80153f0 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5efc14e420 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8015790 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.528+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5efc169670 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.529+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5efc04c3c0 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.529+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ef8015bc0 con 0x7f5ee8004d50 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.530+0000 7f5ef17fa700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 0x7f5ee00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.530+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5ef8085660 con 0x7f5ee8004d50 2023-10-08T16:05:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.530+0000 7f5f0085c700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 0x7f5ee00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.530+0000 7f5f0085c700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 0x7f5ee00693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001d20 tx=0x7f5eec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.534+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ef80528d0 con 0x7f5ee8004d50 2023-10-08T16:05:55.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.716+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5efc14ea70 con 0x7f5ee8004d50 2023-10-08T16:05:55.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.717+0000 7f5ef17fa700 1 -- 172.21.15.170:0/3409221600 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f5ef8055ef0 con 0x7f5ee8004d50 2023-10-08T16:05:55.717 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:55.717 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":51,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:53.008429+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 msgr2=0x7f5ee00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 0x7f5ee00693e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5eec001d20 tx=0x7f5eec009040 comp rx=0 tx=0).stop 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 msgr2=0x7f5efc16a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80032c0 tx=0x7f5ef800b040 comp rx=0 tx=0).stop 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 shutdown_connections 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5efc0a98d0 0x7f5efc06e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5ee0066f30 0x7f5ee00693e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ee8004d50 0x7f5efc16a290 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 --2- 172.21.15.170:0/3409221600 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5efc0c9b70 0x7f5efc169130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 >> 172.21.15.170:0/3409221600 conn(0x7f5efc06a320 msgr2=0x7f5efc151870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 shutdown_connections 2023-10-08T16:05:55.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.719+0000 7f5f032c1700 1 -- 172.21.15.170:0/3409221600 wait complete. 2023-10-08T16:05:55.732 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:05:55.732 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:05:55.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.845+0000 7f3259632700 1 Processor -- start 2023-10-08T16:05:55.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.845+0000 7f3259632700 1 -- start start 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f3259632700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3254159c60 0x7f325415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f3259632700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 0x7f3254069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f3259632700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f325415a160 con 0x7f3254159890 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f3259632700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325415df50 con 0x7f3254159c60 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f3259632700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32540d13a0 con 0x7f325415db10 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3254159c60 0x7f325415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3254159c60 0x7f325415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36884/0 (socket says 172.21.15.170:36884) 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 -- 172.21.15.170:0/2849404977 learned_addr learned my addr 172.21.15.170:0/2849404977 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f32537fe700 1 --2- 172.21.15.170:0/2849404977 >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 0x7f3254069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 -- 172.21.15.170:0/2849404977 >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 msgr2=0x7f3254069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 --2- 172.21.15.170:0/2849404977 >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 0x7f3254069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 -- 172.21.15.170:0/2849404977 >> v1:172.21.15.191:6789/0 conn(0x7f3254159890 legacy=0x7f325415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f324a7fc700 1 -- 172.21.15.170:0/2849404977 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325415e140 con 0x7f3254159c60 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.846+0000 7f32537fe700 1 --2- 172.21.15.170:0/2849404977 >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 0x7f3254069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f324a7fc700 1 --2- 172.21.15.170:0/2849404977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f325415d5d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f325406a890 tx=0x7f323c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f12dc5a4240324 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f32527fc700 1 -- 172.21.15.170:0/2849404977 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00b860 con 0x7f3254159c60 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f32527fc700 1 -- 172.21.15.170:0/2849404977 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c00b9c0 con 0x7f3254159c60 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f32527fc700 1 -- 172.21.15.170:0/2849404977 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c00bd60 con 0x7f3254159c60 2023-10-08T16:05:55.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 -- 172.21.15.170:0/2849404977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 msgr2=0x7f325415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 --2- 172.21.15.170:0/2849404977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f325415d5d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f325406a890 tx=0x7f323c009250 comp rx=0 tx=0).stop 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 -- 172.21.15.170:0/2849404977 shutdown_connections 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 --2- 172.21.15.170:0/2849404977 >> v2:172.21.15.191:3300/0 conn(0x7f325415db10 0x7f3254069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 --2- 172.21.15.170:0/2849404977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f325415d5d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 -- 172.21.15.170:0/2849404977 >> 172.21.15.170:0/2849404977 conn(0x7f3254068e20 msgr2=0x7f32540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 -- 172.21.15.170:0/2849404977 shutdown_connections 2023-10-08T16:05:55.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.847+0000 7f3259632700 1 -- 172.21.15.170:0/2849404977 wait complete. 2023-10-08T16:05:55.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 Processor -- start 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 -- start start 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3254159890 0x7f325414caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f325415db10 0x7f325415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325416bde0 con 0x7f3254159c60 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325416b1d0 con 0x7f325415db10 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3259632700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325416b4b0 con 0x7f3254159890 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f324a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f324a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36900/0 (socket says 172.21.15.170:36900) 2023-10-08T16:05:55.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f324a7fc700 1 -- 172.21.15.170:0/4030703176 learned_addr learned my addr 172.21.15.170:0/4030703176 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f32537fe700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f325415db10 0x7f325415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.849+0000 7f3252ffd700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3254159890 0x7f325414caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 -- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3254159890 msgr2=0x7f325414caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3254159890 0x7f325414caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 -- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f325415db10 msgr2=0x7f325415caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f325415db10 0x7f325415caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 -- 172.21.15.170:0/4030703176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0094f0 con 0x7f3254159c60 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324a7fc700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f323c009290 tx=0x7f323c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c026480 con 0x7f3254159c60 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c01f070 con 0x7f3254159c60 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c026850 con 0x7f3254159c60 2023-10-08T16:05:55.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325416b790 con 0x7f3254159c60 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.850+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32540b3590 con 0x7f3254159c60 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.851+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325404c3c0 con 0x7f3254159c60 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.851+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f323c01a430 con 0x7f3254159c60 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.852+0000 7f324b7fe700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.852+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f323c021070 con 0x7f3254159c60 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.852+0000 7f3252ffd700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:55.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.852+0000 7f3252ffd700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f32540ce190 tx=0x7f3244009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:55.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:55.856+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f323c053690 con 0x7f3254159c60 2023-10-08T16:05:56.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.034+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32540b1aa0 con 0x7f3234066f30 2023-10-08T16:05:56.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.041+0000 7f324b7fe700 1 -- 172.21.15.170:0/4030703176 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274530 (secure 0 0 0) 0x7f32540b1aa0 con 0x7f3234066f30 2023-10-08T16:05:56.048 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.044+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 msgr2=0x7f32340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.044+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 0x7f32340693e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f32540ce190 tx=0x7f3244009040 comp rx=0 tx=0).stop 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.044+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 msgr2=0x7f32540cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.044+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f323c009290 tx=0x7f323c017000 comp rx=0 tx=0).stop 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 shutdown_connections 2023-10-08T16:05:56.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f325415db10 0x7f325415caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3254159890 0x7f325414caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3234066f30 0x7f32340693e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 --2- 172.21.15.170:0/4030703176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3254159c60 0x7f32540cf030 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 >> 172.21.15.170:0/4030703176 conn(0x7f3254068e20 msgr2=0x7f32540c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 shutdown_connections 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.045+0000 7f3259632700 1 -- 172.21.15.170:0/4030703176 wait complete. 2023-10-08T16:05:56.052 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:05:56.057 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":87,"stamp":"2023-10-08T16:05:54.239669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2387896,"kb_used_data":5496,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752586824,"statfs":{"total":773094113280,"available":770648907776,"internally_reserved":0,"allocated":5627904,"data_stored":3012708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.246395"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:50.993409+0000","last_change":"2023-10-08T16:05:50.993409+0000","last_active":"2023-10-08T16:05:50.993409+0000","last_peered":"2023-10-08T16:05:50.993409+0000","last_clean":"2023-10-08T16:05:50.993409+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:50.993409+0000","last_undegraded":"2023-10-08T16:05:50.993409+0000","last_fullsized":"2023-10-08T16:05:50.993409+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T16:05:49.988483+0000","last_change":"2023-10-08T16:05:49.988483+0000","last_active":"2023-10-08T16:05:49.988483+0000","last_peered":"2023-10-08T16:05:49.988483+0000","last_clean":"2023-10-08T16:05:49.988483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:05:49.988483+0000","last_undegraded":"2023-10-08T16:05:49.988483+0000","last_fullsized":"2023-10-08T16:05:49.988483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-08T16:05:46.987086+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:05:46.987086+0000","last_peered":"2023-10-08T16:05:46.987086+0000","last_clean":"2023-10-08T16:05:46.987086+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:05:46.987086+0000","last_undegraded":"2023-10-08T16:05:46.987086+0000","last_fullsized":"2023-10-08T16:05:46.987086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298416,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073424,"statfs":{"total":96636764160,"available":96331186176,"internally_reserved":0,"allocated":1155072,"data_stored":821957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":8.0939999999999994,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.439}]},{"osd":1,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":8.2040000000000006,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.36199999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.51100000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.3409999999999993,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.56599999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":231589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.627}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55100000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57799999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":231589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.318}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302500,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069340,"statfs":{"total":96636764160,"available":96327004160,"internally_reserved":0,"allocated":1142784,"data_stored":816772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.52400000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.53700000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.73199999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":226404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.312}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.70899999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297836,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074004,"statfs":{"total":96636764160,"available":96331780096,"internally_reserved":0,"allocated":561152,"data_stored":231589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.46100000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.52200000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.59499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":226404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.503},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":226404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.497},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.83799999999999997}]},{"osd":1,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":12.227,"5min":12.227,"15min":12.227},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.79900000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42499999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.751}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:05:56.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:05:56.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.211+0000 7f05b359e700 1 Processor -- start 2023-10-08T16:05:56.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.211+0000 7f05b359e700 1 -- start start 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b359e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f05b41597f0 0x7f05b40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 0x7f05b415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b415dbb0 con 0x7f05b4159420 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b359e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b40b11d0 con 0x7f05b41597f0 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40b14b0 con 0x7f05b415d230 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f05b4159420 0x7f05b40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60486/0 (socket says 172.21.15.170:60486) 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b259c700 1 -- 172.21.15.170:0/3559317154 learned_addr learned my addr 172.21.15.170:0/3559317154 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f05b41597f0 0x7f05b40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b2d9d700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 0x7f05b415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 -- 172.21.15.170:0/3559317154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 msgr2=0x7f05b415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 0x7f05b415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 -- 172.21.15.170:0/3559317154 >> v1:172.21.15.170:6789/0 conn(0x7f05b4159420 legacy=0x7f05b40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b1d9b700 1 -- 172.21.15.170:0/3559317154 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38701530 0 0) 0x7f05b415dbb0 con 0x7f05b4159420 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 -- 172.21.15.170:0/3559317154 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40b1790 con 0x7f05b41597f0 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05b2d9d700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 0x7f05b415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.212+0000 7f05abfff700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b41597f0 0x7f05b40b2270 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f059c0098e0 tx=0x7f059c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1219c3e86ae7c29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b1d9b700 1 -- 172.21.15.170:0/3559317154 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c009e80 con 0x7f05b41597f0 2023-10-08T16:05:56.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b1d9b700 1 -- 172.21.15.170:0/3559317154 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c00b860 con 0x7f05b41597f0 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b1d9b700 1 -- 172.21.15.170:0/3559317154 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00bbe0 con 0x7f05b41597f0 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 -- 172.21.15.170:0/3559317154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b41597f0 msgr2=0x7f05b40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b41597f0 0x7f05b40b2270 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f059c0098e0 tx=0x7f059c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 -- 172.21.15.170:0/3559317154 shutdown_connections 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b415d230 0x7f05b415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 --2- 172.21.15.170:0/3559317154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b41597f0 0x7f05b40b2270 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 -- 172.21.15.170:0/3559317154 >> 172.21.15.170:0/3559317154 conn(0x7f05b40d0660 msgr2=0x7f05b40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 -- 172.21.15.170:0/3559317154 shutdown_connections 2023-10-08T16:05:56.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.213+0000 7f05b359e700 1 -- 172.21.15.170:0/3559317154 wait complete. 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.214+0000 7f05b359e700 1 Processor -- start 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.214+0000 7f05b359e700 1 -- start start 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41597f0 0x7f05b4154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 0x7f05b4155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b415cb30 con 0x7f05b4159420 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b41696e0 con 0x7f05b415d230 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b40c59c0 con 0x7f05b41597f0 2023-10-08T16:05:56.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36926/0 (socket says 172.21.15.170:36926) 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 -- 172.21.15.170:0/829842398 learned_addr learned my addr 172.21.15.170:0/829842398 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05abfff700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41597f0 0x7f05b4154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b2d9d700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 0x7f05b4155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 -- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41597f0 msgr2=0x7f05b4154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41597f0 0x7f05b4154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 -- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 msgr2=0x7f05b4155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 0x7f05b4155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 -- 172.21.15.170:0/829842398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c009420 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b2d9d700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 0x7f05b4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.215+0000 7f05b259c700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c7e0 tx=0x7f05a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.216+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4005b30 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.216+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4005c90 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.216+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b40c5ca0 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.216+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40072e0 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.216+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b40c6260 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.217+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b40b6470 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.217+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05a40076d0 con 0x7f05b4159420 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.217+0000 7f05aa7fc700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 0x7f05940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.217+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f05a4083910 con 0x7f05b4159420 2023-10-08T16:05:56.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.217+0000 7f05abfff700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 0x7f05940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.218+0000 7f05abfff700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 0x7f05940693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f05b4159c70 tx=0x7f059c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.221+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a404b5a0 con 0x7f05b4159420 2023-10-08T16:05:56.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.399+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05b4169820 con 0x7f0594066f30 2023-10-08T16:05:56.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.407+0000 7f05aa7fc700 1 -- 172.21.15.170:0/829842398 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289269 (secure 0 0 0) 0x7f05b4169820 con 0x7f0594066f30 2023-10-08T16:05:56.411 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:56.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 msgr2=0x7f05940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 0x7f05940693e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f05b4159c70 tx=0x7f059c017000 comp rx=0 tx=0).stop 2023-10-08T16:05:56.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 msgr2=0x7f05b415c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c7e0 tx=0x7f05a4007df0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 shutdown_connections 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b415d230 0x7f05b4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0594066f30 0x7f05940693e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b4159420 0x7f05b415c5f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 --2- 172.21.15.170:0/829842398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b41597f0 0x7f05b4154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 >> 172.21.15.170:0/829842398 conn(0x7f05b40d0660 msgr2=0x7f05b41550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 shutdown_connections 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.411+0000 7f05b359e700 1 -- 172.21.15.170:0/829842398 wait complete. 2023-10-08T16:05:56.414 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:05:56.423 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2023-10-08T16:05:56.239975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2388356,"kb_used_data":5956,"kb_used_omap":0,"kb_used_meta":2382336,"kb_avail":752586364,"statfs":{"total":773094113280,"available":770648436736,"internally_reserved":0,"allocated":6098944,"data_stored":3220084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2439512064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001027"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020143+0000","last_change":"2023-10-08T16:05:52.017582+0000","last_active":"2023-10-08T16:05:52.020143+0000","last_peered":"2023-10-08T16:05:52.020143+0000","last_clean":"2023-10-08T16:05:52.020143+0000","last_became_active":"2023-10-08T16:05:52.017309+0000","last_became_peered":"2023-10-08T16:05:52.017309+0000","last_unstale":"2023-10-08T16:05:52.020143+0000","last_undegraded":"2023-10-08T16:05:52.020143+0000","last_fullsized":"2023-10-08T16:05:52.020143+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:51.690618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022232+0000","last_change":"2023-10-08T16:05:52.015930+0000","last_active":"2023-10-08T16:05:52.022232+0000","last_peered":"2023-10-08T16:05:52.022232+0000","last_clean":"2023-10-08T16:05:52.022232+0000","last_became_active":"2023-10-08T16:05:52.013694+0000","last_became_peered":"2023-10-08T16:05:52.013694+0000","last_unstale":"2023-10-08T16:05:52.022232+0000","last_undegraded":"2023-10-08T16:05:52.022232+0000","last_fullsized":"2023-10-08T16:05:52.022232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:02.690998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.034759+0000","last_change":"2023-10-08T16:05:52.030841+0000","last_active":"2023-10-08T16:05:52.034759+0000","last_peered":"2023-10-08T16:05:52.034759+0000","last_clean":"2023-10-08T16:05:52.034759+0000","last_became_active":"2023-10-08T16:05:52.030558+0000","last_became_peered":"2023-10-08T16:05:52.030558+0000","last_unstale":"2023-10-08T16:05:52.034759+0000","last_undegraded":"2023-10-08T16:05:52.034759+0000","last_fullsized":"2023-10-08T16:05:52.034759+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:18:09.916634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.031770+0000","last_change":"2023-10-08T16:05:52.025775+0000","last_active":"2023-10-08T16:05:52.031770+0000","last_peered":"2023-10-08T16:05:52.031770+0000","last_clean":"2023-10-08T16:05:52.031770+0000","last_became_active":"2023-10-08T16:05:52.025505+0000","last_became_peered":"2023-10-08T16:05:52.025505+0000","last_unstale":"2023-10-08T16:05:52.031770+0000","last_undegraded":"2023-10-08T16:05:52.031770+0000","last_fullsized":"2023-10-08T16:05:52.031770+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:07.336001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023428+0000","last_change":"2023-10-08T16:05:52.022467+0000","last_active":"2023-10-08T16:05:52.023428+0000","last_peered":"2023-10-08T16:05:52.023428+0000","last_clean":"2023-10-08T16:05:52.023428+0000","last_became_active":"2023-10-08T16:05:52.022227+0000","last_became_peered":"2023-10-08T16:05:52.022227+0000","last_unstale":"2023-10-08T16:05:52.023428+0000","last_undegraded":"2023-10-08T16:05:52.023428+0000","last_fullsized":"2023-10-08T16:05:52.023428+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:31:53.891204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.019927+0000","last_change":"2023-10-08T16:05:52.015164+0000","last_active":"2023-10-08T16:05:52.019927+0000","last_peered":"2023-10-08T16:05:52.019927+0000","last_clean":"2023-10-08T16:05:52.019927+0000","last_became_active":"2023-10-08T16:05:52.014909+0000","last_became_peered":"2023-10-08T16:05:52.014909+0000","last_unstale":"2023-10-08T16:05:52.019927+0000","last_undegraded":"2023-10-08T16:05:52.019927+0000","last_fullsized":"2023-10-08T16:05:52.019927+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:19:02.508753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029684+0000","last_change":"2023-10-08T16:05:52.024220+0000","last_active":"2023-10-08T16:05:52.029684+0000","last_peered":"2023-10-08T16:05:52.029684+0000","last_clean":"2023-10-08T16:05:52.029684+0000","last_became_active":"2023-10-08T16:05:52.023965+0000","last_became_peered":"2023-10-08T16:05:52.023965+0000","last_unstale":"2023-10-08T16:05:52.029684+0000","last_undegraded":"2023-10-08T16:05:52.029684+0000","last_fullsized":"2023-10-08T16:05:52.029684+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:56.186095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025412+0000","last_change":"2023-10-08T16:05:52.021341+0000","last_active":"2023-10-08T16:05:52.025412+0000","last_peered":"2023-10-08T16:05:52.025412+0000","last_clean":"2023-10-08T16:05:52.025412+0000","last_became_active":"2023-10-08T16:05:52.018844+0000","last_became_peered":"2023-10-08T16:05:52.018844+0000","last_unstale":"2023-10-08T16:05:52.025412+0000","last_undegraded":"2023-10-08T16:05:52.025412+0000","last_fullsized":"2023-10-08T16:05:52.025412+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:28.797800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028933+0000","last_change":"2023-10-08T16:05:52.015814+0000","last_active":"2023-10-08T16:05:52.028933+0000","last_peered":"2023-10-08T16:05:52.028933+0000","last_clean":"2023-10-08T16:05:52.028933+0000","last_became_active":"2023-10-08T16:05:52.015482+0000","last_became_peered":"2023-10-08T16:05:52.015482+0000","last_unstale":"2023-10-08T16:05:52.028933+0000","last_undegraded":"2023-10-08T16:05:52.028933+0000","last_fullsized":"2023-10-08T16:05:52.028933+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:24.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021575+0000","last_change":"2023-10-08T16:05:52.013363+0000","last_active":"2023-10-08T16:05:52.021575+0000","last_peered":"2023-10-08T16:05:52.021575+0000","last_clean":"2023-10-08T16:05:52.021575+0000","last_became_active":"2023-10-08T16:05:52.013105+0000","last_became_peered":"2023-10-08T16:05:52.013105+0000","last_unstale":"2023-10-08T16:05:52.021575+0000","last_undegraded":"2023-10-08T16:05:52.021575+0000","last_fullsized":"2023-10-08T16:05:52.021575+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:06:15.501462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021883+0000","last_change":"2023-10-08T16:05:52.016642+0000","last_active":"2023-10-08T16:05:52.021883+0000","last_peered":"2023-10-08T16:05:52.021883+0000","last_clean":"2023-10-08T16:05:52.021883+0000","last_became_active":"2023-10-08T16:05:52.016370+0000","last_became_peered":"2023-10-08T16:05:52.016370+0000","last_unstale":"2023-10-08T16:05:52.021883+0000","last_undegraded":"2023-10-08T16:05:52.021883+0000","last_fullsized":"2023-10-08T16:05:52.021883+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:43.906170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.026882+0000","last_change":"2023-10-08T16:05:52.019282+0000","last_active":"2023-10-08T16:05:52.026882+0000","last_peered":"2023-10-08T16:05:52.026882+0000","last_clean":"2023-10-08T16:05:52.026882+0000","last_became_active":"2023-10-08T16:05:52.019088+0000","last_became_peered":"2023-10-08T16:05:52.019088+0000","last_unstale":"2023-10-08T16:05:52.026882+0000","last_undegraded":"2023-10-08T16:05:52.026882+0000","last_fullsized":"2023-10-08T16:05:52.026882+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:01:29.921431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.018382+0000","last_change":"2023-10-08T16:05:52.013142+0000","last_active":"2023-10-08T16:05:52.018382+0000","last_peered":"2023-10-08T16:05:52.018382+0000","last_clean":"2023-10-08T16:05:52.018382+0000","last_became_active":"2023-10-08T16:05:52.012861+0000","last_became_peered":"2023-10-08T16:05:52.012861+0000","last_unstale":"2023-10-08T16:05:52.018382+0000","last_undegraded":"2023-10-08T16:05:52.018382+0000","last_fullsized":"2023-10-08T16:05:52.018382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:00.476677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029624+0000","last_change":"2023-10-08T16:05:52.021321+0000","last_active":"2023-10-08T16:05:52.029624+0000","last_peered":"2023-10-08T16:05:52.029624+0000","last_clean":"2023-10-08T16:05:52.029624+0000","last_became_active":"2023-10-08T16:05:52.021037+0000","last_became_peered":"2023-10-08T16:05:52.021037+0000","last_unstale":"2023-10-08T16:05:52.029624+0000","last_undegraded":"2023-10-08T16:05:52.029624+0000","last_fullsized":"2023-10-08T16:05:52.029624+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:25.195176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029099+0000","last_change":"2023-10-08T16:05:52.015058+0000","last_active":"2023-10-08T16:05:52.029099+0000","last_peered":"2023-10-08T16:05:52.029099+0000","last_clean":"2023-10-08T16:05:52.029099+0000","last_became_active":"2023-10-08T16:05:52.014791+0000","last_became_peered":"2023-10-08T16:05:52.014791+0000","last_unstale":"2023-10-08T16:05:52.029099+0000","last_undegraded":"2023-10-08T16:05:52.029099+0000","last_fullsized":"2023-10-08T16:05:52.029099+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:06:15.501462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023160+0000","last_change":"2023-10-08T16:05:52.021247+0000","last_active":"2023-10-08T16:05:52.023160+0000","last_peered":"2023-10-08T16:05:52.023160+0000","last_clean":"2023-10-08T16:05:52.023160+0000","last_became_active":"2023-10-08T16:05:52.020961+0000","last_became_peered":"2023-10-08T16:05:52.020961+0000","last_unstale":"2023-10-08T16:05:52.023160+0000","last_undegraded":"2023-10-08T16:05:52.023160+0000","last_fullsized":"2023-10-08T16:05:52.023160+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:37.288369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025355+0000","last_change":"2023-10-08T16:05:52.021014+0000","last_active":"2023-10-08T16:05:52.025355+0000","last_peered":"2023-10-08T16:05:52.025355+0000","last_clean":"2023-10-08T16:05:52.025355+0000","last_became_active":"2023-10-08T16:05:52.019399+0000","last_became_peered":"2023-10-08T16:05:52.019399+0000","last_unstale":"2023-10-08T16:05:52.025355+0000","last_undegraded":"2023-10-08T16:05:52.025355+0000","last_fullsized":"2023-10-08T16:05:52.025355+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:51:43.253902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028631+0000","last_change":"2023-10-08T16:05:52.019432+0000","last_active":"2023-10-08T16:05:52.028631+0000","last_peered":"2023-10-08T16:05:52.028631+0000","last_clean":"2023-10-08T16:05:52.028631+0000","last_became_active":"2023-10-08T16:05:52.019191+0000","last_became_peered":"2023-10-08T16:05:52.019191+0000","last_unstale":"2023-10-08T16:05:52.028631+0000","last_undegraded":"2023-10-08T16:05:52.028631+0000","last_fullsized":"2023-10-08T16:05:52.028631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:12.502843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020149+0000","last_change":"2023-10-08T16:05:52.017504+0000","last_active":"2023-10-08T16:05:52.020149+0000","last_peered":"2023-10-08T16:05:52.020149+0000","last_clean":"2023-10-08T16:05:52.020149+0000","last_became_active":"2023-10-08T16:05:52.017249+0000","last_became_peered":"2023-10-08T16:05:52.017249+0000","last_unstale":"2023-10-08T16:05:52.020149+0000","last_undegraded":"2023-10-08T16:05:52.020149+0000","last_fullsized":"2023-10-08T16:05:52.020149+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:07.577919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023293+0000","last_change":"2023-10-08T16:05:52.018732+0000","last_active":"2023-10-08T16:05:52.023293+0000","last_peered":"2023-10-08T16:05:52.023293+0000","last_clean":"2023-10-08T16:05:52.023293+0000","last_became_active":"2023-10-08T16:05:52.018497+0000","last_became_peered":"2023-10-08T16:05:52.018497+0000","last_unstale":"2023-10-08T16:05:52.023293+0000","last_undegraded":"2023-10-08T16:05:52.023293+0000","last_fullsized":"2023-10-08T16:05:52.023293+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:10.358267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.030771+0000","last_change":"2023-10-08T16:05:52.027246+0000","last_active":"2023-10-08T16:05:52.030771+0000","last_peered":"2023-10-08T16:05:52.030771+0000","last_clean":"2023-10-08T16:05:52.030771+0000","last_became_active":"2023-10-08T16:05:52.026969+0000","last_became_peered":"2023-10-08T16:05:52.026969+0000","last_unstale":"2023-10-08T16:05:52.030771+0000","last_undegraded":"2023-10-08T16:05:52.030771+0000","last_fullsized":"2023-10-08T16:05:52.030771+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:04:46.085625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022492+0000","last_change":"2023-10-08T16:05:52.018159+0000","last_active":"2023-10-08T16:05:52.022492+0000","last_peered":"2023-10-08T16:05:52.022492+0000","last_clean":"2023-10-08T16:05:52.022492+0000","last_became_active":"2023-10-08T16:05:52.017171+0000","last_became_peered":"2023-10-08T16:05:52.017171+0000","last_unstale":"2023-10-08T16:05:52.022492+0000","last_undegraded":"2023-10-08T16:05:52.022492+0000","last_fullsized":"2023-10-08T16:05:52.022492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:32.384794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.030652+0000","last_change":"2023-10-08T16:05:52.025780+0000","last_active":"2023-10-08T16:05:52.030652+0000","last_peered":"2023-10-08T16:05:52.030652+0000","last_clean":"2023-10-08T16:05:52.030652+0000","last_became_active":"2023-10-08T16:05:52.023651+0000","last_became_peered":"2023-10-08T16:05:52.023651+0000","last_unstale":"2023-10-08T16:05:52.030652+0000","last_undegraded":"2023-10-08T16:05:52.030652+0000","last_fullsized":"2023-10-08T16:05:52.030652+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:14.149958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.018340+0000","last_change":"2023-10-08T16:05:52.013920+0000","last_active":"2023-10-08T16:05:52.018340+0000","last_peered":"2023-10-08T16:05:52.018340+0000","last_clean":"2023-10-08T16:05:52.018340+0000","last_became_active":"2023-10-08T16:05:52.013661+0000","last_became_peered":"2023-10-08T16:05:52.013661+0000","last_unstale":"2023-10-08T16:05:52.018340+0000","last_undegraded":"2023-10-08T16:05:52.018340+0000","last_fullsized":"2023-10-08T16:05:52.018340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:37:51.744471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020029+0000","last_change":"2023-10-08T16:05:52.014666+0000","last_active":"2023-10-08T16:05:52.020029+0000","last_peered":"2023-10-08T16:05:52.020029+0000","last_clean":"2023-10-08T16:05:52.020029+0000","last_became_active":"2023-10-08T16:05:52.014416+0000","last_became_peered":"2023-10-08T16:05:52.014416+0000","last_unstale":"2023-10-08T16:05:52.020029+0000","last_undegraded":"2023-10-08T16:05:52.020029+0000","last_fullsized":"2023-10-08T16:05:52.020029+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:41:47.338302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.034612+0000","last_change":"2023-10-08T16:05:52.029628+0000","last_active":"2023-10-08T16:05:52.034612+0000","last_peered":"2023-10-08T16:05:52.034612+0000","last_clean":"2023-10-08T16:05:52.034612+0000","last_became_active":"2023-10-08T16:05:52.029379+0000","last_became_peered":"2023-10-08T16:05:52.029379+0000","last_unstale":"2023-10-08T16:05:52.034612+0000","last_undegraded":"2023-10-08T16:05:52.034612+0000","last_fullsized":"2023-10-08T16:05:52.034612+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:37.288369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021042+0000","last_change":"2023-10-08T16:05:52.017977+0000","last_active":"2023-10-08T16:05:52.021042+0000","last_peered":"2023-10-08T16:05:52.021042+0000","last_clean":"2023-10-08T16:05:52.021042+0000","last_became_active":"2023-10-08T16:05:52.017741+0000","last_became_peered":"2023-10-08T16:05:52.017741+0000","last_unstale":"2023-10-08T16:05:52.021042+0000","last_undegraded":"2023-10-08T16:05:52.021042+0000","last_fullsized":"2023-10-08T16:05:52.021042+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:17.831495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.027857+0000","last_change":"2023-10-08T16:05:52.027442+0000","last_active":"2023-10-08T16:05:52.027857+0000","last_peered":"2023-10-08T16:05:52.027857+0000","last_clean":"2023-10-08T16:05:52.027857+0000","last_became_active":"2023-10-08T16:05:52.027208+0000","last_became_peered":"2023-10-08T16:05:52.027208+0000","last_unstale":"2023-10-08T16:05:52.027857+0000","last_undegraded":"2023-10-08T16:05:52.027857+0000","last_fullsized":"2023-10-08T16:05:52.027857+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:14.678392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025624+0000","last_change":"2023-10-08T16:05:52.021701+0000","last_active":"2023-10-08T16:05:52.025624+0000","last_peered":"2023-10-08T16:05:52.025624+0000","last_clean":"2023-10-08T16:05:52.025624+0000","last_became_active":"2023-10-08T16:05:52.019414+0000","last_became_peered":"2023-10-08T16:05:52.019414+0000","last_unstale":"2023-10-08T16:05:52.025624+0000","last_undegraded":"2023-10-08T16:05:52.025624+0000","last_fullsized":"2023-10-08T16:05:52.025624+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:46:44.126408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.032036+0000","last_change":"2023-10-08T16:05:52.029815+0000","last_active":"2023-10-08T16:05:52.032036+0000","last_peered":"2023-10-08T16:05:52.032036+0000","last_clean":"2023-10-08T16:05:52.032036+0000","last_became_active":"2023-10-08T16:05:52.029424+0000","last_became_peered":"2023-10-08T16:05:52.029424+0000","last_unstale":"2023-10-08T16:05:52.032036+0000","last_undegraded":"2023-10-08T16:05:52.032036+0000","last_fullsized":"2023-10-08T16:05:52.032036+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:04.569960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020059+0000","last_change":"2023-10-08T16:05:52.015059+0000","last_active":"2023-10-08T16:05:52.020059+0000","last_peered":"2023-10-08T16:05:52.020059+0000","last_clean":"2023-10-08T16:05:52.020059+0000","last_became_active":"2023-10-08T16:05:52.014794+0000","last_became_peered":"2023-10-08T16:05:52.014794+0000","last_unstale":"2023-10-08T16:05:52.020059+0000","last_undegraded":"2023-10-08T16:05:52.020059+0000","last_fullsized":"2023-10-08T16:05:52.020059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:45:12.176469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028246+0000","last_change":"2023-10-08T16:05:52.023018+0000","last_active":"2023-10-08T16:05:52.028246+0000","last_peered":"2023-10-08T16:05:52.028246+0000","last_clean":"2023-10-08T16:05:52.028246+0000","last_became_active":"2023-10-08T16:05:52.022749+0000","last_became_peered":"2023-10-08T16:05:52.022749+0000","last_unstale":"2023-10-08T16:05:52.028246+0000","last_undegraded":"2023-10-08T16:05:52.028246+0000","last_fullsized":"2023-10-08T16:05:52.028246+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:02.998124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021097+0000","last_change":"2023-10-08T16:05:52.018328+0000","last_active":"2023-10-08T16:05:52.021097+0000","last_peered":"2023-10-08T16:05:52.021097+0000","last_clean":"2023-10-08T16:05:52.021097+0000","last_became_active":"2023-10-08T16:05:52.018037+0000","last_became_peered":"2023-10-08T16:05:52.018037+0000","last_unstale":"2023-10-08T16:05:52.021097+0000","last_undegraded":"2023-10-08T16:05:52.021097+0000","last_fullsized":"2023-10-08T16:05:52.021097+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:20.687205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"50'4","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.024844+0000","last_change":"2023-10-08T16:05:51.022340+0000","last_active":"2023-10-08T16:05:52.024844+0000","last_peered":"2023-10-08T16:05:52.024844+0000","last_clean":"2023-10-08T16:05:52.024844+0000","last_became_active":"2023-10-08T16:05:51.022065+0000","last_became_peered":"2023-10-08T16:05:51.022065+0000","last_unstale":"2023-10-08T16:05:52.024844+0000","last_undegraded":"2023-10-08T16:05:52.024844+0000","last_fullsized":"2023-10-08T16:05:52.024844+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:16.318467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023137+0000","last_change":"2023-10-08T16:05:52.019707+0000","last_active":"2023-10-08T16:05:52.023137+0000","last_peered":"2023-10-08T16:05:52.023137+0000","last_clean":"2023-10-08T16:05:52.023137+0000","last_became_active":"2023-10-08T16:05:52.019474+0000","last_became_peered":"2023-10-08T16:05:52.019474+0000","last_unstale":"2023-10-08T16:05:52.023137+0000","last_undegraded":"2023-10-08T16:05:52.023137+0000","last_fullsized":"2023-10-08T16:05:52.023137+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:56.186095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023167+0000","last_change":"2023-10-08T16:05:51.015992+0000","last_active":"2023-10-08T16:05:52.023167+0000","last_peered":"2023-10-08T16:05:52.023167+0000","last_clean":"2023-10-08T16:05:52.023167+0000","last_became_active":"2023-10-08T16:05:51.015425+0000","last_became_peered":"2023-10-08T16:05:51.015425+0000","last_unstale":"2023-10-08T16:05:52.023167+0000","last_undegraded":"2023-10-08T16:05:52.023167+0000","last_fullsized":"2023-10-08T16:05:52.023167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:55:29.165548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.019389+0000","last_change":"2023-10-08T16:05:52.013203+0000","last_active":"2023-10-08T16:05:52.019389+0000","last_peered":"2023-10-08T16:05:52.019389+0000","last_clean":"2023-10-08T16:05:52.019389+0000","last_became_active":"2023-10-08T16:05:52.012950+0000","last_became_peered":"2023-10-08T16:05:52.012950+0000","last_unstale":"2023-10-08T16:05:52.019389+0000","last_undegraded":"2023-10-08T16:05:52.019389+0000","last_fullsized":"2023-10-08T16:05:52.019389+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:25.195176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023783+0000","last_change":"2023-10-08T16:05:51.015518+0000","last_active":"2023-10-08T16:05:52.023783+0000","last_peered":"2023-10-08T16:05:52.023783+0000","last_clean":"2023-10-08T16:05:52.023783+0000","last_became_active":"2023-10-08T16:05:51.015275+0000","last_became_peered":"2023-10-08T16:05:51.015275+0000","last_unstale":"2023-10-08T16:05:52.023783+0000","last_undegraded":"2023-10-08T16:05:52.023783+0000","last_fullsized":"2023-10-08T16:05:52.023783+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:18:53.332979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028296+0000","last_change":"2023-10-08T16:05:52.024894+0000","last_active":"2023-10-08T16:05:52.028296+0000","last_peered":"2023-10-08T16:05:52.028296+0000","last_clean":"2023-10-08T16:05:52.028296+0000","last_became_active":"2023-10-08T16:05:52.024624+0000","last_became_peered":"2023-10-08T16:05:52.024624+0000","last_unstale":"2023-10-08T16:05:52.028296+0000","last_undegraded":"2023-10-08T16:05:52.028296+0000","last_fullsized":"2023-10-08T16:05:52.028296+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:07.336001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028770+0000","last_change":"2023-10-08T16:05:51.024169+0000","last_active":"2023-10-08T16:05:52.028770+0000","last_peered":"2023-10-08T16:05:52.028770+0000","last_clean":"2023-10-08T16:05:52.028770+0000","last_became_active":"2023-10-08T16:05:51.023901+0000","last_became_peered":"2023-10-08T16:05:51.023901+0000","last_unstale":"2023-10-08T16:05:52.028770+0000","last_undegraded":"2023-10-08T16:05:52.028770+0000","last_fullsized":"2023-10-08T16:05:52.028770+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:33:19.216262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.024955+0000","last_change":"2023-10-08T16:05:52.019290+0000","last_active":"2023-10-08T16:05:52.024955+0000","last_peered":"2023-10-08T16:05:52.024955+0000","last_clean":"2023-10-08T16:05:52.024955+0000","last_became_active":"2023-10-08T16:05:52.019038+0000","last_became_peered":"2023-10-08T16:05:52.019038+0000","last_unstale":"2023-10-08T16:05:52.024955+0000","last_undegraded":"2023-10-08T16:05:52.024955+0000","last_fullsized":"2023-10-08T16:05:52.024955+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:11:58.842943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028699+0000","last_change":"2023-10-08T16:05:51.015209+0000","last_active":"2023-10-08T16:05:52.028699+0000","last_peered":"2023-10-08T16:05:52.028699+0000","last_clean":"2023-10-08T16:05:52.028699+0000","last_became_active":"2023-10-08T16:05:51.014863+0000","last_became_peered":"2023-10-08T16:05:51.014863+0000","last_unstale":"2023-10-08T16:05:52.028699+0000","last_undegraded":"2023-10-08T16:05:52.028699+0000","last_fullsized":"2023-10-08T16:05:52.028699+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:51.560299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025380+0000","last_change":"2023-10-08T16:05:52.019662+0000","last_active":"2023-10-08T16:05:52.025380+0000","last_peered":"2023-10-08T16:05:52.025380+0000","last_clean":"2023-10-08T16:05:52.025380+0000","last_became_active":"2023-10-08T16:05:52.018274+0000","last_became_peered":"2023-10-08T16:05:52.018274+0000","last_unstale":"2023-10-08T16:05:52.025380+0000","last_undegraded":"2023-10-08T16:05:52.025380+0000","last_fullsized":"2023-10-08T16:05:52.025380+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:08:41.295850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020773+0000","last_change":"2023-10-08T16:05:51.018787+0000","last_active":"2023-10-08T16:05:52.020773+0000","last_peered":"2023-10-08T16:05:52.020773+0000","last_clean":"2023-10-08T16:05:52.020773+0000","last_became_active":"2023-10-08T16:05:51.018577+0000","last_became_peered":"2023-10-08T16:05:51.018577+0000","last_unstale":"2023-10-08T16:05:52.020773+0000","last_undegraded":"2023-10-08T16:05:52.020773+0000","last_fullsized":"2023-10-08T16:05:52.020773+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:49.677892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.027064+0000","last_change":"2023-10-08T16:05:52.020529+0000","last_active":"2023-10-08T16:05:52.027064+0000","last_peered":"2023-10-08T16:05:52.027064+0000","last_clean":"2023-10-08T16:05:52.027064+0000","last_became_active":"2023-10-08T16:05:52.020137+0000","last_became_peered":"2023-10-08T16:05:52.020137+0000","last_unstale":"2023-10-08T16:05:52.027064+0000","last_undegraded":"2023-10-08T16:05:52.027064+0000","last_fullsized":"2023-10-08T16:05:52.027064+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:11:06.316150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.012231+0000","last_change":"2023-10-08T16:05:51.022944+0000","last_active":"2023-10-08T16:05:52.012231+0000","last_peered":"2023-10-08T16:05:52.012231+0000","last_clean":"2023-10-08T16:05:52.012231+0000","last_became_active":"2023-10-08T16:05:51.022377+0000","last_became_peered":"2023-10-08T16:05:51.022377+0000","last_unstale":"2023-10-08T16:05:52.012231+0000","last_undegraded":"2023-10-08T16:05:52.012231+0000","last_fullsized":"2023-10-08T16:05:52.012231+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:17.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022288+0000","last_change":"2023-10-08T16:05:52.017532+0000","last_active":"2023-10-08T16:05:52.022288+0000","last_peered":"2023-10-08T16:05:52.022288+0000","last_clean":"2023-10-08T16:05:52.022288+0000","last_became_active":"2023-10-08T16:05:52.016943+0000","last_became_peered":"2023-10-08T16:05:52.016943+0000","last_unstale":"2023-10-08T16:05:52.022288+0000","last_undegraded":"2023-10-08T16:05:52.022288+0000","last_fullsized":"2023-10-08T16:05:52.022288+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:12.192252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.004888+0000","last_change":"2023-10-08T16:05:51.016894+0000","last_active":"2023-10-08T16:05:52.004888+0000","last_peered":"2023-10-08T16:05:52.004888+0000","last_clean":"2023-10-08T16:05:52.004888+0000","last_became_active":"2023-10-08T16:05:51.016564+0000","last_became_peered":"2023-10-08T16:05:51.016564+0000","last_unstale":"2023-10-08T16:05:52.004888+0000","last_undegraded":"2023-10-08T16:05:52.004888+0000","last_fullsized":"2023-10-08T16:05:52.004888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:52.569116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029547+0000","last_change":"2023-10-08T16:05:52.021091+0000","last_active":"2023-10-08T16:05:52.029547+0000","last_peered":"2023-10-08T16:05:52.029547+0000","last_clean":"2023-10-08T16:05:52.029547+0000","last_became_active":"2023-10-08T16:05:52.014738+0000","last_became_peered":"2023-10-08T16:05:52.014738+0000","last_unstale":"2023-10-08T16:05:52.029547+0000","last_undegraded":"2023-10-08T16:05:52.029547+0000","last_fullsized":"2023-10-08T16:05:52.029547+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:51.380007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.010041+0000","last_change":"2023-10-08T16:05:51.016925+0000","last_active":"2023-10-08T16:05:52.010041+0000","last_peered":"2023-10-08T16:05:52.010041+0000","last_clean":"2023-10-08T16:05:52.010041+0000","last_became_active":"2023-10-08T16:05:51.016617+0000","last_became_peered":"2023-10-08T16:05:51.016617+0000","last_unstale":"2023-10-08T16:05:52.010041+0000","last_undegraded":"2023-10-08T16:05:52.010041+0000","last_fullsized":"2023-10-08T16:05:52.010041+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:50.375081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.030603+0000","last_change":"2023-10-08T16:05:52.023260+0000","last_active":"2023-10-08T16:05:52.030603+0000","last_peered":"2023-10-08T16:05:52.030603+0000","last_clean":"2023-10-08T16:05:52.030603+0000","last_became_active":"2023-10-08T16:05:52.022713+0000","last_became_peered":"2023-10-08T16:05:52.022713+0000","last_unstale":"2023-10-08T16:05:52.030603+0000","last_undegraded":"2023-10-08T16:05:52.030603+0000","last_fullsized":"2023-10-08T16:05:52.030603+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:19.755642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.005777+0000","last_change":"2023-10-08T16:05:51.016134+0000","last_active":"2023-10-08T16:05:52.005777+0000","last_peered":"2023-10-08T16:05:52.005777+0000","last_clean":"2023-10-08T16:05:52.005777+0000","last_became_active":"2023-10-08T16:05:51.015888+0000","last_became_peered":"2023-10-08T16:05:51.015888+0000","last_unstale":"2023-10-08T16:05:52.005777+0000","last_undegraded":"2023-10-08T16:05:52.005777+0000","last_fullsized":"2023-10-08T16:05:52.005777+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:35.532401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.012246+0000","last_change":"2023-10-08T16:05:52.010518+0000","last_active":"2023-10-08T16:05:52.012246+0000","last_peered":"2023-10-08T16:05:52.012246+0000","last_clean":"2023-10-08T16:05:52.012246+0000","last_became_active":"2023-10-08T16:05:52.010089+0000","last_became_peered":"2023-10-08T16:05:52.010089+0000","last_unstale":"2023-10-08T16:05:52.012246+0000","last_undegraded":"2023-10-08T16:05:52.012246+0000","last_fullsized":"2023-10-08T16:05:52.012246+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:37.547773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"50'3","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.026516+0000","last_change":"2023-10-08T16:05:51.010863+0000","last_active":"2023-10-08T16:05:52.026516+0000","last_peered":"2023-10-08T16:05:52.026516+0000","last_clean":"2023-10-08T16:05:52.026516+0000","last_became_active":"2023-10-08T16:05:51.009896+0000","last_became_peered":"2023-10-08T16:05:51.009896+0000","last_unstale":"2023-10-08T16:05:52.026516+0000","last_undegraded":"2023-10-08T16:05:52.026516+0000","last_fullsized":"2023-10-08T16:05:52.026516+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:42.901245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028552+0000","last_change":"2023-10-08T16:05:52.018678+0000","last_active":"2023-10-08T16:05:52.028552+0000","last_peered":"2023-10-08T16:05:52.028552+0000","last_clean":"2023-10-08T16:05:52.028552+0000","last_became_active":"2023-10-08T16:05:52.018372+0000","last_became_peered":"2023-10-08T16:05:52.018372+0000","last_unstale":"2023-10-08T16:05:52.028552+0000","last_undegraded":"2023-10-08T16:05:52.028552+0000","last_fullsized":"2023-10-08T16:05:52.028552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:52:54.449040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"50'6","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.031987+0000","last_change":"2023-10-08T16:05:51.016216+0000","last_active":"2023-10-08T16:05:52.031987+0000","last_peered":"2023-10-08T16:05:52.031987+0000","last_clean":"2023-10-08T16:05:52.031987+0000","last_became_active":"2023-10-08T16:05:51.015907+0000","last_became_peered":"2023-10-08T16:05:51.015907+0000","last_unstale":"2023-10-08T16:05:52.031987+0000","last_undegraded":"2023-10-08T16:05:52.031987+0000","last_fullsized":"2023-10-08T16:05:52.031987+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:59.827491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.024413+0000","last_change":"2023-10-08T16:05:52.017258+0000","last_active":"2023-10-08T16:05:52.024413+0000","last_peered":"2023-10-08T16:05:52.024413+0000","last_clean":"2023-10-08T16:05:52.024413+0000","last_became_active":"2023-10-08T16:05:52.017016+0000","last_became_peered":"2023-10-08T16:05:52.017016+0000","last_unstale":"2023-10-08T16:05:52.024413+0000","last_undegraded":"2023-10-08T16:05:52.024413+0000","last_fullsized":"2023-10-08T16:05:52.024413+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:02.998124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022554+0000","last_change":"2023-10-08T16:05:51.016271+0000","last_active":"2023-10-08T16:05:52.022554+0000","last_peered":"2023-10-08T16:05:52.022554+0000","last_clean":"2023-10-08T16:05:52.022554+0000","last_became_active":"2023-10-08T16:05:51.015585+0000","last_became_peered":"2023-10-08T16:05:51.015585+0000","last_unstale":"2023-10-08T16:05:52.022554+0000","last_undegraded":"2023-10-08T16:05:52.022554+0000","last_fullsized":"2023-10-08T16:05:52.022554+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:19.682280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028833+0000","last_change":"2023-10-08T16:05:52.020582+0000","last_active":"2023-10-08T16:05:52.028833+0000","last_peered":"2023-10-08T16:05:52.028833+0000","last_clean":"2023-10-08T16:05:52.028833+0000","last_became_active":"2023-10-08T16:05:52.020088+0000","last_became_peered":"2023-10-08T16:05:52.020088+0000","last_unstale":"2023-10-08T16:05:52.028833+0000","last_undegraded":"2023-10-08T16:05:52.028833+0000","last_fullsized":"2023-10-08T16:05:52.028833+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:36.537326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.008420+0000","last_change":"2023-10-08T16:05:51.018379+0000","last_active":"2023-10-08T16:05:52.008420+0000","last_peered":"2023-10-08T16:05:52.008420+0000","last_clean":"2023-10-08T16:05:52.008420+0000","last_became_active":"2023-10-08T16:05:51.017986+0000","last_became_peered":"2023-10-08T16:05:51.017986+0000","last_unstale":"2023-10-08T16:05:52.008420+0000","last_undegraded":"2023-10-08T16:05:52.008420+0000","last_fullsized":"2023-10-08T16:05:52.008420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:55:42.762048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029656+0000","last_change":"2023-10-08T16:05:52.022686+0000","last_active":"2023-10-08T16:05:52.029656+0000","last_peered":"2023-10-08T16:05:52.029656+0000","last_clean":"2023-10-08T16:05:52.029656+0000","last_became_active":"2023-10-08T16:05:52.022301+0000","last_became_peered":"2023-10-08T16:05:52.022301+0000","last_unstale":"2023-10-08T16:05:52.029656+0000","last_undegraded":"2023-10-08T16:05:52.029656+0000","last_fullsized":"2023-10-08T16:05:52.029656+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:15:37.547773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.012335+0000","last_change":"2023-10-08T16:05:51.022754+0000","last_active":"2023-10-08T16:05:52.012335+0000","last_peered":"2023-10-08T16:05:52.012335+0000","last_clean":"2023-10-08T16:05:52.012335+0000","last_became_active":"2023-10-08T16:05:51.022083+0000","last_became_peered":"2023-10-08T16:05:51.022083+0000","last_unstale":"2023-10-08T16:05:52.012335+0000","last_undegraded":"2023-10-08T16:05:52.012335+0000","last_fullsized":"2023-10-08T16:05:52.012335+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:54:49.677892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.030630+0000","last_change":"2023-10-08T16:05:52.023797+0000","last_active":"2023-10-08T16:05:52.030630+0000","last_peered":"2023-10-08T16:05:52.030630+0000","last_clean":"2023-10-08T16:05:52.030630+0000","last_became_active":"2023-10-08T16:05:52.023301+0000","last_became_peered":"2023-10-08T16:05:52.023301+0000","last_unstale":"2023-10-08T16:05:52.030630+0000","last_undegraded":"2023-10-08T16:05:52.030630+0000","last_fullsized":"2023-10-08T16:05:52.030630+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:20.687205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"50'8","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028286+0000","last_change":"2023-10-08T16:05:51.024010+0000","last_active":"2023-10-08T16:05:52.028286+0000","last_peered":"2023-10-08T16:05:52.028286+0000","last_clean":"2023-10-08T16:05:52.028286+0000","last_became_active":"2023-10-08T16:05:51.023722+0000","last_became_peered":"2023-10-08T16:05:51.023722+0000","last_unstale":"2023-10-08T16:05:52.028286+0000","last_undegraded":"2023-10-08T16:05:52.028286+0000","last_fullsized":"2023-10-08T16:05:52.028286+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:22:17.635113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.031987+0000","last_change":"2023-10-08T16:05:52.029691+0000","last_active":"2023-10-08T16:05:52.031987+0000","last_peered":"2023-10-08T16:05:52.031987+0000","last_clean":"2023-10-08T16:05:52.031987+0000","last_became_active":"2023-10-08T16:05:52.029429+0000","last_became_peered":"2023-10-08T16:05:52.029429+0000","last_unstale":"2023-10-08T16:05:52.031987+0000","last_undegraded":"2023-10-08T16:05:52.031987+0000","last_fullsized":"2023-10-08T16:05:52.031987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:42:02.998124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022403+0000","last_change":"2023-10-08T16:05:51.019245+0000","last_active":"2023-10-08T16:05:52.022403+0000","last_peered":"2023-10-08T16:05:52.022403+0000","last_clean":"2023-10-08T16:05:52.022403+0000","last_became_active":"2023-10-08T16:05:51.017589+0000","last_became_peered":"2023-10-08T16:05:51.017589+0000","last_unstale":"2023-10-08T16:05:52.022403+0000","last_undegraded":"2023-10-08T16:05:52.022403+0000","last_fullsized":"2023-10-08T16:05:52.022403+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:16.826569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.031727+0000","last_change":"2023-10-08T16:05:52.025365+0000","last_active":"2023-10-08T16:05:52.031727+0000","last_peered":"2023-10-08T16:05:52.031727+0000","last_clean":"2023-10-08T16:05:52.031727+0000","last_became_active":"2023-10-08T16:05:52.024611+0000","last_became_peered":"2023-10-08T16:05:52.024611+0000","last_unstale":"2023-10-08T16:05:52.031727+0000","last_undegraded":"2023-10-08T16:05:52.031727+0000","last_fullsized":"2023-10-08T16:05:52.031727+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:24:17.831495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.020453+0000","last_change":"2023-10-08T16:05:51.015632+0000","last_active":"2023-10-08T16:05:52.020453+0000","last_peered":"2023-10-08T16:05:52.020453+0000","last_clean":"2023-10-08T16:05:52.020453+0000","last_became_active":"2023-10-08T16:05:51.015390+0000","last_became_peered":"2023-10-08T16:05:51.015390+0000","last_unstale":"2023-10-08T16:05:52.020453+0000","last_undegraded":"2023-10-08T16:05:52.020453+0000","last_fullsized":"2023-10-08T16:05:52.020453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:37:39.867564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.026435+0000","last_change":"2023-10-08T16:05:52.025706+0000","last_active":"2023-10-08T16:05:52.026435+0000","last_peered":"2023-10-08T16:05:52.026435+0000","last_clean":"2023-10-08T16:05:52.026435+0000","last_became_active":"2023-10-08T16:05:52.025484+0000","last_became_peered":"2023-10-08T16:05:52.025484+0000","last_unstale":"2023-10-08T16:05:52.026435+0000","last_undegraded":"2023-10-08T16:05:52.026435+0000","last_fullsized":"2023-10-08T16:05:52.026435+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:15.313383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"50'1","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.026424+0000","last_change":"2023-10-08T16:05:51.022334+0000","last_active":"2023-10-08T16:05:52.026424+0000","last_peered":"2023-10-08T16:05:52.026424+0000","last_clean":"2023-10-08T16:05:52.026424+0000","last_became_active":"2023-10-08T16:05:51.021913+0000","last_became_peered":"2023-10-08T16:05:51.021913+0000","last_unstale":"2023-10-08T16:05:52.026424+0000","last_undegraded":"2023-10-08T16:05:52.026424+0000","last_fullsized":"2023-10-08T16:05:52.026424+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:50.375081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029949+0000","last_change":"2023-10-08T16:05:52.022282+0000","last_active":"2023-10-08T16:05:52.029949+0000","last_peered":"2023-10-08T16:05:52.029949+0000","last_clean":"2023-10-08T16:05:52.029949+0000","last_became_active":"2023-10-08T16:05:52.021988+0000","last_became_peered":"2023-10-08T16:05:52.021988+0000","last_unstale":"2023-10-08T16:05:52.029949+0000","last_undegraded":"2023-10-08T16:05:52.029949+0000","last_fullsized":"2023-10-08T16:05:52.029949+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:42:09.158367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.010247+0000","last_change":"2023-10-08T16:05:51.016996+0000","last_active":"2023-10-08T16:05:52.010247+0000","last_peered":"2023-10-08T16:05:52.010247+0000","last_clean":"2023-10-08T16:05:52.010247+0000","last_became_active":"2023-10-08T16:05:51.016719+0000","last_became_peered":"2023-10-08T16:05:51.016719+0000","last_unstale":"2023-10-08T16:05:52.010247+0000","last_undegraded":"2023-10-08T16:05:52.010247+0000","last_fullsized":"2023-10-08T16:05:52.010247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:59.827491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028082+0000","last_change":"2023-10-08T16:05:52.024920+0000","last_active":"2023-10-08T16:05:52.028082+0000","last_peered":"2023-10-08T16:05:52.028082+0000","last_clean":"2023-10-08T16:05:52.028082+0000","last_became_active":"2023-10-08T16:05:52.024559+0000","last_became_peered":"2023-10-08T16:05:52.024559+0000","last_unstale":"2023-10-08T16:05:52.028082+0000","last_undegraded":"2023-10-08T16:05:52.028082+0000","last_fullsized":"2023-10-08T16:05:52.028082+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:55:30.170473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.029140+0000","last_change":"2023-10-08T16:05:51.015633+0000","last_active":"2023-10-08T16:05:52.029140+0000","last_peered":"2023-10-08T16:05:52.029140+0000","last_clean":"2023-10-08T16:05:52.029140+0000","last_became_active":"2023-10-08T16:05:51.015335+0000","last_became_peered":"2023-10-08T16:05:51.015335+0000","last_unstale":"2023-10-08T16:05:52.029140+0000","last_undegraded":"2023-10-08T16:05:52.029140+0000","last_fullsized":"2023-10-08T16:05:52.029140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:06.331076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021399+0000","last_change":"2023-10-08T16:05:52.017982+0000","last_active":"2023-10-08T16:05:52.021399+0000","last_peered":"2023-10-08T16:05:52.021399+0000","last_clean":"2023-10-08T16:05:52.021399+0000","last_became_active":"2023-10-08T16:05:52.017704+0000","last_became_peered":"2023-10-08T16:05:52.017704+0000","last_unstale":"2023-10-08T16:05:52.021399+0000","last_undegraded":"2023-10-08T16:05:52.021399+0000","last_fullsized":"2023-10-08T16:05:52.021399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:56.186095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.001869+0000","last_change":"2023-10-08T16:05:51.015314+0000","last_active":"2023-10-08T16:05:52.001869+0000","last_peered":"2023-10-08T16:05:52.001869+0000","last_clean":"2023-10-08T16:05:52.001869+0000","last_became_active":"2023-10-08T16:05:51.015064+0000","last_became_peered":"2023-10-08T16:05:51.015064+0000","last_unstale":"2023-10-08T16:05:52.001869+0000","last_undegraded":"2023-10-08T16:05:52.001869+0000","last_fullsized":"2023-10-08T16:05:52.001869+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:44.996872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021686+0000","last_change":"2023-10-08T16:05:52.019233+0000","last_active":"2023-10-08T16:05:52.021686+0000","last_peered":"2023-10-08T16:05:52.021686+0000","last_clean":"2023-10-08T16:05:52.021686+0000","last_became_active":"2023-10-08T16:05:52.018987+0000","last_became_peered":"2023-10-08T16:05:52.018987+0000","last_unstale":"2023-10-08T16:05:52.021686+0000","last_undegraded":"2023-10-08T16:05:52.021686+0000","last_fullsized":"2023-10-08T16:05:52.021686+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:17.968741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.011791+0000","last_change":"2023-10-08T16:05:51.022187+0000","last_active":"2023-10-08T16:05:52.011791+0000","last_peered":"2023-10-08T16:05:52.011791+0000","last_clean":"2023-10-08T16:05:52.011791+0000","last_became_active":"2023-10-08T16:05:51.021902+0000","last_became_peered":"2023-10-08T16:05:51.021902+0000","last_unstale":"2023-10-08T16:05:52.011791+0000","last_undegraded":"2023-10-08T16:05:52.011791+0000","last_fullsized":"2023-10-08T16:05:52.011791+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:17:01.050624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.023830+0000","last_change":"2023-10-08T16:05:52.021628+0000","last_active":"2023-10-08T16:05:52.023830+0000","last_peered":"2023-10-08T16:05:52.023830+0000","last_clean":"2023-10-08T16:05:52.023830+0000","last_became_active":"2023-10-08T16:05:52.021331+0000","last_became_peered":"2023-10-08T16:05:52.021331+0000","last_unstale":"2023-10-08T16:05:52.023830+0000","last_undegraded":"2023-10-08T16:05:52.023830+0000","last_fullsized":"2023-10-08T16:05:52.023830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:36.537326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"50'2","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025133+0000","last_change":"2023-10-08T16:05:51.005637+0000","last_active":"2023-10-08T16:05:52.025133+0000","last_peered":"2023-10-08T16:05:52.025133+0000","last_clean":"2023-10-08T16:05:52.025133+0000","last_became_active":"2023-10-08T16:05:51.005376+0000","last_became_peered":"2023-10-08T16:05:51.005376+0000","last_unstale":"2023-10-08T16:05:52.025133+0000","last_undegraded":"2023-10-08T16:05:52.025133+0000","last_fullsized":"2023-10-08T16:05:52.025133+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:52:52.569116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.031828+0000","last_change":"2023-10-08T16:05:52.026788+0000","last_active":"2023-10-08T16:05:52.031828+0000","last_peered":"2023-10-08T16:05:52.031828+0000","last_clean":"2023-10-08T16:05:52.031828+0000","last_became_active":"2023-10-08T16:05:52.026409+0000","last_became_peered":"2023-10-08T16:05:52.026409+0000","last_unstale":"2023-10-08T16:05:52.031828+0000","last_undegraded":"2023-10-08T16:05:52.031828+0000","last_fullsized":"2023-10-08T16:05:52.031828+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:20.687205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.009674+0000","last_change":"2023-10-08T16:05:51.015193+0000","last_active":"2023-10-08T16:05:52.009674+0000","last_peered":"2023-10-08T16:05:52.009674+0000","last_clean":"2023-10-08T16:05:52.009674+0000","last_became_active":"2023-10-08T16:05:51.014929+0000","last_became_peered":"2023-10-08T16:05:51.014929+0000","last_unstale":"2023-10-08T16:05:52.009674+0000","last_undegraded":"2023-10-08T16:05:52.009674+0000","last_fullsized":"2023-10-08T16:05:52.009674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:17:01.050624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.012227+0000","last_change":"2023-10-08T16:05:52.010080+0000","last_active":"2023-10-08T16:05:52.012227+0000","last_peered":"2023-10-08T16:05:52.012227+0000","last_clean":"2023-10-08T16:05:52.012227+0000","last_became_active":"2023-10-08T16:05:52.009792+0000","last_became_peered":"2023-10-08T16:05:52.009792+0000","last_unstale":"2023-10-08T16:05:52.012227+0000","last_undegraded":"2023-10-08T16:05:52.012227+0000","last_fullsized":"2023-10-08T16:05:52.012227+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:24.063114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.008304+0000","last_change":"2023-10-08T16:05:51.011752+0000","last_active":"2023-10-08T16:05:52.008304+0000","last_peered":"2023-10-08T16:05:52.008304+0000","last_clean":"2023-10-08T16:05:52.008304+0000","last_became_active":"2023-10-08T16:05:51.011344+0000","last_became_peered":"2023-10-08T16:05:51.011344+0000","last_unstale":"2023-10-08T16:05:52.008304+0000","last_undegraded":"2023-10-08T16:05:52.008304+0000","last_fullsized":"2023-10-08T16:05:52.008304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:19.749336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028284+0000","last_change":"2023-10-08T16:05:52.024802+0000","last_active":"2023-10-08T16:05:52.028284+0000","last_peered":"2023-10-08T16:05:52.028284+0000","last_clean":"2023-10-08T16:05:52.028284+0000","last_became_active":"2023-10-08T16:05:52.024535+0000","last_became_peered":"2023-10-08T16:05:52.024535+0000","last_unstale":"2023-10-08T16:05:52.028284+0000","last_undegraded":"2023-10-08T16:05:52.028284+0000","last_fullsized":"2023-10-08T16:05:52.028284+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:55:30.170473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.011394+0000","last_change":"2023-10-08T16:05:51.024133+0000","last_active":"2023-10-08T16:05:52.011394+0000","last_peered":"2023-10-08T16:05:52.011394+0000","last_clean":"2023-10-08T16:05:52.011394+0000","last_became_active":"2023-10-08T16:05:51.023865+0000","last_became_peered":"2023-10-08T16:05:51.023865+0000","last_unstale":"2023-10-08T16:05:52.011394+0000","last_undegraded":"2023-10-08T16:05:52.011394+0000","last_fullsized":"2023-10-08T16:05:52.011394+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:42.901245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022156+0000","last_change":"2023-10-08T16:05:52.015874+0000","last_active":"2023-10-08T16:05:52.022156+0000","last_peered":"2023-10-08T16:05:52.022156+0000","last_clean":"2023-10-08T16:05:52.022156+0000","last_became_active":"2023-10-08T16:05:52.013619+0000","last_became_peered":"2023-10-08T16:05:52.013619+0000","last_unstale":"2023-10-08T16:05:52.022156+0000","last_undegraded":"2023-10-08T16:05:52.022156+0000","last_fullsized":"2023-10-08T16:05:52.022156+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:12.578850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.001502+0000","last_change":"2023-10-08T16:05:51.022400+0000","last_active":"2023-10-08T16:05:52.001502+0000","last_peered":"2023-10-08T16:05:52.001502+0000","last_clean":"2023-10-08T16:05:52.001502+0000","last_became_active":"2023-10-08T16:05:51.022149+0000","last_became_peered":"2023-10-08T16:05:51.022149+0000","last_unstale":"2023-10-08T16:05:52.001502+0000","last_undegraded":"2023-10-08T16:05:52.001502+0000","last_fullsized":"2023-10-08T16:05:52.001502+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:51.560299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.028854+0000","last_change":"2023-10-08T16:05:52.020683+0000","last_active":"2023-10-08T16:05:52.028854+0000","last_peered":"2023-10-08T16:05:52.028854+0000","last_clean":"2023-10-08T16:05:52.028854+0000","last_became_active":"2023-10-08T16:05:52.020479+0000","last_became_peered":"2023-10-08T16:05:52.020479+0000","last_unstale":"2023-10-08T16:05:52.028854+0000","last_undegraded":"2023-10-08T16:05:52.028854+0000","last_fullsized":"2023-10-08T16:05:52.028854+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:07.336001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.003591+0000","last_change":"2023-10-08T16:05:51.016178+0000","last_active":"2023-10-08T16:05:52.003591+0000","last_peered":"2023-10-08T16:05:52.003591+0000","last_clean":"2023-10-08T16:05:52.003591+0000","last_became_active":"2023-10-08T16:05:51.015691+0000","last_became_peered":"2023-10-08T16:05:51.015691+0000","last_unstale":"2023-10-08T16:05:52.003591+0000","last_undegraded":"2023-10-08T16:05:52.003591+0000","last_fullsized":"2023-10-08T16:05:52.003591+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:31:44.996872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.021690+0000","last_change":"2023-10-08T16:05:52.013914+0000","last_active":"2023-10-08T16:05:52.021690+0000","last_peered":"2023-10-08T16:05:52.021690+0000","last_clean":"2023-10-08T16:05:52.021690+0000","last_became_active":"2023-10-08T16:05:52.013615+0000","last_became_peered":"2023-10-08T16:05:52.013615+0000","last_unstale":"2023-10-08T16:05:52.021690+0000","last_undegraded":"2023-10-08T16:05:52.021690+0000","last_fullsized":"2023-10-08T16:05:52.021690+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:25.195176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.001466+0000","last_change":"2023-10-08T16:05:51.017318+0000","last_active":"2023-10-08T16:05:52.001466+0000","last_peered":"2023-10-08T16:05:52.001466+0000","last_clean":"2023-10-08T16:05:52.001466+0000","last_became_active":"2023-10-08T16:05:51.017077+0000","last_became_peered":"2023-10-08T16:05:51.017077+0000","last_unstale":"2023-10-08T16:05:52.001466+0000","last_undegraded":"2023-10-08T16:05:52.001466+0000","last_fullsized":"2023-10-08T16:05:52.001466+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:59.827491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.022272+0000","last_change":"2023-10-08T16:05:52.016946+0000","last_active":"2023-10-08T16:05:52.022272+0000","last_peered":"2023-10-08T16:05:52.022272+0000","last_clean":"2023-10-08T16:05:52.022272+0000","last_became_active":"2023-10-08T16:05:52.016708+0000","last_became_peered":"2023-10-08T16:05:52.016708+0000","last_unstale":"2023-10-08T16:05:52.022272+0000","last_undegraded":"2023-10-08T16:05:52.022272+0000","last_fullsized":"2023-10-08T16:05:52.022272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:04:05.511181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.004956+0000","last_change":"2023-10-08T16:05:51.023201+0000","last_active":"2023-10-08T16:05:52.004956+0000","last_peered":"2023-10-08T16:05:52.004956+0000","last_clean":"2023-10-08T16:05:52.004956+0000","last_became_active":"2023-10-08T16:05:51.022892+0000","last_became_peered":"2023-10-08T16:05:51.022892+0000","last_unstale":"2023-10-08T16:05:52.004956+0000","last_undegraded":"2023-10-08T16:05:52.004956+0000","last_fullsized":"2023-10-08T16:05:52.004956+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:36.283443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.025431+0000","last_change":"2023-10-08T16:05:52.021497+0000","last_active":"2023-10-08T16:05:52.025431+0000","last_peered":"2023-10-08T16:05:52.025431+0000","last_clean":"2023-10-08T16:05:52.025431+0000","last_became_active":"2023-10-08T16:05:52.017257+0000","last_became_peered":"2023-10-08T16:05:52.017257+0000","last_unstale":"2023-10-08T16:05:52.025431+0000","last_undegraded":"2023-10-08T16:05:52.025431+0000","last_fullsized":"2023-10-08T16:05:52.025431+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:50.993409+0000","last_clean_scrub_stamp":"2023-10-08T16:05:50.993409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:59.133194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.007831+0000","last_change":"2023-10-08T16:05:51.023301+0000","last_active":"2023-10-08T16:05:52.007831+0000","last_peered":"2023-10-08T16:05:52.007831+0000","last_clean":"2023-10-08T16:05:52.007831+0000","last_became_active":"2023-10-08T16:05:51.023014+0000","last_became_peered":"2023-10-08T16:05:51.023014+0000","last_unstale":"2023-10-08T16:05:52.007831+0000","last_undegraded":"2023-10-08T16:05:52.007831+0000","last_fullsized":"2023-10-08T16:05:52.007831+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:05:49.988483+0000","last_clean_scrub_stamp":"2023-10-08T16:05:49.988483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:02:23.723819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-10-08T16:05:52.009483+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:05:52.009483+0000","last_peered":"2023-10-08T16:05:52.009483+0000","last_clean":"2023-10-08T16:05:52.009483+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:05:52.009483+0000","last_undegraded":"2023-10-08T16:05:52.009483+0000","last_fullsized":"2023-10-08T16:05:52.009483+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298472,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073368,"statfs":{"total":96636764160,"available":96331128832,"internally_reserved":0,"allocated":1212416,"data_stored":847228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":8.0939999999999994,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":8.2040000000000006,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.42899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.3409999999999993,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.68600000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297872,"kb_used_data":584,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073968,"statfs":{"total":96636764160,"available":96331743232,"internally_reserved":0,"allocated":598016,"data_stored":250412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.501},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55800000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.52900000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.436},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.625},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297888,"kb_used_data":600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073952,"statfs":{"total":96636764160,"available":96331726848,"internally_reserved":0,"allocated":614400,"data_stored":256790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.252},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.34300000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.28499999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.44400000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.46200000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.48599999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.46999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302564,"kb_used_data":1180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069276,"statfs":{"total":96636764160,"available":96326938624,"internally_reserved":0,"allocated":1208320,"data_stored":842870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.51000000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.39100000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.53400000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.626},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.52600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":256756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.372}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.41799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.503}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44500000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.62},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297892,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073948,"statfs":{"total":96636764160,"available":96331722752,"internally_reserved":0,"allocated":618496,"data_stored":256756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.30499999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.373}]},{"osd":4,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65600000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297896,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073944,"statfs":{"total":96636764160,"available":96331718656,"internally_reserved":0,"allocated":622592,"data_stored":258236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.497},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.59099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.51200000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297880,"kb_used_data":592,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073960,"statfs":{"total":96636764160,"available":96331735040,"internally_reserved":0,"allocated":606208,"data_stored":251036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.502}]},{"osd":1,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":12.227,"5min":12.227,"15min":12.227},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.45500000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.47499999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.67000000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:05:56.429 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:05:56.429 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:05:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 Processor -- start 2023-10-08T16:05:56.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 -- start start 2023-10-08T16:05:56.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 0x7f2f180af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f180af860 con 0x7f2f18069c70 2023-10-08T16:05:56.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f180cbff0 con 0x7f2f18159950 2023-10-08T16:05:56.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.584+0000 7f2f1e3b5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f180ce380 con 0x7f2f180cbb90 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f177fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2f18069c70 0x7f2f180cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60502/0 (socket says 172.21.15.170:60502) 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f177fe700 1 -- 172.21.15.170:0/1350656512 learned_addr learned my addr 172.21.15.170:0/1350656512 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f1c952700 1 --2- 172.21.15.170:0/1350656512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 0x7f2f180af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901973488 0 0) 0x7f2f180af860 con 0x7f2f18069c70 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f00003570 con 0x7f2f18069c70 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126391972 0 0) 0x7f2f08001110 con 0x7f2f18159950 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f180af860 con 0x7f2f18159950 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f1c952700 1 -- 172.21.15.170:0/1350656512 >> v1:172.21.15.191:6789/0 conn(0x7f2f18159950 legacy=0x7f2f18069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f1c952700 1 -- 172.21.15.170:0/1350656512 >> v1:172.21.15.170:6789/0 conn(0x7f2f18069c70 legacy=0x7f2f180cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f1c952700 1 -- 172.21.15.170:0/1350656512 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180cec20 con 0x7f2f180cbb90 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f1c952700 1 --2- 172.21.15.170:0/1350656512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 0x7f2f180af320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009e90 tx=0x7f2f10009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451bd25936afa636 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000e070 con 0x7f2f180cbb90 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1000bb80 con 0x7f2f180cbb90 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.585+0000 7f2f16ffd700 1 -- 172.21.15.170:0/1350656512 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f10012430 con 0x7f2f180cbb90 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/1350656512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 msgr2=0x7f2f180af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/1350656512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 0x7f2f180af320 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009e90 tx=0x7f2f10009780 comp rx=0 tx=0).stop 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/1350656512 shutdown_connections 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/1350656512 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f180cbb90 0x7f2f180af320 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/1350656512 >> 172.21.15.170:0/1350656512 conn(0x7f2f18068e00 msgr2=0x7f2f180d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/1350656512 shutdown_connections 2023-10-08T16:05:56.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.586+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/1350656512 wait complete. 2023-10-08T16:05:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.587+0000 7f2f1e3b5700 1 Processor -- start 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.587+0000 7f2f1e3b5700 1 -- start start 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f18069c70 0x7f2f180c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f18159950 0x7f2f1815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f180ca6f0 con 0x7f2f180cbb90 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f180b3550 con 0x7f2f18069c70 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1e3b5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1815ad30 con 0x7f2f18159950 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36932/0 (socket says 172.21.15.170:36932) 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 -- 172.21.15.170:0/839554080 learned_addr learned my addr 172.21.15.170:0/839554080 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:56.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f1c952700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f18159950 0x7f2f1815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 -- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f18159950 msgr2=0x7f2f1815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f17fff700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f18069c70 0x7f2f180c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f18159950 0x7f2f1815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 -- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f18069c70 msgr2=0x7f2f180c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f18069c70 0x7f2f180c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 -- 172.21.15.170:0/839554080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f10009a20 con 0x7f2f180cbb90 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.588+0000 7f2f177fe700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c790 tx=0x7f2f0c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.589+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c005b30 con 0x7f2f180cbb90 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.589+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c005c90 con 0x7f2f180cbb90 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.589+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f180c8ec0 con 0x7f2f180cbb90 2023-10-08T16:05:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.589+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0072e0 con 0x7f2f180cbb90 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.589+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f180c9480 con 0x7f2f180cbb90 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.590+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f180b36c0 con 0x7f2f180cbb90 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.590+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f0c01b3b0 con 0x7f2f180cbb90 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.590+0000 7f2efffff700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 0x7f2f040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.590+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2f0c083b10 con 0x7f2f180cbb90 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.590+0000 7f2f17fff700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 0x7f2f040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.591+0000 7f2f17fff700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 0x7f2f040693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002740 tx=0x7f2f08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.595+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f0c050d90 con 0x7f2f180cbb90 2023-10-08T16:05:56.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.851+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2f180718b0 con 0x7f2f180cbb90 2023-10-08T16:05:56.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.854+0000 7f2efffff700 1 -- 172.21.15.170:0/839554080 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2f0c017020 con 0x7f2f180cbb90 2023-10-08T16:05:56.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 msgr2=0x7f2f040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 0x7f2f040693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f08002740 tx=0x7f2f08009040 comp rx=0 tx=0).stop 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 msgr2=0x7f2f180ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c790 tx=0x7f2f0c007ea0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 shutdown_connections 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f18159950 0x7f2f1815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f18069c70 0x7f2f180c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f04066f30 0x7f2f040693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 --2- 172.21.15.170:0/839554080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f180cbb90 0x7f2f180ca1b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 >> 172.21.15.170:0/839554080 conn(0x7f2f18068e00 msgr2=0x7f2f180cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 shutdown_connections 2023-10-08T16:05:56.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.857+0000 7f2f1e3b5700 1 -- 172.21.15.170:0/839554080 wait complete. 2023-10-08T16:05:56.859 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:05:56.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:05:56.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.982+0000 7fea35a6e700 1 Processor -- start 2023-10-08T16:05:56.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.982+0000 7fea35a6e700 1 -- start start 2023-10-08T16:05:56.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea35a6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fea30159c60 0x7fea3015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea35a6e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3015a160 con 0x7fea3015db10 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea35a6e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea300d13a0 con 0x7fea30159890 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea35a6e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3015df50 con 0x7fea30159c60 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea2ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fea3015db10 0x7fea30069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60514/0 (socket says 172.21.15.170:60514) 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea2ffff700 1 -- 172.21.15.170:0/2454547131 learned_addr learned my addr 172.21.15.170:0/2454547131 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.983+0000 7fea2effd700 1 --2- 172.21.15.170:0/2454547131 >> v2:172.21.15.191:3300/0 conn(0x7fea30159c60 0x7fea3015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810118696 0 0) 0x7fea3015a160 con 0x7fea3015db10 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea18003570 con 0x7fea3015db10 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249957575 0 0) 0x7fea18003570 con 0x7fea3015db10 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2effd700 1 -- 172.21.15.170:0/2454547131 >> v1:172.21.15.191:6789/0 conn(0x7fea30159890 legacy=0x7fea3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2effd700 1 -- 172.21.15.170:0/2454547131 >> v1:172.21.15.170:6789/0 conn(0x7fea3015db10 legacy=0x7fea30069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2effd700 1 -- 172.21.15.170:0/2454547131 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3015e140 con 0x7fea30159c60 2023-10-08T16:05:56.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2effd700 1 --2- 172.21.15.170:0/2454547131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea3015d5d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea240098e0 tx=0x7fea240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3219f898c5b8ddeb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28001470 con 0x7fea30159c60 2023-10-08T16:05:56.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea24011420 con 0x7fea30159c60 2023-10-08T16:05:56.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.984+0000 7fea2e7fc700 1 -- 172.21.15.170:0/2454547131 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea240117c0 con 0x7fea30159c60 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 -- 172.21.15.170:0/2454547131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 msgr2=0x7fea3015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 --2- 172.21.15.170:0/2454547131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea3015d5d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fea240098e0 tx=0x7fea240091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 -- 172.21.15.170:0/2454547131 shutdown_connections 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 --2- 172.21.15.170:0/2454547131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea3015d5d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 -- 172.21.15.170:0/2454547131 >> 172.21.15.170:0/2454547131 conn(0x7fea30068e20 msgr2=0x7fea300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 -- 172.21.15.170:0/2454547131 shutdown_connections 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.985+0000 7fea35a6e700 1 -- 172.21.15.170:0/2454547131 wait complete. 2023-10-08T16:05:56.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.986+0000 7fea35a6e700 1 Processor -- start 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.986+0000 7fea35a6e700 1 -- start start 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea301528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3015db10 0x7fea30154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30153cc0 con 0x7fea30159890 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30152e10 con 0x7fea30159c60 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea301530b0 con 0x7fea3015db10 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36940/0 (socket says 172.21.15.170:36940) 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 -- 172.21.15.170:0/3114905465 learned_addr learned my addr 172.21.15.170:0/3114905465 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:56.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2effd700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea301528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2ffff700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3015db10 0x7fea30154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 -- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3015db10 msgr2=0x7fea30154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3015db10 0x7fea30154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 -- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 msgr2=0x7fea301528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea301528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 -- 172.21.15.170:0/3114905465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea24009420 con 0x7fea30159890 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea2f7fe700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea2000cb10 tx=0x7fea200017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000a760 con 0x7fea30159890 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2000a8e0 con 0x7fea30159890 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea30153390 con 0x7fea30159890 2023-10-08T16:05:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.987+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000aca0 con 0x7fea30159890 2023-10-08T16:05:56.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.988+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea30153950 con 0x7fea30159890 2023-10-08T16:05:56.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.988+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3004c3c0 con 0x7fea30159890 2023-10-08T16:05:56.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.989+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea20003030 con 0x7fea30159890 2023-10-08T16:05:56.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.989+0000 7fea177fe700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 0x7fea1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:56.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.989+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fea20084630 con 0x7fea30159890 2023-10-08T16:05:56.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.989+0000 7fea2effd700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 0x7fea1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:56.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.990+0000 7fea2effd700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 0x7fea1c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fd0 tx=0x7fea24016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:56.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:56.993+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea200518a0 con 0x7fea30159890 2023-10-08T16:05:57.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.217+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea30153fa0 con 0x7fea30159890 2023-10-08T16:05:57.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.218+0000 7fea177fe700 1 -- 172.21.15.170:0/3114905465 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fea20054ec0 con 0x7fea30159890 2023-10-08T16:05:57.218 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:05:57.218 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:05:57.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 msgr2=0x7fea1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 0x7fea1c0693e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fea24009fd0 tx=0x7fea24016000 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 msgr2=0x7fea300cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea2000cb10 tx=0x7fea200017b0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 shutdown_connections 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fea3015db10 0x7fea30154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fea1c066f30 0x7fea1c0693e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea30159890 0x7fea300cf0b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 --2- 172.21.15.170:0/3114905465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fea30159c60 0x7fea301528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 >> 172.21.15.170:0/3114905465 conn(0x7fea30068e20 msgr2=0x7fea30155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 shutdown_connections 2023-10-08T16:05:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.220+0000 7fea35a6e700 1 -- 172.21.15.170:0/3114905465 wait complete. 2023-10-08T16:05:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 30 2023-10-08T16:05:57.301 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:05:57.301+0000 7f7e2f43f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:05:57.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:05:51.383446+0000', 'modified': '2023-10-08T16:05:55.018929+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5304}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5304': {'gid': 5304, 'name': 'd', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6834/2304240445', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 2304240445}, {'type': 'v1', 'addr': '172.21.15.191:6834', 'nonce': 2304240445}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:05:57.635 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:05:57.635 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T16:05:57.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.750+0000 7f87b06c4700 1 Processor -- start 2023-10-08T16:05:57.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.750+0000 7f87b06c4700 1 -- start start 2023-10-08T16:05:57.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87a806bcd0 0x7f87a80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a806c0a0 0x7f87a815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a815df50 0x7f87a815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a815d7e0 con 0x7f87a815df50 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a815dac0 con 0x7f87a806bcd0 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87b06c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a815aa30 con 0x7f87a806c0a0 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87aec61700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a815df50 0x7f87a815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87aec61700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87a815df50 0x7f87a815e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36944/0 (socket says 172.21.15.170:36944) 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.751+0000 7f87aec61700 1 -- 172.21.15.170:0/3305850999 learned_addr learned my addr 172.21.15.170:0/3305850999 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87adc5f700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a806c0a0 0x7f87a815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 -- 172.21.15.170:0/3305850999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a806c0a0 msgr2=0x7f87a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87ae460700 1 --2- 172.21.15.170:0/3305850999 >> v2:172.21.15.191:3300/0 conn(0x7f87a806bcd0 0x7f87a80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a806c0a0 0x7f87a815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 -- 172.21.15.170:0/3305850999 >> v2:172.21.15.191:3300/0 conn(0x7f87a806bcd0 msgr2=0x7f87a80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 --2- 172.21.15.170:0/3305850999 >> v2:172.21.15.191:3300/0 conn(0x7f87a806bcd0 0x7f87a80caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 -- 172.21.15.170:0/3305850999 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a815ad10 con 0x7f87a815df50 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87aec61700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a815df50 0x7f87a815e3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87a80691a0 tx=0x7f87a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34def0735d7d556b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87ad45e700 1 -- 172.21.15.170:0/3305850999 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000e070 con 0x7f87a815df50 2023-10-08T16:05:57.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87ad45e700 1 -- 172.21.15.170:0/3305850999 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000ba50 con 0x7f87a815df50 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.752+0000 7f87ad45e700 1 -- 172.21.15.170:0/3305850999 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00092e0 con 0x7f87a815df50 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 -- 172.21.15.170:0/3305850999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a815df50 msgr2=0x7f87a815e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a815df50 0x7f87a815e3c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f87a80691a0 tx=0x7f87a0009840 comp rx=0 tx=0).stop 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 -- 172.21.15.170:0/3305850999 shutdown_connections 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a815df50 0x7f87a815e3c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 --2- 172.21.15.170:0/3305850999 >> v2:172.21.15.191:3300/0 conn(0x7f87a806bcd0 0x7f87a80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 --2- 172.21.15.170:0/3305850999 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a806c0a0 0x7f87a815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 -- 172.21.15.170:0/3305850999 >> 172.21.15.170:0/3305850999 conn(0x7f87a80ce600 msgr2=0x7f87a80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 -- 172.21.15.170:0/3305850999 shutdown_connections 2023-10-08T16:05:57.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.753+0000 7f87b06c4700 1 -- 172.21.15.170:0/3305850999 wait complete. 2023-10-08T16:05:57.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.754+0000 7f87b06c4700 1 Processor -- start 2023-10-08T16:05:57.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.754+0000 7f87b06c4700 1 -- start start 2023-10-08T16:05:57.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a806bcd0 0x7f87a814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a815df50 0x7f87a80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8158bb0 con 0x7f87a806c0a0 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80c83e0 con 0x7f87a806bcd0 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8158280 con 0x7f87a815df50 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36954/0 (socket says 172.21.15.170:36954) 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 -- 172.21.15.170:0/1920038588 learned_addr learned my addr 172.21.15.170:0/1920038588 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87aec61700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a815df50 0x7f87a80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87ae460700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a806bcd0 0x7f87a814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 -- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a815df50 msgr2=0x7f87a80c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a815df50 0x7f87a80c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 -- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a806bcd0 msgr2=0x7f87a814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a806bcd0 0x7f87a814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 -- 172.21.15.170:0/1920038588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0009ae0 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f879800c880 tx=0x7f8798008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f879e7fc700 1 -- 172.21.15.170:0/1920038588 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011600 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f879e7fc700 1 -- 172.21.15.170:0/1920038588 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798011780 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a8158560 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f879e7fc700 1 -- 172.21.15.170:0/1920038588 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798011b40 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.755+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80b13a0 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.757+0000 7f879e7fc700 1 -- 172.21.15.170:0/1920038588 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8798018360 con 0x7f87a806c0a0 2023-10-08T16:05:57.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.757+0000 7f879e7fc700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.757+0000 7f879e7fc700 1 -- 172.21.15.170:0/1920038588 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8798084230 con 0x7f87a806c0a0 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.757+0000 7f87ae460700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.757+0000 7f87ae460700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 0x7f878c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.759+0000 7f87b99bcb80 1 Processor -- start 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.759+0000 7f87b99bcb80 1 -- start start 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9320ea70 con 0x55ee932e0f60 2023-10-08T16:05:57.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9320c270 con 0x55ee93301d00 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee93208060 con 0x55ee932b7a50 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36956/0 (socket says 172.21.15.170:36956) 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 -- 172.21.15.170:0/3103916397 learned_addr learned my addr 172.21.15.170:0/3103916397 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87ae460700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87adc5f700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 -- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 msgr2=0x55ee932b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 -- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 msgr2=0x55ee932e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 -- 172.21.15.170:0/3103916397 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee930a2270 con 0x55ee932e0f60 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87ae460700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f87aec61700 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009380 tx=0x7f87a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133da96e9d86dad4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f878b7fe700 1 -- 172.21.15.170:0/3103916397 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000bbc0 con 0x55ee932e0f60 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f878b7fe700 1 -- 172.21.15.170:0/3103916397 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000bd20 con 0x55ee932e0f60 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.760+0000 7f878b7fe700 1 -- 172.21.15.170:0/3103916397 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0007860 con 0x55ee932e0f60 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 -- 172.21.15.170:0/3103916397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 msgr2=0x55ee932b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009380 tx=0x7f87a0018000 comp rx=0 tx=0).stop 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 -- 172.21.15.170:0/3103916397 shutdown_connections 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee932b3cf0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/3103916397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 -- 172.21.15.170:0/3103916397 >> 172.21.15.170:0/3103916397 conn(0x55ee932b9370 msgr2=0x55ee932b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 -- 172.21.15.170:0/3103916397 shutdown_connections 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.761+0000 7f87b99bcb80 1 -- 172.21.15.170:0/3103916397 wait complete. 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.762+0000 7f87b99bcb80 1 -- start start 2023-10-08T16:05:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.762+0000 7f87b99bcb80 1 Processor -- start 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee933016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee931b3890 con 0x55ee932e0f60 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee931ceea0 con 0x55ee93301d00 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee9308ae30 con 0x55ee932b7a50 2023-10-08T16:05:57.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 10 client.0 _ll_register_callbacks cb 0x55ee9327b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36968/0 (socket says 172.21.15.170:36968) 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 -- 172.21.15.170:0/1381169475 learned_addr learned my addr 172.21.15.170:0/1381169475 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87aec61700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87ae460700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee933016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 -- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 msgr2=0x55ee933016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee933016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 -- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 msgr2=0x55ee932f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 -- 172.21.15.170:0/1381169475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a815ad10 con 0x55ee932e0f60 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87aec61700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:05:57.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8798000c00 tx=0x7f8798084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc880d47be22cf74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f8788ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879805c100 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f879806ce40 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee932acb70 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f879805bc70 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 10 client.5442 fetch_fsmap learned FSMap version 30 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55ee93044fe0 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.763+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ee932acb70 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f87980822f0 con 0x55ee932e0f60 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 10 client.5442 fetch_fsmap finished waiting for FSMap version 30 2023-10-08T16:05:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 10 client.5442 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 20 client.5442 populate_metadata read hostname 'smithi170' 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee932fe670 0x55ee932fea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 4 client.5442 mds_command: new command op to 5304 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 --> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee93289aa0 con 0x55ee932fe670 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87ae460700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee932fe670 0x55ee932fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f87ae460700 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee932fe670 0x55ee932fea40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28140eddd96915f server_cookie=adde17d2e8671ee in_seq=0 out_seq=0 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.764+0000 7f8788ff9700 10 client.5442 ms_handle_connect on v2:172.21.15.191:6832/2304240445 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f8788ff9700 1 -- 172.21.15.170:0/1381169475 <== mds.0 v2:172.21.15.191:6832/2304240445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ee93289aa0 con 0x55ee932fe670 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f8788ff9700 10 client.5442 handle_command_reply: tid=0 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 client.5442 shutdown 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 msgr2=0x55ee93311e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8798000c00 tx=0x7f8798084040 comp rx=0 tx=0).stop 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 shutdown_connections 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee932fe670 0x55ee932fea40 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee933016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1381169475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932e0f60 0x55ee93311e40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f8788ff9700 0 client.5442 ms_handle_reset on v2:172.21.15.191:6832/2304240445 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 >> 172.21.15.170:0/1381169475 conn(0x55ee932b9370 msgr2=0x55ee932fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 shutdown_connections 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1381169475 wait complete. 2023-10-08T16:05:57.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.765+0000 7f87b99bcb80 20 client.5442 trim_cache size 0 max 16384 2023-10-08T16:05:57.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.782+0000 7f87b99bcb80 1 Processor -- start 2023-10-08T16:05:57.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.782+0000 7f87b99bcb80 1 -- start start 2023-10-08T16:05:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932fca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee932e0f60 0x55ee9330d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee932b2e80 con 0x55ee93301d00 2023-10-08T16:05:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee932ffd30 con 0x55ee932e0f60 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee9330dab0 con 0x55ee932b7a50 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36984/0 (socket says 172.21.15.170:36984) 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 -- 172.21.15.170:0/1400393946 learned_addr learned my addr 172.21.15.170:0/1400393946 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87adc5f700 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee932e0f60 0x55ee9330d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87ae460700 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932fca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 -- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 msgr2=0x55ee932fca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932fca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 -- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee932e0f60 msgr2=0x55ee9330d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee932e0f60 0x55ee9330d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 -- 172.21.15.170:0/1400393946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee931d6780 con 0x55ee93301d00 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f87aec61700 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a00094e0 tx=0x7f87a0020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b45698c0714dcbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f878bfff700 1 -- 172.21.15.170:0/1400393946 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0009970 con 0x55ee93301d00 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f878bfff700 1 -- 172.21.15.170:0/1400393946 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a002d040 con 0x55ee93301d00 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.783+0000 7f878bfff700 1 -- 172.21.15.170:0/1400393946 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00032f0 con 0x55ee93301d00 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1400393946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 msgr2=0x55ee932e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f87a00094e0 tx=0x7f87a0020070 comp rx=0 tx=0).stop 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1400393946 shutdown_connections 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee93301d00 0x55ee932e7800 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932b7a50 0x55ee932fca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/1400393946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee932e0f60 0x55ee9330d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1400393946 >> 172.21.15.170:0/1400393946 conn(0x55ee932b9370 msgr2=0x55ee932b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1400393946 shutdown_connections 2023-10-08T16:05:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- 172.21.15.170:0/1400393946 wait complete. 2023-10-08T16:05:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.784+0000 7f87b99bcb80 1 -- start start 2023-10-08T16:05:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 Processor -- start 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932e0f60 0x55ee932df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee932e7800 con 0x55ee932b7a50 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee933648d0 con 0x55ee93301d00 2023-10-08T16:05:57.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ee93364cf0 con 0x55ee932e0f60 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87b99bcb80 10 client.0 _ll_register_callbacks cb 0x55ee932ba5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.785+0000 7f87ae460700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36992/0 (socket says 172.21.15.170:36992) 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 -- 172.21.15.170:0/2124456946 learned_addr learned my addr 172.21.15.170:0/2124456946 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87aec61700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87adc5f700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932e0f60 0x55ee932df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 -- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932e0f60 msgr2=0x55ee932df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932e0f60 0x55ee932df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 -- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 msgr2=0x55ee932dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 -- 172.21.15.170:0/2124456946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee930a2270 con 0x55ee932b7a50 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87ae460700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f87a400f4c0 tx=0x7f87a4024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5465ce787e16b580 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4027040 con 0x55ee932b7a50 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87a4014b00 con 0x55ee932b7a50 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87a000bd20 con 0x55ee932b7a50 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87a402b460 con 0x55ee932b7a50 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87b99bcb80 10 client.5448 fetch_fsmap learned FSMap version 30 2023-10-08T16:05:57.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55ee931bd470 con 0x55ee932b7a50 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000bd20 con 0x55ee932b7a50 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f87a402b5c0 con 0x55ee932b7a50 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.786+0000 7f87b99bcb80 10 client.5448 fetch_fsmap finished waiting for FSMap version 30 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 10 client.5448 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 20 client.5448 populate_metadata read hostname 'smithi170' 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee93363d60 0x55ee93364130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 4 client.5448 mds_command: new command op to 5304 tid=0[{"prefix": "status"}] 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 --> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ee931aae00 con 0x55ee93363d60 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87adc5f700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee93363d60 0x55ee93364130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f8788ff9700 10 client.5448 ms_handle_connect on v2:172.21.15.191:6832/2304240445 2023-10-08T16:05:57.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87adc5f700 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee93363d60 0x55ee93364130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9a053f7337721c server_cookie=5280f661d75cb98e in_seq=0 out_seq=0 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f8788ff9700 1 -- 172.21.15.170:0/2124456946 <== mds.0 v2:172.21.15.191:6832/2304240445 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ee931aae00 con 0x55ee93363d60 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f8788ff9700 10 client.5448 handle_command_reply: tid=0 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 client.5448 shutdown 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 msgr2=0x55ee93360ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f87a400f4c0 tx=0x7f87a4024040 comp rx=0 tx=0).stop 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 shutdown_connections 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:6832/2304240445,v1:172.21.15.191:6834/2304240445] conn(0x55ee93363d60 0x55ee93364130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55ee93301d00 0x55ee932dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f8788ff9700 0 client.5448 ms_handle_reset on v2:172.21.15.191:6832/2304240445 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee932b7a50 0x55ee93360ce0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 --2- 172.21.15.170:0/2124456946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55ee932e0f60 0x55ee932df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.787+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 >> 172.21.15.170:0/2124456946 conn(0x55ee932b9370 msgr2=0x55ee932b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 shutdown_connections 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b99bcb80 1 -- 172.21.15.170:0/2124456946 wait complete. 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b99bcb80 20 client.5448 trim_cache size 0 max 16384 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 5304, 2023-10-08T16:05:57.797 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.7883084360000003, 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 30, 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 51, 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 51, 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.893959423 2023-10-08T16:05:57.798 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 msgr2=0x7f878c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 0x7f878c0693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f87a4006fd0 tx=0x7f87a4008040 comp rx=0 tx=0).stop 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 msgr2=0x7f87a80c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f879800c880 tx=0x7f8798008040 comp rx=0 tx=0).stop 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 shutdown_connections 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87a815df50 0x7f87a80c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87a806bcd0 0x7f87a814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f878c066f30 0x7f878c0693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 --2- 172.21.15.170:0/1920038588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87a806c0a0 0x7f87a80c7860 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 >> 172.21.15.170:0/1920038588 conn(0x7f87a80ce600 msgr2=0x7f87a806a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 shutdown_connections 2023-10-08T16:05:57.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.788+0000 7f87b06c4700 1 -- 172.21.15.170:0/1920038588 wait complete. 2023-10-08T16:05:57.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:05:57.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.915+0000 7f9667f5b700 1 Processor -- start 2023-10-08T16:05:57.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.916+0000 7f9667f5b700 1 -- start start 2023-10-08T16:05:57.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.916+0000 7f9667f5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f966006c0a0 0x7f96600caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.916+0000 7f9667f5b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966015d7e0 con 0x7f966006bcd0 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.916+0000 7f9667f5b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f966015dac0 con 0x7f966015df50 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.916+0000 7f9667f5b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f966015aa30 con 0x7f966006c0a0 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9665cf7700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f966006bcd0 0x7f966015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60524/0 (socket says 172.21.15.170:60524) 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9665cf7700 1 -- 172.21.15.170:0/1869398079 learned_addr learned my addr 172.21.15.170:0/1869398079 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f96654f6700 1 --2- 172.21.15.170:0/1869398079 >> v2:172.21.15.170:3300/0 conn(0x7f966006c0a0 0x7f96600caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f96654f6700 1 -- 172.21.15.170:0/1869398079 >> v1:172.21.15.191:6789/0 conn(0x7f966015df50 legacy=0x7f96600c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f96654f6700 1 -- 172.21.15.170:0/1869398079 >> v1:172.21.15.170:6789/0 conn(0x7f966006bcd0 legacy=0x7f966015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f96654f6700 1 -- 172.21.15.170:0/1869398079 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966015ad10 con 0x7f966006c0a0 2023-10-08T16:05:57.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9664cf5700 1 -- 172.21.15.170:0/1869398079 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912744210 0 0) 0x7f966015d7e0 con 0x7f966006bcd0 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f96654f6700 1 --2- 172.21.15.170:0/1869398079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966006c0a0 0x7f96600caa90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96500098e0 tx=0x7f96500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9212334c09851958 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9664cf5700 1 -- 172.21.15.170:0/1869398079 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9650009e80 con 0x7f966006c0a0 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9664cf5700 1 -- 172.21.15.170:0/1869398079 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f965000b860 con 0x7f966006c0a0 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.917+0000 7f9664cf5700 1 -- 172.21.15.170:0/1869398079 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965000bbe0 con 0x7f966006c0a0 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 -- 172.21.15.170:0/1869398079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966006c0a0 msgr2=0x7f96600caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 --2- 172.21.15.170:0/1869398079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966006c0a0 0x7f96600caa90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96500098e0 tx=0x7f96500091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 -- 172.21.15.170:0/1869398079 shutdown_connections 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 --2- 172.21.15.170:0/1869398079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966006c0a0 0x7f96600caa90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 -- 172.21.15.170:0/1869398079 >> 172.21.15.170:0/1869398079 conn(0x7f96600ce600 msgr2=0x7f96600cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:57.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 -- 172.21.15.170:0/1869398079 shutdown_connections 2023-10-08T16:05:57.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.918+0000 7f9667f5b700 1 -- 172.21.15.170:0/1869398079 wait complete. 2023-10-08T16:05:57.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 Processor -- start 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 -- start start 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966006bcd0 0x7f9660153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966006c0a0 0x7f9660157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b3d70 con 0x7f966015df50 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601584e0 con 0x7f966006c0a0 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.919+0000 7f9667f5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96600b2650 con 0x7f966006bcd0 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37016/0 (socket says 172.21.15.170:37016) 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 -- 172.21.15.170:0/3230532515 learned_addr learned my addr 172.21.15.170:0/3230532515 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9665cf7700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966006bcd0 0x7f9660153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96654f6700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966006c0a0 0x7f9660157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 -- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966006bcd0 msgr2=0x7f9660153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966006bcd0 0x7f9660153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 -- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966006c0a0 msgr2=0x7f9660157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966006c0a0 0x7f9660157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 -- 172.21.15.170:0/3230532515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9650009420 con 0x7f966015df50 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f96664f8700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f965800c4c0 tx=0x7f9658007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965800ce70 con 0x7f966015df50 2023-10-08T16:05:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9658011590 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9660154020 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9658011970 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.920+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96600b3990 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.921+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96600b3b50 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.922+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9658011ce0 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.922+0000 7f9655ffb700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.922+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9658083810 con 0x7f966015df50 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.922+0000 7f9665cf7700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.922+0000 7f9665cf7700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 0x7f96440693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f965c000f80 tx=0x7f965c001950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:57.926+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9658050a80 con 0x7f966015df50 2023-10-08T16:05:58.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.155+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f96600b3eb0 con 0x7f966015df50 2023-10-08T16:05:58.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.155+0000 7f9655ffb700 1 -- 172.21.15.170:0/3230532515 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f96600b3eb0 con 0x7f966015df50 2023-10-08T16:05:58.156 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:05:58.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 msgr2=0x7f96440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 0x7f96440693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f965c000f80 tx=0x7f965c001950 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 msgr2=0x7f96600b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f965800c4c0 tx=0x7f9658007f50 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 shutdown_connections 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9644066f30 0x7f96440693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f966015df50 0x7f96600b2110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f966006bcd0 0x7f9660153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 --2- 172.21.15.170:0/3230532515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f966006c0a0 0x7f9660157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 >> 172.21.15.170:0/3230532515 conn(0x7f96600ce600 msgr2=0x7f966015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 shutdown_connections 2023-10-08T16:05:58.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.158+0000 7f9667f5b700 1 -- 172.21.15.170:0/3230532515 wait complete. 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:05:58.170 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:58.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.171 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:05:58.186 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:05:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:05:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:05:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.186 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:05:58.187 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42393sec preferred_lft 42393sec 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:05:58.188 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.189 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:05:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:05:58.189 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:58.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.189 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:05:58.249 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:05:58.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.251 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:05:58.307 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:05:58.307 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:05:58.307 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:05:58.307 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:05:58.308 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:05:58.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.308 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:05:58.308 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:05:58.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:05:58.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:05:58.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:05:58.309 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:05:58.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.407 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:05:58.422 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:05:58.422 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:05:58.422 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:58.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.422 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:05:58.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:58.487 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:58.542 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:05:58.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:58.542 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:58.596 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:05:58.596 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:05:58.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:58.675 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:58.676 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:58.694 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:58.695 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:58.716 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:05:58.717 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:05:58.718 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:05:58.718 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:05:58.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:58.719 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:05:58.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:05:58.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.938+0000 7fb6a3996700 1 Processor -- start 2023-10-08T16:05:58.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.939+0000 7fb6a3996700 1 -- start start 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.939+0000 7fb6a3996700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb69c06bcd0 0x7fb69c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.939+0000 7fb6a3996700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 0x7fb69c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a3996700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c15d7e0 con 0x7fb69c15df50 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a3996700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c15dac0 con 0x7fb69c06bcd0 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a3996700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c15aa30 con 0x7fb69c06c0a0 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb69c06bcd0 0x7fb69c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a0f31700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 0x7fb69c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a0f31700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 0x7fb69c15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:38696/0 (socket says 172.21.15.170:38696) 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb69c06bcd0 0x7fb69c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:56922/0 (socket says 172.21.15.170:56922) 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a0f31700 1 -- 172.21.15.170:0/319141397 learned_addr learned my addr 172.21.15.170:0/319141397 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 -- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 msgr2=0x7fb69c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 --2- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 0x7fb69c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 -- 172.21.15.170:0/319141397 >> v1:172.21.15.191:6789/0 conn(0x7fb69c15df50 legacy=0x7fb69c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb693fff700 1 -- 172.21.15.170:0/319141397 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886579406 0 0) 0x7fb69c15d7e0 con 0x7fb69c15df50 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 -- 172.21.15.170:0/319141397 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c15ad10 con 0x7fb69c06bcd0 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.940+0000 7fb6a1732700 1 --2- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c06bcd0 0x7fb69c0caa90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6980098e0 tx=0x7fb6980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac4e89057ed42a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb693fff700 1 -- 172.21.15.170:0/319141397 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69800b640 con 0x7fb69c06bcd0 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb693fff700 1 -- 172.21.15.170:0/319141397 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69800b7a0 con 0x7fb69c06bcd0 2023-10-08T16:05:58.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb693fff700 1 -- 172.21.15.170:0/319141397 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69800bb40 con 0x7fb69c06bcd0 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 -- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c06bcd0 msgr2=0x7fb69c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 --2- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c06bcd0 0x7fb69c0caa90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6980098e0 tx=0x7fb6980091a0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 -- 172.21.15.170:0/319141397 shutdown_connections 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 --2- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c06bcd0 0x7fb69c0caa90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 --2- 172.21.15.170:0/319141397 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06c0a0 0x7fb69c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 -- 172.21.15.170:0/319141397 >> 172.21.15.170:0/319141397 conn(0x7fb69c0ce600 msgr2=0x7fb69c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 -- 172.21.15.170:0/319141397 shutdown_connections 2023-10-08T16:05:58.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.941+0000 7fb6a3996700 1 -- 172.21.15.170:0/319141397 wait complete. 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.942+0000 7fb6a3996700 1 Processor -- start 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.942+0000 7fb6a3996700 1 -- start start 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06bcd0 0x7fb69c16daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c15df50 0x7fb69c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0b1560 con 0x7fb69c06c0a0 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c16e9d0 con 0x7fb69c15df50 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a3996700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c16e130 con 0x7fb69c06bcd0 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a0f31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a0f31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37022/0 (socket says 172.21.15.170:37022) 2023-10-08T16:05:58.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a0f31700 1 -- 172.21.15.170:0/3376544551 learned_addr learned my addr 172.21.15.170:0/3376544551 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a1732700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06bcd0 0x7fb69c16daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.943+0000 7fb6a1f33700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c15df50 0x7fb69c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 -- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06bcd0 msgr2=0x7fb69c16daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06bcd0 0x7fb69c16daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 -- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c15df50 msgr2=0x7fb69c14c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c15df50 0x7fb69c14c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 -- 172.21.15.170:0/3376544551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698009420 con 0x7fb69c06c0a0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a0f31700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00ea50 tx=0x7fb68c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00c640 con 0x7fb69c06c0a0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c00c7a0 con 0x7fb69c06c0a0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c16e410 con 0x7fb69c06c0a0 2023-10-08T16:05:58.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00cb20 con 0x7fb69c06c0a0 2023-10-08T16:05:58.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.944+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c14c720 con 0x7fb69c06c0a0 2023-10-08T16:05:58.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.945+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0b16d0 con 0x7fb69c06c0a0 2023-10-08T16:05:58.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.945+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb68c013470 con 0x7fb69c06c0a0 2023-10-08T16:05:58.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.946+0000 7fb6917fa700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 0x7fb6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:58.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.946+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb68c084740 con 0x7fb69c06c0a0 2023-10-08T16:05:58.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.946+0000 7fb6a1732700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 0x7fb6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:58.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.946+0000 7fb6a1732700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 0x7fb6800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6980093f0 tx=0x7fb698017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:58.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:58.950+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb69c0b16d0 con 0x7fb69c06c0a0 2023-10-08T16:05:59.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.179+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb69c0b16d0 con 0x7fb69c06c0a0 2023-10-08T16:05:59.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.180+0000 7fb6917fa700 1 -- 172.21.15.170:0/3376544551 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb69c0b16d0 con 0x7fb69c06c0a0 2023-10-08T16:05:59.180 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:05:59.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 msgr2=0x7fb6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 0x7fb6800693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb6980093f0 tx=0x7fb698017000 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 msgr2=0x7fb69c154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb68c00ea50 tx=0x7fb68c009250 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 shutdown_connections 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb69c15df50 0x7fb69c14c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb69c06bcd0 0x7fb69c16daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb680066f30 0x7fb6800693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 --2- 172.21.15.170:0/3376544551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb69c06c0a0 0x7fb69c154240 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 >> 172.21.15.170:0/3376544551 conn(0x7fb69c0ce600 msgr2=0x7fb69c156260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 shutdown_connections 2023-10-08T16:05:59.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.182+0000 7fb6a3996700 1 -- 172.21.15.170:0/3376544551 wait complete. 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:05:59.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:05:59.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:59.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.195 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:05:59.213 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:59.214 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42392sec preferred_lft 42392sec 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:05:59.215 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:05:59.216 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:05:59.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.217 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:05:59.281 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:05:59.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.283 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:05:59.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:05:59.342 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:05:59.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.446 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:05:59.461 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:05:59.461 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:05:59.461 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:59.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.461 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:05:59.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:59.526 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:59.581 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:05:59.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:05:59.581 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:59.635 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:05:59.635 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:05:59.635 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:59.716 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:59.716 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:59.734 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:59.735 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:59.756 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:05:59.756 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:05:59.757 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:05:59.759 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:05:59.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:05:59.759 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:05:59.823 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:05:59.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.945+0000 7fd36318a700 1 Processor -- start 2023-10-08T16:05:59.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.945+0000 7fd36318a700 1 -- start start 2023-10-08T16:05:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd36318a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd35c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd36318a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c0b5b00 con 0x7fd35c0cf350 2023-10-08T16:05:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd36318a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35c0b5da0 con 0x7fd35c0b53d0 2023-10-08T16:05:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd36318a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0b6080 con 0x7fd35c159ec0 2023-10-08T16:05:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd361727700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd35c0cf350 0x7fd35c0cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60540/0 (socket says 172.21.15.170:60540) 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd361727700 1 -- 172.21.15.170:0/583922060 learned_addr learned my addr 172.21.15.170:0/583922060 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd360f26700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd35c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535712941 0 0) 0x7fd35c0b5b00 con 0x7fd35c0cf350 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd34c003570 con 0x7fd35c0cf350 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192253926 0 0) 0x7fd34c003570 con 0x7fd35c0cf350 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 msgr2=0x7fd35c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd35c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 >> v1:172.21.15.191:6789/0 conn(0x7fd35c0b53d0 legacy=0x7fd35c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c0cc100 con 0x7fd35c0cf350 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fd3540017f0 con 0x7fd35c0cf350 2023-10-08T16:05:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.946+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 >> v1:172.21.15.170:6789/0 conn(0x7fd35c0cf350 legacy=0x7fd35c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd34c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd34c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd34c004d50 0x7fd34c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c0b6080 con 0x7fd34c004010 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c00bbf0 con 0x7fd34c004d50 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c00bdb0 con 0x7fd35c159ec0 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd34c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd360f26700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd34c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd361727700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd34c004d50 0x7fd34c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 -- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 msgr2=0x7fd34c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c159ec0 0x7fd34c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 -- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd34c004d50 msgr2=0x7fd34c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd34c004d50 0x7fd34c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 -- 172.21.15.170:0/583922060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c003570 con 0x7fd34c004010 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35bfff700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd34c004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd348001880 tx=0x7fd348007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafa4b2400d0e792 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348012420 con 0x7fd34c004010 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348012580 con 0x7fd34c004010 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.947+0000 7fd35b7fe700 1 -- 172.21.15.170:0/583922060 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348012920 con 0x7fd34c004010 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 -- 172.21.15.170:0/583922060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 msgr2=0x7fd34c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 --2- 172.21.15.170:0/583922060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd34c004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd348001880 tx=0x7fd348007d30 comp rx=0 tx=0).stop 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd361727700 1 -- 172.21.15.170:0/583922060 reap_dead start 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 -- 172.21.15.170:0/583922060 shutdown_connections 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 -- 172.21.15.170:0/583922060 >> 172.21.15.170:0/583922060 conn(0x7fd35c0999f0 msgr2=0x7fd35c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:05:59.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 -- 172.21.15.170:0/583922060 shutdown_connections 2023-10-08T16:05:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.948+0000 7fd36318a700 1 -- 172.21.15.170:0/583922060 wait complete. 2023-10-08T16:05:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.949+0000 7fd36318a700 1 Processor -- start 2023-10-08T16:05:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.949+0000 7fd36318a700 1 -- start start 2023-10-08T16:05:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd35c068390 0x7fd35c068800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c15c590 0x7fd35c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c068d40 con 0x7fd34c004010 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c15cf30 con 0x7fd35c068390 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c068f70 con 0x7fd35c15c590 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37038/0 (socket says 172.21.15.170:37038) 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 -- 172.21.15.170:0/1657592978 learned_addr learned my addr 172.21.15.170:0/1657592978 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd35bfff700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd35c068390 0x7fd35c068800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 -- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c15c590 msgr2=0x7fd35c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c15c590 0x7fd35c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 -- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd35c068390 msgr2=0x7fd35c068800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd35c068390 0x7fd35c068800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:05:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 -- 172.21.15.170:0/1657592978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c069250 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd360f26700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd350011b60 tx=0x7fd350009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350005c10 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350005d70 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35c069530 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35000fa60 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.950+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35c15a330 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.952+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35c063c90 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.952+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd350006050 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.952+0000 7fd358ff9700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.952+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd350084e20 con 0x7fd34c004010 2023-10-08T16:05:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.952+0000 7fd35bfff700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:05:59.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.953+0000 7fd35bfff700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd348008fd0 tx=0x7fd348015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:05:59.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:05:59.957+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd350052090 con 0x7fd34c004010 2023-10-08T16:06:00.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:00.186+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd35c0718b0 con 0x7fd34c004010 2023-10-08T16:06:01.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.039+0000 7fd358ff9700 1 -- 172.21.15.170:0/1657592978 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd3500556b0 con 0x7fd34c004010 2023-10-08T16:06:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 msgr2=0x7fd3400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 0x7fd3400693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd348008fd0 tx=0x7fd348015000 comp rx=0 tx=0).stop 2023-10-08T16:06:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 msgr2=0x7fd35c14c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd350011b60 tx=0x7fd350009120 comp rx=0 tx=0).stop 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 shutdown_connections 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd35c15c590 0x7fd35c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340066f30 0x7fd3400693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34c004010 0x7fd35c14c1a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 --2- 172.21.15.170:0/1657592978 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd35c068390 0x7fd35c068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 >> 172.21.15.170:0/1657592978 conn(0x7fd35c0999f0 msgr2=0x7fd35c15ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 shutdown_connections 2023-10-08T16:06:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.042+0000 7fd36318a700 1 -- 172.21.15.170:0/1657592978 wait complete. 2023-10-08T16:06:01.055 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:06:01.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.172+0000 7faa23a77700 1 Processor -- start 2023-10-08T16:06:01.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.172+0000 7faa23a77700 1 -- start start 2023-10-08T16:06:01.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa23a77700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faa1c069d60 0x7faa1c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa23a77700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c06bdb0 con 0x7faa1c0c81f0 2023-10-08T16:06:01.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa23a77700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c0cb370 con 0x7faa1c0aca80 2023-10-08T16:06:01.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa23a77700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c159f60 con 0x7faa1c069d60 2023-10-08T16:06:01.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa22014700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7faa1c0c81f0 0x7faa1c0cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60544/0 (socket says 172.21.15.170:60544) 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.173+0000 7faa22014700 1 -- 172.21.15.170:0/423211241 learned_addr learned my addr 172.21.15.170:0/423211241 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa21012700 1 --2- 172.21.15.170:0/423211241 >> v2:172.21.15.191:3300/0 conn(0x7faa1c069d60 0x7faa1c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529120443 0 0) 0x7faa1c06bdb0 con 0x7faa1c0c81f0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa08003570 con 0x7faa1c0c81f0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3385466829 0 0) 0x7faa08003570 con 0x7faa1c0c81f0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 >> v2:172.21.15.191:3300/0 conn(0x7faa1c069d60 msgr2=0x7faa1c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 --2- 172.21.15.170:0/423211241 >> v2:172.21.15.191:3300/0 conn(0x7faa1c069d60 0x7faa1c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 >> v1:172.21.15.191:6789/0 conn(0x7faa1c0aca80 legacy=0x7faa1c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c06b2b0 con 0x7faa1c0c81f0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7faa140017f0 con 0x7faa1c0c81f0 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 >> v1:172.21.15.170:6789/0 conn(0x7faa1c0c81f0 legacy=0x7faa1c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa08003a60 0x7faa08003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08004810 0x7faa08004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 0x7faa0800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c159f60 con 0x7faa080051c0 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0800c0d0 con 0x7faa08004810 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0800c290 con 0x7faa08003a60 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 0x7faa0800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa21012700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08004810 0x7faa08004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa21813700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa08003a60 0x7faa08003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 -- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa08003a60 msgr2=0x7faa08003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa08003a60 0x7faa08003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 -- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08004810 msgr2=0x7faa08004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa08004810 0x7faa08004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 -- 172.21.15.170:0/423211241 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa08003570 con 0x7faa080051c0 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa21813700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa08003a60 0x7faa08003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.174+0000 7faa22014700 1 -- 172.21.15.170:0/423211241 reap_dead start 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa22014700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 0x7faa0800ba60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faa1400a930 tx=0x7faa140021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231b8e76bfde7910 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140070d0 con 0x7faa080051c0 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14007230 con 0x7faa080051c0 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa20811700 1 -- 172.21.15.170:0/423211241 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140075d0 con 0x7faa080051c0 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 -- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 msgr2=0x7faa0800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 0x7faa0800ba60 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7faa1400a930 tx=0x7faa140021d0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 -- 172.21.15.170:0/423211241 shutdown_connections 2023-10-08T16:06:01.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 --2- 172.21.15.170:0/423211241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa080051c0 0x7faa0800ba60 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 -- 172.21.15.170:0/423211241 >> 172.21.15.170:0/423211241 conn(0x7faa1c0cd670 msgr2=0x7faa1c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 -- 172.21.15.170:0/423211241 shutdown_connections 2023-10-08T16:06:01.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.175+0000 7faa23a77700 1 -- 172.21.15.170:0/423211241 wait complete. 2023-10-08T16:06:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.176+0000 7faa23a77700 1 Processor -- start 2023-10-08T16:06:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.176+0000 7faa23a77700 1 -- start start 2023-10-08T16:06:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa080051c0 0x7faa1c159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b2da0 0x7faa1c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15d8b0 con 0x7faa1c15e4d0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c159bc0 con 0x7faa080051c0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa23a77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0b17c0 con 0x7faa1c0b2da0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa21012700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa21012700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37054/0 (socket says 172.21.15.170:37054) 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa21012700 1 -- 172.21.15.170:0/2817320226 learned_addr learned my addr 172.21.15.170:0/2817320226 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa21813700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa080051c0 0x7faa1c159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.177+0000 7faa22014700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b2da0 0x7faa1c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 -- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b2da0 msgr2=0x7faa1c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b2da0 0x7faa1c15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 -- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa080051c0 msgr2=0x7faa1c159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa080051c0 0x7faa1c159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 -- 172.21.15.170:0/2817320226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0b1aa0 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa21012700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fd0 tx=0x7faa0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00f790 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00f8f0 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c0b1d80 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00fc70 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.178+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c0b2340 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.179+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c0b26b0 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.179+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faa0c014650 con 0x7faa1c15e4d0 2023-10-08T16:06:01.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.180+0000 7faa11ffb700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.180+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faa0c018030 con 0x7faa1c15e4d0 2023-10-08T16:06:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.180+0000 7faa21813700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.180+0000 7faa21813700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa18000c00 tx=0x7faa1800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.184+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa0c055aa0 con 0x7faa1c15e4d0 2023-10-08T16:06:01.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.393+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1c04c3c0 con 0x7faa00066f30 2023-10-08T16:06:01.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.394+0000 7faa11ffb700 1 -- 172.21.15.170:0/2817320226 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7faa1c04c3c0 con 0x7faa00066f30 2023-10-08T16:06:01.395 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:06:01.395 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:01.395 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:06:01.395 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:01.395 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 msgr2=0x7faa000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 0x7faa000693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faa18000c00 tx=0x7faa1800c040 comp rx=0 tx=0).stop 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 msgr2=0x7faa1c0b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fd0 tx=0x7faa0c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 shutdown_connections 2023-10-08T16:06:01.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faa1c0b2da0 0x7faa1c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faa080051c0 0x7faa1c159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faa00066f30 0x7faa000693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 --2- 172.21.15.170:0/2817320226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa1c15e4d0 0x7faa1c0b2860 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 >> 172.21.15.170:0/2817320226 conn(0x7faa1c0cd670 msgr2=0x7faa1c06a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 shutdown_connections 2023-10-08T16:06:01.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.396+0000 7faa23a77700 1 -- 172.21.15.170:0/2817320226 wait complete. 2023-10-08T16:06:01.409 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:06:01.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.525+0000 7f8ffa863700 1 Processor -- start 2023-10-08T16:06:01.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.525+0000 7f8ffa863700 1 -- start start 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ffa863700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ff406bcd0 0x7f8ff40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ffa863700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff415d7e0 con 0x7f8ff415df50 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ffa863700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff415dac0 con 0x7f8ff406c0a0 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ffa863700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff415aa30 con 0x7f8ff406bcd0 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff8e00700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8ff415df50 0x7f8ff40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60554/0 (socket says 172.21.15.170:60554) 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff8e00700 1 -- 172.21.15.170:0/2045557267 learned_addr learned my addr 172.21.15.170:0/2045557267 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff3fff700 1 --2- 172.21.15.170:0/2045557267 >> v2:172.21.15.191:3300/0 conn(0x7f8ff406bcd0 0x7f8ff40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff2ffd700 1 -- 172.21.15.170:0/2045557267 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510639571 0 0) 0x7f8ff415d7e0 con 0x7f8ff415df50 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff2ffd700 1 -- 172.21.15.170:0/2045557267 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fdc003570 con 0x7f8ff415df50 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff3fff700 1 -- 172.21.15.170:0/2045557267 >> v1:172.21.15.191:6789/0 conn(0x7f8ff406c0a0 legacy=0x7f8ff415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff3fff700 1 -- 172.21.15.170:0/2045557267 >> v1:172.21.15.170:6789/0 conn(0x7f8ff415df50 legacy=0x7f8ff40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.526+0000 7f8ff3fff700 1 -- 172.21.15.170:0/2045557267 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff415ad10 con 0x7f8ff406bcd0 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ff3fff700 1 --2- 172.21.15.170:0/2045557267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406bcd0 0x7f8ff40caa90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098e0 tx=0x7f8fe80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d716146278ea8bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ff2ffd700 1 -- 172.21.15.170:0/2045557267 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800b640 con 0x7f8ff406bcd0 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ff2ffd700 1 -- 172.21.15.170:0/2045557267 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe800b7a0 con 0x7f8ff406bcd0 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ff2ffd700 1 -- 172.21.15.170:0/2045557267 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800bb20 con 0x7f8ff406bcd0 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 -- 172.21.15.170:0/2045557267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406bcd0 msgr2=0x7f8ff40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 --2- 172.21.15.170:0/2045557267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406bcd0 0x7f8ff40caa90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098e0 tx=0x7f8fe80091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 -- 172.21.15.170:0/2045557267 shutdown_connections 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 --2- 172.21.15.170:0/2045557267 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406bcd0 0x7f8ff40caa90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 -- 172.21.15.170:0/2045557267 >> 172.21.15.170:0/2045557267 conn(0x7f8ff40ce600 msgr2=0x7f8ff40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 -- 172.21.15.170:0/2045557267 shutdown_connections 2023-10-08T16:06:01.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.527+0000 7f8ffa863700 1 -- 172.21.15.170:0/2045557267 wait complete. 2023-10-08T16:06:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.528+0000 7f8ffa863700 1 Processor -- start 2023-10-08T16:06:01.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 -- start start 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ff406bcd0 0x7f8ff40c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406c0a0 0x7f8ff4149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff416ae20 con 0x7f8ff415df50 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff416bc80 con 0x7f8ff406c0a0 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.529+0000 7f8ffa863700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40c8960 con 0x7f8ff406bcd0 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37068/0 (socket says 172.21.15.170:37068) 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 -- 172.21.15.170:0/1000398349 learned_addr learned my addr 172.21.15.170:0/1000398349 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff37fe700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406c0a0 0x7f8ff4149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff3fff700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ff406bcd0 0x7f8ff40c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 -- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ff406bcd0 msgr2=0x7f8ff40c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ff406bcd0 0x7f8ff40c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 -- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406c0a0 msgr2=0x7f8ff4149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406c0a0 0x7f8ff4149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 -- 172.21.15.170:0/1000398349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8009420 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ff8e00700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00cc40 tx=0x7f8fec001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00a940 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00aaa0 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff41498a0 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00ae20 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.530+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff416b350 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.531+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff404c3c0 con 0x7f8ff415df50 2023-10-08T16:06:01.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.531+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fec007300 con 0x7f8ff415df50 2023-10-08T16:06:01.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.532+0000 7f8fdbfff700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 0x7f8fe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.532+0000 7f8ff3fff700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 0x7f8fe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.532+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8fec0848d0 con 0x7f8ff415df50 2023-10-08T16:06:01.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.532+0000 7f8ff3fff700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 0x7f8fe00693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80093f0 tx=0x7f8fe8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.536+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fec051bc0 con 0x7f8ff415df50 2023-10-08T16:06:01.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.717+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8ff4071280 con 0x7f8ff415df50 2023-10-08T16:06:01.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.718+0000 7f8fdbfff700 1 -- 172.21.15.170:0/1000398349 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8fec0551e0 con 0x7f8ff415df50 2023-10-08T16:06:01.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 msgr2=0x7f8fe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 0x7f8fe00693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80093f0 tx=0x7f8fe8017000 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 msgr2=0x7f8ff40cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8fec00cc40 tx=0x7f8fec001a40 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 shutdown_connections 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8fe0066f30 0x7f8fe00693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ff415df50 0x7f8ff40cc3e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ff406bcd0 0x7f8ff40c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 --2- 172.21.15.170:0/1000398349 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ff406c0a0 0x7f8ff4149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 >> 172.21.15.170:0/1000398349 conn(0x7f8ff40ce600 msgr2=0x7f8ff415b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 shutdown_connections 2023-10-08T16:06:01.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.720+0000 7f8ffa863700 1 -- 172.21.15.170:0/1000398349 wait complete. 2023-10-08T16:06:01.732 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000414781 2023-10-08T16:06:01.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.849+0000 7f2436cbf700 1 Processor -- start 2023-10-08T16:06:01.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.849+0000 7f2436cbf700 1 -- start start 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2436cbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2436cbf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2430159c60 0x7f243015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2436cbf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243015a160 con 0x7f243015db10 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2436cbf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24300d13a0 con 0x7f2430159c60 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2436cbf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243015e140 con 0x7f2430159890 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f242ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2430159c60 0x7f243015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f243525c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f243015db10 0x7f2430069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60560/0 (socket says 172.21.15.170:60560) 2023-10-08T16:06:01.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f242ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2430159c60 0x7f243015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37072/0 (socket says 172.21.15.170:37072) 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f243525c700 1 -- 172.21.15.170:0/876449471 learned_addr learned my addr 172.21.15.170:0/876449471 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.850+0000 7f2434a5b700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242ffff700 1 -- 172.21.15.170:0/876449471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 msgr2=0x7f243015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242ffff700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242ffff700 1 -- 172.21.15.170:0/876449471 >> v1:172.21.15.170:6789/0 conn(0x7f243015db10 legacy=0x7f2430069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242ffff700 1 -- 172.21.15.170:0/876449471 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243015a3d0 con 0x7f2430159c60 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242f7fe700 1 -- 172.21.15.170:0/876449471 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220462052 0 0) 0x7f243015a160 con 0x7f243015db10 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2434a5b700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242ffff700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f243015d5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f24180098e0 tx=0x7f24180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925513afd5ad2285 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242f7fe700 1 -- 172.21.15.170:0/876449471 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418009e80 con 0x7f2430159c60 2023-10-08T16:06:01.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242f7fe700 1 -- 172.21.15.170:0/876449471 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800b860 con 0x7f2430159c60 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f242f7fe700 1 -- 172.21.15.170:0/876449471 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800bbe0 con 0x7f2430159c60 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 -- 172.21.15.170:0/876449471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 msgr2=0x7f243015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f243015d5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f24180098e0 tx=0x7f24180091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 -- 172.21.15.170:0/876449471 shutdown_connections 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2430159890 0x7f243015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 --2- 172.21.15.170:0/876449471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f243015d5d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 -- 172.21.15.170:0/876449471 >> 172.21.15.170:0/876449471 conn(0x7f2430068e20 msgr2=0x7f24300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 -- 172.21.15.170:0/876449471 shutdown_connections 2023-10-08T16:06:01.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.851+0000 7f2436cbf700 1 -- 172.21.15.170:0/876449471 wait complete. 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.852+0000 7f2436cbf700 1 Processor -- start 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 -- start start 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 0x7f2430140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 0x7f2430155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24300b64a0 con 0x7f2430159c60 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243016db60 con 0x7f2430159890 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f2436cbf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24300b61c0 con 0x7f243015db10 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f242ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f242ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37078/0 (socket says 172.21.15.170:37078) 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.853+0000 7f242ffff700 1 -- 172.21.15.170:0/1312032818 learned_addr learned my addr 172.21.15.170:0/1312032818 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:01.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f243525c700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 0x7f2430155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f2434a5b700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 0x7f2430140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 -- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 msgr2=0x7f2430155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 0x7f2430155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 -- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 msgr2=0x7f2430140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 0x7f2430140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 -- 172.21.15.170:0/1312032818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009420 con 0x7f2430159c60 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f243525c700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 0x7f2430155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f2434a5b700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 0x7f2430140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242ffff700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2418013000 tx=0x7f2418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800b9d0 con 0x7f2430159c60 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800bb30 con 0x7f2430159c60 2023-10-08T16:06:01.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243016d510 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418025650 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.854+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243016dff0 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.855+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243016d370 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.855+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2418023070 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.856+0000 7f242cff9700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 0x7f24100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.856+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2418085ae0 con 0x7f2430159c60 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.856+0000 7f2434a5b700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 0x7f24100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:01.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.856+0000 7f2434a5b700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 0x7f24100693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f24200076e0 tx=0x7f24200098f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:01.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:01.860+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f243016d370 con 0x7f2430159c60 2023-10-08T16:06:02.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.069+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000414781", "target": ["mon-mgr", ""]}) v1 -- 0x7f243016d370 con 0x7f2410066f30 2023-10-08T16:06:02.073 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:02.073+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:02.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:02.073+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:02.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:02.073+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:02.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:02.073+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:02.074 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:02.073+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:02.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.087+0000 7f242cff9700 1 -- 172.21.15.170:0/1312032818 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f243016d370 con 0x7f2410066f30 2023-10-08T16:06:02.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 msgr2=0x7f24100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 0x7f24100693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f24200076e0 tx=0x7f24200098f0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 msgr2=0x7f2430154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2418013000 tx=0x7f2418017000 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 shutdown_connections 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243015db10 0x7f2430155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2430159890 0x7f2430140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2410066f30 0x7f24100693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 --2- 172.21.15.170:0/1312032818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430159c60 0x7f2430154ef0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 >> 172.21.15.170:0/1312032818 conn(0x7f2430068e20 msgr2=0x7f2430153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 shutdown_connections 2023-10-08T16:06:02.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.089+0000 7f2436cbf700 1 -- 172.21.15.170:0/1312032818 wait complete. 2023-10-08T16:06:02.101 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-08T16:06:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.217+0000 7fce2c8d5700 1 Processor -- start 2023-10-08T16:06:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.217+0000 7fce2c8d5700 1 -- start start 2023-10-08T16:06:02.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2c8d5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fce280b4e70 0x7fce280ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2c8d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fce280b16d0 0x7fce280b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2c8d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce280b2080 con 0x7fce2815d240 2023-10-08T16:06:02.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2c8d5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce280cfb00 con 0x7fce280b4e70 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2c8d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce280b5c40 con 0x7fce280b16d0 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fce280b4e70 0x7fce280ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fce280b4e70 0x7fce280ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37094/0 (socket says 172.21.15.170:37094) 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 -- 172.21.15.170:0/1138557478 learned_addr learned my addr 172.21.15.170:0/1138557478 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce2659c700 1 --2- 172.21.15.170:0/1138557478 >> v2:172.21.15.191:3300/0 conn(0x7fce280b16d0 0x7fce280b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 -- 172.21.15.170:0/1138557478 >> v2:172.21.15.191:3300/0 conn(0x7fce280b16d0 msgr2=0x7fce280b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 --2- 172.21.15.170:0/1138557478 >> v2:172.21.15.191:3300/0 conn(0x7fce280b16d0 0x7fce280b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 -- 172.21.15.170:0/1138557478 >> v1:172.21.15.191:6789/0 conn(0x7fce2815d240 legacy=0x7fce2815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 -- 172.21.15.170:0/1138557478 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce280b5f20 con 0x7fce280b4e70 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce26d9d700 1 --2- 172.21.15.170:0/1138557478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce280ccff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce28089830 tx=0x7fce20009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c28b433f819127 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce25d9b700 1 -- 172.21.15.170:0/1138557478 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000e070 con 0x7fce280b4e70 2023-10-08T16:06:02.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce25d9b700 1 -- 172.21.15.170:0/1138557478 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000ba20 con 0x7fce280b4e70 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.218+0000 7fce25d9b700 1 -- 172.21.15.170:0/1138557478 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce200092e0 con 0x7fce280b4e70 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1138557478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 msgr2=0x7fce280ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1138557478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce280ccff0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fce28089830 tx=0x7fce20009840 comp rx=0 tx=0).stop 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1138557478 shutdown_connections 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1138557478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce280ccff0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1138557478 >> v2:172.21.15.191:3300/0 conn(0x7fce280b16d0 0x7fce280b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1138557478 >> 172.21.15.170:0/1138557478 conn(0x7fce280d10c0 msgr2=0x7fce280affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1138557478 shutdown_connections 2023-10-08T16:06:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.219+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1138557478 wait complete. 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.220+0000 7fce2c8d5700 1 Processor -- start 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.220+0000 7fce2c8d5700 1 -- start start 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce280b16d0 0x7fce280c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2815d240 0x7fce2815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2816d420 con 0x7fce280b4e70 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28157cb0 con 0x7fce2815d240 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2c8d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2816d0f0 con 0x7fce280b16d0 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37108/0 (socket says 172.21.15.170:37108) 2023-10-08T16:06:02.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 -- 172.21.15.170:0/1053434818 learned_addr learned my addr 172.21.15.170:0/1053434818 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce26d9d700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce280b16d0 0x7fce280c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2759e700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2815d240 0x7fce2815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 -- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce280b16d0 msgr2=0x7fce280c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce280b16d0 0x7fce280c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 -- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2815d240 msgr2=0x7fce2815e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2815d240 0x7fce2815e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 -- 172.21.15.170:0/1053434818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20009ae0 con 0x7fce280b4e70 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.221+0000 7fce2659c700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fce1000ec80 tx=0x7fce100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.222+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000c850 con 0x7fce280b4e70 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.222+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1000c9b0 con 0x7fce280b4e70 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.222+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce28158720 con 0x7fce280b4e70 2023-10-08T16:06:02.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.222+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1000cd50 con 0x7fce280b4e70 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.222+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce280c9ed0 con 0x7fce280b4e70 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.223+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2816d590 con 0x7fce280b4e70 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.223+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce10013680 con 0x7fce280b4e70 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.223+0000 7fce1effd700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.223+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce100847f0 con 0x7fce280b4e70 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.224+0000 7fce26d9d700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.224+0000 7fce26d9d700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce20009880 tx=0x7fce20018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.228+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce10051a60 con 0x7fce280b4e70 2023-10-08T16:06:02.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.438+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7fce2804bb80 con 0x7fce08066f30 2023-10-08T16:06:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.440+0000 7fce1effd700 1 -- 172.21.15.170:0/1053434818 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+531 (secure 0 0 0) 0x7fce2804bb80 con 0x7fce08066f30 2023-10-08T16:06:02.440 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:06:02.440 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2023-10-08T16:06:02.440 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "pending_subvolume_deletions": 0, 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "pools": { 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "data": [ 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365980352512, 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_data", 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 0 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:02.441 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": [ 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365980352512, 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_metadata", 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 65536 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout: "used_size": 0 2023-10-08T16:06:02.442 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:06:02.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 msgr2=0x7fce080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fce20009880 tx=0x7fce20018000 comp rx=0 tx=0).stop 2023-10-08T16:06:02.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 msgr2=0x7fce28157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fce1000ec80 tx=0x7fce100091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 shutdown_connections 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fce2815d240 0x7fce2815e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fce280b16d0 0x7fce280c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 --2- 172.21.15.170:0/1053434818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce280b4e70 0x7fce28157770 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 >> 172.21.15.170:0/1053434818 conn(0x7fce280d10c0 msgr2=0x7fce28158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 shutdown_connections 2023-10-08T16:06:02.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.442+0000 7fce2c8d5700 1 -- 172.21.15.170:0/1053434818 wait complete. 2023-10-08T16:06:02.454 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:06:02.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:06:02.490 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:06:02.541 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:06:02.541 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:06:02.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:02.542 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:06:02.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:06:02.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:02.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:06:02.634 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:06:02.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:02.634 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:06:02.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:06:02.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:02.677 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:06:02.733 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:02.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.848+0000 7eff58306700 1 Processor -- start 2023-10-08T16:06:02.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.848+0000 7eff58306700 1 -- start start 2023-10-08T16:06:02.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff50140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff5006c0a0 0x7eff50141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 0x7eff500c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5015c2a0 con 0x7eff5006c0a0 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5015b690 con 0x7eff500c8e00 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff58306700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5015b970 con 0x7eff5006bcd0 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff558a1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff5006c0a0 0x7eff50141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff558a1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7eff5006c0a0 0x7eff50141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37124/0 (socket says 172.21.15.170:37124) 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.849+0000 7eff558a1700 1 -- 172.21.15.170:0/3161235629 learned_addr learned my addr 172.21.15.170:0/3161235629 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff568a3700 1 --2- 172.21.15.170:0/3161235629 >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 0x7eff500c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff560a2700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff50140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 -- 172.21.15.170:0/3161235629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 msgr2=0x7eff50140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:02.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff50140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 -- 172.21.15.170:0/3161235629 >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 msgr2=0x7eff500c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 --2- 172.21.15.170:0/3161235629 >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 0x7eff500c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 -- 172.21.15.170:0/3161235629 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5015bc50 con 0x7eff5006c0a0 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff568a3700 1 --2- 172.21.15.170:0/3161235629 >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 0x7eff500c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff558a1700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50141330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff5015b530 tx=0x7eff40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110d3441af8ef293 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff550a0700 1 -- 172.21.15.170:0/3161235629 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000b860 con 0x7eff5006c0a0 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff550a0700 1 -- 172.21.15.170:0/3161235629 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4000b9c0 con 0x7eff5006c0a0 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.850+0000 7eff550a0700 1 -- 172.21.15.170:0/3161235629 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4000bd60 con 0x7eff5006c0a0 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 -- 172.21.15.170:0/3161235629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 msgr2=0x7eff50141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50141330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff5015b530 tx=0x7eff40009250 comp rx=0 tx=0).stop 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 -- 172.21.15.170:0/3161235629 shutdown_connections 2023-10-08T16:06:02.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 --2- 172.21.15.170:0/3161235629 >> v2:172.21.15.191:3300/0 conn(0x7eff500c8e00 0x7eff500c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff50140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 --2- 172.21.15.170:0/3161235629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50141330 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 -- 172.21.15.170:0/3161235629 >> 172.21.15.170:0/3161235629 conn(0x7eff500ce600 msgr2=0x7eff500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:02.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 -- 172.21.15.170:0/3161235629 shutdown_connections 2023-10-08T16:06:02.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.851+0000 7eff58306700 1 -- 172.21.15.170:0/3161235629 wait complete. 2023-10-08T16:06:02.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.852+0000 7eff58306700 1 Processor -- start 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.852+0000 7eff58306700 1 -- start start 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff5015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff500c8e00 0x7eff50167ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5015b150 con 0x7eff5006c0a0 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5015ced0 con 0x7eff500c8e00 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5015ae70 con 0x7eff5006bcd0 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37138/0 (socket says 172.21.15.170:37138) 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 -- 172.21.15.170:0/1445911449 learned_addr learned my addr 172.21.15.170:0/1445911449 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff560a2700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff5015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff568a3700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff500c8e00 0x7eff50167ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 -- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 msgr2=0x7eff5015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff5015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 -- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff500c8e00 msgr2=0x7eff50167ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff500c8e00 0x7eff50167ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 -- 172.21.15.170:0/1445911449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400094f0 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff558a1700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff400090b0 tx=0x7eff40017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011e00 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4001f070 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5015c880 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400266d0 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.853+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff501595d0 con 0x7eff5006c0a0 2023-10-08T16:06:02.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.854+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff500c9570 con 0x7eff5006c0a0 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.855+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff40026a40 con 0x7eff5006c0a0 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.855+0000 7eff467fc700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 0x7eff340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.855+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7eff40021070 con 0x7eff5006c0a0 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.855+0000 7eff560a2700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 0x7eff340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.855+0000 7eff560a2700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 0x7eff340693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009fd0 tx=0x7eff4c009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:02.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:02.859+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff400534c0 con 0x7eff5006c0a0 2023-10-08T16:06:03.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.047+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7eff40026cf0 con 0x7eff5006c0a0 2023-10-08T16:06:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.091+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff5004c3c0 con 0x7eff5006c0a0 2023-10-08T16:06:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.091+0000 7eff467fc700 1 -- 172.21.15.170:0/1445911449 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4600 (secure 0 0 0) 0x7eff40058190 con 0x7eff5006c0a0 2023-10-08T16:06:03.092 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:03.092 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:06:03.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 msgr2=0x7eff340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 0x7eff340693e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009fd0 tx=0x7eff4c009880 comp rx=0 tx=0).stop 2023-10-08T16:06:03.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 msgr2=0x7eff50157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff400090b0 tx=0x7eff40017000 comp rx=0 tx=0).stop 2023-10-08T16:06:03.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 shutdown_connections 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7eff500c8e00 0x7eff50167ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7eff5006bcd0 0x7eff5015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.093+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff34066f30 0x7eff340693e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.094+0000 7eff58306700 1 --2- 172.21.15.170:0/1445911449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff5006c0a0 0x7eff50157680 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.094+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 >> 172.21.15.170:0/1445911449 conn(0x7eff500ce600 msgr2=0x7eff500cc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.094+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 shutdown_connections 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.094+0000 7eff58306700 1 -- 172.21.15.170:0/1445911449 wait complete. 2023-10-08T16:06:03.095 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 31 2023-10-08T16:06:03.106 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:03.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.219+0000 7f3de3c3d700 1 Processor -- start 2023-10-08T16:06:03.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.219+0000 7f3de3c3d700 1 -- start start 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de3c3d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3ddc06bcd0 0x7f3ddc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de3c3d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3ddc06c0a0 0x7f3ddc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de3c3d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc15d7e0 con 0x7f3ddc15df50 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de3c3d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc15dac0 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de3c3d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc15aa30 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de11d8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3ddc06c0a0 0x7f3ddc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de21da700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3ddc15df50 0x7f3ddc0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60574/0 (socket says 172.21.15.170:60574) 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de21da700 1 -- 172.21.15.170:0/344163177 learned_addr learned my addr 172.21.15.170:0/344163177 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.220+0000 7f3de19d9700 1 --2- 172.21.15.170:0/344163177 >> v2:172.21.15.191:3300/0 conn(0x7f3ddc06bcd0 0x7f3ddc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de11d8700 1 -- 172.21.15.170:0/344163177 >> v2:172.21.15.191:3300/0 conn(0x7f3ddc06bcd0 msgr2=0x7f3ddc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de11d8700 1 --2- 172.21.15.170:0/344163177 >> v2:172.21.15.191:3300/0 conn(0x7f3ddc06bcd0 0x7f3ddc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de11d8700 1 -- 172.21.15.170:0/344163177 >> v1:172.21.15.170:6789/0 conn(0x7f3ddc15df50 legacy=0x7f3ddc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de09d7700 1 -- 172.21.15.170:0/344163177 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227489623 0 0) 0x7f3ddc15d7e0 con 0x7f3ddc15df50 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de11d8700 1 -- 172.21.15.170:0/344163177 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc15ad10 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de11d8700 1 --2- 172.21.15.170:0/344163177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06c0a0 0x7f3ddc15a1e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0691a0 tx=0x7f3dcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71748fe9718c22f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de09d7700 1 -- 172.21.15.170:0/344163177 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011630 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de09d7700 1 -- 172.21.15.170:0/344163177 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc011790 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de09d7700 1 -- 172.21.15.170:0/344163177 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011b30 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 -- 172.21.15.170:0/344163177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06c0a0 msgr2=0x7f3ddc15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/344163177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06c0a0 0x7f3ddc15a1e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0691a0 tx=0x7f3dcc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 -- 172.21.15.170:0/344163177 shutdown_connections 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/344163177 >> v2:172.21.15.191:3300/0 conn(0x7f3ddc06bcd0 0x7f3ddc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/344163177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06c0a0 0x7f3ddc15a1e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.221+0000 7f3de3c3d700 1 -- 172.21.15.170:0/344163177 >> 172.21.15.170:0/344163177 conn(0x7f3ddc0ce600 msgr2=0x7f3ddc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.222+0000 7f3de3c3d700 1 -- 172.21.15.170:0/344163177 shutdown_connections 2023-10-08T16:06:03.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.222+0000 7f3de3c3d700 1 -- 172.21.15.170:0/344163177 wait complete. 2023-10-08T16:06:03.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 Processor -- start 2023-10-08T16:06:03.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 -- start start 2023-10-08T16:06:03.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 0x7f3ddc0c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc15df50 0x7f3ddc0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1533b0 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc153c50 con 0x7f3ddc15df50 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de3c3d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc14bf70 con 0x7f3ddc06c0a0 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de19d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de19d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37154/0 (socket says 172.21.15.170:37154) 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.223+0000 7f3de19d9700 1 -- 172.21.15.170:0/1527117112 learned_addr learned my addr 172.21.15.170:0/1527117112 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de21da700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc15df50 0x7f3ddc0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de11d8700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 0x7f3ddc0c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 -- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 msgr2=0x7f3ddc0c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 0x7f3ddc0c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 -- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc15df50 msgr2=0x7f3ddc0b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc15df50 0x7f3ddc0b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 -- 172.21.15.170:0/1527117112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc009420 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de11d8700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 0x7f3ddc0c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de19d9700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e80 tx=0x7f3dd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800cb50 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800f040 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc0c4b50 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8014480 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.224+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc1538e0 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.225+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc153f30 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.225+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3dd80147f0 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.226+0000 7f3dd1ffb700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 0x7f3dc0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.226+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3dd8085000 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.226+0000 7f3de11d8700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 0x7f3dc0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.226+0000 7f3de11d8700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 0x7f3dc0069470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0093f0 tx=0x7f3dcc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.230+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dd8052170 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.413+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ddc04c3c0 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.414+0000 7f3dd1ffb700 1 -- 172.21.15.170:0/1527117112 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f3dd8055790 con 0x7f3ddc06bcd0 2023-10-08T16:06:03.414 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:03.414 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":51,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:53.008429+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 msgr2=0x7f3dc0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 0x7f3dc0069470 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0093f0 tx=0x7f3dcc017000 comp rx=0 tx=0).stop 2023-10-08T16:06:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 msgr2=0x7f3ddc14ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009e80 tx=0x7f3dd800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:03.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 shutdown_connections 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ddc15df50 0x7f3ddc0b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3dc0066fc0 0x7f3dc0069470 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ddc06bcd0 0x7f3ddc14ba30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 --2- 172.21.15.170:0/1527117112 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ddc06c0a0 0x7f3ddc0c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 >> 172.21.15.170:0/1527117112 conn(0x7f3ddc0ce600 msgr2=0x7f3ddc0cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 shutdown_connections 2023-10-08T16:06:03.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.416+0000 7f3de3c3d700 1 -- 172.21.15.170:0/1527117112 wait complete. 2023-10-08T16:06:03.429 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:05:49.716147+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:05:50.344843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:06:03.430 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:06:03.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.546+0000 7fa9e5e35700 1 Processor -- start 2023-10-08T16:06:03.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.546+0000 7fa9e5e35700 1 -- start start 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5e35700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 0x7fa9e00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5e35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 0x7fa9e00cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5e35700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e00cb900 con 0x7fa9e00b1450 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5e35700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e00cc700 con 0x7fa9e0157680 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5e35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e00cd240 con 0x7fa9e0157a50 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e4e33700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 0x7fa9e00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9dffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 0x7fa9e00cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5634700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fa9e00b1450 0x7fa9e00c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55860/0 (socket says 172.21.15.170:55860) 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9e5634700 1 -- 172.21.15.170:0/1266516203 learned_addr learned my addr 172.21.15.170:0/1266516203 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9df7fe700 1 -- 172.21.15.170:0/1266516203 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247339767 0 0) 0x7fa9c8001110 con 0x7fa9e00b1450 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.547+0000 7fa9df7fe700 1 -- 172.21.15.170:0/1266516203 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9cc003570 con 0x7fa9e00b1450 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9dffff700 1 -- 172.21.15.170:0/1266516203 >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 msgr2=0x7fa9e00cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9dffff700 1 --2- 172.21.15.170:0/1266516203 >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 0x7fa9e00cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9dffff700 1 -- 172.21.15.170:0/1266516203 >> v1:172.21.15.191:6789/0 conn(0x7fa9e00b1450 legacy=0x7fa9e00c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9dffff700 1 -- 172.21.15.170:0/1266516203 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e00c9a20 con 0x7fa9e0157a50 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e4e33700 1 --2- 172.21.15.170:0/1266516203 >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 0x7fa9e00cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:03.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9dffff700 1 --2- 172.21.15.170:0/1266516203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 0x7fa9e00cb3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098e0 tx=0x7fa9d00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7305a41b229cff08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9df7fe700 1 -- 172.21.15.170:0/1266516203 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0011630 con 0x7fa9e0157a50 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9df7fe700 1 -- 172.21.15.170:0/1266516203 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d0011790 con 0x7fa9e0157a50 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9df7fe700 1 -- 172.21.15.170:0/1266516203 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0011b10 con 0x7fa9e0157a50 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 -- 172.21.15.170:0/1266516203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 msgr2=0x7fa9e00cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/1266516203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 0x7fa9e00cb3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00098e0 tx=0x7fa9d00091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 -- 172.21.15.170:0/1266516203 shutdown_connections 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/1266516203 >> v2:172.21.15.191:3300/0 conn(0x7fa9e0157680 0x7fa9e00cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/1266516203 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e0157a50 0x7fa9e00cb3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.548+0000 7fa9e5e35700 1 -- 172.21.15.170:0/1266516203 >> 172.21.15.170:0/1266516203 conn(0x7fa9e006baa0 msgr2=0x7fa9e006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.549+0000 7fa9e5e35700 1 -- 172.21.15.170:0/1266516203 shutdown_connections 2023-10-08T16:06:03.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.549+0000 7fa9e5e35700 1 -- 172.21.15.170:0/1266516203 wait complete. 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.550+0000 7fa9e5e35700 1 Processor -- start 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.550+0000 7fa9e5e35700 1 -- start start 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.550+0000 7fa9e5e35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e00b1450 0x7fa9e00c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.550+0000 7fa9e5e35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e0157680 0x7fa9e01512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0158ba0 con 0x7fa9e0157a50 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0151d40 con 0x7fa9e0157680 2023-10-08T16:06:03.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0157f90 con 0x7fa9e00b1450 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37164/0 (socket says 172.21.15.170:37164) 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 -- 172.21.15.170:0/261637521 learned_addr learned my addr 172.21.15.170:0/261637521 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9dffff700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e0157680 0x7fa9e01512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e4e33700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e00b1450 0x7fa9e00c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 -- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e00b1450 msgr2=0x7fa9e00c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e00b1450 0x7fa9e00c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 -- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e0157680 msgr2=0x7fa9e01512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e0157680 0x7fa9e01512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 -- 172.21.15.170:0/261637521 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8009a20 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5634700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c790 tx=0x7fa9c8007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8005b30 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8005c90 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d0009420 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8007300 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.551+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e0158550 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.552+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0158d50 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.553+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa9c801b400 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.553+0000 7fa9dcff9700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 0x7fa9c0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.553+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa9c8083e90 con 0x7fa9e0157a50 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.553+0000 7fa9e4e33700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 0x7fa9c0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.553+0000 7fa9e4e33700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 0x7fa9c0069470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80099f0 tx=0x7fa9d800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.557+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9e0158d50 con 0x7fa9e0157a50 2023-10-08T16:06:03.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.738+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9e00cb900 con 0x7fa9e0157a50 2023-10-08T16:06:03.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.738+0000 7fa9dcff9700 1 -- 172.21.15.170:0/261637521 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9e00cb900 con 0x7fa9e0157a50 2023-10-08T16:06:03.739 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 msgr2=0x7fa9c0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 0x7fa9c0069470 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80099f0 tx=0x7fa9d800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 msgr2=0x7fa9e0151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800c790 tx=0x7fa9c8007c30 comp rx=0 tx=0).stop 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 shutdown_connections 2023-10-08T16:06:03.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa9c0066fc0 0x7fa9c0069470 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa9e0157a50 0x7fa9e0151800 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa9e00b1450 0x7fa9e00c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 --2- 172.21.15.170:0/261637521 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa9e0157680 0x7fa9e01512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 >> 172.21.15.170:0/261637521 conn(0x7fa9e006baa0 msgr2=0x7fa9e00c8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.740+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 shutdown_connections 2023-10-08T16:06:03.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.741+0000 7fa9e5e35700 1 -- 172.21.15.170:0/261637521 wait complete. 2023-10-08T16:06:03.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:06:03.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.870+0000 7f1a1119d700 1 Processor -- start 2023-10-08T16:06:03.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.870+0000 7f1a1119d700 1 -- start start 2023-10-08T16:06:03.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.870+0000 7f1a1119d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1a0c159c60 0x7f1a0c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a1119d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c15a160 con 0x7f1a0c15db10 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a1119d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0c0d13a0 con 0x7f1a0c159890 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a1119d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0c15df50 con 0x7f1a0c159c60 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0b59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1a0c15db10 0x7f1a0c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60586/0 (socket says 172.21.15.170:60586) 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0b59e700 1 -- 172.21.15.170:0/81817254 learned_addr learned my addr 172.21.15.170:0/81817254 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:03.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/81817254 >> v2:172.21.15.191:3300/0 conn(0x7f1a0c159c60 0x7f1a0c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a09d9b700 1 -- 172.21.15.170:0/81817254 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990271240 0 0) 0x7f1a0c15a160 con 0x7f1a0c15db10 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a09d9b700 1 -- 172.21.15.170:0/81817254 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19f8003570 con 0x7f1a0c15db10 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0a59c700 1 -- 172.21.15.170:0/81817254 >> v1:172.21.15.191:6789/0 conn(0x7f1a0c159890 legacy=0x7f1a0c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0a59c700 1 -- 172.21.15.170:0/81817254 >> v1:172.21.15.170:6789/0 conn(0x7f1a0c15db10 legacy=0x7f1a0c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0a59c700 1 -- 172.21.15.170:0/81817254 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c15e140 con 0x7f1a0c159c60 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/81817254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c159c60 0x7f1a0c15d5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f19f40098e0 tx=0x7f19f40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1816980406525784 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a09d9b700 1 -- 172.21.15.170:0/81817254 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400b640 con 0x7f1a0c159c60 2023-10-08T16:06:03.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a09d9b700 1 -- 172.21.15.170:0/81817254 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f400b7a0 con 0x7f1a0c159c60 2023-10-08T16:06:03.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.871+0000 7f1a09d9b700 1 -- 172.21.15.170:0/81817254 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400bb40 con 0x7f1a0c159c60 2023-10-08T16:06:03.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 -- 172.21.15.170:0/81817254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c159c60 msgr2=0x7f1a0c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 --2- 172.21.15.170:0/81817254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c159c60 0x7f1a0c15d5d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f19f40098e0 tx=0x7f19f40091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 -- 172.21.15.170:0/81817254 shutdown_connections 2023-10-08T16:06:03.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 --2- 172.21.15.170:0/81817254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c159c60 0x7f1a0c15d5d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 -- 172.21.15.170:0/81817254 >> 172.21.15.170:0/81817254 conn(0x7f1a0c068e20 msgr2=0x7f1a0c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:03.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 -- 172.21.15.170:0/81817254 shutdown_connections 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.872+0000 7f1a1119d700 1 -- 172.21.15.170:0/81817254 wait complete. 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.873+0000 7f1a1119d700 1 Processor -- start 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.873+0000 7f1a1119d700 1 -- start start 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 0x7f1a0c0b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c15db10 0x7f1a0c0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c16a940 con 0x7f1a0c159890 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c16b760 con 0x7f1a0c15db10 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a1119d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c16ab50 con 0x7f1a0c159c60 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37178/0 (socket says 172.21.15.170:37178) 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 -- 172.21.15.170:0/3847767271 learned_addr learned my addr 172.21.15.170:0/3847767271 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0b59e700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c15db10 0x7f1a0c0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 0x7f1a0c0b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 -- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 msgr2=0x7f1a0c0b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 0x7f1a0c0b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 -- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c15db10 msgr2=0x7f1a0c0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c15db10 0x7f1a0c0b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0ad9d700 1 -- 172.21.15.170:0/3847767271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009420 con 0x7f1a0c159890 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.874+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 0x7f1a0c0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a0ad9d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cb10 tx=0x7f1a040017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400a760 con 0x7f1a0c159890 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0400a8e0 con 0x7f1a0c159890 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c16ae30 con 0x7f1a0c159890 2023-10-08T16:06:03.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0400aca0 con 0x7f1a0c159890 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.875+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c16b3f0 con 0x7f1a0c159890 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.876+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c04c3c0 con 0x7f1a0c159890 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.876+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f1a04003030 con 0x7f1a0c159890 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.876+0000 7f1a02ffd700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 0x7f19ec069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.876+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1a040845f0 con 0x7f1a0c159890 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.876+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 0x7f19ec069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.877+0000 7f1a0a59c700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 0x7f19ec069470 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0b1340 tx=0x7f19f4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:03.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:03.881+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a04051870 con 0x7f1a0c159890 2023-10-08T16:06:04.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.060+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a0c06a5b0 con 0x7f1a0c159890 2023-10-08T16:06:04.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.061+0000 7f1a02ffd700 1 -- 172.21.15.170:0/3847767271 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a04054e90 con 0x7f1a0c159890 2023-10-08T16:06:04.061 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 msgr2=0x7f19ec069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 0x7f19ec069470 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0b1340 tx=0x7f19f4017000 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 msgr2=0x7f1a0c152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400cb10 tx=0x7f1a040017b0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 shutdown_connections 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1a0c15db10 0x7f1a0c0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f19ec066fc0 0x7f19ec069470 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1a0c159890 0x7f1a0c152cc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 --2- 172.21.15.170:0/3847767271 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1a0c159c60 0x7f1a0c0b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 >> 172.21.15.170:0/3847767271 conn(0x7f1a0c068e20 msgr2=0x7f1a0c1585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 shutdown_connections 2023-10-08T16:06:04.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.063+0000 7f1a1119d700 1 -- 172.21.15.170:0/3847767271 wait complete. 2023-10-08T16:06:04.076 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:06:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.192+0000 7f868650f700 1 Processor -- start 2023-10-08T16:06:04.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.192+0000 7f868650f700 1 -- start start 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f868650f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8680069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f868650f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8680068aa0 0x7f86800c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f868650f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800c7de0 con 0x7f868006a1c0 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f868650f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8680140c10 con 0x7f8680068aa0 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f868650f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8680141490 con 0x7f8680140350 2023-10-08T16:06:04.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f867f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f868006a1c0 0x7f8680068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60600/0 (socket says 172.21.15.170:60600) 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.193+0000 7f867f7fe700 1 -- 172.21.15.170:0/2968049163 learned_addr learned my addr 172.21.15.170:0/2968049163 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 --2- 172.21.15.170:0/2968049163 >> v2:172.21.15.191:3300/0 conn(0x7f8680068aa0 0x7f86800c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867ffff700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8680069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783304947 0 0) 0x7f86800c7de0 con 0x7f868006a1c0 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8668003570 con 0x7f868006a1c0 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 484825879 0 0) 0x7f8668003570 con 0x7f868006a1c0 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 msgr2=0x7f8680069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8680069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 >> v2:172.21.15.191:3300/0 conn(0x7f8680068aa0 msgr2=0x7f86800c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 --2- 172.21.15.170:0/2968049163 >> v2:172.21.15.191:3300/0 conn(0x7f8680068aa0 0x7f86800c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680140ed0 con 0x7f868006a1c0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8670001470 con 0x7f868006a1c0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 >> v1:172.21.15.170:6789/0 conn(0x7f868006a1c0 legacy=0x7f8680068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8668003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668003fe0 0x7f8668004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8668004d50 0x7f86680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680140c10 con 0x7f8668004d50 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680141490 con 0x7f8668003fe0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866800ba90 con 0x7f8680140350 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8668004d50 0x7f86680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867ffff700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8668003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f867f7fe700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668003fe0 0x7f8668004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 -- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 msgr2=0x7f8668003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8680140350 0x7f8668003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 -- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668003fe0 msgr2=0x7f8668004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8668003fe0 0x7f8668004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 -- 172.21.15.170:0/2968049163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8668003570 con 0x7f8668004d50 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.194+0000 7f8684aac700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8668004d50 0x7f86680051c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8680068ea0 tx=0x7f867800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee796deb125ccdae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800ea30 con 0x7f8668004d50 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867800f040 con 0x7f8668004d50 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f867effd700 1 -- 172.21.15.170:0/2968049163 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867800ede0 con 0x7f8668004d50 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 -- 172.21.15.170:0/2968049163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8668004d50 msgr2=0x7f86680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 --2- 172.21.15.170:0/2968049163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8668004d50 0x7f86680051c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8680068ea0 tx=0x7f867800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:04.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f8684aac700 1 -- 172.21.15.170:0/2968049163 reap_dead start 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 -- 172.21.15.170:0/2968049163 shutdown_connections 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 -- 172.21.15.170:0/2968049163 >> 172.21.15.170:0/2968049163 conn(0x7f86800cd670 msgr2=0x7f86800cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 -- 172.21.15.170:0/2968049163 shutdown_connections 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.195+0000 7f868650f700 1 -- 172.21.15.170:0/2968049163 wait complete. 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.196+0000 7f868650f700 1 Processor -- start 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.196+0000 7f868650f700 1 -- start start 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86800c5930 0x7f86800c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86801656a0 0x7f86800b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800caf60 con 0x7f86800ca5a0 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800b1400 con 0x7f86800c5930 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f868650f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c4c60 con 0x7f86801656a0 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37202/0 (socket says 172.21.15.170:37202) 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 -- 172.21.15.170:0/3935258916 learned_addr learned my addr 172.21.15.170:0/3935258916 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f867f7fe700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86801656a0 0x7f86800b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f867ffff700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86800c5930 0x7f86800c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 -- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86801656a0 msgr2=0x7f86800b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86801656a0 0x7f86800b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 -- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86800c5930 msgr2=0x7f86800c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86800c5930 0x7f86800c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.197+0000 7f8684aac700 1 -- 172.21.15.170:0/3935258916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680165ba0 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f8684aac700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f867800afd0 tx=0x7f867801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867801d630 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867801d7b0 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867801db70 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8680165e80 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.198+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8680166440 con 0x7f86800ca5a0 2023-10-08T16:06:04.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.199+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680166630 con 0x7f86800ca5a0 2023-10-08T16:06:04.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.199+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8678003040 con 0x7f86800ca5a0 2023-10-08T16:06:04.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.200+0000 7f8667fff700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 0x7f866c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.200+0000 7f867ffff700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 0x7f866c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.200+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8678086450 con 0x7f86800ca5a0 2023-10-08T16:06:04.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.200+0000 7f867ffff700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 0x7f866c069470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8674006e50 tx=0x7f8674009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.204+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8680166630 con 0x7f86800ca5a0 2023-10-08T16:06:04.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.385+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86800b15b0 con 0x7f86800ca5a0 2023-10-08T16:06:04.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.385+0000 7f8667fff700 1 -- 172.21.15.170:0/3935258916 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86800b15b0 con 0x7f86800ca5a0 2023-10-08T16:06:04.386 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:06:04.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.387+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 msgr2=0x7f866c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.387+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 0x7f866c069470 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8674006e50 tx=0x7f8674009120 comp rx=0 tx=0).stop 2023-10-08T16:06:04.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 msgr2=0x7f86800caa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f867800afd0 tx=0x7f867801a000 comp rx=0 tx=0).stop 2023-10-08T16:06:04.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 shutdown_connections 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c066fc0 0x7f866c069470 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f86800ca5a0 0x7f86800caa20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f86800c5930 0x7f86800c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 --2- 172.21.15.170:0/3935258916 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f86801656a0 0x7f86800b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 >> 172.21.15.170:0/3935258916 conn(0x7f86800cd670 msgr2=0x7f86800c3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 shutdown_connections 2023-10-08T16:06:04.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.388+0000 7f868650f700 1 -- 172.21.15.170:0/3935258916 wait complete. 2023-10-08T16:06:04.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:04.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.515+0000 7efd81d7f700 1 Processor -- start 2023-10-08T16:06:04.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.515+0000 7efd81d7f700 1 -- start start 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd81d7f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efd7c15d230 0x7efd7c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd81d7f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c0b2980 con 0x7efd7c159420 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd81d7f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c15db70 con 0x7efd7c1597f0 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd81d7f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0b11d0 con 0x7efd7c15d230 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd8157e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efd7c15d230 0x7efd7c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd80d7d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efd7c159420 0x7efd7c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:60602/0 (socket says 172.21.15.170:60602) 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd80d7d700 1 -- 172.21.15.170:0/4042617409 learned_addr learned my addr 172.21.15.170:0/4042617409 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:04.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd8157e700 1 -- 172.21.15.170:0/4042617409 >> v1:172.21.15.191:6789/0 conn(0x7efd7c1597f0 legacy=0x7efd7c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd8157e700 1 -- 172.21.15.170:0/4042617409 >> v1:172.21.15.170:6789/0 conn(0x7efd7c159420 legacy=0x7efd7c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd8157e700 1 -- 172.21.15.170:0/4042617409 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0b14b0 con 0x7efd7c15d230 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.516+0000 7efd737fe700 1 -- 172.21.15.170:0/4042617409 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653033528 0 0) 0x7efd7c0b2980 con 0x7efd7c159420 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd8157e700 1 --2- 172.21.15.170:0/4042617409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c15d230 0x7efd7c15d630 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efd74009e60 tx=0x7efd74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2602700ebc0c79fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd737fe700 1 -- 172.21.15.170:0/4042617409 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400e070 con 0x7efd7c15d230 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd737fe700 1 -- 172.21.15.170:0/4042617409 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7400bbf0 con 0x7efd7c15d230 2023-10-08T16:06:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd737fe700 1 -- 172.21.15.170:0/4042617409 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74012430 con 0x7efd7c15d230 2023-10-08T16:06:04.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 -- 172.21.15.170:0/4042617409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c15d230 msgr2=0x7efd7c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 --2- 172.21.15.170:0/4042617409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c15d230 0x7efd7c15d630 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efd74009e60 tx=0x7efd74009780 comp rx=0 tx=0).stop 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 -- 172.21.15.170:0/4042617409 shutdown_connections 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 --2- 172.21.15.170:0/4042617409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c15d230 0x7efd7c15d630 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 -- 172.21.15.170:0/4042617409 >> 172.21.15.170:0/4042617409 conn(0x7efd7c0d0660 msgr2=0x7efd7c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 -- 172.21.15.170:0/4042617409 shutdown_connections 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.517+0000 7efd81d7f700 1 -- 172.21.15.170:0/4042617409 wait complete. 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.518+0000 7efd81d7f700 1 Processor -- start 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 -- start start 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd7c159420 0x7efd7c154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd7c15d230 0x7efd7c169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c14f840 con 0x7efd7c1597f0 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c15f1c0 con 0x7efd7c159420 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd81d7f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c15a1e0 con 0x7efd7c15d230 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd73fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd73fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37214/0 (socket says 172.21.15.170:37214) 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.519+0000 7efd73fff700 1 -- 172.21.15.170:0/3804120300 learned_addr learned my addr 172.21.15.170:0/3804120300 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd8157e700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd7c15d230 0x7efd7c169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd80d7d700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd7c159420 0x7efd7c154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 -- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd7c15d230 msgr2=0x7efd7c169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd7c15d230 0x7efd7c169770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 -- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd7c159420 msgr2=0x7efd7c154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd7c159420 0x7efd7c154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 -- 172.21.15.170:0/3804120300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009a20 con 0x7efd7c1597f0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd73fff700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b7e0 tx=0x7efd6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00f510 con 0x7efd7c1597f0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00f670 con 0x7efd7c1597f0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00fa10 con 0x7efd7c1597f0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c155420 con 0x7efd7c1597f0 2023-10-08T16:06:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.520+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c15eb70 con 0x7efd7c1597f0 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.521+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c15f330 con 0x7efd7c1597f0 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.521+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7efd6c01a400 con 0x7efd7c1597f0 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.522+0000 7efd70ff9700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 0x7efd68069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.522+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd6c084000 con 0x7efd7c1597f0 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.522+0000 7efd80d7d700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 0x7efd68069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.522+0000 7efd80d7d700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 0x7efd68069470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efd78007fd0 tx=0x7efd78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.526+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd6c051300 con 0x7efd7c1597f0 2023-10-08T16:06:04.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.748+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd7c0b1790 con 0x7efd7c1597f0 2023-10-08T16:06:04.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.749+0000 7efd70ff9700 1 -- 172.21.15.170:0/3804120300 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4600 (secure 0 0 0) 0x7efd6c054920 con 0x7efd7c1597f0 2023-10-08T16:06:04.749 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:04.749 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:06:04.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 msgr2=0x7efd68069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 0x7efd68069470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efd78007fd0 tx=0x7efd78009040 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 msgr2=0x7efd7c159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00b7e0 tx=0x7efd6c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 shutdown_connections 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd7c15d230 0x7efd7c169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd7c159420 0x7efd7c154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd68066fc0 0x7efd68069470 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 --2- 172.21.15.170:0/3804120300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd7c1597f0 0x7efd7c159ca0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 >> 172.21.15.170:0/3804120300 conn(0x7efd7c0d0660 msgr2=0x7efd7c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 shutdown_connections 2023-10-08T16:06:04.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.751+0000 7efd81d7f700 1 -- 172.21.15.170:0/3804120300 wait complete. 2023-10-08T16:06:04.753 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 31 2023-10-08T16:06:04.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:04.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.878+0000 7f26214f5700 1 Processor -- start 2023-10-08T16:06:04.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.878+0000 7f26214f5700 1 -- start start 2023-10-08T16:06:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26214f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c1597e0 0x7f261c06c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26214f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f261c06c050 0x7f261c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26214f5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c06bbc0 con 0x7f261c159bb0 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26214f5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c0c8660 con 0x7f261c06c050 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26214f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06ad10 con 0x7f261c1597e0 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.879+0000 7f26127fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f261c06c050 0x7f261c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f261c06c050 0x7f261c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37222/0 (socket says 172.21.15.170:37222) 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 -- 172.21.15.170:0/3607712566 learned_addr learned my addr 172.21.15.170:0/3607712566 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f261affd700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c1597e0 0x7f261c06c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 -- 172.21.15.170:0/3607712566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c1597e0 msgr2=0x7f261c06c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c1597e0 0x7f261c06c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 -- 172.21.15.170:0/3607712566 >> v1:172.21.15.191:6789/0 conn(0x7f261c159bb0 legacy=0x7f261c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 -- 172.21.15.170:0/3607712566 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c06afb0 con 0x7f261c06c050 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f261a7fc700 1 -- 172.21.15.170:0/3607712566 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393248358 0 0) 0x7f261c06bbc0 con 0x7f261c159bb0 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f26127fc700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c068390 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26040099b0 tx=0x7f2604009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ab4c4933916ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f261a7fc700 1 -- 172.21.15.170:0/3607712566 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604009ea0 con 0x7f261c06c050 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f261a7fc700 1 -- 172.21.15.170:0/3607712566 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400b860 con 0x7f261c06c050 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.880+0000 7f261a7fc700 1 -- 172.21.15.170:0/3607712566 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400bc00 con 0x7f261c06c050 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 -- 172.21.15.170:0/3607712566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 msgr2=0x7f261c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c068390 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f26040099b0 tx=0x7f2604009250 comp rx=0 tx=0).stop 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 -- 172.21.15.170:0/3607712566 shutdown_connections 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c1597e0 0x7f261c06c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 --2- 172.21.15.170:0/3607712566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c068390 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 -- 172.21.15.170:0/3607712566 >> 172.21.15.170:0/3607712566 conn(0x7f261c0cdd80 msgr2=0x7f261c0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:04.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 -- 172.21.15.170:0/3607712566 shutdown_connections 2023-10-08T16:06:04.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.881+0000 7f26214f5700 1 -- 172.21.15.170:0/3607712566 wait complete. 2023-10-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.882+0000 7f26214f5700 1 Processor -- start 2023-10-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.882+0000 7f26214f5700 1 -- start start 2023-10-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 0x7f261c0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 0x7f261c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c15baa0 con 0x7f261c06c050 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0c5610 con 0x7f261c1597e0 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c15b450 con 0x7f261c159bb0 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37234/0 (socket says 172.21.15.170:37234) 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 -- 172.21.15.170:0/3238065040 learned_addr learned my addr 172.21.15.170:0/3238065040 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261b7fe700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 0x7f261c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26127fc700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 0x7f261c0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 -- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 msgr2=0x7f261c152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 0x7f261c152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 -- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 msgr2=0x7f261c0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 0x7f261c0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 -- 172.21.15.170:0/3238065040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26040094f0 con 0x7f261c06c050 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261b7fe700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 0x7f261c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26127fc700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 0x7f261c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f261affd700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f260c009f10 tx=0x7f260c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00cc80 con 0x7f261c06c050 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00f040 con 0x7f261c06c050 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c15b730 con 0x7f261c06c050 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c014480 con 0x7f261c06c050 2023-10-08T16:06:04.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.883+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c0c7620 con 0x7f261c06c050 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.884+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c04c3c0 con 0x7f261c06c050 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.885+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f260c014830 con 0x7f261c06c050 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.885+0000 7f26137fe700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 0x7f25fc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.885+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f260c085160 con 0x7f261c06c050 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.885+0000 7f26127fc700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 0x7f25fc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.886+0000 7f26127fc700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 0x7f25fc069470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26040099b0 tx=0x7f2604017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:04.889+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f260c0522d0 con 0x7f261c06c050 2023-10-08T16:06:05.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.076+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f261c06b290 con 0x7f261c06c050 2023-10-08T16:06:05.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.077+0000 7f26137fe700 1 -- 172.21.15.170:0/3238065040 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f260c0558f0 con 0x7f261c06c050 2023-10-08T16:06:05.078 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:05.078 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":51,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:53.008429+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:05.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.079+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 msgr2=0x7f25fc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.079+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 0x7f25fc069470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f26040099b0 tx=0x7f2604017000 comp rx=0 tx=0).stop 2023-10-08T16:06:05.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.079+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 msgr2=0x7f261c0c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.079+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f260c009f10 tx=0x7f260c00b040 comp rx=0 tx=0).stop 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 shutdown_connections 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f261c159bb0 0x7f261c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25fc066fc0 0x7f25fc069470 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261c06c050 0x7f261c0c70e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 --2- 172.21.15.170:0/3238065040 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261c1597e0 0x7f261c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 >> 172.21.15.170:0/3238065040 conn(0x7f261c0cdd80 msgr2=0x7f261c0c2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 shutdown_connections 2023-10-08T16:06:05.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.080+0000 7f26214f5700 1 -- 172.21.15.170:0/3238065040 wait complete. 2023-10-08T16:06:05.092 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:06:05.092 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:05.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.206+0000 7f419759e700 1 Processor -- start 2023-10-08T16:06:05.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.206+0000 7f419759e700 1 -- start start 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4198159420 0x7f41980b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f419815d230 0x7f419815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419759e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41980b2980 con 0x7f41981597f0 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419815dbb0 con 0x7f4198159420 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41980b11d0 con 0x7f419815d230 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4198159420 0x7f41980b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f418dd9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f41981597f0 0x7f41980b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58120/0 (socket says 172.21.15.170:58120) 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f418dd9b700 1 -- 172.21.15.170:0/2178869771 learned_addr learned my addr 172.21.15.170:0/2178869771 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f4196d9d700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f419815d230 0x7f419815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419659c700 1 -- 172.21.15.170:0/2178869771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f419815d230 msgr2=0x7f419815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419659c700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f419815d230 0x7f419815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419659c700 1 -- 172.21.15.170:0/2178869771 >> v1:172.21.15.170:6789/0 conn(0x7f41981597f0 legacy=0x7f41980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f419659c700 1 -- 172.21.15.170:0/2178869771 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980b14b0 con 0x7f4198159420 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.207+0000 7f4195d9b700 1 -- 172.21.15.170:0/2178869771 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376277760 0 0) 0x7f41980b2980 con 0x7f41981597f0 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419659c700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4198159420 0x7f41980b5f40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41880098e0 tx=0x7f41880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8781869e73565bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f4195d9b700 1 -- 172.21.15.170:0/2178869771 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188009e80 con 0x7f4198159420 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f4195d9b700 1 -- 172.21.15.170:0/2178869771 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418800b860 con 0x7f4198159420 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f4195d9b700 1 -- 172.21.15.170:0/2178869771 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800bbe0 con 0x7f4198159420 2023-10-08T16:06:05.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 -- 172.21.15.170:0/2178869771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4198159420 msgr2=0x7f41980b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4198159420 0x7f41980b5f40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41880098e0 tx=0x7f41880091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 -- 172.21.15.170:0/2178869771 shutdown_connections 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f419815d230 0x7f419815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 --2- 172.21.15.170:0/2178869771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4198159420 0x7f41980b5f40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 -- 172.21.15.170:0/2178869771 >> 172.21.15.170:0/2178869771 conn(0x7f41980d0660 msgr2=0x7f41980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 -- 172.21.15.170:0/2178869771 shutdown_connections 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.208+0000 7f419759e700 1 -- 172.21.15.170:0/2178869771 wait complete. 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.209+0000 7f419759e700 1 Processor -- start 2023-10-08T16:06:05.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 -- start start 2023-10-08T16:06:05.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 0x7f4198141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419815d230 0x7f41981539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198141640 con 0x7f41981597f0 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41981545f0 con 0x7f419815d230 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4198154030 con 0x7f4198159420 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f418dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f418dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44664/0 (socket says 172.21.15.170:44664) 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f418dd9b700 1 -- 172.21.15.170:0/151735403 learned_addr learned my addr 172.21.15.170:0/151735403 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.210+0000 7f419659c700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 0x7f4198141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f4196d9d700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419815d230 0x7f41981539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 -- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 msgr2=0x7f4198141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 0x7f4198141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 -- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419815d230 msgr2=0x7f41981539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419815d230 0x7f41981539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 -- 172.21.15.170:0/151735403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188009420 con 0x7f41981597f0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f419659c700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 0x7f4198141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418dd9b700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f418000c840 tx=0x7f4180007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180005b30 con 0x7f41981597f0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4180005c90 con 0x7f41981597f0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4198154310 con 0x7f41981597f0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180007320 con 0x7f41981597f0 2023-10-08T16:06:05.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.211+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419815a0b0 con 0x7f41981597f0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.212+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419804c3c0 con 0x7f41981597f0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.212+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f418001b3b0 con 0x7f41981597f0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.213+0000 7f418effd700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 0x7f4178069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.213+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4180083b00 con 0x7f41981597f0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.213+0000 7f419659c700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 0x7f4178069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.213+0000 7f419659c700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 0x7f4178069470 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41880091e0 tx=0x7f4188017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.217+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4180050d80 con 0x7f41981597f0 2023-10-08T16:06:05.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.440+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41980b1790 con 0x7f41981597f0 2023-10-08T16:06:05.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.440+0000 7f418effd700 1 -- 172.21.15.170:0/151735403 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f4180017020 con 0x7f41981597f0 2023-10-08T16:06:05.442 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:05.442 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:06:05.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 msgr2=0x7f4178069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 0x7f4178069470 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f41880091e0 tx=0x7f4188017000 comp rx=0 tx=0).stop 2023-10-08T16:06:05.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 msgr2=0x7f4198153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f418000c840 tx=0x7f4180007db0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 shutdown_connections 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f419815d230 0x7f41981539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4198159420 0x7f4198141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4178066fc0 0x7f4178069470 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 --2- 172.21.15.170:0/151735403 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41981597f0 0x7f4198153490 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.442+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 >> 172.21.15.170:0/151735403 conn(0x7f41980d0660 msgr2=0x7f41980ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.443+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 shutdown_connections 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.443+0000 7f419759e700 1 -- 172.21.15.170:0/151735403 wait complete. 2023-10-08T16:06:05.445 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 31 2023-10-08T16:06:05.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:05.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.570+0000 7f48d13e4700 1 Processor -- start 2023-10-08T16:06:05.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.570+0000 7f48d13e4700 1 -- start start 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc140350 0x7f48cc069c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f48cc06a1c0 0x7f48cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48cc140720 0x7f48cc0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0c7de0 con 0x7f48cc140720 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc140c10 con 0x7f48cc06a1c0 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48d13e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc141490 con 0x7f48cc140350 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48cc140720 0x7f48cc0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f48cc140720 0x7f48cc0688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44680/0 (socket says 172.21.15.170:44680) 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 -- 172.21.15.170:0/226352130 learned_addr learned my addr 172.21.15.170:0/226352130 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48caffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc140350 0x7f48cc069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48ca7fc700 1 --2- 172.21.15.170:0/226352130 >> v2:172.21.15.191:3300/0 conn(0x7f48cc06a1c0 0x7f48cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 -- 172.21.15.170:0/226352130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc140350 msgr2=0x7f48cc069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/226352130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc140350 0x7f48cc069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 -- 172.21.15.170:0/226352130 >> v2:172.21.15.191:3300/0 conn(0x7f48cc06a1c0 msgr2=0x7f48cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/226352130 >> v2:172.21.15.191:3300/0 conn(0x7f48cc06a1c0 0x7f48cc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 -- 172.21.15.170:0/226352130 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc140ed0 con 0x7f48cc140720 2023-10-08T16:06:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/226352130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0688d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f48cc06e0a0 tx=0x7f48c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda44991e326d083 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48c9ffb700 1 -- 172.21.15.170:0/226352130 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c400e070 con 0x7f48cc140720 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48c9ffb700 1 -- 172.21.15.170:0/226352130 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c400b970 con 0x7f48cc140720 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.571+0000 7f48c9ffb700 1 -- 172.21.15.170:0/226352130 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40092e0 con 0x7f48cc140720 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 -- 172.21.15.170:0/226352130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 msgr2=0x7f48cc0688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 --2- 172.21.15.170:0/226352130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0688d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f48cc06e0a0 tx=0x7f48c4009840 comp rx=0 tx=0).stop 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 -- 172.21.15.170:0/226352130 shutdown_connections 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 --2- 172.21.15.170:0/226352130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0688d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 --2- 172.21.15.170:0/226352130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc140350 0x7f48cc069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 --2- 172.21.15.170:0/226352130 >> v2:172.21.15.191:3300/0 conn(0x7f48cc06a1c0 0x7f48cc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 -- 172.21.15.170:0/226352130 >> 172.21.15.170:0/226352130 conn(0x7f48cc0cd670 msgr2=0x7f48cc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 -- 172.21.15.170:0/226352130 shutdown_connections 2023-10-08T16:06:05.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.572+0000 7f48d13e4700 1 -- 172.21.15.170:0/226352130 wait complete. 2023-10-08T16:06:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.573+0000 7f48d13e4700 1 Processor -- start 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.573+0000 7f48d13e4700 1 -- start start 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc06a1c0 0x7f48cc0c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48cc140350 0x7f48cc0b13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc159c50 con 0x7f48cc140720 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0b2dc0 con 0x7f48cc140350 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48d13e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc159040 con 0x7f48cc06a1c0 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44686/0 (socket says 172.21.15.170:44686) 2023-10-08T16:06:05.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 -- 172.21.15.170:0/3402795458 learned_addr learned my addr 172.21.15.170:0/3402795458 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48caffd700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc06a1c0 0x7f48cc0c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48ca7fc700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48cc140350 0x7f48cc0b13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 -- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc06a1c0 msgr2=0x7f48cc0c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc06a1c0 0x7f48cc0c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 -- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48cc140350 msgr2=0x7f48cc0b13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48cc140350 0x7f48cc0b13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 -- 172.21.15.170:0/3402795458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c4009ae0 con 0x7f48cc140720 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.574+0000 7f48cb7fe700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009880 tx=0x7f48c4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.575+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4008b10 con 0x7f48cc140720 2023-10-08T16:06:05.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.575+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c4008c70 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.575+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc159320 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.575+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c4025460 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.575+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc1598e0 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.576+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc04c3c0 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.576+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f48c4003040 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.576+0000 7f48b2ffd700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 0x7f48b8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.576+0000 7f48caffd700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 0x7f48b8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.576+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48c40862a0 con 0x7f48cc140720 2023-10-08T16:06:05.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.577+0000 7f48caffd700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 0x7f48b8069470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009da0 tx=0x7f48bc009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.581+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48c4053310 con 0x7f48cc140720 2023-10-08T16:06:05.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.809+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48cc1411b0 con 0x7f48cc140720 2023-10-08T16:06:05.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.809+0000 7f48b2ffd700 1 -- 172.21.15.170:0/3402795458 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f48c4056930 con 0x7f48cc140720 2023-10-08T16:06:05.811 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:05.812 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:05:51.383446+0000","modified":"2023-10-08T16:05:55.018929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5304},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5304":{"gid":5304,"name":"d","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/2304240445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2304240445},{"type":"v1","addr":"172.21.15.191:6834","nonce":2304240445}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T16:06:05.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.811+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 msgr2=0x7f48b8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.811+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 0x7f48b8069470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48bc009da0 tx=0x7f48bc009120 comp rx=0 tx=0).stop 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.811+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 msgr2=0x7f48cc0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.811+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009880 tx=0x7f48c4018000 comp rx=0 tx=0).stop 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 shutdown_connections 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48b8066fc0 0x7f48b8069470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48cc140720 0x7f48cc0b1920 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48cc06a1c0 0x7f48cc0c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 --2- 172.21.15.170:0/3402795458 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f48cc140350 0x7f48cc0b13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 >> 172.21.15.170:0/3402795458 conn(0x7f48cc0cd670 msgr2=0x7f48cc0c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 shutdown_connections 2023-10-08T16:06:05.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.812+0000 7f48d13e4700 1 -- 172.21.15.170:0/3402795458 wait complete. 2023-10-08T16:06:05.816 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 31 2023-10-08T16:06:05.825 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.938+0000 7fa111d86700 1 Processor -- start 2023-10-08T16:06:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.938+0000 7fa111d86700 1 -- start start 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa111d86700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa10c159890 0x7fa10c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa111d86700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa111d86700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10c15a160 con 0x7fa10c15db10 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa111d86700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10c0d13a0 con 0x7fa10c159890 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa111d86700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c15e140 con 0x7fa10c159c60 2023-10-08T16:06:05.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa10b7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa10c159890 0x7fa10c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa10affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa10bfff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fa10c15db10 0x7fa10c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54410/0 (socket says 172.21.15.170:54410) 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa10bfff700 1 -- 172.21.15.170:0/2660268856 learned_addr learned my addr 172.21.15.170:0/2660268856 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.939+0000 7fa10affd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:48252/0 (socket says 172.21.15.170:48252) 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10b7fe700 1 -- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 msgr2=0x7fa10c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:05.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10b7fe700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10b7fe700 1 -- 172.21.15.170:0/2660268856 >> v1:172.21.15.191:6789/0 conn(0x7fa10c15db10 legacy=0x7fa10c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10a7fc700 1 -- 172.21.15.170:0/2660268856 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991649795 0 0) 0x7fa10c15a160 con 0x7fa10c15db10 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10b7fe700 1 -- 172.21.15.170:0/2660268856 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c15a3d0 con 0x7fa10c159890 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10affd700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10b7fe700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159890 0x7fa10c15abd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0098e0 tx=0x7fa0fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74d20881bf12bba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10a7fc700 1 -- 172.21.15.170:0/2660268856 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00b640 con 0x7fa10c159890 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10a7fc700 1 -- 172.21.15.170:0/2660268856 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00b7a0 con 0x7fa10c159890 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.940+0000 7fa10a7fc700 1 -- 172.21.15.170:0/2660268856 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00bb40 con 0x7fa10c159890 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 -- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159890 msgr2=0x7fa10c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159890 0x7fa10c15abd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0098e0 tx=0x7fa0fc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 -- 172.21.15.170:0/2660268856 shutdown_connections 2023-10-08T16:06:05.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159890 0x7fa10c15abd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 --2- 172.21.15.170:0/2660268856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c159c60 0x7fa10c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 -- 172.21.15.170:0/2660268856 >> 172.21.15.170:0/2660268856 conn(0x7fa10c068e20 msgr2=0x7fa10c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:05.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 -- 172.21.15.170:0/2660268856 shutdown_connections 2023-10-08T16:06:05.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.941+0000 7fa111d86700 1 -- 172.21.15.170:0/2660268856 wait complete. 2023-10-08T16:06:05.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.942+0000 7fa111d86700 1 Processor -- start 2023-10-08T16:06:05.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.942+0000 7fa111d86700 1 -- start start 2023-10-08T16:06:05.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 0x7fa10c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 0x7fa10c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0b53a0 con 0x7fa10c159890 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0b49d0 con 0x7fa10c159c60 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c0b3dc0 con 0x7fa10c15db10 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44690/0 (socket says 172.21.15.170:44690) 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 -- 172.21.15.170:0/1137437318 learned_addr learned my addr 172.21.15.170:0/1137437318 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10bfff700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 0x7fa10c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10affd700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 0x7fa10c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 -- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 msgr2=0x7fa10c152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 0x7fa10c152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 -- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 msgr2=0x7fa10c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 0x7fa10c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 -- 172.21.15.170:0/1137437318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009420 con 0x7fa10c159890 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10bfff700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 0x7fa10c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10affd700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 0x7fa10c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa10b7fe700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009f80 tx=0x7fa0fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00bb20 con 0x7fa10c159890 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc011420 con 0x7fa10c159890 2023-10-08T16:06:05.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc009420 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc011800 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.943+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10c0b4380 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.944+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10c04c3c0 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.945+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fa0fc011c30 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.945+0000 7fa0f37fe700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 0x7fa0f8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.945+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa0fc086180 con 0x7fa10c159890 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.945+0000 7fa10affd700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 0x7fa0f8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.946+0000 7fa10affd700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 0x7fa0f8069470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1000093f0 tx=0x7fa100008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:05.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:05.949+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0fc053400 con 0x7fa10c159890 2023-10-08T16:06:06.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.135+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa10c06a5b0 con 0x7fa10c159890 2023-10-08T16:06:06.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.136+0000 7fa0f37fe700 1 -- 172.21.15.170:0/1137437318 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fa0fc056a20 con 0x7fa10c159890 2023-10-08T16:06:06.138 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:06.138 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":51,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:05:53.008429+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:05:49.716147+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T16:05:50.344843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 msgr2=0x7fa0f8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 0x7fa0f8069470 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa1000093f0 tx=0x7fa100008040 comp rx=0 tx=0).stop 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 msgr2=0x7fa10c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009f80 tx=0x7fa0fc017000 comp rx=0 tx=0).stop 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 shutdown_connections 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa10c15db10 0x7fa10c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f8066fc0 0x7fa0f8069470 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa10c159890 0x7fa10c0b4e60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 --2- 172.21.15.170:0/1137437318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa10c159c60 0x7fa10c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 >> 172.21.15.170:0/1137437318 conn(0x7fa10c068e20 msgr2=0x7fa10c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 shutdown_connections 2023-10-08T16:06:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.138+0000 7fa111d86700 1 -- 172.21.15.170:0/1137437318 wait complete. 2023-10-08T16:06:06.152 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:06:06.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.266+0000 7f28955e5700 1 Processor -- start 2023-10-08T16:06:06.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.266+0000 7f28955e5700 1 -- start start 2023-10-08T16:06:06.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f289006c0a0 0x7f289015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f289015df50 0x7f289015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289015d7e0 con 0x7f289006c0a0 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289015dac0 con 0x7f289015df50 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28955e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289015aa30 con 0x7f289006bcd0 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28867fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f289006c0a0 0x7f289015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28867fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f289006c0a0 0x7f289015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44698/0 (socket says 172.21.15.170:44698) 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.267+0000 7f28867fc700 1 -- 172.21.15.170:0/3666761114 learned_addr learned my addr 172.21.15.170:0/3666761114 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f288effd700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f288f7fe700 1 --2- 172.21.15.170:0/3666761114 >> v2:172.21.15.191:3300/0 conn(0x7f289015df50 0x7f289015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 -- 172.21.15.170:0/3666761114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 msgr2=0x7f28900caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 -- 172.21.15.170:0/3666761114 >> v2:172.21.15.191:3300/0 conn(0x7f289015df50 msgr2=0x7f289015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 --2- 172.21.15.170:0/3666761114 >> v2:172.21.15.191:3300/0 conn(0x7f289015df50 0x7f289015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 -- 172.21.15.170:0/3666761114 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289015ad10 con 0x7f289006c0a0 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f28867fc700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289006c0a0 0x7f289015a1e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f289006b620 tx=0x7f2878009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8351357a03d93582 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f288e7fc700 1 -- 172.21.15.170:0/3666761114 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287800b640 con 0x7f289006c0a0 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f288e7fc700 1 -- 172.21.15.170:0/3666761114 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287800b7a0 con 0x7f289006c0a0 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.268+0000 7f288e7fc700 1 -- 172.21.15.170:0/3666761114 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287800bb40 con 0x7f289006c0a0 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 -- 172.21.15.170:0/3666761114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289006c0a0 msgr2=0x7f289015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289006c0a0 0x7f289015a1e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f289006b620 tx=0x7f2878009250 comp rx=0 tx=0).stop 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 -- 172.21.15.170:0/3666761114 shutdown_connections 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 --2- 172.21.15.170:0/3666761114 >> v2:172.21.15.191:3300/0 conn(0x7f289015df50 0x7f289015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 --2- 172.21.15.170:0/3666761114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289006c0a0 0x7f289015a1e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 -- 172.21.15.170:0/3666761114 >> 172.21.15.170:0/3666761114 conn(0x7f28900ce600 msgr2=0x7f28900cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 -- 172.21.15.170:0/3666761114 shutdown_connections 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.269+0000 7f28955e5700 1 -- 172.21.15.170:0/3666761114 wait complete. 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.270+0000 7f28955e5700 1 Processor -- start 2023-10-08T16:06:06.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.270+0000 7f28955e5700 1 -- start start 2023-10-08T16:06:06.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 0x7f2890157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890158630 con 0x7f289015df50 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890158910 con 0x7f289006c0a0 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2890158bf0 con 0x7f289006bcd0 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44710/0 (socket says 172.21.15.170:44710) 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 -- 172.21.15.170:0/4104585119 learned_addr learned my addr 172.21.15.170:0/4104585119 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28867fc700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 0x7f2890157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288effd700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 -- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 msgr2=0x7f28900b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 -- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 msgr2=0x7f2890157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 0x7f2890157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 -- 172.21.15.170:0/4104585119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28780094f0 con 0x7f289015df50 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28867fc700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 0x7f2890157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288effd700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f288f7fe700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2888009e90 tx=0x7f288800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288800cc20 con 0x7f289015df50 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288800f040 con 0x7f289015df50 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.271+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2890158ed0 con 0x7f289015df50 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.272+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888014480 con 0x7f289015df50 2023-10-08T16:06:06.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.272+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2890158320 con 0x7f289015df50 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.272+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28900c6510 con 0x7f289015df50 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.273+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f2888014830 con 0x7f289015df50 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.273+0000 7f28877fe700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 0x7f2870069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.273+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2888085030 con 0x7f289015df50 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.273+0000 7f288effd700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 0x7f2870069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.274+0000 7f288effd700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 0x7f2870069470 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2880006f00 tx=0x7f2880008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.277+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28900c6510 con 0x7f289015df50 2023-10-08T16:06:06.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.503+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f28900c6510 con 0x7f289015df50 2023-10-08T16:06:06.505 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:06:06.504+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:06:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.507+0000 7f28877fe700 1 -- 172.21.15.170:0/4104585119 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7f28900c6510 con 0x7f289015df50 2023-10-08T16:06:06.510 INFO:tasks.ceph.mds.d.smithi191.stderr: -1764> 2023-10-08T16:05:57.301+0000 7f7e2f43f700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:06:06.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 msgr2=0x7f2870069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 0x7f2870069470 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2880006f00 tx=0x7f2880008040 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 msgr2=0x7f2890156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2888009e90 tx=0x7f288800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 shutdown_connections 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2870066fc0 0x7f2870069470 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f289015df50 0x7f2890156350 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f289006bcd0 0x7f28900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 --2- 172.21.15.170:0/4104585119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f289006c0a0 0x7f2890157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 >> 172.21.15.170:0/4104585119 conn(0x7f28900ce600 msgr2=0x7f28900c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:06.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 shutdown_connections 2023-10-08T16:06:06.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.510+0000 7f28955e5700 1 -- 172.21.15.170:0/4104585119 wait complete. 2023-10-08T16:06:06.513 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:06:06.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:06:06.534 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:06:06.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.639+0000 7fdb00aea700 1 Processor -- start 2023-10-08T16:06:06.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.639+0000 7fdb00aea700 1 -- start start 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdb00aea700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdafc159890 0x7fdafc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdb00aea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 0x7fdafc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdb00aea700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdafc069ee0 con 0x7fdafc159c60 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdb00aea700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdafc15a160 con 0x7fdafc159890 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdb00aea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc0d13a0 con 0x7fdafc069560 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 0x7fdafc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafa59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdafc159890 0x7fdafc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdaf9d9b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fdafc159c60 0x7fdafc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54420/0 (socket says 172.21.15.170:54420) 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdaf9d9b700 1 -- 172.21.15.170:0/3988615747 learned_addr learned my addr 172.21.15.170:0/3988615747 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 -- 172.21.15.170:0/3988615747 >> v2:172.21.15.191:3300/0 conn(0x7fdafc159890 msgr2=0x7fdafc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 --2- 172.21.15.170:0/3988615747 >> v2:172.21.15.191:3300/0 conn(0x7fdafc159890 0x7fdafc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 -- 172.21.15.170:0/3988615747 >> v1:172.21.15.191:6789/0 conn(0x7fdafc159c60 legacy=0x7fdafc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 -- 172.21.15.170:0/3988615747 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc15e140 con 0x7fdafc069560 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.640+0000 7fdafad9d700 1 --2- 172.21.15.170:0/3988615747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 0x7fdafc0699a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009e60 tx=0x7fdaf4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936816175f66cc0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdaf959a700 1 -- 172.21.15.170:0/3988615747 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400e070 con 0x7fdafc069560 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdaf959a700 1 -- 172.21.15.170:0/3988615747 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf400bb60 con 0x7fdafc069560 2023-10-08T16:06:06.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdaf959a700 1 -- 172.21.15.170:0/3988615747 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4012430 con 0x7fdafc069560 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 -- 172.21.15.170:0/3988615747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 msgr2=0x7fdafc0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 --2- 172.21.15.170:0/3988615747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 0x7fdafc0699a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009e60 tx=0x7fdaf4009780 comp rx=0 tx=0).stop 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 -- 172.21.15.170:0/3988615747 shutdown_connections 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 --2- 172.21.15.170:0/3988615747 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc069560 0x7fdafc0699a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 --2- 172.21.15.170:0/3988615747 >> v2:172.21.15.191:3300/0 conn(0x7fdafc159890 0x7fdafc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 -- 172.21.15.170:0/3988615747 >> 172.21.15.170:0/3988615747 conn(0x7fdafc068e20 msgr2=0x7fdafc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 -- 172.21.15.170:0/3988615747 shutdown_connections 2023-10-08T16:06:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.641+0000 7fdb00aea700 1 -- 172.21.15.170:0/3988615747 wait complete. 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.642+0000 7fdb00aea700 1 Processor -- start 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.642+0000 7fdb00aea700 1 -- start start 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdafc069560 0x7fdafc0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 0x7fdafc169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1558e0 con 0x7fdafc159890 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc0c5630 con 0x7fdafc069560 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdb00aea700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc0c5910 con 0x7fdafc159c60 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdaf9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdaf9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44726/0 (socket says 172.21.15.170:44726) 2023-10-08T16:06:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdaf9d9b700 1 -- 172.21.15.170:0/2389950643 learned_addr learned my addr 172.21.15.170:0/2389950643 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdafa59c700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdafc069560 0x7fdafc0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.643+0000 7fdafad9d700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 0x7fdafc169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 -- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 msgr2=0x7fdafc169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 0x7fdafc169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 -- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdafc069560 msgr2=0x7fdafc0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdafc069560 0x7fdafc0cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 -- 172.21.15.170:0/2389950643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4009a20 con 0x7fdafc159890 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdafad9d700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 0x7fdafc169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaf9d9b700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c7c0 tx=0x7fdaf0007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005b30 con 0x7fdafc159890 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0005c90 con 0x7fdafc159890 2023-10-08T16:06:06.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc0c5bf0 con 0x7fdafc159890 2023-10-08T16:06:06.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004a80 con 0x7fdafc159890 2023-10-08T16:06:06.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.644+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc0ce080 con 0x7fdafc159890 2023-10-08T16:06:06.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.645+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc0c6110 con 0x7fdafc159890 2023-10-08T16:06:06.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.645+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fdaf00070d0 con 0x7fdafc159890 2023-10-08T16:06:06.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.646+0000 7fdaea7fc700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 0x7fdadc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:06.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.646+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fdaf00839a0 con 0x7fdafc159890 2023-10-08T16:06:06.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.646+0000 7fdafa59c700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 0x7fdadc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:06.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.646+0000 7fdafa59c700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 0x7fdadc069470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009db0 tx=0x7fdaec0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:06.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.650+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdaf004b5a0 con 0x7fdafc159890 2023-10-08T16:06:06.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:06.874+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdafc0ce270 con 0x7fdafc159890 2023-10-08T16:06:07.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.508+0000 7fdaea7fc700 1 -- 172.21.15.170:0/2389950643 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdafc0c6110 con 0x7fdafc159890 2023-10-08T16:06:07.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 msgr2=0x7fdadc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 0x7fdadc069470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009db0 tx=0x7fdaec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 msgr2=0x7fdafc150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000c7c0 tx=0x7fdaf0007f70 comp rx=0 tx=0).stop 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 shutdown_connections 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdafc159c60 0x7fdafc169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdafc069560 0x7fdafc0cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc066fc0 0x7fdadc069470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 --2- 172.21.15.170:0/2389950643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdafc159890 0x7fdafc150000 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 >> 172.21.15.170:0/2389950643 conn(0x7fdafc068e20 msgr2=0x7fdafc157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 shutdown_connections 2023-10-08T16:06:07.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.511+0000 7fdb00aea700 1 -- 172.21.15.170:0/2389950643 wait complete. 2023-10-08T16:06:07.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:06:07.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.641+0000 7f8ccf82a700 1 Processor -- start 2023-10-08T16:06:07.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccf82a700 1 -- start start 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccf82a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc80b55a0 0x7f8cc8164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccf82a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80cfa60 con 0x7f8cc80cd910 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccf82a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc80b5b00 con 0x7f8cc8159ec0 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccf82a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc80b5da0 con 0x7f8cc80b55a0 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.642+0000 7f8ccddc7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc80b55a0 0x7f8cc8164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:07.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cccdc5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8cc80cd910 0x7f8cc80ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58126/0 (socket says 172.21.15.170:58126) 2023-10-08T16:06:07.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cccdc5700 1 -- 172.21.15.170:0/1887119388 learned_addr learned my addr 172.21.15.170:0/1887119388 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:07.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8ccddc7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc80b55a0 0x7f8cc8164400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44728/0 (socket says 172.21.15.170:44728) 2023-10-08T16:06:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1887119388 >> v1:172.21.15.191:6789/0 conn(0x7f8cc8159ec0 legacy=0x7f8cc80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1887119388 >> v1:172.21.15.170:6789/0 conn(0x7f8cc80cd910 legacy=0x7f8cc80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1887119388 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc80b6080 con 0x7f8cc80b55a0 2023-10-08T16:06:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cc7fff700 1 -- 172.21.15.170:0/1887119388 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254847785 0 0) 0x7f8cc80cfa60 con 0x7f8cc80cd910 2023-10-08T16:06:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8ccddc7700 1 --2- 172.21.15.170:0/1887119388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc80b55a0 0x7f8cc8164400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009e90 tx=0x7f8cb8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12a5c9203a0ced5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cc7fff700 1 -- 172.21.15.170:0/1887119388 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800e070 con 0x7f8cc80b55a0 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cc7fff700 1 -- 172.21.15.170:0/1887119388 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800bbb0 con 0x7f8cc80b55a0 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.643+0000 7f8cc7fff700 1 -- 172.21.15.170:0/1887119388 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8012430 con 0x7f8cc80b55a0 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1887119388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc80b55a0 msgr2=0x7f8cc8164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1887119388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc80b55a0 0x7f8cc8164400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009e90 tx=0x7f8cb8009780 comp rx=0 tx=0).stop 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1887119388 shutdown_connections 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1887119388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc80b55a0 0x7f8cc8164400 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1887119388 >> 172.21.15.170:0/1887119388 conn(0x7f8cc80999f0 msgr2=0x7f8cc80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1887119388 shutdown_connections 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.644+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1887119388 wait complete. 2023-10-08T16:06:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.645+0000 7f8ccf82a700 1 Processor -- start 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.645+0000 7f8ccf82a700 1 -- start start 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc80b55a0 0x7f8cc8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 0x7f8cc809dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc816d0d0 con 0x7f8cc816ddc0 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc816d930 con 0x7f8cc8159ec0 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccf82a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc809e320 con 0x7f8cc80b55a0 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44734/0 (socket says 172.21.15.170:44734) 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1403824445 learned_addr learned my addr 172.21.15.170:0/1403824445 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:07.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8cccdc5700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 0x7f8cc809dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccd5c6700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc80b55a0 0x7f8cc8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc80b55a0 msgr2=0x7f8cc8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc80b55a0 0x7f8cc8157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 msgr2=0x7f8cc809dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 0x7f8cc809dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 -- 172.21.15.170:0/1403824445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb8009a20 con 0x7f8cc816ddc0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8cccdc5700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 0x7f8cc809dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.646+0000 7f8ccddc7700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80099f0 tx=0x7f8cb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.647+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb801d710 con 0x7f8cc816ddc0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.647+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb801d890 con 0x7f8cc816ddc0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.647+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc8157960 con 0x7f8cc816ddc0 2023-10-08T16:06:07.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.647+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb801dc50 con 0x7f8cc816ddc0 2023-10-08T16:06:07.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.647+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc816d5c0 con 0x7f8cc816ddc0 2023-10-08T16:06:07.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.648+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc8063c90 con 0x7f8cc816ddc0 2023-10-08T16:06:07.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.648+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8cb8024070 con 0x7f8cc816ddc0 2023-10-08T16:06:07.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.648+0000 7f8cc57fa700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 0x7f8cb4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:07.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.648+0000 7f8ccd5c6700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 0x7f8cb4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:07.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.649+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8cb80870a0 con 0x7f8cc816ddc0 2023-10-08T16:06:07.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.649+0000 7f8ccd5c6700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 0x7f8cb4069470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00038a0 tx=0x7f8cc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:07.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.653+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cb80542d0 con 0x7f8cc816ddc0 2023-10-08T16:06:07.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:07.835+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8cc80718b0 con 0x7f8cc816ddc0 2023-10-08T16:06:08.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.510+0000 7f8cc57fa700 1 -- 172.21.15.170:0/1403824445 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8cb801ddb0 con 0x7f8cc816ddc0 2023-10-08T16:06:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 msgr2=0x7f8cb4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 0x7f8cb4069470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00038a0 tx=0x7f8cc0009040 comp rx=0 tx=0).stop 2023-10-08T16:06:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 msgr2=0x7f8cc81537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80099f0 tx=0x7f8cb8018000 comp rx=0 tx=0).stop 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 shutdown_connections 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cb4066fc0 0x7f8cb4069470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc816ddc0 0x7f8cc81537a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc80b55a0 0x7f8cc8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 --2- 172.21.15.170:0/1403824445 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8159ec0 0x7f8cc809dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 >> 172.21.15.170:0/1403824445 conn(0x7f8cc80999f0 msgr2=0x7f8cc80c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 shutdown_connections 2023-10-08T16:06:08.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.513+0000 7f8ccf82a700 1 -- 172.21.15.170:0/1403824445 wait complete. 2023-10-08T16:06:08.517 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:06:08.526 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:06:08.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.643+0000 7f3588f6a700 1 Processor -- start 2023-10-08T16:06:08.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.643+0000 7f3588f6a700 1 -- start start 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3588f6a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35840b55a0 0x7f3584164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3588f6a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35840cfa60 con 0x7f35840cd910 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3588f6a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35840b5b00 con 0x7f3584159ec0 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3588f6a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35840b5da0 con 0x7f35840b55a0 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3582d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35840b55a0 0x7f3584164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3579d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f35840cd910 0x7f35840ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58136/0 (socket says 172.21.15.170:58136) 2023-10-08T16:06:08.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3582d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f35840b55a0 0x7f3584164400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44746/0 (socket says 172.21.15.170:44746) 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3579d9b700 1 -- 172.21.15.170:0/2551095854 learned_addr learned my addr 172.21.15.170:0/2551095854 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3582d9d700 1 -- 172.21.15.170:0/2551095854 >> v1:172.21.15.191:6789/0 conn(0x7f3584159ec0 legacy=0x7f35840b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.644+0000 7f3582d9d700 1 -- 172.21.15.170:0/2551095854 >> v1:172.21.15.170:6789/0 conn(0x7f35840cd910 legacy=0x7f35840ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3582d9d700 1 -- 172.21.15.170:0/2551095854 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35840b6080 con 0x7f35840b55a0 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3581d9b700 1 -- 172.21.15.170:0/2551095854 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325827213 0 0) 0x7f35840cfa60 con 0x7f35840cd910 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3582d9d700 1 --2- 172.21.15.170:0/2551095854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840b55a0 0x7f3584164400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f357c009e90 tx=0x7f357c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509c69ee576d03ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3581d9b700 1 -- 172.21.15.170:0/2551095854 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c00e070 con 0x7f35840b55a0 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3581d9b700 1 -- 172.21.15.170:0/2551095854 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c00bbf0 con 0x7f35840b55a0 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3581d9b700 1 -- 172.21.15.170:0/2551095854 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c012430 con 0x7f35840b55a0 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 -- 172.21.15.170:0/2551095854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840b55a0 msgr2=0x7f3584164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 --2- 172.21.15.170:0/2551095854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840b55a0 0x7f3584164400 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f357c009e90 tx=0x7f357c009780 comp rx=0 tx=0).stop 2023-10-08T16:06:08.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 -- 172.21.15.170:0/2551095854 shutdown_connections 2023-10-08T16:06:08.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 --2- 172.21.15.170:0/2551095854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840b55a0 0x7f3584164400 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 -- 172.21.15.170:0/2551095854 >> 172.21.15.170:0/2551095854 conn(0x7f35840999f0 msgr2=0x7f35840b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:08.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 -- 172.21.15.170:0/2551095854 shutdown_connections 2023-10-08T16:06:08.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.645+0000 7f3588f6a700 1 -- 172.21.15.170:0/2551095854 wait complete. 2023-10-08T16:06:08.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 Processor -- start 2023-10-08T16:06:08.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 -- start start 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35840b55a0 0x7f3584147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3584159ec0 0x7f3584157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358406a980 con 0x7f35840d0c10 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35841477a0 con 0x7f3584159ec0 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3588f6a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358406ae10 con 0x7f35840b55a0 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3582d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3582d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44758/0 (socket says 172.21.15.170:44758) 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.647+0000 7f3582d9d700 1 -- 172.21.15.170:0/3822194318 learned_addr learned my addr 172.21.15.170:0/3822194318 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f358259c700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35840b55a0 0x7f3584147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:08.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 -- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35840b55a0 msgr2=0x7f3584147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35840b55a0 0x7f3584147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 -- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3584159ec0 msgr2=0x7f3584157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3579d9b700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3584159ec0 0x7f3584157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3584159ec0 0x7f3584157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 -- 172.21.15.170:0/3822194318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357c009a20 con 0x7f35840d0c10 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3582d9d700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f357c0099f0 tx=0x7f357c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c01b630 con 0x7f35840d0c10 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c01b790 con 0x7f35840d0c10 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358406a050 con 0x7f35840d0c10 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c01bb30 con 0x7f35840d0c10 2023-10-08T16:06:08.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.648+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358406a610 con 0x7f35840d0c10 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.649+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3584063c90 con 0x7f35840d0c10 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.649+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f357c01c400 con 0x7f35840d0c10 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.650+0000 7f357affd700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 0x7f3564069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.650+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f357c086480 con 0x7f35840d0c10 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.650+0000 7f358259c700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 0x7f3564069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:08.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.650+0000 7f358259c700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 0x7f3564069470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35740017b0 tx=0x7f3574009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:08.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.654+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f357c053750 con 0x7f35840d0c10 2023-10-08T16:06:08.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:08.837+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f35840718b0 con 0x7f35840d0c10 2023-10-08T16:06:09.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.513+0000 7f357affd700 1 -- 172.21.15.170:0/3822194318 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f357c056d70 con 0x7f35840d0c10 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 msgr2=0x7f3564069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 0x7f3564069470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f35740017b0 tx=0x7f3574009040 comp rx=0 tx=0).stop 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 msgr2=0x7f3584158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f357c0099f0 tx=0x7f357c019000 comp rx=0 tx=0).stop 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 shutdown_connections 2023-10-08T16:06:09.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3564066fc0 0x7f3564069470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35840d0c10 0x7f3584158150 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f35840b55a0 0x7f3584147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 --2- 172.21.15.170:0/3822194318 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3584159ec0 0x7f3584157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 >> 172.21.15.170:0/3822194318 conn(0x7f35840999f0 msgr2=0x7f35840c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.517+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 shutdown_connections 2023-10-08T16:06:09.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.518+0000 7f3588f6a700 1 -- 172.21.15.170:0/3822194318 wait complete. 2023-10-08T16:06:09.520 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:06:09.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:06:09.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.647+0000 7f3b98295700 1 Processor -- start 2023-10-08T16:06:09.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.647+0000 7f3b98295700 1 -- start start 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b98295700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b90069d60 0x7f3b900c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b98295700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9006bdb0 con 0x7f3b900aca80 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b98295700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b900cb370 con 0x7f3b900c81f0 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b98295700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b90159f60 con 0x7f3b90069d60 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b96031700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3b900aca80 0x7f3b9006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58144/0 (socket says 172.21.15.170:58144) 2023-10-08T16:06:09.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b96031700 1 -- 172.21.15.170:0/783933998 learned_addr learned my addr 172.21.15.170:0/783933998 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b95830700 1 --2- 172.21.15.170:0/783933998 >> v2:172.21.15.170:3300/0 conn(0x7f3b90069d60 0x7f3b900c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b9502f700 1 -- 172.21.15.170:0/783933998 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587165603 0 0) 0x7f3b9006bdb0 con 0x7f3b900aca80 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b9502f700 1 -- 172.21.15.170:0/783933998 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b7c003570 con 0x7f3b900aca80 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.648+0000 7f3b95830700 1 -- 172.21.15.170:0/783933998 >> v1:172.21.15.191:6789/0 conn(0x7f3b900c81f0 legacy=0x7f3b900cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b95830700 1 -- 172.21.15.170:0/783933998 >> v1:172.21.15.170:6789/0 conn(0x7f3b900aca80 legacy=0x7f3b9006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b95830700 1 -- 172.21.15.170:0/783933998 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9006b2b0 con 0x7f3b90069d60 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b95830700 1 --2- 172.21.15.170:0/783933998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b90069d60 0x7f3b900c7cb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b800098e0 tx=0x7f3b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f8b247452088a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b9502f700 1 -- 172.21.15.170:0/783933998 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000b640 con 0x7f3b90069d60 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b9502f700 1 -- 172.21.15.170:0/783933998 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000b7a0 con 0x7f3b90069d60 2023-10-08T16:06:09.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b9502f700 1 -- 172.21.15.170:0/783933998 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8000bb40 con 0x7f3b90069d60 2023-10-08T16:06:09.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b98295700 1 -- 172.21.15.170:0/783933998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b90069d60 msgr2=0x7f3b900c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.649+0000 7f3b98295700 1 --2- 172.21.15.170:0/783933998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b90069d60 0x7f3b900c7cb0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3b800098e0 tx=0x7f3b800091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.650+0000 7f3b98295700 1 -- 172.21.15.170:0/783933998 shutdown_connections 2023-10-08T16:06:09.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.650+0000 7f3b98295700 1 --2- 172.21.15.170:0/783933998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b90069d60 0x7f3b900c7cb0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.650+0000 7f3b98295700 1 -- 172.21.15.170:0/783933998 >> 172.21.15.170:0/783933998 conn(0x7f3b900cd670 msgr2=0x7f3b900cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.650+0000 7f3b98295700 1 -- 172.21.15.170:0/783933998 shutdown_connections 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.650+0000 7f3b98295700 1 -- 172.21.15.170:0/783933998 wait complete. 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 Processor -- start 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 -- start start 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90069d60 0x7f3b901579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 0x7f3b9016db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90157f00 con 0x7f3b900aca80 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90156920 con 0x7f3b90069d60 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b98295700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90156c00 con 0x7f3b900c81f0 2023-10-08T16:06:09.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.651+0000 7f3b95830700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44776/0 (socket says 172.21.15.170:44776) 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 -- 172.21.15.170:0/1294828643 learned_addr learned my addr 172.21.15.170:0/1294828643 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b96031700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90069d60 0x7f3b901579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b96832700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 0x7f3b9016db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 -- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 msgr2=0x7f3b9016db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 0x7f3b9016db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 -- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90069d60 msgr2=0x7f3b901579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90069d60 0x7f3b901579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 -- 172.21.15.170:0/1294828643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80009420 con 0x7f3b900aca80 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b96832700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 0x7f3b9016db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:09.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b95830700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013000 tx=0x7f3b80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80000d10 con 0x7f3b900aca80 2023-10-08T16:06:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8000bb20 con 0x7f3b900aca80 2023-10-08T16:06:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b90156ee0 con 0x7f3b900aca80 2023-10-08T16:06:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80011650 con 0x7f3b900aca80 2023-10-08T16:06:09.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.652+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b901574a0 con 0x7f3b900aca80 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.653+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90157690 con 0x7f3b900aca80 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.653+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3b80011a00 con 0x7f3b900aca80 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.654+0000 7f3b867fc700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 0x7f3b74069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.654+0000 7f3b96031700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 0x7f3b74069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.654+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3b800857e0 con 0x7f3b900aca80 2023-10-08T16:06:09.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.654+0000 7f3b96031700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 0x7f3b74069470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b900ad0d0 tx=0x7f3b8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.658+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b80052d60 con 0x7f3b900aca80 2023-10-08T16:06:09.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.850+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b9004c3c0 con 0x7f3b900aca80 2023-10-08T16:06:09.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.853+0000 7f3b867fc700 1 -- 172.21.15.170:0/1294828643 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b80011cb0 con 0x7f3b900aca80 2023-10-08T16:06:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.855+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 msgr2=0x7f3b74069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.855+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 0x7f3b74069470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3b900ad0d0 tx=0x7f3b8c009040 comp rx=0 tx=0).stop 2023-10-08T16:06:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.855+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 msgr2=0x7f3b9016d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.855+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3b80013000 tx=0x7f3b80017000 comp rx=0 tx=0).stop 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.855+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 shutdown_connections 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b900c81f0 0x7f3b9016db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b90069d60 0x7f3b901579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b74066fc0 0x7f3b74069470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 --2- 172.21.15.170:0/1294828643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b900aca80 0x7f3b9016d640 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 >> 172.21.15.170:0/1294828643 conn(0x7f3b900cd670 msgr2=0x7f3b900c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 shutdown_connections 2023-10-08T16:06:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.856+0000 7f3b98295700 1 -- 172.21.15.170:0/1294828643 wait complete. 2023-10-08T16:06:09.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info' 2023-10-08T16:06:09.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.981+0000 7f748aa40700 1 Processor -- start 2023-10-08T16:06:09.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f748aa40700 1 -- start start 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f748aa40700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7484159c10 0x7f748415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f748aa40700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748415c480 con 0x7f748415c910 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f748aa40700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748415b5d0 con 0x7f74840b5ea0 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f748aa40700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840b2a40 con 0x7f7484159c10 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f7488fdd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f748415c910 0x7f7484159fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58158/0 (socket says 172.21.15.170:58158) 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.982+0000 7f7488fdd700 1 -- 172.21.15.170:0/1407429353 learned_addr learned my addr 172.21.15.170:0/1407429353 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7483fff700 1 --2- 172.21.15.170:0/1407429353 >> v2:172.21.15.191:3300/0 conn(0x7f7484159c10 0x7f748415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848945635 0 0) 0x7f748415c480 con 0x7f748415c910 2023-10-08T16:06:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c003570 con 0x7f748415c910 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1777193066 0 0) 0x7f746c003570 con 0x7f748415c910 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 >> v2:172.21.15.191:3300/0 conn(0x7f7484159c10 msgr2=0x7f748415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 --2- 172.21.15.170:0/1407429353 >> v2:172.21.15.191:3300/0 conn(0x7f7484159c10 0x7f748415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 >> v1:172.21.15.191:6789/0 conn(0x7f74840b5ea0 legacy=0x7f7484068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748415b870 con 0x7f748415c910 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f747c0017f0 con 0x7f748415c910 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 >> v1:172.21.15.170:6789/0 conn(0x7f748415c910 legacy=0x7f7484159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746c003a60 0x7f746c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 0x7f746c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c0051c0 0x7f746c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00c130 con 0x7f746c004810 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00c2a0 con 0x7f746c003a60 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746c00c460 con 0x7f746c0051c0 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f74837fe700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 0x7f746c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7483fff700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746c003a60 0x7f746c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.983+0000 7f7488fdd700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c0051c0 0x7f746c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 -- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c0051c0 msgr2=0x7f746c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c0051c0 0x7f746c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 -- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746c003a60 msgr2=0x7f746c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f746c003a60 0x7f746c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 -- 172.21.15.170:0/1407429353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c003570 con 0x7f746c004810 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f7488fdd700 1 -- 172.21.15.170:0/1407429353 reap_dead start 2023-10-08T16:06:09.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f74837fe700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 0x7f746c004c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7478012f40 tx=0x7f7478007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d50f0db44df2f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478017070 con 0x7f746c004810 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478001aa0 con 0x7f746c004810 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f7482ffd700 1 -- 172.21.15.170:0/1407429353 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478001e80 con 0x7f746c004810 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 -- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 msgr2=0x7f746c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 0x7f746c004c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7478012f40 tx=0x7f7478007dc0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 -- 172.21.15.170:0/1407429353 shutdown_connections 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 --2- 172.21.15.170:0/1407429353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f746c004810 0x7f746c004c80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 -- 172.21.15.170:0/1407429353 >> 172.21.15.170:0/1407429353 conn(0x7f7484099a20 msgr2=0x7f74840af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 -- 172.21.15.170:0/1407429353 shutdown_connections 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.984+0000 7f748aa40700 1 -- 172.21.15.170:0/1407429353 wait complete. 2023-10-08T16:06:09.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.985+0000 7f748aa40700 1 Processor -- start 2023-10-08T16:06:09.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 -- start start 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c004810 0x7f74840aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74840afd40 0x7f7484154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484159850 con 0x7f7484158ec0 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484154c40 con 0x7f74840afd40 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.986+0000 7f748aa40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840b0490 con 0x7f746c004810 2023-10-08T16:06:09.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44788/0 (socket says 172.21.15.170:44788) 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 -- 172.21.15.170:0/443842405 learned_addr learned my addr 172.21.15.170:0/443842405 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f74837fe700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74840afd40 0x7f7484154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7483fff700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c004810 0x7f74840aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 -- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c004810 msgr2=0x7f74840aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c004810 0x7f74840aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 -- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74840afd40 msgr2=0x7f7484154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74840afd40 0x7f7484154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 -- 172.21.15.170:0/443842405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74840b0770 con 0x7f7484158ec0 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f7488fdd700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f74840b6390 tx=0x7f747c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00ac90 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00ae10 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74840b0a50 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c013640 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.987+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74840b01b0 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.988+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748404c3c0 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.989+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f747c0139f0 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.989+0000 7f746bfff700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 0x7f7470069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.989+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f747c084a90 con 0x7f7484158ec0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.989+0000 7f7483fff700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 0x7f7470069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:09.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.989+0000 7f7483fff700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 0x7f7470069470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7474006f00 tx=0x7f74740091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:09.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:09.993+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f747c052010 con 0x7f7484158ec0 2023-10-08T16:06:10.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.253+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]} v 0) v1 -- 0x7f748415bb50 con 0x7f7484158ec0 2023-10-08T16:06:10.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.515+0000 7f746bfff700 1 -- 172.21.15.170:0/443842405 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info"]}]=0 v108) v1 ==== 133+0+0 (secure 0 0 0) 0x7f747c055630 con 0x7f7484158ec0 2023-10-08T16:06:10.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.517+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 msgr2=0x7f7470069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:10.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.517+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 0x7f7470069470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7474006f00 tx=0x7f74740091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.517+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 msgr2=0x7f7484159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:10.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.517+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f74840b6390 tx=0x7f747c0021d0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 shutdown_connections 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7470066fc0 0x7f7470069470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7484158ec0 0x7f7484159310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f746c004810 0x7f74840aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 --2- 172.21.15.170:0/443842405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f74840afd40 0x7f7484154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 >> 172.21.15.170:0/443842405 conn(0x7f7484099a20 msgr2=0x7f748409abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 shutdown_connections 2023-10-08T16:06:10.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.518+0000 7f748aa40700 1 -- 172.21.15.170:0/443842405 wait complete. 2023-10-08T16:06:10.531 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command ... ok 2023-10-08T16:06:10.532 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) Tests the pending_subvolume_deletions in 'fs volume info' command 2023-10-08T16:06:10.532 INFO:tasks.cephfs_test_runner:test_volume_info_pending_subvol_deletions (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:06:10.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-08T16:06:10.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.645+0000 7fdff4c77700 1 Processor -- start 2023-10-08T16:06:10.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.645+0000 7fdff4c77700 1 -- start start 2023-10-08T16:06:10.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdff4c77700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdff0159420 0x7fdff00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdff4c77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff01597f0 0x7fdff00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdff4c77700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff00b11d0 con 0x7fdff015d230 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdff4c77700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff00b14b0 con 0x7fdff0159420 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdff4c77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff00b1790 con 0x7fdff01597f0 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdfef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdff0159420 0x7fdff00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdfef7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdff0159420 0x7fdff00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44796/0 (socket says 172.21.15.170:44796) 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.646+0000 7fdfef7fe700 1 -- 172.21.15.170:0/1797244608 learned_addr learned my addr 172.21.15.170:0/1797244608 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfef7fe700 1 -- 172.21.15.170:0/1797244608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff01597f0 msgr2=0x7fdff00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfeeffd700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff01597f0 0x7fdff00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff01597f0 0x7fdff00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfef7fe700 1 -- 172.21.15.170:0/1797244608 >> v1:172.21.15.191:6789/0 conn(0x7fdff015d230 legacy=0x7fdff00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:10.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfef7fe700 1 -- 172.21.15.170:0/1797244608 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff00b1a70 con 0x7fdff0159420 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff0159420 0x7fdff00b5f40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009f20 tx=0x7fdfe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf35e9cc5c3c0bf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfee7fc700 1 -- 172.21.15.170:0/1797244608 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe000e070 con 0x7fdff0159420 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfee7fc700 1 -- 172.21.15.170:0/1797244608 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe000ba40 con 0x7fdff0159420 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.647+0000 7fdfee7fc700 1 -- 172.21.15.170:0/1797244608 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00092e0 con 0x7fdff0159420 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 -- 172.21.15.170:0/1797244608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff0159420 msgr2=0x7fdff00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff0159420 0x7fdff00b5f40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009f20 tx=0x7fdfe0009840 comp rx=0 tx=0).stop 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 -- 172.21.15.170:0/1797244608 shutdown_connections 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff0159420 0x7fdff00b5f40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1797244608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff01597f0 0x7fdff00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 -- 172.21.15.170:0/1797244608 >> 172.21.15.170:0/1797244608 conn(0x7fdff00d0660 msgr2=0x7fdff00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 -- 172.21.15.170:0/1797244608 shutdown_connections 2023-10-08T16:06:10.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.648+0000 7fdff4c77700 1 -- 172.21.15.170:0/1797244608 wait complete. 2023-10-08T16:06:10.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.649+0000 7fdff4c77700 1 Processor -- start 2023-10-08T16:06:10.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.649+0000 7fdff4c77700 1 -- start start 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 0x7fdff01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 0x7fdff00ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff016b490 con 0x7fdff01597f0 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff0155810 con 0x7fdff015d230 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff016a880 con 0x7fdff0159420 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44808/0 (socket says 172.21.15.170:44808) 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 -- 172.21.15.170:0/1149060467 learned_addr learned my addr 172.21.15.170:0/1149060467 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:10.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 0x7fdff01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeffff700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 0x7fdff00ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 -- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 msgr2=0x7fdff01552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 0x7fdff01552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 -- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 msgr2=0x7fdff00ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 0x7fdff00ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 -- 172.21.15.170:0/1149060467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe0009ae0 con 0x7fdff01597f0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 0x7fdff01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeffff700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 0x7fdff00ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfeeffd700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c800 tx=0x7fdfe4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011600 con 0x7fdff01597f0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe4011760 con 0x7fdff01597f0 2023-10-08T16:06:10.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff016ab60 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4011b20 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.650+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff016b120 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.651+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff016b640 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.652+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fdfe40193f0 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.652+0000 7fdfd77fe700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 0x7fdfdc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.652+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdfe4084760 con 0x7fdff01597f0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.652+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 0x7fdfdc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:10.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.652+0000 7fdfef7fe700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 0x7fdfdc069470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009880 tx=0x7fdfe0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:10.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.656+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdff016b640 con 0x7fdff01597f0 2023-10-08T16:06:10.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:10.918+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fdff016b640 con 0x7fdff01597f0 2023-10-08T16:06:11.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.516+0000 7fdfd77fe700 1 -- 172.21.15.170:0/1149060467 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v109) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdff016b640 con 0x7fdff01597f0 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 msgr2=0x7fdfdc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 0x7fdfdc069470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009880 tx=0x7fdfe0017000 comp rx=0 tx=0).stop 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 msgr2=0x7fdff00cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdfe400c800 tx=0x7fdfe4008040 comp rx=0 tx=0).stop 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 shutdown_connections 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdff015d230 0x7fdff00ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdff0159420 0x7fdff01552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdfdc066fc0 0x7fdfdc069470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 --2- 172.21.15.170:0/1149060467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdff01597f0 0x7fdff00cf850 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.518+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 >> 172.21.15.170:0/1149060467 conn(0x7fdff00d0660 msgr2=0x7fdff0069920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:11.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.519+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 shutdown_connections 2023-10-08T16:06:11.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.519+0000 7fdff4c77700 1 -- 172.21.15.170:0/1149060467 wait complete. 2023-10-08T16:06:11.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:06:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.649+0000 7ff33758c700 1 Processor -- start 2023-10-08T16:06:11.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.649+0000 7ff33758c700 1 -- start start 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff33758c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3300aca80 0x7ff33006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff33758c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff33758c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33006bdb0 con 0x7ff3300c81f0 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff33758c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3300c8670 con 0x7ff3300aca80 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff33758c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300cb370 con 0x7ff330069d60 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff335328700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3300aca80 0x7ff33006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff335328700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3300aca80 0x7ff33006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44812/0 (socket says 172.21.15.170:44812) 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff335328700 1 -- 172.21.15.170:0/3853967699 learned_addr learned my addr 172.21.15.170:0/3853967699 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:11.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff334b27700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff335328700 1 -- 172.21.15.170:0/3853967699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 msgr2=0x7ff3300c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.650+0000 7ff335328700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff335328700 1 -- 172.21.15.170:0/3853967699 >> v1:172.21.15.191:6789/0 conn(0x7ff3300c81f0 legacy=0x7ff330068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff335328700 1 -- 172.21.15.170:0/3853967699 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330159f60 con 0x7ff3300aca80 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff334b27700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff335328700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff33006aba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009990 tx=0x7ff32c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3251ab2e5634fddb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff327fff700 1 -- 172.21.15.170:0/3853967699 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005c10 con 0x7ff3300aca80 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff327fff700 1 -- 172.21.15.170:0/3853967699 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c005d70 con 0x7ff3300aca80 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff327fff700 1 -- 172.21.15.170:0/3853967699 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00b880 con 0x7ff3300aca80 2023-10-08T16:06:11.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 -- 172.21.15.170:0/3853967699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 msgr2=0x7ff33006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff33006aba0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009990 tx=0x7ff32c009220 comp rx=0 tx=0).stop 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 -- 172.21.15.170:0/3853967699 shutdown_connections 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff33006aba0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 --2- 172.21.15.170:0/3853967699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 -- 172.21.15.170:0/3853967699 >> 172.21.15.170:0/3853967699 conn(0x7ff3300cd670 msgr2=0x7ff3300cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 -- 172.21.15.170:0/3853967699 shutdown_connections 2023-10-08T16:06:11.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.651+0000 7ff33758c700 1 -- 172.21.15.170:0/3853967699 wait complete. 2023-10-08T16:06:11.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 Processor -- start 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 -- start start 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300c81f0 0x7ff330157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015a9a0 con 0x7ff3300aca80 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33016d7f0 con 0x7ff3300c81f0 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff33758c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33016dad0 con 0x7ff330069d60 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.653+0000 7ff334b27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44820/0 (socket says 172.21.15.170:44820) 2023-10-08T16:06:11.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 -- 172.21.15.170:0/3431820615 learned_addr learned my addr 172.21.15.170:0/3431820615 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff335328700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff335b29700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300c81f0 0x7ff330157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 -- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 msgr2=0x7ff3300af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 -- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300c81f0 msgr2=0x7ff330157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300c81f0 0x7ff330157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 -- 172.21.15.170:0/3431820615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c009480 con 0x7ff3300aca80 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff334b27700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e8a0 tx=0x7ff320008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320013410 con 0x7ff3300aca80 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff320013570 con 0x7ff3300aca80 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33016ddb0 con 0x7ff3300aca80 2023-10-08T16:06:11.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff320013930 con 0x7ff3300aca80 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.654+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33016e370 con 0x7ff3300aca80 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.655+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33004c3c0 con 0x7ff3300aca80 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.655+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff320013ca0 con 0x7ff3300aca80 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.656+0000 7ff3257fa700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 0x7ff314069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.656+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff320084010 con 0x7ff3300aca80 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.656+0000 7ff335328700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 0x7ff314069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:11.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.656+0000 7ff335328700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 0x7ff314069470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009260 tx=0x7ff32c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:11.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.660+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff320054f10 con 0x7ff3300aca80 2023-10-08T16:06:11.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.844+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff33006b510 con 0x7ff3300aca80 2023-10-08T16:06:11.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.847+0000 7ff3257fa700 1 -- 172.21.15.170:0/3431820615 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff320054aa0 con 0x7ff3300aca80 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 msgr2=0x7ff314069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 0x7ff314069470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009260 tx=0x7ff32c016000 comp rx=0 tx=0).stop 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 msgr2=0x7ff3300ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff32000e8a0 tx=0x7ff320008040 comp rx=0 tx=0).stop 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 shutdown_connections 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff3300c81f0 0x7ff330157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff330069d60 0x7ff3300af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff314066fc0 0x7ff314069470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 --2- 172.21.15.170:0/3431820615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3300aca80 0x7ff3300ab9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 >> 172.21.15.170:0/3431820615 conn(0x7ff3300cd670 msgr2=0x7ff330157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 shutdown_connections 2023-10-08T16:06:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:11.849+0000 7ff33758c700 1 -- 172.21.15.170:0/3431820615 wait complete. 2023-10-08T16:06:11.863 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:06:11.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:06:11.903 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:06:11.952 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:06:11.953 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:06:11.953 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:12.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.065+0000 7ffbe91fe700 1 Processor -- start 2023-10-08T16:06:12.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.065+0000 7ffbe91fe700 1 -- start start 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe91fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffbe4159890 0x7ffbe415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe91fe700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe415a160 con 0x7ffbe4159c60 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe91fe700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40d13a0 con 0x7ffbe4069560 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe91fe700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe415e140 con 0x7ffbe4159890 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ffbe4159890 0x7ffbe415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbda59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ffbe4159c60 0x7ffbe415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58172/0 (socket says 172.21.15.170:58172) 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbda59c700 1 -- 172.21.15.170:0/3594696307 learned_addr learned my addr 172.21.15.170:0/3594696307 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/3594696307 >> v1:172.21.15.191:6789/0 conn(0x7ffbe4069560 legacy=0x7ffbe40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/3594696307 >> v1:172.21.15.170:6789/0 conn(0x7ffbe4159c60 legacy=0x7ffbe415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe259c700 1 -- 172.21.15.170:0/3594696307 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597351098 0 0) 0x7ffbe415a160 con 0x7ffbe4159c60 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.066+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/3594696307 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe415a3d0 con 0x7ffbe4159890 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe2d9d700 1 --2- 172.21.15.170:0/3594696307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4159890 0x7ffbe415abd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40098e0 tx=0x7ffbd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb321be0c0c2ff3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe259c700 1 -- 172.21.15.170:0/3594696307 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4009e80 con 0x7ffbe4159890 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe259c700 1 -- 172.21.15.170:0/3594696307 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd400b820 con 0x7ffbe4159890 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe259c700 1 -- 172.21.15.170:0/3594696307 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400bba0 con 0x7ffbe4159890 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 -- 172.21.15.170:0/3594696307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4159890 msgr2=0x7ffbe415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/3594696307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4159890 0x7ffbe415abd0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40098e0 tx=0x7ffbd40091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 -- 172.21.15.170:0/3594696307 shutdown_connections 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/3594696307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4159890 0x7ffbe415abd0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 -- 172.21.15.170:0/3594696307 >> 172.21.15.170:0/3594696307 conn(0x7ffbe4068e20 msgr2=0x7ffbe40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:12.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 -- 172.21.15.170:0/3594696307 shutdown_connections 2023-10-08T16:06:12.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.067+0000 7ffbe91fe700 1 -- 172.21.15.170:0/3594696307 wait complete. 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 Processor -- start 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 -- start start 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe4159890 0x7ffbe415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe4159c60 0x7ffbe40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b4850 con 0x7ffbe4069560 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b4ac0 con 0x7ffbe4159c60 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe91fe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40b4da0 con 0x7ffbe4159890 2023-10-08T16:06:12.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44844/0 (socket says 172.21.15.170:44844) 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.069+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/4246037123 learned_addr learned my addr 172.21.15.170:0/4246037123 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe359e700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe4159c60 0x7ffbe40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbda59c700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe4159890 0x7ffbe415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe4159890 msgr2=0x7ffbe415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe4159890 0x7ffbe415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe4159c60 msgr2=0x7ffbe40ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe4159c60 0x7ffbe40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 -- 172.21.15.170:0/4246037123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4009420 con 0x7ffbe4069560 2023-10-08T16:06:12.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe2d9d700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40030c0 tx=0x7ffbd4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400bae0 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd4020070 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe40b5080 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd401b6a0 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.070+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe414d930 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.071+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe40b55a0 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.071+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ffbd4022030 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.072+0000 7ffbdb7fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 0x7ffbc4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.072+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ffbd40862c0 con 0x7ffbe4069560 2023-10-08T16:06:12.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.072+0000 7ffbda59c700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 0x7ffbc4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.072+0000 7ffbda59c700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 0x7ffbc4069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc007fd0 tx=0x7ffbcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.076+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbe40b55a0 con 0x7ffbe4069560 2023-10-08T16:06:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.304+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbe40cc820 con 0x7ffbe4069560 2023-10-08T16:06:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.304+0000 7ffbdb7fe700 1 -- 172.21.15.170:0/4246037123 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3532 (secure 0 0 0) 0x7ffbd4053840 con 0x7ffbe4069560 2023-10-08T16:06:12.306 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:12.306 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4865,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/8757553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":8757553},{"type":"v1","addr":"172.21.15.170:6836","nonce":8757553}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3781589215","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3781589215},{"type":"v1","addr":"172.21.15.191:6835","nonce":3781589215}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5048,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1185822746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1185822746},{"type":"v1","addr":"172.21.15.191:6834","nonce":1185822746}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5298,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2177212382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2177212382},{"type":"v1","addr":"172.21.15.170:6837","nonce":2177212382}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-10-08T16:06:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 msgr2=0x7ffbc4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 0x7ffbc4069470 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc007fd0 tx=0x7ffbcc009040 comp rx=0 tx=0).stop 2023-10-08T16:06:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 msgr2=0x7ffbe414d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40030c0 tx=0x7ffbd4016000 comp rx=0 tx=0).stop 2023-10-08T16:06:12.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 shutdown_connections 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffbe4159c60 0x7ffbe40ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffbc4066fc0 0x7ffbc4069470 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffbe4069560 0x7ffbe414d3f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 --2- 172.21.15.170:0/4246037123 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffbe4159890 0x7ffbe415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.306+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 >> 172.21.15.170:0/4246037123 conn(0x7ffbe4068e20 msgr2=0x7ffbe40cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.307+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 shutdown_connections 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.307+0000 7ffbe91fe700 1 -- 172.21.15.170:0/4246037123 wait complete. 2023-10-08T16:06:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 33 2023-10-08T16:06:12.320 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:06:12.320 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:06:12.321 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:06:12.321 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:06:12.321 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:06:12.321 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:06:12.321 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:06:12.322 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:06:12.322 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:06:12.322 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:06:12.322 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:06:12.322 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:06:12.322 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:06:12.322 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:06:12.323 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:06:12.323 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:06:12.330 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:06:12.330 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:06:12.331 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:06:12.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:06:12.333 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:06:12.334 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:06:12.334 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:06:12.335 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:06:12.372 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:06:12.373 DEBUG:teuthology.parallel:result is None 2023-10-08T16:06:12.373 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:06:12.373 DEBUG:teuthology.parallel:result is None 2023-10-08T16:06:12.376 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:06:12.376 DEBUG:teuthology.parallel:result is None 2023-10-08T16:06:12.377 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:06:12.377 DEBUG:teuthology.parallel:result is None 2023-10-08T16:06:12.377 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:06:12.446 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:06:12.446 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:06:12.458 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:06:12.459 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:06:12.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 Processor -- start 2023-10-08T16:06:12.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 -- start start 2023-10-08T16:06:12.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f903015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300b2980 con 0x7f9030159420 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903015dbb0 con 0x7f90301597f0 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.497+0000 7f9034803700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300b11d0 con 0x7f903015d230 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9030159420 0x7f90300b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58190/0 (socket says 172.21.15.170:58190) 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902ed9d700 1 -- 172.21.15.170:0/383900216 learned_addr learned my addr 172.21.15.170:0/383900216 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902f59e700 1 --2- 172.21.15.170:0/383900216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f903015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518541982 0 0) 0x7f90300b2980 con 0x7f9030159420 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901c003570 con 0x7f9030159420 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563703111 0 0) 0x7f903015dbb0 con 0x7f90301597f0 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90300b2980 con 0x7f90301597f0 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902f59e700 1 -- 172.21.15.170:0/383900216 >> v1:172.21.15.191:6789/0 conn(0x7f90301597f0 legacy=0x7f90300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902f59e700 1 -- 172.21.15.170:0/383900216 >> v1:172.21.15.170:6789/0 conn(0x7f9030159420 legacy=0x7f90300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902f59e700 1 -- 172.21.15.170:0/383900216 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300b14b0 con 0x7f903015d230 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902f59e700 1 --2- 172.21.15.170:0/383900216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f903015d670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9028009e90 tx=0x7f9028009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87b3424ff4f5fd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800e070 con 0x7f903015d230 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800bb80 con 0x7f903015d230 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.498+0000 7f902dd9b700 1 -- 172.21.15.170:0/383900216 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028012430 con 0x7f903015d230 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 -- 172.21.15.170:0/383900216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 msgr2=0x7f903015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 --2- 172.21.15.170:0/383900216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f903015d670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9028009e90 tx=0x7f9028009780 comp rx=0 tx=0).stop 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 -- 172.21.15.170:0/383900216 shutdown_connections 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 --2- 172.21.15.170:0/383900216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f903015d670 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 -- 172.21.15.170:0/383900216 >> 172.21.15.170:0/383900216 conn(0x7f90300d0660 msgr2=0x7f90300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:12.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 -- 172.21.15.170:0/383900216 shutdown_connections 2023-10-08T16:06:12.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.499+0000 7f9034803700 1 -- 172.21.15.170:0/383900216 wait complete. 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.500+0000 7f9034803700 1 Processor -- start 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.500+0000 7f9034803700 1 -- start start 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90301597f0 0x7f90300c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f90300c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903015cd50 con 0x7f9030159420 2023-10-08T16:06:12.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300c9a10 con 0x7f90301597f0 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f9034803700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300c5fe0 con 0x7f903015d230 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44894/0 (socket says 172.21.15.170:44894) 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 -- 172.21.15.170:0/3423692301 learned_addr learned my addr 172.21.15.170:0/3423692301 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902f59e700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f90300c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902e59c700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90301597f0 0x7f90300c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 -- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 msgr2=0x7f90300c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f90300c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 -- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90301597f0 msgr2=0x7f90300c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90301597f0 0x7f90300c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:12.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.501+0000 7f902ed9d700 1 -- 172.21.15.170:0/3423692301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9028009a20 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f902ed9d700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f902000c790 tx=0x7f9020007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020005b30 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9020005c90 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90300c62c0 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020007300 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.502+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90300c9630 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.503+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903004c3c0 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.503+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f902001b400 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.503+0000 7f9026ffd700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 0x7f9010069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.503+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9020083d90 con 0x7f9030159420 2023-10-08T16:06:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.503+0000 7f902e59c700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 0x7f9010069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.504+0000 7f902e59c700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 0x7f9010069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f903016cc10 tx=0x7f9018009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.507+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9020051440 con 0x7f9030159420 2023-10-08T16:06:12.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:12.708+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90300b1790 con 0x7f9030159420 2023-10-08T16:06:12.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:06:12.989 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:06:13.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.534+0000 7f9026ffd700 1 -- 172.21.15.170:0/3423692301 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f90200076f0 con 0x7f9030159420 2023-10-08T16:06:13.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 msgr2=0x7f9010069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 0x7f9010069470 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f903016cc10 tx=0x7f9018009040 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 msgr2=0x7f90301550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f902000c790 tx=0x7f9020007ea0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 shutdown_connections 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f903015d230 0x7f90300c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9010066fc0 0x7f9010069470 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9030159420 0x7f90301550b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 --2- 172.21.15.170:0/3423692301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90301597f0 0x7f90300c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 >> 172.21.15.170:0/3423692301 conn(0x7f90300d0660 msgr2=0x7f9030155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 shutdown_connections 2023-10-08T16:06:13.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.537+0000 7f9034803700 1 -- 172.21.15.170:0/3423692301 wait complete. 2023-10-08T16:06:13.540 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:06:13.551 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:06:13.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.664+0000 7fc5c20b8700 1 Processor -- start 2023-10-08T16:06:13.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.664+0000 7fc5c20b8700 1 -- start start 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c20b8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5bc159420 0x7fc5bc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c20b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c20b8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc0b2980 con 0x7fc5bc1597f0 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c20b8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc15dbb0 con 0x7fc5bc159420 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c20b8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b11d0 con 0x7fc5bc15d230 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c10b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5bc159420 0x7fc5bc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c18b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c10b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5bc159420 0x7fc5bc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:46978/0 (socket says 172.21.15.170:46978) 2023-10-08T16:06:13.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c18b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:48446/0 (socket says 172.21.15.170:48446) 2023-10-08T16:06:13.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.665+0000 7fc5c10b6700 1 -- 172.21.15.170:0/1708215400 learned_addr learned my addr 172.21.15.170:0/1708215400 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:13.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c10b6700 1 -- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 msgr2=0x7fc5bc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:13.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c10b6700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c10b6700 1 -- 172.21.15.170:0/1708215400 >> v1:172.21.15.191:6789/0 conn(0x7fc5bc1597f0 legacy=0x7fc5bc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c10b6700 1 -- 172.21.15.170:0/1708215400 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0b14b0 con 0x7fc5bc159420 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c18b7700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5c10b6700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc159420 0x7fc5bc0b5f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80099b0 tx=0x7fc5b8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393b6e96f4c5faa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5b3fff700 1 -- 172.21.15.170:0/1708215400 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8009ea0 con 0x7fc5bc159420 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5b3fff700 1 -- 172.21.15.170:0/1708215400 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b800b860 con 0x7fc5bc159420 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.666+0000 7fc5b3fff700 1 -- 172.21.15.170:0/1708215400 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b800bc00 con 0x7fc5bc159420 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 -- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc159420 msgr2=0x7fc5bc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc159420 0x7fc5bc0b5f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80099b0 tx=0x7fc5b8009250 comp rx=0 tx=0).stop 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 -- 172.21.15.170:0/1708215400 shutdown_connections 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc15d230 0x7fc5bc15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/1708215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc159420 0x7fc5bc0b5f40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 -- 172.21.15.170:0/1708215400 >> 172.21.15.170:0/1708215400 conn(0x7fc5bc0d0660 msgr2=0x7fc5bc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:13.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 -- 172.21.15.170:0/1708215400 shutdown_connections 2023-10-08T16:06:13.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.667+0000 7fc5c20b8700 1 -- 172.21.15.170:0/1708215400 wait complete. 2023-10-08T16:06:13.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.668+0000 7fc5c20b8700 1 Processor -- start 2023-10-08T16:06:13.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.668+0000 7fc5c20b8700 1 -- start start 2023-10-08T16:06:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc159420 0x7fc5bc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc15d230 0x7fc5bc0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0c6120 con 0x7fc5bc1597f0 2023-10-08T16:06:13.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb230 con 0x7fc5bc15d230 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0cb510 con 0x7fc5bc159420 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44906/0 (socket says 172.21.15.170:44906) 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 -- 172.21.15.170:0/395796106 learned_addr learned my addr 172.21.15.170:0/395796106 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c10b6700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc159420 0x7fc5bc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c18b7700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc15d230 0x7fc5bc0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 -- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc159420 msgr2=0x7fc5bc155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc159420 0x7fc5bc155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 -- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc15d230 msgr2=0x7fc5bc0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc15d230 0x7fc5bc0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 -- 172.21.15.170:0/395796106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009a20 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c08b5700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c7c0 tx=0x7fc5ac007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac011590 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac011710 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac011ad0 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.669+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b80094f0 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.670+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0cbad0 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.670+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc0cbcf0 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.671+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fc5ac01a360 con 0x7fc5bc1597f0 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.671+0000 7fc5b17fa700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 0x7fc5a0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:13.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.671+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc5ac083180 con 0x7fc5bc1597f0 2023-10-08T16:06:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.671+0000 7fc5c10b6700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 0x7fc5a0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.672+0000 7fc5c10b6700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 0x7fc5a0069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80099b0 tx=0x7fc5b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:13.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.675+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5ac050810 con 0x7fc5bc1597f0 2023-10-08T16:06:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.932+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5bc04c3c0 con 0x7fc5bc1597f0 2023-10-08T16:06:13.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.932+0000 7fc5b17fa700 1 -- 172.21.15.170:0/395796106 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc5ac053e30 con 0x7fc5bc1597f0 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.934 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:06:13.935 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.936 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.937 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:06:13.938 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.939 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.940 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.941 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.942 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.943 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.944 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:06:13.945 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.946 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.947 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:06:13.948 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:06:13.949 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:06:13.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 msgr2=0x7fc5a0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 0x7fc5a0069470 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80099b0 tx=0x7fc5b8017000 comp rx=0 tx=0).stop 2023-10-08T16:06:13.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 msgr2=0x7fc5bc0c78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c7c0 tx=0x7fc5ac007f50 comp rx=0 tx=0).stop 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 shutdown_connections 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5bc15d230 0x7fc5bc0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5bc159420 0x7fc5bc155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5a0066fc0 0x7fc5a0069470 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 --2- 172.21.15.170:0/395796106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5bc1597f0 0x7fc5bc0c78d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 >> 172.21.15.170:0/395796106 conn(0x7fc5bc0d0660 msgr2=0x7fc5bc156120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 shutdown_connections 2023-10-08T16:06:13.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:13.934+0000 7fc5c20b8700 1 -- 172.21.15.170:0/395796106 wait complete. 2023-10-08T16:06:13.952 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:14.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.065+0000 7f8d26de1700 1 Processor -- start 2023-10-08T16:06:14.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.065+0000 7f8d26de1700 1 -- start start 2023-10-08T16:06:14.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d26de1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8d20159c60 0x7f8d2015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d26de1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d20069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d26de1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2015a160 con 0x7f8d20159890 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d26de1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2015df50 con 0x7f8d20159c60 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d26de1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200d13a0 con 0x7f8d2015db10 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d17fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8d20159c60 0x7f8d2015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.066+0000 7f8d2537e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d20069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8d20159c60 0x7f8d2015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:46994/0 (socket says 172.21.15.170:46994) 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 -- 172.21.15.170:0/2572938885 learned_addr learned my addr 172.21.15.170:0/2572938885 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d24b7d700 1 --1- 172.21.15.170:0/2572938885 >> v1:172.21.15.191:6789/0 conn(0x7f8d20159890 0x7f8d2015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54496/0 (socket says 172.21.15.170:54496) 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 -- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 msgr2=0x7f8d20069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d20069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 -- 172.21.15.170:0/2572938885 >> v1:172.21.15.191:6789/0 conn(0x7f8d20159890 legacy=0x7f8d2015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 -- 172.21.15.170:0/2572938885 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2015e140 con 0x7f8d20159c60 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d177fe700 1 -- 172.21.15.170:0/2572938885 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357753172 0 0) 0x7f8d2015a160 con 0x7f8d20159890 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d2537e700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d20069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d17fff700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d2015d5d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d191d5f034a50ed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d177fe700 1 -- 172.21.15.170:0/2572938885 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10009ea0 con 0x7f8d20159c60 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d177fe700 1 -- 172.21.15.170:0/2572938885 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1000b860 con 0x7f8d20159c60 2023-10-08T16:06:14.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.067+0000 7f8d177fe700 1 -- 172.21.15.170:0/2572938885 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000bc00 con 0x7f8d20159c60 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 -- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 msgr2=0x7f8d2015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d2015d5d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10009250 comp rx=0 tx=0).stop 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 -- 172.21.15.170:0/2572938885 shutdown_connections 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d20069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 --2- 172.21.15.170:0/2572938885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d2015d5d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 -- 172.21.15.170:0/2572938885 >> 172.21.15.170:0/2572938885 conn(0x7f8d20068e20 msgr2=0x7f8d200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 -- 172.21.15.170:0/2572938885 shutdown_connections 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.068+0000 7f8d26de1700 1 -- 172.21.15.170:0/2572938885 wait complete. 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.069+0000 7f8d26de1700 1 Processor -- start 2023-10-08T16:06:14.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.069+0000 7f8d26de1700 1 -- start start 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d20153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d200b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d20169fd0 con 0x7f8d20159890 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2016ae30 con 0x7f8d20159c60 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2016a220 con 0x7f8d2015db10 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44912/0 (socket says 172.21.15.170:44912) 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 -- 172.21.15.170:0/4017216220 learned_addr learned my addr 172.21.15.170:0/4017216220 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d17fff700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d20153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d2537e700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d200b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 -- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 msgr2=0x7f8d200b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d200b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 -- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 msgr2=0x7f8d20153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d20153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 -- 172.21.15.170:0/4017216220 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18009420 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d24b7d700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00ce00 tx=0x7f8d1c001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c005df0 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d100094f0 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c00a920 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.070+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c00ace0 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.071+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2016a7e0 con 0x7f8d20159890 2023-10-08T16:06:14.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.072+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2004c3c0 con 0x7f8d20159890 2023-10-08T16:06:14.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.072+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8d1c007300 con 0x7f8d20159890 2023-10-08T16:06:14.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.072+0000 7f8d14ff9700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 0x7f8d0c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.072+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8d1c084370 con 0x7f8d20159890 2023-10-08T16:06:14.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.072+0000 7f8d17fff700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 0x7f8d0c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.073+0000 7f8d17fff700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 0x7f8d0c069470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.076+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d1c051a00 con 0x7f8d20159890 2023-10-08T16:06:14.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.261+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d2006a5b0 con 0x7f8d20159890 2023-10-08T16:06:14.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.261+0000 7f8d14ff9700 1 -- 172.21.15.170:0/4017216220 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f8d1c055020 con 0x7f8d20159890 2023-10-08T16:06:14.263 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:14.263 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":59,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:13.534033+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 msgr2=0x7f8d0c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 0x7f8d0c069470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099b0 tx=0x7f8d10017000 comp rx=0 tx=0).stop 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 msgr2=0x7f8d200ceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00ce00 tx=0x7f8d1c001bc0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 shutdown_connections 2023-10-08T16:06:14.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d2015db10 0x7f8d200b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8d0c066fc0 0x7f8d0c069470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d20159890 0x7f8d200ceee0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 --2- 172.21.15.170:0/4017216220 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d20159c60 0x7f8d20153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.263+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 >> 172.21.15.170:0/4017216220 conn(0x7f8d20068e20 msgr2=0x7f8d20156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.264+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 shutdown_connections 2023-10-08T16:06:14.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.264+0000 7f8d26de1700 1 -- 172.21.15.170:0/4017216220 wait complete. 2023-10-08T16:06:14.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:06:14.277 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:06:14.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.393+0000 7f87f51dd700 1 Processor -- start 2023-10-08T16:06:14.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.393+0000 7f87f51dd700 1 -- start start 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f006b160 0x7f87f006b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87f00637d0 0x7f87f0068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 0x7f87f006a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f00ac450 con 0x7f87f00637d0 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f0069d00 con 0x7f87f006a2b0 2023-10-08T16:06:14.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87f51dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0068830 con 0x7f87f006b160 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87e659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87f00637d0 0x7f87f0068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87e659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87f00637d0 0x7f87f0068cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44918/0 (socket says 172.21.15.170:44918) 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.394+0000 7f87e659c700 1 -- 172.21.15.170:0/1930854411 learned_addr learned my addr 172.21.15.170:0/1930854411 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87eed9d700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f006b160 0x7f87f006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87ef59e700 1 --2- 172.21.15.170:0/1930854411 >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 0x7f87f006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 -- 172.21.15.170:0/1930854411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f006b160 msgr2=0x7f87f006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f006b160 0x7f87f006b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 -- 172.21.15.170:0/1930854411 >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 msgr2=0x7f87f006a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 --2- 172.21.15.170:0/1930854411 >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 0x7f87f006a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 -- 172.21.15.170:0/1930854411 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0069740 con 0x7f87f00637d0 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87ef59e700 1 --2- 172.21.15.170:0/1930854411 >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 0x7f87f006a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87e659c700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00637d0 0x7f87f0068cc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f87f0063560 tx=0x7f87d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da67cbd5bf52de61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87ee59c700 1 -- 172.21.15.170:0/1930854411 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b860 con 0x7f87f00637d0 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87ee59c700 1 -- 172.21.15.170:0/1930854411 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800b9c0 con 0x7f87f00637d0 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87ee59c700 1 -- 172.21.15.170:0/1930854411 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800bd80 con 0x7f87f00637d0 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 -- 172.21.15.170:0/1930854411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00637d0 msgr2=0x7f87f0068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00637d0 0x7f87f0068cc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f87f0063560 tx=0x7f87d8009250 comp rx=0 tx=0).stop 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 -- 172.21.15.170:0/1930854411 shutdown_connections 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 --2- 172.21.15.170:0/1930854411 >> v2:172.21.15.191:3300/0 conn(0x7f87f006a2b0 0x7f87f006a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f006b160 0x7f87f006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 --2- 172.21.15.170:0/1930854411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f00637d0 0x7f87f0068cc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 -- 172.21.15.170:0/1930854411 >> 172.21.15.170:0/1930854411 conn(0x7f87f006ded0 msgr2=0x7f87f006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 -- 172.21.15.170:0/1930854411 shutdown_connections 2023-10-08T16:06:14.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.395+0000 7f87f51dd700 1 -- 172.21.15.170:0/1930854411 wait complete. 2023-10-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 Processor -- start 2023-10-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 -- start start 2023-10-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 0x7f87f016c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f006a2b0 0x7f87f0157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0158430 con 0x7f87f006b160 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0158710 con 0x7f87f006a2b0 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87f51dd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01589f0 con 0x7f87f00637d0 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.397+0000 7f87ef59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44920/0 (socket says 172.21.15.170:44920) 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 -- 172.21.15.170:0/147188848 learned_addr learned my addr 172.21.15.170:0/147188848 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87e659c700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f006a2b0 0x7f87f0157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87eed9d700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 0x7f87f016c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 -- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 msgr2=0x7f87f016c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 0x7f87f016c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 -- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f006a2b0 msgr2=0x7f87f0157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f006a2b0 0x7f87f0157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 -- 172.21.15.170:0/147188848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80094f0 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87eed9d700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 0x7f87f016c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87ef59e700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87e800a380 tx=0x7f87e8007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8015440 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e80155a0 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8015960 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f0158cd0 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.398+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f016cb10 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.399+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f01591f0 con 0x7f87f006b160 2023-10-08T16:06:14.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.399+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f87e800d440 con 0x7f87f006b160 2023-10-08T16:06:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.400+0000 7f87e77fe700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 0x7f87d0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.400+0000 7f87eed9d700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 0x7f87d0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.400+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f87e8084d90 con 0x7f87f006b160 2023-10-08T16:06:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.400+0000 7f87eed9d700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 0x7f87d0069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87f0156a20 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.404+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87f01591f0 con 0x7f87f006b160 2023-10-08T16:06:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.587+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87f01591f0 con 0x7f87f006b160 2023-10-08T16:06:14.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.587+0000 7f87e77fe700 1 -- 172.21.15.170:0/147188848 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87f01591f0 con 0x7f87f006b160 2023-10-08T16:06:14.589 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 msgr2=0x7f87d0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 0x7f87d0069470 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87f0156a20 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 msgr2=0x7f87f015bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f87e800a380 tx=0x7f87e8007900 comp rx=0 tx=0).stop 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 shutdown_connections 2023-10-08T16:06:14.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87d0066fc0 0x7f87d0069470 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f006b160 0x7f87f015bf10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87f00637d0 0x7f87f016c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 --2- 172.21.15.170:0/147188848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f006a2b0 0x7f87f0157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.589+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 >> 172.21.15.170:0/147188848 conn(0x7f87f006ded0 msgr2=0x7f87f006a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.590+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 shutdown_connections 2023-10-08T16:06:14.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.590+0000 7f87f51dd700 1 -- 172.21.15.170:0/147188848 wait complete. 2023-10-08T16:06:14.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:06:14.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.716+0000 7f3ec40e0700 1 Processor -- start 2023-10-08T16:06:14.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.716+0000 7f3ec40e0700 1 -- start start 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec40e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 0x7f3ebc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec40e0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc15a160 con 0x7f3ebc069560 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec40e0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc0d13a0 con 0x7f3ebc159c60 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec40e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc15e140 con 0x7f3ebc159890 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec267d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3ebc069560 0x7f3ebc0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58196/0 (socket says 172.21.15.170:58196) 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec267d700 1 -- 172.21.15.170:0/785498051 learned_addr learned my addr 172.21.15.170:0/785498051 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec1e7c700 1 --2- 172.21.15.170:0/785498051 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 0x7f3ebc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724103350 0 0) 0x7f3ebc15a160 con 0x7f3ebc069560 2023-10-08T16:06:14.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ea8003570 con 0x7f3ebc069560 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645715143 0 0) 0x7f3ebc0d13a0 con 0x7f3ebc159c60 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ebc15a160 con 0x7f3ebc159c60 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec1e7c700 1 -- 172.21.15.170:0/785498051 >> v1:172.21.15.191:6789/0 conn(0x7f3ebc159c60 legacy=0x7f3ebc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec1e7c700 1 -- 172.21.15.170:0/785498051 >> v1:172.21.15.170:6789/0 conn(0x7f3ebc069560 legacy=0x7f3ebc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec1e7c700 1 -- 172.21.15.170:0/785498051 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc15a3d0 con 0x7f3ebc159890 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.717+0000 7f3ec1e7c700 1 --2- 172.21.15.170:0/785498051 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 0x7f3ebc15abd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009910 tx=0x7f3eb80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f840071886286e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8011630 con 0x7f3ebc159890 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb8011790 con 0x7f3ebc159890 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec0e7a700 1 -- 172.21.15.170:0/785498051 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb8011b30 con 0x7f3ebc159890 2023-10-08T16:06:14.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 -- 172.21.15.170:0/785498051 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 msgr2=0x7f3ebc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/785498051 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 0x7f3ebc15abd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009910 tx=0x7f3eb80091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 -- 172.21.15.170:0/785498051 shutdown_connections 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/785498051 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc159890 0x7f3ebc15abd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 -- 172.21.15.170:0/785498051 >> 172.21.15.170:0/785498051 conn(0x7f3ebc068e20 msgr2=0x7f3ebc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 -- 172.21.15.170:0/785498051 shutdown_connections 2023-10-08T16:06:14.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.718+0000 7f3ec40e0700 1 -- 172.21.15.170:0/785498051 wait complete. 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.719+0000 7f3ec40e0700 1 Processor -- start 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.719+0000 7f3ec40e0700 1 -- start start 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc069560 0x7f3ebc14d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ebc159890 0x7f3ebc0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b62e0 con 0x7f3ebc159c60 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b56d0 con 0x7f3ebc159890 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec40e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0b59b0 con 0x7f3ebc069560 2023-10-08T16:06:14.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44936/0 (socket says 172.21.15.170:44936) 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 -- 172.21.15.170:0/1199109208 learned_addr learned my addr 172.21.15.170:0/1199109208 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec1e7c700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc069560 0x7f3ebc14d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec167b700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ebc159890 0x7f3ebc0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 -- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc069560 msgr2=0x7f3ebc14d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc069560 0x7f3ebc14d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 -- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ebc159890 msgr2=0x7f3ebc0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ebc159890 0x7f3ebc0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.720+0000 7f3ec267d700 1 -- 172.21.15.170:0/1199109208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8009420 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3ec267d700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cc60 tx=0x7f3eb4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400a8b0 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb400aa30 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc0b5c90 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb400adf0 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.721+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc0b53c0 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.722+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc0b6490 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.722+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3eb4007300 con 0x7f3ebc159c60 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.723+0000 7f3eb1ffb700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 0x7f3ea0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:14.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.723+0000 7f3ec1e7c700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 0x7f3ea0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.723+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3eb4084530 con 0x7f3ebc159c60 2023-10-08T16:06:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.723+0000 7f3ec1e7c700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 0x7f3ea0069470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009860 tx=0x7f3eb8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:14.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.727+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ebc0b6490 con 0x7f3ebc159c60 2023-10-08T16:06:14.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.913+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3ebc04c3c0 con 0x7f3ebc159c60 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.914+0000 7f3eb1ffb700 1 -- 172.21.15.170:0/1199109208 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3eb4055610 con 0x7f3ebc159c60 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:06:14.915 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:06:14.916 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:14.916 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 msgr2=0x7f3ea0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 0x7f3ea0069470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009860 tx=0x7f3eb8017000 comp rx=0 tx=0).stop 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 msgr2=0x7f3ebc15d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400cc60 tx=0x7f3eb4001ae0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 shutdown_connections 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ea0066fc0 0x7f3ea0069470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ebc159c60 0x7f3ebc15d250 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ebc069560 0x7f3ebc14d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 --2- 172.21.15.170:0/1199109208 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ebc159890 0x7f3ebc0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.915+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 >> 172.21.15.170:0/1199109208 conn(0x7f3ebc068e20 msgr2=0x7f3ebc154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.916+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 shutdown_connections 2023-10-08T16:06:14.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:14.916+0000 7f3ec40e0700 1 -- 172.21.15.170:0/1199109208 wait complete. 2023-10-08T16:06:14.929 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:06:14.929 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:06:15.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.047+0000 7f7f654c2700 1 Processor -- start 2023-10-08T16:06:15.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.047+0000 7f7f654c2700 1 -- start start 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f654c2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7f60069a50 0x7f7f600c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f654c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f6015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f654c2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f600ac030 con 0x7f7f60069e20 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f654c2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f60068aa0 con 0x7f7f60069a50 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f654c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600ab820 con 0x7f7f600c8490 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7f60069a50 0x7f7f600c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7f60069e20 0x7f7f60068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58198/0 (socket says 172.21.15.170:58198) 2023-10-08T16:06:15.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5f7fe700 1 -- 172.21.15.170:0/520837238 learned_addr learned my addr 172.21.15.170:0/520837238 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5e7fc700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f6015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 -- 172.21.15.170:0/520837238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 msgr2=0x7f7f6015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f6015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 -- 172.21.15.170:0/520837238 >> v1:172.21.15.170:6789/0 conn(0x7f7f60069e20 legacy=0x7f7f60068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 -- 172.21.15.170:0/520837238 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6006dbb0 con 0x7f7f60069a50 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5dffb700 1 -- 172.21.15.170:0/520837238 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178744397 0 0) 0x7f7f50001110 con 0x7f7f60069e20 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5effd700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069a50 0x7f7f600c6ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009e90 tx=0x7f7f58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47573e864437f49 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5dffb700 1 -- 172.21.15.170:0/520837238 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5800e070 con 0x7f7f60069a50 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5dffb700 1 -- 172.21.15.170:0/520837238 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5800bbf0 con 0x7f7f60069a50 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.048+0000 7f7f5dffb700 1 -- 172.21.15.170:0/520837238 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58012430 con 0x7f7f60069a50 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 -- 172.21.15.170:0/520837238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069a50 msgr2=0x7f7f600c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069a50 0x7f7f600c6ac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009e90 tx=0x7f7f58009780 comp rx=0 tx=0).stop 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 -- 172.21.15.170:0/520837238 shutdown_connections 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069a50 0x7f7f600c6ac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 --2- 172.21.15.170:0/520837238 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f6015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 -- 172.21.15.170:0/520837238 >> 172.21.15.170:0/520837238 conn(0x7f7f600c8f90 msgr2=0x7f7f600c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 -- 172.21.15.170:0/520837238 shutdown_connections 2023-10-08T16:06:15.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.049+0000 7f7f654c2700 1 -- 172.21.15.170:0/520837238 wait complete. 2023-10-08T16:06:15.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.050+0000 7f7f654c2700 1 Processor -- start 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.050+0000 7f7f654c2700 1 -- start start 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 0x7f7f60169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f600c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600c6e50 con 0x7f7f60069e20 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600c46b0 con 0x7f7f60069a50 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f654c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6016a020 con 0x7f7f600c8490 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44954/0 (socket says 172.21.15.170:44954) 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 -- 172.21.15.170:0/3213772141 learned_addr learned my addr 172.21.15.170:0/3213772141 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:15.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5f7fe700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f600c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5effd700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 0x7f7f60169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 -- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 msgr2=0x7f7f600c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f600c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 -- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 msgr2=0x7f7f60169ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 0x7f7f60169ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5e7fc700 1 -- 172.21.15.170:0/3213772141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58009a20 con 0x7f7f60069e20 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.051+0000 7f7f5effd700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 0x7f7f60169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f5e7fc700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d890 tx=0x7f7f48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f480125f0 con 0x7f7f60069e20 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f48012750 con 0x7f7f60069e20 2023-10-08T16:06:15.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f60167bd0 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f48012b10 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.052+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f600c3d80 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.053+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f600718b0 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.053+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7f4801e460 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.053+0000 7f7f56ffd700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 0x7f7f40069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.054+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7f48084880 con 0x7f7f60069e20 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.054+0000 7f7f5effd700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 0x7f7f40069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.054+0000 7f7f5effd700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 0x7f7f40069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009fd0 tx=0x7f7f58019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.058+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f48051d50 con 0x7f7f60069e20 2023-10-08T16:06:15.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.238+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7f600c6f90 con 0x7f7f60069e20 2023-10-08T16:06:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.544+0000 7f7f56ffd700 1 -- 172.21.15.170:0/3213772141 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f48055370 con 0x7f7f60069e20 2023-10-08T16:06:15.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 msgr2=0x7f7f40069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 0x7f7f40069470 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009fd0 tx=0x7f7f58019000 comp rx=0 tx=0).stop 2023-10-08T16:06:15.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 msgr2=0x7f7f60167690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800d890 tx=0x7f7f48008040 comp rx=0 tx=0).stop 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 shutdown_connections 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f600c8490 0x7f7f600c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f60069a50 0x7f7f60169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f40066fc0 0x7f7f40069470 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 --2- 172.21.15.170:0/3213772141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f60069e20 0x7f7f60167690 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 >> 172.21.15.170:0/3213772141 conn(0x7f7f600c8f90 msgr2=0x7f7f6006e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 shutdown_connections 2023-10-08T16:06:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.547+0000 7f7f654c2700 1 -- 172.21.15.170:0/3213772141 wait complete. 2023-10-08T16:06:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:06:15.563 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:06:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.683+0000 7fb100467700 1 Processor -- start 2023-10-08T16:06:15.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.683+0000 7fb100467700 1 -- start start 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.684+0000 7fb100467700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0f80cbb90 0x7fb0f80af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.684+0000 7fb100467700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f80af860 con 0x7fb0f8159950 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.684+0000 7fb100467700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f80ce380 con 0x7fb0f8069c70 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.684+0000 7fb100467700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f80cec20 con 0x7fb0f80cbb90 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fda02700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fb0f8069c70 0x7fb0f80cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:56896/0 (socket says 172.21.15.170:56896) 2023-10-08T16:06:15.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fda02700 1 -- 172.21.15.170:0/668826917 learned_addr learned my addr 172.21.15.170:0/668826917 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fe203700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb0f8159950 0x7fb0f8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51944/0 (socket says 172.21.15.170:51944) 2023-10-08T16:06:15.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fea04700 1 --2- 172.21.15.170:0/668826917 >> v2:172.21.15.191:3300/0 conn(0x7fb0f80cbb90 0x7fb0f80af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fea04700 1 -- 172.21.15.170:0/668826917 >> v1:172.21.15.191:6789/0 conn(0x7fb0f8069c70 legacy=0x7fb0f80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fea04700 1 -- 172.21.15.170:0/668826917 >> v1:172.21.15.170:6789/0 conn(0x7fb0f8159950 legacy=0x7fb0f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fea04700 1 -- 172.21.15.170:0/668826917 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f80cbfb0 con 0x7fb0f80cbb90 2023-10-08T16:06:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fd201700 1 -- 172.21.15.170:0/668826917 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406370818 0 0) 0x7fb0f80af860 con 0x7fb0f8159950 2023-10-08T16:06:15.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.685+0000 7fb0fea04700 1 --2- 172.21.15.170:0/668826917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f80af320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009de0 tx=0x7fb0f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c659134beaa2d3ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb0fd201700 1 -- 172.21.15.170:0/668826917 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000e070 con 0x7fb0f80cbb90 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb0fd201700 1 -- 172.21.15.170:0/668826917 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000bbb0 con 0x7fb0f80cbb90 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb0fd201700 1 -- 172.21.15.170:0/668826917 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0012430 con 0x7fb0f80cbb90 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 -- 172.21.15.170:0/668826917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 msgr2=0x7fb0f80af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 --2- 172.21.15.170:0/668826917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f80af320 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009de0 tx=0x7fb0f0009780 comp rx=0 tx=0).stop 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 -- 172.21.15.170:0/668826917 shutdown_connections 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 --2- 172.21.15.170:0/668826917 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f80af320 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 -- 172.21.15.170:0/668826917 >> 172.21.15.170:0/668826917 conn(0x7fb0f8068e00 msgr2=0x7fb0f80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 -- 172.21.15.170:0/668826917 shutdown_connections 2023-10-08T16:06:15.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.686+0000 7fb100467700 1 -- 172.21.15.170:0/668826917 wait complete. 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 Processor -- start 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 -- start start 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f8150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0f8159950 0x7fb0f81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f814f660 con 0x7fb0f8069c70 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f816a7b0 con 0x7fb0f80cbb90 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.688+0000 7fb100467700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f81573f0 con 0x7fb0f8159950 2023-10-08T16:06:15.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44088/0 (socket says 172.21.15.170:44088) 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 -- 172.21.15.170:0/2865750652 learned_addr learned my addr 172.21.15.170:0/2865750652 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fda02700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f8150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fea04700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0f8159950 0x7fb0f81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 -- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0f8159950 msgr2=0x7fb0f81513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0f8159950 0x7fb0f81513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 -- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 msgr2=0x7fb0f8150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f8150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 -- 172.21.15.170:0/2865750652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f0009a20 con 0x7fb0f8069c70 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0fe203700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c560 tx=0x7fb0f4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4005a70 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4005bd0 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f81576d0 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40070d0 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.689+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f816aa70 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.690+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f80718b0 con 0x7fb0f8069c70 2023-10-08T16:06:15.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.691+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fb0f4007480 con 0x7fb0f8069c70 2023-10-08T16:06:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.691+0000 7fb0ee7fc700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 0x7fb0dc069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.691+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fb0f4083900 con 0x7fb0f8069c70 2023-10-08T16:06:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.691+0000 7fb0fda02700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 0x7fb0dc069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:15.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.692+0000 7fb0fda02700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 0x7fb0dc069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0f815e3a0 tx=0x7fb0e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:15.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.696+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f4050d60 con 0x7fb0f8069c70 2023-10-08T16:06:15.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:15.880+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb0f80afa90 con 0x7fb0f8069c70 2023-10-08T16:06:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.553+0000 7fb0ee7fc700 1 -- 172.21.15.170:0/2865750652 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb0f4017070 con 0x7fb0f8069c70 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.563+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 msgr2=0x7fb0dc069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.563+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 0x7fb0dc069470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0f815e3a0 tx=0x7fb0e8009040 comp rx=0 tx=0).stop 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.563+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 msgr2=0x7fb0f814f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.563+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c560 tx=0x7fb0f4007f60 comp rx=0 tx=0).stop 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 shutdown_connections 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0f8159950 0x7fb0f81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0dc066fc0 0x7fb0dc069470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0f8069c70 0x7fb0f814f120 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 --2- 172.21.15.170:0/2865750652 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0f80cbb90 0x7fb0f8150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.565+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 >> 172.21.15.170:0/2865750652 conn(0x7fb0f8068e00 msgr2=0x7fb0f80b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.566+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 shutdown_connections 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.566+0000 7fb100467700 1 -- 172.21.15.170:0/2865750652 wait complete. 2023-10-08T16:06:16.612 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:06:16.612 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:06:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.730+0000 7f67c5ee5700 1 Processor -- start 2023-10-08T16:06:16.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.730+0000 7f67c5ee5700 1 -- start start 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67c5ee5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f67c0159c60 0x7f67c015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67c5ee5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c015db10 0x7f67c0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67c5ee5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67c015a160 con 0x7f67c0159890 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67c5ee5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67c015df50 con 0x7f67c0159c60 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67c5ee5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00d13a0 con 0x7f67c015db10 2023-10-08T16:06:16.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67beffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f67c0159c60 0x7f67c015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67bf7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f67c0159890 0x7f67c015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51954/0 (socket says 172.21.15.170:51954) 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67bf7fe700 1 -- 172.21.15.170:0/76952009 learned_addr learned my addr 172.21.15.170:0/76952009 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.731+0000 7f67bffff700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c015db10 0x7f67c0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67beffd700 1 -- 172.21.15.170:0/76952009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c015db10 msgr2=0x7f67c0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67beffd700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c015db10 0x7f67c0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67beffd700 1 -- 172.21.15.170:0/76952009 >> v1:172.21.15.170:6789/0 conn(0x7f67c0159890 legacy=0x7f67c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67be7fc700 1 -- 172.21.15.170:0/76952009 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029846575 0 0) 0x7f67c015a160 con 0x7f67c0159890 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67beffd700 1 -- 172.21.15.170:0/76952009 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c015e140 con 0x7f67c0159c60 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67beffd700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c0159c60 0x7f67c015d5d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67b40098e0 tx=0x7f67b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c722c7c25c6fd82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67be7fc700 1 -- 172.21.15.170:0/76952009 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4009e80 con 0x7f67c0159c60 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67be7fc700 1 -- 172.21.15.170:0/76952009 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b400b860 con 0x7f67c0159c60 2023-10-08T16:06:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67be7fc700 1 -- 172.21.15.170:0/76952009 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b400bbe0 con 0x7f67c0159c60 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 -- 172.21.15.170:0/76952009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c0159c60 msgr2=0x7f67c015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c0159c60 0x7f67c015d5d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67b40098e0 tx=0x7f67b40091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 -- 172.21.15.170:0/76952009 shutdown_connections 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c015db10 0x7f67c0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/76952009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c0159c60 0x7f67c015d5d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 -- 172.21.15.170:0/76952009 >> 172.21.15.170:0/76952009 conn(0x7f67c0068e20 msgr2=0x7f67c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 -- 172.21.15.170:0/76952009 shutdown_connections 2023-10-08T16:06:16.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.732+0000 7f67c5ee5700 1 -- 172.21.15.170:0/76952009 wait complete. 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 Processor -- start 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 -- start start 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c0159890 0x7f67c00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c0159c60 0x7f67c0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c00cf640 con 0x7f67c015db10 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c014ab10 con 0x7f67c0159890 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67c5ee5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c014b3b0 con 0x7f67c0159c60 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.734+0000 7f67bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44106/0 (socket says 172.21.15.170:44106) 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 -- 172.21.15.170:0/2094925258 learned_addr learned my addr 172.21.15.170:0/2094925258 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67beffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c0159c60 0x7f67c0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bf7fe700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c0159890 0x7f67c00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 -- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c0159c60 msgr2=0x7f67c0151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c0159c60 0x7f67c0151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 -- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c0159890 msgr2=0x7f67c00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c0159890 0x7f67c00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 -- 172.21.15.170:0/2094925258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b4009420 con 0x7f67c015db10 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67bffff700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d860 tx=0x7f67b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b80125f0 con 0x7f67c015db10 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b8012790 con 0x7f67c015db10 2023-10-08T16:06:16.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b8012b50 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c00cd740 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.735+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c014ad60 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.736+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c004c3c0 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.737+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f67b8013480 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.737+0000 7f67a77fe700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 0x7f67ac069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.737+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f67b80847c0 con 0x7f67c015db10 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.737+0000 7f67bf7fe700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 0x7f67ac069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:16.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.738+0000 7f67bf7fe700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 0x7f67ac069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67b0002250 tx=0x7f67b0001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:16.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.742+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b8051aa0 con 0x7f67c015db10 2023-10-08T16:06:16.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:16.965+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f67c014b4f0 con 0x7f67c015db10 2023-10-08T16:06:17.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.554+0000 7f67a77fe700 1 -- 172.21.15.170:0/2094925258 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7f67b80550c0 con 0x7f67c015db10 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 msgr2=0x7f67ac069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 0x7f67ac069470 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67b0002250 tx=0x7f67b0001a20 comp rx=0 tx=0).stop 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 msgr2=0x7f67c01523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67b800d860 tx=0x7f67b8008040 comp rx=0 tx=0).stop 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 shutdown_connections 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67ac066fc0 0x7f67ac069470 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67c015db10 0x7f67c01523d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67c0159890 0x7f67c00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 --2- 172.21.15.170:0/2094925258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67c0159c60 0x7f67c0151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.557+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 >> 172.21.15.170:0/2094925258 conn(0x7f67c0068e20 msgr2=0x7f67c014d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.558+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 shutdown_connections 2023-10-08T16:06:17.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.559+0000 7f67c5ee5700 1 -- 172.21.15.170:0/2094925258 wait complete. 2023-10-08T16:06:17.575 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 10 and data pool 11 2023-10-08T16:06:17.588 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:17.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.701+0000 7f730b3fd700 1 Processor -- start 2023-10-08T16:06:17.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.701+0000 7f730b3fd700 1 -- start start 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730b3fd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 0x7f73040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730b3fd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f730415d230 0x7f730415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730b3fd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730415db70 con 0x7f7304159420 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730b3fd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73040b11d0 con 0x7f730415d230 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730b3fd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73040b14b0 con 0x7f73041597f0 2023-10-08T16:06:17.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730abfc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f730415d230 0x7f730415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730abfc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f730415d230 0x7f730415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44120/0 (socket says 172.21.15.170:44120) 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730abfc700 1 -- 172.21.15.170:0/4146482506 learned_addr learned my addr 172.21.15.170:0/4146482506 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730a3fb700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7304159420 0x7f73040b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51964/0 (socket says 172.21.15.170:51964) 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f7309bfa700 1 --2- 172.21.15.170:0/4146482506 >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 0x7f73040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.702+0000 7f730abfc700 1 -- 172.21.15.170:0/4146482506 >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 msgr2=0x7f73040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730abfc700 1 --2- 172.21.15.170:0/4146482506 >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 0x7f73040b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730abfc700 1 -- 172.21.15.170:0/4146482506 >> v1:172.21.15.170:6789/0 conn(0x7f7304159420 legacy=0x7f73040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f73093f9700 1 -- 172.21.15.170:0/4146482506 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451755165 0 0) 0x7f730415db70 con 0x7f7304159420 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730abfc700 1 -- 172.21.15.170:0/4146482506 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73040b1790 con 0x7f730415d230 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f7309bfa700 1 --2- 172.21.15.170:0/4146482506 >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 0x7f73040b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:17.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730abfc700 1 --2- 172.21.15.170:0/4146482506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415d230 0x7f730415d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7304089750 tx=0x7f72fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8137b6c8df29ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f73093f9700 1 -- 172.21.15.170:0/4146482506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc00e070 con 0x7f730415d230 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f73093f9700 1 -- 172.21.15.170:0/4146482506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc00ba50 con 0x7f730415d230 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f73093f9700 1 -- 172.21.15.170:0/4146482506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0092e0 con 0x7f730415d230 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 -- 172.21.15.170:0/4146482506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415d230 msgr2=0x7f730415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 --2- 172.21.15.170:0/4146482506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415d230 0x7f730415d630 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7304089750 tx=0x7f72fc009840 comp rx=0 tx=0).stop 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 -- 172.21.15.170:0/4146482506 shutdown_connections 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 --2- 172.21.15.170:0/4146482506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f730415d230 0x7f730415d630 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 --2- 172.21.15.170:0/4146482506 >> v2:172.21.15.191:3300/0 conn(0x7f73041597f0 0x7f73040b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 -- 172.21.15.170:0/4146482506 >> 172.21.15.170:0/4146482506 conn(0x7f73040d0660 msgr2=0x7f73040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.703+0000 7f730b3fd700 1 -- 172.21.15.170:0/4146482506 shutdown_connections 2023-10-08T16:06:17.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.704+0000 7f730b3fd700 1 -- 172.21.15.170:0/4146482506 wait complete. 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 Processor -- start 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 -- start start 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159420 0x7f73040696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f730415d230 0x7f7304169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304150c80 con 0x7f73041597f0 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304150000 con 0x7f7304159420 2023-10-08T16:06:17.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730b3fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73041502e0 con 0x7f730415d230 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f7309bfa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f7309bfa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44122/0 (socket says 172.21.15.170:44122) 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f7309bfa700 1 -- 172.21.15.170:0/3935293081 learned_addr learned my addr 172.21.15.170:0/3935293081 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.705+0000 7f730abfc700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f730415d230 0x7f7304169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f730a3fb700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159420 0x7f73040696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 -- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f730415d230 msgr2=0x7f7304169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f730415d230 0x7f7304169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 -- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159420 msgr2=0x7f73040696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159420 0x7f73040696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 -- 172.21.15.170:0/3935293081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc009ae0 con 0x7f73041597f0 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f7309bfa700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e8a0 tx=0x7f72f4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:17.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400c430 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f400c590 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73041505c0 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f400c950 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.706+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730414fc90 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.707+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730404c3c0 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.707+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f72f400cd00 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.708+0000 7f72fa7fc700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 0x7f72e8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.708+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72f4084490 con 0x7f73041597f0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.708+0000 7f730a3fb700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 0x7f72e8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:17.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.708+0000 7f730a3fb700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 0x7f72e8069470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f73000011e0 tx=0x7f7300007bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:17.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.713+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72f4051600 con 0x7f73041597f0 2023-10-08T16:06:17.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.895+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73040b1a70 con 0x7f73041597f0 2023-10-08T16:06:17.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.896+0000 7f72fa7fc700 1 -- 172.21.15.170:0/3935293081 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f72f4054c20 con 0x7f73041597f0 2023-10-08T16:06:17.896 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:17.897 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:16.967983+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 msgr2=0x7f72e8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 0x7f72e8069470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f73000011e0 tx=0x7f7300007bf0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 msgr2=0x7f7304069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f72f400e8a0 tx=0x7f72f4008040 comp rx=0 tx=0).stop 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 shutdown_connections 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f730415d230 0x7f7304169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7304159420 0x7f73040696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f72e8066fc0 0x7f72e8069470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 --2- 172.21.15.170:0/3935293081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73041597f0 0x7f7304069bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:17.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 >> 172.21.15.170:0/3935293081 conn(0x7f73040d0660 msgr2=0x7f7304156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:17.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 shutdown_connections 2023-10-08T16:06:17.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:17.898+0000 7f730b3fd700 1 -- 172.21.15.170:0/3935293081 wait complete. 2023-10-08T16:06:17.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:18.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.025+0000 7fefa4d0c700 1 Processor -- start 2023-10-08T16:06:18.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.025+0000 7fefa4d0c700 1 -- start start 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fefa4d0c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fefa0159c60 0x7fefa015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fefa4d0c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fefa015db10 0x7fefa0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fefa4d0c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa015a160 con 0x7fefa0159890 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fefa4d0c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa015df50 con 0x7fefa0159c60 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fefa4d0c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa00d13a0 con 0x7fefa015db10 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef95d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fefa0159c60 0x7fefa015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef9e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fefa0159890 0x7fefa015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51970/0 (socket says 172.21.15.170:51970) 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef9e59c700 1 -- 172.21.15.170:0/2639114761 learned_addr learned my addr 172.21.15.170:0/2639114761 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:18.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef9ed9d700 1 --2- 172.21.15.170:0/2639114761 >> v2:172.21.15.191:3300/0 conn(0x7fefa015db10 0x7fefa0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef95d9b700 1 -- 172.21.15.170:0/2639114761 >> v2:172.21.15.191:3300/0 conn(0x7fefa015db10 msgr2=0x7fefa0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef95d9b700 1 --2- 172.21.15.170:0/2639114761 >> v2:172.21.15.191:3300/0 conn(0x7fefa015db10 0x7fefa0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef95d9b700 1 -- 172.21.15.170:0/2639114761 >> v1:172.21.15.170:6789/0 conn(0x7fefa0159890 legacy=0x7fefa015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef95d9b700 1 -- 172.21.15.170:0/2639114761 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa015e140 con 0x7fefa0159c60 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.026+0000 7fef9dd9b700 1 -- 172.21.15.170:0/2639114761 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342657830 0 0) 0x7fefa015a160 con 0x7fefa0159890 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fef95d9b700 1 --2- 172.21.15.170:0/2639114761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa0159c60 0x7fefa015d5d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fefa006a890 tx=0x7fef880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc7a1c0b97b466f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fef9dd9b700 1 -- 172.21.15.170:0/2639114761 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88011630 con 0x7fefa0159c60 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fef9dd9b700 1 -- 172.21.15.170:0/2639114761 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88011790 con 0x7fefa0159c60 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fef9dd9b700 1 -- 172.21.15.170:0/2639114761 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef88011b30 con 0x7fefa0159c60 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 -- 172.21.15.170:0/2639114761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa0159c60 msgr2=0x7fefa015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/2639114761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa0159c60 0x7fefa015d5d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fefa006a890 tx=0x7fef880091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 -- 172.21.15.170:0/2639114761 shutdown_connections 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/2639114761 >> v2:172.21.15.191:3300/0 conn(0x7fefa015db10 0x7fefa0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/2639114761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa0159c60 0x7fefa015d5d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 -- 172.21.15.170:0/2639114761 >> 172.21.15.170:0/2639114761 conn(0x7fefa0068e20 msgr2=0x7fefa00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 -- 172.21.15.170:0/2639114761 shutdown_connections 2023-10-08T16:06:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.027+0000 7fefa4d0c700 1 -- 172.21.15.170:0/2639114761 wait complete. 2023-10-08T16:06:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 Processor -- start 2023-10-08T16:06:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 -- start start 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa0159890 0x7fefa014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa0159c60 0x7fefa00cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00b1fb0 con 0x7fefa015db10 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c5ff0 con 0x7fefa0159890 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fefa4d0c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c56c0 con 0x7fefa0159c60 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fef9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fef9ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44148/0 (socket says 172.21.15.170:44148) 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.029+0000 7fef9ed9d700 1 -- 172.21.15.170:0/1587033132 learned_addr learned my addr 172.21.15.170:0/1587033132 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef95d9b700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa0159c60 0x7fefa00cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9e59c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa0159890 0x7fefa014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 -- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa0159c60 msgr2=0x7fefa00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa0159c60 0x7fefa00cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 -- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa0159890 msgr2=0x7fefa014b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa0159890 0x7fefa014b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 -- 172.21.15.170:0/1587033132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef88009420 con 0x7fefa015db10 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef9ed9d700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fef98009df0 tx=0x7fef9800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800caa0 con 0x7fefa015db10 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9800f040 con 0x7fefa015db10 2023-10-08T16:06:18.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00c59a0 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800ce70 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.030+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa014bdd0 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.031+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa00c6160 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.031+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fef98014690 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.032+0000 7fef96ffd700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 0x7fef80069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.032+0000 7fef9e59c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 0x7fef80069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.032+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef98084e70 con 0x7fefa015db10 2023-10-08T16:06:18.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.032+0000 7fef9e59c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 0x7fef80069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef90007fd0 tx=0x7fef90007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.036+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef98055a70 con 0x7fefa015db10 2023-10-08T16:06:18.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.218+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa004c3c0 con 0x7fefa015db10 2023-10-08T16:06:18.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.219+0000 7fef96ffd700 1 -- 172.21.15.170:0/1587033132 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+15414 (secure 0 0 0) 0x7fef98055600 con 0x7fefa015db10 2023-10-08T16:06:18.220 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:18.220 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":62,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:16.967983+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 msgr2=0x7fef80069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 0x7fef80069470 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fef90007fd0 tx=0x7fef90007d30 comp rx=0 tx=0).stop 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 msgr2=0x7fefa0151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fef98009df0 tx=0x7fef9800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 shutdown_connections 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef80066fc0 0x7fef80069470 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefa015db10 0x7fefa0151d80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefa0159890 0x7fefa014b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 --2- 172.21.15.170:0/1587033132 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefa0159c60 0x7fefa00cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 >> 172.21.15.170:0/1587033132 conn(0x7fefa0068e20 msgr2=0x7fefa01573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 shutdown_connections 2023-10-08T16:06:18.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.221+0000 7fefa4d0c700 1 -- 172.21.15.170:0/1587033132 wait complete. 2023-10-08T16:06:18.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:06:18.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 Processor -- start 2023-10-08T16:06:18.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 -- start start 2023-10-08T16:06:18.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff680159890 0x7ff68015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68015a160 con 0x7ff680069560 2023-10-08T16:06:18.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800d13a0 con 0x7ff680159c60 2023-10-08T16:06:18.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.351+0000 7ff685822700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68015e140 con 0x7ff680159890 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff680069560 0x7ff6800ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51986/0 (socket says 172.21.15.170:51986) 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67f7fe700 1 -- 172.21.15.170:0/3203499475 learned_addr learned my addr 172.21.15.170:0/3203499475 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67effd700 1 --2- 172.21.15.170:0/3203499475 >> v2:172.21.15.191:3300/0 conn(0x7ff680159890 0x7ff68015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616425184 0 0) 0x7ff68015a160 con 0x7ff680069560 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c003570 con 0x7ff680069560 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4020345021 0 0) 0x7ff66c003570 con 0x7ff680069560 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 >> v2:172.21.15.191:3300/0 conn(0x7ff680159890 msgr2=0x7ff68015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 --2- 172.21.15.170:0/3203499475 >> v2:172.21.15.191:3300/0 conn(0x7ff680159890 0x7ff68015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 >> v1:172.21.15.191:6789/0 conn(0x7ff680159c60 legacy=0x7ff68015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68015a3d0 con 0x7ff680069560 2023-10-08T16:06:18.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7ff6780017f0 con 0x7ff680069560 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 >> v1:172.21.15.170:6789/0 conn(0x7ff680069560 legacy=0x7ff6800ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff66c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff66c004810 0x7ff66c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff66c0051c0 0x7ff66c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68015e140 con 0x7ff66c003a60 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c00c0d0 con 0x7ff66c004810 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c00c290 con 0x7ff66c0051c0 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67effd700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff66c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff6767fc700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff66c004810 0x7ff66c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.352+0000 7ff67f7fe700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff66c0051c0 0x7ff66c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 -- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff66c0051c0 msgr2=0x7ff66c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:18.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff66c0051c0 0x7ff66c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 -- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff66c004810 msgr2=0x7ff66c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff66c004810 0x7ff66c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 -- 172.21.15.170:0/3203499475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c003570 con 0x7ff66c003a60 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67f7fe700 1 -- 172.21.15.170:0/3203499475 reap_dead start 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67effd700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff66c003e70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff670009c00 tx=0x7ff6700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360fae36a6aefa01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000e710 con 0x7ff66c003a60 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67000e870 con 0x7ff66c003a60 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff67e7fc700 1 -- 172.21.15.170:0/3203499475 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000ec30 con 0x7ff66c003a60 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 -- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 msgr2=0x7ff66c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff66c003e70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff670009c00 tx=0x7ff6700091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 -- 172.21.15.170:0/3203499475 shutdown_connections 2023-10-08T16:06:18.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 --2- 172.21.15.170:0/3203499475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff66c003e70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 -- 172.21.15.170:0/3203499475 >> 172.21.15.170:0/3203499475 conn(0x7ff680068e20 msgr2=0x7ff6800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:18.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 -- 172.21.15.170:0/3203499475 shutdown_connections 2023-10-08T16:06:18.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.353+0000 7ff685822700 1 -- 172.21.15.170:0/3203499475 wait complete. 2023-10-08T16:06:18.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 Processor -- start 2023-10-08T16:06:18.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 -- start start 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff680151ed0 0x7ff6800cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6800cf5f0 0x7ff680169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800cfa60 con 0x7ff66c003a60 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800ce0b0 con 0x7ff6800cf5f0 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff685822700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68016e020 con 0x7ff680151ed0 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff67effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff6767fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff680151ed0 0x7ff6800cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff67effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44176/0 (socket says 172.21.15.170:44176) 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.355+0000 7ff67effd700 1 -- 172.21.15.170:0/2911077080 learned_addr learned my addr 172.21.15.170:0/2911077080 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:18.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67f7fe700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6800cf5f0 0x7ff680169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 -- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff680151ed0 msgr2=0x7ff6800cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff680151ed0 0x7ff6800cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 -- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6800cf5f0 msgr2=0x7ff680169ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6800cf5f0 0x7ff680169ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 -- 172.21.15.170:0/2911077080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800cd4a0 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff67effd700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e4b0 tx=0x7ff67001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670022070 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67000e9e0 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6800cd780 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670027470 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.356+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6800cdd40 con 0x7ff66c003a60 2023-10-08T16:06:18.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.357+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68004c3c0 con 0x7ff66c003a60 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.357+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7ff67001c450 con 0x7ff66c003a60 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.358+0000 7ff6777fe700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 0x7ff660069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.358+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff670085f20 con 0x7ff66c003a60 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.358+0000 7ff6767fc700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 0x7ff660069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.358+0000 7ff6767fc700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 0x7ff660069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff680153f10 tx=0x7ff668007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:18.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.362+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6700b75a0 con 0x7ff66c003a60 2023-10-08T16:06:18.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:18.590+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff68006a5b0 con 0x7ff66c003a60 2023-10-08T16:06:19.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.567+0000 7ff6777fe700 1 -- 172.21.15.170:0/2911077080 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6700532a0 con 0x7ff66c003a60 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 msgr2=0x7ff660069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 0x7ff660069470 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff680153f10 tx=0x7ff668007d30 comp rx=0 tx=0).stop 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 msgr2=0x7ff68016dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff67000e4b0 tx=0x7ff67001a000 comp rx=0 tx=0).stop 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 shutdown_connections 2023-10-08T16:06:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6800cf5f0 0x7ff680169ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff660066fc0 0x7ff660069470 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff66c003a60 0x7ff68016dae0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 --2- 172.21.15.170:0/2911077080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff680151ed0 0x7ff6800cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 >> 172.21.15.170:0/2911077080 conn(0x7ff680068e20 msgr2=0x7ff680152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 shutdown_connections 2023-10-08T16:06:19.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.569+0000 7ff685822700 1 -- 172.21.15.170:0/2911077080 wait complete. 2023-10-08T16:06:19.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:06:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.701+0000 7f177c439700 1 Processor -- start 2023-10-08T16:06:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.701+0000 7f177c439700 1 -- start start 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177c439700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f177415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177c439700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f177415db10 0x7f1774069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177c439700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f177415a160 con 0x7f1774159890 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177c439700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17740d13a0 con 0x7f177415db10 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177c439700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177415e140 con 0x7f1774159c60 2023-10-08T16:06:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177a9d6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f177415db10 0x7f1774069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177a9d6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f177415db10 0x7f1774069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44178/0 (socket says 172.21.15.170:44178) 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f177a9d6700 1 -- 172.21.15.170:0/174403788 learned_addr learned my addr 172.21.15.170:0/174403788 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.702+0000 7f17799d4700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f177415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177a9d6700 1 -- 172.21.15.170:0/174403788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 msgr2=0x7f177415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177a9d6700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f177415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177a9d6700 1 -- 172.21.15.170:0/174403788 >> v1:172.21.15.191:6789/0 conn(0x7f1774159890 legacy=0x7f177415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177a9d6700 1 -- 172.21.15.170:0/174403788 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177415a3d0 con 0x7f177415db10 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177a9d6700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774069560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f176c0099b0 tx=0x7f176c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40aa796ded92a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f17791d3700 1 -- 172.21.15.170:0/174403788 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c00b820 con 0x7f177415db10 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f17791d3700 1 -- 172.21.15.170:0/174403788 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c00b980 con 0x7f177415db10 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f17791d3700 1 -- 172.21.15.170:0/174403788 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c00bd20 con 0x7f177415db10 2023-10-08T16:06:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 -- 172.21.15.170:0/174403788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 msgr2=0x7f1774069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774069560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f176c0099b0 tx=0x7f176c009250 comp rx=0 tx=0).stop 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 -- 172.21.15.170:0/174403788 shutdown_connections 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774069560 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 --2- 172.21.15.170:0/174403788 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f177415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.703+0000 7f177c439700 1 -- 172.21.15.170:0/174403788 >> 172.21.15.170:0/174403788 conn(0x7f1774068e20 msgr2=0x7f17740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.704+0000 7f177c439700 1 -- 172.21.15.170:0/174403788 shutdown_connections 2023-10-08T16:06:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.704+0000 7f177c439700 1 -- 172.21.15.170:0/174403788 wait complete. 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 Processor -- start 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 -- start start 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1774159890 0x7f17740b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f1774153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740b6510 con 0x7f177415db10 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740b1fd0 con 0x7f1774159890 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177c439700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740b1b40 con 0x7f1774159c60 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177a9d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177a9d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44192/0 (socket says 172.21.15.170:44192) 2023-10-08T16:06:19.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177a9d6700 1 -- 172.21.15.170:0/1864523270 learned_addr learned my addr 172.21.15.170:0/1864523270 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f17799d4700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f1774153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.705+0000 7f177a1d5700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1774159890 0x7f17740b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 -- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 msgr2=0x7f1774153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f1774153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 -- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1774159890 msgr2=0x7f17740b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1774159890 0x7f17740b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 -- 172.21.15.170:0/1864523270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0094f0 con 0x7f177415db10 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f17799d4700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f1774153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177a9d6700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f17740899c0 tx=0x7f176c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c00b820 con 0x7f177415db10 2023-10-08T16:06:19.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c020070 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17740b0f30 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c01b6a0 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.706+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17740b14f0 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.707+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177404c3c0 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.707+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f176c022030 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.708+0000 7f176a7fc700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 0x7f1758069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.708+0000 7f177a1d5700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 0x7f1758069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.708+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f176c0864c0 con 0x7f177415db10 2023-10-08T16:06:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.708+0000 7f177a1d5700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 0x7f1758069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1770007fd0 tx=0x7f17700026a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:19.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.712+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f176c053740 con 0x7f177415db10 2023-10-08T16:06:19.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:19.937+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f177404c3c0 con 0x7f177415db10 2023-10-08T16:06:20.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.577+0000 7f176a7fc700 1 -- 172.21.15.170:0/1864523270 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f176c056d60 con 0x7f177415db10 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 msgr2=0x7f1758069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 0x7f1758069470 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1770007fd0 tx=0x7f17700026a0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 msgr2=0x7f1774153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f17740899c0 tx=0x7f176c016000 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 shutdown_connections 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1758066fc0 0x7f1758069470 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177415db10 0x7f1774153e70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1774159890 0x7f17740b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 --2- 172.21.15.170:0/1864523270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1774159c60 0x7f1774153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 >> 172.21.15.170:0/1864523270 conn(0x7f1774068e20 msgr2=0x7f1774156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:20.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 shutdown_connections 2023-10-08T16:06:20.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.579+0000 7f177c439700 1 -- 172.21.15.170:0/1864523270 wait complete. 2023-10-08T16:06:20.592 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.706+0000 7feac4748700 1 Processor -- start 2023-10-08T16:06:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.706+0000 7feac4748700 1 -- start start 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac4748700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7feabc0b1420 0x7feabc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac4748700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feabc0ccf90 0x7feabc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac4748700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feabc0cc300 con 0x7feabc0cbf30 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac4748700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc156bf0 con 0x7feabc0ccf90 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac4748700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feabc0ae950 con 0x7feabc0b1420 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac2ce5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feabc0ccf90 0x7feabc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac24e4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7feabc0cbf30 0x7feabc0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51988/0 (socket says 172.21.15.170:51988) 2023-10-08T16:06:20.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.707+0000 7feac24e4700 1 -- 172.21.15.170:0/3348460849 learned_addr learned my addr 172.21.15.170:0/3348460849 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac1ce3700 1 --2- 172.21.15.170:0/3348460849 >> v2:172.21.15.191:3300/0 conn(0x7feabc0b1420 0x7feabc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac2ce5700 1 -- 172.21.15.170:0/3348460849 >> v2:172.21.15.191:3300/0 conn(0x7feabc0b1420 msgr2=0x7feabc0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac2ce5700 1 --2- 172.21.15.170:0/3348460849 >> v2:172.21.15.191:3300/0 conn(0x7feabc0b1420 0x7feabc0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac2ce5700 1 -- 172.21.15.170:0/3348460849 >> v1:172.21.15.170:6789/0 conn(0x7feabc0cbf30 legacy=0x7feabc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac2ce5700 1 -- 172.21.15.170:0/3348460849 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feabc0ad370 con 0x7feabc0ccf90 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac14e2700 1 -- 172.21.15.170:0/3348460849 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72125072 0 0) 0x7feabc0cc300 con 0x7feabc0cbf30 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac2ce5700 1 --2- 172.21.15.170:0/3348460849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc0c9740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feabc0c9280 tx=0x7feab4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558207bfc18edbcf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac14e2700 1 -- 172.21.15.170:0/3348460849 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400e070 con 0x7feabc0ccf90 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac14e2700 1 -- 172.21.15.170:0/3348460849 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab400bbf0 con 0x7feabc0ccf90 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac14e2700 1 -- 172.21.15.170:0/3348460849 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4012430 con 0x7feabc0ccf90 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac4748700 1 -- 172.21.15.170:0/3348460849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 msgr2=0x7feabc0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.708+0000 7feac4748700 1 --2- 172.21.15.170:0/3348460849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc0c9740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7feabc0c9280 tx=0x7feab4009780 comp rx=0 tx=0).stop 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 -- 172.21.15.170:0/3348460849 shutdown_connections 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 --2- 172.21.15.170:0/3348460849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc0c9740 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 --2- 172.21.15.170:0/3348460849 >> v2:172.21.15.191:3300/0 conn(0x7feabc0b1420 0x7feabc0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 -- 172.21.15.170:0/3348460849 >> 172.21.15.170:0/3348460849 conn(0x7feabc06a320 msgr2=0x7feabc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 -- 172.21.15.170:0/3348460849 shutdown_connections 2023-10-08T16:06:20.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.709+0000 7feac4748700 1 -- 172.21.15.170:0/3348460849 wait complete. 2023-10-08T16:06:20.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 Processor -- start 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 -- start start 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feabc0b1420 0x7feabc149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 0x7feabc155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc167c70 con 0x7feabc0ccf90 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0aa3a0 con 0x7feabc0b1420 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.710+0000 7feac4748700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feabc0aa680 con 0x7feabc0cbf30 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44222/0 (socket says 172.21.15.170:44222) 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 -- 172.21.15.170:0/1668303466 learned_addr learned my addr 172.21.15.170:0/1668303466 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:20.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac1ce3700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 0x7feabc155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac24e4700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feabc0b1420 0x7feabc149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 -- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 msgr2=0x7feabc155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 0x7feabc155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 -- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feabc0b1420 msgr2=0x7feabc149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feabc0b1420 0x7feabc149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 -- 172.21.15.170:0/1668303466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009a20 con 0x7feabc0ccf90 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac1ce3700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 0x7feabc155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac2ce5700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feab400b6c0 tx=0x7feab4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab401b630 con 0x7feabc0ccf90 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab401b790 con 0x7feabc0ccf90 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc0aa960 con 0x7feabc0ccf90 2023-10-08T16:06:20.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab401bb50 con 0x7feabc0ccf90 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.711+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc14a3d0 con 0x7feabc0ccf90 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.712+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc167e20 con 0x7feabc0ccf90 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.712+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7feab401c400 con 0x7feabc0ccf90 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.713+0000 7feab27fc700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 0x7feaa0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.713+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7feab4086580 con 0x7feabc0ccf90 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.713+0000 7feac24e4700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 0x7feaa0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:20.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.713+0000 7feac24e4700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 0x7feaa0069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feab80011e0 tx=0x7feab8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:20.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.717+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feabc167e20 con 0x7feabc0ccf90 2023-10-08T16:06:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.941+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feabc0cca80 con 0x7feabc0ccf90 2023-10-08T16:06:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.941+0000 7feab27fc700 1 -- 172.21.15.170:0/1668303466 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4608 (secure 0 0 0) 0x7feab40536f0 con 0x7feabc0ccf90 2023-10-08T16:06:20.942 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:20.942 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 msgr2=0x7feaa0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 0x7feaa0069470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7feab80011e0 tx=0x7feab8009040 comp rx=0 tx=0).stop 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 msgr2=0x7feabc156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7feab400b6c0 tx=0x7feab4019000 comp rx=0 tx=0).stop 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 shutdown_connections 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feaa0066fc0 0x7feaa0069470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feabc0ccf90 0x7feabc156080 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feabc0b1420 0x7feabc149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 --2- 172.21.15.170:0/1668303466 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feabc0cbf30 0x7feabc155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 >> 172.21.15.170:0/1668303466 conn(0x7feabc06a320 msgr2=0x7feabc0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 shutdown_connections 2023-10-08T16:06:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:20.943+0000 7feac4748700 1 -- 172.21.15.170:0/1668303466 wait complete. 2023-10-08T16:06:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2023-10-08T16:06:20.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:06:21.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.069+0000 7f651b942700 1 Processor -- start 2023-10-08T16:06:21.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.069+0000 7f651b942700 1 -- start start 2023-10-08T16:06:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f651b942700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6514069560 0x7f65140699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f651b942700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6514069ee0 con 0x7f6514159c60 2023-10-08T16:06:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f651b942700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651415a160 con 0x7f6514159890 2023-10-08T16:06:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f651b942700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65140d13a0 con 0x7f6514069560 2023-10-08T16:06:21.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f6518edd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6514159c60 0x7f651415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51998/0 (socket says 172.21.15.170:51998) 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.070+0000 7f6518edd700 1 -- 172.21.15.170:0/3075304649 learned_addr learned my addr 172.21.15.170:0/3075304649 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 --2- 172.21.15.170:0/3075304649 >> v2:172.21.15.191:3300/0 conn(0x7f6514069560 0x7f65140699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239251154 0 0) 0x7f6514069ee0 con 0x7f6514159c60 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6500003570 con 0x7f6514159c60 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 130131179 0 0) 0x7f6500003570 con 0x7f6514159c60 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 >> v2:172.21.15.191:3300/0 conn(0x7f6514069560 msgr2=0x7f65140699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:21.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 --2- 172.21.15.170:0/3075304649 >> v2:172.21.15.191:3300/0 conn(0x7f6514069560 0x7f65140699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 >> v1:172.21.15.191:6789/0 conn(0x7f6514159890 legacy=0x7f651415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651415e140 con 0x7f6514159c60 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6504001470 con 0x7f6514159c60 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 >> v1:172.21.15.170:6789/0 conn(0x7f6514159c60 legacy=0x7f651415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6500003a60 0x7f6500003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6500004810 0x7f6500004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 0x7f650000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65140d13a0 con 0x7f65000051c0 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650000c0d0 con 0x7f6500004810 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650000c290 con 0x7f6500003a60 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 0x7f650000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6518edd700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6500004810 0x7f6500004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f65196de700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6500003a60 0x7f6500003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 -- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6500003a60 msgr2=0x7f6500003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6500003a60 0x7f6500003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 -- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6500004810 msgr2=0x7f6500004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6500004810 0x7f6500004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 -- 172.21.15.170:0/3075304649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6500003570 con 0x7f65000051c0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.071+0000 7f6519edf700 1 -- 172.21.15.170:0/3075304649 reap_dead start 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f6519edf700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 0x7f650000ba60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f650c009c00 tx=0x7f650c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a45d816b5fb270 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00e710 con 0x7f65000051c0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c00e890 con 0x7f65000051c0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f650bfff700 1 -- 172.21.15.170:0/3075304649 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00ec50 con 0x7f65000051c0 2023-10-08T16:06:21.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 -- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 msgr2=0x7f650000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 0x7f650000ba60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f650c009c00 tx=0x7f650c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 -- 172.21.15.170:0/3075304649 shutdown_connections 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 --2- 172.21.15.170:0/3075304649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65000051c0 0x7f650000ba60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 -- 172.21.15.170:0/3075304649 >> 172.21.15.170:0/3075304649 conn(0x7f6514068e20 msgr2=0x7f65140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 -- 172.21.15.170:0/3075304649 shutdown_connections 2023-10-08T16:06:21.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.072+0000 7f651b942700 1 -- 172.21.15.170:0/3075304649 wait complete. 2023-10-08T16:06:21.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.073+0000 7f651b942700 1 Processor -- start 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 -- start start 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65000051c0 0x7f6514149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6514155dc0 0x7f6514156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65140b3590 con 0x7f65140cf5d0 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65140b43f0 con 0x7f65000051c0 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.074+0000 7f651b942700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651414a3a0 con 0x7f6514155dc0 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44236/0 (socket says 172.21.15.170:44236) 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 -- 172.21.15.170:0/2385180009 learned_addr learned my addr 172.21.15.170:0/2385180009 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:21.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f65196de700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65000051c0 0x7f6514149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6518edd700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6514155dc0 0x7f6514156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 -- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6514155dc0 msgr2=0x7f6514156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6514155dc0 0x7f6514156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 -- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65000051c0 msgr2=0x7f6514149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65000051c0 0x7f6514149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 -- 172.21.15.170:0/2385180009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65140b37e0 con 0x7f65140cf5d0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f6519edf700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f651406a300 tx=0x7f650c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c022070 con 0x7f65140cf5d0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c00e710 con 0x7f65140cf5d0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65140b3ac0 con 0x7f65140cf5d0 2023-10-08T16:06:21.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.075+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c006e50 con 0x7f65140cf5d0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.076+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65140b4080 con 0x7f65140cf5d0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.076+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f650c01c920 con 0x7f65140cf5d0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.076+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65140b4240 con 0x7f65140cf5d0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.077+0000 7f65097fa700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 0x7f64f8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.077+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f650c085ce0 con 0x7f65140cf5d0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.077+0000 7f65196de700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 0x7f64f8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.077+0000 7f65196de700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 0x7f64f8069470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6510001750 tx=0x7f6510001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.081+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f650c052fc0 con 0x7f65140cf5d0 2023-10-08T16:06:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.264+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f651404c3c0 con 0x7f65140cf5d0 2023-10-08T16:06:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.265+0000 7f65097fa700 1 -- 172.21.15.170:0/2385180009 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f650c0565e0 con 0x7f65140cf5d0 2023-10-08T16:06:21.266 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:21.266 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":63,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:18.560944+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.267+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 msgr2=0x7f64f8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.267+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 0x7f64f8069470 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6510001750 tx=0x7f6510001640 comp rx=0 tx=0).stop 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.267+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 msgr2=0x7f65140cfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.267+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f651406a300 tx=0x7f650c019000 comp rx=0 tx=0).stop 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 shutdown_connections 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64f8066fc0 0x7f64f8069470 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65140cf5d0 0x7f65140cfa40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65000051c0 0x7f6514149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 --2- 172.21.15.170:0/2385180009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6514155dc0 0x7f6514156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 >> 172.21.15.170:0/2385180009 conn(0x7f6514068e20 msgr2=0x7f65140b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 shutdown_connections 2023-10-08T16:06:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.268+0000 7f651b942700 1 -- 172.21.15.170:0/2385180009 wait complete. 2023-10-08T16:06:21.280 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:06:21.281 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:06:21.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.394+0000 7fe3ec608700 1 Processor -- start 2023-10-08T16:06:21.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.394+0000 7fe3ec608700 1 -- start start 2023-10-08T16:06:21.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.395+0000 7fe3ec608700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe3e4159890 0x7fe3e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.395+0000 7fe3ec608700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4069560 0x7fe3e40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.395+0000 7fe3ec608700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e4069ee0 con 0x7fe3e4159c60 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.395+0000 7fe3ec608700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e415a160 con 0x7fe3e4159890 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.395+0000 7fe3ec608700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e40d13a0 con 0x7fe3e4069560 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe3e4159890 0x7fe3e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe3e4159890 0x7fe3e415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44248/0 (socket says 172.21.15.170:44248) 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/3094535427 learned_addr learned my addr 172.21.15.170:0/3094535427 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3eaba5700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4069560 0x7fe3e40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/3094535427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4069560 msgr2=0x7fe3e40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4069560 0x7fe3e40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/3094535427 >> v1:172.21.15.191:6789/0 conn(0x7fe3e4159c60 legacy=0x7fe3e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/3094535427 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e415e140 con 0x7fe3e4159890 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3ea3a4700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4159890 0x7fe3e415abd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00099b0 tx=0x7fe3e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25026fa4f20dc21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3e93a2700 1 -- 172.21.15.170:0/3094535427 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000b860 con 0x7fe3e4159890 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3e93a2700 1 -- 172.21.15.170:0/3094535427 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000b9c0 con 0x7fe3e4159890 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.396+0000 7fe3e93a2700 1 -- 172.21.15.170:0/3094535427 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000bd60 con 0x7fe3e4159890 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 -- 172.21.15.170:0/3094535427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4159890 msgr2=0x7fe3e415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4159890 0x7fe3e415abd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00099b0 tx=0x7fe3e0009250 comp rx=0 tx=0).stop 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 -- 172.21.15.170:0/3094535427 shutdown_connections 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4069560 0x7fe3e40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 --2- 172.21.15.170:0/3094535427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4159890 0x7fe3e415abd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 -- 172.21.15.170:0/3094535427 >> 172.21.15.170:0/3094535427 conn(0x7fe3e4068e20 msgr2=0x7fe3e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 -- 172.21.15.170:0/3094535427 shutdown_connections 2023-10-08T16:06:21.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.397+0000 7fe3ec608700 1 -- 172.21.15.170:0/3094535427 wait complete. 2023-10-08T16:06:21.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.398+0000 7fe3ec608700 1 Processor -- start 2023-10-08T16:06:21.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.398+0000 7fe3ec608700 1 -- start start 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.398+0000 7fe3ec608700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.398+0000 7fe3ec608700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e4159890 0x7fe3e40b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4159c60 0x7fe3e41523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e416a020 con 0x7fe3e4069560 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e416a300 con 0x7fe3e4159890 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e416a5e0 con 0x7fe3e4159c60 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44258/0 (socket says 172.21.15.170:44258) 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/2304341167 learned_addr learned my addr 172.21.15.170:0/2304341167 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3eaba5700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4159c60 0x7fe3e41523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3e9ba3700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e4159890 0x7fe3e40b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4159c60 msgr2=0x7fe3e41523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4159c60 0x7fe3e41523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e4159890 msgr2=0x7fe3e40b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e4159890 0x7fe3e40b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 -- 172.21.15.170:0/2304341167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00094f0 con 0x7fe3e4069560 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ea3a4700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40899c0 tx=0x7fe3e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0024440 con 0x7fe3e4069560 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e00245c0 con 0x7fe3e4069560 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0024980 con 0x7fe3e4069560 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e416a8c0 con 0x7fe3e4069560 2023-10-08T16:06:21.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.399+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4169db0 con 0x7fe3e4069560 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.400+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e416aa80 con 0x7fe3e4069560 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.401+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fe3e001a420 con 0x7fe3e4069560 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.401+0000 7fe3da7fc700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 0x7fe3c8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.401+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe3e00862b0 con 0x7fe3e4069560 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.401+0000 7fe3e9ba3700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 0x7fe3c8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.401+0000 7fe3e9ba3700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 0x7fe3c8069470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40d00d0 tx=0x7fe3d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.405+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e0053530 con 0x7fe3e4069560 2023-10-08T16:06:21.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.586+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3e404c3c0 con 0x7fe3c8066fc0 2023-10-08T16:06:21.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.594+0000 7fe3da7fc700 1 -- 172.21.15.170:0/2304341167 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289326 (secure 0 0 0) 0x7fe3e404c3c0 con 0x7fe3c8066fc0 2023-10-08T16:06:21.598 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 msgr2=0x7fe3c8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 0x7fe3c8069470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40d00d0 tx=0x7fe3d4009040 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 msgr2=0x7fe3e41501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40899c0 tx=0x7fe3e0017000 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 shutdown_connections 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3e4159c60 0x7fe3e41523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3c8066fc0 0x7fe3c8069470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3e4069560 0x7fe3e41501e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 --2- 172.21.15.170:0/2304341167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe3e4159890 0x7fe3e40b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 >> 172.21.15.170:0/2304341167 conn(0x7fe3e4068e20 msgr2=0x7fe3e4150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 shutdown_connections 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.598+0000 7fe3ec608700 1 -- 172.21.15.170:0/2304341167 wait complete. 2023-10-08T16:06:21.601 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:06:21.610 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-10-08T16:06:20.244779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397728,"kb_used_data":7136,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752576992,"statfs":{"total":773094113280,"available":770638839808,"internally_reserved":0,"allocated":7307264,"data_stored":3748188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.700184"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580172+0000","last_change":"2023-10-08T16:06:17.568234+0000","last_active":"2023-10-08T16:06:17.580172+0000","last_peered":"2023-10-08T16:06:17.580172+0000","last_clean":"2023-10-08T16:06:17.580172+0000","last_became_active":"2023-10-08T16:06:17.567998+0000","last_became_peered":"2023-10-08T16:06:17.567998+0000","last_unstale":"2023-10-08T16:06:17.580172+0000","last_undegraded":"2023-10-08T16:06:17.580172+0000","last_fullsized":"2023-10-08T16:06:17.580172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:24.238411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587668+0000","last_change":"2023-10-08T16:06:17.583044+0000","last_active":"2023-10-08T16:06:17.587668+0000","last_peered":"2023-10-08T16:06:17.587668+0000","last_clean":"2023-10-08T16:06:17.587668+0000","last_became_active":"2023-10-08T16:06:17.582788+0000","last_became_peered":"2023-10-08T16:06:17.582788+0000","last_unstale":"2023-10-08T16:06:17.587668+0000","last_undegraded":"2023-10-08T16:06:17.587668+0000","last_fullsized":"2023-10-08T16:06:17.587668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:01:40.644016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569762+0000","last_change":"2023-10-08T16:06:17.569762+0000","last_active":"2023-10-08T16:06:17.569762+0000","last_peered":"2023-10-08T16:06:17.569762+0000","last_clean":"2023-10-08T16:06:17.569762+0000","last_became_active":"2023-10-08T16:06:17.569466+0000","last_became_peered":"2023-10-08T16:06:17.569466+0000","last_unstale":"2023-10-08T16:06:17.569762+0000","last_undegraded":"2023-10-08T16:06:17.569762+0000","last_fullsized":"2023-10-08T16:06:17.569762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:16.538590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.586595+0000","last_change":"2023-10-08T16:06:17.585962+0000","last_active":"2023-10-08T16:06:17.586595+0000","last_peered":"2023-10-08T16:06:17.586595+0000","last_clean":"2023-10-08T16:06:17.586595+0000","last_became_active":"2023-10-08T16:06:17.585690+0000","last_became_peered":"2023-10-08T16:06:17.585690+0000","last_unstale":"2023-10-08T16:06:17.586595+0000","last_undegraded":"2023-10-08T16:06:17.586595+0000","last_fullsized":"2023-10-08T16:06:17.586595+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:01.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575469+0000","last_change":"2023-10-08T16:06:17.575469+0000","last_active":"2023-10-08T16:06:17.575469+0000","last_peered":"2023-10-08T16:06:17.575469+0000","last_clean":"2023-10-08T16:06:17.575469+0000","last_became_active":"2023-10-08T16:06:17.575245+0000","last_became_peered":"2023-10-08T16:06:17.575245+0000","last_unstale":"2023-10-08T16:06:17.575469+0000","last_undegraded":"2023-10-08T16:06:17.575469+0000","last_fullsized":"2023-10-08T16:06:17.575469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:40.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587610+0000","last_change":"2023-10-08T16:06:17.582179+0000","last_active":"2023-10-08T16:06:17.587610+0000","last_peered":"2023-10-08T16:06:17.587610+0000","last_clean":"2023-10-08T16:06:17.587610+0000","last_became_active":"2023-10-08T16:06:17.581943+0000","last_became_peered":"2023-10-08T16:06:17.581943+0000","last_unstale":"2023-10-08T16:06:17.587610+0000","last_undegraded":"2023-10-08T16:06:17.587610+0000","last_fullsized":"2023-10-08T16:06:17.587610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:52.648731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588121+0000","last_change":"2023-10-08T16:06:17.580126+0000","last_active":"2023-10-08T16:06:17.588121+0000","last_peered":"2023-10-08T16:06:17.588121+0000","last_clean":"2023-10-08T16:06:17.588121+0000","last_became_active":"2023-10-08T16:06:17.579772+0000","last_became_peered":"2023-10-08T16:06:17.579772+0000","last_unstale":"2023-10-08T16:06:17.588121+0000","last_undegraded":"2023-10-08T16:06:17.588121+0000","last_fullsized":"2023-10-08T16:06:17.588121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:26.856007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587577+0000","last_change":"2023-10-08T16:06:17.582290+0000","last_active":"2023-10-08T16:06:17.587577+0000","last_peered":"2023-10-08T16:06:17.587577+0000","last_clean":"2023-10-08T16:06:17.587577+0000","last_became_active":"2023-10-08T16:06:17.581534+0000","last_became_peered":"2023-10-08T16:06:17.581534+0000","last_unstale":"2023-10-08T16:06:17.587577+0000","last_undegraded":"2023-10-08T16:06:17.587577+0000","last_fullsized":"2023-10-08T16:06:17.587577+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:06.225262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570270+0000","last_change":"2023-10-08T16:06:17.570270+0000","last_active":"2023-10-08T16:06:17.570270+0000","last_peered":"2023-10-08T16:06:17.570270+0000","last_clean":"2023-10-08T16:06:17.570270+0000","last_became_active":"2023-10-08T16:06:17.570008+0000","last_became_peered":"2023-10-08T16:06:17.570008+0000","last_unstale":"2023-10-08T16:06:17.570270+0000","last_undegraded":"2023-10-08T16:06:17.570270+0000","last_fullsized":"2023-10-08T16:06:17.570270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:20.000317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571817+0000","last_change":"2023-10-08T16:06:17.571817+0000","last_active":"2023-10-08T16:06:17.571817+0000","last_peered":"2023-10-08T16:06:17.571817+0000","last_clean":"2023-10-08T16:06:17.571817+0000","last_became_active":"2023-10-08T16:06:17.571439+0000","last_became_peered":"2023-10-08T16:06:17.571439+0000","last_unstale":"2023-10-08T16:06:17.571817+0000","last_undegraded":"2023-10-08T16:06:17.571817+0000","last_fullsized":"2023-10-08T16:06:17.571817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:22.728342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576316+0000","last_change":"2023-10-08T16:06:17.573822+0000","last_active":"2023-10-08T16:06:17.576316+0000","last_peered":"2023-10-08T16:06:17.576316+0000","last_clean":"2023-10-08T16:06:17.576316+0000","last_became_active":"2023-10-08T16:06:17.573550+0000","last_became_peered":"2023-10-08T16:06:17.573550+0000","last_unstale":"2023-10-08T16:06:17.576316+0000","last_undegraded":"2023-10-08T16:06:17.576316+0000","last_fullsized":"2023-10-08T16:06:17.576316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:45:31.341258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577121+0000","last_change":"2023-10-08T16:06:17.569314+0000","last_active":"2023-10-08T16:06:17.577121+0000","last_peered":"2023-10-08T16:06:17.577121+0000","last_clean":"2023-10-08T16:06:17.577121+0000","last_became_active":"2023-10-08T16:06:17.569111+0000","last_became_peered":"2023-10-08T16:06:17.569111+0000","last_unstale":"2023-10-08T16:06:17.577121+0000","last_undegraded":"2023-10-08T16:06:17.577121+0000","last_fullsized":"2023-10-08T16:06:17.577121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:06.775660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.585032+0000","last_change":"2023-10-08T16:06:17.585032+0000","last_active":"2023-10-08T16:06:17.585032+0000","last_peered":"2023-10-08T16:06:17.585032+0000","last_clean":"2023-10-08T16:06:17.585032+0000","last_became_active":"2023-10-08T16:06:17.584781+0000","last_became_peered":"2023-10-08T16:06:17.584781+0000","last_unstale":"2023-10-08T16:06:17.585032+0000","last_undegraded":"2023-10-08T16:06:17.585032+0000","last_fullsized":"2023-10-08T16:06:17.585032+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:43.420587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574312+0000","last_change":"2023-10-08T16:06:17.574312+0000","last_active":"2023-10-08T16:06:17.574312+0000","last_peered":"2023-10-08T16:06:17.574312+0000","last_clean":"2023-10-08T16:06:17.574312+0000","last_became_active":"2023-10-08T16:06:17.573841+0000","last_became_peered":"2023-10-08T16:06:17.573841+0000","last_unstale":"2023-10-08T16:06:17.574312+0000","last_undegraded":"2023-10-08T16:06:17.574312+0000","last_fullsized":"2023-10-08T16:06:17.574312+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:37.743529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570509+0000","last_change":"2023-10-08T16:06:17.570509+0000","last_active":"2023-10-08T16:06:17.570509+0000","last_peered":"2023-10-08T16:06:17.570509+0000","last_clean":"2023-10-08T16:06:17.570509+0000","last_became_active":"2023-10-08T16:06:17.570252+0000","last_became_peered":"2023-10-08T16:06:17.570252+0000","last_unstale":"2023-10-08T16:06:17.570509+0000","last_undegraded":"2023-10-08T16:06:17.570509+0000","last_fullsized":"2023-10-08T16:06:17.570509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:33:30.868127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583530+0000","last_change":"2023-10-08T16:06:17.583530+0000","last_active":"2023-10-08T16:06:17.583530+0000","last_peered":"2023-10-08T16:06:17.583530+0000","last_clean":"2023-10-08T16:06:17.583530+0000","last_became_active":"2023-10-08T16:06:17.583293+0000","last_became_peered":"2023-10-08T16:06:17.583293+0000","last_unstale":"2023-10-08T16:06:17.583530+0000","last_undegraded":"2023-10-08T16:06:17.583530+0000","last_fullsized":"2023-10-08T16:06:17.583530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:01.142901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576153+0000","last_change":"2023-10-08T16:06:17.576153+0000","last_active":"2023-10-08T16:06:17.576153+0000","last_peered":"2023-10-08T16:06:17.576153+0000","last_clean":"2023-10-08T16:06:17.576153+0000","last_became_active":"2023-10-08T16:06:17.575914+0000","last_became_peered":"2023-10-08T16:06:17.575914+0000","last_unstale":"2023-10-08T16:06:17.576153+0000","last_undegraded":"2023-10-08T16:06:17.576153+0000","last_fullsized":"2023-10-08T16:06:17.576153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:04:31.062457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.586782+0000","last_change":"2023-10-08T16:06:17.586108+0000","last_active":"2023-10-08T16:06:17.586782+0000","last_peered":"2023-10-08T16:06:17.586782+0000","last_clean":"2023-10-08T16:06:17.586782+0000","last_became_active":"2023-10-08T16:06:17.585852+0000","last_became_peered":"2023-10-08T16:06:17.585852+0000","last_unstale":"2023-10-08T16:06:17.586782+0000","last_undegraded":"2023-10-08T16:06:17.586782+0000","last_fullsized":"2023-10-08T16:06:17.586782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:39.944057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587946+0000","last_change":"2023-10-08T16:06:17.584639+0000","last_active":"2023-10-08T16:06:17.587946+0000","last_peered":"2023-10-08T16:06:17.587946+0000","last_clean":"2023-10-08T16:06:17.587946+0000","last_became_active":"2023-10-08T16:06:17.583921+0000","last_became_peered":"2023-10-08T16:06:17.583921+0000","last_unstale":"2023-10-08T16:06:17.587946+0000","last_undegraded":"2023-10-08T16:06:17.587946+0000","last_fullsized":"2023-10-08T16:06:17.587946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:38.013447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575797+0000","last_change":"2023-10-08T16:06:17.575797+0000","last_active":"2023-10-08T16:06:17.575797+0000","last_peered":"2023-10-08T16:06:17.575797+0000","last_clean":"2023-10-08T16:06:17.575797+0000","last_became_active":"2023-10-08T16:06:17.575463+0000","last_became_peered":"2023-10-08T16:06:17.575463+0000","last_unstale":"2023-10-08T16:06:17.575797+0000","last_undegraded":"2023-10-08T16:06:17.575797+0000","last_fullsized":"2023-10-08T16:06:17.575797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:33.723028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569742+0000","last_change":"2023-10-08T16:06:17.569742+0000","last_active":"2023-10-08T16:06:17.569742+0000","last_peered":"2023-10-08T16:06:17.569742+0000","last_clean":"2023-10-08T16:06:17.569742+0000","last_became_active":"2023-10-08T16:06:17.569484+0000","last_became_peered":"2023-10-08T16:06:17.569484+0000","last_unstale":"2023-10-08T16:06:17.569742+0000","last_undegraded":"2023-10-08T16:06:17.569742+0000","last_fullsized":"2023-10-08T16:06:17.569742+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:24.198646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572410+0000","last_change":"2023-10-08T16:06:17.572410+0000","last_active":"2023-10-08T16:06:17.572410+0000","last_peered":"2023-10-08T16:06:17.572410+0000","last_clean":"2023-10-08T16:06:17.572410+0000","last_became_active":"2023-10-08T16:06:17.572111+0000","last_became_peered":"2023-10-08T16:06:17.572111+0000","last_unstale":"2023-10-08T16:06:17.572410+0000","last_undegraded":"2023-10-08T16:06:17.572410+0000","last_fullsized":"2023-10-08T16:06:17.572410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:53:06.363027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578580+0000","last_change":"2023-10-08T16:06:17.575819+0000","last_active":"2023-10-08T16:06:17.578580+0000","last_peered":"2023-10-08T16:06:17.578580+0000","last_clean":"2023-10-08T16:06:17.578580+0000","last_became_active":"2023-10-08T16:06:17.575489+0000","last_became_peered":"2023-10-08T16:06:17.575489+0000","last_unstale":"2023-10-08T16:06:17.578580+0000","last_undegraded":"2023-10-08T16:06:17.578580+0000","last_fullsized":"2023-10-08T16:06:17.578580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:41:26.128329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584458+0000","last_change":"2023-10-08T16:06:17.582694+0000","last_active":"2023-10-08T16:06:17.584458+0000","last_peered":"2023-10-08T16:06:17.584458+0000","last_clean":"2023-10-08T16:06:17.584458+0000","last_became_active":"2023-10-08T16:06:17.582448+0000","last_became_peered":"2023-10-08T16:06:17.582448+0000","last_unstale":"2023-10-08T16:06:17.584458+0000","last_undegraded":"2023-10-08T16:06:17.584458+0000","last_fullsized":"2023-10-08T16:06:17.584458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:04.149377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581421+0000","last_change":"2023-10-08T16:06:17.581421+0000","last_active":"2023-10-08T16:06:17.581421+0000","last_peered":"2023-10-08T16:06:17.581421+0000","last_clean":"2023-10-08T16:06:17.581421+0000","last_became_active":"2023-10-08T16:06:17.581144+0000","last_became_peered":"2023-10-08T16:06:17.581144+0000","last_unstale":"2023-10-08T16:06:17.581421+0000","last_undegraded":"2023-10-08T16:06:17.581421+0000","last_fullsized":"2023-10-08T16:06:17.581421+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:13.035462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.590160+0000","last_change":"2023-10-08T16:06:17.586190+0000","last_active":"2023-10-08T16:06:17.590160+0000","last_peered":"2023-10-08T16:06:17.590160+0000","last_clean":"2023-10-08T16:06:17.590160+0000","last_became_active":"2023-10-08T16:06:17.585906+0000","last_became_peered":"2023-10-08T16:06:17.585906+0000","last_unstale":"2023-10-08T16:06:17.590160+0000","last_undegraded":"2023-10-08T16:06:17.590160+0000","last_fullsized":"2023-10-08T16:06:17.590160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:38.553596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569630+0000","last_change":"2023-10-08T16:06:17.569630+0000","last_active":"2023-10-08T16:06:17.569630+0000","last_peered":"2023-10-08T16:06:17.569630+0000","last_clean":"2023-10-08T16:06:17.569630+0000","last_became_active":"2023-10-08T16:06:17.569334+0000","last_became_peered":"2023-10-08T16:06:17.569334+0000","last_unstale":"2023-10-08T16:06:17.569630+0000","last_undegraded":"2023-10-08T16:06:17.569630+0000","last_fullsized":"2023-10-08T16:06:17.569630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:55:54.632994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570845+0000","last_change":"2023-10-08T16:06:17.570845+0000","last_active":"2023-10-08T16:06:17.570845+0000","last_peered":"2023-10-08T16:06:17.570845+0000","last_clean":"2023-10-08T16:06:17.570845+0000","last_became_active":"2023-10-08T16:06:17.570561+0000","last_became_peered":"2023-10-08T16:06:17.570561+0000","last_unstale":"2023-10-08T16:06:17.570845+0000","last_undegraded":"2023-10-08T16:06:17.570845+0000","last_fullsized":"2023-10-08T16:06:17.570845+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:44:26.187851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574170+0000","last_change":"2023-10-08T16:06:17.574170+0000","last_active":"2023-10-08T16:06:17.574170+0000","last_peered":"2023-10-08T16:06:17.574170+0000","last_clean":"2023-10-08T16:06:17.574170+0000","last_became_active":"2023-10-08T16:06:17.573932+0000","last_became_peered":"2023-10-08T16:06:17.573932+0000","last_unstale":"2023-10-08T16:06:17.574170+0000","last_undegraded":"2023-10-08T16:06:17.574170+0000","last_fullsized":"2023-10-08T16:06:17.574170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:20:54.757888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.590168+0000","last_change":"2023-10-08T16:06:17.586233+0000","last_active":"2023-10-08T16:06:17.590168+0000","last_peered":"2023-10-08T16:06:17.590168+0000","last_clean":"2023-10-08T16:06:17.590168+0000","last_became_active":"2023-10-08T16:06:17.585961+0000","last_became_peered":"2023-10-08T16:06:17.585961+0000","last_unstale":"2023-10-08T16:06:17.590168+0000","last_undegraded":"2023-10-08T16:06:17.590168+0000","last_fullsized":"2023-10-08T16:06:17.590168+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:36.461052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587902+0000","last_change":"2023-10-08T16:06:17.579178+0000","last_active":"2023-10-08T16:06:17.587902+0000","last_peered":"2023-10-08T16:06:17.587902+0000","last_clean":"2023-10-08T16:06:17.587902+0000","last_became_active":"2023-10-08T16:06:17.578917+0000","last_became_peered":"2023-10-08T16:06:17.578917+0000","last_unstale":"2023-10-08T16:06:17.587902+0000","last_undegraded":"2023-10-08T16:06:17.587902+0000","last_fullsized":"2023-10-08T16:06:17.587902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:44.200579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587601+0000","last_change":"2023-10-08T16:06:17.582483+0000","last_active":"2023-10-08T16:06:17.587601+0000","last_peered":"2023-10-08T16:06:17.587601+0000","last_clean":"2023-10-08T16:06:17.587601+0000","last_became_active":"2023-10-08T16:06:17.581826+0000","last_became_peered":"2023-10-08T16:06:17.581826+0000","last_unstale":"2023-10-08T16:06:17.587601+0000","last_undegraded":"2023-10-08T16:06:17.587601+0000","last_fullsized":"2023-10-08T16:06:17.587601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:00.269685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587433+0000","last_change":"2023-10-08T16:06:17.583441+0000","last_active":"2023-10-08T16:06:17.587433+0000","last_peered":"2023-10-08T16:06:17.587433+0000","last_clean":"2023-10-08T16:06:17.587433+0000","last_became_active":"2023-10-08T16:06:17.583226+0000","last_became_peered":"2023-10-08T16:06:17.583226+0000","last_unstale":"2023-10-08T16:06:17.587433+0000","last_undegraded":"2023-10-08T16:06:17.587433+0000","last_fullsized":"2023-10-08T16:06:17.587433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:07.326980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580034+0000","last_change":"2023-10-08T16:06:16.575523+0000","last_active":"2023-10-08T16:06:17.580034+0000","last_peered":"2023-10-08T16:06:17.580034+0000","last_clean":"2023-10-08T16:06:17.580034+0000","last_became_active":"2023-10-08T16:06:16.575219+0000","last_became_peered":"2023-10-08T16:06:16.575219+0000","last_unstale":"2023-10-08T16:06:17.580034+0000","last_undegraded":"2023-10-08T16:06:17.580034+0000","last_fullsized":"2023-10-08T16:06:17.580034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:42.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572793+0000","last_change":"2023-10-08T16:06:17.572793+0000","last_active":"2023-10-08T16:06:17.572793+0000","last_peered":"2023-10-08T16:06:17.572793+0000","last_clean":"2023-10-08T16:06:17.572793+0000","last_became_active":"2023-10-08T16:06:17.572520+0000","last_became_peered":"2023-10-08T16:06:17.572520+0000","last_unstale":"2023-10-08T16:06:17.572793+0000","last_undegraded":"2023-10-08T16:06:17.572793+0000","last_fullsized":"2023-10-08T16:06:17.572793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:11.636902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.567579+0000","last_change":"2023-10-08T16:06:16.568287+0000","last_active":"2023-10-08T16:06:17.567579+0000","last_peered":"2023-10-08T16:06:17.567579+0000","last_clean":"2023-10-08T16:06:17.567579+0000","last_became_active":"2023-10-08T16:06:16.568049+0000","last_became_peered":"2023-10-08T16:06:16.568049+0000","last_unstale":"2023-10-08T16:06:17.567579+0000","last_undegraded":"2023-10-08T16:06:17.567579+0000","last_fullsized":"2023-10-08T16:06:17.567579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:05.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588057+0000","last_change":"2023-10-08T16:06:17.585911+0000","last_active":"2023-10-08T16:06:17.588057+0000","last_peered":"2023-10-08T16:06:17.588057+0000","last_clean":"2023-10-08T16:06:17.588057+0000","last_became_active":"2023-10-08T16:06:17.585651+0000","last_became_peered":"2023-10-08T16:06:17.585651+0000","last_unstale":"2023-10-08T16:06:17.588057+0000","last_undegraded":"2023-10-08T16:06:17.588057+0000","last_fullsized":"2023-10-08T16:06:17.588057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:06.775660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584433+0000","last_change":"2023-10-08T16:06:16.567215+0000","last_active":"2023-10-08T16:06:17.584433+0000","last_peered":"2023-10-08T16:06:17.584433+0000","last_clean":"2023-10-08T16:06:17.584433+0000","last_became_active":"2023-10-08T16:06:16.566911+0000","last_became_peered":"2023-10-08T16:06:16.566911+0000","last_unstale":"2023-10-08T16:06:17.584433+0000","last_undegraded":"2023-10-08T16:06:17.584433+0000","last_fullsized":"2023-10-08T16:06:17.584433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:01:39.638715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575292+0000","last_change":"2023-10-08T16:06:17.569196+0000","last_active":"2023-10-08T16:06:17.575292+0000","last_peered":"2023-10-08T16:06:17.575292+0000","last_clean":"2023-10-08T16:06:17.575292+0000","last_became_active":"2023-10-08T16:06:17.568888+0000","last_became_peered":"2023-10-08T16:06:17.568888+0000","last_unstale":"2023-10-08T16:06:17.575292+0000","last_undegraded":"2023-10-08T16:06:17.575292+0000","last_fullsized":"2023-10-08T16:06:17.575292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:57:49.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587385+0000","last_change":"2023-10-08T16:06:16.570685+0000","last_active":"2023-10-08T16:06:17.587385+0000","last_peered":"2023-10-08T16:06:17.587385+0000","last_clean":"2023-10-08T16:06:17.587385+0000","last_became_active":"2023-10-08T16:06:16.570337+0000","last_became_peered":"2023-10-08T16:06:16.570337+0000","last_unstale":"2023-10-08T16:06:17.587385+0000","last_undegraded":"2023-10-08T16:06:17.587385+0000","last_fullsized":"2023-10-08T16:06:17.587385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:37.048819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581285+0000","last_change":"2023-10-08T16:06:17.574860+0000","last_active":"2023-10-08T16:06:17.581285+0000","last_peered":"2023-10-08T16:06:17.581285+0000","last_clean":"2023-10-08T16:06:17.581285+0000","last_became_active":"2023-10-08T16:06:17.574560+0000","last_became_peered":"2023-10-08T16:06:17.574560+0000","last_unstale":"2023-10-08T16:06:17.581285+0000","last_undegraded":"2023-10-08T16:06:17.581285+0000","last_fullsized":"2023-10-08T16:06:17.581285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:33.723028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.558927+0000","last_change":"2023-10-08T16:06:16.566783+0000","last_active":"2023-10-08T16:06:17.558927+0000","last_peered":"2023-10-08T16:06:17.558927+0000","last_clean":"2023-10-08T16:06:17.558927+0000","last_became_active":"2023-10-08T16:06:16.566483+0000","last_became_peered":"2023-10-08T16:06:16.566483+0000","last_unstale":"2023-10-08T16:06:17.558927+0000","last_undegraded":"2023-10-08T16:06:17.558927+0000","last_fullsized":"2023-10-08T16:06:17.558927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:13.463499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576349+0000","last_change":"2023-10-08T16:06:17.576349+0000","last_active":"2023-10-08T16:06:17.576349+0000","last_peered":"2023-10-08T16:06:17.576349+0000","last_clean":"2023-10-08T16:06:17.576349+0000","last_became_active":"2023-10-08T16:06:17.576002+0000","last_became_peered":"2023-10-08T16:06:17.576002+0000","last_unstale":"2023-10-08T16:06:17.576349+0000","last_undegraded":"2023-10-08T16:06:17.576349+0000","last_fullsized":"2023-10-08T16:06:17.576349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:43.420587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583615+0000","last_change":"2023-10-08T16:06:16.566809+0000","last_active":"2023-10-08T16:06:17.583615+0000","last_peered":"2023-10-08T16:06:17.583615+0000","last_clean":"2023-10-08T16:06:17.583615+0000","last_became_active":"2023-10-08T16:06:16.566496+0000","last_became_peered":"2023-10-08T16:06:16.566496+0000","last_unstale":"2023-10-08T16:06:17.583615+0000","last_undegraded":"2023-10-08T16:06:17.583615+0000","last_fullsized":"2023-10-08T16:06:17.583615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:05.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570775+0000","last_change":"2023-10-08T16:06:17.570775+0000","last_active":"2023-10-08T16:06:17.570775+0000","last_peered":"2023-10-08T16:06:17.570775+0000","last_clean":"2023-10-08T16:06:17.570775+0000","last_became_active":"2023-10-08T16:06:17.570541+0000","last_became_peered":"2023-10-08T16:06:17.570541+0000","last_unstale":"2023-10-08T16:06:17.570775+0000","last_undegraded":"2023-10-08T16:06:17.570775+0000","last_fullsized":"2023-10-08T16:06:17.570775+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:30.121237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.554848+0000","last_change":"2023-10-08T16:06:16.564774+0000","last_active":"2023-10-08T16:06:17.554848+0000","last_peered":"2023-10-08T16:06:17.554848+0000","last_clean":"2023-10-08T16:06:17.554848+0000","last_became_active":"2023-10-08T16:06:16.564485+0000","last_became_peered":"2023-10-08T16:06:16.564485+0000","last_unstale":"2023-10-08T16:06:17.554848+0000","last_undegraded":"2023-10-08T16:06:17.554848+0000","last_fullsized":"2023-10-08T16:06:17.554848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:53.343775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576232+0000","last_change":"2023-10-08T16:06:17.576232+0000","last_active":"2023-10-08T16:06:17.576232+0000","last_peered":"2023-10-08T16:06:17.576232+0000","last_clean":"2023-10-08T16:06:17.576232+0000","last_became_active":"2023-10-08T16:06:17.575675+0000","last_became_peered":"2023-10-08T16:06:17.575675+0000","last_unstale":"2023-10-08T16:06:17.576232+0000","last_undegraded":"2023-10-08T16:06:17.576232+0000","last_fullsized":"2023-10-08T16:06:17.576232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:26.856007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567386+0000","last_change":"2023-10-08T16:06:16.564164+0000","last_active":"2023-10-08T16:06:16.567386+0000","last_peered":"2023-10-08T16:06:16.567386+0000","last_clean":"2023-10-08T16:06:16.567386+0000","last_became_active":"2023-10-08T16:06:16.563910+0000","last_became_peered":"2023-10-08T16:06:16.563910+0000","last_unstale":"2023-10-08T16:06:16.567386+0000","last_undegraded":"2023-10-08T16:06:16.567386+0000","last_fullsized":"2023-10-08T16:06:16.567386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:18.369942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581337+0000","last_change":"2023-10-08T16:06:17.578005+0000","last_active":"2023-10-08T16:06:17.581337+0000","last_peered":"2023-10-08T16:06:17.581337+0000","last_clean":"2023-10-08T16:06:17.581337+0000","last_became_active":"2023-10-08T16:06:17.577737+0000","last_became_peered":"2023-10-08T16:06:17.577737+0000","last_unstale":"2023-10-08T16:06:17.581337+0000","last_undegraded":"2023-10-08T16:06:17.581337+0000","last_fullsized":"2023-10-08T16:06:17.581337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:01.759178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.568086+0000","last_change":"2023-10-08T16:06:16.563719+0000","last_active":"2023-10-08T16:06:16.568086+0000","last_peered":"2023-10-08T16:06:16.568086+0000","last_clean":"2023-10-08T16:06:16.568086+0000","last_became_active":"2023-10-08T16:06:16.563409+0000","last_became_peered":"2023-10-08T16:06:16.563409+0000","last_unstale":"2023-10-08T16:06:16.568086+0000","last_undegraded":"2023-10-08T16:06:16.568086+0000","last_fullsized":"2023-10-08T16:06:16.568086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:34.904243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587945+0000","last_change":"2023-10-08T16:06:17.586859+0000","last_active":"2023-10-08T16:06:17.587945+0000","last_peered":"2023-10-08T16:06:17.587945+0000","last_clean":"2023-10-08T16:06:17.587945+0000","last_became_active":"2023-10-08T16:06:17.586669+0000","last_became_peered":"2023-10-08T16:06:17.586669+0000","last_unstale":"2023-10-08T16:06:17.587945+0000","last_undegraded":"2023-10-08T16:06:17.587945+0000","last_fullsized":"2023-10-08T16:06:17.587945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:14.468800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575990+0000","last_change":"2023-10-08T16:06:16.568819+0000","last_active":"2023-10-08T16:06:17.575990+0000","last_peered":"2023-10-08T16:06:17.575990+0000","last_clean":"2023-10-08T16:06:17.575990+0000","last_became_active":"2023-10-08T16:06:16.568296+0000","last_became_peered":"2023-10-08T16:06:16.568296+0000","last_unstale":"2023-10-08T16:06:17.575990+0000","last_undegraded":"2023-10-08T16:06:17.575990+0000","last_fullsized":"2023-10-08T16:06:17.575990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:57.703766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587960+0000","last_change":"2023-10-08T16:06:17.583544+0000","last_active":"2023-10-08T16:06:17.587960+0000","last_peered":"2023-10-08T16:06:17.587960+0000","last_clean":"2023-10-08T16:06:17.587960+0000","last_became_active":"2023-10-08T16:06:17.583272+0000","last_became_peered":"2023-10-08T16:06:17.583272+0000","last_unstale":"2023-10-08T16:06:17.587960+0000","last_undegraded":"2023-10-08T16:06:17.587960+0000","last_fullsized":"2023-10-08T16:06:17.587960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:37.113115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569787+0000","last_change":"2023-10-08T16:06:16.566219+0000","last_active":"2023-10-08T16:06:17.569787+0000","last_peered":"2023-10-08T16:06:17.569787+0000","last_clean":"2023-10-08T16:06:17.569787+0000","last_became_active":"2023-10-08T16:06:16.565768+0000","last_became_peered":"2023-10-08T16:06:16.565768+0000","last_unstale":"2023-10-08T16:06:17.569787+0000","last_undegraded":"2023-10-08T16:06:17.569787+0000","last_fullsized":"2023-10-08T16:06:17.569787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:23.233110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573859+0000","last_change":"2023-10-08T16:06:17.573859+0000","last_active":"2023-10-08T16:06:17.573859+0000","last_peered":"2023-10-08T16:06:17.573859+0000","last_clean":"2023-10-08T16:06:17.573859+0000","last_became_active":"2023-10-08T16:06:17.573038+0000","last_became_peered":"2023-10-08T16:06:17.573038+0000","last_unstale":"2023-10-08T16:06:17.573859+0000","last_undegraded":"2023-10-08T16:06:17.573859+0000","last_fullsized":"2023-10-08T16:06:17.573859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:39.701235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577903+0000","last_change":"2023-10-08T16:06:16.563616+0000","last_active":"2023-10-08T16:06:17.577903+0000","last_peered":"2023-10-08T16:06:17.577903+0000","last_clean":"2023-10-08T16:06:17.577903+0000","last_became_active":"2023-10-08T16:06:16.563288+0000","last_became_peered":"2023-10-08T16:06:16.563288+0000","last_unstale":"2023-10-08T16:06:17.577903+0000","last_undegraded":"2023-10-08T16:06:17.577903+0000","last_fullsized":"2023-10-08T16:06:17.577903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:34.904243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575110+0000","last_change":"2023-10-08T16:06:17.569203+0000","last_active":"2023-10-08T16:06:17.575110+0000","last_peered":"2023-10-08T16:06:17.575110+0000","last_clean":"2023-10-08T16:06:17.575110+0000","last_became_active":"2023-10-08T16:06:17.568974+0000","last_became_peered":"2023-10-08T16:06:17.568974+0000","last_unstale":"2023-10-08T16:06:17.575110+0000","last_undegraded":"2023-10-08T16:06:17.575110+0000","last_fullsized":"2023-10-08T16:06:17.575110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:38.364733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.566943+0000","last_change":"2023-10-08T16:06:16.575012+0000","last_active":"2023-10-08T16:06:17.566943+0000","last_peered":"2023-10-08T16:06:17.566943+0000","last_clean":"2023-10-08T16:06:17.566943+0000","last_became_active":"2023-10-08T16:06:16.574772+0000","last_became_peered":"2023-10-08T16:06:16.574772+0000","last_unstale":"2023-10-08T16:06:17.566943+0000","last_undegraded":"2023-10-08T16:06:17.566943+0000","last_fullsized":"2023-10-08T16:06:17.566943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:56.930770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581334+0000","last_change":"2023-10-08T16:06:17.576974+0000","last_active":"2023-10-08T16:06:17.581334+0000","last_peered":"2023-10-08T16:06:17.581334+0000","last_clean":"2023-10-08T16:06:17.581334+0000","last_became_active":"2023-10-08T16:06:17.576726+0000","last_became_peered":"2023-10-08T16:06:17.576726+0000","last_unstale":"2023-10-08T16:06:17.581334+0000","last_undegraded":"2023-10-08T16:06:17.581334+0000","last_fullsized":"2023-10-08T16:06:17.581334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:53.909664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578592+0000","last_change":"2023-10-08T16:06:16.568193+0000","last_active":"2023-10-08T16:06:17.578592+0000","last_peered":"2023-10-08T16:06:17.578592+0000","last_clean":"2023-10-08T16:06:17.578592+0000","last_became_active":"2023-10-08T16:06:16.567897+0000","last_became_peered":"2023-10-08T16:06:16.567897+0000","last_unstale":"2023-10-08T16:06:17.578592+0000","last_undegraded":"2023-10-08T16:06:17.578592+0000","last_fullsized":"2023-10-08T16:06:17.578592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:23.679170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.582733+0000","last_change":"2023-10-08T16:06:17.582733+0000","last_active":"2023-10-08T16:06:17.582733+0000","last_peered":"2023-10-08T16:06:17.582733+0000","last_clean":"2023-10-08T16:06:17.582733+0000","last_became_active":"2023-10-08T16:06:17.582435+0000","last_became_peered":"2023-10-08T16:06:17.582435+0000","last_unstale":"2023-10-08T16:06:17.582733+0000","last_undegraded":"2023-10-08T16:06:17.582733+0000","last_fullsized":"2023-10-08T16:06:17.582733+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:44.200579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573837+0000","last_change":"2023-10-08T16:06:16.565813+0000","last_active":"2023-10-08T16:06:17.573837+0000","last_peered":"2023-10-08T16:06:17.573837+0000","last_clean":"2023-10-08T16:06:17.573837+0000","last_became_active":"2023-10-08T16:06:16.565453+0000","last_became_peered":"2023-10-08T16:06:16.565453+0000","last_unstale":"2023-10-08T16:06:17.573837+0000","last_undegraded":"2023-10-08T16:06:17.573837+0000","last_fullsized":"2023-10-08T16:06:17.573837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:08.675593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572578+0000","last_change":"2023-10-08T16:06:17.572578+0000","last_active":"2023-10-08T16:06:17.572578+0000","last_peered":"2023-10-08T16:06:17.572578+0000","last_clean":"2023-10-08T16:06:17.572578+0000","last_became_active":"2023-10-08T16:06:17.572344+0000","last_became_peered":"2023-10-08T16:06:17.572344+0000","last_unstale":"2023-10-08T16:06:17.572578+0000","last_undegraded":"2023-10-08T16:06:17.572578+0000","last_fullsized":"2023-10-08T16:06:17.572578+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:45.306918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576599+0000","last_change":"2023-10-08T16:06:16.575950+0000","last_active":"2023-10-08T16:06:17.576599+0000","last_peered":"2023-10-08T16:06:17.576599+0000","last_clean":"2023-10-08T16:06:17.576599+0000","last_became_active":"2023-10-08T16:06:16.575609+0000","last_became_peered":"2023-10-08T16:06:16.575609+0000","last_unstale":"2023-10-08T16:06:17.576599+0000","last_undegraded":"2023-10-08T16:06:17.576599+0000","last_fullsized":"2023-10-08T16:06:17.576599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:56.930770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571695+0000","last_change":"2023-10-08T16:06:17.571695+0000","last_active":"2023-10-08T16:06:17.571695+0000","last_peered":"2023-10-08T16:06:17.571695+0000","last_clean":"2023-10-08T16:06:17.571695+0000","last_became_active":"2023-10-08T16:06:17.571189+0000","last_became_peered":"2023-10-08T16:06:17.571189+0000","last_unstale":"2023-10-08T16:06:17.571695+0000","last_undegraded":"2023-10-08T16:06:17.571695+0000","last_fullsized":"2023-10-08T16:06:17.571695+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:01.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583741+0000","last_change":"2023-10-08T16:06:16.569672+0000","last_active":"2023-10-08T16:06:17.583741+0000","last_peered":"2023-10-08T16:06:17.583741+0000","last_clean":"2023-10-08T16:06:17.583741+0000","last_became_active":"2023-10-08T16:06:16.569348+0000","last_became_peered":"2023-10-08T16:06:16.569348+0000","last_unstale":"2023-10-08T16:06:17.583741+0000","last_undegraded":"2023-10-08T16:06:17.583741+0000","last_fullsized":"2023-10-08T16:06:17.583741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:32.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580120+0000","last_change":"2023-10-08T16:06:17.567245+0000","last_active":"2023-10-08T16:06:17.580120+0000","last_peered":"2023-10-08T16:06:17.580120+0000","last_clean":"2023-10-08T16:06:17.580120+0000","last_became_active":"2023-10-08T16:06:17.566060+0000","last_became_peered":"2023-10-08T16:06:17.566060+0000","last_unstale":"2023-10-08T16:06:17.580120+0000","last_undegraded":"2023-10-08T16:06:17.580120+0000","last_fullsized":"2023-10-08T16:06:17.580120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:21.572253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584369+0000","last_change":"2023-10-08T16:06:16.569102+0000","last_active":"2023-10-08T16:06:17.584369+0000","last_peered":"2023-10-08T16:06:17.584369+0000","last_clean":"2023-10-08T16:06:17.584369+0000","last_became_active":"2023-10-08T16:06:16.568872+0000","last_became_peered":"2023-10-08T16:06:16.568872+0000","last_unstale":"2023-10-08T16:06:17.584369+0000","last_undegraded":"2023-10-08T16:06:17.584369+0000","last_fullsized":"2023-10-08T16:06:17.584369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:01.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577101+0000","last_change":"2023-10-08T16:06:17.569298+0000","last_active":"2023-10-08T16:06:17.577101+0000","last_peered":"2023-10-08T16:06:17.577101+0000","last_clean":"2023-10-08T16:06:17.577101+0000","last_became_active":"2023-10-08T16:06:17.569030+0000","last_became_peered":"2023-10-08T16:06:17.569030+0000","last_unstale":"2023-10-08T16:06:17.577101+0000","last_undegraded":"2023-10-08T16:06:17.577101+0000","last_fullsized":"2023-10-08T16:06:17.577101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:18.434623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588290+0000","last_change":"2023-10-08T16:06:16.569843+0000","last_active":"2023-10-08T16:06:17.588290+0000","last_peered":"2023-10-08T16:06:17.588290+0000","last_clean":"2023-10-08T16:06:17.588290+0000","last_became_active":"2023-10-08T16:06:16.568502+0000","last_became_peered":"2023-10-08T16:06:16.568502+0000","last_unstale":"2023-10-08T16:06:17.588290+0000","last_undegraded":"2023-10-08T16:06:17.588290+0000","last_fullsized":"2023-10-08T16:06:17.588290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:10.631601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570330+0000","last_change":"2023-10-08T16:06:17.570330+0000","last_active":"2023-10-08T16:06:17.570330+0000","last_peered":"2023-10-08T16:06:17.570330+0000","last_clean":"2023-10-08T16:06:17.570330+0000","last_became_active":"2023-10-08T16:06:17.570095+0000","last_became_peered":"2023-10-08T16:06:17.570095+0000","last_unstale":"2023-10-08T16:06:17.570330+0000","last_undegraded":"2023-10-08T16:06:17.570330+0000","last_fullsized":"2023-10-08T16:06:17.570330+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.709067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.560852+0000","last_change":"2023-10-08T16:06:16.575536+0000","last_active":"2023-10-08T16:06:17.560852+0000","last_peered":"2023-10-08T16:06:17.560852+0000","last_clean":"2023-10-08T16:06:17.560852+0000","last_became_active":"2023-10-08T16:06:16.575227+0000","last_became_peered":"2023-10-08T16:06:16.575227+0000","last_unstale":"2023-10-08T16:06:17.560852+0000","last_undegraded":"2023-10-08T16:06:17.560852+0000","last_fullsized":"2023-10-08T16:06:17.560852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:21.723041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.563281+0000","last_change":"2023-10-08T16:06:17.563281+0000","last_active":"2023-10-08T16:06:17.563281+0000","last_peered":"2023-10-08T16:06:17.563281+0000","last_clean":"2023-10-08T16:06:17.563281+0000","last_became_active":"2023-10-08T16:06:17.563038+0000","last_became_peered":"2023-10-08T16:06:17.563038+0000","last_unstale":"2023-10-08T16:06:17.563281+0000","last_undegraded":"2023-10-08T16:06:17.563281+0000","last_fullsized":"2023-10-08T16:06:17.563281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:34:04.939546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578687+0000","last_change":"2023-10-08T16:06:16.566260+0000","last_active":"2023-10-08T16:06:17.578687+0000","last_peered":"2023-10-08T16:06:17.578687+0000","last_clean":"2023-10-08T16:06:17.578687+0000","last_became_active":"2023-10-08T16:06:16.565937+0000","last_became_peered":"2023-10-08T16:06:16.565937+0000","last_unstale":"2023-10-08T16:06:17.578687+0000","last_undegraded":"2023-10-08T16:06:17.578687+0000","last_fullsized":"2023-10-08T16:06:17.578687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:21:22.211819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583353+0000","last_change":"2023-10-08T16:06:17.583353+0000","last_active":"2023-10-08T16:06:17.583353+0000","last_peered":"2023-10-08T16:06:17.583353+0000","last_clean":"2023-10-08T16:06:17.583353+0000","last_became_active":"2023-10-08T16:06:17.583134+0000","last_became_peered":"2023-10-08T16:06:17.583134+0000","last_unstale":"2023-10-08T16:06:17.583353+0000","last_undegraded":"2023-10-08T16:06:17.583353+0000","last_fullsized":"2023-10-08T16:06:17.583353+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:31:43.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573960+0000","last_change":"2023-10-08T16:06:16.568615+0000","last_active":"2023-10-08T16:06:17.573960+0000","last_peered":"2023-10-08T16:06:17.573960+0000","last_clean":"2023-10-08T16:06:17.573960+0000","last_became_active":"2023-10-08T16:06:16.567453+0000","last_became_peered":"2023-10-08T16:06:16.567453+0000","last_unstale":"2023-10-08T16:06:17.573960+0000","last_undegraded":"2023-10-08T16:06:17.573960+0000","last_fullsized":"2023-10-08T16:06:17.573960+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:05.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583922+0000","last_change":"2023-10-08T16:06:17.583922+0000","last_active":"2023-10-08T16:06:17.583922+0000","last_peered":"2023-10-08T16:06:17.583922+0000","last_clean":"2023-10-08T16:06:17.583922+0000","last_became_active":"2023-10-08T16:06:17.583698+0000","last_became_peered":"2023-10-08T16:06:17.583698+0000","last_unstale":"2023-10-08T16:06:17.583922+0000","last_undegraded":"2023-10-08T16:06:17.583922+0000","last_fullsized":"2023-10-08T16:06:17.583922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:40.309552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.569552+0000","last_change":"2023-10-08T16:06:16.568225+0000","last_active":"2023-10-08T16:06:16.569552+0000","last_peered":"2023-10-08T16:06:16.569552+0000","last_clean":"2023-10-08T16:06:16.569552+0000","last_became_active":"2023-10-08T16:06:16.567958+0000","last_became_peered":"2023-10-08T16:06:16.567958+0000","last_unstale":"2023-10-08T16:06:16.569552+0000","last_undegraded":"2023-10-08T16:06:16.569552+0000","last_fullsized":"2023-10-08T16:06:16.569552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:38.938757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570681+0000","last_change":"2023-10-08T16:06:17.570681+0000","last_active":"2023-10-08T16:06:17.570681+0000","last_peered":"2023-10-08T16:06:17.570681+0000","last_clean":"2023-10-08T16:06:17.570681+0000","last_became_active":"2023-10-08T16:06:17.570361+0000","last_became_peered":"2023-10-08T16:06:17.570361+0000","last_unstale":"2023-10-08T16:06:17.570681+0000","last_undegraded":"2023-10-08T16:06:17.570681+0000","last_fullsized":"2023-10-08T16:06:17.570681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:53.909664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587710+0000","last_change":"2023-10-08T16:06:16.569545+0000","last_active":"2023-10-08T16:06:17.587710+0000","last_peered":"2023-10-08T16:06:17.587710+0000","last_clean":"2023-10-08T16:06:17.587710+0000","last_became_active":"2023-10-08T16:06:16.569052+0000","last_became_peered":"2023-10-08T16:06:16.569052+0000","last_unstale":"2023-10-08T16:06:17.587710+0000","last_undegraded":"2023-10-08T16:06:17.587710+0000","last_fullsized":"2023-10-08T16:06:17.587710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:39.859359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588021+0000","last_change":"2023-10-08T16:06:17.579285+0000","last_active":"2023-10-08T16:06:17.588021+0000","last_peered":"2023-10-08T16:06:17.588021+0000","last_clean":"2023-10-08T16:06:17.588021+0000","last_became_active":"2023-10-08T16:06:17.579024+0000","last_became_peered":"2023-10-08T16:06:17.579024+0000","last_unstale":"2023-10-08T16:06:17.588021+0000","last_undegraded":"2023-10-08T16:06:17.588021+0000","last_fullsized":"2023-10-08T16:06:17.588021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:01.142901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571674+0000","last_change":"2023-10-08T16:06:16.565882+0000","last_active":"2023-10-08T16:06:17.571674+0000","last_peered":"2023-10-08T16:06:17.571674+0000","last_clean":"2023-10-08T16:06:17.571674+0000","last_became_active":"2023-10-08T16:06:16.565503+0000","last_became_peered":"2023-10-08T16:06:16.565503+0000","last_unstale":"2023-10-08T16:06:17.571674+0000","last_undegraded":"2023-10-08T16:06:17.571674+0000","last_fullsized":"2023-10-08T16:06:17.571674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:05.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584434+0000","last_change":"2023-10-08T16:06:17.582081+0000","last_active":"2023-10-08T16:06:17.584434+0000","last_peered":"2023-10-08T16:06:17.584434+0000","last_clean":"2023-10-08T16:06:17.584434+0000","last_became_active":"2023-10-08T16:06:17.581841+0000","last_became_peered":"2023-10-08T16:06:17.581841+0000","last_unstale":"2023-10-08T16:06:17.584434+0000","last_undegraded":"2023-10-08T16:06:17.584434+0000","last_fullsized":"2023-10-08T16:06:17.584434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:09.872516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567798+0000","last_change":"2023-10-08T16:06:16.565980+0000","last_active":"2023-10-08T16:06:16.567798+0000","last_peered":"2023-10-08T16:06:16.567798+0000","last_clean":"2023-10-08T16:06:16.567798+0000","last_became_active":"2023-10-08T16:06:16.565187+0000","last_became_peered":"2023-10-08T16:06:16.565187+0000","last_unstale":"2023-10-08T16:06:16.567798+0000","last_undegraded":"2023-10-08T16:06:16.567798+0000","last_fullsized":"2023-10-08T16:06:16.567798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:42.514716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576334+0000","last_change":"2023-10-08T16:06:17.574881+0000","last_active":"2023-10-08T16:06:17.576334+0000","last_peered":"2023-10-08T16:06:17.576334+0000","last_clean":"2023-10-08T16:06:17.576334+0000","last_became_active":"2023-10-08T16:06:17.574405+0000","last_became_peered":"2023-10-08T16:06:17.574405+0000","last_unstale":"2023-10-08T16:06:17.576334+0000","last_undegraded":"2023-10-08T16:06:17.576334+0000","last_fullsized":"2023-10-08T16:06:17.576334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:57.285585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.569097+0000","last_change":"2023-10-08T16:06:16.567331+0000","last_active":"2023-10-08T16:06:16.569097+0000","last_peered":"2023-10-08T16:06:16.569097+0000","last_clean":"2023-10-08T16:06:16.569097+0000","last_became_active":"2023-10-08T16:06:16.567062+0000","last_became_peered":"2023-10-08T16:06:16.567062+0000","last_unstale":"2023-10-08T16:06:16.569097+0000","last_undegraded":"2023-10-08T16:06:16.569097+0000","last_fullsized":"2023-10-08T16:06:16.569097+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:08.675593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574119+0000","last_change":"2023-10-08T16:06:17.574119+0000","last_active":"2023-10-08T16:06:17.574119+0000","last_peered":"2023-10-08T16:06:17.574119+0000","last_clean":"2023-10-08T16:06:17.574119+0000","last_became_active":"2023-10-08T16:06:17.573303+0000","last_became_peered":"2023-10-08T16:06:17.573303+0000","last_unstale":"2023-10-08T16:06:17.574119+0000","last_undegraded":"2023-10-08T16:06:17.574119+0000","last_fullsized":"2023-10-08T16:06:17.574119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:38.054120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567455+0000","last_change":"2023-10-08T16:06:16.564016+0000","last_active":"2023-10-08T16:06:16.567455+0000","last_peered":"2023-10-08T16:06:16.567455+0000","last_clean":"2023-10-08T16:06:16.567455+0000","last_became_active":"2023-10-08T16:06:16.563425+0000","last_became_peered":"2023-10-08T16:06:16.563425+0000","last_unstale":"2023-10-08T16:06:16.567455+0000","last_undegraded":"2023-10-08T16:06:16.567455+0000","last_fullsized":"2023-10-08T16:06:16.567455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:01:54.467407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587679+0000","last_change":"2023-10-08T16:06:17.582917+0000","last_active":"2023-10-08T16:06:17.587679+0000","last_peered":"2023-10-08T16:06:17.587679+0000","last_clean":"2023-10-08T16:06:17.587679+0000","last_became_active":"2023-10-08T16:06:17.582519+0000","last_became_peered":"2023-10-08T16:06:17.582519+0000","last_unstale":"2023-10-08T16:06:17.587679+0000","last_undegraded":"2023-10-08T16:06:17.587679+0000","last_fullsized":"2023-10-08T16:06:17.587679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:18.434623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.554838+0000","last_change":"2023-10-08T16:06:16.564166+0000","last_active":"2023-10-08T16:06:17.554838+0000","last_peered":"2023-10-08T16:06:17.554838+0000","last_clean":"2023-10-08T16:06:17.554838+0000","last_became_active":"2023-10-08T16:06:16.563910+0000","last_became_peered":"2023-10-08T16:06:16.563910+0000","last_unstale":"2023-10-08T16:06:17.554838+0000","last_undegraded":"2023-10-08T16:06:17.554838+0000","last_fullsized":"2023-10-08T16:06:17.554838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:01.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.566959+0000","last_change":"2023-10-08T16:06:17.564528+0000","last_active":"2023-10-08T16:06:17.566959+0000","last_peered":"2023-10-08T16:06:17.566959+0000","last_clean":"2023-10-08T16:06:17.566959+0000","last_became_active":"2023-10-08T16:06:17.564208+0000","last_became_peered":"2023-10-08T16:06:17.564208+0000","last_unstale":"2023-10-08T16:06:17.566959+0000","last_undegraded":"2023-10-08T16:06:17.566959+0000","last_fullsized":"2023-10-08T16:06:17.566959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:09.680894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.579605+0000","last_change":"2023-10-08T16:06:16.575359+0000","last_active":"2023-10-08T16:06:16.579605+0000","last_peered":"2023-10-08T16:06:16.579605+0000","last_clean":"2023-10-08T16:06:16.579605+0000","last_became_active":"2023-10-08T16:06:16.574975+0000","last_became_peered":"2023-10-08T16:06:16.574975+0000","last_unstale":"2023-10-08T16:06:16.579605+0000","last_undegraded":"2023-10-08T16:06:16.579605+0000","last_fullsized":"2023-10-08T16:06:16.579605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:53.343775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587153+0000","last_change":"2023-10-08T16:06:17.581897+0000","last_active":"2023-10-08T16:06:17.587153+0000","last_peered":"2023-10-08T16:06:17.587153+0000","last_clean":"2023-10-08T16:06:17.587153+0000","last_became_active":"2023-10-08T16:06:17.581619+0000","last_became_peered":"2023-10-08T16:06:17.581619+0000","last_unstale":"2023-10-08T16:06:17.587153+0000","last_undegraded":"2023-10-08T16:06:17.587153+0000","last_fullsized":"2023-10-08T16:06:17.587153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:19.090222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.565001+0000","last_change":"2023-10-08T16:06:16.570676+0000","last_active":"2023-10-08T16:06:17.565001+0000","last_peered":"2023-10-08T16:06:17.565001+0000","last_clean":"2023-10-08T16:06:17.565001+0000","last_became_active":"2023-10-08T16:06:16.570313+0000","last_became_peered":"2023-10-08T16:06:16.570313+0000","last_unstale":"2023-10-08T16:06:17.565001+0000","last_undegraded":"2023-10-08T16:06:17.565001+0000","last_fullsized":"2023-10-08T16:06:17.565001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:44.301618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.585985+0000","last_change":"2023-10-08T16:06:17.585985+0000","last_active":"2023-10-08T16:06:17.585985+0000","last_peered":"2023-10-08T16:06:17.585985+0000","last_clean":"2023-10-08T16:06:17.585985+0000","last_became_active":"2023-10-08T16:06:17.585689+0000","last_became_peered":"2023-10-08T16:06:17.585689+0000","last_unstale":"2023-10-08T16:06:17.585985+0000","last_undegraded":"2023-10-08T16:06:17.585985+0000","last_fullsized":"2023-10-08T16:06:17.585985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:51.248645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584327+0000","last_change":"2023-10-08T16:06:16.570493+0000","last_active":"2023-10-08T16:06:17.584327+0000","last_peered":"2023-10-08T16:06:17.584327+0000","last_clean":"2023-10-08T16:06:17.584327+0000","last_became_active":"2023-10-08T16:06:16.570214+0000","last_became_peered":"2023-10-08T16:06:16.570214+0000","last_unstale":"2023-10-08T16:06:17.584327+0000","last_undegraded":"2023-10-08T16:06:17.584327+0000","last_fullsized":"2023-10-08T16:06:17.584327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:18.995016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":199,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.560956+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:06:17.560956+0000","last_peered":"2023-10-08T16:06:17.560956+0000","last_clean":"2023-10-08T16:06:17.560956+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:06:17.560956+0000","last_undegraded":"2023-10-08T16:06:17.560956+0000","last_fullsized":"2023-10-08T16:06:17.560956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":916027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":8.0939999999999994,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":8.2040000000000006,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.30399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.373},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.3409999999999993,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.5}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":318875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67000000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42299999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":320907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54000000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.439},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.51600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":909151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.434},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.37}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.63100000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":320319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.371}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.316},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.48799999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302128,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069712,"statfs":{"total":96636764160,"available":96327385088,"internally_reserved":0,"allocated":761856,"data_stored":318895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.371},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.29099999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.628}]},{"osd":7,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":318853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.501},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.51500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":325161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":12.227,"5min":12.227,"15min":12.227},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.50900000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.441}]},{"osd":6,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:06:21.619 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:06:21.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.766+0000 7f7af513d700 1 Processor -- start 2023-10-08T16:06:21.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.766+0000 7f7af513d700 1 -- start start 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7af513d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7af0159890 0x7f7af015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7af513d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0069560 0x7f7af00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7af513d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af0069ee0 con 0x7f7af0159c60 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7af513d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af015a160 con 0x7f7af0159890 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7af513d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00d13a0 con 0x7f7af0069560 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7af0159890 0x7f7af015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7af0159890 0x7f7af015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44266/0 (socket says 172.21.15.170:44266) 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 -- 172.21.15.170:0/3462360361 learned_addr learned my addr 172.21.15.170:0/3462360361 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:21.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aef59e700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0069560 0x7f7af00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 -- 172.21.15.170:0/3462360361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0069560 msgr2=0x7f7af00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0069560 0x7f7af00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 -- 172.21.15.170:0/3462360361 >> v1:172.21.15.191:6789/0 conn(0x7f7af0159c60 legacy=0x7f7af015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.767+0000 7f7aeed9d700 1 -- 172.21.15.170:0/3462360361 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af015e140 con 0x7f7af0159890 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159890 0x7f7af015abd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00099b0 tx=0x7f7ae0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8163dc6580a7f61b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7aee59c700 1 -- 172.21.15.170:0/3462360361 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0009ea0 con 0x7f7af0159890 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7aee59c700 1 -- 172.21.15.170:0/3462360361 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000b860 con 0x7f7af0159890 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7aee59c700 1 -- 172.21.15.170:0/3462360361 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000bc00 con 0x7f7af0159890 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 -- 172.21.15.170:0/3462360361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159890 msgr2=0x7f7af015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159890 0x7f7af015abd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00099b0 tx=0x7f7ae0009250 comp rx=0 tx=0).stop 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 -- 172.21.15.170:0/3462360361 shutdown_connections 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0069560 0x7f7af00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 --2- 172.21.15.170:0/3462360361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159890 0x7f7af015abd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 -- 172.21.15.170:0/3462360361 >> 172.21.15.170:0/3462360361 conn(0x7f7af0068e20 msgr2=0x7f7af00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 -- 172.21.15.170:0/3462360361 shutdown_connections 2023-10-08T16:06:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.768+0000 7f7af513d700 1 -- 172.21.15.170:0/3462360361 wait complete. 2023-10-08T16:06:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.769+0000 7f7af513d700 1 Processor -- start 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.769+0000 7f7af513d700 1 -- start start 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 0x7f7af014cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 0x7f7af00c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af014d330 con 0x7f7af0159c60 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00ca380 con 0x7f7af0069560 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7af513d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00c8b00 con 0x7f7af0159890 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7aef59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7aef59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44274/0 (socket says 172.21.15.170:44274) 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7aef59e700 1 -- 172.21.15.170:0/131968662 learned_addr learned my addr 172.21.15.170:0/131968662 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 0x7f7af014cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.770+0000 7f7ae659c700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 0x7f7af00c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 -- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 msgr2=0x7f7af00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 0x7f7af00c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 -- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 msgr2=0x7f7af014cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 0x7f7af014cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 -- 172.21.15.170:0/131968662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00094f0 con 0x7f7af0159c60 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 0x7f7af014cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7ae659c700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 0x7f7af00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7aef59e700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009df0 tx=0x7f7ae800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800cb50 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae800f040 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8014480 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af00c9770 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.771+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af00c9d30 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.772+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af00ca530 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.772+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7ae80147f0 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.773+0000 7f7ae77fe700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 0x7f7ad0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.773+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ae8085300 con 0x7f7af0159c60 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.773+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 0x7f7ad0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:21.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.773+0000 7f7aeed9d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 0x7f7ad0069470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00099b0 tx=0x7f7ae0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:21.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.777+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7af00ca530 con 0x7f7af0159c60 2023-10-08T16:06:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.954+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7af00ca710 con 0x7f7ad0066fc0 2023-10-08T16:06:21.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.962+0000 7f7ae77fe700 1 -- 172.21.15.170:0/131968662 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289326 (secure 0 0 0) 0x7f7af00ca710 con 0x7f7ad0066fc0 2023-10-08T16:06:21.966 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 msgr2=0x7f7ad0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 0x7f7ad0069470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00099b0 tx=0x7f7ae0017000 comp rx=0 tx=0).stop 2023-10-08T16:06:21.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 msgr2=0x7f7af01524b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009df0 tx=0x7f7ae800b040 comp rx=0 tx=0).stop 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 shutdown_connections 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ad0066fc0 0x7f7ad0069470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7af0159c60 0x7f7af01524b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7af0069560 0x7f7af014cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 --2- 172.21.15.170:0/131968662 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7af0159890 0x7f7af00c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 >> 172.21.15.170:0/131968662 conn(0x7f7af0068e20 msgr2=0x7f7af015c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:21.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 shutdown_connections 2023-10-08T16:06:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:21.965+0000 7f7af513d700 1 -- 172.21.15.170:0/131968662 wait complete. 2023-10-08T16:06:21.970 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:06:21.977 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2023-10-08T16:06:20.244779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397728,"kb_used_data":7136,"kb_used_omap":0,"kb_used_meta":2390528,"kb_avail":752576992,"statfs":{"total":773094113280,"available":770638839808,"internally_reserved":0,"allocated":7307264,"data_stored":3748188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2447900672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.700184"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580172+0000","last_change":"2023-10-08T16:06:17.568234+0000","last_active":"2023-10-08T16:06:17.580172+0000","last_peered":"2023-10-08T16:06:17.580172+0000","last_clean":"2023-10-08T16:06:17.580172+0000","last_became_active":"2023-10-08T16:06:17.567998+0000","last_became_peered":"2023-10-08T16:06:17.567998+0000","last_unstale":"2023-10-08T16:06:17.580172+0000","last_undegraded":"2023-10-08T16:06:17.580172+0000","last_fullsized":"2023-10-08T16:06:17.580172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:24.238411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587668+0000","last_change":"2023-10-08T16:06:17.583044+0000","last_active":"2023-10-08T16:06:17.587668+0000","last_peered":"2023-10-08T16:06:17.587668+0000","last_clean":"2023-10-08T16:06:17.587668+0000","last_became_active":"2023-10-08T16:06:17.582788+0000","last_became_peered":"2023-10-08T16:06:17.582788+0000","last_unstale":"2023-10-08T16:06:17.587668+0000","last_undegraded":"2023-10-08T16:06:17.587668+0000","last_fullsized":"2023-10-08T16:06:17.587668+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:01:40.644016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569762+0000","last_change":"2023-10-08T16:06:17.569762+0000","last_active":"2023-10-08T16:06:17.569762+0000","last_peered":"2023-10-08T16:06:17.569762+0000","last_clean":"2023-10-08T16:06:17.569762+0000","last_became_active":"2023-10-08T16:06:17.569466+0000","last_became_peered":"2023-10-08T16:06:17.569466+0000","last_unstale":"2023-10-08T16:06:17.569762+0000","last_undegraded":"2023-10-08T16:06:17.569762+0000","last_fullsized":"2023-10-08T16:06:17.569762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:16.538590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.586595+0000","last_change":"2023-10-08T16:06:17.585962+0000","last_active":"2023-10-08T16:06:17.586595+0000","last_peered":"2023-10-08T16:06:17.586595+0000","last_clean":"2023-10-08T16:06:17.586595+0000","last_became_active":"2023-10-08T16:06:17.585690+0000","last_became_peered":"2023-10-08T16:06:17.585690+0000","last_unstale":"2023-10-08T16:06:17.586595+0000","last_undegraded":"2023-10-08T16:06:17.586595+0000","last_fullsized":"2023-10-08T16:06:17.586595+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:01.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575469+0000","last_change":"2023-10-08T16:06:17.575469+0000","last_active":"2023-10-08T16:06:17.575469+0000","last_peered":"2023-10-08T16:06:17.575469+0000","last_clean":"2023-10-08T16:06:17.575469+0000","last_became_active":"2023-10-08T16:06:17.575245+0000","last_became_peered":"2023-10-08T16:06:17.575245+0000","last_unstale":"2023-10-08T16:06:17.575469+0000","last_undegraded":"2023-10-08T16:06:17.575469+0000","last_fullsized":"2023-10-08T16:06:17.575469+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:40.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587610+0000","last_change":"2023-10-08T16:06:17.582179+0000","last_active":"2023-10-08T16:06:17.587610+0000","last_peered":"2023-10-08T16:06:17.587610+0000","last_clean":"2023-10-08T16:06:17.587610+0000","last_became_active":"2023-10-08T16:06:17.581943+0000","last_became_peered":"2023-10-08T16:06:17.581943+0000","last_unstale":"2023-10-08T16:06:17.587610+0000","last_undegraded":"2023-10-08T16:06:17.587610+0000","last_fullsized":"2023-10-08T16:06:17.587610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:52.648731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588121+0000","last_change":"2023-10-08T16:06:17.580126+0000","last_active":"2023-10-08T16:06:17.588121+0000","last_peered":"2023-10-08T16:06:17.588121+0000","last_clean":"2023-10-08T16:06:17.588121+0000","last_became_active":"2023-10-08T16:06:17.579772+0000","last_became_peered":"2023-10-08T16:06:17.579772+0000","last_unstale":"2023-10-08T16:06:17.588121+0000","last_undegraded":"2023-10-08T16:06:17.588121+0000","last_fullsized":"2023-10-08T16:06:17.588121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:26.856007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587577+0000","last_change":"2023-10-08T16:06:17.582290+0000","last_active":"2023-10-08T16:06:17.587577+0000","last_peered":"2023-10-08T16:06:17.587577+0000","last_clean":"2023-10-08T16:06:17.587577+0000","last_became_active":"2023-10-08T16:06:17.581534+0000","last_became_peered":"2023-10-08T16:06:17.581534+0000","last_unstale":"2023-10-08T16:06:17.587577+0000","last_undegraded":"2023-10-08T16:06:17.587577+0000","last_fullsized":"2023-10-08T16:06:17.587577+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:06.225262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570270+0000","last_change":"2023-10-08T16:06:17.570270+0000","last_active":"2023-10-08T16:06:17.570270+0000","last_peered":"2023-10-08T16:06:17.570270+0000","last_clean":"2023-10-08T16:06:17.570270+0000","last_became_active":"2023-10-08T16:06:17.570008+0000","last_became_peered":"2023-10-08T16:06:17.570008+0000","last_unstale":"2023-10-08T16:06:17.570270+0000","last_undegraded":"2023-10-08T16:06:17.570270+0000","last_fullsized":"2023-10-08T16:06:17.570270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:20.000317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571817+0000","last_change":"2023-10-08T16:06:17.571817+0000","last_active":"2023-10-08T16:06:17.571817+0000","last_peered":"2023-10-08T16:06:17.571817+0000","last_clean":"2023-10-08T16:06:17.571817+0000","last_became_active":"2023-10-08T16:06:17.571439+0000","last_became_peered":"2023-10-08T16:06:17.571439+0000","last_unstale":"2023-10-08T16:06:17.571817+0000","last_undegraded":"2023-10-08T16:06:17.571817+0000","last_fullsized":"2023-10-08T16:06:17.571817+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:22.728342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576316+0000","last_change":"2023-10-08T16:06:17.573822+0000","last_active":"2023-10-08T16:06:17.576316+0000","last_peered":"2023-10-08T16:06:17.576316+0000","last_clean":"2023-10-08T16:06:17.576316+0000","last_became_active":"2023-10-08T16:06:17.573550+0000","last_became_peered":"2023-10-08T16:06:17.573550+0000","last_unstale":"2023-10-08T16:06:17.576316+0000","last_undegraded":"2023-10-08T16:06:17.576316+0000","last_fullsized":"2023-10-08T16:06:17.576316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:45:31.341258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577121+0000","last_change":"2023-10-08T16:06:17.569314+0000","last_active":"2023-10-08T16:06:17.577121+0000","last_peered":"2023-10-08T16:06:17.577121+0000","last_clean":"2023-10-08T16:06:17.577121+0000","last_became_active":"2023-10-08T16:06:17.569111+0000","last_became_peered":"2023-10-08T16:06:17.569111+0000","last_unstale":"2023-10-08T16:06:17.577121+0000","last_undegraded":"2023-10-08T16:06:17.577121+0000","last_fullsized":"2023-10-08T16:06:17.577121+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:06.775660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.585032+0000","last_change":"2023-10-08T16:06:17.585032+0000","last_active":"2023-10-08T16:06:17.585032+0000","last_peered":"2023-10-08T16:06:17.585032+0000","last_clean":"2023-10-08T16:06:17.585032+0000","last_became_active":"2023-10-08T16:06:17.584781+0000","last_became_peered":"2023-10-08T16:06:17.584781+0000","last_unstale":"2023-10-08T16:06:17.585032+0000","last_undegraded":"2023-10-08T16:06:17.585032+0000","last_fullsized":"2023-10-08T16:06:17.585032+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:43.420587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574312+0000","last_change":"2023-10-08T16:06:17.574312+0000","last_active":"2023-10-08T16:06:17.574312+0000","last_peered":"2023-10-08T16:06:17.574312+0000","last_clean":"2023-10-08T16:06:17.574312+0000","last_became_active":"2023-10-08T16:06:17.573841+0000","last_became_peered":"2023-10-08T16:06:17.573841+0000","last_unstale":"2023-10-08T16:06:17.574312+0000","last_undegraded":"2023-10-08T16:06:17.574312+0000","last_fullsized":"2023-10-08T16:06:17.574312+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:37.743529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570509+0000","last_change":"2023-10-08T16:06:17.570509+0000","last_active":"2023-10-08T16:06:17.570509+0000","last_peered":"2023-10-08T16:06:17.570509+0000","last_clean":"2023-10-08T16:06:17.570509+0000","last_became_active":"2023-10-08T16:06:17.570252+0000","last_became_peered":"2023-10-08T16:06:17.570252+0000","last_unstale":"2023-10-08T16:06:17.570509+0000","last_undegraded":"2023-10-08T16:06:17.570509+0000","last_fullsized":"2023-10-08T16:06:17.570509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:33:30.868127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583530+0000","last_change":"2023-10-08T16:06:17.583530+0000","last_active":"2023-10-08T16:06:17.583530+0000","last_peered":"2023-10-08T16:06:17.583530+0000","last_clean":"2023-10-08T16:06:17.583530+0000","last_became_active":"2023-10-08T16:06:17.583293+0000","last_became_peered":"2023-10-08T16:06:17.583293+0000","last_unstale":"2023-10-08T16:06:17.583530+0000","last_undegraded":"2023-10-08T16:06:17.583530+0000","last_fullsized":"2023-10-08T16:06:17.583530+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:01.142901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576153+0000","last_change":"2023-10-08T16:06:17.576153+0000","last_active":"2023-10-08T16:06:17.576153+0000","last_peered":"2023-10-08T16:06:17.576153+0000","last_clean":"2023-10-08T16:06:17.576153+0000","last_became_active":"2023-10-08T16:06:17.575914+0000","last_became_peered":"2023-10-08T16:06:17.575914+0000","last_unstale":"2023-10-08T16:06:17.576153+0000","last_undegraded":"2023-10-08T16:06:17.576153+0000","last_fullsized":"2023-10-08T16:06:17.576153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:04:31.062457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.586782+0000","last_change":"2023-10-08T16:06:17.586108+0000","last_active":"2023-10-08T16:06:17.586782+0000","last_peered":"2023-10-08T16:06:17.586782+0000","last_clean":"2023-10-08T16:06:17.586782+0000","last_became_active":"2023-10-08T16:06:17.585852+0000","last_became_peered":"2023-10-08T16:06:17.585852+0000","last_unstale":"2023-10-08T16:06:17.586782+0000","last_undegraded":"2023-10-08T16:06:17.586782+0000","last_fullsized":"2023-10-08T16:06:17.586782+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:39.944057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587946+0000","last_change":"2023-10-08T16:06:17.584639+0000","last_active":"2023-10-08T16:06:17.587946+0000","last_peered":"2023-10-08T16:06:17.587946+0000","last_clean":"2023-10-08T16:06:17.587946+0000","last_became_active":"2023-10-08T16:06:17.583921+0000","last_became_peered":"2023-10-08T16:06:17.583921+0000","last_unstale":"2023-10-08T16:06:17.587946+0000","last_undegraded":"2023-10-08T16:06:17.587946+0000","last_fullsized":"2023-10-08T16:06:17.587946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:21:38.013447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575797+0000","last_change":"2023-10-08T16:06:17.575797+0000","last_active":"2023-10-08T16:06:17.575797+0000","last_peered":"2023-10-08T16:06:17.575797+0000","last_clean":"2023-10-08T16:06:17.575797+0000","last_became_active":"2023-10-08T16:06:17.575463+0000","last_became_peered":"2023-10-08T16:06:17.575463+0000","last_unstale":"2023-10-08T16:06:17.575797+0000","last_undegraded":"2023-10-08T16:06:17.575797+0000","last_fullsized":"2023-10-08T16:06:17.575797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:33.723028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569742+0000","last_change":"2023-10-08T16:06:17.569742+0000","last_active":"2023-10-08T16:06:17.569742+0000","last_peered":"2023-10-08T16:06:17.569742+0000","last_clean":"2023-10-08T16:06:17.569742+0000","last_became_active":"2023-10-08T16:06:17.569484+0000","last_became_peered":"2023-10-08T16:06:17.569484+0000","last_unstale":"2023-10-08T16:06:17.569742+0000","last_undegraded":"2023-10-08T16:06:17.569742+0000","last_fullsized":"2023-10-08T16:06:17.569742+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:38:24.198646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572410+0000","last_change":"2023-10-08T16:06:17.572410+0000","last_active":"2023-10-08T16:06:17.572410+0000","last_peered":"2023-10-08T16:06:17.572410+0000","last_clean":"2023-10-08T16:06:17.572410+0000","last_became_active":"2023-10-08T16:06:17.572111+0000","last_became_peered":"2023-10-08T16:06:17.572111+0000","last_unstale":"2023-10-08T16:06:17.572410+0000","last_undegraded":"2023-10-08T16:06:17.572410+0000","last_fullsized":"2023-10-08T16:06:17.572410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:53:06.363027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578580+0000","last_change":"2023-10-08T16:06:17.575819+0000","last_active":"2023-10-08T16:06:17.578580+0000","last_peered":"2023-10-08T16:06:17.578580+0000","last_clean":"2023-10-08T16:06:17.578580+0000","last_became_active":"2023-10-08T16:06:17.575489+0000","last_became_peered":"2023-10-08T16:06:17.575489+0000","last_unstale":"2023-10-08T16:06:17.578580+0000","last_undegraded":"2023-10-08T16:06:17.578580+0000","last_fullsized":"2023-10-08T16:06:17.578580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:41:26.128329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584458+0000","last_change":"2023-10-08T16:06:17.582694+0000","last_active":"2023-10-08T16:06:17.584458+0000","last_peered":"2023-10-08T16:06:17.584458+0000","last_clean":"2023-10-08T16:06:17.584458+0000","last_became_active":"2023-10-08T16:06:17.582448+0000","last_became_peered":"2023-10-08T16:06:17.582448+0000","last_unstale":"2023-10-08T16:06:17.584458+0000","last_undegraded":"2023-10-08T16:06:17.584458+0000","last_fullsized":"2023-10-08T16:06:17.584458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:04.149377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581421+0000","last_change":"2023-10-08T16:06:17.581421+0000","last_active":"2023-10-08T16:06:17.581421+0000","last_peered":"2023-10-08T16:06:17.581421+0000","last_clean":"2023-10-08T16:06:17.581421+0000","last_became_active":"2023-10-08T16:06:17.581144+0000","last_became_peered":"2023-10-08T16:06:17.581144+0000","last_unstale":"2023-10-08T16:06:17.581421+0000","last_undegraded":"2023-10-08T16:06:17.581421+0000","last_fullsized":"2023-10-08T16:06:17.581421+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:13.035462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.590160+0000","last_change":"2023-10-08T16:06:17.586190+0000","last_active":"2023-10-08T16:06:17.590160+0000","last_peered":"2023-10-08T16:06:17.590160+0000","last_clean":"2023-10-08T16:06:17.590160+0000","last_became_active":"2023-10-08T16:06:17.585906+0000","last_became_peered":"2023-10-08T16:06:17.585906+0000","last_unstale":"2023-10-08T16:06:17.590160+0000","last_undegraded":"2023-10-08T16:06:17.590160+0000","last_fullsized":"2023-10-08T16:06:17.590160+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:52:38.553596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569630+0000","last_change":"2023-10-08T16:06:17.569630+0000","last_active":"2023-10-08T16:06:17.569630+0000","last_peered":"2023-10-08T16:06:17.569630+0000","last_clean":"2023-10-08T16:06:17.569630+0000","last_became_active":"2023-10-08T16:06:17.569334+0000","last_became_peered":"2023-10-08T16:06:17.569334+0000","last_unstale":"2023-10-08T16:06:17.569630+0000","last_undegraded":"2023-10-08T16:06:17.569630+0000","last_fullsized":"2023-10-08T16:06:17.569630+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:55:54.632994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570845+0000","last_change":"2023-10-08T16:06:17.570845+0000","last_active":"2023-10-08T16:06:17.570845+0000","last_peered":"2023-10-08T16:06:17.570845+0000","last_clean":"2023-10-08T16:06:17.570845+0000","last_became_active":"2023-10-08T16:06:17.570561+0000","last_became_peered":"2023-10-08T16:06:17.570561+0000","last_unstale":"2023-10-08T16:06:17.570845+0000","last_undegraded":"2023-10-08T16:06:17.570845+0000","last_fullsized":"2023-10-08T16:06:17.570845+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:44:26.187851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574170+0000","last_change":"2023-10-08T16:06:17.574170+0000","last_active":"2023-10-08T16:06:17.574170+0000","last_peered":"2023-10-08T16:06:17.574170+0000","last_clean":"2023-10-08T16:06:17.574170+0000","last_became_active":"2023-10-08T16:06:17.573932+0000","last_became_peered":"2023-10-08T16:06:17.573932+0000","last_unstale":"2023-10-08T16:06:17.574170+0000","last_undegraded":"2023-10-08T16:06:17.574170+0000","last_fullsized":"2023-10-08T16:06:17.574170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:20:54.757888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.590168+0000","last_change":"2023-10-08T16:06:17.586233+0000","last_active":"2023-10-08T16:06:17.590168+0000","last_peered":"2023-10-08T16:06:17.590168+0000","last_clean":"2023-10-08T16:06:17.590168+0000","last_became_active":"2023-10-08T16:06:17.585961+0000","last_became_peered":"2023-10-08T16:06:17.585961+0000","last_unstale":"2023-10-08T16:06:17.590168+0000","last_undegraded":"2023-10-08T16:06:17.590168+0000","last_fullsized":"2023-10-08T16:06:17.590168+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:36.461052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587902+0000","last_change":"2023-10-08T16:06:17.579178+0000","last_active":"2023-10-08T16:06:17.587902+0000","last_peered":"2023-10-08T16:06:17.587902+0000","last_clean":"2023-10-08T16:06:17.587902+0000","last_became_active":"2023-10-08T16:06:17.578917+0000","last_became_peered":"2023-10-08T16:06:17.578917+0000","last_unstale":"2023-10-08T16:06:17.587902+0000","last_undegraded":"2023-10-08T16:06:17.587902+0000","last_fullsized":"2023-10-08T16:06:17.587902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:44.200579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587601+0000","last_change":"2023-10-08T16:06:17.582483+0000","last_active":"2023-10-08T16:06:17.587601+0000","last_peered":"2023-10-08T16:06:17.587601+0000","last_clean":"2023-10-08T16:06:17.587601+0000","last_became_active":"2023-10-08T16:06:17.581826+0000","last_became_peered":"2023-10-08T16:06:17.581826+0000","last_unstale":"2023-10-08T16:06:17.587601+0000","last_undegraded":"2023-10-08T16:06:17.587601+0000","last_fullsized":"2023-10-08T16:06:17.587601+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:57:00.269685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587433+0000","last_change":"2023-10-08T16:06:17.583441+0000","last_active":"2023-10-08T16:06:17.587433+0000","last_peered":"2023-10-08T16:06:17.587433+0000","last_clean":"2023-10-08T16:06:17.587433+0000","last_became_active":"2023-10-08T16:06:17.583226+0000","last_became_peered":"2023-10-08T16:06:17.583226+0000","last_unstale":"2023-10-08T16:06:17.587433+0000","last_undegraded":"2023-10-08T16:06:17.587433+0000","last_fullsized":"2023-10-08T16:06:17.587433+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:07.326980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"62'4","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580034+0000","last_change":"2023-10-08T16:06:16.575523+0000","last_active":"2023-10-08T16:06:17.580034+0000","last_peered":"2023-10-08T16:06:17.580034+0000","last_clean":"2023-10-08T16:06:17.580034+0000","last_became_active":"2023-10-08T16:06:16.575219+0000","last_became_peered":"2023-10-08T16:06:16.575219+0000","last_unstale":"2023-10-08T16:06:17.580034+0000","last_undegraded":"2023-10-08T16:06:17.580034+0000","last_fullsized":"2023-10-08T16:06:17.580034+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:42.415287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572793+0000","last_change":"2023-10-08T16:06:17.572793+0000","last_active":"2023-10-08T16:06:17.572793+0000","last_peered":"2023-10-08T16:06:17.572793+0000","last_clean":"2023-10-08T16:06:17.572793+0000","last_became_active":"2023-10-08T16:06:17.572520+0000","last_became_peered":"2023-10-08T16:06:17.572520+0000","last_unstale":"2023-10-08T16:06:17.572793+0000","last_undegraded":"2023-10-08T16:06:17.572793+0000","last_fullsized":"2023-10-08T16:06:17.572793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:11.636902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"62'1","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.567579+0000","last_change":"2023-10-08T16:06:16.568287+0000","last_active":"2023-10-08T16:06:17.567579+0000","last_peered":"2023-10-08T16:06:17.567579+0000","last_clean":"2023-10-08T16:06:17.567579+0000","last_became_active":"2023-10-08T16:06:16.568049+0000","last_became_peered":"2023-10-08T16:06:16.568049+0000","last_unstale":"2023-10-08T16:06:17.567579+0000","last_undegraded":"2023-10-08T16:06:17.567579+0000","last_fullsized":"2023-10-08T16:06:17.567579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:05.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588057+0000","last_change":"2023-10-08T16:06:17.585911+0000","last_active":"2023-10-08T16:06:17.588057+0000","last_peered":"2023-10-08T16:06:17.588057+0000","last_clean":"2023-10-08T16:06:17.588057+0000","last_became_active":"2023-10-08T16:06:17.585651+0000","last_became_peered":"2023-10-08T16:06:17.585651+0000","last_unstale":"2023-10-08T16:06:17.588057+0000","last_undegraded":"2023-10-08T16:06:17.588057+0000","last_fullsized":"2023-10-08T16:06:17.588057+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:06.775660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"62'2","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584433+0000","last_change":"2023-10-08T16:06:16.567215+0000","last_active":"2023-10-08T16:06:17.584433+0000","last_peered":"2023-10-08T16:06:17.584433+0000","last_clean":"2023-10-08T16:06:17.584433+0000","last_became_active":"2023-10-08T16:06:16.566911+0000","last_became_peered":"2023-10-08T16:06:16.566911+0000","last_unstale":"2023-10-08T16:06:17.584433+0000","last_undegraded":"2023-10-08T16:06:17.584433+0000","last_fullsized":"2023-10-08T16:06:17.584433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:01:39.638715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575292+0000","last_change":"2023-10-08T16:06:17.569196+0000","last_active":"2023-10-08T16:06:17.575292+0000","last_peered":"2023-10-08T16:06:17.575292+0000","last_clean":"2023-10-08T16:06:17.575292+0000","last_became_active":"2023-10-08T16:06:17.568888+0000","last_became_peered":"2023-10-08T16:06:17.568888+0000","last_unstale":"2023-10-08T16:06:17.575292+0000","last_undegraded":"2023-10-08T16:06:17.575292+0000","last_fullsized":"2023-10-08T16:06:17.575292+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:57:49.527701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587385+0000","last_change":"2023-10-08T16:06:16.570685+0000","last_active":"2023-10-08T16:06:17.587385+0000","last_peered":"2023-10-08T16:06:17.587385+0000","last_clean":"2023-10-08T16:06:17.587385+0000","last_became_active":"2023-10-08T16:06:16.570337+0000","last_became_peered":"2023-10-08T16:06:16.570337+0000","last_unstale":"2023-10-08T16:06:17.587385+0000","last_undegraded":"2023-10-08T16:06:17.587385+0000","last_fullsized":"2023-10-08T16:06:17.587385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:37.048819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581285+0000","last_change":"2023-10-08T16:06:17.574860+0000","last_active":"2023-10-08T16:06:17.581285+0000","last_peered":"2023-10-08T16:06:17.581285+0000","last_clean":"2023-10-08T16:06:17.581285+0000","last_became_active":"2023-10-08T16:06:17.574560+0000","last_became_peered":"2023-10-08T16:06:17.574560+0000","last_unstale":"2023-10-08T16:06:17.581285+0000","last_undegraded":"2023-10-08T16:06:17.581285+0000","last_fullsized":"2023-10-08T16:06:17.581285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:52:33.723028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"62'1","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.558927+0000","last_change":"2023-10-08T16:06:16.566783+0000","last_active":"2023-10-08T16:06:17.558927+0000","last_peered":"2023-10-08T16:06:17.558927+0000","last_clean":"2023-10-08T16:06:17.558927+0000","last_became_active":"2023-10-08T16:06:16.566483+0000","last_became_peered":"2023-10-08T16:06:16.566483+0000","last_unstale":"2023-10-08T16:06:17.558927+0000","last_undegraded":"2023-10-08T16:06:17.558927+0000","last_fullsized":"2023-10-08T16:06:17.558927+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:13.463499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576349+0000","last_change":"2023-10-08T16:06:17.576349+0000","last_active":"2023-10-08T16:06:17.576349+0000","last_peered":"2023-10-08T16:06:17.576349+0000","last_clean":"2023-10-08T16:06:17.576349+0000","last_became_active":"2023-10-08T16:06:17.576002+0000","last_became_peered":"2023-10-08T16:06:17.576002+0000","last_unstale":"2023-10-08T16:06:17.576349+0000","last_undegraded":"2023-10-08T16:06:17.576349+0000","last_fullsized":"2023-10-08T16:06:17.576349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:49:43.420587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583615+0000","last_change":"2023-10-08T16:06:16.566809+0000","last_active":"2023-10-08T16:06:17.583615+0000","last_peered":"2023-10-08T16:06:17.583615+0000","last_clean":"2023-10-08T16:06:17.583615+0000","last_became_active":"2023-10-08T16:06:16.566496+0000","last_became_peered":"2023-10-08T16:06:16.566496+0000","last_unstale":"2023-10-08T16:06:17.583615+0000","last_undegraded":"2023-10-08T16:06:17.583615+0000","last_fullsized":"2023-10-08T16:06:17.583615+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:09:05.841825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570775+0000","last_change":"2023-10-08T16:06:17.570775+0000","last_active":"2023-10-08T16:06:17.570775+0000","last_peered":"2023-10-08T16:06:17.570775+0000","last_clean":"2023-10-08T16:06:17.570775+0000","last_became_active":"2023-10-08T16:06:17.570541+0000","last_became_peered":"2023-10-08T16:06:17.570541+0000","last_unstale":"2023-10-08T16:06:17.570775+0000","last_undegraded":"2023-10-08T16:06:17.570775+0000","last_fullsized":"2023-10-08T16:06:17.570775+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:30.121237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.554848+0000","last_change":"2023-10-08T16:06:16.564774+0000","last_active":"2023-10-08T16:06:17.554848+0000","last_peered":"2023-10-08T16:06:17.554848+0000","last_clean":"2023-10-08T16:06:17.554848+0000","last_became_active":"2023-10-08T16:06:16.564485+0000","last_became_peered":"2023-10-08T16:06:16.564485+0000","last_unstale":"2023-10-08T16:06:17.554848+0000","last_undegraded":"2023-10-08T16:06:17.554848+0000","last_fullsized":"2023-10-08T16:06:17.554848+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:53.343775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576232+0000","last_change":"2023-10-08T16:06:17.576232+0000","last_active":"2023-10-08T16:06:17.576232+0000","last_peered":"2023-10-08T16:06:17.576232+0000","last_clean":"2023-10-08T16:06:17.576232+0000","last_became_active":"2023-10-08T16:06:17.575675+0000","last_became_peered":"2023-10-08T16:06:17.575675+0000","last_unstale":"2023-10-08T16:06:17.576232+0000","last_undegraded":"2023-10-08T16:06:17.576232+0000","last_fullsized":"2023-10-08T16:06:17.576232+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:26.856007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567386+0000","last_change":"2023-10-08T16:06:16.564164+0000","last_active":"2023-10-08T16:06:16.567386+0000","last_peered":"2023-10-08T16:06:16.567386+0000","last_clean":"2023-10-08T16:06:16.567386+0000","last_became_active":"2023-10-08T16:06:16.563910+0000","last_became_peered":"2023-10-08T16:06:16.563910+0000","last_unstale":"2023-10-08T16:06:16.567386+0000","last_undegraded":"2023-10-08T16:06:16.567386+0000","last_fullsized":"2023-10-08T16:06:16.567386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:18.369942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581337+0000","last_change":"2023-10-08T16:06:17.578005+0000","last_active":"2023-10-08T16:06:17.581337+0000","last_peered":"2023-10-08T16:06:17.581337+0000","last_clean":"2023-10-08T16:06:17.581337+0000","last_became_active":"2023-10-08T16:06:17.577737+0000","last_became_peered":"2023-10-08T16:06:17.577737+0000","last_unstale":"2023-10-08T16:06:17.581337+0000","last_undegraded":"2023-10-08T16:06:17.581337+0000","last_fullsized":"2023-10-08T16:06:17.581337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:30:01.759178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.568086+0000","last_change":"2023-10-08T16:06:16.563719+0000","last_active":"2023-10-08T16:06:16.568086+0000","last_peered":"2023-10-08T16:06:16.568086+0000","last_clean":"2023-10-08T16:06:16.568086+0000","last_became_active":"2023-10-08T16:06:16.563409+0000","last_became_peered":"2023-10-08T16:06:16.563409+0000","last_unstale":"2023-10-08T16:06:16.568086+0000","last_undegraded":"2023-10-08T16:06:16.568086+0000","last_fullsized":"2023-10-08T16:06:16.568086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:34.904243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587945+0000","last_change":"2023-10-08T16:06:17.586859+0000","last_active":"2023-10-08T16:06:17.587945+0000","last_peered":"2023-10-08T16:06:17.587945+0000","last_clean":"2023-10-08T16:06:17.587945+0000","last_became_active":"2023-10-08T16:06:17.586669+0000","last_became_peered":"2023-10-08T16:06:17.586669+0000","last_unstale":"2023-10-08T16:06:17.587945+0000","last_undegraded":"2023-10-08T16:06:17.587945+0000","last_fullsized":"2023-10-08T16:06:17.587945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:14.468800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575990+0000","last_change":"2023-10-08T16:06:16.568819+0000","last_active":"2023-10-08T16:06:17.575990+0000","last_peered":"2023-10-08T16:06:17.575990+0000","last_clean":"2023-10-08T16:06:17.575990+0000","last_became_active":"2023-10-08T16:06:16.568296+0000","last_became_peered":"2023-10-08T16:06:16.568296+0000","last_unstale":"2023-10-08T16:06:17.575990+0000","last_undegraded":"2023-10-08T16:06:17.575990+0000","last_fullsized":"2023-10-08T16:06:17.575990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:57.703766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587960+0000","last_change":"2023-10-08T16:06:17.583544+0000","last_active":"2023-10-08T16:06:17.587960+0000","last_peered":"2023-10-08T16:06:17.587960+0000","last_clean":"2023-10-08T16:06:17.587960+0000","last_became_active":"2023-10-08T16:06:17.583272+0000","last_became_peered":"2023-10-08T16:06:17.583272+0000","last_unstale":"2023-10-08T16:06:17.587960+0000","last_undegraded":"2023-10-08T16:06:17.587960+0000","last_fullsized":"2023-10-08T16:06:17.587960+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:46:37.113115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"62'3","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.569787+0000","last_change":"2023-10-08T16:06:16.566219+0000","last_active":"2023-10-08T16:06:17.569787+0000","last_peered":"2023-10-08T16:06:17.569787+0000","last_clean":"2023-10-08T16:06:17.569787+0000","last_became_active":"2023-10-08T16:06:16.565768+0000","last_became_peered":"2023-10-08T16:06:16.565768+0000","last_unstale":"2023-10-08T16:06:17.569787+0000","last_undegraded":"2023-10-08T16:06:17.569787+0000","last_fullsized":"2023-10-08T16:06:17.569787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:23.233110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573859+0000","last_change":"2023-10-08T16:06:17.573859+0000","last_active":"2023-10-08T16:06:17.573859+0000","last_peered":"2023-10-08T16:06:17.573859+0000","last_clean":"2023-10-08T16:06:17.573859+0000","last_became_active":"2023-10-08T16:06:17.573038+0000","last_became_peered":"2023-10-08T16:06:17.573038+0000","last_unstale":"2023-10-08T16:06:17.573859+0000","last_undegraded":"2023-10-08T16:06:17.573859+0000","last_fullsized":"2023-10-08T16:06:17.573859+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:16:39.701235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"62'6","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577903+0000","last_change":"2023-10-08T16:06:16.563616+0000","last_active":"2023-10-08T16:06:17.577903+0000","last_peered":"2023-10-08T16:06:17.577903+0000","last_clean":"2023-10-08T16:06:17.577903+0000","last_became_active":"2023-10-08T16:06:16.563288+0000","last_became_peered":"2023-10-08T16:06:16.563288+0000","last_unstale":"2023-10-08T16:06:17.577903+0000","last_undegraded":"2023-10-08T16:06:17.577903+0000","last_fullsized":"2023-10-08T16:06:17.577903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:34.904243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.575110+0000","last_change":"2023-10-08T16:06:17.569203+0000","last_active":"2023-10-08T16:06:17.575110+0000","last_peered":"2023-10-08T16:06:17.575110+0000","last_clean":"2023-10-08T16:06:17.575110+0000","last_became_active":"2023-10-08T16:06:17.568974+0000","last_became_peered":"2023-10-08T16:06:17.568974+0000","last_unstale":"2023-10-08T16:06:17.575110+0000","last_undegraded":"2023-10-08T16:06:17.575110+0000","last_fullsized":"2023-10-08T16:06:17.575110+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:09:38.364733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.566943+0000","last_change":"2023-10-08T16:06:16.575012+0000","last_active":"2023-10-08T16:06:17.566943+0000","last_peered":"2023-10-08T16:06:17.566943+0000","last_clean":"2023-10-08T16:06:17.566943+0000","last_became_active":"2023-10-08T16:06:16.574772+0000","last_became_peered":"2023-10-08T16:06:16.574772+0000","last_unstale":"2023-10-08T16:06:17.566943+0000","last_undegraded":"2023-10-08T16:06:17.566943+0000","last_fullsized":"2023-10-08T16:06:17.566943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:56.930770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.581334+0000","last_change":"2023-10-08T16:06:17.576974+0000","last_active":"2023-10-08T16:06:17.581334+0000","last_peered":"2023-10-08T16:06:17.581334+0000","last_clean":"2023-10-08T16:06:17.581334+0000","last_became_active":"2023-10-08T16:06:17.576726+0000","last_became_peered":"2023-10-08T16:06:17.576726+0000","last_unstale":"2023-10-08T16:06:17.581334+0000","last_undegraded":"2023-10-08T16:06:17.581334+0000","last_fullsized":"2023-10-08T16:06:17.581334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:53.909664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578592+0000","last_change":"2023-10-08T16:06:16.568193+0000","last_active":"2023-10-08T16:06:17.578592+0000","last_peered":"2023-10-08T16:06:17.578592+0000","last_clean":"2023-10-08T16:06:17.578592+0000","last_became_active":"2023-10-08T16:06:16.567897+0000","last_became_peered":"2023-10-08T16:06:16.567897+0000","last_unstale":"2023-10-08T16:06:17.578592+0000","last_undegraded":"2023-10-08T16:06:17.578592+0000","last_fullsized":"2023-10-08T16:06:17.578592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:23.679170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.582733+0000","last_change":"2023-10-08T16:06:17.582733+0000","last_active":"2023-10-08T16:06:17.582733+0000","last_peered":"2023-10-08T16:06:17.582733+0000","last_clean":"2023-10-08T16:06:17.582733+0000","last_became_active":"2023-10-08T16:06:17.582435+0000","last_became_peered":"2023-10-08T16:06:17.582435+0000","last_unstale":"2023-10-08T16:06:17.582733+0000","last_undegraded":"2023-10-08T16:06:17.582733+0000","last_fullsized":"2023-10-08T16:06:17.582733+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:44.200579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573837+0000","last_change":"2023-10-08T16:06:16.565813+0000","last_active":"2023-10-08T16:06:17.573837+0000","last_peered":"2023-10-08T16:06:17.573837+0000","last_clean":"2023-10-08T16:06:17.573837+0000","last_became_active":"2023-10-08T16:06:16.565453+0000","last_became_peered":"2023-10-08T16:06:16.565453+0000","last_unstale":"2023-10-08T16:06:17.573837+0000","last_undegraded":"2023-10-08T16:06:17.573837+0000","last_fullsized":"2023-10-08T16:06:17.573837+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:08.675593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.572578+0000","last_change":"2023-10-08T16:06:17.572578+0000","last_active":"2023-10-08T16:06:17.572578+0000","last_peered":"2023-10-08T16:06:17.572578+0000","last_clean":"2023-10-08T16:06:17.572578+0000","last_became_active":"2023-10-08T16:06:17.572344+0000","last_became_peered":"2023-10-08T16:06:17.572344+0000","last_unstale":"2023-10-08T16:06:17.572578+0000","last_undegraded":"2023-10-08T16:06:17.572578+0000","last_fullsized":"2023-10-08T16:06:17.572578+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:45.306918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"62'8","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576599+0000","last_change":"2023-10-08T16:06:16.575950+0000","last_active":"2023-10-08T16:06:17.576599+0000","last_peered":"2023-10-08T16:06:17.576599+0000","last_clean":"2023-10-08T16:06:17.576599+0000","last_became_active":"2023-10-08T16:06:16.575609+0000","last_became_peered":"2023-10-08T16:06:16.575609+0000","last_unstale":"2023-10-08T16:06:17.576599+0000","last_undegraded":"2023-10-08T16:06:17.576599+0000","last_fullsized":"2023-10-08T16:06:17.576599+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:56.930770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571695+0000","last_change":"2023-10-08T16:06:17.571695+0000","last_active":"2023-10-08T16:06:17.571695+0000","last_peered":"2023-10-08T16:06:17.571695+0000","last_clean":"2023-10-08T16:06:17.571695+0000","last_became_active":"2023-10-08T16:06:17.571189+0000","last_became_peered":"2023-10-08T16:06:17.571189+0000","last_unstale":"2023-10-08T16:06:17.571695+0000","last_undegraded":"2023-10-08T16:06:17.571695+0000","last_fullsized":"2023-10-08T16:06:17.571695+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:01.867877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583741+0000","last_change":"2023-10-08T16:06:16.569672+0000","last_active":"2023-10-08T16:06:17.583741+0000","last_peered":"2023-10-08T16:06:17.583741+0000","last_clean":"2023-10-08T16:06:17.583741+0000","last_became_active":"2023-10-08T16:06:16.569348+0000","last_became_peered":"2023-10-08T16:06:16.569348+0000","last_unstale":"2023-10-08T16:06:17.583741+0000","last_undegraded":"2023-10-08T16:06:17.583741+0000","last_fullsized":"2023-10-08T16:06:17.583741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:32.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.580120+0000","last_change":"2023-10-08T16:06:17.567245+0000","last_active":"2023-10-08T16:06:17.580120+0000","last_peered":"2023-10-08T16:06:17.580120+0000","last_clean":"2023-10-08T16:06:17.580120+0000","last_became_active":"2023-10-08T16:06:17.566060+0000","last_became_peered":"2023-10-08T16:06:17.566060+0000","last_unstale":"2023-10-08T16:06:17.580120+0000","last_undegraded":"2023-10-08T16:06:17.580120+0000","last_fullsized":"2023-10-08T16:06:17.580120+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:21.572253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"62'2","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584369+0000","last_change":"2023-10-08T16:06:16.569102+0000","last_active":"2023-10-08T16:06:17.584369+0000","last_peered":"2023-10-08T16:06:17.584369+0000","last_clean":"2023-10-08T16:06:17.584369+0000","last_became_active":"2023-10-08T16:06:16.568872+0000","last_became_peered":"2023-10-08T16:06:16.568872+0000","last_unstale":"2023-10-08T16:06:17.584369+0000","last_undegraded":"2023-10-08T16:06:17.584369+0000","last_fullsized":"2023-10-08T16:06:17.584369+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:01.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.577101+0000","last_change":"2023-10-08T16:06:17.569298+0000","last_active":"2023-10-08T16:06:17.577101+0000","last_peered":"2023-10-08T16:06:17.577101+0000","last_clean":"2023-10-08T16:06:17.577101+0000","last_became_active":"2023-10-08T16:06:17.569030+0000","last_became_peered":"2023-10-08T16:06:17.569030+0000","last_unstale":"2023-10-08T16:06:17.577101+0000","last_undegraded":"2023-10-08T16:06:17.577101+0000","last_fullsized":"2023-10-08T16:06:17.577101+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:18.434623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"62'1","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588290+0000","last_change":"2023-10-08T16:06:16.569843+0000","last_active":"2023-10-08T16:06:17.588290+0000","last_peered":"2023-10-08T16:06:17.588290+0000","last_clean":"2023-10-08T16:06:17.588290+0000","last_became_active":"2023-10-08T16:06:16.568502+0000","last_became_peered":"2023-10-08T16:06:16.568502+0000","last_unstale":"2023-10-08T16:06:17.588290+0000","last_undegraded":"2023-10-08T16:06:17.588290+0000","last_fullsized":"2023-10-08T16:06:17.588290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:05:10.631601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570330+0000","last_change":"2023-10-08T16:06:17.570330+0000","last_active":"2023-10-08T16:06:17.570330+0000","last_peered":"2023-10-08T16:06:17.570330+0000","last_clean":"2023-10-08T16:06:17.570330+0000","last_became_active":"2023-10-08T16:06:17.570095+0000","last_became_peered":"2023-10-08T16:06:17.570095+0000","last_unstale":"2023-10-08T16:06:17.570330+0000","last_undegraded":"2023-10-08T16:06:17.570330+0000","last_fullsized":"2023-10-08T16:06:17.570330+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.709067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.560852+0000","last_change":"2023-10-08T16:06:16.575536+0000","last_active":"2023-10-08T16:06:17.560852+0000","last_peered":"2023-10-08T16:06:17.560852+0000","last_clean":"2023-10-08T16:06:17.560852+0000","last_became_active":"2023-10-08T16:06:16.575227+0000","last_became_peered":"2023-10-08T16:06:16.575227+0000","last_unstale":"2023-10-08T16:06:17.560852+0000","last_undegraded":"2023-10-08T16:06:17.560852+0000","last_fullsized":"2023-10-08T16:06:17.560852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:21.723041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.563281+0000","last_change":"2023-10-08T16:06:17.563281+0000","last_active":"2023-10-08T16:06:17.563281+0000","last_peered":"2023-10-08T16:06:17.563281+0000","last_clean":"2023-10-08T16:06:17.563281+0000","last_became_active":"2023-10-08T16:06:17.563038+0000","last_became_peered":"2023-10-08T16:06:17.563038+0000","last_unstale":"2023-10-08T16:06:17.563281+0000","last_undegraded":"2023-10-08T16:06:17.563281+0000","last_fullsized":"2023-10-08T16:06:17.563281+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:34:04.939546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.578687+0000","last_change":"2023-10-08T16:06:16.566260+0000","last_active":"2023-10-08T16:06:17.578687+0000","last_peered":"2023-10-08T16:06:17.578687+0000","last_clean":"2023-10-08T16:06:17.578687+0000","last_became_active":"2023-10-08T16:06:16.565937+0000","last_became_peered":"2023-10-08T16:06:16.565937+0000","last_unstale":"2023-10-08T16:06:17.578687+0000","last_undegraded":"2023-10-08T16:06:17.578687+0000","last_fullsized":"2023-10-08T16:06:17.578687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:21:22.211819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583353+0000","last_change":"2023-10-08T16:06:17.583353+0000","last_active":"2023-10-08T16:06:17.583353+0000","last_peered":"2023-10-08T16:06:17.583353+0000","last_clean":"2023-10-08T16:06:17.583353+0000","last_became_active":"2023-10-08T16:06:17.583134+0000","last_became_peered":"2023-10-08T16:06:17.583134+0000","last_unstale":"2023-10-08T16:06:17.583353+0000","last_undegraded":"2023-10-08T16:06:17.583353+0000","last_fullsized":"2023-10-08T16:06:17.583353+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:31:43.414492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.573960+0000","last_change":"2023-10-08T16:06:16.568615+0000","last_active":"2023-10-08T16:06:17.573960+0000","last_peered":"2023-10-08T16:06:17.573960+0000","last_clean":"2023-10-08T16:06:17.573960+0000","last_became_active":"2023-10-08T16:06:16.567453+0000","last_became_peered":"2023-10-08T16:06:16.567453+0000","last_unstale":"2023-10-08T16:06:17.573960+0000","last_undegraded":"2023-10-08T16:06:17.573960+0000","last_fullsized":"2023-10-08T16:06:17.573960+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:05.770359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.583922+0000","last_change":"2023-10-08T16:06:17.583922+0000","last_active":"2023-10-08T16:06:17.583922+0000","last_peered":"2023-10-08T16:06:17.583922+0000","last_clean":"2023-10-08T16:06:17.583922+0000","last_became_active":"2023-10-08T16:06:17.583698+0000","last_became_peered":"2023-10-08T16:06:17.583698+0000","last_unstale":"2023-10-08T16:06:17.583922+0000","last_undegraded":"2023-10-08T16:06:17.583922+0000","last_fullsized":"2023-10-08T16:06:17.583922+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:40.309552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.569552+0000","last_change":"2023-10-08T16:06:16.568225+0000","last_active":"2023-10-08T16:06:16.569552+0000","last_peered":"2023-10-08T16:06:16.569552+0000","last_clean":"2023-10-08T16:06:16.569552+0000","last_became_active":"2023-10-08T16:06:16.567958+0000","last_became_peered":"2023-10-08T16:06:16.567958+0000","last_unstale":"2023-10-08T16:06:16.569552+0000","last_undegraded":"2023-10-08T16:06:16.569552+0000","last_fullsized":"2023-10-08T16:06:16.569552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:38.938757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.570681+0000","last_change":"2023-10-08T16:06:17.570681+0000","last_active":"2023-10-08T16:06:17.570681+0000","last_peered":"2023-10-08T16:06:17.570681+0000","last_clean":"2023-10-08T16:06:17.570681+0000","last_became_active":"2023-10-08T16:06:17.570361+0000","last_became_peered":"2023-10-08T16:06:17.570361+0000","last_unstale":"2023-10-08T16:06:17.570681+0000","last_undegraded":"2023-10-08T16:06:17.570681+0000","last_fullsized":"2023-10-08T16:06:17.570681+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:53.909664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587710+0000","last_change":"2023-10-08T16:06:16.569545+0000","last_active":"2023-10-08T16:06:17.587710+0000","last_peered":"2023-10-08T16:06:17.587710+0000","last_clean":"2023-10-08T16:06:17.587710+0000","last_became_active":"2023-10-08T16:06:16.569052+0000","last_became_peered":"2023-10-08T16:06:16.569052+0000","last_unstale":"2023-10-08T16:06:17.587710+0000","last_undegraded":"2023-10-08T16:06:17.587710+0000","last_fullsized":"2023-10-08T16:06:17.587710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:32:39.859359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.588021+0000","last_change":"2023-10-08T16:06:17.579285+0000","last_active":"2023-10-08T16:06:17.588021+0000","last_peered":"2023-10-08T16:06:17.588021+0000","last_clean":"2023-10-08T16:06:17.588021+0000","last_became_active":"2023-10-08T16:06:17.579024+0000","last_became_peered":"2023-10-08T16:06:17.579024+0000","last_unstale":"2023-10-08T16:06:17.588021+0000","last_undegraded":"2023-10-08T16:06:17.588021+0000","last_fullsized":"2023-10-08T16:06:17.588021+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:01.142901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.571674+0000","last_change":"2023-10-08T16:06:16.565882+0000","last_active":"2023-10-08T16:06:17.571674+0000","last_peered":"2023-10-08T16:06:17.571674+0000","last_clean":"2023-10-08T16:06:17.571674+0000","last_became_active":"2023-10-08T16:06:16.565503+0000","last_became_peered":"2023-10-08T16:06:16.565503+0000","last_unstale":"2023-10-08T16:06:17.571674+0000","last_undegraded":"2023-10-08T16:06:17.571674+0000","last_fullsized":"2023-10-08T16:06:17.571674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:53:05.219962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584434+0000","last_change":"2023-10-08T16:06:17.582081+0000","last_active":"2023-10-08T16:06:17.584434+0000","last_peered":"2023-10-08T16:06:17.584434+0000","last_clean":"2023-10-08T16:06:17.584434+0000","last_became_active":"2023-10-08T16:06:17.581841+0000","last_became_peered":"2023-10-08T16:06:17.581841+0000","last_unstale":"2023-10-08T16:06:17.584434+0000","last_undegraded":"2023-10-08T16:06:17.584434+0000","last_fullsized":"2023-10-08T16:06:17.584434+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:09.872516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567798+0000","last_change":"2023-10-08T16:06:16.565980+0000","last_active":"2023-10-08T16:06:16.567798+0000","last_peered":"2023-10-08T16:06:16.567798+0000","last_clean":"2023-10-08T16:06:16.567798+0000","last_became_active":"2023-10-08T16:06:16.565187+0000","last_became_peered":"2023-10-08T16:06:16.565187+0000","last_unstale":"2023-10-08T16:06:16.567798+0000","last_undegraded":"2023-10-08T16:06:16.567798+0000","last_fullsized":"2023-10-08T16:06:16.567798+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:42.514716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.576334+0000","last_change":"2023-10-08T16:06:17.574881+0000","last_active":"2023-10-08T16:06:17.576334+0000","last_peered":"2023-10-08T16:06:17.576334+0000","last_clean":"2023-10-08T16:06:17.576334+0000","last_became_active":"2023-10-08T16:06:17.574405+0000","last_became_peered":"2023-10-08T16:06:17.574405+0000","last_unstale":"2023-10-08T16:06:17.576334+0000","last_undegraded":"2023-10-08T16:06:17.576334+0000","last_fullsized":"2023-10-08T16:06:17.576334+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:57.285585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.569097+0000","last_change":"2023-10-08T16:06:16.567331+0000","last_active":"2023-10-08T16:06:16.569097+0000","last_peered":"2023-10-08T16:06:16.569097+0000","last_clean":"2023-10-08T16:06:16.569097+0000","last_became_active":"2023-10-08T16:06:16.567062+0000","last_became_peered":"2023-10-08T16:06:16.567062+0000","last_unstale":"2023-10-08T16:06:16.569097+0000","last_undegraded":"2023-10-08T16:06:16.569097+0000","last_fullsized":"2023-10-08T16:06:16.569097+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:08.675593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.574119+0000","last_change":"2023-10-08T16:06:17.574119+0000","last_active":"2023-10-08T16:06:17.574119+0000","last_peered":"2023-10-08T16:06:17.574119+0000","last_clean":"2023-10-08T16:06:17.574119+0000","last_became_active":"2023-10-08T16:06:17.573303+0000","last_became_peered":"2023-10-08T16:06:17.573303+0000","last_unstale":"2023-10-08T16:06:17.574119+0000","last_undegraded":"2023-10-08T16:06:17.574119+0000","last_fullsized":"2023-10-08T16:06:17.574119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:38.054120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.567455+0000","last_change":"2023-10-08T16:06:16.564016+0000","last_active":"2023-10-08T16:06:16.567455+0000","last_peered":"2023-10-08T16:06:16.567455+0000","last_clean":"2023-10-08T16:06:16.567455+0000","last_became_active":"2023-10-08T16:06:16.563425+0000","last_became_peered":"2023-10-08T16:06:16.563425+0000","last_unstale":"2023-10-08T16:06:16.567455+0000","last_undegraded":"2023-10-08T16:06:16.567455+0000","last_fullsized":"2023-10-08T16:06:16.567455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:01:54.467407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587679+0000","last_change":"2023-10-08T16:06:17.582917+0000","last_active":"2023-10-08T16:06:17.587679+0000","last_peered":"2023-10-08T16:06:17.587679+0000","last_clean":"2023-10-08T16:06:17.587679+0000","last_became_active":"2023-10-08T16:06:17.582519+0000","last_became_peered":"2023-10-08T16:06:17.582519+0000","last_unstale":"2023-10-08T16:06:17.587679+0000","last_undegraded":"2023-10-08T16:06:17.587679+0000","last_fullsized":"2023-10-08T16:06:17.587679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:18.434623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.554838+0000","last_change":"2023-10-08T16:06:16.564166+0000","last_active":"2023-10-08T16:06:17.554838+0000","last_peered":"2023-10-08T16:06:17.554838+0000","last_clean":"2023-10-08T16:06:17.554838+0000","last_became_active":"2023-10-08T16:06:16.563910+0000","last_became_peered":"2023-10-08T16:06:16.563910+0000","last_unstale":"2023-10-08T16:06:17.554838+0000","last_undegraded":"2023-10-08T16:06:17.554838+0000","last_fullsized":"2023-10-08T16:06:17.554838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:44:01.742329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.566959+0000","last_change":"2023-10-08T16:06:17.564528+0000","last_active":"2023-10-08T16:06:17.566959+0000","last_peered":"2023-10-08T16:06:17.566959+0000","last_clean":"2023-10-08T16:06:17.566959+0000","last_became_active":"2023-10-08T16:06:17.564208+0000","last_became_peered":"2023-10-08T16:06:17.564208+0000","last_unstale":"2023-10-08T16:06:17.566959+0000","last_undegraded":"2023-10-08T16:06:17.566959+0000","last_fullsized":"2023-10-08T16:06:17.566959+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:40:09.680894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-10-08T16:06:16.579605+0000","last_change":"2023-10-08T16:06:16.575359+0000","last_active":"2023-10-08T16:06:16.579605+0000","last_peered":"2023-10-08T16:06:16.579605+0000","last_clean":"2023-10-08T16:06:16.579605+0000","last_became_active":"2023-10-08T16:06:16.574975+0000","last_became_peered":"2023-10-08T16:06:16.574975+0000","last_unstale":"2023-10-08T16:06:16.579605+0000","last_undegraded":"2023-10-08T16:06:16.579605+0000","last_fullsized":"2023-10-08T16:06:16.579605+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:10:53.343775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.587153+0000","last_change":"2023-10-08T16:06:17.581897+0000","last_active":"2023-10-08T16:06:17.587153+0000","last_peered":"2023-10-08T16:06:17.587153+0000","last_clean":"2023-10-08T16:06:17.587153+0000","last_became_active":"2023-10-08T16:06:17.581619+0000","last_became_peered":"2023-10-08T16:06:17.581619+0000","last_unstale":"2023-10-08T16:06:17.587153+0000","last_undegraded":"2023-10-08T16:06:17.587153+0000","last_fullsized":"2023-10-08T16:06:17.587153+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:09:19.090222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.565001+0000","last_change":"2023-10-08T16:06:16.570676+0000","last_active":"2023-10-08T16:06:17.565001+0000","last_peered":"2023-10-08T16:06:17.565001+0000","last_clean":"2023-10-08T16:06:17.565001+0000","last_became_active":"2023-10-08T16:06:16.570313+0000","last_became_peered":"2023-10-08T16:06:16.570313+0000","last_unstale":"2023-10-08T16:06:17.565001+0000","last_undegraded":"2023-10-08T16:06:17.565001+0000","last_fullsized":"2023-10-08T16:06:17.565001+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:44.301618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":8,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.585985+0000","last_change":"2023-10-08T16:06:17.585985+0000","last_active":"2023-10-08T16:06:17.585985+0000","last_peered":"2023-10-08T16:06:17.585985+0000","last_clean":"2023-10-08T16:06:17.585985+0000","last_became_active":"2023-10-08T16:06:17.585689+0000","last_became_peered":"2023-10-08T16:06:17.585689+0000","last_unstale":"2023-10-08T16:06:17.585985+0000","last_undegraded":"2023-10-08T16:06:17.585985+0000","last_fullsized":"2023-10-08T16:06:17.585985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:16.544685+0000","last_clean_scrub_stamp":"2023-10-08T16:06:16.544685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:51.248645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"62'2","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.584327+0000","last_change":"2023-10-08T16:06:16.570493+0000","last_active":"2023-10-08T16:06:17.584327+0000","last_peered":"2023-10-08T16:06:17.584327+0000","last_clean":"2023-10-08T16:06:17.584327+0000","last_became_active":"2023-10-08T16:06:16.570214+0000","last_became_peered":"2023-10-08T16:06:16.570214+0000","last_unstale":"2023-10-08T16:06:17.584327+0000","last_undegraded":"2023-10-08T16:06:17.584327+0000","last_fullsized":"2023-10-08T16:06:17.584327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:06:15.539385+0000","last_clean_scrub_stamp":"2023-10-08T16:06:15.539385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:18.995016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":199,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-10-08T16:06:17.560956+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:06:17.560956+0000","last_peered":"2023-10-08T16:06:17.560956+0000","last_clean":"2023-10-08T16:06:17.560956+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:06:17.560956+0000","last_undegraded":"2023-10-08T16:06:17.560956+0000","last_fullsized":"2023-10-08T16:06:17.560956+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298624,"kb_used_data":1336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073216,"statfs":{"total":96636764160,"available":96330973184,"internally_reserved":0,"allocated":1368064,"data_stored":916027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":8.0939999999999994,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.41199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":8.2040000000000006,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.30399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":8.4339999999999993,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.373},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.3409999999999993,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.46999999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.5}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":318875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67000000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.755}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42299999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302140,"kb_used_data":756,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069700,"statfs":{"total":96636764160,"available":96327372800,"internally_reserved":0,"allocated":774144,"data_stored":320907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.312,"15min":0.312},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38},{"interface":"front","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.48299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52900000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.54000000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.439},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.51600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302700,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069140,"statfs":{"total":96636764160,"available":96326799360,"internally_reserved":0,"allocated":1347584,"data_stored":909151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":5.5599999999999996,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.434},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.37}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":5.5970000000000004,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.63100000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":3.2890000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52400000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.5049999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.68300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298036,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073804,"statfs":{"total":96636764160,"available":96331575296,"internally_reserved":0,"allocated":765952,"data_stored":320319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.371}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.316},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45500000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.48799999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.38900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302128,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069712,"statfs":{"total":96636764160,"available":96327385088,"internally_reserved":0,"allocated":761856,"data_stored":318895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.371},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.26600000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.38},{"interface":"front","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.29099999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.33600000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.628}]},{"osd":7,"last update":"Sun Oct 8 16:05:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.76100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":318853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.501},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.51500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.61099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.46800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":770048,"data_stored":325161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":4.0119999999999996,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.47899999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":12.227,"5min":12.227,"15min":12.227},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.50900000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.45300000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":9.3170000000000002,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.8719999999999999,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.441}]},{"osd":6,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:06:21.983 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:06:21.983 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:06:22.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.138+0000 7f1603b1d700 1 Processor -- start 2023-10-08T16:06:22.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.138+0000 7f1603b1d700 1 -- start start 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f1603b1d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15fc069c70 0x7f15fc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f1603b1d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f15fc0af320 0x7f15fc0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f1603b1d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc0ce380 con 0x7f15fc159950 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f1603b1d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0cec20 con 0x7f15fc0af320 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f1603b1d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc0afa90 con 0x7f15fc069c70 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f15fc0af320 0x7f15fc0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16018b9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f15fc159950 0x7f15fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52002/0 (socket says 172.21.15.170:52002) 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16018b9700 1 -- 172.21.15.170:0/3016360631 learned_addr learned my addr 172.21.15.170:0/3016360631 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16010b8700 1 --2- 172.21.15.170:0/3016360631 >> v2:172.21.15.191:3300/0 conn(0x7f15fc069c70 0x7f15fc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 -- 172.21.15.170:0/3016360631 >> v2:172.21.15.191:3300/0 conn(0x7f15fc069c70 msgr2=0x7f15fc0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 --2- 172.21.15.170:0/3016360631 >> v2:172.21.15.191:3300/0 conn(0x7f15fc069c70 0x7f15fc0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 -- 172.21.15.170:0/3016360631 >> v1:172.21.15.170:6789/0 conn(0x7f15fc159950 legacy=0x7f15fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 -- 172.21.15.170:0/3016360631 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc0ce5d0 con 0x7f15fc0af320 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16008b7700 1 -- 172.21.15.170:0/3016360631 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090287728 0 0) 0x7f15fc0ce380 con 0x7f15fc159950 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.139+0000 7f16020ba700 1 --2- 172.21.15.170:0/3016360631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc0af320 0x7f15fc0cf0b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f15fc15d450 tx=0x7f15f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801c6f5595333e2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f16008b7700 1 -- 172.21.15.170:0/3016360631 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f400e070 con 0x7f15fc0af320 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f16008b7700 1 -- 172.21.15.170:0/3016360631 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f400bc30 con 0x7f15fc0af320 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f16008b7700 1 -- 172.21.15.170:0/3016360631 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4012430 con 0x7f15fc0af320 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 -- 172.21.15.170:0/3016360631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc0af320 msgr2=0x7f15fc0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 --2- 172.21.15.170:0/3016360631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc0af320 0x7f15fc0cf0b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f15fc15d450 tx=0x7f15f4009780 comp rx=0 tx=0).stop 2023-10-08T16:06:22.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 -- 172.21.15.170:0/3016360631 shutdown_connections 2023-10-08T16:06:22.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 --2- 172.21.15.170:0/3016360631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc0af320 0x7f15fc0cf0b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 --2- 172.21.15.170:0/3016360631 >> v2:172.21.15.191:3300/0 conn(0x7f15fc069c70 0x7f15fc0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 -- 172.21.15.170:0/3016360631 >> 172.21.15.170:0/3016360631 conn(0x7f15fc068e00 msgr2=0x7f15fc0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:22.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 -- 172.21.15.170:0/3016360631 shutdown_connections 2023-10-08T16:06:22.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.140+0000 7f1603b1d700 1 -- 172.21.15.170:0/3016360631 wait complete. 2023-10-08T16:06:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.141+0000 7f1603b1d700 1 Processor -- start 2023-10-08T16:06:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.141+0000 7f1603b1d700 1 -- start start 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15fc069c70 0x7f15fc0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15fc0af320 0x7f15fc0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0b41c0 con 0x7f15fc159950 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0c8240 con 0x7f15fc0af320 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f1603b1d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc0c8520 con 0x7f15fc069c70 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44286/0 (socket says 172.21.15.170:44286) 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 -- 172.21.15.170:0/1953215400 learned_addr learned my addr 172.21.15.170:0/1953215400 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16018b9700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15fc069c70 0x7f15fc0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16010b8700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15fc0af320 0x7f15fc0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 -- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15fc069c70 msgr2=0x7f15fc0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15fc069c70 0x7f15fc0b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 -- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15fc0af320 msgr2=0x7f15fc0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15fc0af320 0x7f15fc0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.142+0000 7f16020ba700 1 -- 172.21.15.170:0/1953215400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009a20 con 0x7f15fc159950 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f16020ba700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f15fc070e20 tx=0x7f15f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4012ea0 con 0x7f15fc159950 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f401d630 con 0x7f15fc159950 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc15ad90 con 0x7f15fc159950 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f401d9f0 con 0x7f15fc159950 2023-10-08T16:06:22.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.143+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc0c8800 con 0x7f15fc159950 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.144+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc15b2b0 con 0x7f15fc159950 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.144+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f15f4024070 con 0x7f15fc159950 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.145+0000 7f15f1ffb700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 0x7f15e0069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.145+0000 7f16018b9700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 0x7f15e0069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.145+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15f4086db0 con 0x7f15fc159950 2023-10-08T16:06:22.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.145+0000 7f16018b9700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 0x7f15e0069470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15fc16ce00 tx=0x7f15f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.149+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15fc15b2b0 con 0x7f15fc159950 2023-10-08T16:06:22.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.407+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f15fc15b2b0 con 0x7f15fc159950 2023-10-08T16:06:22.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.410+0000 7f15f1ffb700 1 -- 172.21.15.170:0/1953215400 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f15fc15b2b0 con 0x7f15fc159950 2023-10-08T16:06:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 msgr2=0x7f15e0069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 0x7f15e0069470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15fc16ce00 tx=0x7f15f8009040 comp rx=0 tx=0).stop 2023-10-08T16:06:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 msgr2=0x7f15fc0d0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f15fc070e20 tx=0x7f15f4019000 comp rx=0 tx=0).stop 2023-10-08T16:06:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 shutdown_connections 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15e0066fc0 0x7f15e0069470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15fc159950 0x7f15fc0d0780 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15fc069c70 0x7f15fc0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 --2- 172.21.15.170:0/1953215400 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15fc0af320 0x7f15fc0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 >> 172.21.15.170:0/1953215400 conn(0x7f15fc068e00 msgr2=0x7f15fc0c5c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 shutdown_connections 2023-10-08T16:06:22.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.412+0000 7f1603b1d700 1 -- 172.21.15.170:0/1953215400 wait complete. 2023-10-08T16:06:22.415 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:06:22.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:06:22.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.538+0000 7f8332fa1700 1 Processor -- start 2023-10-08T16:06:22.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.538+0000 7f8332fa1700 1 -- start start 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8332fa1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f832c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8332fa1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f832c15db10 0x7f832c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8332fa1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832c15a160 con 0x7f832c159890 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8332fa1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0d13a0 con 0x7f832c15db10 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8332fa1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15e140 con 0x7f832c159c60 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8330d3d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f832c159890 0x7f832c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52012/0 (socket says 172.21.15.170:52012) 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f8330d3d700 1 -- 172.21.15.170:0/3208633989 learned_addr learned my addr 172.21.15.170:0/3208633989 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f832bfff700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f832c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f833153e700 1 --2- 172.21.15.170:0/3208633989 >> v2:172.21.15.191:3300/0 conn(0x7f832c15db10 0x7f832c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.539+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704304699 0 0) 0x7f832c15a160 con 0x7f832c159890 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8318003570 con 0x7f832c159890 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278004729 0 0) 0x7f8318003570 con 0x7f832c159890 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 msgr2=0x7f832c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f832c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 >> v2:172.21.15.191:3300/0 conn(0x7f832c15db10 msgr2=0x7f832c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 --2- 172.21.15.170:0/3208633989 >> v2:172.21.15.191:3300/0 conn(0x7f832c15db10 0x7f832c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c15a3d0 con 0x7f832c159890 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8314001470 con 0x7f832c159890 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 >> v1:172.21.15.170:6789/0 conn(0x7f832c159890 legacy=0x7f832c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318003a60 0x7f8318003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8318004810 0x7f8318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f83180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0d13a0 con 0x7f8318004810 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15e140 con 0x7f8318003a60 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831800ba60 con 0x7f832c159c60 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8318004810 0x7f8318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f8330d3d700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318003a60 0x7f8318003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f833153e700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f83180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 -- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 msgr2=0x7f83180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c159c60 0x7f83180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 -- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318003a60 msgr2=0x7f8318003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318003a60 0x7f8318003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 -- 172.21.15.170:0/3208633989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8318003570 con 0x7f8318004810 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832bfff700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8318004810 0x7f8318004c80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e210 tx=0x7f831c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62871f039fef6ed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00eac0 con 0x7f8318004810 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.540+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00f040 con 0x7f8318004810 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f832b7fe700 1 -- 172.21.15.170:0/3208633989 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00ee70 con 0x7f8318004810 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 -- 172.21.15.170:0/3208633989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8318004810 msgr2=0x7f8318004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3208633989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8318004810 0x7f8318004c80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f831c00e210 tx=0x7f831c00b040 comp rx=0 tx=0).stop 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f833153e700 1 -- 172.21.15.170:0/3208633989 reap_dead start 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 -- 172.21.15.170:0/3208633989 shutdown_connections 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 -- 172.21.15.170:0/3208633989 >> 172.21.15.170:0/3208633989 conn(0x7f832c068e20 msgr2=0x7f832c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 -- 172.21.15.170:0/3208633989 shutdown_connections 2023-10-08T16:06:22.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.541+0000 7f8332fa1700 1 -- 172.21.15.170:0/3208633989 wait complete. 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.542+0000 7f8332fa1700 1 Processor -- start 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.542+0000 7f8332fa1700 1 -- start start 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318004810 0x7f832c0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c0c9940 0x7f832c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c156b10 con 0x7f832c0cdb10 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0b38b0 con 0x7f8318004810 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8332fa1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c0ca2f0 con 0x7f832c0c9940 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44314/0 (socket says 172.21.15.170:44314) 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 -- 172.21.15.170:0/3410240229 learned_addr learned my addr 172.21.15.170:0/3410240229 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f8330d3d700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318004810 0x7f832c0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f833153e700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c0c9940 0x7f832c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 -- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c0c9940 msgr2=0x7f832c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c0c9940 0x7f832c0c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 -- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318004810 msgr2=0x7f832c0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318004810 0x7f832c0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 -- 172.21.15.170:0/3410240229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0ce2f0 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.543+0000 7f832bfff700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f831c009c30 tx=0x7f831c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.544+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c01d420 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.544+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c01d5a0 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.544+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0ce5d0 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.544+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c01d960 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.544+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c0ceb90 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.545+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0b3be0 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.545+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f831c003040 con 0x7f832c0cdb10 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.545+0000 7f8328ff9700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 0x7f830c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.545+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f831c0865d0 con 0x7f832c0cdb10 2023-10-08T16:06:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.545+0000 7f8330d3d700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 0x7f830c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.546+0000 7f8330d3d700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 0x7f830c069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8314001d20 tx=0x7f8314009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.550+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f831c053740 con 0x7f832c0cdb10 2023-10-08T16:06:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.776+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f832c04c3c0 con 0x7f832c0cdb10 2023-10-08T16:06:22.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.777+0000 7f8328ff9700 1 -- 172.21.15.170:0/3410240229 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f831c018090 con 0x7f832c0cdb10 2023-10-08T16:06:22.777 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:06:22.777 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:06:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 msgr2=0x7f830c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 0x7f830c069470 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8314001d20 tx=0x7f8314009040 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 msgr2=0x7f832c0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f831c009c30 tx=0x7f831c01a000 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 shutdown_connections 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f832c0c9940 0x7f832c0c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8318004810 0x7f832c0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f830c066fc0 0x7f830c069470 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 --2- 172.21.15.170:0/3410240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f832c0cdb10 0x7f832c0c9400 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 >> 172.21.15.170:0/3410240229 conn(0x7f832c068e20 msgr2=0x7f832c1584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 shutdown_connections 2023-10-08T16:06:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:22.779+0000 7f8332fa1700 1 -- 172.21.15.170:0/3410240229 wait complete. 2023-10-08T16:06:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 38 2023-10-08T16:06:22.857 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T16:06:22.857+0000 7ff898171700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:06:23.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:06:16.968284+0000', 'modified': '2023-10-08T16:06:20.570046+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5667}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5667': {'gid': 5667, 'name': 'b', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6834/1489757560', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 1489757560}, {'type': 'v1', 'addr': '172.21.15.191:6834', 'nonce': 1489757560}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:06:23.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:06:23.194 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T16:06:23.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.311+0000 7f9894fe3700 1 Processor -- start 2023-10-08T16:06:23.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.311+0000 7f9894fe3700 1 -- start start 2023-10-08T16:06:23.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f9894fe3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9890069d60 0x7f98900c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f9894fe3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f989006bdb0 con 0x7f98900aca80 2023-10-08T16:06:23.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f9894fe3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900cb370 con 0x7f98900c81f0 2023-10-08T16:06:23.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f9894fe3700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9890159f60 con 0x7f9890069d60 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f988dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9890069d60 0x7f98900c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f988e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f98900aca80 0x7f989006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52022/0 (socket says 172.21.15.170:52022) 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.312+0000 7f988e59c700 1 -- 172.21.15.170:0/3079456644 learned_addr learned my addr 172.21.15.170:0/3079456644 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988dd9b700 1 -- 172.21.15.170:0/3079456644 >> v1:172.21.15.191:6789/0 conn(0x7f98900c81f0 legacy=0x7f98900cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988dd9b700 1 -- 172.21.15.170:0/3079456644 >> v1:172.21.15.170:6789/0 conn(0x7f98900aca80 legacy=0x7f989006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988dd9b700 1 -- 172.21.15.170:0/3079456644 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989006b2b0 con 0x7f9890069d60 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988d59a700 1 -- 172.21.15.170:0/3079456644 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172840541 0 0) 0x7f989006bdb0 con 0x7f98900aca80 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988dd9b700 1 --2- 172.21.15.170:0/3079456644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900c7cb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f98840098e0 tx=0x7f98840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc0baad593f7e12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988d59a700 1 -- 172.21.15.170:0/3079456644 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884009e80 con 0x7f9890069d60 2023-10-08T16:06:23.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988d59a700 1 -- 172.21.15.170:0/3079456644 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400b820 con 0x7f9890069d60 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f988d59a700 1 -- 172.21.15.170:0/3079456644 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400bba0 con 0x7f9890069d60 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f9894fe3700 1 -- 172.21.15.170:0/3079456644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 msgr2=0x7f98900c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f9894fe3700 1 --2- 172.21.15.170:0/3079456644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900c7cb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f98840098e0 tx=0x7f98840091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f9894fe3700 1 -- 172.21.15.170:0/3079456644 shutdown_connections 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f9894fe3700 1 --2- 172.21.15.170:0/3079456644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900c7cb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.313+0000 7f9894fe3700 1 -- 172.21.15.170:0/3079456644 >> 172.21.15.170:0/3079456644 conn(0x7f98900cd670 msgr2=0x7f98900cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.314+0000 7f9894fe3700 1 -- 172.21.15.170:0/3079456644 shutdown_connections 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.314+0000 7f9894fe3700 1 -- 172.21.15.170:0/3079456644 wait complete. 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 Processor -- start 2023-10-08T16:06:23.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 -- start start 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98900aca80 0x7f98900b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98900c81f0 0x7f989015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b0950 con 0x7f9890069d60 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b0b80 con 0x7f98900aca80 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f9894fe3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b0e60 con 0x7f98900c81f0 2023-10-08T16:06:23.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f988e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f988e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44338/0 (socket says 172.21.15.170:44338) 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.315+0000 7f988e59c700 1 -- 172.21.15.170:0/1401951308 learned_addr learned my addr 172.21.15.170:0/1401951308 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988dd9b700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98900aca80 0x7f98900b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988ed9d700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98900c81f0 0x7f989015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 -- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98900c81f0 msgr2=0x7f989015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98900c81f0 0x7f989015a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 -- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98900aca80 msgr2=0x7f98900b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98900aca80 0x7f98900b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 -- 172.21.15.170:0/1401951308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9884009420 con 0x7f9890069d60 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f988e59c700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f988000cb60 tx=0x7f98800017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f987e7fc700 1 -- 172.21.15.170:0/1401951308 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880005d80 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f987e7fc700 1 -- 172.21.15.170:0/1401951308 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988000a5d0 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98900b1140 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f987e7fc700 1 -- 172.21.15.170:0/1401951308 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000a990 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.316+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900acef0 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.317+0000 7f987e7fc700 1 -- 172.21.15.170:0/1401951308 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f988000ad40 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.318+0000 7f987e7fc700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 0x7f9870069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.318+0000 7f987e7fc700 1 -- 172.21.15.170:0/1401951308 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9880084310 con 0x7f9890069d60 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.318+0000 7f988dd9b700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 0x7f9870069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.318+0000 7f988dd9b700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 0x7f9870069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f989006a130 tx=0x7f9884016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.320+0000 7f989e2dbb80 1 Processor -- start 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.320+0000 7f989e2dbb80 1 -- start start 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923bdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923bd8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923b33a70 con 0x558923c26d00 2023-10-08T16:06:23.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923b31270 con 0x558923c05f60 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x558923b2d060 con 0x558923bdca50 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44352/0 (socket says 172.21.15.170:44352) 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 -- 172.21.15.170:0/342970328 learned_addr learned my addr 172.21.15.170:0/342970328 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988e59c700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923bdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988ed9d700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923bd8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 -- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 msgr2=0x558923bdce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923bdce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 -- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 msgr2=0x558923bd8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923bd8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988dd9b700 1 -- 172.21.15.170:0/342970328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589239c7270 con 0x558923c26d00 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.321+0000 7f988ed9d700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923bd8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f988dd9b700 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f988401b4c0 tx=0x7f9884021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20abb11ecd447320 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f986f7fe700 1 -- 172.21.15.170:0/342970328 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884021070 con 0x558923c26d00 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f986f7fe700 1 -- 172.21.15.170:0/342970328 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988401bae0 con 0x558923c26d00 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f986f7fe700 1 -- 172.21.15.170:0/342970328 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988401bc50 con 0x558923c26d00 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 -- 172.21.15.170:0/342970328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 msgr2=0x558923c05a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f988401b4c0 tx=0x7f9884021000 comp rx=0 tx=0).stop 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 -- 172.21.15.170:0/342970328 shutdown_connections 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923bd8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923bdce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/342970328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c05a20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.322+0000 7f989e2dbb80 1 -- 172.21.15.170:0/342970328 >> 172.21.15.170:0/342970328 conn(0x558923bde370 msgr2=0x558923bde740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.323+0000 7f989e2dbb80 1 -- 172.21.15.170:0/342970328 shutdown_connections 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.323+0000 7f989e2dbb80 1 -- 172.21.15.170:0/342970328 wait complete. 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.323+0000 7f989e2dbb80 1 -- start start 2023-10-08T16:06:23.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.323+0000 7f989e2dbb80 1 Processor -- start 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 0x558923bd7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923bd8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923ad8890 con 0x558923bdca50 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923af3ea0 con 0x558923c26d00 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589239afe30 con 0x558923c05f60 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f989e2dbb80 10 client.0 _ll_register_callbacks cb 0x558923ba0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44354/0 (socket says 172.21.15.170:44354) 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 -- 172.21.15.170:0/3464539065 learned_addr learned my addr 172.21.15.170:0/3464539065 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988dd9b700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 0x558923bd7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988ed9d700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923bd8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 -- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 msgr2=0x558923bd7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 0x558923bd7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 -- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 msgr2=0x558923bd8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923bd8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988e59c700 1 -- 172.21.15.170:0/3464539065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f989006b2b0 con 0x558923bdca50 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988dd9b700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 0x558923bd7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.324+0000 7f988ed9d700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923bd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f988e59c700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98800014b0 tx=0x7f9880084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3700b4bec2b2b7ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:06:23.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880058520 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f988005c270 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558923bd1b70 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9880055740 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 10 client.5802 fetch_fsmap learned FSMap version 38 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x558923969fe0 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558923bd1b70 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 38) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f98800547c0 con 0x558923bdca50 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 10 client.5802 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 10 client.5802 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 20 client.5802 populate_metadata read hostname 'smithi170' 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c01d20 0x558923c1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 4 client.5802 mds_command: new command op to 5667 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 --> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558923baeaa0 con 0x558923c01d20 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.325+0000 7f988dd9b700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c01d20 0x558923c1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.326+0000 7f986cff9700 10 client.5802 ms_handle_connect on v2:172.21.15.191:6832/1489757560 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.326+0000 7f988dd9b700 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c01d20 0x558923c1e1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693cdd2510c38fe3 server_cookie=5a61c974aa5c7e21 in_seq=0 out_seq=0 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.326+0000 7f986cff9700 1 -- 172.21.15.170:0/3464539065 <== mds.0 v2:172.21.15.191:6832/1489757560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558923baeaa0 con 0x558923c01d20 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.326+0000 7f986cff9700 10 client.5802 handle_command_reply: tid=0 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.326+0000 7f989e2dbb80 1 client.5802 shutdown 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 msgr2=0x558923c23a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f98800014b0 tx=0x7f9880084040 comp rx=0 tx=0).stop 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 shutdown_connections 2023-10-08T16:06:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923bd8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c01d20 0x558923c1e1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923bdca50 0x558923c23a00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f986cff9700 0 client.5802 ms_handle_reset on v2:172.21.15.191:6832/1489757560 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/3464539065 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923c05f60 0x558923bd7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 >> 172.21.15.170:0/3464539065 conn(0x558923bde370 msgr2=0x558923c22550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 shutdown_connections 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 1 -- 172.21.15.170:0/3464539065 wait complete. 2023-10-08T16:06:23.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.327+0000 7f989e2dbb80 20 client.5802 trim_cache size 0 max 16384 2023-10-08T16:06:23.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.343+0000 7f989e2dbb80 1 Processor -- start 2023-10-08T16:06:23.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.343+0000 7f989e2dbb80 1 -- start start 2023-10-08T16:06:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923c008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923be0630 con 0x558923c26d00 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923be02f0 con 0x558923c05f60 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x558923bdae10 con 0x558923bdca50 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44356/0 (socket says 172.21.15.170:44356) 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 -- 172.21.15.170:0/4054727562 learned_addr learned my addr 172.21.15.170:0/4054727562 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 -- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 msgr2=0x558923c27110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 -- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 msgr2=0x558923c008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923c008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.344+0000 7f988ed9d700 1 -- 172.21.15.170:0/4054727562 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558923afb780 con 0x558923c26d00 2023-10-08T16:06:23.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f988ed9d700 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f988800f8b0 tx=0x7f9888009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8a28bf12cb3bca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f986ffff700 1 -- 172.21.15.170:0/4054727562 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888016680 con 0x558923c26d00 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f986ffff700 1 -- 172.21.15.170:0/4054727562 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988800d050 con 0x558923c26d00 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f986ffff700 1 -- 172.21.15.170:0/4054727562 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9888016a50 con 0x558923c26d00 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4054727562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 msgr2=0x558923c25850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f988800f8b0 tx=0x7f9888009040 comp rx=0 tx=0).stop 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4054727562 shutdown_connections 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c26d00 0x558923c25850 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4054727562 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c05f60 0x558923c008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4054727562 >> 172.21.15.170:0/4054727562 conn(0x558923bde370 msgr2=0x558923bd54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4054727562 shutdown_connections 2023-10-08T16:06:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.345+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4054727562 wait complete. 2023-10-08T16:06:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.346+0000 7f989e2dbb80 1 -- start start 2023-10-08T16:06:23.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.346+0000 7f989e2dbb80 1 Processor -- start 2023-10-08T16:06:23.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c6b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923c22a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923c7f590 con 0x558923c05f60 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923bdfdb0 con 0x558923c26d00 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x558923bdfff0 con 0x558923bdca50 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 10 client.0 _ll_register_callbacks cb 0x558923ba0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44364/0 (socket says 172.21.15.170:44364) 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 -- 172.21.15.170:0/4158278749 learned_addr learned my addr 172.21.15.170:0/4158278749 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988e59c700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c6b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988ed9d700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923c22a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 -- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 msgr2=0x558923c6b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c6b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 -- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 msgr2=0x558923c22a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923c22a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 -- 172.21.15.170:0/4158278749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988800f5e0 con 0x558923c05f60 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f988dd9b700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9884000e00 tx=0x7f9884013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef0845fa544dc25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f986cff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884039040 con 0x558923c05f60 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9884033640 con 0x558923c05f60 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.347+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f988401bae0 con 0x558923c05f60 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9884021040 con 0x558923c05f60 2023-10-08T16:06:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 10 client.5808 fetch_fsmap learned FSMap version 38 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x558923ae2470 con 0x558923c05f60 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988401bae0 con 0x558923c05f60 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 38) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f9884033a10 con 0x558923c05f60 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 10 client.5808 fetch_fsmap finished waiting for FSMap version 38 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 10 client.5808 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 20 client.5808 populate_metadata read hostname 'smithi170' 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c19560 0x558923c19930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 4 client.5808 mds_command: new command op to 5667 tid=0[{"prefix": "status"}] 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 --> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558923acfe00 con 0x558923c19560 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f988e59c700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c19560 0x558923c19930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f986cff9700 10 client.5808 ms_handle_connect on v2:172.21.15.191:6832/1489757560 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f988e59c700 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c19560 0x558923c19930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b03cf90c2539d77 server_cookie=b89bc2ce11b88419 in_seq=0 out_seq=0 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.348+0000 7f986cff9700 1 -- 172.21.15.170:0/4158278749 <== mds.0 v2:172.21.15.191:6832/1489757560 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x558923acfe00 con 0x558923c19560 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f986cff9700 10 client.5808 handle_command_reply: tid=0 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 client.5808 shutdown 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 msgr2=0x558923c22550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9884000e00 tx=0x7f9884013000 comp rx=0 tx=0).stop 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 shutdown_connections 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x558923c26d00 0x558923c22a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x558923bdca50 0x558923c6b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.191:6832/1489757560,v1:172.21.15.191:6834/1489757560] conn(0x558923c19560 0x558923c19930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 --2- 172.21.15.170:0/4158278749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558923c05f60 0x558923c22550 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f986cff9700 0 client.5808 ms_handle_reset on v2:172.21.15.191:6832/1489757560 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 >> 172.21.15.170:0/4158278749 conn(0x558923bde370 msgr2=0x558923bd7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 shutdown_connections 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 1 -- 172.21.15.170:0/4158278749 wait complete. 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f989e2dbb80 20 client.5808 trim_cache size 0 max 16384 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:06:23.357 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 5667, 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.7961362630000002, 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 38, 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 63, 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 63, 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.888905628 2023-10-08T16:06:23.358 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 msgr2=0x7f9870069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 0x7f9870069470 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f989006a130 tx=0x7f9884016000 comp rx=0 tx=0).stop 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 msgr2=0x7f98900aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f988000cb60 tx=0x7f98800017b0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 shutdown_connections 2023-10-08T16:06:23.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98900c81f0 0x7f989015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9870066fc0 0x7f9870069470 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9890069d60 0x7f98900aed60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 --2- 172.21.15.170:0/1401951308 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98900aca80 0x7f98900b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 >> 172.21.15.170:0/1401951308 conn(0x7f98900cd670 msgr2=0x7f98900c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 shutdown_connections 2023-10-08T16:06:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.349+0000 7f9894fe3700 1 -- 172.21.15.170:0/1401951308 wait complete. 2023-10-08T16:06:23.361 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:06:23.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 Processor -- start 2023-10-08T16:06:23.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 -- start start 2023-10-08T16:06:23.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5b24159c60 0x7f5b2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2415a160 con 0x7f5b2415db10 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240d13a0 con 0x7f5b24159890 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.476+0000 7f5b2add5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2415df50 con 0x7f5b24159c60 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b29372700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5b2415db10 0x7f5b24069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52028/0 (socket says 172.21.15.170:52028) 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b29372700 1 -- 172.21.15.170:0/2197734130 learned_addr learned my addr 172.21.15.170:0/2197734130 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b23fff700 1 --2- 172.21.15.170:0/2197734130 >> v2:172.21.15.191:3300/0 conn(0x7f5b24159c60 0x7f5b2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906504319 0 0) 0x7f5b2415a160 con 0x7f5b2415db10 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b14003570 con 0x7f5b2415db10 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151513339 0 0) 0x7f5b14003570 con 0x7f5b2415db10 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b23fff700 1 -- 172.21.15.170:0/2197734130 >> v1:172.21.15.191:6789/0 conn(0x7f5b24159890 legacy=0x7f5b2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b23fff700 1 -- 172.21.15.170:0/2197734130 >> v1:172.21.15.170:6789/0 conn(0x7f5b2415db10 legacy=0x7f5b24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b23fff700 1 -- 172.21.15.170:0/2197734130 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2415e140 con 0x7f5b24159c60 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b23fff700 1 --2- 172.21.15.170:0/2197734130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159c60 0x7f5b2415d5d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b100098e0 tx=0x7f5b100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73e04596c27221c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c0017f0 con 0x7f5b24159c60 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b10011420 con 0x7f5b24159c60 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.477+0000 7f5b237fe700 1 -- 172.21.15.170:0/2197734130 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b100117c0 con 0x7f5b24159c60 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 -- 172.21.15.170:0/2197734130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159c60 msgr2=0x7f5b2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 --2- 172.21.15.170:0/2197734130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159c60 0x7f5b2415d5d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5b100098e0 tx=0x7f5b100091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 -- 172.21.15.170:0/2197734130 shutdown_connections 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 --2- 172.21.15.170:0/2197734130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159c60 0x7f5b2415d5d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 -- 172.21.15.170:0/2197734130 >> 172.21.15.170:0/2197734130 conn(0x7f5b24068e20 msgr2=0x7f5b240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 -- 172.21.15.170:0/2197734130 shutdown_connections 2023-10-08T16:06:23.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.478+0000 7f5b2add5700 1 -- 172.21.15.170:0/2197734130 wait complete. 2023-10-08T16:06:23.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.479+0000 7f5b2add5700 1 Processor -- start 2023-10-08T16:06:23.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.479+0000 7f5b2add5700 1 -- start start 2023-10-08T16:06:23.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159890 0x7f5b2416a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24159c60 0x7f5b2415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2416a470 con 0x7f5b2415db10 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b6060 con 0x7f5b24159890 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b2add5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b6340 con 0x7f5b24159c60 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44372/0 (socket says 172.21.15.170:44372) 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 -- 172.21.15.170:0/232859781 learned_addr learned my addr 172.21.15.170:0/232859781 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b23fff700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24159c60 0x7f5b2415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 -- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24159c60 msgr2=0x7f5b2415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24159c60 0x7f5b2415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b28b71700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159890 0x7f5b2416a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 -- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159890 msgr2=0x7f5b2416a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159890 0x7f5b2416a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 -- 172.21.15.170:0/232859781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b10009420 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.480+0000 7f5b29372700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00cbe0 tx=0x7f5b1c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.481+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c004dc0 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.481+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00a6b0 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.481+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2416a100 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.481+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00aa70 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.481+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2415d050 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.482+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2404c3c0 con 0x7f5b2415db10 2023-10-08T16:06:23.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.482+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f5b1c007150 con 0x7f5b2415db10 2023-10-08T16:06:23.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.482+0000 7f5b20ff9700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 0x7f5b08069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:23.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.482+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b1c083f50 con 0x7f5b2415db10 2023-10-08T16:06:23.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.482+0000 7f5b28b71700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 0x7f5b08069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:23.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.483+0000 7f5b28b71700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 0x7f5b08069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b240b1b50 tx=0x7f5b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:23.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.487+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b1c051250 con 0x7f5b2415db10 2023-10-08T16:06:23.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.716+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b2406a5b0 con 0x7f5b2415db10 2023-10-08T16:06:23.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.716+0000 7f5b20ff9700 1 -- 172.21.15.170:0/232859781 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5b1c018070 con 0x7f5b2415db10 2023-10-08T16:06:23.716 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 msgr2=0x7f5b08069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 0x7f5b08069470 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b240b1b50 tx=0x7f5b18009040 comp rx=0 tx=0).stop 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 msgr2=0x7f5b240b5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00cbe0 tx=0x7f5b1c001640 comp rx=0 tx=0).stop 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 shutdown_connections 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5b08066fc0 0x7f5b08069470 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b2415db10 0x7f5b240b5660 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5b24159890 0x7f5b2416a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 --2- 172.21.15.170:0/232859781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5b24159c60 0x7f5b2415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 >> 172.21.15.170:0/232859781 conn(0x7f5b24068e20 msgr2=0x7f5b24156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 shutdown_connections 2023-10-08T16:06:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:23.718+0000 7f5b2add5700 1 -- 172.21.15.170:0/232859781 wait complete. 2023-10-08T16:06:23.730 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:06:23.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:06:23.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:06:23.730 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:06:23.731 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:06:23.731 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:06:23.731 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:06:23.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:06:23.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:06:23.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.731 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:06:23.746 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:06:23.746 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:06:23.746 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:06:23.746 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.746 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:06:23.747 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:06:23.748 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:23.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.749 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:06:23.808 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:06:23.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.810 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:06:23.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.866 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:06:23.867 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:06:23.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:06:23.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:06:23.867 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:06:23.867 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:06:23.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.965 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:06:23.979 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:06:23.979 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:06:23.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:06:23.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:23.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:06:24.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:06:24.044 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:06:24.099 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:06:24.099 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:06:24.099 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:06:24.153 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:06:24.153 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:06:24.153 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:24.232 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:24.232 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:24.251 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:24.251 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:24.251 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:06:24.252 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:06:24.252 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:06:24.252 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:06:24.252 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:06:24.252 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:06:24.275 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:06:24.276 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:06:24.278 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:06:24.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:24.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:06:24.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.498+0000 7f64dd3de700 1 Processor -- start 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.498+0000 7f64dd3de700 1 -- start start 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64dd3de700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 0x7f64d8068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64dd3de700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f64d815e010 0x7f64d806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64dd3de700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d806ec70 con 0x7f64d806f640 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64dd3de700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80b3430 con 0x7f64d815e010 2023-10-08T16:06:24.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64dd3de700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80b2e70 con 0x7f64d8068550 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64ce7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f64d815e010 0x7f64d806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64d77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f64d806f640 0x7f64d806a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52032/0 (socket says 172.21.15.170:52032) 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64d77fe700 1 -- 172.21.15.170:0/2524383006 learned_addr learned my addr 172.21.15.170:0/2524383006 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.499+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/2524383006 >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 0x7f64d8068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64ce7fc700 1 -- 172.21.15.170:0/2524383006 >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 msgr2=0x7f64d8068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64ce7fc700 1 --2- 172.21.15.170:0/2524383006 >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 0x7f64d8068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64ce7fc700 1 -- 172.21.15.170:0/2524383006 >> v1:172.21.15.170:6789/0 conn(0x7f64d806f640 legacy=0x7f64d806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64ce7fc700 1 -- 172.21.15.170:0/2524383006 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80b3150 con 0x7f64d815e010 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/2524383006 >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 0x7f64d8068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64d67fc700 1 -- 172.21.15.170:0/2524383006 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232224316 0 0) 0x7f64d806ec70 con 0x7f64d806f640 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64ce7fc700 1 --2- 172.21.15.170:0/2524383006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d815e010 0x7f64d806f100 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f64d806e170 tx=0x7f64c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09e36d5c3cec257 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:24.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64d67fc700 1 -- 172.21.15.170:0/2524383006 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000b6b0 con 0x7f64d815e010 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64d67fc700 1 -- 172.21.15.170:0/2524383006 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c000b810 con 0x7f64d815e010 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64d67fc700 1 -- 172.21.15.170:0/2524383006 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000bbb0 con 0x7f64d815e010 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64dd3de700 1 -- 172.21.15.170:0/2524383006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d815e010 msgr2=0x7f64d806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64dd3de700 1 --2- 172.21.15.170:0/2524383006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d815e010 0x7f64d806f100 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f64d806e170 tx=0x7f64c00091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64dd3de700 1 -- 172.21.15.170:0/2524383006 shutdown_connections 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64dd3de700 1 --2- 172.21.15.170:0/2524383006 >> v2:172.21.15.191:3300/0 conn(0x7f64d8068550 0x7f64d8068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.500+0000 7f64dd3de700 1 --2- 172.21.15.170:0/2524383006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d815e010 0x7f64d806f100 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.501+0000 7f64dd3de700 1 -- 172.21.15.170:0/2524383006 >> 172.21.15.170:0/2524383006 conn(0x7f64d80cd480 msgr2=0x7f64d80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.501+0000 7f64dd3de700 1 -- 172.21.15.170:0/2524383006 shutdown_connections 2023-10-08T16:06:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.501+0000 7f64dd3de700 1 -- 172.21.15.170:0/2524383006 wait complete. 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 Processor -- start 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 -- start start 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 0x7f64d80a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64d815e010 0x7f64d80ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80a9e10 con 0x7f64d806f640 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80aa040 con 0x7f64d815e010 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.502+0000 7f64dd3de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80aa320 con 0x7f64d8068550 2023-10-08T16:06:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44400/0 (socket says 172.21.15.170:44400) 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 -- 172.21.15.170:0/650814778 learned_addr learned my addr 172.21.15.170:0/650814778 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64d77fe700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64d815e010 0x7f64d80ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 0x7f64d80a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 -- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 msgr2=0x7f64d80a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 0x7f64d80a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 -- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64d815e010 msgr2=0x7f64d80ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64d815e010 0x7f64d80ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 -- 172.21.15.170:0/650814778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c0009420 con 0x7f64d806f640 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 0x7f64d80a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64ce7fc700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f64d80ce220 tx=0x7f64c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c000bb90 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c0011420 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80aa600 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c00117c0 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.503+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d80aabc0 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.504+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d804c3c0 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.505+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f64c0011b70 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.505+0000 7f64cf7fe700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 0x7f64b8069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.505+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f64c0085d50 con 0x7f64d806f640 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.505+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 0x7f64b8069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:24.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.505+0000 7f64d6ffd700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 0x7f64b8069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f64c8006f00 tx=0x7f64c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:24.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.509+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64c0052fd0 con 0x7f64d806f640 2023-10-08T16:06:24.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.743+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64d806e900 con 0x7f64d806f640 2023-10-08T16:06:24.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.744+0000 7f64cf7fe700 1 -- 172.21.15.170:0/650814778 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f64c0015060 con 0x7f64d806f640 2023-10-08T16:06:24.744 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 msgr2=0x7f64b8069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 0x7f64b8069470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f64c8006f00 tx=0x7f64c8008040 comp rx=0 tx=0).stop 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 msgr2=0x7f64d814a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f64d80ce220 tx=0x7f64c0017000 comp rx=0 tx=0).stop 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 shutdown_connections 2023-10-08T16:06:24.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f64d815e010 0x7f64d80ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f64d8068550 0x7f64d80a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f64b8066fc0 0x7f64b8069470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 --2- 172.21.15.170:0/650814778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64d806f640 0x7f64d814a0b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.745+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 >> 172.21.15.170:0/650814778 conn(0x7f64d80cd480 msgr2=0x7f64d8153580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.746+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 shutdown_connections 2023-10-08T16:06:24.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:24.746+0000 7f64dd3de700 1 -- 172.21.15.170:0/650814778 wait complete. 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:06:24.758 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:06:24.759 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:06:24.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:24.759 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:06:24.776 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:06:24.777 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:06:24.778 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:06:24.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:24.780 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:06:24.844 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:06:24.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:24.845 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:06:24.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:06:24.907 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:06:25.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:25.011 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:06:25.025 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:06:25.025 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:06:25.025 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:06:25.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:25.025 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:06:25.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:06:25.089 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:06:25.143 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:06:25.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:06:25.144 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:06:25.198 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:06:25.198 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:06:25.198 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:25.277 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:25.277 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:25.297 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:25.297 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:25.297 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:06:25.297 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:06:25.297 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:06:25.298 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:06:25.298 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:06:25.298 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:06:25.317 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:06:25.318 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:06:25.320 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:06:25.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:06:25.320 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:06:25.384 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:06:25.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.503+0000 7fcb5a734700 1 Processor -- start 2023-10-08T16:06:25.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.503+0000 7fcb5a734700 1 -- start start 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb5a734700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb54159ec0 0x7fcb540b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb5a734700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 0x7fcb540cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb5a734700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540b5b00 con 0x7fcb540cce40 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb5a734700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540b5850 con 0x7fcb54159ec0 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb5a734700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540b5da0 con 0x7fcb540b53d0 2023-10-08T16:06:25.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb54159ec0 0x7fcb540b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb54159ec0 0x7fcb540b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50922/0 (socket says 172.21.15.170:50922) 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 -- 172.21.15.170:0/1529723336 learned_addr learned my addr 172.21.15.170:0/1529723336 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb537fe700 1 --2- 172.21.15.170:0/1529723336 >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 0x7fcb540cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 -- 172.21.15.170:0/1529723336 >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 msgr2=0x7fcb540cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 --2- 172.21.15.170:0/1529723336 >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 0x7fcb540cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 -- 172.21.15.170:0/1529723336 >> v1:172.21.15.191:6789/0 conn(0x7fcb540cce40 legacy=0x7fcb540cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.504+0000 7fcb53fff700 1 -- 172.21.15.170:0/1529723336 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540b6080 con 0x7fcb54159ec0 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb537fe700 1 --2- 172.21.15.170:0/1529723336 >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 0x7fcb540cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb53fff700 1 --2- 172.21.15.170:0/1529723336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540b4e90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb540b64d0 tx=0x7fcb44009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d549a53e9291 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb52ffd700 1 -- 172.21.15.170:0/1529723336 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb44005c10 con 0x7fcb54159ec0 2023-10-08T16:06:25.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb52ffd700 1 -- 172.21.15.170:0/1529723336 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb44005d70 con 0x7fcb54159ec0 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb52ffd700 1 -- 172.21.15.170:0/1529723336 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400b880 con 0x7fcb54159ec0 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 -- 172.21.15.170:0/1529723336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 msgr2=0x7fcb540b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 --2- 172.21.15.170:0/1529723336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540b4e90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb540b64d0 tx=0x7fcb44009220 comp rx=0 tx=0).stop 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 -- 172.21.15.170:0/1529723336 shutdown_connections 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 --2- 172.21.15.170:0/1529723336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540b4e90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 --2- 172.21.15.170:0/1529723336 >> v2:172.21.15.191:3300/0 conn(0x7fcb540b53d0 0x7fcb540cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 -- 172.21.15.170:0/1529723336 >> 172.21.15.170:0/1529723336 conn(0x7fcb540999f0 msgr2=0x7fcb540b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 -- 172.21.15.170:0/1529723336 shutdown_connections 2023-10-08T16:06:25.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.505+0000 7fcb5a734700 1 -- 172.21.15.170:0/1529723336 wait complete. 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.506+0000 7fcb5a734700 1 Processor -- start 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 -- start start 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb540b53d0 0x7fcb5414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb540cce40 0x7fcb540c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54156120 con 0x7fcb54159ec0 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5414e6e0 con 0x7fcb540b53d0 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.507+0000 7fcb5a734700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5414e110 con 0x7fcb540cce40 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb537fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb540cce40 0x7fcb540c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb540b53d0 0x7fcb5414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50926/0 (socket says 172.21.15.170:50926) 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 -- 172.21.15.170:0/2597207092 learned_addr learned my addr 172.21.15.170:0/2597207092 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 -- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb540cce40 msgr2=0x7fcb540c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb540cce40 0x7fcb540c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 -- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb540b53d0 msgr2=0x7fcb5414dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb540b53d0 0x7fcb5414dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 -- 172.21.15.170:0/2597207092 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44009480 con 0x7fcb54159ec0 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb58cd1700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c8e0 tx=0x7fcb4c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c012630 con 0x7fcb54159ec0 2023-10-08T16:06:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0127b0 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5416a260 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c012b70 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.508+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5416a820 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.509+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54063c90 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.510+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcb4c003030 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.510+0000 7fcb3bfff700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 0x7fcb40069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.510+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fcb4c084460 con 0x7fcb54159ec0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.510+0000 7fcb53fff700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 0x7fcb40069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.510+0000 7fcb53fff700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 0x7fcb40069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb540b4290 tx=0x7fcb44016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:25.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.514+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb4c051670 con 0x7fcb54159ec0 2023-10-08T16:06:25.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:25.748+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcb540718b0 con 0x7fcb54159ec0 2023-10-08T16:06:26.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.598+0000 7fcb3bfff700 1 -- 172.21.15.170:0/2597207092 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v39) v1 ==== 136+0+0 (secure 0 0 0) 0x7fcb4c054c90 con 0x7fcb54159ec0 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 msgr2=0x7fcb40069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 0x7fcb40069470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb540b4290 tx=0x7fcb44016000 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 msgr2=0x7fcb540c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c8e0 tx=0x7fcb4c001710 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 shutdown_connections 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb40066fc0 0x7fcb40069470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb54159ec0 0x7fcb540c9000 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb540b53d0 0x7fcb5414dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 --2- 172.21.15.170:0/2597207092 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb540cce40 0x7fcb540c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 >> 172.21.15.170:0/2597207092 conn(0x7fcb540999f0 msgr2=0x7fcb540cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:26.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 shutdown_connections 2023-10-08T16:06:26.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.600+0000 7fcb5a734700 1 -- 172.21.15.170:0/2597207092 wait complete. 2023-10-08T16:06:26.612 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:06:26.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.727+0000 7f76267f6700 1 Processor -- start 2023-10-08T16:06:26.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.727+0000 7f76267f6700 1 -- start start 2023-10-08T16:06:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f762015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7620159c60 0x7f762015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 0x7f7620069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762015a160 con 0x7f7620159c60 2023-10-08T16:06:26.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76200d13a0 con 0x7f762015db10 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f76267f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762015e140 con 0x7f7620159890 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f761f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7620159c60 0x7f762015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f761f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7620159c60 0x7f762015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50930/0 (socket says 172.21.15.170:50930) 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f761f7fe700 1 -- 172.21.15.170:0/992136392 learned_addr learned my addr 172.21.15.170:0/992136392 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f761ffff700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f762015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.728+0000 7f7624d93700 1 --2- 172.21.15.170:0/992136392 >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 0x7f7620069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 -- 172.21.15.170:0/992136392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 msgr2=0x7f762015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f762015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 -- 172.21.15.170:0/992136392 >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 msgr2=0x7f7620069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 --2- 172.21.15.170:0/992136392 >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 0x7f7620069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 -- 172.21.15.170:0/992136392 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762015a3d0 con 0x7f7620159c60 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761ffff700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f762015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f7624d93700 1 --2- 172.21.15.170:0/992136392 >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 0x7f7620069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761f7fe700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7620159c60 0x7f762015d5d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7620089b50 tx=0x7f7614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a2bddff2384176 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761effd700 1 -- 172.21.15.170:0/992136392 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400b860 con 0x7f7620159c60 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761effd700 1 -- 172.21.15.170:0/992136392 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761400b9c0 con 0x7f7620159c60 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f761effd700 1 -- 172.21.15.170:0/992136392 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400bd60 con 0x7f7620159c60 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 -- 172.21.15.170:0/992136392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7620159c60 msgr2=0x7f762015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7620159c60 0x7f762015d5d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7620089b50 tx=0x7f7614009250 comp rx=0 tx=0).stop 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 -- 172.21.15.170:0/992136392 shutdown_connections 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 --2- 172.21.15.170:0/992136392 >> v2:172.21.15.191:3300/0 conn(0x7f762015db10 0x7f7620069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f762015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.729+0000 7f76267f6700 1 --2- 172.21.15.170:0/992136392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7620159c60 0x7f762015d5d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.730+0000 7f76267f6700 1 -- 172.21.15.170:0/992136392 >> 172.21.15.170:0/992136392 conn(0x7f7620068e20 msgr2=0x7f76200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:26.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.730+0000 7f76267f6700 1 -- 172.21.15.170:0/992136392 shutdown_connections 2023-10-08T16:06:26.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.730+0000 7f76267f6700 1 -- 172.21.15.170:0/992136392 wait complete. 2023-10-08T16:06:26.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.731+0000 7f76267f6700 1 Processor -- start 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.731+0000 7f76267f6700 1 -- start start 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.731+0000 7f76267f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f76200cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.731+0000 7f76267f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 0x7f76200cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cd6a0 con 0x7f762015db10 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cd980 con 0x7f7620159c60 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76200cdc60 con 0x7f7620159890 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50942/0 (socket says 172.21.15.170:50942) 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 -- 172.21.15.170:0/3446594986 learned_addr learned my addr 172.21.15.170:0/3446594986 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:26.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f761ffff700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f76200cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f761f7fe700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 0x7f76200cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 -- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 msgr2=0x7f76200cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f76200cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 -- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 msgr2=0x7f76200cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 0x7f76200cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 -- 172.21.15.170:0/3446594986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76140094f0 con 0x7f762015db10 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f761f7fe700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 0x7f76200cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7624d93700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f761800cef0 tx=0x7f7618007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800ae00 con 0x7f762015db10 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618016420 con 0x7f762015db10 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76200cdf40 con 0x7f762015db10 2023-10-08T16:06:26.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180167e0 con 0x7f762015db10 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.732+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76200cfd30 con 0x7f762015db10 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.733+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76200ce460 con 0x7f762015db10 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.734+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f7618016bd0 con 0x7f762015db10 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.734+0000 7f7607fff700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 0x7f760c069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.734+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f7618012070 con 0x7f762015db10 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.734+0000 7f761ffff700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 0x7f760c069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.734+0000 7f761ffff700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 0x7f760c069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f761000ab30 tx=0x7f7610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:26.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.738+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f761804e5a0 con 0x7f762015db10 2023-10-08T16:06:26.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.946+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f76200b4710 con 0x7f760c066fc0 2023-10-08T16:06:26.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.948+0000 7f7607fff700 1 -- 172.21.15.170:0/3446594986 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f76200b4710 con 0x7f760c066fc0 2023-10-08T16:06:26.948 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:06:26.949 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:06:26.949 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:06:26.949 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:06:26.949 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:06:26.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 msgr2=0x7f760c069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 0x7f760c069470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f761000ab30 tx=0x7f7610008040 comp rx=0 tx=0).stop 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 msgr2=0x7f7620153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f761800cef0 tx=0x7f7618007860 comp rx=0 tx=0).stop 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 shutdown_connections 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f760c066fc0 0x7f760c069470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762015db10 0x7f7620153570 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7620159890 0x7f76200cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 --2- 172.21.15.170:0/3446594986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7620159c60 0x7f76200cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:26.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 >> 172.21.15.170:0/3446594986 conn(0x7f7620068e20 msgr2=0x7f7620158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:26.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 shutdown_connections 2023-10-08T16:06:26.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:26.950+0000 7f76267f6700 1 -- 172.21.15.170:0/3446594986 wait complete. 2023-10-08T16:06:26.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:06:27.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.079+0000 7f3e715f1700 1 Processor -- start 2023-10-08T16:06:27.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.079+0000 7f3e715f1700 1 -- start start 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e715f1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3e6c153900 0x7f3e6c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e715f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0c9db0 0x7f3e6c06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e715f1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e6c06e150 con 0x7f3e6c157130 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e715f1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e6c0c7800 con 0x7f3e6c153900 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e715f1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0c7ae0 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e6bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3e6c153900 0x7f3e6c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e6bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3e6c153900 0x7f3e6c0c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50954/0 (socket says 172.21.15.170:50954) 2023-10-08T16:06:27.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e6bfff700 1 -- 172.21.15.170:0/2511856987 learned_addr learned my addr 172.21.15.170:0/2511856987 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.080+0000 7f3e6b7fe700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0c9db0 0x7f3e6c06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6bfff700 1 -- 172.21.15.170:0/2511856987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0c9db0 msgr2=0x7f3e6c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6bfff700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0c9db0 0x7f3e6c06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6bfff700 1 -- 172.21.15.170:0/2511856987 >> v1:172.21.15.191:6789/0 conn(0x7f3e6c157130 legacy=0x7f3e6c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6bfff700 1 -- 172.21.15.170:0/2511856987 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0c7dc0 con 0x7f3e6c153900 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6bfff700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c153900 0x7f3e6c0c6f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00ab50 tx=0x7f3e5c00a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35219f719084fa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6affd700 1 -- 172.21.15.170:0/2511856987 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00ca60 con 0x7f3e6c153900 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6affd700 1 -- 172.21.15.170:0/2511856987 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c00cbc0 con 0x7f3e6c153900 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e6affd700 1 -- 172.21.15.170:0/2511856987 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c018400 con 0x7f3e6c153900 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 -- 172.21.15.170:0/2511856987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c153900 msgr2=0x7f3e6c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c153900 0x7f3e6c0c6f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00ab50 tx=0x7f3e5c00a3f0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 -- 172.21.15.170:0/2511856987 shutdown_connections 2023-10-08T16:06:27.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c153900 0x7f3e6c0c6f00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 --2- 172.21.15.170:0/2511856987 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0c9db0 0x7f3e6c06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 -- 172.21.15.170:0/2511856987 >> 172.21.15.170:0/2511856987 conn(0x7f3e6c0cc1e0 msgr2=0x7f3e6c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:27.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 -- 172.21.15.170:0/2511856987 shutdown_connections 2023-10-08T16:06:27.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.081+0000 7f3e715f1700 1 -- 172.21.15.170:0/2511856987 wait complete. 2023-10-08T16:06:27.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.082+0000 7f3e715f1700 1 Processor -- start 2023-10-08T16:06:27.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 -- start start 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e6c157130 0x7f3e6c0b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0cb5e0 0x7f3e6c0af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0b1e50 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c16a020 con 0x7f3e6c157130 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.083+0000 7f3e715f1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0ac4c0 con 0x7f3e6c0cb5e0 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50968/0 (socket says 172.21.15.170:50968) 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 -- 172.21.15.170:0/3036385470 learned_addr learned my addr 172.21.15.170:0/3036385470 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6b7fe700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e6c157130 0x7f3e6c0b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e70df0700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0cb5e0 0x7f3e6c0af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 -- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0cb5e0 msgr2=0x7f3e6c0af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0cb5e0 0x7f3e6c0af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 -- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e6c157130 msgr2=0x7f3e6c0b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e6c157130 0x7f3e6c0b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 -- 172.21.15.170:0/3036385470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5c00a690 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e6bfff700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c006fd0 tx=0x7f3e5c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c023440 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c0235a0 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c169780 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c023960 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.084+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c169d40 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.085+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c158cd0 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.085+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f3e5c0103f0 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.086+0000 7f3e53fff700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 0x7f3e58069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.086+0000 7f3e6b7fe700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 0x7f3e58069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.086+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f3e5c086070 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.086+0000 7f3e6b7fe700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 0x7f3e58069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0b0f60 tx=0x7f3e60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.090+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e5c053280 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.273+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3e6c06dd20 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.273+0000 7f3e53fff700 1 -- 172.21.15.170:0/3036385470 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3e5c0568a0 con 0x7f3e6c0c9db0 2023-10-08T16:06:27.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.275+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 msgr2=0x7f3e58069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 0x7f3e58069470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0b0f60 tx=0x7f3e60008040 comp rx=0 tx=0).stop 2023-10-08T16:06:27.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 msgr2=0x7f3e6c1690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c006fd0 tx=0x7f3e5c01a000 comp rx=0 tx=0).stop 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 shutdown_connections 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3e6c0cb5e0 0x7f3e6c0af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3e58066fc0 0x7f3e58069470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e6c0c9db0 0x7f3e6c1690e0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 --2- 172.21.15.170:0/3036385470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3e6c157130 0x7f3e6c0b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 >> 172.21.15.170:0/3036385470 conn(0x7f3e6c0cc1e0 msgr2=0x7f3e6c0c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 shutdown_connections 2023-10-08T16:06:27.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.276+0000 7f3e715f1700 1 -- 172.21.15.170:0/3036385470 wait complete. 2023-10-08T16:06:27.288 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvolume_0000000000439977 --mode=777 2023-10-08T16:06:27.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.405+0000 7f7168b35700 1 Processor -- start 2023-10-08T16:06:27.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.405+0000 7f7168b35700 1 -- start start 2023-10-08T16:06:27.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7168b35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71640cd910 0x7f71640ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7168b35700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71640b5b00 con 0x7f71640b55a0 2023-10-08T16:06:27.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7168b35700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71640b5da0 con 0x7f7164159ec0 2023-10-08T16:06:27.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7168b35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71640b6080 con 0x7f71640cd910 2023-10-08T16:06:27.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7162d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f71640b55a0 0x7f71640ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:56196/0 (socket says 172.21.15.170:56196) 2023-10-08T16:06:27.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.406+0000 7f7162d9d700 1 -- 172.21.15.170:0/1911025898 learned_addr learned my addr 172.21.15.170:0/1911025898 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7161d9b700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71640cd910 0x7f71640ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073327579 0 0) 0x7f71640b5b00 con 0x7f71640b55a0 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f714c003570 con 0x7f71640b55a0 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622061775 0 0) 0x7f71640b5da0 con 0x7f7164159ec0 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71640b5b00 con 0x7f7164159ec0 2023-10-08T16:06:27.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3232421227 0 0) 0x7f714c003570 con 0x7f71640b55a0 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71640cd910 msgr2=0x7f71640ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71640cd910 0x7f71640ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 >> v1:172.21.15.191:6789/0 conn(0x7f7164159ec0 legacy=0x7f71640b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640cc100 con 0x7f71640b55a0 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f715c001470 con 0x7f71640b55a0 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7161d9b700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71640cd910 0x7f71640ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 >> v1:172.21.15.170:6789/0 conn(0x7f71640b55a0 legacy=0x7f71640ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c003c20 0x7f714c003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714c004810 0x7f714c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f714c0051c0 0x7f714c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71640b6080 con 0x7f714c0051c0 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c00bfa0 con 0x7f714c004810 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c00c130 con 0x7f714c003c20 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f714c0051c0 0x7f714c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7161d9b700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714c004810 0x7f714c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716259c700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c003c20 0x7f714c003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 -- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c003c20 msgr2=0x7f714c003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c003c20 0x7f714c003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 -- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714c004810 msgr2=0x7f714c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f714c004810 0x7f714c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f7162d9d700 1 -- 172.21.15.170:0/1911025898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7158009420 con 0x7f714c0051c0 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.407+0000 7f716259c700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c003c20 0x7f714c003ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:27.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7162d9d700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f714c0051c0 0x7f714c00ba60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c300 tx=0x7f715c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88886bfd46b405b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00cb90 con 0x7f714c0051c0 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c00ccf0 con 0x7f714c0051c0 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f716159a700 1 -- 172.21.15.170:0/1911025898 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c005b70 con 0x7f714c0051c0 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 -- 172.21.15.170:0/1911025898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f714c0051c0 msgr2=0x7f714c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 --2- 172.21.15.170:0/1911025898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f714c0051c0 0x7f714c00ba60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c300 tx=0x7f715c001e10 comp rx=0 tx=0).stop 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7162d9d700 1 -- 172.21.15.170:0/1911025898 reap_dead start 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 -- 172.21.15.170:0/1911025898 shutdown_connections 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 -- 172.21.15.170:0/1911025898 >> 172.21.15.170:0/1911025898 conn(0x7f71640999f0 msgr2=0x7f71640b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 -- 172.21.15.170:0/1911025898 shutdown_connections 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.408+0000 7f7168b35700 1 -- 172.21.15.170:0/1911025898 wait complete. 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.409+0000 7f7168b35700 1 Processor -- start 2023-10-08T16:06:27.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 -- start start 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c0051c0 0x7f716409d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716409ae70 0x7f7164155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716416a6e0 con 0x7f7164169530 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164169ee0 con 0x7f716409ae70 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7168b35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716409df70 con 0x7f714c0051c0 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7162d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7162d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50978/0 (socket says 172.21.15.170:50978) 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7162d9d700 1 -- 172.21.15.170:0/223266279 learned_addr learned my addr 172.21.15.170:0/223266279 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f716259c700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c0051c0 0x7f716409d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.410+0000 7f7161d9b700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716409ae70 0x7f7164155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 -- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c0051c0 msgr2=0x7f716409d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c0051c0 0x7f716409d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 -- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716409ae70 msgr2=0x7f7164155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716409ae70 0x7f7164155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 -- 172.21.15.170:0/223266279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716409d9b0 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7162d9d700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c2a0 tx=0x7f715c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c000e80 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c000fe0 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716409dc90 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c007320 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.411+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71640ce0c0 con 0x7f7164169530 2023-10-08T16:06:27.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.412+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164063c90 con 0x7f7164169530 2023-10-08T16:06:27.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.412+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f715c019030 con 0x7f7164169530 2023-10-08T16:06:27.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.413+0000 7f71527fc700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 0x7f7144069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.413+0000 7f716259c700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 0x7f7144069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.413+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f715c085560 con 0x7f7164169530 2023-10-08T16:06:27.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.413+0000 7f716259c700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 0x7f7144069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7154000f80 tx=0x7f715400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.417+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f715c052770 con 0x7f7164169530 2023-10-08T16:06:27.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.631+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvolume_0000000000439977", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f71640ce2b0 con 0x7f7144066fc0 2023-10-08T16:06:27.632 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.632+0000 7f60a202f700 -1 client.5493 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-08T16:06:02.082330+0000 mtime=2023-10-08T16:06:02.082330+0000 ctime=2023-10-08T16:06:02.087825+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000414781"] has_dir_layout 0x560b3e5fa000) 2023-10-08T16:06:27.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.636+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:27.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.636+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:27.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.636+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:27.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.636+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:27.636 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:06:27.636+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:06:27.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.666+0000 7f71527fc700 1 -- 172.21.15.170:0/223266279 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f71640ce2b0 con 0x7f7144066fc0 2023-10-08T16:06:27.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 msgr2=0x7f7144069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 0x7f7144069470 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7154000f80 tx=0x7f715400c040 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 msgr2=0x7f71641699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c2a0 tx=0x7f715c015000 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 shutdown_connections 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7144066fc0 0x7f7144069470 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7164169530 0x7f71641699a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f714c0051c0 0x7f716409d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 --2- 172.21.15.170:0/223266279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f716409ae70 0x7f7164155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 >> 172.21.15.170:0/223266279 conn(0x7f71640999f0 msgr2=0x7f71640c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 shutdown_connections 2023-10-08T16:06:27.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.669+0000 7f7168b35700 1 -- 172.21.15.170:0/223266279 wait complete. 2023-10-08T16:06:27.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvolume_0000000000439977 2023-10-08T16:06:27.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.799+0000 7f8f89522700 1 Processor -- start 2023-10-08T16:06:27.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.799+0000 7f8f89522700 1 -- start start 2023-10-08T16:06:27.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f89522700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f840b4e70 0x7f8f840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f89522700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f840b16d0 0x7f8f840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f89522700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840b2080 con 0x7f8f8415d240 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f89522700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840cfb00 con 0x7f8f840b4e70 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f89522700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840b5c40 con 0x7f8f840b16d0 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f840b4e70 0x7f8f840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f88d21700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f8415d240 0x7f8f8415d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:56206/0 (socket says 172.21.15.170:56206) 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f88d21700 1 -- 172.21.15.170:0/1260946473 learned_addr learned my addr 172.21.15.170:0/1260946473 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f837fe700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f840b16d0 0x7f8f840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 -- 172.21.15.170:0/1260946473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f840b16d0 msgr2=0x7f8f840b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f840b16d0 0x7f8f840b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 -- 172.21.15.170:0/1260946473 >> v1:172.21.15.170:6789/0 conn(0x7f8f8415d240 legacy=0x7f8f8415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f82ffd700 1 -- 172.21.15.170:0/1260946473 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564581082 0 0) 0x7f8f840b2080 con 0x7f8f8415d240 2023-10-08T16:06:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 -- 172.21.15.170:0/1260946473 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840b5f20 con 0x7f8f840b4e70 2023-10-08T16:06:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f83fff700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b4e70 0x7f8f840ccff0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f780098e0 tx=0x7f8f780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0684e779e5a350 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f82ffd700 1 -- 172.21.15.170:0/1260946473 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800b640 con 0x7f8f840b4e70 2023-10-08T16:06:27.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f82ffd700 1 -- 172.21.15.170:0/1260946473 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7800b7a0 con 0x7f8f840b4e70 2023-10-08T16:06:27.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.800+0000 7f8f82ffd700 1 -- 172.21.15.170:0/1260946473 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800bb40 con 0x7f8f840b4e70 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 -- 172.21.15.170:0/1260946473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b4e70 msgr2=0x7f8f840ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b4e70 0x7f8f840ccff0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f780098e0 tx=0x7f8f780091a0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 -- 172.21.15.170:0/1260946473 shutdown_connections 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b4e70 0x7f8f840ccff0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 --2- 172.21.15.170:0/1260946473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f840b16d0 0x7f8f840b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 -- 172.21.15.170:0/1260946473 >> 172.21.15.170:0/1260946473 conn(0x7f8f840d10c0 msgr2=0x7f8f840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 -- 172.21.15.170:0/1260946473 shutdown_connections 2023-10-08T16:06:27.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.801+0000 7f8f89522700 1 -- 172.21.15.170:0/1260946473 wait complete. 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.802+0000 7f8f89522700 1 Processor -- start 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.802+0000 7f8f89522700 1 -- start start 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 0x7f8f840cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f8415d240 0x7f8f840cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840cc430 con 0x7f8f840b16d0 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c6770 con 0x7f8f840b4e70 2023-10-08T16:06:27.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f89522700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c9e90 con 0x7f8f8415d240 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51010/0 (socket says 172.21.15.170:51010) 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 -- 172.21.15.170:0/3677820252 learned_addr learned my addr 172.21.15.170:0/3677820252 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f88d21700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f8415d240 0x7f8f840cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f837fe700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 0x7f8f840cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 -- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f8415d240 msgr2=0x7f8f840cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f8415d240 0x7f8f840cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 -- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 msgr2=0x7f8f840cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 0x7f8f840cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 -- 172.21.15.170:0/3677820252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f78009420 con 0x7f8f840b16d0 2023-10-08T16:06:27.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f837fe700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 0x7f8f840cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.803+0000 7f8f83fff700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f780093f0 tx=0x7f8f78017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.804+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7801f070 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.804+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78000bc0 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.804+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800bb40 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.804+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f840ca170 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.804+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f840ca450 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.805+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f84088990 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.805+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f8f78023070 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.805+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 0x7f8f70069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.806+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f8f78085b20 con 0x7f8f840b16d0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.806+0000 7f8f837fe700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 0x7f8f70069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:06:27.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.806+0000 7f8f837fe700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 0x7f8f70069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f840cf510 tx=0x7f8f74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:06:27.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:27.810+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f78087e60 con 0x7f8f840b16d0 2023-10-08T16:06:28.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.023+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvolume_0000000000439977", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f840c68b0 con 0x7f8f70066fc0 2023-10-08T16:06:28.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.028+0000 7f8f6bfff700 1 -- 172.21.15.170:0/3677820252 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f8f74009aa0 con 0x7f8f70066fc0 2023-10-08T16:06:28.029 INFO:teuthology.orchestra.run.smithi170.stdout:/volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 msgr2=0x7f8f70069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 0x7f8f70069470 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f840cf510 tx=0x7f8f74008040 comp rx=0 tx=0).stop 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 msgr2=0x7f8f841576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f780093f0 tx=0x7f8f78017000 comp rx=0 tx=0).stop 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 shutdown_connections 2023-10-08T16:06:28.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f8415d240 0x7f8f840cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f70066fc0 0x7f8f70069470 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f840b16d0 0x7f8f841576e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 --2- 172.21.15.170:0/3677820252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f840b4e70 0x7f8f840cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 >> 172.21.15.170:0/3677820252 conn(0x7f8f840d10c0 msgr2=0x7f8f8415c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 shutdown_connections 2023-10-08T16:06:28.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:06:28.030+0000 7f8f89522700 1 -- 172.21.15.170:0/3677820252 wait complete. 2023-10-08T16:06:28.043 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvolume_0000000000439977 with 3000 files each 0.1MB size under directory volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37 2023-10-08T16:06:28.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.043 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.0 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.066 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3603e-05 s, 0.0 kB/s 2023-10-08T16:06:28.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.067 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2599e-05 s, 0.0 kB/s 2023-10-08T16:06:28.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.179 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7469e-05 s, 0.0 kB/s 2023-10-08T16:06:28.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.3 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.234 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3984e-05 s, 0.0 kB/s 2023-10-08T16:06:28.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.235 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.4 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.291 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0219e-05 s, 0.0 kB/s 2023-10-08T16:06:28.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.291 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.5 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.346 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.347 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1002e-05 s, 0.0 kB/s 2023-10-08T16:06:28.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.347 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.6 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1965e-05 s, 0.0 kB/s 2023-10-08T16:06:28.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.403 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.7 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.458 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0139e-05 s, 0.0 kB/s 2023-10-08T16:06:28.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.459 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.8 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.514 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.514 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9319e-05 s, 0.0 kB/s 2023-10-08T16:06:28.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.514 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.9 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.149e-05 s, 0.0 kB/s 2023-10-08T16:06:28.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.570 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.10 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.625 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1486e-05 s, 0.0 kB/s 2023-10-08T16:06:28.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.627 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.11 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7431e-05 s, 0.0 kB/s 2023-10-08T16:06:28.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.684 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.12 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.740 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8294e-05 s, 0.0 kB/s 2023-10-08T16:06:28.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.740 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.13 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.795 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6831e-05 s, 0.0 kB/s 2023-10-08T16:06:28.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.796 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.14 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.851 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0384e-05 s, 0.0 kB/s 2023-10-08T16:06:28.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.852 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.15 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.905 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1931e-05 s, 0.0 kB/s 2023-10-08T16:06:28.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.16 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:28.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:28.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:28.963 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-10-08T16:06:28.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:28.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.17 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.019 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0546e-05 s, 0.0 kB/s 2023-10-08T16:06:29.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.019 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.18 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.076 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8197e-05 s, 0.0 kB/s 2023-10-08T16:06:29.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.077 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.19 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.132 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.132 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.132 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5418e-05 s, 0.0 kB/s 2023-10-08T16:06:29.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.133 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.20 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9908e-05 s, 0.0 kB/s 2023-10-08T16:06:29.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.191 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.21 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.245 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.85e-05 s, 0.0 kB/s 2023-10-08T16:06:29.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.246 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.22 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.301 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8291e-05 s, 0.0 kB/s 2023-10-08T16:06:29.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.301 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.23 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3292e-05 s, 0.0 kB/s 2023-10-08T16:06:29.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.24 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.412 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.807e-05 s, 0.0 kB/s 2023-10-08T16:06:29.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.412 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.25 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.468 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6102e-05 s, 0.0 kB/s 2023-10-08T16:06:29.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.469 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.26 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.523 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.001e-05 s, 0.0 kB/s 2023-10-08T16:06:29.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.27 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5362e-05 s, 0.0 kB/s 2023-10-08T16:06:29.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.28 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0208e-05 s, 0.0 kB/s 2023-10-08T16:06:29.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.636 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.29 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.691 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0031e-05 s, 0.0 kB/s 2023-10-08T16:06:29.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.692 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.30 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.748 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0592e-05 s, 0.0 kB/s 2023-10-08T16:06:29.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.748 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.31 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.803 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.803 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.803 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8742e-05 s, 0.0 kB/s 2023-10-08T16:06:29.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.804 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.32 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.860 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.821e-05 s, 0.0 kB/s 2023-10-08T16:06:29.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.860 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.33 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.901e-05 s, 0.0 kB/s 2023-10-08T16:06:29.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.916 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.34 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9121e-05 s, 0.0 kB/s 2023-10-08T16:06:29.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:29.972 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.35 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.028 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0828e-05 s, 0.0 kB/s 2023-10-08T16:06:30.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.029 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.36 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.084 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6621e-05 s, 0.0 kB/s 2023-10-08T16:06:30.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.084 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.37 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.140 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3871e-05 s, 0.0 kB/s 2023-10-08T16:06:30.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.141 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.38 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.197 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8385e-05 s, 0.0 kB/s 2023-10-08T16:06:30.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.197 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.39 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.253 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8922e-05 s, 0.0 kB/s 2023-10-08T16:06:30.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.254 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.40 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.309 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.309 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.309 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4005e-05 s, 0.0 kB/s 2023-10-08T16:06:30.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.309 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.41 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.365 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3378e-05 s, 0.0 kB/s 2023-10-08T16:06:30.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.365 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.42 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.420 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.421 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6422e-05 s, 0.0 kB/s 2023-10-08T16:06:30.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.421 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.43 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.476 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.476 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.476 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4791e-05 s, 0.0 kB/s 2023-10-08T16:06:30.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.477 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.44 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.532 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.532 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.532 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7118e-05 s, 0.0 kB/s 2023-10-08T16:06:30.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.532 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.45 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.588 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-08T16:06:30.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.588 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.46 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.645 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2325e-05 s, 0.0 kB/s 2023-10-08T16:06:30.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.646 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.47 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.702 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3126e-05 s, 0.0 kB/s 2023-10-08T16:06:30.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.703 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.48 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.758 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.759 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.759 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0256e-05 s, 0.0 kB/s 2023-10-08T16:06:30.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.759 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.49 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.814 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.5567e-05 s, 0.0 kB/s 2023-10-08T16:06:30.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.815 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.50 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.871 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.871 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2078e-05 s, 0.0 kB/s 2023-10-08T16:06:30.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.871 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.51 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1155e-05 s, 0.0 kB/s 2023-10-08T16:06:30.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.927 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.52 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9809e-05 s, 0.0 kB/s 2023-10-08T16:06:30.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:30.983 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.53 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.040 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.036e-05 s, 0.0 kB/s 2023-10-08T16:06:31.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.54 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.095 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0975e-05 s, 0.0 kB/s 2023-10-08T16:06:31.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.096 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.55 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.152 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5275e-05 s, 0.0 kB/s 2023-10-08T16:06:31.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.152 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.56 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.208 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1418e-05 s, 0.0 kB/s 2023-10-08T16:06:31.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.208 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.57 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.264 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.264 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0512e-05 s, 0.0 kB/s 2023-10-08T16:06:31.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.264 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.58 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.320 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4892e-05 s, 0.0 kB/s 2023-10-08T16:06:31.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.321 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.59 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.377 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.377 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.377 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8785e-05 s, 0.0 kB/s 2023-10-08T16:06:31.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.377 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.60 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.433 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9931e-05 s, 0.0 kB/s 2023-10-08T16:06:31.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.433 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.61 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1664e-05 s, 0.0 kB/s 2023-10-08T16:06:31.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.489 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.62 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.543 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9338e-05 s, 0.0 kB/s 2023-10-08T16:06:31.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.544 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.63 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.599 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.599 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.599 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4482e-05 s, 0.0 kB/s 2023-10-08T16:06:31.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.600 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.64 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.656 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-10-08T16:06:31.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.657 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.65 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.712 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7812e-05 s, 0.0 kB/s 2023-10-08T16:06:31.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.713 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.66 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.768 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9446e-05 s, 0.0 kB/s 2023-10-08T16:06:31.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.67 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.823 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5355e-05 s, 0.0 kB/s 2023-10-08T16:06:31.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.824 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.68 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.879 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9844e-05 s, 0.0 kB/s 2023-10-08T16:06:31.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.880 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.69 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.936 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6321e-05 s, 0.0 kB/s 2023-10-08T16:06:31.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.70 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:31.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:31.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:31.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0451e-05 s, 0.0 kB/s 2023-10-08T16:06:31.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:31.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.71 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9261e-05 s, 0.0 kB/s 2023-10-08T16:06:32.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.049 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.72 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.105 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8966e-05 s, 0.0 kB/s 2023-10-08T16:06:32.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.73 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.161 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.161 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-10-08T16:06:32.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.161 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.74 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.217 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4764e-05 s, 0.0 kB/s 2023-10-08T16:06:32.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.217 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.75 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.273 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6417e-05 s, 0.0 kB/s 2023-10-08T16:06:32.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.273 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.76 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.329 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.131e-05 s, 0.0 kB/s 2023-10-08T16:06:32.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.77 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.386 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8675e-05 s, 0.0 kB/s 2023-10-08T16:06:32.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.387 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.78 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.442 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0185e-05 s, 0.0 kB/s 2023-10-08T16:06:32.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.442 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.79 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.499 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.499 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.499 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3885e-05 s, 0.0 kB/s 2023-10-08T16:06:32.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.499 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.80 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.554 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1125e-05 s, 0.0 kB/s 2023-10-08T16:06:32.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.555 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.81 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.611 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.611 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.611 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1119e-05 s, 0.0 kB/s 2023-10-08T16:06:32.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.82 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.667 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9248e-05 s, 0.0 kB/s 2023-10-08T16:06:32.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.668 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.83 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.723 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0894e-05 s, 0.0 kB/s 2023-10-08T16:06:32.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.723 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.84 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.779 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-08T16:06:32.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.779 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.85 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.834 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-08T16:06:32.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.835 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.86 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3739e-05 s, 0.0 kB/s 2023-10-08T16:06:32.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.890 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.87 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:32.945 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:32.945 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:32.945 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8359e-05 s, 0.0 kB/s 2023-10-08T16:06:32.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:32.946 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.88 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.001 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5521e-05 s, 0.0 kB/s 2023-10-08T16:06:33.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.002 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.89 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.058 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6595e-05 s, 0.0 kB/s 2023-10-08T16:06:33.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.058 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.90 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.113 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.089e-05 s, 0.0 kB/s 2023-10-08T16:06:33.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.113 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.91 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.168 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.169 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3648e-05 s, 0.0 kB/s 2023-10-08T16:06:33.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.169 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.92 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.225 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1264e-05 s, 0.0 kB/s 2023-10-08T16:06:33.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.225 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.93 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.280 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.280 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.280 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6502e-05 s, 0.0 kB/s 2023-10-08T16:06:33.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.281 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.94 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3305e-05 s, 0.0 kB/s 2023-10-08T16:06:33.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.95 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0829e-05 s, 0.0 kB/s 2023-10-08T16:06:33.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.392 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.96 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.448 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2357e-05 s, 0.0 kB/s 2023-10-08T16:06:33.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.448 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.97 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0436e-05 s, 0.0 kB/s 2023-10-08T16:06:33.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.504 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.98 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7725e-05 s, 0.0 kB/s 2023-10-08T16:06:33.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.560 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.99 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.616 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.616 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7885e-05 s, 0.0 kB/s 2023-10-08T16:06:33.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.616 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9405e-05 s, 0.0 kB/s 2023-10-08T16:06:33.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.673 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.727 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0507e-05 s, 0.0 kB/s 2023-10-08T16:06:33.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.728 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.784 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6561e-05 s, 0.0 kB/s 2023-10-08T16:06:33.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.784 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.840 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9499e-05 s, 0.0 kB/s 2023-10-08T16:06:33.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.895 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0618e-05 s, 0.0 kB/s 2023-10-08T16:06:33.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.895 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:33.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:33.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:33.951 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4688e-05 s, 0.0 kB/s 2023-10-08T16:06:33.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:33.951 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.006 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0908e-05 s, 0.0 kB/s 2023-10-08T16:06:34.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.007 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.062 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.062 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.062 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2688e-05 s, 0.0 kB/s 2023-10-08T16:06:34.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.063 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8861e-05 s, 0.0 kB/s 2023-10-08T16:06:34.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.119 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.174 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8109e-05 s, 0.0 kB/s 2023-10-08T16:06:34.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.175 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.230 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1676e-05 s, 0.0 kB/s 2023-10-08T16:06:34.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.231 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.285 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.285 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.285 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0859e-05 s, 0.0 kB/s 2023-10-08T16:06:34.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.286 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.341 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.341 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.341 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0306e-05 s, 0.0 kB/s 2023-10-08T16:06:34.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.342 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.396 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.396 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1469e-05 s, 0.0 kB/s 2023-10-08T16:06:34.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.397 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.453 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.453 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.453 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9322e-05 s, 0.0 kB/s 2023-10-08T16:06:34.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.453 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.511 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.848e-05 s, 0.0 kB/s 2023-10-08T16:06:34.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.512 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.565 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.565 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.565 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9492e-05 s, 0.0 kB/s 2023-10-08T16:06:34.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.567 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9482e-05 s, 0.0 kB/s 2023-10-08T16:06:34.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.622 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.678 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9548e-05 s, 0.0 kB/s 2023-10-08T16:06:34.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.678 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.733 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.733 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.733 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6075e-05 s, 0.0 kB/s 2023-10-08T16:06:34.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.735 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.790 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.887e-05 s, 0.0 kB/s 2023-10-08T16:06:34.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.791 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.845 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.845 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.845 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7685e-05 s, 0.0 kB/s 2023-10-08T16:06:34.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.846 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.901 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.901 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.901 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.112e-05 s, 0.0 kB/s 2023-10-08T16:06:34.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.902 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:34.956 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:34.956 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:34.956 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0246e-05 s, 0.0 kB/s 2023-10-08T16:06:34.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:34.957 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.011 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6245e-05 s, 0.0 kB/s 2023-10-08T16:06:35.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.012 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.067 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1545e-05 s, 0.0 kB/s 2023-10-08T16:06:35.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0581e-05 s, 0.0 kB/s 2023-10-08T16:06:35.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.179 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0672e-05 s, 0.0 kB/s 2023-10-08T16:06:35.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9434e-05 s, 0.0 kB/s 2023-10-08T16:06:35.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.234 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.289 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3162e-05 s, 0.0 kB/s 2023-10-08T16:06:35.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.290 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.345 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2525e-05 s, 0.0 kB/s 2023-10-08T16:06:35.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.345 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.401 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.999e-05 s, 0.0 kB/s 2023-10-08T16:06:35.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.456 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.952e-05 s, 0.0 kB/s 2023-10-08T16:06:35.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.457 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.512 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8511e-05 s, 0.0 kB/s 2023-10-08T16:06:35.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.512 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.567 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.567 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.567 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6114e-05 s, 0.0 kB/s 2023-10-08T16:06:35.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.568 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.625 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2134e-05 s, 0.0 kB/s 2023-10-08T16:06:35.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.682 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1687e-05 s, 0.0 kB/s 2023-10-08T16:06:35.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.682 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.738 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0108e-05 s, 0.0 kB/s 2023-10-08T16:06:35.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.738 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.793 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.794 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7018e-05 s, 0.0 kB/s 2023-10-08T16:06:35.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.794 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.849 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0181e-05 s, 0.0 kB/s 2023-10-08T16:06:35.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.850 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.902e-05 s, 0.0 kB/s 2023-10-08T16:06:35.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:35.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:35.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:35.963 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000108526 s, 0.0 kB/s 2023-10-08T16:06:35.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:35.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.019 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0652e-05 s, 0.0 kB/s 2023-10-08T16:06:36.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.019 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.074 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8546e-05 s, 0.0 kB/s 2023-10-08T16:06:36.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.075 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.131 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9564e-05 s, 0.0 kB/s 2023-10-08T16:06:36.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.131 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.186 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.187 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3336e-05 s, 0.0 kB/s 2023-10-08T16:06:36.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.187 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9094e-05 s, 0.0 kB/s 2023-10-08T16:06:36.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.244 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.299 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.369 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7167e-05 s, 0.0 kB/s 2023-10-08T16:06:36.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.386 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1465e-05 s, 0.0 kB/s 2023-10-08T16:06:36.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.387 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.441 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.441 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.441 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9672e-05 s, 0.0 kB/s 2023-10-08T16:06:36.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.442 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.497 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1246e-05 s, 0.0 kB/s 2023-10-08T16:06:36.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.497 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.554 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2421e-05 s, 0.0 kB/s 2023-10-08T16:06:36.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.555 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.610 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2028e-05 s, 0.0 kB/s 2023-10-08T16:06:36.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.666 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.666 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.666 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1201e-05 s, 0.0 kB/s 2023-10-08T16:06:36.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.667 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.722 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9369e-05 s, 0.0 kB/s 2023-10-08T16:06:36.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.722 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.849 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0352e-05 s, 0.0 kB/s 2023-10-08T16:06:36.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.854 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1351e-05 s, 0.0 kB/s 2023-10-08T16:06:36.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.870 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0779e-05 s, 0.0 kB/s 2023-10-08T16:06:36.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:36.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:36.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:36.981 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2138e-05 s, 0.0 kB/s 2023-10-08T16:06:36.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:36.982 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.037 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1322e-05 s, 0.0 kB/s 2023-10-08T16:06:37.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.038 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.092 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8815e-05 s, 0.0 kB/s 2023-10-08T16:06:37.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.092 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.148 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.148 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5015e-05 s, 0.0 kB/s 2023-10-08T16:06:37.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.148 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.204 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.204 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6308e-05 s, 0.0 kB/s 2023-10-08T16:06:37.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.204 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.259 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2918e-05 s, 0.0 kB/s 2023-10-08T16:06:37.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.314 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8335e-05 s, 0.0 kB/s 2023-10-08T16:06:37.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2415e-05 s, 0.0 kB/s 2023-10-08T16:06:37.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.372 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.427 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0928e-05 s, 0.0 kB/s 2023-10-08T16:06:37.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.427 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8347e-05 s, 0.0 kB/s 2023-10-08T16:06:37.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.485 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0987e-05 s, 0.0 kB/s 2023-10-08T16:06:37.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8713e-05 s, 0.0 kB/s 2023-10-08T16:06:37.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0798e-05 s, 0.0 kB/s 2023-10-08T16:06:37.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.653 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.707 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0201e-05 s, 0.0 kB/s 2023-10-08T16:06:37.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.708 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4845e-05 s, 0.0 kB/s 2023-10-08T16:06:37.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.764 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.819 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.819 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.819 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1205e-05 s, 0.0 kB/s 2023-10-08T16:06:37.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.820 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.875 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0224e-05 s, 0.0 kB/s 2023-10-08T16:06:37.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.876 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.931 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.931 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.639e-05 s, 0.0 kB/s 2023-10-08T16:06:37.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.931 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:37.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:37.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:37.987 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8051e-05 s, 0.0 kB/s 2023-10-08T16:06:37.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:37.987 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.042 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.042 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.042 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0948e-05 s, 0.0 kB/s 2023-10-08T16:06:38.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.043 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.099 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9708e-05 s, 0.0 kB/s 2023-10-08T16:06:38.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.099 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.155 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1816e-05 s, 0.0 kB/s 2023-10-08T16:06:38.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.155 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.211 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.1709e-05 s, 0.0 kB/s 2023-10-08T16:06:38.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.211 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.267 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0431e-05 s, 0.0 kB/s 2023-10-08T16:06:38.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.267 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.322 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.079e-05 s, 0.0 kB/s 2023-10-08T16:06:38.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.323 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.377 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.377 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.377 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0732e-05 s, 0.0 kB/s 2023-10-08T16:06:38.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.378 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.433 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-08T16:06:38.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.434 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.489 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.490 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000104497 s, 0.0 kB/s 2023-10-08T16:06:38.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.490 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.545 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5438e-05 s, 0.0 kB/s 2023-10-08T16:06:38.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.546 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9495e-05 s, 0.0 kB/s 2023-10-08T16:06:38.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.601 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.656 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3333e-05 s, 0.0 kB/s 2023-10-08T16:06:38.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.656 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7811e-05 s, 0.0 kB/s 2023-10-08T16:06:38.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2392e-05 s, 0.0 kB/s 2023-10-08T16:06:38.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9953e-05 s, 0.0 kB/s 2023-10-08T16:06:38.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7299e-05 s, 0.0 kB/s 2023-10-08T16:06:38.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8872e-05 s, 0.0 kB/s 2023-10-08T16:06:38.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.937 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:38.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:38.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:38.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.122e-05 s, 0.0 kB/s 2023-10-08T16:06:38.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:38.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3364e-05 s, 0.0 kB/s 2023-10-08T16:06:39.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.048 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0995e-05 s, 0.0 kB/s 2023-10-08T16:06:39.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9115e-05 s, 0.0 kB/s 2023-10-08T16:06:39.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9819e-05 s, 0.0 kB/s 2023-10-08T16:06:39.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.4128e-05 s, 0.0 kB/s 2023-10-08T16:06:39.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.328 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9919e-05 s, 0.0 kB/s 2023-10-08T16:06:39.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.384 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0081e-05 s, 0.0 kB/s 2023-10-08T16:06:39.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4701e-05 s, 0.0 kB/s 2023-10-08T16:06:39.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.496 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9485e-05 s, 0.0 kB/s 2023-10-08T16:06:39.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.497 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.552 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.552 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2308e-05 s, 0.0 kB/s 2023-10-08T16:06:39.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.552 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.608 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8782e-05 s, 0.0 kB/s 2023-10-08T16:06:39.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.608 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.663 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.663 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.663 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4164e-05 s, 0.0 kB/s 2023-10-08T16:06:39.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.664 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.720 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2092e-05 s, 0.0 kB/s 2023-10-08T16:06:39.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.721 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.776 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0488e-05 s, 0.0 kB/s 2023-10-08T16:06:39.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.777 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.832 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.832 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.832 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0561e-05 s, 0.0 kB/s 2023-10-08T16:06:39.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.833 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.887 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.888 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.888 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8584e-05 s, 0.0 kB/s 2023-10-08T16:06:39.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.888 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.942 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.943 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.943 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4608e-05 s, 0.0 kB/s 2023-10-08T16:06:39.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:39.944 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:39.998 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:39.999 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0007e-05 s, 0.0 kB/s 2023-10-08T16:06:39.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.000 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.056 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1638e-05 s, 0.0 kB/s 2023-10-08T16:06:40.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.056 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.112 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3181e-05 s, 0.0 kB/s 2023-10-08T16:06:40.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.112 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.167 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0127e-05 s, 0.0 kB/s 2023-10-08T16:06:40.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.168 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7749e-05 s, 0.0 kB/s 2023-10-08T16:06:40.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.224 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.280 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.280 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4552e-05 s, 0.0 kB/s 2023-10-08T16:06:40.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.280 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6288e-05 s, 0.0 kB/s 2023-10-08T16:06:40.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5545e-05 s, 0.0 kB/s 2023-10-08T16:06:40.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.392 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1648e-05 s, 0.0 kB/s 2023-10-08T16:06:40.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.448 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0957e-05 s, 0.0 kB/s 2023-10-08T16:06:40.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.504 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7662e-05 s, 0.0 kB/s 2023-10-08T16:06:40.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.559 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.615 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2425e-05 s, 0.0 kB/s 2023-10-08T16:06:40.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.616 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.671 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9729e-05 s, 0.0 kB/s 2023-10-08T16:06:40.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.672 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.727 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1764e-05 s, 0.0 kB/s 2023-10-08T16:06:40.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.783 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.862e-05 s, 0.0 kB/s 2023-10-08T16:06:40.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.783 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.838 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.839 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7522e-05 s, 0.0 kB/s 2023-10-08T16:06:40.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.839 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.894 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.895 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1383e-05 s, 0.0 kB/s 2023-10-08T16:06:40.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.895 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:40.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:40.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:40.950 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0007e-05 s, 0.0 kB/s 2023-10-08T16:06:40.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:40.951 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.007 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.007 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6703e-05 s, 0.0 kB/s 2023-10-08T16:06:41.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.007 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6164e-05 s, 0.0 kB/s 2023-10-08T16:06:41.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.065 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.120 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9464e-05 s, 0.0 kB/s 2023-10-08T16:06:41.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.120 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.175 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1776e-05 s, 0.0 kB/s 2023-10-08T16:06:41.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.230 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6805e-05 s, 0.0 kB/s 2023-10-08T16:06:41.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.231 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.286 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.286 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.287 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0436e-05 s, 0.0 kB/s 2023-10-08T16:06:41.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.287 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.342 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.342 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.343 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1375e-05 s, 0.0 kB/s 2023-10-08T16:06:41.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.343 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.399 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6915e-05 s, 0.0 kB/s 2023-10-08T16:06:41.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.400 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.455 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.455 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.455 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9975e-05 s, 0.0 kB/s 2023-10-08T16:06:41.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.456 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.510 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.511 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9138e-05 s, 0.0 kB/s 2023-10-08T16:06:41.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.511 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2321e-05 s, 0.0 kB/s 2023-10-08T16:06:41.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.566 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.621 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.621 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.621 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4808e-05 s, 0.0 kB/s 2023-10-08T16:06:41.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.622 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.678 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3528e-05 s, 0.0 kB/s 2023-10-08T16:06:41.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.679 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.733 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.733 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.733 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0815e-05 s, 0.0 kB/s 2023-10-08T16:06:41.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.734 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6881e-05 s, 0.0 kB/s 2023-10-08T16:06:41.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.791 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.845 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.845 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.845 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.9913e-05 s, 0.0 kB/s 2023-10-08T16:06:41.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.846 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.901 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.901 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.901 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.585e-05 s, 0.0 kB/s 2023-10-08T16:06:41.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.902 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:41.956 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:41.956 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:41.956 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8685e-05 s, 0.0 kB/s 2023-10-08T16:06:41.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:41.957 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.011 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4402e-05 s, 0.0 kB/s 2023-10-08T16:06:42.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.012 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.067 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9396e-05 s, 0.0 kB/s 2023-10-08T16:06:42.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0782e-05 s, 0.0 kB/s 2023-10-08T16:06:42.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.179 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2311e-05 s, 0.0 kB/s 2023-10-08T16:06:42.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.235 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1372e-05 s, 0.0 kB/s 2023-10-08T16:06:42.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.235 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.290 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.291 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2332e-05 s, 0.0 kB/s 2023-10-08T16:06:42.291 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.291 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.347 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.1383e-05 s, 0.0 kB/s 2023-10-08T16:06:42.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.347 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9368e-05 s, 0.0 kB/s 2023-10-08T16:06:42.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.403 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9105e-05 s, 0.0 kB/s 2023-10-08T16:06:42.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.459 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.515 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8035e-05 s, 0.0 kB/s 2023-10-08T16:06:42.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.515 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2644e-05 s, 0.0 kB/s 2023-10-08T16:06:42.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.626 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4411e-05 s, 0.0 kB/s 2023-10-08T16:06:42.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.627 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9948e-05 s, 0.0 kB/s 2023-10-08T16:06:42.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.683 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.738 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7111e-05 s, 0.0 kB/s 2023-10-08T16:06:42.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.739 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.795 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5102e-05 s, 0.0 kB/s 2023-10-08T16:06:42.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.850 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6878e-05 s, 0.0 kB/s 2023-10-08T16:06:42.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.908 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6815e-05 s, 0.0 kB/s 2023-10-08T16:06:42.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.908 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:42.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:42.964 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8748e-05 s, 0.0 kB/s 2023-10-08T16:06:42.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:42.964 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.020 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1981e-05 s, 0.0 kB/s 2023-10-08T16:06:43.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.020 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:06:43.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.076 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6538e-05 s, 0.0 kB/s 2023-10-08T16:06:43.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.076 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.118 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:06:43.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.133 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1379e-05 s, 0.0 kB/s 2023-10-08T16:06:43.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.133 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.189 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1892e-05 s, 0.0 kB/s 2023-10-08T16:06:43.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.189 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9524e-05 s, 0.0 kB/s 2023-10-08T16:06:43.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.244 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.300 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3542e-05 s, 0.0 kB/s 2023-10-08T16:06:43.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.301 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1472e-05 s, 0.0 kB/s 2023-10-08T16:06:43.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.411 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.251e-05 s, 0.0 kB/s 2023-10-08T16:06:43.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.412 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.467 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9396e-05 s, 0.0 kB/s 2023-10-08T16:06:43.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.467 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.524 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.524 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.525 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0178e-05 s, 0.0 kB/s 2023-10-08T16:06:43.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.525 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.580 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0455e-05 s, 0.0 kB/s 2023-10-08T16:06:43.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.581 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.636 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.637 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3828e-05 s, 0.0 kB/s 2023-10-08T16:06:43.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.638 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.693 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.693 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.693 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0181e-05 s, 0.0 kB/s 2023-10-08T16:06:43.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.693 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0001e-05 s, 0.0 kB/s 2023-10-08T16:06:43.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.750 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.805 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.805 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.805 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7957e-05 s, 0.0 kB/s 2023-10-08T16:06:43.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.806 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.051e-05 s, 0.0 kB/s 2023-10-08T16:06:43.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.861 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.917 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.917 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.917 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7111e-05 s, 0.0 kB/s 2023-10-08T16:06:43.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.918 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:43.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:43.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:43.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0199e-05 s, 0.0 kB/s 2023-10-08T16:06:43.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:43.973 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.027 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8582e-05 s, 0.0 kB/s 2023-10-08T16:06:44.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.083 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5979e-05 s, 0.0 kB/s 2023-10-08T16:06:44.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.083 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1318e-05 s, 0.0 kB/s 2023-10-08T16:06:44.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.141 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.196 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.196 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.196 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2297e-05 s, 0.0 kB/s 2023-10-08T16:06:44.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.197 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.251 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2243e-05 s, 0.0 kB/s 2023-10-08T16:06:44.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.251 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8982e-05 s, 0.0 kB/s 2023-10-08T16:06:44.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.307 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.363 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9553e-05 s, 0.0 kB/s 2023-10-08T16:06:44.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.363 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.418 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.4602e-05 s, 0.0 kB/s 2023-10-08T16:06:44.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.419 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.475 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-08T16:06:44.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.475 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.530 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1111e-05 s, 0.0 kB/s 2023-10-08T16:06:44.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.533 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.588 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.646e-05 s, 0.0 kB/s 2023-10-08T16:06:44.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.589 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9534e-05 s, 0.0 kB/s 2023-10-08T16:06:44.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.698 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1215e-05 s, 0.0 kB/s 2023-10-08T16:06:44.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.699 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.754 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.754 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9058e-05 s, 0.0 kB/s 2023-10-08T16:06:44.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.754 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.175e-05 s, 0.0 kB/s 2023-10-08T16:06:44.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6695e-05 s, 0.0 kB/s 2023-10-08T16:06:44.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.922 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2803e-05 s, 0.0 kB/s 2023-10-08T16:06:44.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:44.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:44.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:44.978 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.049e-05 s, 0.0 kB/s 2023-10-08T16:06:44.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:44.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.033 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.033 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.033 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.965e-05 s, 0.0 kB/s 2023-10-08T16:06:45.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.034 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.090 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.090 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1869e-05 s, 0.0 kB/s 2023-10-08T16:06:45.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.091 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.147 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2465e-05 s, 0.0 kB/s 2023-10-08T16:06:45.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.147 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.201 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.202 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0264e-05 s, 0.0 kB/s 2023-10-08T16:06:45.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.202 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.256 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.257 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9456e-05 s, 0.0 kB/s 2023-10-08T16:06:45.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.313 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0774e-05 s, 0.0 kB/s 2023-10-08T16:06:45.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.313 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.368 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.369 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4185e-05 s, 0.0 kB/s 2023-10-08T16:06:45.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.369 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.424 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9281e-05 s, 0.0 kB/s 2023-10-08T16:06:45.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.425 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.479 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.480 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.480 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1515e-05 s, 0.0 kB/s 2023-10-08T16:06:45.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.480 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.535 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.535 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.535 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2563e-05 s, 0.0 kB/s 2023-10-08T16:06:45.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.536 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.590 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.591 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5965e-05 s, 0.0 kB/s 2023-10-08T16:06:45.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.592 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.648 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1997e-05 s, 0.0 kB/s 2023-10-08T16:06:45.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.649 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.704 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0416e-05 s, 0.0 kB/s 2023-10-08T16:06:45.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.704 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.759 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.759 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.759 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9849e-05 s, 0.0 kB/s 2023-10-08T16:06:45.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.760 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.816 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1639e-05 s, 0.0 kB/s 2023-10-08T16:06:45.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.816 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.872 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.186e-05 s, 0.0 kB/s 2023-10-08T16:06:45.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.873 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0108e-05 s, 0.0 kB/s 2023-10-08T16:06:45.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.929 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:45.983 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:45.984 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:45.985 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8391e-05 s, 0.0 kB/s 2023-10-08T16:06:45.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:45.985 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.038 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8395e-05 s, 0.0 kB/s 2023-10-08T16:06:46.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.039 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.093 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9614e-05 s, 0.0 kB/s 2023-10-08T16:06:46.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.149 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0514e-05 s, 0.0 kB/s 2023-10-08T16:06:46.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.150 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.204 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.204 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1113e-05 s, 0.0 kB/s 2023-10-08T16:06:46.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.205 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.260 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.260 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.260 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9717e-05 s, 0.0 kB/s 2023-10-08T16:06:46.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.261 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.315 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8729e-05 s, 0.0 kB/s 2023-10-08T16:06:46.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0452e-05 s, 0.0 kB/s 2023-10-08T16:06:46.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.372 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.428 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1695e-05 s, 0.0 kB/s 2023-10-08T16:06:46.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.428 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.483 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6565e-05 s, 0.0 kB/s 2023-10-08T16:06:46.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.483 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.538 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.539 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8495e-05 s, 0.0 kB/s 2023-10-08T16:06:46.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.539 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.595 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-08T16:06:46.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.595 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.651 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8525e-05 s, 0.0 kB/s 2023-10-08T16:06:46.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.651 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.707 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9686e-05 s, 0.0 kB/s 2023-10-08T16:06:46.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.707 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.762 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.762 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.763 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0792e-05 s, 0.0 kB/s 2023-10-08T16:06:46.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.764 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2212e-05 s, 0.0 kB/s 2023-10-08T16:06:46.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.821 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.876 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9335e-05 s, 0.0 kB/s 2023-10-08T16:06:46.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.932 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.057e-05 s, 0.0 kB/s 2023-10-08T16:06:46.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:46.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:46.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:46.988 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9445e-05 s, 0.0 kB/s 2023-10-08T16:06:46.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:46.988 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.044 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.044 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.044 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0812e-05 s, 0.0 kB/s 2023-10-08T16:06:47.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.044 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.100 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8765e-05 s, 0.0 kB/s 2023-10-08T16:06:47.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.156 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1815e-05 s, 0.0 kB/s 2023-10-08T16:06:47.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.1015e-05 s, 0.0 kB/s 2023-10-08T16:06:47.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.267 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.206e-05 s, 0.0 kB/s 2023-10-08T16:06:47.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0014e-05 s, 0.0 kB/s 2023-10-08T16:06:47.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.324 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7648e-05 s, 0.0 kB/s 2023-10-08T16:06:47.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.379 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.434 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9781e-05 s, 0.0 kB/s 2023-10-08T16:06:47.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.435 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.492 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9552e-05 s, 0.0 kB/s 2023-10-08T16:06:47.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.492 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2392e-05 s, 0.0 kB/s 2023-10-08T16:06:47.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.548 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.639 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.639 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9276e-05 s, 0.0 kB/s 2023-10-08T16:06:47.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.640 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7722e-05 s, 0.0 kB/s 2023-10-08T16:06:47.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9962e-05 s, 0.0 kB/s 2023-10-08T16:06:47.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.771 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0471e-05 s, 0.0 kB/s 2023-10-08T16:06:47.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.772 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.827 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1416e-05 s, 0.0 kB/s 2023-10-08T16:06:47.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.828 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6198e-05 s, 0.0 kB/s 2023-10-08T16:06:47.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.884 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.939 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9495e-05 s, 0.0 kB/s 2023-10-08T16:06:47.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.940 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:47.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:47.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:47.995 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9995e-05 s, 0.0 kB/s 2023-10-08T16:06:47.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:47.996 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.050 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.247 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9899e-05 s, 0.0 kB/s 2023-10-08T16:06:48.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.248 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.304 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9829e-05 s, 0.0 kB/s 2023-10-08T16:06:48.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.305 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.360 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5701e-05 s, 0.0 kB/s 2023-10-08T16:06:48.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.361 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8144e-05 s, 0.0 kB/s 2023-10-08T16:06:48.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.417 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.473 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.473 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.473 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.135e-05 s, 0.0 kB/s 2023-10-08T16:06:48.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.474 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.528 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5055e-05 s, 0.0 kB/s 2023-10-08T16:06:48.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.529 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.584 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1497e-05 s, 0.0 kB/s 2023-10-08T16:06:48.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.584 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4687e-05 s, 0.0 kB/s 2023-10-08T16:06:48.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.641 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.696 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.696 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.696 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5141e-05 s, 0.0 kB/s 2023-10-08T16:06:48.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.697 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.753 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-08T16:06:48.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.754 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.809 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6769e-05 s, 0.0 kB/s 2023-10-08T16:06:48.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.809 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.865 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.865 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.865 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0177e-05 s, 0.0 kB/s 2023-10-08T16:06:48.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.865 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.920 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.920 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.920 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4458e-05 s, 0.0 kB/s 2023-10-08T16:06:48.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.921 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:48.976 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:48.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:48.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2936e-05 s, 0.0 kB/s 2023-10-08T16:06:48.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:48.996 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.033 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.034 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9449e-05 s, 0.0 kB/s 2023-10-08T16:06:49.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.034 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.089 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9748e-05 s, 0.0 kB/s 2023-10-08T16:06:49.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.090 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.145 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1574e-05 s, 0.0 kB/s 2023-10-08T16:06:49.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.146 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.200 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.200 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.200 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.2478e-05 s, 0.0 kB/s 2023-10-08T16:06:49.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.201 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.257 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0075e-05 s, 0.0 kB/s 2023-10-08T16:06:49.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.257 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.312 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.313 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7825e-05 s, 0.0 kB/s 2023-10-08T16:06:49.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.313 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.368 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.368 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.368 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4042e-05 s, 0.0 kB/s 2023-10-08T16:06:49.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.369 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.424 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.425 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7658e-05 s, 0.0 kB/s 2023-10-08T16:06:49.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.425 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.481 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0139e-05 s, 0.0 kB/s 2023-10-08T16:06:49.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.481 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.536 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.736e-05 s, 0.0 kB/s 2023-10-08T16:06:49.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.591 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0373e-05 s, 0.0 kB/s 2023-10-08T16:06:49.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.592 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.647 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.648 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4114e-05 s, 0.0 kB/s 2023-10-08T16:06:49.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.706 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.799 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.799 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.837e-05 s, 0.0 kB/s 2023-10-08T16:06:49.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.800 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.858 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-08T16:06:49.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.860 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1937e-05 s, 0.0 kB/s 2023-10-08T16:06:49.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:49.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:49.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:49.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7535e-05 s, 0.0 kB/s 2023-10-08T16:06:49.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:49.973 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.029 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.029 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4222e-05 s, 0.0 kB/s 2023-10-08T16:06:50.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.029 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.084 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7285e-05 s, 0.0 kB/s 2023-10-08T16:06:50.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.085 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-10-08T16:06:50.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.140 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.196 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.196 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-10-08T16:06:50.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.196 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.251 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-08T16:06:50.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.252 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.307 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.307 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7691e-05 s, 0.0 kB/s 2023-10-08T16:06:50.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.307 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.363 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.363 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7364e-05 s, 0.0 kB/s 2023-10-08T16:06:50.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.363 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9962e-05 s, 0.0 kB/s 2023-10-08T16:06:50.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.419 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.474 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5971e-05 s, 0.0 kB/s 2023-10-08T16:06:50.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.476 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.531 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9659e-05 s, 0.0 kB/s 2023-10-08T16:06:50.532 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.532 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0257e-05 s, 0.0 kB/s 2023-10-08T16:06:50.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.642 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.642 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.642 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6294e-05 s, 0.0 kB/s 2023-10-08T16:06:50.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.642 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.698 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8368e-05 s, 0.0 kB/s 2023-10-08T16:06:50.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.698 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.754 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1918e-05 s, 0.0 kB/s 2023-10-08T16:06:50.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.754 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.810 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9931e-05 s, 0.0 kB/s 2023-10-08T16:06:50.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.810 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.865 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.865 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.865 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9902e-05 s, 0.0 kB/s 2023-10-08T16:06:50.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.866 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1764e-05 s, 0.0 kB/s 2023-10-08T16:06:50.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.921 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:50.976 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:50.976 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:50.976 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3566e-05 s, 0.0 kB/s 2023-10-08T16:06:50.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:50.977 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.032 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.032 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0945e-05 s, 0.0 kB/s 2023-10-08T16:06:51.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.032 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.088 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3715e-05 s, 0.0 kB/s 2023-10-08T16:06:51.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.088 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.144 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.144 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7566e-05 s, 0.0 kB/s 2023-10-08T16:06:51.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.144 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.199 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.200 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.200 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2388e-05 s, 0.0 kB/s 2023-10-08T16:06:51.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.200 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.256 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.256 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.023e-05 s, 0.0 kB/s 2023-10-08T16:06:51.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.256 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.311 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.312 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.312 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8969e-05 s, 0.0 kB/s 2023-10-08T16:06:51.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.312 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.367 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0042e-05 s, 0.0 kB/s 2023-10-08T16:06:51.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.368 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.423 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2143e-05 s, 0.0 kB/s 2023-10-08T16:06:51.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.424 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.479 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.479 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.479 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0251e-05 s, 0.0 kB/s 2023-10-08T16:06:51.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.479 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.534 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6492e-05 s, 0.0 kB/s 2023-10-08T16:06:51.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1702e-05 s, 0.0 kB/s 2023-10-08T16:06:51.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.590 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.824e-05 s, 0.0 kB/s 2023-10-08T16:06:51.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.005e-05 s, 0.0 kB/s 2023-10-08T16:06:51.702 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.702 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.756 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8559e-05 s, 0.0 kB/s 2023-10-08T16:06:51.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.757 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.5303e-05 s, 0.0 kB/s 2023-10-08T16:06:51.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.812 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.868 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-10-08T16:06:51.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.869 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2572e-05 s, 0.0 kB/s 2023-10-08T16:06:51.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.925 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:51.979 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9634e-05 s, 0.0 kB/s 2023-10-08T16:06:51.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:51.980 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.036 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.036 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.832e-05 s, 0.0 kB/s 2023-10-08T16:06:52.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.036 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.092 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9462e-05 s, 0.0 kB/s 2023-10-08T16:06:52.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.092 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.148 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1992e-05 s, 0.0 kB/s 2023-10-08T16:06:52.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.148 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0468e-05 s, 0.0 kB/s 2023-10-08T16:06:52.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.204 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.258 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.108e-05 s, 0.0 kB/s 2023-10-08T16:06:52.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.313 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0135e-05 s, 0.0 kB/s 2023-10-08T16:06:52.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.314 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.370 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0897e-05 s, 0.0 kB/s 2023-10-08T16:06:52.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6079e-05 s, 0.0 kB/s 2023-10-08T16:06:52.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.426 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.482 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9827e-05 s, 0.0 kB/s 2023-10-08T16:06:52.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.482 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.537 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.538 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.538 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.559e-05 s, 0.0 kB/s 2023-10-08T16:06:52.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.538 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.594 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.594 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1395e-05 s, 0.0 kB/s 2023-10-08T16:06:52.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.594 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.650 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0729e-05 s, 0.0 kB/s 2023-10-08T16:06:52.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0467e-05 s, 0.0 kB/s 2023-10-08T16:06:52.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.706 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3128e-05 s, 0.0 kB/s 2023-10-08T16:06:52.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.763 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.817 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.818 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.818 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2476e-05 s, 0.0 kB/s 2023-10-08T16:06:52.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.818 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.874 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8981e-05 s, 0.0 kB/s 2023-10-08T16:06:52.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.874 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0885e-05 s, 0.0 kB/s 2023-10-08T16:06:52.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.930 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:52.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:52.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:52.985 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6353e-05 s, 0.0 kB/s 2023-10-08T16:06:52.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:52.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.040 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9894e-05 s, 0.0 kB/s 2023-10-08T16:06:53.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.095 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9749e-05 s, 0.0 kB/s 2023-10-08T16:06:53.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.096 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.151 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0626e-05 s, 0.0 kB/s 2023-10-08T16:06:53.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.152 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9363e-05 s, 0.0 kB/s 2023-10-08T16:06:53.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.207 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.261 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.85e-05 s, 0.0 kB/s 2023-10-08T16:06:53.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.261 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.315 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8645e-05 s, 0.0 kB/s 2023-10-08T16:06:53.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.372 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9753e-05 s, 0.0 kB/s 2023-10-08T16:06:53.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.428 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.428 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7008e-05 s, 0.0 kB/s 2023-10-08T16:06:53.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.428 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1465e-05 s, 0.0 kB/s 2023-10-08T16:06:53.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.485 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.541 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1302e-05 s, 0.0 kB/s 2023-10-08T16:06:53.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.541 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.596 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0238e-05 s, 0.0 kB/s 2023-10-08T16:06:53.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.597 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.651 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7998e-05 s, 0.0 kB/s 2023-10-08T16:06:53.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.652 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.708 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9929e-05 s, 0.0 kB/s 2023-10-08T16:06:53.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.708 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.763 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.763 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.763 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1315e-05 s, 0.0 kB/s 2023-10-08T16:06:53.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.764 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.819 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.819 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.819 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0718e-05 s, 0.0 kB/s 2023-10-08T16:06:53.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.820 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.875 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1953e-05 s, 0.0 kB/s 2023-10-08T16:06:53.875 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.875 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.930 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9271e-05 s, 0.0 kB/s 2023-10-08T16:06:53.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.931 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:53.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:53.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:53.987 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8013e-05 s, 0.0 kB/s 2023-10-08T16:06:53.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:53.987 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.042 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.042 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.042 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0081e-05 s, 0.0 kB/s 2023-10-08T16:06:54.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.042 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.097 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.097 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.097 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.75e-05 s, 0.0 kB/s 2023-10-08T16:06:54.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.098 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.152 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-08T16:06:54.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.153 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.208 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9606e-05 s, 0.0 kB/s 2023-10-08T16:06:54.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.209 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.264 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7874e-05 s, 0.0 kB/s 2023-10-08T16:06:54.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.264 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.318 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.319 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7927e-05 s, 0.0 kB/s 2023-10-08T16:06:54.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.320 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.375 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0025e-05 s, 0.0 kB/s 2023-10-08T16:06:54.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.376 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1612e-05 s, 0.0 kB/s 2023-10-08T16:06:54.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.431 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.487 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.487 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7582e-05 s, 0.0 kB/s 2023-10-08T16:06:54.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.487 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.543 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.543 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9966e-05 s, 0.0 kB/s 2023-10-08T16:06:54.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.543 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.598 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9793e-05 s, 0.0 kB/s 2023-10-08T16:06:54.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.599 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5126e-05 s, 0.0 kB/s 2023-10-08T16:06:54.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.653 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.709 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8886e-05 s, 0.0 kB/s 2023-10-08T16:06:54.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.710 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.766 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.766 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.766 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1253e-05 s, 0.0 kB/s 2023-10-08T16:06:54.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.766 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8826e-05 s, 0.0 kB/s 2023-10-08T16:06:54.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.822 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.876 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0248e-05 s, 0.0 kB/s 2023-10-08T16:06:54.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.933 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0079e-05 s, 0.0 kB/s 2023-10-08T16:06:54.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.934 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:54.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:54.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:54.988 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000100741 s, 0.0 kB/s 2023-10-08T16:06:54.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:54.988 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.043 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.044 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.044 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1648e-05 s, 0.0 kB/s 2023-10-08T16:06:55.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.044 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.098 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.099 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.099 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-08T16:06:55.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.100 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.155 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2232e-05 s, 0.0 kB/s 2023-10-08T16:06:55.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9661e-05 s, 0.0 kB/s 2023-10-08T16:06:55.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.267 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1571e-05 s, 0.0 kB/s 2023-10-08T16:06:55.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.322 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.322 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.047e-05 s, 0.0 kB/s 2023-10-08T16:06:55.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.323 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.378 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-10-08T16:06:55.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.379 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.433 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1468e-05 s, 0.0 kB/s 2023-10-08T16:06:55.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.434 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.489 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.489 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.489 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1175e-05 s, 0.0 kB/s 2023-10-08T16:06:55.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.490 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.545 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6968e-05 s, 0.0 kB/s 2023-10-08T16:06:55.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.546 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.600 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3977e-05 s, 0.0 kB/s 2023-10-08T16:06:55.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.601 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.655 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0347e-05 s, 0.0 kB/s 2023-10-08T16:06:55.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.656 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.705e-05 s, 0.0 kB/s 2023-10-08T16:06:55.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.767 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3268e-05 s, 0.0 kB/s 2023-10-08T16:06:55.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.824 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9558e-05 s, 0.0 kB/s 2023-10-08T16:06:55.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.824 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.879 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6819e-05 s, 0.0 kB/s 2023-10-08T16:06:55.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.880 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1576e-05 s, 0.0 kB/s 2023-10-08T16:06:55.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.935 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:55.990 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:55.990 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:55.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0635e-05 s, 0.0 kB/s 2023-10-08T16:06:55.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:55.991 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7087e-05 s, 0.0 kB/s 2023-10-08T16:06:56.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1572e-05 s, 0.0 kB/s 2023-10-08T16:06:56.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9317e-05 s, 0.0 kB/s 2023-10-08T16:06:56.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.159 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.213 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.214 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7694e-05 s, 0.0 kB/s 2023-10-08T16:06:56.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.214 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.269 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.269 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2478e-05 s, 0.0 kB/s 2023-10-08T16:06:56.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.269 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8975e-05 s, 0.0 kB/s 2023-10-08T16:06:56.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.379 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9106e-05 s, 0.0 kB/s 2023-10-08T16:06:56.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.434 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.434 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.434 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6082e-05 s, 0.0 kB/s 2023-10-08T16:06:56.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.435 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.490 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.117e-05 s, 0.0 kB/s 2023-10-08T16:06:56.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.491 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.546 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.547 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1881e-05 s, 0.0 kB/s 2023-10-08T16:06:56.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.547 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1544e-05 s, 0.0 kB/s 2023-10-08T16:06:56.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.659 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.659 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.659 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-08T16:06:56.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.660 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0222e-05 s, 0.0 kB/s 2023-10-08T16:06:56.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.715 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.771 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1649e-05 s, 0.0 kB/s 2023-10-08T16:06:56.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9054e-05 s, 0.0 kB/s 2023-10-08T16:06:56.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1477e-05 s, 0.0 kB/s 2023-10-08T16:06:56.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7829e-05 s, 0.0 kB/s 2023-10-08T16:06:56.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:56.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6677e-05 s, 0.0 kB/s 2023-10-08T16:06:56.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:56.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.046 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0316e-05 s, 0.0 kB/s 2023-10-08T16:06:57.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2078e-05 s, 0.0 kB/s 2023-10-08T16:06:57.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.102 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.157 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0768e-05 s, 0.0 kB/s 2023-10-08T16:06:57.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.158 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9884e-05 s, 0.0 kB/s 2023-10-08T16:06:57.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.213 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.267 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0568e-05 s, 0.0 kB/s 2023-10-08T16:06:57.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.137e-05 s, 0.0 kB/s 2023-10-08T16:06:57.323 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.323 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.378 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1591e-05 s, 0.0 kB/s 2023-10-08T16:06:57.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.379 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.433 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1489e-05 s, 0.0 kB/s 2023-10-08T16:06:57.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.434 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.490 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3755e-05 s, 0.0 kB/s 2023-10-08T16:06:57.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.490 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.545 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0408e-05 s, 0.0 kB/s 2023-10-08T16:06:57.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.546 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.601 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.601 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.601 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0304e-05 s, 0.0 kB/s 2023-10-08T16:06:57.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.601 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.657 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.657 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5437e-05 s, 0.0 kB/s 2023-10-08T16:06:57.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.657 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.712 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.116e-05 s, 0.0 kB/s 2023-10-08T16:06:57.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.713 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.768 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8231e-05 s, 0.0 kB/s 2023-10-08T16:06:57.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.824 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0613e-05 s, 0.0 kB/s 2023-10-08T16:06:57.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0807e-05 s, 0.0 kB/s 2023-10-08T16:06:57.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1058e-05 s, 0.0 kB/s 2023-10-08T16:06:57.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:57.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:57.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:57.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8152e-05 s, 0.0 kB/s 2023-10-08T16:06:57.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:57.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1244e-05 s, 0.0 kB/s 2023-10-08T16:06:58.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.102 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5191e-05 s, 0.0 kB/s 2023-10-08T16:06:58.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.157 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9684e-05 s, 0.0 kB/s 2023-10-08T16:06:58.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.158 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5898e-05 s, 0.0 kB/s 2023-10-08T16:06:58.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.213 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6695e-05 s, 0.0 kB/s 2023-10-08T16:06:58.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.269 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6849e-05 s, 0.0 kB/s 2023-10-08T16:06:58.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.379 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-10-08T16:06:58.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.436 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2764e-05 s, 0.0 kB/s 2023-10-08T16:06:58.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.437 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.493 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8781e-05 s, 0.0 kB/s 2023-10-08T16:06:58.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.493 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.548 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0738e-05 s, 0.0 kB/s 2023-10-08T16:06:58.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.549 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1928e-05 s, 0.0 kB/s 2023-10-08T16:06:58.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0677e-05 s, 0.0 kB/s 2023-10-08T16:06:58.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.660 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-10-08T16:06:58.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.770 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5868e-05 s, 0.0 kB/s 2023-10-08T16:06:58.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9623e-05 s, 0.0 kB/s 2023-10-08T16:06:58.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9234e-05 s, 0.0 kB/s 2023-10-08T16:06:58.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.937 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.023e-05 s, 0.0 kB/s 2023-10-08T16:06:58.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.937 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:58.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:58.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:58.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.055e-05 s, 0.0 kB/s 2023-10-08T16:06:58.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:58.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9509e-05 s, 0.0 kB/s 2023-10-08T16:06:59.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.048 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8954e-05 s, 0.0 kB/s 2023-10-08T16:06:59.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7781e-05 s, 0.0 kB/s 2023-10-08T16:06:59.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.159 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.214 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.214 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.214 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9407e-05 s, 0.0 kB/s 2023-10-08T16:06:59.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.271 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.08e-05 s, 0.0 kB/s 2023-10-08T16:06:59.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.271 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.326 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.774e-05 s, 0.0 kB/s 2023-10-08T16:06:59.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.327 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.382 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.382 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.382 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9252e-05 s, 0.0 kB/s 2023-10-08T16:06:59.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.382 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.437 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.437 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.437 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3075e-05 s, 0.0 kB/s 2023-10-08T16:06:59.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.438 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5024e-05 s, 0.0 kB/s 2023-10-08T16:06:59.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5057e-05 s, 0.0 kB/s 2023-10-08T16:06:59.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.604 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6745e-05 s, 0.0 kB/s 2023-10-08T16:06:59.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.659 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.659 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9687e-05 s, 0.0 kB/s 2023-10-08T16:06:59.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.660 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-08T16:06:59.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.771 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9699e-05 s, 0.0 kB/s 2023-10-08T16:06:59.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.826 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8311e-05 s, 0.0 kB/s 2023-10-08T16:06:59.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.827 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.883 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0389e-05 s, 0.0 kB/s 2023-10-08T16:06:59.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.883 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.939 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9445e-05 s, 0.0 kB/s 2023-10-08T16:06:59.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.940 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:06:59.994 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:06:59.994 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:06:59.994 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9189e-05 s, 0.0 kB/s 2023-10-08T16:06:59.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:06:59.994 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6668e-05 s, 0.0 kB/s 2023-10-08T16:07:00.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.050 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.104 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2548e-05 s, 0.0 kB/s 2023-10-08T16:07:00.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.159 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8728e-05 s, 0.0 kB/s 2023-10-08T16:07:00.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7785e-05 s, 0.0 kB/s 2023-10-08T16:07:00.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0348e-05 s, 0.0 kB/s 2023-10-08T16:07:00.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.328 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9936e-05 s, 0.0 kB/s 2023-10-08T16:07:00.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2989e-05 s, 0.0 kB/s 2023-10-08T16:07:00.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.438 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0337e-05 s, 0.0 kB/s 2023-10-08T16:07:00.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.438 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.858e-05 s, 0.0 kB/s 2023-10-08T16:07:00.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.494 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0227e-05 s, 0.0 kB/s 2023-10-08T16:07:00.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.604 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1336e-05 s, 0.0 kB/s 2023-10-08T16:07:00.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7501e-05 s, 0.0 kB/s 2023-10-08T16:07:00.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.660 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.717 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0209e-05 s, 0.0 kB/s 2023-10-08T16:07:00.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.772 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000101925 s, 0.0 kB/s 2023-10-08T16:07:00.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.773 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.829 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0644e-05 s, 0.0 kB/s 2023-10-08T16:07:00.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.829 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.884 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0654e-05 s, 0.0 kB/s 2023-10-08T16:07:00.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.885 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.940 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2695e-05 s, 0.0 kB/s 2023-10-08T16:07:00.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:00.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1614e-05 s, 0.0 kB/s 2023-10-08T16:07:00.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:00.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.051 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9113e-05 s, 0.0 kB/s 2023-10-08T16:07:01.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.106 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.107 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9609e-05 s, 0.0 kB/s 2023-10-08T16:07:01.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.163 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0941e-05 s, 0.0 kB/s 2023-10-08T16:07:01.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.164 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.219 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5324e-05 s, 0.0 kB/s 2023-10-08T16:07:01.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.220 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.274 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8267e-05 s, 0.0 kB/s 2023-10-08T16:07:01.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.275 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.329 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1669e-05 s, 0.0 kB/s 2023-10-08T16:07:01.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.384 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.384 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.384 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8024e-05 s, 0.0 kB/s 2023-10-08T16:07:01.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.385 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.439 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.439 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.439 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0544e-05 s, 0.0 kB/s 2023-10-08T16:07:01.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.440 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9501e-05 s, 0.0 kB/s 2023-10-08T16:07:01.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9527e-05 s, 0.0 kB/s 2023-10-08T16:07:01.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.604 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2569e-05 s, 0.0 kB/s 2023-10-08T16:07:01.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9426e-05 s, 0.0 kB/s 2023-10-08T16:07:01.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5821e-05 s, 0.0 kB/s 2023-10-08T16:07:01.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.772 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5292e-05 s, 0.0 kB/s 2023-10-08T16:07:01.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.773 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.828 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9539e-05 s, 0.0 kB/s 2023-10-08T16:07:01.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.829 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.884 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5214e-05 s, 0.0 kB/s 2023-10-08T16:07:01.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.884 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.940 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5248e-05 s, 0.0 kB/s 2023-10-08T16:07:01.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:01.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:01.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:01.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.057e-05 s, 0.0 kB/s 2023-10-08T16:07:01.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:01.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.051 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1108e-05 s, 0.0 kB/s 2023-10-08T16:07:02.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.106 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.107 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1291e-05 s, 0.0 kB/s 2023-10-08T16:07:02.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.107 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.162 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6071e-05 s, 0.0 kB/s 2023-10-08T16:07:02.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.163 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.217 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3598e-05 s, 0.0 kB/s 2023-10-08T16:07:02.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.218 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.273 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8914e-05 s, 0.0 kB/s 2023-10-08T16:07:02.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.274 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.328 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1825e-05 s, 0.0 kB/s 2023-10-08T16:07:02.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.329 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.384 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4825e-05 s, 0.0 kB/s 2023-10-08T16:07:02.385 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.385 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8109e-05 s, 0.0 kB/s 2023-10-08T16:07:02.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.495 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.496 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2578e-05 s, 0.0 kB/s 2023-10-08T16:07:02.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.497 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.553 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9149e-05 s, 0.0 kB/s 2023-10-08T16:07:02.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.553 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.609 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.609 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.609 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1091e-05 s, 0.0 kB/s 2023-10-08T16:07:02.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.609 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.665 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.665 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8431e-05 s, 0.0 kB/s 2023-10-08T16:07:02.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.665 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0197e-05 s, 0.0 kB/s 2023-10-08T16:07:02.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.721 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6837e-05 s, 0.0 kB/s 2023-10-08T16:07:02.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.778 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2611e-05 s, 0.0 kB/s 2023-10-08T16:07:02.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.834 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.890 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9636e-05 s, 0.0 kB/s 2023-10-08T16:07:02.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.891 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:02.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:02.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:02.947 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5245e-05 s, 0.0 kB/s 2023-10-08T16:07:02.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:02.948 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.002 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.002 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.002 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1332e-05 s, 0.0 kB/s 2023-10-08T16:07:03.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.003 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.057 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9548e-05 s, 0.0 kB/s 2023-10-08T16:07:03.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.058 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.113 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1032e-05 s, 0.0 kB/s 2023-10-08T16:07:03.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.114 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.169 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5005e-05 s, 0.0 kB/s 2023-10-08T16:07:03.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.170 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0727e-05 s, 0.0 kB/s 2023-10-08T16:07:03.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.226 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7612e-05 s, 0.0 kB/s 2023-10-08T16:07:03.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.283 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.338 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.339 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.339 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0053e-05 s, 0.0 kB/s 2023-10-08T16:07:03.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.339 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7045e-05 s, 0.0 kB/s 2023-10-08T16:07:03.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0361e-05 s, 0.0 kB/s 2023-10-08T16:07:03.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.451 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.506 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1502e-05 s, 0.0 kB/s 2023-10-08T16:07:03.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.506 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.561 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.562 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9117e-05 s, 0.0 kB/s 2023-10-08T16:07:03.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.562 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.617 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0338e-05 s, 0.0 kB/s 2023-10-08T16:07:03.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.618 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.673 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.673 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.673 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0462e-05 s, 0.0 kB/s 2023-10-08T16:07:03.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.673 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.728 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4392e-05 s, 0.0 kB/s 2023-10-08T16:07:03.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.729 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7931e-05 s, 0.0 kB/s 2023-10-08T16:07:03.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.785 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.841 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0428e-05 s, 0.0 kB/s 2023-10-08T16:07:03.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.841 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.896 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.897 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.897 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0956e-05 s, 0.0 kB/s 2023-10-08T16:07:03.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.897 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:03.953 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:03.953 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:03.953 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8599e-05 s, 0.0 kB/s 2023-10-08T16:07:03.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:03.954 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0429e-05 s, 0.0 kB/s 2023-10-08T16:07:04.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.009 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6675e-05 s, 0.0 kB/s 2023-10-08T16:07:04.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.065 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.121 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1822e-05 s, 0.0 kB/s 2023-10-08T16:07:04.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.121 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1362e-05 s, 0.0 kB/s 2023-10-08T16:07:04.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.177 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.231 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.232 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6003e-05 s, 0.0 kB/s 2023-10-08T16:07:04.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.232 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.288 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6253e-05 s, 0.0 kB/s 2023-10-08T16:07:04.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.288 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.343 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.343 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.343 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5355e-05 s, 0.0 kB/s 2023-10-08T16:07:04.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.344 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.399 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2131e-05 s, 0.0 kB/s 2023-10-08T16:07:04.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.400 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.456 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2408e-05 s, 0.0 kB/s 2023-10-08T16:07:04.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.456 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.513 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1712e-05 s, 0.0 kB/s 2023-10-08T16:07:04.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.569 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.978e-05 s, 0.0 kB/s 2023-10-08T16:07:04.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.625 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.859e-05 s, 0.0 kB/s 2023-10-08T16:07:04.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.681 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5935e-05 s, 0.0 kB/s 2023-10-08T16:07:04.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.681 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5565e-05 s, 0.0 kB/s 2023-10-08T16:07:04.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.738 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.794 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9333e-05 s, 0.0 kB/s 2023-10-08T16:07:04.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.794 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.849 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9517e-05 s, 0.0 kB/s 2023-10-08T16:07:04.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.850 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8405e-05 s, 0.0 kB/s 2023-10-08T16:07:04.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.905 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:04.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:04.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:04.960 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6058e-05 s, 0.0 kB/s 2023-10-08T16:07:04.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:04.961 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1261e-05 s, 0.0 kB/s 2023-10-08T16:07:05.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.017 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.072 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.072 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.073 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0378e-05 s, 0.0 kB/s 2023-10-08T16:07:05.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.073 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1612e-05 s, 0.0 kB/s 2023-10-08T16:07:05.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.129 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9427e-05 s, 0.0 kB/s 2023-10-08T16:07:05.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.185 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8961e-05 s, 0.0 kB/s 2023-10-08T16:07:05.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.241 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.296 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.296 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.296 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9125e-05 s, 0.0 kB/s 2023-10-08T16:07:05.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.297 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.351 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7532e-05 s, 0.0 kB/s 2023-10-08T16:07:05.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.352 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.406 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.407 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.407 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1865e-05 s, 0.0 kB/s 2023-10-08T16:07:05.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.408 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.463 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.464 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1037e-05 s, 0.0 kB/s 2023-10-08T16:07:05.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.464 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.519 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.519 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.519 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000102438 s, 0.0 kB/s 2023-10-08T16:07:05.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.520 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8639e-05 s, 0.0 kB/s 2023-10-08T16:07:05.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.575 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.630 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0621e-05 s, 0.0 kB/s 2023-10-08T16:07:05.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.631 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.686 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1087e-05 s, 0.0 kB/s 2023-10-08T16:07:05.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.687 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.742 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8133e-05 s, 0.0 kB/s 2023-10-08T16:07:05.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.742 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.798 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.798 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.798 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9564e-05 s, 0.0 kB/s 2023-10-08T16:07:05.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.798 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3426e-05 s, 0.0 kB/s 2023-10-08T16:07:05.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.854 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.909 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0151e-05 s, 0.0 kB/s 2023-10-08T16:07:05.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.910 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:05.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:05.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:05.965 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0112e-05 s, 0.0 kB/s 2023-10-08T16:07:05.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:05.966 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.022 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.022 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.022 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9584e-05 s, 0.0 kB/s 2023-10-08T16:07:06.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.022 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.078 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9449e-05 s, 0.0 kB/s 2023-10-08T16:07:06.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.078 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.134 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.134 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0169e-05 s, 0.0 kB/s 2023-10-08T16:07:06.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.134 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.190 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9378e-05 s, 0.0 kB/s 2023-10-08T16:07:06.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.190 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.246 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9268e-05 s, 0.0 kB/s 2023-10-08T16:07:06.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.246 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.301 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8482e-05 s, 0.0 kB/s 2023-10-08T16:07:06.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.302 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.358 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7354e-05 s, 0.0 kB/s 2023-10-08T16:07:06.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.359 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.415 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.415 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7938e-05 s, 0.0 kB/s 2023-10-08T16:07:06.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.415 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.470 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.470 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.470 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1557e-05 s, 0.0 kB/s 2023-10-08T16:07:06.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.471 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.525 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.525 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.525 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-10-08T16:07:06.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.526 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.581 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.581 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.581 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.062e-05 s, 0.0 kB/s 2023-10-08T16:07:06.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.582 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.637 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9739e-05 s, 0.0 kB/s 2023-10-08T16:07:06.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.637 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.693 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.4549e-05 s, 0.0 kB/s 2023-10-08T16:07:06.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.694 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.750 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.750 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.750 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1857e-05 s, 0.0 kB/s 2023-10-08T16:07:06.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.750 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.805 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.806 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.806 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0644e-05 s, 0.0 kB/s 2023-10-08T16:07:06.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.806 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.862 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2881e-05 s, 0.0 kB/s 2023-10-08T16:07:06.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.862 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.917 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.918 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6158e-05 s, 0.0 kB/s 2023-10-08T16:07:06.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.918 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:06.974 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:06.974 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:06.974 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3565e-05 s, 0.0 kB/s 2023-10-08T16:07:06.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:06.975 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.030 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.031 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9944e-05 s, 0.0 kB/s 2023-10-08T16:07:07.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.031 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.087 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9981e-05 s, 0.0 kB/s 2023-10-08T16:07:07.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.087 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9864e-05 s, 0.0 kB/s 2023-10-08T16:07:07.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.143 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.199 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.199 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1292e-05 s, 0.0 kB/s 2023-10-08T16:07:07.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.199 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.255 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9054e-05 s, 0.0 kB/s 2023-10-08T16:07:07.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.255 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.310 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7978e-05 s, 0.0 kB/s 2023-10-08T16:07:07.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.311 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.366 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1826e-05 s, 0.0 kB/s 2023-10-08T16:07:07.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.367 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.423 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4218e-05 s, 0.0 kB/s 2023-10-08T16:07:07.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.424 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.479 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.480 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.480 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2111e-05 s, 0.0 kB/s 2023-10-08T16:07:07.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.480 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.536 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1719e-05 s, 0.0 kB/s 2023-10-08T16:07:07.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.592 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3631e-05 s, 0.0 kB/s 2023-10-08T16:07:07.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.592 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.647 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.647 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.647 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1134e-05 s, 0.0 kB/s 2023-10-08T16:07:07.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.648 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.704 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.174e-05 s, 0.0 kB/s 2023-10-08T16:07:07.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.705 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.759 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.759 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.759 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8366e-05 s, 0.0 kB/s 2023-10-08T16:07:07.759 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.759 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.814 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0099e-05 s, 0.0 kB/s 2023-10-08T16:07:07.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.815 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.871 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.871 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.872 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9716e-05 s, 0.0 kB/s 2023-10-08T16:07:07.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.873 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1781e-05 s, 0.0 kB/s 2023-10-08T16:07:07.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.929 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:07.984 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:07.984 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:07.984 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3732e-05 s, 0.0 kB/s 2023-10-08T16:07:07.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:07.985 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.040 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8211e-05 s, 0.0 kB/s 2023-10-08T16:07:08.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.097 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.097 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1174e-05 s, 0.0 kB/s 2023-10-08T16:07:08.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.097 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.152 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9223e-05 s, 0.0 kB/s 2023-10-08T16:07:08.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.153 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.207 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.207 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.207 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0456e-05 s, 0.0 kB/s 2023-10-08T16:07:08.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.208 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.263 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2282e-05 s, 0.0 kB/s 2023-10-08T16:07:08.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.263 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.318 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.318 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.319 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0818e-05 s, 0.0 kB/s 2023-10-08T16:07:08.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.319 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3715e-05 s, 0.0 kB/s 2023-10-08T16:07:08.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.375 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.431 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0425e-05 s, 0.0 kB/s 2023-10-08T16:07:08.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.432 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3985e-05 s, 0.0 kB/s 2023-10-08T16:07:08.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.489 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.544 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6675e-05 s, 0.0 kB/s 2023-10-08T16:07:08.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.545 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.599 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.599 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.599 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8748e-05 s, 0.0 kB/s 2023-10-08T16:07:08.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.600 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.657 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5362e-05 s, 0.0 kB/s 2023-10-08T16:07:08.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.658 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.714 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1725e-05 s, 0.0 kB/s 2023-10-08T16:07:08.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.770 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-10-08T16:07:08.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.826 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2155e-05 s, 0.0 kB/s 2023-10-08T16:07:08.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.826 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1139e-05 s, 0.0 kB/s 2023-10-08T16:07:08.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.882 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.936 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3911e-05 s, 0.0 kB/s 2023-10-08T16:07:08.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.937 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:08.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:08.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:08.993 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9067e-05 s, 0.0 kB/s 2023-10-08T16:07:08.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:08.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.048 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8345e-05 s, 0.0 kB/s 2023-10-08T16:07:09.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.049 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.104 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2348e-05 s, 0.0 kB/s 2023-10-08T16:07:09.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0537e-05 s, 0.0 kB/s 2023-10-08T16:07:09.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9355e-05 s, 0.0 kB/s 2023-10-08T16:07:09.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.271 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2508e-05 s, 0.0 kB/s 2023-10-08T16:07:09.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.271 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9846e-05 s, 0.0 kB/s 2023-10-08T16:07:09.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.382 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7721e-05 s, 0.0 kB/s 2023-10-08T16:07:09.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.383 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9199e-05 s, 0.0 kB/s 2023-10-08T16:07:09.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2255e-05 s, 0.0 kB/s 2023-10-08T16:07:09.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8887e-05 s, 0.0 kB/s 2023-10-08T16:07:09.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9982e-05 s, 0.0 kB/s 2023-10-08T16:07:09.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.661 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0451e-05 s, 0.0 kB/s 2023-10-08T16:07:09.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.716 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.716 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.716 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8316e-05 s, 0.0 kB/s 2023-10-08T16:07:09.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.717 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.771 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0239e-05 s, 0.0 kB/s 2023-10-08T16:07:09.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.772 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.827 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0113e-05 s, 0.0 kB/s 2023-10-08T16:07:09.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.828 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.882 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5211e-05 s, 0.0 kB/s 2023-10-08T16:07:09.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.883 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.938 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1828e-05 s, 0.0 kB/s 2023-10-08T16:07:09.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.938 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:09.994 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9808e-05 s, 0.0 kB/s 2023-10-08T16:07:09.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:09.995 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0159e-05 s, 0.0 kB/s 2023-10-08T16:07:10.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.050 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8008e-05 s, 0.0 kB/s 2023-10-08T16:07:10.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.160 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.644e-05 s, 0.0 kB/s 2023-10-08T16:07:10.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.161 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.207e-05 s, 0.0 kB/s 2023-10-08T16:07:10.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.270 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1058e-05 s, 0.0 kB/s 2023-10-08T16:07:10.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.271 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.326 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.326 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.326 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.92e-05 s, 0.0 kB/s 2023-10-08T16:07:10.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.327 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.381 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.381 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.382 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1389e-05 s, 0.0 kB/s 2023-10-08T16:07:10.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.382 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.437 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.437 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.437 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0189e-05 s, 0.0 kB/s 2023-10-08T16:07:10.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.438 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.493 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9666e-05 s, 0.0 kB/s 2023-10-08T16:07:10.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.494 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2483e-05 s, 0.0 kB/s 2023-10-08T16:07:10.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7162e-05 s, 0.0 kB/s 2023-10-08T16:07:10.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.606 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.661 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8241e-05 s, 0.0 kB/s 2023-10-08T16:07:10.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.662 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6755e-05 s, 0.0 kB/s 2023-10-08T16:07:10.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.719 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.773 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0506e-05 s, 0.0 kB/s 2023-10-08T16:07:10.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.774 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9389e-05 s, 0.0 kB/s 2023-10-08T16:07:10.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.830 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1996e-05 s, 0.0 kB/s 2023-10-08T16:07:10.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.940 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0178e-05 s, 0.0 kB/s 2023-10-08T16:07:10.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:10.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:10.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:10.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8865e-05 s, 0.0 kB/s 2023-10-08T16:07:10.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:10.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.052 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0382e-05 s, 0.0 kB/s 2023-10-08T16:07:11.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.053 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.107 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0131e-05 s, 0.0 kB/s 2023-10-08T16:07:11.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.163 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0344e-05 s, 0.0 kB/s 2023-10-08T16:07:11.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.164 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9458e-05 s, 0.0 kB/s 2023-10-08T16:07:11.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.219 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.275 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8013e-05 s, 0.0 kB/s 2023-10-08T16:07:11.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.275 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.330 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.331 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.4711e-05 s, 0.0 kB/s 2023-10-08T16:07:11.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.331 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.385 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0186e-05 s, 0.0 kB/s 2023-10-08T16:07:11.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.387 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.441 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.442 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5421e-05 s, 0.0 kB/s 2023-10-08T16:07:11.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.443 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.498 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9608e-05 s, 0.0 kB/s 2023-10-08T16:07:11.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.499 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.555 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.555 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.555 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7154e-05 s, 0.0 kB/s 2023-10-08T16:07:11.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.556 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.610 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6677e-05 s, 0.0 kB/s 2023-10-08T16:07:11.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.667 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6345e-05 s, 0.0 kB/s 2023-10-08T16:07:11.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.667 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9579e-05 s, 0.0 kB/s 2023-10-08T16:07:11.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.723 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.778 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8565e-05 s, 0.0 kB/s 2023-10-08T16:07:11.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.779 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.834 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8881e-05 s, 0.0 kB/s 2023-10-08T16:07:11.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.835 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.891 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2098e-05 s, 0.0 kB/s 2023-10-08T16:07:11.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.891 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:11.946 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:11.947 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6016e-05 s, 0.0 kB/s 2023-10-08T16:07:11.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:11.947 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.003 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.003 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.003 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0639e-05 s, 0.0 kB/s 2023-10-08T16:07:12.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.004 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.059 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.059 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.059 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1034e-05 s, 0.0 kB/s 2023-10-08T16:07:12.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.059 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.114 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1957e-05 s, 0.0 kB/s 2023-10-08T16:07:12.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.115 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.172 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8507e-05 s, 0.0 kB/s 2023-10-08T16:07:12.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.172 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.227 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.227 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.227 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8818e-05 s, 0.0 kB/s 2023-10-08T16:07:12.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.228 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.284 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0196e-05 s, 0.0 kB/s 2023-10-08T16:07:12.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.284 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.339 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7211e-05 s, 0.0 kB/s 2023-10-08T16:07:12.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.340 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3831e-05 s, 0.0 kB/s 2023-10-08T16:07:12.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.452 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.401e-05 s, 0.0 kB/s 2023-10-08T16:07:12.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.452 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.508 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9549e-05 s, 0.0 kB/s 2023-10-08T16:07:12.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.509 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.564 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.466e-05 s, 0.0 kB/s 2023-10-08T16:07:12.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.565 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.621 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6841e-05 s, 0.0 kB/s 2023-10-08T16:07:12.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.621 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.676 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.677 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2965e-05 s, 0.0 kB/s 2023-10-08T16:07:12.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.677 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.732 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7141e-05 s, 0.0 kB/s 2023-10-08T16:07:12.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.733 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.788 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.788 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.788 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1876e-05 s, 0.0 kB/s 2023-10-08T16:07:12.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.789 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.844 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.878e-05 s, 0.0 kB/s 2023-10-08T16:07:12.845 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.845 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1612e-05 s, 0.0 kB/s 2023-10-08T16:07:12.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.900 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:12.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0922e-05 s, 0.0 kB/s 2023-10-08T16:07:12.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:12.956 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.012 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.012 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8891e-05 s, 0.0 kB/s 2023-10-08T16:07:13.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.013 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.068 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0234e-05 s, 0.0 kB/s 2023-10-08T16:07:13.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.069 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8281e-05 s, 0.0 kB/s 2023-10-08T16:07:13.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.179 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8284e-05 s, 0.0 kB/s 2023-10-08T16:07:13.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.180 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:07:13.222 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:07:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.237 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2625e-05 s, 0.0 kB/s 2023-10-08T16:07:13.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.237 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4159e-05 s, 0.0 kB/s 2023-10-08T16:07:13.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.294 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.355 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.355 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0669e-05 s, 0.0 kB/s 2023-10-08T16:07:13.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.412 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9956e-05 s, 0.0 kB/s 2023-10-08T16:07:13.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.413 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.468 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1258e-05 s, 0.0 kB/s 2023-10-08T16:07:13.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.468 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.524 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000100334 s, 0.0 kB/s 2023-10-08T16:07:13.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.580 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9855e-05 s, 0.0 kB/s 2023-10-08T16:07:13.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.580 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6772e-05 s, 0.0 kB/s 2023-10-08T16:07:13.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.636 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.691 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0973e-05 s, 0.0 kB/s 2023-10-08T16:07:13.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.692 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8861e-05 s, 0.0 kB/s 2023-10-08T16:07:13.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.748 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.805 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.805 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0535e-05 s, 0.0 kB/s 2023-10-08T16:07:13.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.805 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.860 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1655e-05 s, 0.0 kB/s 2023-10-08T16:07:13.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.861 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0122e-05 s, 0.0 kB/s 2023-10-08T16:07:13.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:13.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:13.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:13.971 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9639e-05 s, 0.0 kB/s 2023-10-08T16:07:13.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:13.972 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.028 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.038e-05 s, 0.0 kB/s 2023-10-08T16:07:14.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.029 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.084 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4765e-05 s, 0.0 kB/s 2023-10-08T16:07:14.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.084 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.139 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.139 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.139 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4811e-05 s, 0.0 kB/s 2023-10-08T16:07:14.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.140 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.196 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0352e-05 s, 0.0 kB/s 2023-10-08T16:07:14.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.196 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.252 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.252 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.252 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1804e-05 s, 0.0 kB/s 2023-10-08T16:07:14.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.252 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.308 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.308 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.308 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8983e-05 s, 0.0 kB/s 2023-10-08T16:07:14.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.308 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.364 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5995e-05 s, 0.0 kB/s 2023-10-08T16:07:14.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.365 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1185e-05 s, 0.0 kB/s 2023-10-08T16:07:14.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.475 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9001e-05 s, 0.0 kB/s 2023-10-08T16:07:14.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.475 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.531 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0057e-05 s, 0.0 kB/s 2023-10-08T16:07:14.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.531 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0132e-05 s, 0.0 kB/s 2023-10-08T16:07:14.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.977e-05 s, 0.0 kB/s 2023-10-08T16:07:14.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.644 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-10-08T16:07:14.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.700 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8664e-05 s, 0.0 kB/s 2023-10-08T16:07:14.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.756 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1696e-05 s, 0.0 kB/s 2023-10-08T16:07:14.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0135e-05 s, 0.0 kB/s 2023-10-08T16:07:14.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8612e-05 s, 0.0 kB/s 2023-10-08T16:07:14.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0514e-05 s, 0.0 kB/s 2023-10-08T16:07:14.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:14.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0478e-05 s, 0.0 kB/s 2023-10-08T16:07:15.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.036 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3198e-05 s, 0.0 kB/s 2023-10-08T16:07:15.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.092 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.147 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9471e-05 s, 0.0 kB/s 2023-10-08T16:07:15.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.148 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.204 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.204 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.204 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1038e-05 s, 0.0 kB/s 2023-10-08T16:07:15.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.204 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.260 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.261 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1042e-05 s, 0.0 kB/s 2023-10-08T16:07:15.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.261 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.317 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1762e-05 s, 0.0 kB/s 2023-10-08T16:07:15.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.317 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.372 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9332e-05 s, 0.0 kB/s 2023-10-08T16:07:15.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.428 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.429 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5538e-05 s, 0.0 kB/s 2023-10-08T16:07:15.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.430 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2098e-05 s, 0.0 kB/s 2023-10-08T16:07:15.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.486 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0909e-05 s, 0.0 kB/s 2023-10-08T16:07:15.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.542 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.598 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3112e-05 s, 0.0 kB/s 2023-10-08T16:07:15.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.598 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8376e-05 s, 0.0 kB/s 2023-10-08T16:07:15.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.655 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.710 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.710 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.710 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.4812e-05 s, 0.0 kB/s 2023-10-08T16:07:15.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.711 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.765 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9885e-05 s, 0.0 kB/s 2023-10-08T16:07:15.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.766 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.822 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.822 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.822 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1214e-05 s, 0.0 kB/s 2023-10-08T16:07:15.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.822 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.878 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1605e-05 s, 0.0 kB/s 2023-10-08T16:07:15.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.878 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.934 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-08T16:07:15.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.934 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:15.989 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:15.989 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:15.989 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8002e-05 s, 0.0 kB/s 2023-10-08T16:07:15.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:15.990 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5629e-05 s, 0.0 kB/s 2023-10-08T16:07:16.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0122e-05 s, 0.0 kB/s 2023-10-08T16:07:16.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9684e-05 s, 0.0 kB/s 2023-10-08T16:07:16.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0194e-05 s, 0.0 kB/s 2023-10-08T16:07:16.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8774e-05 s, 0.0 kB/s 2023-10-08T16:07:16.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.328 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1022e-05 s, 0.0 kB/s 2023-10-08T16:07:16.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1649e-05 s, 0.0 kB/s 2023-10-08T16:07:16.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.439 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.439 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8174e-05 s, 0.0 kB/s 2023-10-08T16:07:16.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5261e-05 s, 0.0 kB/s 2023-10-08T16:07:16.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.550 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9212e-05 s, 0.0 kB/s 2023-10-08T16:07:16.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.551 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.606 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1147e-05 s, 0.0 kB/s 2023-10-08T16:07:16.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.663 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9245e-05 s, 0.0 kB/s 2023-10-08T16:07:16.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.663 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.719 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.9523e-05 s, 0.0 kB/s 2023-10-08T16:07:16.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.719 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.791 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1829e-05 s, 0.0 kB/s 2023-10-08T16:07:16.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6923e-05 s, 0.0 kB/s 2023-10-08T16:07:16.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.830 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.886 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8936e-05 s, 0.0 kB/s 2023-10-08T16:07:16.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:16.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:16.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0116e-05 s, 0.0 kB/s 2023-10-08T16:07:16.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:16.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:16.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.007 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.008 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9822e-05 s, 0.0 kB/s 2023-10-08T16:07:17.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.008 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7788e-05 s, 0.0 kB/s 2023-10-08T16:07:17.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.118 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1212e-05 s, 0.0 kB/s 2023-10-08T16:07:17.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.119 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.220 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.220 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1113e-05 s, 0.0 kB/s 2023-10-08T16:07:17.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.220 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8841e-05 s, 0.0 kB/s 2023-10-08T16:07:17.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.269 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2603e-05 s, 0.0 kB/s 2023-10-08T16:07:17.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.292 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.352 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6708e-05 s, 0.0 kB/s 2023-10-08T16:07:17.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.352 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.411 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3638e-05 s, 0.0 kB/s 2023-10-08T16:07:17.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.411 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.8032e-05 s, 0.0 kB/s 2023-10-08T16:07:17.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.460 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.515 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7058e-05 s, 0.0 kB/s 2023-10-08T16:07:17.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.515 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.203e-05 s, 0.0 kB/s 2023-10-08T16:07:17.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.1343e-05 s, 0.0 kB/s 2023-10-08T16:07:17.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.626 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.680 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8071e-05 s, 0.0 kB/s 2023-10-08T16:07:17.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.681 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.735 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8622e-05 s, 0.0 kB/s 2023-10-08T16:07:17.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.736 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0481e-05 s, 0.0 kB/s 2023-10-08T16:07:17.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.847 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1805e-05 s, 0.0 kB/s 2023-10-08T16:07:17.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.847 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.902 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.902 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.902 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1742e-05 s, 0.0 kB/s 2023-10-08T16:07:17.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.902 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:17.957 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:17.957 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:17.957 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7685e-05 s, 0.0 kB/s 2023-10-08T16:07:17.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:17.958 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.013 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0527e-05 s, 0.0 kB/s 2023-10-08T16:07:18.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.014 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.070 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.85e-05 s, 0.0 kB/s 2023-10-08T16:07:18.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.071 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8318e-05 s, 0.0 kB/s 2023-10-08T16:07:18.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.126 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7279e-05 s, 0.0 kB/s 2023-10-08T16:07:18.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.181 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8479e-05 s, 0.0 kB/s 2023-10-08T16:07:18.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9108e-05 s, 0.0 kB/s 2023-10-08T16:07:18.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.292 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.347 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0528e-05 s, 0.0 kB/s 2023-10-08T16:07:18.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.348 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0326e-05 s, 0.0 kB/s 2023-10-08T16:07:18.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.403 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.457 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8375e-05 s, 0.0 kB/s 2023-10-08T16:07:18.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.457 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.512 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4802e-05 s, 0.0 kB/s 2023-10-08T16:07:18.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0692e-05 s, 0.0 kB/s 2023-10-08T16:07:18.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.623 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.624 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9593e-05 s, 0.0 kB/s 2023-10-08T16:07:18.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.681 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9718e-05 s, 0.0 kB/s 2023-10-08T16:07:18.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.681 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.737 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.737 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.737 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6553e-05 s, 0.0 kB/s 2023-10-08T16:07:18.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.737 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9902e-05 s, 0.0 kB/s 2023-10-08T16:07:18.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.793 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9901e-05 s, 0.0 kB/s 2023-10-08T16:07:18.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.847 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.902 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.902 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.903 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0846e-05 s, 0.0 kB/s 2023-10-08T16:07:18.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.903 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:18.957 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:18.958 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:18.958 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3261e-05 s, 0.0 kB/s 2023-10-08T16:07:18.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:18.959 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.014 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0911e-05 s, 0.0 kB/s 2023-10-08T16:07:19.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.015 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.070 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8987e-05 s, 0.0 kB/s 2023-10-08T16:07:19.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.070 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.126 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0177e-05 s, 0.0 kB/s 2023-10-08T16:07:19.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.127 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.182 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9444e-05 s, 0.0 kB/s 2023-10-08T16:07:19.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.182 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.238 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0288e-05 s, 0.0 kB/s 2023-10-08T16:07:19.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.239 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.293 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9349e-05 s, 0.0 kB/s 2023-10-08T16:07:19.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.294 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.348 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.097e-05 s, 0.0 kB/s 2023-10-08T16:07:19.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.349 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.404 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4605e-05 s, 0.0 kB/s 2023-10-08T16:07:19.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.404 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0016e-05 s, 0.0 kB/s 2023-10-08T16:07:19.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.460 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.515 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0915e-05 s, 0.0 kB/s 2023-10-08T16:07:19.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.515 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3e-05 s, 0.0 kB/s 2023-10-08T16:07:19.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0526e-05 s, 0.0 kB/s 2023-10-08T16:07:19.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.628 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0294e-05 s, 0.0 kB/s 2023-10-08T16:07:19.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.768 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6754e-05 s, 0.0 kB/s 2023-10-08T16:07:19.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.823 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3762e-05 s, 0.0 kB/s 2023-10-08T16:07:19.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.824 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.879 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3145e-05 s, 0.0 kB/s 2023-10-08T16:07:19.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.880 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7971e-05 s, 0.0 kB/s 2023-10-08T16:07:19.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.935 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1862e-05 s, 0.0 kB/s 2023-10-08T16:07:19.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:19.991 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3345e-05 s, 0.0 kB/s 2023-10-08T16:07:20.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.102 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9008e-05 s, 0.0 kB/s 2023-10-08T16:07:20.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.159 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7075e-05 s, 0.0 kB/s 2023-10-08T16:07:20.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.986e-05 s, 0.0 kB/s 2023-10-08T16:07:20.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8461e-05 s, 0.0 kB/s 2023-10-08T16:07:20.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1328e-05 s, 0.0 kB/s 2023-10-08T16:07:20.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.382 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0163e-05 s, 0.0 kB/s 2023-10-08T16:07:20.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.383 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.438 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.984e-05 s, 0.0 kB/s 2023-10-08T16:07:20.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8965e-05 s, 0.0 kB/s 2023-10-08T16:07:20.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.551 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8359e-05 s, 0.0 kB/s 2023-10-08T16:07:20.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.551 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.607 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9368e-05 s, 0.0 kB/s 2023-10-08T16:07:20.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.662 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4858e-05 s, 0.0 kB/s 2023-10-08T16:07:20.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.663 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.719 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5012e-05 s, 0.0 kB/s 2023-10-08T16:07:20.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.720 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.775 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8206e-05 s, 0.0 kB/s 2023-10-08T16:07:20.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.775 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8673e-05 s, 0.0 kB/s 2023-10-08T16:07:20.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.885 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0853e-05 s, 0.0 kB/s 2023-10-08T16:07:20.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6666e-05 s, 0.0 kB/s 2023-10-08T16:07:20.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:20.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:20.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:20.997 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0542e-05 s, 0.0 kB/s 2023-10-08T16:07:20.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:20.998 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.054 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9e-05 s, 0.0 kB/s 2023-10-08T16:07:21.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.054 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8653e-05 s, 0.0 kB/s 2023-10-08T16:07:21.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.110 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.165 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7047e-05 s, 0.0 kB/s 2023-10-08T16:07:21.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.166 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.222 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-08T16:07:21.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.222 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.278 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1376e-05 s, 0.0 kB/s 2023-10-08T16:07:21.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.333 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.334 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.334 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.826e-05 s, 0.0 kB/s 2023-10-08T16:07:21.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.334 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.390 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6778e-05 s, 0.0 kB/s 2023-10-08T16:07:21.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.390 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.445 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9977e-05 s, 0.0 kB/s 2023-10-08T16:07:21.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.446 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.501 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.501 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.502 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6521e-05 s, 0.0 kB/s 2023-10-08T16:07:21.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.502 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.557 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.558 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000131459 s, 0.0 kB/s 2023-10-08T16:07:21.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.558 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.613 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.613 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.613 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.107e-05 s, 0.0 kB/s 2023-10-08T16:07:21.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.614 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.669 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.670 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.670 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4208e-05 s, 0.0 kB/s 2023-10-08T16:07:21.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.670 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.726 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0125e-05 s, 0.0 kB/s 2023-10-08T16:07:21.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.781 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.781 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.781 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1108e-05 s, 0.0 kB/s 2023-10-08T16:07:21.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.782 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.836 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.96e-05 s, 0.0 kB/s 2023-10-08T16:07:21.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.837 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.890 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.891 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9342e-05 s, 0.0 kB/s 2023-10-08T16:07:21.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.892 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:21.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:21.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:21.947 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3549e-05 s, 0.0 kB/s 2023-10-08T16:07:21.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:21.948 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.003 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.003 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.003 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2799e-05 s, 0.0 kB/s 2023-10-08T16:07:22.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.003 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.059 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7514e-05 s, 0.0 kB/s 2023-10-08T16:07:22.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.059 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7714e-05 s, 0.0 kB/s 2023-10-08T16:07:22.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.115 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5531e-05 s, 0.0 kB/s 2023-10-08T16:07:22.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.172 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.228 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1151e-05 s, 0.0 kB/s 2023-10-08T16:07:22.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.228 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.283 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.284 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3165e-05 s, 0.0 kB/s 2023-10-08T16:07:22.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.284 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.339 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.339 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.186e-05 s, 0.0 kB/s 2023-10-08T16:07:22.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.340 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8522e-05 s, 0.0 kB/s 2023-10-08T16:07:22.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.395 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.450 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2865e-05 s, 0.0 kB/s 2023-10-08T16:07:22.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.450 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.506 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1341e-05 s, 0.0 kB/s 2023-10-08T16:07:22.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.507 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.562 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1837e-05 s, 0.0 kB/s 2023-10-08T16:07:22.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.563 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.617 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.15e-05 s, 0.0 kB/s 2023-10-08T16:07:22.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.617 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7931e-05 s, 0.0 kB/s 2023-10-08T16:07:22.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.673 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.728 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.728 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.728 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3642e-05 s, 0.0 kB/s 2023-10-08T16:07:22.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.728 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.783 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9996e-05 s, 0.0 kB/s 2023-10-08T16:07:22.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.784 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.839 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3805e-05 s, 0.0 kB/s 2023-10-08T16:07:22.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.839 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7778e-05 s, 0.0 kB/s 2023-10-08T16:07:22.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.895 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9921e-05 s, 0.0 kB/s 2023-10-08T16:07:22.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:22.951 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.005 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.006 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6831e-05 s, 0.0 kB/s 2023-10-08T16:07:23.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.006 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.061 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8735e-05 s, 0.0 kB/s 2023-10-08T16:07:23.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.062 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.117 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.157e-05 s, 0.0 kB/s 2023-10-08T16:07:23.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.118 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.174 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.029e-05 s, 0.0 kB/s 2023-10-08T16:07:23.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.175 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7548e-05 s, 0.0 kB/s 2023-10-08T16:07:23.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.230 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.285 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.285 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.285 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2315e-05 s, 0.0 kB/s 2023-10-08T16:07:23.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.286 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.340 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0552e-05 s, 0.0 kB/s 2023-10-08T16:07:23.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.341 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4908e-05 s, 0.0 kB/s 2023-10-08T16:07:23.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.451 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0641e-05 s, 0.0 kB/s 2023-10-08T16:07:23.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.452 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.507 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5992e-05 s, 0.0 kB/s 2023-10-08T16:07:23.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.508 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.563 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.563 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1195e-05 s, 0.0 kB/s 2023-10-08T16:07:23.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.564 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2928e-05 s, 0.0 kB/s 2023-10-08T16:07:23.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.620 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.676 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8488e-05 s, 0.0 kB/s 2023-10-08T16:07:23.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.676 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.732 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5333e-05 s, 0.0 kB/s 2023-10-08T16:07:23.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.732 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1835e-05 s, 0.0 kB/s 2023-10-08T16:07:23.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7808e-05 s, 0.0 kB/s 2023-10-08T16:07:23.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.898 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5322e-05 s, 0.0 kB/s 2023-10-08T16:07:23.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:23.953 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:23.954 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:23.954 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9311e-05 s, 0.0 kB/s 2023-10-08T16:07:23.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:23.955 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.010 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8934e-05 s, 0.0 kB/s 2023-10-08T16:07:24.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.010 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.065 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2278e-05 s, 0.0 kB/s 2023-10-08T16:07:24.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.066 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.121 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9632e-05 s, 0.0 kB/s 2023-10-08T16:07:24.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.122 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.178 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8712e-05 s, 0.0 kB/s 2023-10-08T16:07:24.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.178 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.233 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.233 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.233 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1248e-05 s, 0.0 kB/s 2023-10-08T16:07:24.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.234 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.288 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2491e-05 s, 0.0 kB/s 2023-10-08T16:07:24.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.289 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.343 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.344 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.183e-05 s, 0.0 kB/s 2023-10-08T16:07:24.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.345 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.401 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9176e-05 s, 0.0 kB/s 2023-10-08T16:07:24.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.457 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0734e-05 s, 0.0 kB/s 2023-10-08T16:07:24.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.457 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.513 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9559e-05 s, 0.0 kB/s 2023-10-08T16:07:24.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.568 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3563e-05 s, 0.0 kB/s 2023-10-08T16:07:24.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.624 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8916e-05 s, 0.0 kB/s 2023-10-08T16:07:24.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.624 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.679 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2738e-05 s, 0.0 kB/s 2023-10-08T16:07:24.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.680 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.736 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2259e-05 s, 0.0 kB/s 2023-10-08T16:07:24.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.736 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0699e-05 s, 0.0 kB/s 2023-10-08T16:07:24.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7207e-05 s, 0.0 kB/s 2023-10-08T16:07:24.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.848 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1863e-05 s, 0.0 kB/s 2023-10-08T16:07:24.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.904 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.923e-05 s, 0.0 kB/s 2023-10-08T16:07:24.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:24.959 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.013 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0691e-05 s, 0.0 kB/s 2023-10-08T16:07:25.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.014 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.068 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6705e-05 s, 0.0 kB/s 2023-10-08T16:07:25.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.069 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.124 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.124 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.124 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7537e-05 s, 0.0 kB/s 2023-10-08T16:07:25.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.125 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8695e-05 s, 0.0 kB/s 2023-10-08T16:07:25.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.180 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8937e-05 s, 0.0 kB/s 2023-10-08T16:07:25.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8118e-05 s, 0.0 kB/s 2023-10-08T16:07:25.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.293 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.347 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2703e-05 s, 0.0 kB/s 2023-10-08T16:07:25.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.348 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1961e-05 s, 0.0 kB/s 2023-10-08T16:07:25.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.402 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.458 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5681e-05 s, 0.0 kB/s 2023-10-08T16:07:25.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.458 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.513 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7548e-05 s, 0.0 kB/s 2023-10-08T16:07:25.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.514 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.569 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.77e-05 s, 0.0 kB/s 2023-10-08T16:07:25.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0249e-05 s, 0.0 kB/s 2023-10-08T16:07:25.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.679 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9341e-05 s, 0.0 kB/s 2023-10-08T16:07:25.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.680 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.737 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.737 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6279e-05 s, 0.0 kB/s 2023-10-08T16:07:25.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.737 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4832e-05 s, 0.0 kB/s 2023-10-08T16:07:25.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.793 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.848 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8497e-05 s, 0.0 kB/s 2023-10-08T16:07:25.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0045e-05 s, 0.0 kB/s 2023-10-08T16:07:25.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.905 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:25.959 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:25.959 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:25.959 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6313e-05 s, 0.0 kB/s 2023-10-08T16:07:25.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:25.960 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.015 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.015 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.015 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.072e-05 s, 0.0 kB/s 2023-10-08T16:07:26.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.016 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.071 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6609e-05 s, 0.0 kB/s 2023-10-08T16:07:26.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.111 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.127 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.127 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.127 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.041e-05 s, 0.0 kB/s 2023-10-08T16:07:26.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.128 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.183 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8692e-05 s, 0.0 kB/s 2023-10-08T16:07:26.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.183 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.239 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9794e-05 s, 0.0 kB/s 2023-10-08T16:07:26.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.239 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.295 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.295 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.295 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2592e-05 s, 0.0 kB/s 2023-10-08T16:07:26.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.295 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.351 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7784e-05 s, 0.0 kB/s 2023-10-08T16:07:26.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.351 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.407 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9444e-05 s, 0.0 kB/s 2023-10-08T16:07:26.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.407 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.462 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.463 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.463 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9884e-05 s, 0.0 kB/s 2023-10-08T16:07:26.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.463 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.518 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.518 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.518 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7688e-05 s, 0.0 kB/s 2023-10-08T16:07:26.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.519 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.574 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8524e-05 s, 0.0 kB/s 2023-10-08T16:07:26.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.574 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.629 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.629 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.629 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8967e-05 s, 0.0 kB/s 2023-10-08T16:07:26.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.630 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.684 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.684 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.684 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4803e-05 s, 0.0 kB/s 2023-10-08T16:07:26.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.685 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.739 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9896e-05 s, 0.0 kB/s 2023-10-08T16:07:26.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.740 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.796 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1781e-05 s, 0.0 kB/s 2023-10-08T16:07:26.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.796 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.852 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.852 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9884e-05 s, 0.0 kB/s 2023-10-08T16:07:26.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.852 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.907 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.908 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9769e-05 s, 0.0 kB/s 2023-10-08T16:07:26.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.908 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:26.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:26.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:26.963 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0451e-05 s, 0.0 kB/s 2023-10-08T16:07:26.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:26.964 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.020 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9865e-05 s, 0.0 kB/s 2023-10-08T16:07:27.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.020 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.075 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9182e-05 s, 0.0 kB/s 2023-10-08T16:07:27.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.076 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.130 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1159e-05 s, 0.0 kB/s 2023-10-08T16:07:27.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.131 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8249e-05 s, 0.0 kB/s 2023-10-08T16:07:27.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.187 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7789e-05 s, 0.0 kB/s 2023-10-08T16:07:27.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.243 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.299 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.299 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0777e-05 s, 0.0 kB/s 2023-10-08T16:07:27.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.299 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.354 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3293e-05 s, 0.0 kB/s 2023-10-08T16:07:27.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.355 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.409 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2045e-05 s, 0.0 kB/s 2023-10-08T16:07:27.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.410 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.466 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.2579e-05 s, 0.0 kB/s 2023-10-08T16:07:27.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.467 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.523 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2537e-05 s, 0.0 kB/s 2023-10-08T16:07:27.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.523 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2272e-05 s, 0.0 kB/s 2023-10-08T16:07:27.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0051e-05 s, 0.0 kB/s 2023-10-08T16:07:27.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.635 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.690 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.690 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.690 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0232e-05 s, 0.0 kB/s 2023-10-08T16:07:27.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.691 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.748 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7711e-05 s, 0.0 kB/s 2023-10-08T16:07:27.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.748 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.803 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0462e-05 s, 0.0 kB/s 2023-10-08T16:07:27.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.803 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.858 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3741e-05 s, 0.0 kB/s 2023-10-08T16:07:27.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.859 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.915 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.915 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.915 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2345e-05 s, 0.0 kB/s 2023-10-08T16:07:27.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.916 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:27.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:27.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:27.971 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6728e-05 s, 0.0 kB/s 2023-10-08T16:07:27.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:27.971 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.027 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3161e-05 s, 0.0 kB/s 2023-10-08T16:07:28.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.027 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.082 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8518e-05 s, 0.0 kB/s 2023-10-08T16:07:28.085 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.085 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.137 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.137 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.138 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6011e-05 s, 0.0 kB/s 2023-10-08T16:07:28.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.138 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.193 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3374e-05 s, 0.0 kB/s 2023-10-08T16:07:28.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.194 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1675e-05 s, 0.0 kB/s 2023-10-08T16:07:28.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.250 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.304 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000100361 s, 0.0 kB/s 2023-10-08T16:07:28.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.305 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.360 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9358e-05 s, 0.0 kB/s 2023-10-08T16:07:28.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.361 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8491e-05 s, 0.0 kB/s 2023-10-08T16:07:28.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.417 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.472 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.473 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.473 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2172e-05 s, 0.0 kB/s 2023-10-08T16:07:28.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.474 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.530 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8343e-05 s, 0.0 kB/s 2023-10-08T16:07:28.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.530 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2016e-05 s, 0.0 kB/s 2023-10-08T16:07:28.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.642 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.642 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.642 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9141e-05 s, 0.0 kB/s 2023-10-08T16:07:28.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2133e-05 s, 0.0 kB/s 2023-10-08T16:07:28.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.699 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.754 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3373e-05 s, 0.0 kB/s 2023-10-08T16:07:28.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.755 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.810 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4354e-05 s, 0.0 kB/s 2023-10-08T16:07:28.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.866 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3335e-05 s, 0.0 kB/s 2023-10-08T16:07:28.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.922 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0575e-05 s, 0.0 kB/s 2023-10-08T16:07:28.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:28.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:28.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:28.978 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9216e-05 s, 0.0 kB/s 2023-10-08T16:07:28.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:28.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.034 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1068e-05 s, 0.0 kB/s 2023-10-08T16:07:29.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.035 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.089 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9998e-05 s, 0.0 kB/s 2023-10-08T16:07:29.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.090 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.145 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8804e-05 s, 0.0 kB/s 2023-10-08T16:07:29.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.146 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.201 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.202 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9679e-05 s, 0.0 kB/s 2023-10-08T16:07:29.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.202 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.257 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0083e-05 s, 0.0 kB/s 2023-10-08T16:07:29.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.312 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.313 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3769e-05 s, 0.0 kB/s 2023-10-08T16:07:29.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.313 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.367 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.367 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.367 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.888e-05 s, 0.0 kB/s 2023-10-08T16:07:29.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.368 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.422 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6921e-05 s, 0.0 kB/s 2023-10-08T16:07:29.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.423 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1261e-05 s, 0.0 kB/s 2023-10-08T16:07:29.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.479 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.533 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.533 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.533 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8511e-05 s, 0.0 kB/s 2023-10-08T16:07:29.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.588 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9488e-05 s, 0.0 kB/s 2023-10-08T16:07:29.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.589 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.645 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.645 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1459e-05 s, 0.0 kB/s 2023-10-08T16:07:29.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8995e-05 s, 0.0 kB/s 2023-10-08T16:07:29.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.701 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.757 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8499e-05 s, 0.0 kB/s 2023-10-08T16:07:29.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.757 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.812 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0795e-05 s, 0.0 kB/s 2023-10-08T16:07:29.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.813 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0765e-05 s, 0.0 kB/s 2023-10-08T16:07:29.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.868 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0081e-05 s, 0.0 kB/s 2023-10-08T16:07:29.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:29.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:29.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:29.980 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1074e-05 s, 0.0 kB/s 2023-10-08T16:07:29.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:29.981 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.036 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.036 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.036 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1476e-05 s, 0.0 kB/s 2023-10-08T16:07:30.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.036 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8617e-05 s, 0.0 kB/s 2023-10-08T16:07:30.092 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.092 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.146 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3972e-05 s, 0.0 kB/s 2023-10-08T16:07:30.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.147 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0333e-05 s, 0.0 kB/s 2023-10-08T16:07:30.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.259 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8934e-05 s, 0.0 kB/s 2023-10-08T16:07:30.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.316 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5495e-05 s, 0.0 kB/s 2023-10-08T16:07:30.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.317 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5175e-05 s, 0.0 kB/s 2023-10-08T16:07:30.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.372 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.428 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1565e-05 s, 0.0 kB/s 2023-10-08T16:07:30.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.429 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.4154e-05 s, 0.0 kB/s 2023-10-08T16:07:30.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.485 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9652e-05 s, 0.0 kB/s 2023-10-08T16:07:30.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.596 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8746e-05 s, 0.0 kB/s 2023-10-08T16:07:30.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9568e-05 s, 0.0 kB/s 2023-10-08T16:07:30.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.652 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.708 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9806e-05 s, 0.0 kB/s 2023-10-08T16:07:30.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.709 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4769e-05 s, 0.0 kB/s 2023-10-08T16:07:30.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.766 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-08T16:07:30.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.821 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.594e-05 s, 0.0 kB/s 2023-10-08T16:07:30.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.933 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3058e-05 s, 0.0 kB/s 2023-10-08T16:07:30.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.934 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:30.989 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9772e-05 s, 0.0 kB/s 2023-10-08T16:07:30.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:30.990 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.046 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9327e-05 s, 0.0 kB/s 2023-10-08T16:07:31.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.046 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.102 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2215e-05 s, 0.0 kB/s 2023-10-08T16:07:31.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.102 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0925e-05 s, 0.0 kB/s 2023-10-08T16:07:31.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.158 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5325e-05 s, 0.0 kB/s 2023-10-08T16:07:31.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0866e-05 s, 0.0 kB/s 2023-10-08T16:07:31.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.271 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0212e-05 s, 0.0 kB/s 2023-10-08T16:07:31.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.327 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0788e-05 s, 0.0 kB/s 2023-10-08T16:07:31.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.439 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9961e-05 s, 0.0 kB/s 2023-10-08T16:07:31.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.495 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-10-08T16:07:31.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.551 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1432e-05 s, 0.0 kB/s 2023-10-08T16:07:31.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.551 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.607 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8951e-05 s, 0.0 kB/s 2023-10-08T16:07:31.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.663 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2622e-05 s, 0.0 kB/s 2023-10-08T16:07:31.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.663 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.719 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2436e-05 s, 0.0 kB/s 2023-10-08T16:07:31.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.719 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.775 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.071e-05 s, 0.0 kB/s 2023-10-08T16:07:31.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.775 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9539e-05 s, 0.0 kB/s 2023-10-08T16:07:31.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.886 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9093e-05 s, 0.0 kB/s 2023-10-08T16:07:31.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0804e-05 s, 0.0 kB/s 2023-10-08T16:07:31.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:31.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:31.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:31.997 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1571e-05 s, 0.0 kB/s 2023-10-08T16:07:31.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:31.998 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.053 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5855e-05 s, 0.0 kB/s 2023-10-08T16:07:32.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.054 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0032e-05 s, 0.0 kB/s 2023-10-08T16:07:32.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.110 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2498e-05 s, 0.0 kB/s 2023-10-08T16:07:32.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.222 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.223 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.223 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9755e-05 s, 0.0 kB/s 2023-10-08T16:07:32.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.223 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0605e-05 s, 0.0 kB/s 2023-10-08T16:07:32.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.335 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0828e-05 s, 0.0 kB/s 2023-10-08T16:07:32.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9185e-05 s, 0.0 kB/s 2023-10-08T16:07:32.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.392 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.447 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8901e-05 s, 0.0 kB/s 2023-10-08T16:07:32.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.448 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.503 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3214e-05 s, 0.0 kB/s 2023-10-08T16:07:32.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.504 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1117e-05 s, 0.0 kB/s 2023-10-08T16:07:32.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.559 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.615 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8211e-05 s, 0.0 kB/s 2023-10-08T16:07:32.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.615 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.671 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8679e-05 s, 0.0 kB/s 2023-10-08T16:07:32.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.671 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.726 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7682e-05 s, 0.0 kB/s 2023-10-08T16:07:32.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.784 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5948e-05 s, 0.0 kB/s 2023-10-08T16:07:32.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.785 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9177e-05 s, 0.0 kB/s 2023-10-08T16:07:32.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9056e-05 s, 0.0 kB/s 2023-10-08T16:07:32.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.897 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:32.951 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:32.951 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:32.951 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.939e-05 s, 0.0 kB/s 2023-10-08T16:07:32.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:32.952 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.008 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9759e-05 s, 0.0 kB/s 2023-10-08T16:07:33.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.008 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.063 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5453e-05 s, 0.0 kB/s 2023-10-08T16:07:33.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.064 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3882e-05 s, 0.0 kB/s 2023-10-08T16:07:33.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.120 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5722e-05 s, 0.0 kB/s 2023-10-08T16:07:33.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.233 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.233 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9395e-05 s, 0.0 kB/s 2023-10-08T16:07:33.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.233 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.288 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2104e-05 s, 0.0 kB/s 2023-10-08T16:07:33.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.289 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.345 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0269e-05 s, 0.0 kB/s 2023-10-08T16:07:33.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.346 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.401 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.401 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9914e-05 s, 0.0 kB/s 2023-10-08T16:07:33.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.403 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2503e-05 s, 0.0 kB/s 2023-10-08T16:07:33.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.460 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.516 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9858e-05 s, 0.0 kB/s 2023-10-08T16:07:33.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.516 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8284e-05 s, 0.0 kB/s 2023-10-08T16:07:33.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.572 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.628 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.628 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0376e-05 s, 0.0 kB/s 2023-10-08T16:07:33.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.628 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2128e-05 s, 0.0 kB/s 2023-10-08T16:07:33.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.684 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.740 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.740 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.988e-05 s, 0.0 kB/s 2023-10-08T16:07:33.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.741 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.797 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0454e-05 s, 0.0 kB/s 2023-10-08T16:07:33.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.798 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.853 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9807e-05 s, 0.0 kB/s 2023-10-08T16:07:33.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.853 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9078e-05 s, 0.0 kB/s 2023-10-08T16:07:33.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.909 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:33.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:33.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:33.965 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8735e-05 s, 0.0 kB/s 2023-10-08T16:07:33.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:33.965 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.020 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.020 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.020 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6018e-05 s, 0.0 kB/s 2023-10-08T16:07:34.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.021 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.077 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9385e-05 s, 0.0 kB/s 2023-10-08T16:07:34.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.078 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.134 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.134 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.134 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0058e-05 s, 0.0 kB/s 2023-10-08T16:07:34.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.134 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.190 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0121e-05 s, 0.0 kB/s 2023-10-08T16:07:34.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.190 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.246 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1649e-05 s, 0.0 kB/s 2023-10-08T16:07:34.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.247 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.302 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.302 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.302 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0467e-05 s, 0.0 kB/s 2023-10-08T16:07:34.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.303 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.359 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5863e-05 s, 0.0 kB/s 2023-10-08T16:07:34.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.360 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.415 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9921e-05 s, 0.0 kB/s 2023-10-08T16:07:34.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.415 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.470 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.471 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.471 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.049e-05 s, 0.0 kB/s 2023-10-08T16:07:34.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.471 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.527 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9293e-05 s, 0.0 kB/s 2023-10-08T16:07:34.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.527 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.582 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.583 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.145e-05 s, 0.0 kB/s 2023-10-08T16:07:34.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.584 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.639 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.640 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8008e-05 s, 0.0 kB/s 2023-10-08T16:07:34.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.640 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.696 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3436e-05 s, 0.0 kB/s 2023-10-08T16:07:34.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.696 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.752 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9191e-05 s, 0.0 kB/s 2023-10-08T16:07:34.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.752 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.807 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.027e-05 s, 0.0 kB/s 2023-10-08T16:07:34.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.808 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.863 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6214e-05 s, 0.0 kB/s 2023-10-08T16:07:34.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.863 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.918 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.919 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6457e-05 s, 0.0 kB/s 2023-10-08T16:07:34.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.919 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:34.974 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:34.974 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:34.974 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8985e-05 s, 0.0 kB/s 2023-10-08T16:07:34.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:34.974 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.029 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.029 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.030 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0341e-05 s, 0.0 kB/s 2023-10-08T16:07:35.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.030 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.086 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1518e-05 s, 0.0 kB/s 2023-10-08T16:07:35.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.086 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.141 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5148e-05 s, 0.0 kB/s 2023-10-08T16:07:35.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.143 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.198 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5966e-05 s, 0.0 kB/s 2023-10-08T16:07:35.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.199 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.255 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9201e-05 s, 0.0 kB/s 2023-10-08T16:07:35.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.255 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.310 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8347e-05 s, 0.0 kB/s 2023-10-08T16:07:35.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.311 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.367 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5195e-05 s, 0.0 kB/s 2023-10-08T16:07:35.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.367 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.423 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-10-08T16:07:35.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.423 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.478 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.478 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.478 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1362e-05 s, 0.0 kB/s 2023-10-08T16:07:35.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.479 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.534 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9489e-05 s, 0.0 kB/s 2023-10-08T16:07:35.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.589 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.998e-05 s, 0.0 kB/s 2023-10-08T16:07:35.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.589 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.644 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1122e-05 s, 0.0 kB/s 2023-10-08T16:07:35.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.701 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0657e-05 s, 0.0 kB/s 2023-10-08T16:07:35.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.701 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8039e-05 s, 0.0 kB/s 2023-10-08T16:07:35.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.757 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4592e-05 s, 0.0 kB/s 2023-10-08T16:07:35.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.812 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.868 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.868 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.868 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8727e-05 s, 0.0 kB/s 2023-10-08T16:07:35.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.869 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.924 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.925 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2143e-05 s, 0.0 kB/s 2023-10-08T16:07:35.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.925 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:35.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:35.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:35.980 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0429e-05 s, 0.0 kB/s 2023-10-08T16:07:35.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:35.981 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.036 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7069e-05 s, 0.0 kB/s 2023-10-08T16:07:36.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.037 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.092 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.092 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0127e-05 s, 0.0 kB/s 2023-10-08T16:07:36.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.093 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0606e-05 s, 0.0 kB/s 2023-10-08T16:07:36.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.147 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.202 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9541e-05 s, 0.0 kB/s 2023-10-08T16:07:36.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.202 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.258 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8347e-05 s, 0.0 kB/s 2023-10-08T16:07:36.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.314 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9782e-05 s, 0.0 kB/s 2023-10-08T16:07:36.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.370 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2094e-05 s, 0.0 kB/s 2023-10-08T16:07:36.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.426 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.049e-05 s, 0.0 kB/s 2023-10-08T16:07:36.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.426 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.481 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7589e-05 s, 0.0 kB/s 2023-10-08T16:07:36.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.482 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.537 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8784e-05 s, 0.0 kB/s 2023-10-08T16:07:36.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.592 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8282e-05 s, 0.0 kB/s 2023-10-08T16:07:36.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.593 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.649 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1553e-05 s, 0.0 kB/s 2023-10-08T16:07:36.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.705 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3174e-05 s, 0.0 kB/s 2023-10-08T16:07:36.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.705 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.761 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8576e-05 s, 0.0 kB/s 2023-10-08T16:07:36.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.761 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.815 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.815 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.815 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000113984 s, 0.0 kB/s 2023-10-08T16:07:36.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.817 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.949e-05 s, 0.0 kB/s 2023-10-08T16:07:36.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.873 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.927 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7956e-05 s, 0.0 kB/s 2023-10-08T16:07:36.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.928 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:36.984 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:36.984 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:36.984 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3943e-05 s, 0.0 kB/s 2023-10-08T16:07:36.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:36.984 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.040 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9413e-05 s, 0.0 kB/s 2023-10-08T16:07:37.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.040 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.095 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.096 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0464e-05 s, 0.0 kB/s 2023-10-08T16:07:37.096 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.096 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.151 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9611e-05 s, 0.0 kB/s 2023-10-08T16:07:37.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.152 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.207 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.207 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.207 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7309e-05 s, 0.0 kB/s 2023-10-08T16:07:37.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.208 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.262 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.262 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.262 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3114e-05 s, 0.0 kB/s 2023-10-08T16:07:37.263 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.263 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.319 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7381e-05 s, 0.0 kB/s 2023-10-08T16:07:37.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.319 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.375 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6092e-05 s, 0.0 kB/s 2023-10-08T16:07:37.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.376 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.432 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9039e-05 s, 0.0 kB/s 2023-10-08T16:07:37.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.433 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.487 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.487 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.488 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9949e-05 s, 0.0 kB/s 2023-10-08T16:07:37.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.489 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9116e-05 s, 0.0 kB/s 2023-10-08T16:07:37.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.546 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8001e-05 s, 0.0 kB/s 2023-10-08T16:07:37.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.603 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-10-08T16:07:37.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.658 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9874e-05 s, 0.0 kB/s 2023-10-08T16:07:37.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.713 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.769 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2307e-05 s, 0.0 kB/s 2023-10-08T16:07:37.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.826 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8689e-05 s, 0.0 kB/s 2023-10-08T16:07:37.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.826 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2881e-05 s, 0.0 kB/s 2023-10-08T16:07:37.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.936 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8249e-05 s, 0.0 kB/s 2023-10-08T16:07:37.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:37.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:37.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:37.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9952e-05 s, 0.0 kB/s 2023-10-08T16:07:37.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:37.991 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2589e-05 s, 0.0 kB/s 2023-10-08T16:07:38.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0775e-05 s, 0.0 kB/s 2023-10-08T16:07:38.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9518e-05 s, 0.0 kB/s 2023-10-08T16:07:38.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.158 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.213 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2148e-05 s, 0.0 kB/s 2023-10-08T16:07:38.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.214 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.269 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.269 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.269 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8756e-05 s, 0.0 kB/s 2023-10-08T16:07:38.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.269 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3734e-05 s, 0.0 kB/s 2023-10-08T16:07:38.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.380 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1004e-05 s, 0.0 kB/s 2023-10-08T16:07:38.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.381 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.436 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.848e-05 s, 0.0 kB/s 2023-10-08T16:07:38.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.437 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.491 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7475e-05 s, 0.0 kB/s 2023-10-08T16:07:38.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.492 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.548 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.776e-05 s, 0.0 kB/s 2023-10-08T16:07:38.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.548 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0754e-05 s, 0.0 kB/s 2023-10-08T16:07:38.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8285e-05 s, 0.0 kB/s 2023-10-08T16:07:38.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4058e-05 s, 0.0 kB/s 2023-10-08T16:07:38.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.769 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3295e-05 s, 0.0 kB/s 2023-10-08T16:07:38.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3179e-05 s, 0.0 kB/s 2023-10-08T16:07:38.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8859e-05 s, 0.0 kB/s 2023-10-08T16:07:38.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.880 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8754e-05 s, 0.0 kB/s 2023-10-08T16:07:38.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:38.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:38.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:38.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8093e-05 s, 0.0 kB/s 2023-10-08T16:07:38.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:38.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.046 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.046 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5928e-05 s, 0.0 kB/s 2023-10-08T16:07:39.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1051e-05 s, 0.0 kB/s 2023-10-08T16:07:39.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.102 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.157 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8356e-05 s, 0.0 kB/s 2023-10-08T16:07:39.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.157 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.213 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0605e-05 s, 0.0 kB/s 2023-10-08T16:07:39.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.213 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5867e-05 s, 0.0 kB/s 2023-10-08T16:07:39.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7572e-05 s, 0.0 kB/s 2023-10-08T16:07:39.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.380 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0355e-05 s, 0.0 kB/s 2023-10-08T16:07:39.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0626e-05 s, 0.0 kB/s 2023-10-08T16:07:39.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.436 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.491 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8133e-05 s, 0.0 kB/s 2023-10-08T16:07:39.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.491 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.546 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.547 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5739e-05 s, 0.0 kB/s 2023-10-08T16:07:39.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.547 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5308e-05 s, 0.0 kB/s 2023-10-08T16:07:39.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.603 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7009e-05 s, 0.0 kB/s 2023-10-08T16:07:39.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.713 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1683e-05 s, 0.0 kB/s 2023-10-08T16:07:39.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.769 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1572e-05 s, 0.0 kB/s 2023-10-08T16:07:39.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.770 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5598e-05 s, 0.0 kB/s 2023-10-08T16:07:39.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.826 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.881 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1059e-05 s, 0.0 kB/s 2023-10-08T16:07:39.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.937 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0631e-05 s, 0.0 kB/s 2023-10-08T16:07:39.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.937 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:39.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7754e-05 s, 0.0 kB/s 2023-10-08T16:07:39.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:39.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.048 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4148e-05 s, 0.0 kB/s 2023-10-08T16:07:40.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.048 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8468e-05 s, 0.0 kB/s 2023-10-08T16:07:40.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0658e-05 s, 0.0 kB/s 2023-10-08T16:07:40.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.223 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7715e-05 s, 0.0 kB/s 2023-10-08T16:07:40.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.223 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1722e-05 s, 0.0 kB/s 2023-10-08T16:07:40.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.336 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0426e-05 s, 0.0 kB/s 2023-10-08T16:07:40.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.392 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.392 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.392 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1726e-05 s, 0.0 kB/s 2023-10-08T16:07:40.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.393 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.448 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0081e-05 s, 0.0 kB/s 2023-10-08T16:07:40.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.448 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0436e-05 s, 0.0 kB/s 2023-10-08T16:07:40.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.504 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9424e-05 s, 0.0 kB/s 2023-10-08T16:07:40.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.560 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.614 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8201e-05 s, 0.0 kB/s 2023-10-08T16:07:40.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.615 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.669 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9941e-05 s, 0.0 kB/s 2023-10-08T16:07:40.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.670 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.726 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1944e-05 s, 0.0 kB/s 2023-10-08T16:07:40.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7986e-05 s, 0.0 kB/s 2023-10-08T16:07:40.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.782 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.837 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.837 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.838 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000101438 s, 0.0 kB/s 2023-10-08T16:07:40.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.838 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.893 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.893 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.893 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1725e-05 s, 0.0 kB/s 2023-10-08T16:07:40.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.894 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:40.949 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:40.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:40.950 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-08T16:07:40.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:40.950 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.006 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8643e-05 s, 0.0 kB/s 2023-10-08T16:07:41.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.006 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.061 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4655e-05 s, 0.0 kB/s 2023-10-08T16:07:41.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.062 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.117 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.117 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.117 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1438e-05 s, 0.0 kB/s 2023-10-08T16:07:41.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.118 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.172 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9028e-05 s, 0.0 kB/s 2023-10-08T16:07:41.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.173 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0151e-05 s, 0.0 kB/s 2023-10-08T16:07:41.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.228 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.283 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.284 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1888e-05 s, 0.0 kB/s 2023-10-08T16:07:41.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.284 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.340 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0001e-05 s, 0.0 kB/s 2023-10-08T16:07:41.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.341 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.396 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5448e-05 s, 0.0 kB/s 2023-10-08T16:07:41.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.452 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.452 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5205e-05 s, 0.0 kB/s 2023-10-08T16:07:41.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.452 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.508 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6887e-05 s, 0.0 kB/s 2023-10-08T16:07:41.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.508 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.564 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0465e-05 s, 0.0 kB/s 2023-10-08T16:07:41.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.565 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.619 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2798e-05 s, 0.0 kB/s 2023-10-08T16:07:41.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.620 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.674 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.675 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5599e-05 s, 0.0 kB/s 2023-10-08T16:07:41.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.675 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.731 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3005e-05 s, 0.0 kB/s 2023-10-08T16:07:41.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.731 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2305e-05 s, 0.0 kB/s 2023-10-08T16:07:41.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.843 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0618e-05 s, 0.0 kB/s 2023-10-08T16:07:41.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.898 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3519e-05 s, 0.0 kB/s 2023-10-08T16:07:41.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:41.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5021e-05 s, 0.0 kB/s 2023-10-08T16:07:41.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:41.955 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.010 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0937e-05 s, 0.0 kB/s 2023-10-08T16:07:42.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.011 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.065 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6449e-05 s, 0.0 kB/s 2023-10-08T16:07:42.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.066 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.122 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8814e-05 s, 0.0 kB/s 2023-10-08T16:07:42.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.122 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.177 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.177 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.178 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.759e-05 s, 0.0 kB/s 2023-10-08T16:07:42.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.178 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.282 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3171e-05 s, 0.0 kB/s 2023-10-08T16:07:42.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.282 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.297 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2351e-05 s, 0.0 kB/s 2023-10-08T16:07:42.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.298 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.354 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9961e-05 s, 0.0 kB/s 2023-10-08T16:07:42.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.354 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9004e-05 s, 0.0 kB/s 2023-10-08T16:07:42.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.410 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.467 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.968e-05 s, 0.0 kB/s 2023-10-08T16:07:42.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.468 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1351e-05 s, 0.0 kB/s 2023-10-08T16:07:42.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.523 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.578 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9805e-05 s, 0.0 kB/s 2023-10-08T16:07:42.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1191e-05 s, 0.0 kB/s 2023-10-08T16:07:42.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.636 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.690 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.690 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.690 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000110341 s, 0.0 kB/s 2023-10-08T16:07:42.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.691 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.745 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.746 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.746 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8965e-05 s, 0.0 kB/s 2023-10-08T16:07:42.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.746 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1385e-05 s, 0.0 kB/s 2023-10-08T16:07:42.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.802 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.857 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9231e-05 s, 0.0 kB/s 2023-10-08T16:07:42.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.858 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.912 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.912 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.912 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0508e-05 s, 0.0 kB/s 2023-10-08T16:07:42.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.912 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:42.967 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.73e-05 s, 0.0 kB/s 2023-10-08T16:07:42.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:42.968 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.023 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.023 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.024 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9736e-05 s, 0.0 kB/s 2023-10-08T16:07:43.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.024 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.079 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.079 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.080 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9678e-05 s, 0.0 kB/s 2023-10-08T16:07:43.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.080 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.135 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.135 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.136 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7839e-05 s, 0.0 kB/s 2023-10-08T16:07:43.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.136 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.191 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.191 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.192 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0745e-05 s, 0.0 kB/s 2023-10-08T16:07:43.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.193 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0219e-05 s, 0.0 kB/s 2023-10-08T16:07:43.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.249 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:07:43.305 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.305 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.305 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6162e-05 s, 0.0 kB/s 2023-10-08T16:07:43.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.306 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.306 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:07:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.363 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.976e-05 s, 0.0 kB/s 2023-10-08T16:07:43.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.364 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1619e-05 s, 0.0 kB/s 2023-10-08T16:07:43.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.475 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.176e-05 s, 0.0 kB/s 2023-10-08T16:07:43.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.476 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.530 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9576e-05 s, 0.0 kB/s 2023-10-08T16:07:43.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.531 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2424e-05 s, 0.0 kB/s 2023-10-08T16:07:43.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.643 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1258e-05 s, 0.0 kB/s 2023-10-08T16:07:43.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0562e-05 s, 0.0 kB/s 2023-10-08T16:07:43.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.699 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0181e-05 s, 0.0 kB/s 2023-10-08T16:07:43.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.756 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9718e-05 s, 0.0 kB/s 2023-10-08T16:07:43.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.881e-05 s, 0.0 kB/s 2023-10-08T16:07:43.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.923 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.923 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9323e-05 s, 0.0 kB/s 2023-10-08T16:07:43.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:43.979 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:43.979 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:43.979 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5942e-05 s, 0.0 kB/s 2023-10-08T16:07:43.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:43.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.034 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1422e-05 s, 0.0 kB/s 2023-10-08T16:07:44.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.035 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4348e-05 s, 0.0 kB/s 2023-10-08T16:07:44.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.091 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8145e-05 s, 0.0 kB/s 2023-10-08T16:07:44.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.148 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9536e-05 s, 0.0 kB/s 2023-10-08T16:07:44.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.259 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8819e-05 s, 0.0 kB/s 2023-10-08T16:07:44.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.260 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.316 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9531e-05 s, 0.0 kB/s 2023-10-08T16:07:44.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.317 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.374 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8776e-05 s, 0.0 kB/s 2023-10-08T16:07:44.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.374 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.429 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5605e-05 s, 0.0 kB/s 2023-10-08T16:07:44.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.430 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6638e-05 s, 0.0 kB/s 2023-10-08T16:07:44.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.488 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.543 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-10-08T16:07:44.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.544 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.600 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.600 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0534e-05 s, 0.0 kB/s 2023-10-08T16:07:44.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.601 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.656 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.657 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.657 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0473e-05 s, 0.0 kB/s 2023-10-08T16:07:44.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.657 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.712 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.713 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5185e-05 s, 0.0 kB/s 2023-10-08T16:07:44.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.713 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.769 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0899e-05 s, 0.0 kB/s 2023-10-08T16:07:44.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.769 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3546e-05 s, 0.0 kB/s 2023-10-08T16:07:44.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.881 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.077e-05 s, 0.0 kB/s 2023-10-08T16:07:44.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.937 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6892e-05 s, 0.0 kB/s 2023-10-08T16:07:44.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.938 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:44.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:44.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:44.993 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4246e-05 s, 0.0 kB/s 2023-10-08T16:07:44.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:44.994 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.048 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9429e-05 s, 0.0 kB/s 2023-10-08T16:07:45.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.049 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.104 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.894e-05 s, 0.0 kB/s 2023-10-08T16:07:45.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.160 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0258e-05 s, 0.0 kB/s 2023-10-08T16:07:45.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4345e-05 s, 0.0 kB/s 2023-10-08T16:07:45.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.271 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0883e-05 s, 0.0 kB/s 2023-10-08T16:07:45.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.329 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4638e-05 s, 0.0 kB/s 2023-10-08T16:07:45.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.329 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.385 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5278e-05 s, 0.0 kB/s 2023-10-08T16:07:45.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.386 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4246e-05 s, 0.0 kB/s 2023-10-08T16:07:45.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2904e-05 s, 0.0 kB/s 2023-10-08T16:07:45.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.496 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.551 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5373e-05 s, 0.0 kB/s 2023-10-08T16:07:45.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.552 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.607 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5518e-05 s, 0.0 kB/s 2023-10-08T16:07:45.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.662 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1654e-05 s, 0.0 kB/s 2023-10-08T16:07:45.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.663 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.717 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-10-08T16:07:45.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.774 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6712e-05 s, 0.0 kB/s 2023-10-08T16:07:45.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.774 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9675e-05 s, 0.0 kB/s 2023-10-08T16:07:45.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.830 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.885 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6335e-05 s, 0.0 kB/s 2023-10-08T16:07:45.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.885 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8677e-05 s, 0.0 kB/s 2023-10-08T16:07:45.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:45.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:45.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:45.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0603e-05 s, 0.0 kB/s 2023-10-08T16:07:45.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:45.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.052 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9785e-05 s, 0.0 kB/s 2023-10-08T16:07:46.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.108 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.542e-05 s, 0.0 kB/s 2023-10-08T16:07:46.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.163 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1712e-05 s, 0.0 kB/s 2023-10-08T16:07:46.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.164 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.218 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3731e-05 s, 0.0 kB/s 2023-10-08T16:07:46.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.219 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.274 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3115e-05 s, 0.0 kB/s 2023-10-08T16:07:46.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.275 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.330 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.865e-05 s, 0.0 kB/s 2023-10-08T16:07:46.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.388 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.389 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.389 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8292e-05 s, 0.0 kB/s 2023-10-08T16:07:46.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.389 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.445 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.445 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.445 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9258e-05 s, 0.0 kB/s 2023-10-08T16:07:46.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.445 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.500 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.500 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.500 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9993e-05 s, 0.0 kB/s 2023-10-08T16:07:46.501 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.501 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.556 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.556 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.556 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2685e-05 s, 0.0 kB/s 2023-10-08T16:07:46.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.557 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.612 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.612 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.612 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3855e-05 s, 0.0 kB/s 2023-10-08T16:07:46.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.613 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.669 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.670 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.670 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9127e-05 s, 0.0 kB/s 2023-10-08T16:07:46.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.670 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.725 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.725 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.725 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6562e-05 s, 0.0 kB/s 2023-10-08T16:07:46.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.726 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.782 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.782 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.782 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0884e-05 s, 0.0 kB/s 2023-10-08T16:07:46.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.782 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5442e-05 s, 0.0 kB/s 2023-10-08T16:07:46.837 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.838 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.892 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.892 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.893 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8966e-05 s, 0.0 kB/s 2023-10-08T16:07:46.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.894 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:46.949 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:46.949 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:46.950 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.984e-05 s, 0.0 kB/s 2023-10-08T16:07:46.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:46.950 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.005 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.005 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.005 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.764e-05 s, 0.0 kB/s 2023-10-08T16:07:47.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.006 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.060 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.060 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.060 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1796e-05 s, 0.0 kB/s 2023-10-08T16:07:47.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.061 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.115 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.115 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.115 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0509e-05 s, 0.0 kB/s 2023-10-08T16:07:47.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.116 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.170 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5695e-05 s, 0.0 kB/s 2023-10-08T16:07:47.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.171 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.226 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4245e-05 s, 0.0 kB/s 2023-10-08T16:07:47.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.227 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.282 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.726e-05 s, 0.0 kB/s 2023-10-08T16:07:47.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.283 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.336 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.336 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.337 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2585e-05 s, 0.0 kB/s 2023-10-08T16:07:47.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.337 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.393 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9161e-05 s, 0.0 kB/s 2023-10-08T16:07:47.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.394 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.448 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0713e-05 s, 0.0 kB/s 2023-10-08T16:07:47.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.449 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8947e-05 s, 0.0 kB/s 2023-10-08T16:07:47.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.504 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.560 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9664e-05 s, 0.0 kB/s 2023-10-08T16:07:47.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.560 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.616 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1235e-05 s, 0.0 kB/s 2023-10-08T16:07:47.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.616 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1755e-05 s, 0.0 kB/s 2023-10-08T16:07:47.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.672 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.728 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.728 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3289e-05 s, 0.0 kB/s 2023-10-08T16:07:47.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.728 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.784 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3371e-05 s, 0.0 kB/s 2023-10-08T16:07:47.784 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.784 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.839 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0962e-05 s, 0.0 kB/s 2023-10-08T16:07:47.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.894 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.895 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7949e-05 s, 0.0 kB/s 2023-10-08T16:07:47.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.895 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:47.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:47.950 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:47.950 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9782e-05 s, 0.0 kB/s 2023-10-08T16:07:47.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:47.951 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.005 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.006 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.006 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9996e-05 s, 0.0 kB/s 2023-10-08T16:07:48.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.007 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.062 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1002e-05 s, 0.0 kB/s 2023-10-08T16:07:48.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.063 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.118 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8365e-05 s, 0.0 kB/s 2023-10-08T16:07:48.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.119 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.174 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.175 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1649e-05 s, 0.0 kB/s 2023-10-08T16:07:48.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.175 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6916e-05 s, 0.0 kB/s 2023-10-08T16:07:48.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.231 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.286 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.286 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.286 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0241e-05 s, 0.0 kB/s 2023-10-08T16:07:48.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.287 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.342 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.342 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.342 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.917e-05 s, 0.0 kB/s 2023-10-08T16:07:48.342 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.342 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.398 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.398 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.398 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0415e-05 s, 0.0 kB/s 2023-10-08T16:07:48.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.398 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.453 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.453 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.453 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.357e-05 s, 0.0 kB/s 2023-10-08T16:07:48.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.454 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.509 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.509 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.509 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9875e-05 s, 0.0 kB/s 2023-10-08T16:07:48.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.510 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.565 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5676e-05 s, 0.0 kB/s 2023-10-08T16:07:48.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.565 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9816e-05 s, 0.0 kB/s 2023-10-08T16:07:48.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.621 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.98e-05 s, 0.0 kB/s 2023-10-08T16:07:48.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.676 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.731 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2125e-05 s, 0.0 kB/s 2023-10-08T16:07:48.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.732 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0883e-05 s, 0.0 kB/s 2023-10-08T16:07:48.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.842 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.843 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9006e-05 s, 0.0 kB/s 2023-10-08T16:07:48.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7886e-05 s, 0.0 kB/s 2023-10-08T16:07:48.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:48.954 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:48.954 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:48.954 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5137e-05 s, 0.0 kB/s 2023-10-08T16:07:48.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:48.954 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5398e-05 s, 0.0 kB/s 2023-10-08T16:07:49.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.010 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.065 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1558e-05 s, 0.0 kB/s 2023-10-08T16:07:49.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.066 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1605e-05 s, 0.0 kB/s 2023-10-08T16:07:49.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.122 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8841e-05 s, 0.0 kB/s 2023-10-08T16:07:49.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.177 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.231 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.231 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.232 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3018e-05 s, 0.0 kB/s 2023-10-08T16:07:49.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.232 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.287 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7608e-05 s, 0.0 kB/s 2023-10-08T16:07:49.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.288 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.343 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0258e-05 s, 0.0 kB/s 2023-10-08T16:07:49.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.344 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.399 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0622e-05 s, 0.0 kB/s 2023-10-08T16:07:49.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.399 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.146e-05 s, 0.0 kB/s 2023-10-08T16:07:49.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.455 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.511 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2578e-05 s, 0.0 kB/s 2023-10-08T16:07:49.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.511 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.566 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.566 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.567 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9921e-05 s, 0.0 kB/s 2023-10-08T16:07:49.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.567 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.622 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.623 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.623 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9107e-05 s, 0.0 kB/s 2023-10-08T16:07:49.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.623 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.678 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.679 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.966e-05 s, 0.0 kB/s 2023-10-08T16:07:49.679 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.679 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.735 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9942e-05 s, 0.0 kB/s 2023-10-08T16:07:49.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.735 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.790 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.791 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2768e-05 s, 0.0 kB/s 2023-10-08T16:07:49.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.791 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.848 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8815e-05 s, 0.0 kB/s 2023-10-08T16:07:49.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.848 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.903 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8798e-05 s, 0.0 kB/s 2023-10-08T16:07:49.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.904 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:49.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:49.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:49.960 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.5805e-05 s, 0.0 kB/s 2023-10-08T16:07:49.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:49.960 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.015 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.015 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.015 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7607e-05 s, 0.0 kB/s 2023-10-08T16:07:50.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.016 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.071 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.071 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.071 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5969e-05 s, 0.0 kB/s 2023-10-08T16:07:50.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.072 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.126 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2518e-05 s, 0.0 kB/s 2023-10-08T16:07:50.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.127 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.183 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.2116e-05 s, 0.0 kB/s 2023-10-08T16:07:50.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.183 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.239 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7134e-05 s, 0.0 kB/s 2023-10-08T16:07:50.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.239 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.294 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0514e-05 s, 0.0 kB/s 2023-10-08T16:07:50.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.295 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.350 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9191e-05 s, 0.0 kB/s 2023-10-08T16:07:50.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.351 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.405 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.406 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.406 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9078e-05 s, 0.0 kB/s 2023-10-08T16:07:50.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.406 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.462 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8292e-05 s, 0.0 kB/s 2023-10-08T16:07:50.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.463 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.517 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.517 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.518 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1421e-05 s, 0.0 kB/s 2023-10-08T16:07:50.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.518 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.575 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2689e-05 s, 0.0 kB/s 2023-10-08T16:07:50.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.575 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.631 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8727e-05 s, 0.0 kB/s 2023-10-08T16:07:50.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.632 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.685 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.686 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.686 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9828e-05 s, 0.0 kB/s 2023-10-08T16:07:50.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.686 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.742 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.743 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.743 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2658e-05 s, 0.0 kB/s 2023-10-08T16:07:50.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.743 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.801 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.801 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.216e-05 s, 0.0 kB/s 2023-10-08T16:07:50.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.801 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1072e-05 s, 0.0 kB/s 2023-10-08T16:07:50.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.857 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.913 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.913 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8258e-05 s, 0.0 kB/s 2023-10-08T16:07:50.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.914 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:50.969 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:50.970 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:50.970 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9959e-05 s, 0.0 kB/s 2023-10-08T16:07:50.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:50.970 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.025 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.026 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0206e-05 s, 0.0 kB/s 2023-10-08T16:07:51.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.026 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.081 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0528e-05 s, 0.0 kB/s 2023-10-08T16:07:51.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.083 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.137 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.137 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.138 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0942e-05 s, 0.0 kB/s 2023-10-08T16:07:51.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.139 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.194 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.194 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.195 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4805e-05 s, 0.0 kB/s 2023-10-08T16:07:51.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.195 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.251 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3053e-05 s, 0.0 kB/s 2023-10-08T16:07:51.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.251 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.307 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1675e-05 s, 0.0 kB/s 2023-10-08T16:07:51.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.307 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.363 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9576e-05 s, 0.0 kB/s 2023-10-08T16:07:51.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.363 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.420 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0069e-05 s, 0.0 kB/s 2023-10-08T16:07:51.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.476 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3175e-05 s, 0.0 kB/s 2023-10-08T16:07:51.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.477 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.533 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.533 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.533 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0009e-05 s, 0.0 kB/s 2023-10-08T16:07:51.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.588 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.589 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3275e-05 s, 0.0 kB/s 2023-10-08T16:07:51.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.589 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.645 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2812e-05 s, 0.0 kB/s 2023-10-08T16:07:51.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9568e-05 s, 0.0 kB/s 2023-10-08T16:07:51.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.701 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.757 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.757 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.757 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9438e-05 s, 0.0 kB/s 2023-10-08T16:07:51.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.758 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2428e-05 s, 0.0 kB/s 2023-10-08T16:07:51.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.814 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8351e-05 s, 0.0 kB/s 2023-10-08T16:07:51.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.871 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.495e-05 s, 0.0 kB/s 2023-10-08T16:07:51.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0661e-05 s, 0.0 kB/s 2023-10-08T16:07:51.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:51.982 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.037 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0981e-05 s, 0.0 kB/s 2023-10-08T16:07:52.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.038 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.094 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.094 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0031e-05 s, 0.0 kB/s 2023-10-08T16:07:52.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.150 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.150 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8872e-05 s, 0.0 kB/s 2023-10-08T16:07:52.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.150 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.206 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.206 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.206 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1077e-05 s, 0.0 kB/s 2023-10-08T16:07:52.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.207 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.261 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2168e-05 s, 0.0 kB/s 2023-10-08T16:07:52.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.262 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.317 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.087e-05 s, 0.0 kB/s 2023-10-08T16:07:52.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.318 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.373 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.373 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.373 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0729e-05 s, 0.0 kB/s 2023-10-08T16:07:52.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.429 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1368e-05 s, 0.0 kB/s 2023-10-08T16:07:52.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.430 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.539 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1282e-05 s, 0.0 kB/s 2023-10-08T16:07:52.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9719e-05 s, 0.0 kB/s 2023-10-08T16:07:52.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.556 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.611 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0811e-05 s, 0.0 kB/s 2023-10-08T16:07:52.611 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.611 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.667 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9803e-05 s, 0.0 kB/s 2023-10-08T16:07:52.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.668 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.723 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.723 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.723 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6511e-05 s, 0.0 kB/s 2023-10-08T16:07:52.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.724 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.778 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2235e-05 s, 0.0 kB/s 2023-10-08T16:07:52.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.780 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7966e-05 s, 0.0 kB/s 2023-10-08T16:07:52.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.837 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.892 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.892 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.892 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8293e-05 s, 0.0 kB/s 2023-10-08T16:07:52.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.892 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:52.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:52.947 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:52.947 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0679e-05 s, 0.0 kB/s 2023-10-08T16:07:52.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:52.949 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.004 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.004 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.005 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0015e-05 s, 0.0 kB/s 2023-10-08T16:07:53.005 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.005 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.061 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.061 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0704e-05 s, 0.0 kB/s 2023-10-08T16:07:53.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.062 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9899e-05 s, 0.0 kB/s 2023-10-08T16:07:53.117 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.117 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.172 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.172 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3897e-05 s, 0.0 kB/s 2023-10-08T16:07:53.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.173 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.228 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.229 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1785e-05 s, 0.0 kB/s 2023-10-08T16:07:53.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.229 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.284 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.284 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2298e-05 s, 0.0 kB/s 2023-10-08T16:07:53.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.285 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.340 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.340 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9564e-05 s, 0.0 kB/s 2023-10-08T16:07:53.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.340 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9577e-05 s, 0.0 kB/s 2023-10-08T16:07:53.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.451 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2668e-05 s, 0.0 kB/s 2023-10-08T16:07:53.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.452 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.507 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0161e-05 s, 0.0 kB/s 2023-10-08T16:07:53.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.508 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0239e-05 s, 0.0 kB/s 2023-10-08T16:07:53.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.564 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2088e-05 s, 0.0 kB/s 2023-10-08T16:07:53.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.620 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.676 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.676 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3204e-05 s, 0.0 kB/s 2023-10-08T16:07:53.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.676 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5409e-05 s, 0.0 kB/s 2023-10-08T16:07:53.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.732 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.788 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0684e-05 s, 0.0 kB/s 2023-10-08T16:07:53.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.788 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.844 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.025e-05 s, 0.0 kB/s 2023-10-08T16:07:53.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.844 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.392e-05 s, 0.0 kB/s 2023-10-08T16:07:53.901 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.901 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:53.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:53.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:53.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2971e-05 s, 0.0 kB/s 2023-10-08T16:07:53.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:53.956 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.012 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.012 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.012 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8533e-05 s, 0.0 kB/s 2023-10-08T16:07:54.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.012 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0141e-05 s, 0.0 kB/s 2023-10-08T16:07:54.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.124 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3991e-05 s, 0.0 kB/s 2023-10-08T16:07:54.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1162e-05 s, 0.0 kB/s 2023-10-08T16:07:54.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.180 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1573e-05 s, 0.0 kB/s 2023-10-08T16:07:54.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.301 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.2878e-05 s, 0.0 kB/s 2023-10-08T16:07:54.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.302 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.358 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.359 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5817e-05 s, 0.0 kB/s 2023-10-08T16:07:54.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.359 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.415 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3358e-05 s, 0.0 kB/s 2023-10-08T16:07:54.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.415 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.471 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6291e-05 s, 0.0 kB/s 2023-10-08T16:07:54.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.471 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.526 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7977e-05 s, 0.0 kB/s 2023-10-08T16:07:54.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.527 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.583 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.583 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.583 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.0677e-05 s, 0.0 kB/s 2023-10-08T16:07:54.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.583 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.638 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.639 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.639 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0336e-05 s, 0.0 kB/s 2023-10-08T16:07:54.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.640 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.696 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2938e-05 s, 0.0 kB/s 2023-10-08T16:07:54.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.696 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.751 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.751 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.752 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5881e-05 s, 0.0 kB/s 2023-10-08T16:07:54.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.753 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.809 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4812e-05 s, 0.0 kB/s 2023-10-08T16:07:54.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.809 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8909e-05 s, 0.0 kB/s 2023-10-08T16:07:54.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.865 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.920 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.920 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1157e-05 s, 0.0 kB/s 2023-10-08T16:07:54.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.920 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:54.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:54.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:54.975 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9085e-05 s, 0.0 kB/s 2023-10-08T16:07:54.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:54.976 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.031 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8011e-05 s, 0.0 kB/s 2023-10-08T16:07:55.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.032 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.087 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9461e-05 s, 0.0 kB/s 2023-10-08T16:07:55.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.087 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.142 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.143 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0337e-05 s, 0.0 kB/s 2023-10-08T16:07:55.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.143 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.198 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8798e-05 s, 0.0 kB/s 2023-10-08T16:07:55.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.200 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.258 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5919e-05 s, 0.0 kB/s 2023-10-08T16:07:55.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.314 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5475e-05 s, 0.0 kB/s 2023-10-08T16:07:55.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.373 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1802e-05 s, 0.0 kB/s 2023-10-08T16:07:55.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.427 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.428 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8069e-05 s, 0.0 kB/s 2023-10-08T16:07:55.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.429 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.485 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.485 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.485 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1423e-05 s, 0.0 kB/s 2023-10-08T16:07:55.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.486 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.543 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9578e-05 s, 0.0 kB/s 2023-10-08T16:07:55.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.543 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.597 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.598 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1746e-05 s, 0.0 kB/s 2023-10-08T16:07:55.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.598 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.653 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.653 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.653 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1356e-05 s, 0.0 kB/s 2023-10-08T16:07:55.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.654 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.709 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8099e-05 s, 0.0 kB/s 2023-10-08T16:07:55.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.710 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.765 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0299e-05 s, 0.0 kB/s 2023-10-08T16:07:55.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.765 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0929e-05 s, 0.0 kB/s 2023-10-08T16:07:55.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.822 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.876 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1282e-05 s, 0.0 kB/s 2023-10-08T16:07:55.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.933 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0357e-05 s, 0.0 kB/s 2023-10-08T16:07:55.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:55.989 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:55.989 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:55.989 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9892e-05 s, 0.0 kB/s 2023-10-08T16:07:55.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:55.989 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.045 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4045e-05 s, 0.0 kB/s 2023-10-08T16:07:56.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.046 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.100 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9385e-05 s, 0.0 kB/s 2023-10-08T16:07:56.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.155 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0629e-05 s, 0.0 kB/s 2023-10-08T16:07:56.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.213 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2593e-05 s, 0.0 kB/s 2023-10-08T16:07:56.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.213 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7972e-05 s, 0.0 kB/s 2023-10-08T16:07:56.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.269 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0718e-05 s, 0.0 kB/s 2023-10-08T16:07:56.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.324 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.380 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1097e-05 s, 0.0 kB/s 2023-10-08T16:07:56.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.435 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4196e-05 s, 0.0 kB/s 2023-10-08T16:07:56.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.436 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.491 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2665e-05 s, 0.0 kB/s 2023-10-08T16:07:56.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.492 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.547 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0574e-05 s, 0.0 kB/s 2023-10-08T16:07:56.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.548 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6853e-05 s, 0.0 kB/s 2023-10-08T16:07:56.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.659 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1331e-05 s, 0.0 kB/s 2023-10-08T16:07:56.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1943e-05 s, 0.0 kB/s 2023-10-08T16:07:56.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.715 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.771 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.771 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0959e-05 s, 0.0 kB/s 2023-10-08T16:07:56.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.827 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2069e-05 s, 0.0 kB/s 2023-10-08T16:07:56.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.827 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.882 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8591e-05 s, 0.0 kB/s 2023-10-08T16:07:56.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.883 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.939 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7545e-05 s, 0.0 kB/s 2023-10-08T16:07:56.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.940 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:56.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:56.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:56.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0471e-05 s, 0.0 kB/s 2023-10-08T16:07:56.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:56.996 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.052 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3605e-05 s, 0.0 kB/s 2023-10-08T16:07:57.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.108 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.108 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.109 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1582e-05 s, 0.0 kB/s 2023-10-08T16:07:57.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.109 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.164 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.164 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.164 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.914e-05 s, 0.0 kB/s 2023-10-08T16:07:57.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.165 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.221 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9177e-05 s, 0.0 kB/s 2023-10-08T16:07:57.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.221 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.276 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.276 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.276 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1679e-05 s, 0.0 kB/s 2023-10-08T16:07:57.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.277 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.331 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.332 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.332 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0159e-05 s, 0.0 kB/s 2023-10-08T16:07:57.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.332 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.388 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0915e-05 s, 0.0 kB/s 2023-10-08T16:07:57.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.388 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.443 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.443 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.443 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0217e-05 s, 0.0 kB/s 2023-10-08T16:07:57.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.444 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.501 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.501 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.501 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8566e-05 s, 0.0 kB/s 2023-10-08T16:07:57.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.502 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.556 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.556 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.556 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0743e-05 s, 0.0 kB/s 2023-10-08T16:07:57.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.557 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.611 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.612 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.612 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5054e-05 s, 0.0 kB/s 2023-10-08T16:07:57.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.612 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.667 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.667 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2662e-05 s, 0.0 kB/s 2023-10-08T16:07:57.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.668 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.723 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.723 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.723 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4505e-05 s, 0.0 kB/s 2023-10-08T16:07:57.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.723 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.779 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.780 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.780 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0608e-05 s, 0.0 kB/s 2023-10-08T16:07:57.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.780 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.835 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.835 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.836 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0761e-05 s, 0.0 kB/s 2023-10-08T16:07:57.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.836 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.891 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.891 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.891 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9667e-05 s, 0.0 kB/s 2023-10-08T16:07:57.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.891 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:57.946 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:57.946 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:57.946 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1805e-05 s, 0.0 kB/s 2023-10-08T16:07:57.947 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:57.947 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2301e-05 s, 0.0 kB/s 2023-10-08T16:07:58.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.002 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.057 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0494e-05 s, 0.0 kB/s 2023-10-08T16:07:58.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.058 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.113 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.114 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9448e-05 s, 0.0 kB/s 2023-10-08T16:07:58.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.114 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.170 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9034e-05 s, 0.0 kB/s 2023-10-08T16:07:58.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.170 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.225 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9414e-05 s, 0.0 kB/s 2023-10-08T16:07:58.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.226 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.280 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.280 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2745e-05 s, 0.0 kB/s 2023-10-08T16:07:58.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.281 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.337 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0979e-05 s, 0.0 kB/s 2023-10-08T16:07:58.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.337 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.392 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.393 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1508e-05 s, 0.0 kB/s 2023-10-08T16:07:58.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.393 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.448 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1284e-05 s, 0.0 kB/s 2023-10-08T16:07:58.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.449 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.505 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.089e-05 s, 0.0 kB/s 2023-10-08T16:07:58.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.505 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.560 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1999e-05 s, 0.0 kB/s 2023-10-08T16:07:58.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.561 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.668 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.668 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.668 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8644e-05 s, 0.0 kB/s 2023-10-08T16:07:58.668 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.668 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.684 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0591e-05 s, 0.0 kB/s 2023-10-08T16:07:58.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.685 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.741 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4769e-05 s, 0.0 kB/s 2023-10-08T16:07:58.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.742 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.797 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8202e-05 s, 0.0 kB/s 2023-10-08T16:07:58.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.798 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.852 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.852 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.853 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8939e-05 s, 0.0 kB/s 2023-10-08T16:07:58.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.853 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.908 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.909 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9047e-05 s, 0.0 kB/s 2023-10-08T16:07:58.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.909 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:58.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:58.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:58.965 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1278e-05 s, 0.0 kB/s 2023-10-08T16:07:58.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:58.966 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.020 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.020 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.020 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9944e-05 s, 0.0 kB/s 2023-10-08T16:07:59.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.021 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.076 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1021e-05 s, 0.0 kB/s 2023-10-08T16:07:59.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.077 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.131 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.131 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.131 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5548e-05 s, 0.0 kB/s 2023-10-08T16:07:59.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.132 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.188 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.188 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8017e-05 s, 0.0 kB/s 2023-10-08T16:07:59.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.188 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2205e-05 s, 0.0 kB/s 2023-10-08T16:07:59.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.243 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.299 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9685e-05 s, 0.0 kB/s 2023-10-08T16:07:59.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.299 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.355 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4415e-05 s, 0.0 kB/s 2023-10-08T16:07:59.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.355 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0102e-05 s, 0.0 kB/s 2023-10-08T16:07:59.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.411 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.466 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.467 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8951e-05 s, 0.0 kB/s 2023-10-08T16:07:59.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.467 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3064e-05 s, 0.0 kB/s 2023-10-08T16:07:59.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9861e-05 s, 0.0 kB/s 2023-10-08T16:07:59.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.580 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.634 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.01e-05 s, 0.0 kB/s 2023-10-08T16:07:59.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.635 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2297e-05 s, 0.0 kB/s 2023-10-08T16:07:59.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.690 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.745 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7295e-05 s, 0.0 kB/s 2023-10-08T16:07:59.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.745 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.181e-05 s, 0.0 kB/s 2023-10-08T16:07:59.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.801 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9488e-05 s, 0.0 kB/s 2023-10-08T16:07:59.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.856 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.910 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.911 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.911 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.755e-05 s, 0.0 kB/s 2023-10-08T16:07:59.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.911 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:07:59.966 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:07:59.966 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:07:59.966 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.931e-05 s, 0.0 kB/s 2023-10-08T16:07:59.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:07:59.967 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.021 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.021 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.022 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0398e-05 s, 0.0 kB/s 2023-10-08T16:08:00.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.022 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.077 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.077 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.077 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8458e-05 s, 0.0 kB/s 2023-10-08T16:08:00.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.078 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.132 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.132 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.132 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9602e-05 s, 0.0 kB/s 2023-10-08T16:08:00.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.134 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.190 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7549e-05 s, 0.0 kB/s 2023-10-08T16:08:00.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.190 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.246 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1941e-05 s, 0.0 kB/s 2023-10-08T16:08:00.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.246 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.302 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0599e-05 s, 0.0 kB/s 2023-10-08T16:08:00.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.302 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.357 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.357 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.357 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1779e-05 s, 0.0 kB/s 2023-10-08T16:08:00.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.358 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.413 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9612e-05 s, 0.0 kB/s 2023-10-08T16:08:00.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.413 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.469 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.469 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9115e-05 s, 0.0 kB/s 2023-10-08T16:08:00.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.469 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.525 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3945e-05 s, 0.0 kB/s 2023-10-08T16:08:00.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.526 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.581 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.581 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1682e-05 s, 0.0 kB/s 2023-10-08T16:08:00.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.581 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.637 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.7889e-05 s, 0.0 kB/s 2023-10-08T16:08:00.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.638 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.692 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0572e-05 s, 0.0 kB/s 2023-10-08T16:08:00.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.693 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.748 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1048e-05 s, 0.0 kB/s 2023-10-08T16:08:00.748 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.748 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.803 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.804 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.022e-05 s, 0.0 kB/s 2023-10-08T16:08:00.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.804 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.859 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.859 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.859 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7786e-05 s, 0.0 kB/s 2023-10-08T16:08:00.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.860 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.915 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6274e-05 s, 0.0 kB/s 2023-10-08T16:08:00.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0251e-05 s, 0.0 kB/s 2023-10-08T16:08:00.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:00.973 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0783e-05 s, 0.0 kB/s 2023-10-08T16:08:01.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.082 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9775e-05 s, 0.0 kB/s 2023-10-08T16:08:01.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.083 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.138 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7918e-05 s, 0.0 kB/s 2023-10-08T16:08:01.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.138 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.194 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3331e-05 s, 0.0 kB/s 2023-10-08T16:08:01.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.194 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8231e-05 s, 0.0 kB/s 2023-10-08T16:08:01.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.250 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.304 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8896e-05 s, 0.0 kB/s 2023-10-08T16:08:01.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.305 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.359 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.360 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0098e-05 s, 0.0 kB/s 2023-10-08T16:08:01.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.360 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.415 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.415 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.415 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9659e-05 s, 0.0 kB/s 2023-10-08T16:08:01.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.416 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.470 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.470 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.471 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9332e-05 s, 0.0 kB/s 2023-10-08T16:08:01.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.471 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.527 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.528 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8481e-05 s, 0.0 kB/s 2023-10-08T16:08:01.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.528 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.584 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0658e-05 s, 0.0 kB/s 2023-10-08T16:08:01.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.585 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.640 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.640 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.640 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2065e-05 s, 0.0 kB/s 2023-10-08T16:08:01.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.641 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.695 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4518e-05 s, 0.0 kB/s 2023-10-08T16:08:01.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.696 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.751 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.751 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.751 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4701e-05 s, 0.0 kB/s 2023-10-08T16:08:01.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.752 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.806 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.807 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9206e-05 s, 0.0 kB/s 2023-10-08T16:08:01.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.808 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.864 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7477e-05 s, 0.0 kB/s 2023-10-08T16:08:01.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.865 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.919 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3815e-05 s, 0.0 kB/s 2023-10-08T16:08:01.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.920 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:01.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:01.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:01.975 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1687e-05 s, 0.0 kB/s 2023-10-08T16:08:01.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:01.975 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7098e-05 s, 0.0 kB/s 2023-10-08T16:08:02.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.031 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.086 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1069e-05 s, 0.0 kB/s 2023-10-08T16:08:02.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.087 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.143 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1231e-05 s, 0.0 kB/s 2023-10-08T16:08:02.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.144 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.199 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.199 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.199 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-08T16:08:02.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.199 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.255 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1458e-05 s, 0.0 kB/s 2023-10-08T16:08:02.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.255 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.311 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0647e-05 s, 0.0 kB/s 2023-10-08T16:08:02.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.311 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.376 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6094e-05 s, 0.0 kB/s 2023-10-08T16:08:02.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.376 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.442 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4103e-05 s, 0.0 kB/s 2023-10-08T16:08:02.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.443 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.502 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.502 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.502 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1495e-05 s, 0.0 kB/s 2023-10-08T16:08:02.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.502 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1805e-05 s, 0.0 kB/s 2023-10-08T16:08:02.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.559 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.615 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2058e-05 s, 0.0 kB/s 2023-10-08T16:08:02.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.615 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.671 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.131e-05 s, 0.0 kB/s 2023-10-08T16:08:02.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.671 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.727 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0143e-05 s, 0.0 kB/s 2023-10-08T16:08:02.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.783 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-08T16:08:02.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.783 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5141e-05 s, 0.0 kB/s 2023-10-08T16:08:02.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.895 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.138e-05 s, 0.0 kB/s 2023-10-08T16:08:02.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.895 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:02.951 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:02.951 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:02.951 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5312e-05 s, 0.0 kB/s 2023-10-08T16:08:02.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:02.951 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.007 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.007 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.007 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.38e-05 s, 0.0 kB/s 2023-10-08T16:08:03.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.008 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.063 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.063 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.063 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3721e-05 s, 0.0 kB/s 2023-10-08T16:08:03.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.064 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6004e-05 s, 0.0 kB/s 2023-10-08T16:08:03.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.120 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4534e-05 s, 0.0 kB/s 2023-10-08T16:08:03.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.232 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5834e-05 s, 0.0 kB/s 2023-10-08T16:08:03.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.233 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.288 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.288 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1592e-05 s, 0.0 kB/s 2023-10-08T16:08:03.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.288 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.344 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1005e-05 s, 0.0 kB/s 2023-10-08T16:08:03.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.344 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.400 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.408e-05 s, 0.0 kB/s 2023-10-08T16:08:03.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.400 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.456 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1365e-05 s, 0.0 kB/s 2023-10-08T16:08:03.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.456 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.512 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3777e-05 s, 0.0 kB/s 2023-10-08T16:08:03.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.568 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3081e-05 s, 0.0 kB/s 2023-10-08T16:08:03.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.623 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.623 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.623 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8973e-05 s, 0.0 kB/s 2023-10-08T16:08:03.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.624 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.680 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2099e-05 s, 0.0 kB/s 2023-10-08T16:08:03.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.680 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.735 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2247e-05 s, 0.0 kB/s 2023-10-08T16:08:03.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.736 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9163e-05 s, 0.0 kB/s 2023-10-08T16:08:03.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.848 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3159e-05 s, 0.0 kB/s 2023-10-08T16:08:03.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.848 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1069e-05 s, 0.0 kB/s 2023-10-08T16:08:03.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.904 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:03.959 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:03.959 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:03.959 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.166e-05 s, 0.0 kB/s 2023-10-08T16:08:03.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:03.960 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.014 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.014 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.014 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0038e-05 s, 0.0 kB/s 2023-10-08T16:08:04.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.014 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.070 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2009e-05 s, 0.0 kB/s 2023-10-08T16:08:04.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.070 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.126 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9811e-05 s, 0.0 kB/s 2023-10-08T16:08:04.126 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.126 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0957e-05 s, 0.0 kB/s 2023-10-08T16:08:04.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.181 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1642e-05 s, 0.0 kB/s 2023-10-08T16:08:04.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.301 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0922e-05 s, 0.0 kB/s 2023-10-08T16:08:04.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.301 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9955e-05 s, 0.0 kB/s 2023-10-08T16:08:04.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.412 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2506e-05 s, 0.0 kB/s 2023-10-08T16:08:04.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.412 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.468 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7669e-05 s, 0.0 kB/s 2023-10-08T16:08:04.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.469 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.524 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6062e-05 s, 0.0 kB/s 2023-10-08T16:08:04.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.580 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8447e-05 s, 0.0 kB/s 2023-10-08T16:08:04.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.580 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.636 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.636 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5803e-05 s, 0.0 kB/s 2023-10-08T16:08:04.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.636 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.692 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6251e-05 s, 0.0 kB/s 2023-10-08T16:08:04.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.692 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.748 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8556e-05 s, 0.0 kB/s 2023-10-08T16:08:04.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.749 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.805 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.805 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7032e-05 s, 0.0 kB/s 2023-10-08T16:08:04.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.805 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.861 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.861 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0403e-05 s, 0.0 kB/s 2023-10-08T16:08:04.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.861 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.917 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1202e-05 s, 0.0 kB/s 2023-10-08T16:08:04.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:04.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:04.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:04.971 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7256e-05 s, 0.0 kB/s 2023-10-08T16:08:04.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:04.972 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.028 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4387e-05 s, 0.0 kB/s 2023-10-08T16:08:05.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.084 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5998e-05 s, 0.0 kB/s 2023-10-08T16:08:05.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.084 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.140 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.140 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.894e-05 s, 0.0 kB/s 2023-10-08T16:08:05.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.141 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.195 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0907e-05 s, 0.0 kB/s 2023-10-08T16:08:05.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.196 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.250 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.251 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3645e-05 s, 0.0 kB/s 2023-10-08T16:08:05.251 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.251 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.306 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.306 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3912e-05 s, 0.0 kB/s 2023-10-08T16:08:05.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.307 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.362 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.906e-05 s, 0.0 kB/s 2023-10-08T16:08:05.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.363 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8992e-05 s, 0.0 kB/s 2023-10-08T16:08:05.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.418 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.472 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1828e-05 s, 0.0 kB/s 2023-10-08T16:08:05.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.473 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.527 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.528 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.528 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1889e-05 s, 0.0 kB/s 2023-10-08T16:08:05.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.528 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.584 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8383e-05 s, 0.0 kB/s 2023-10-08T16:08:05.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.585 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.639 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.639 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.640 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0528e-05 s, 0.0 kB/s 2023-10-08T16:08:05.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.640 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.695 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7471e-05 s, 0.0 kB/s 2023-10-08T16:08:05.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.696 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.750 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.750 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.750 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9629e-05 s, 0.0 kB/s 2023-10-08T16:08:05.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.751 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.807 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9912e-05 s, 0.0 kB/s 2023-10-08T16:08:05.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.807 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.862 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.863 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2117e-05 s, 0.0 kB/s 2023-10-08T16:08:05.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.863 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.918 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.918 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.918 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1568e-05 s, 0.0 kB/s 2023-10-08T16:08:05.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.919 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:05.973 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:05.973 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:05.973 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2378e-05 s, 0.0 kB/s 2023-10-08T16:08:05.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:05.974 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.029 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.029 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.030 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7882e-05 s, 0.0 kB/s 2023-10-08T16:08:06.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.030 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.086 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.086 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.0895e-05 s, 0.0 kB/s 2023-10-08T16:08:06.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.086 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.141 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9592e-05 s, 0.0 kB/s 2023-10-08T16:08:06.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.142 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.197 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0522e-05 s, 0.0 kB/s 2023-10-08T16:08:06.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.198 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.253 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.254 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.254 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7331e-05 s, 0.0 kB/s 2023-10-08T16:08:06.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.254 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.310 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1067e-05 s, 0.0 kB/s 2023-10-08T16:08:06.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.310 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.365 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.366 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5773e-05 s, 0.0 kB/s 2023-10-08T16:08:06.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.366 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.421 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.421 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.421 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3459e-05 s, 0.0 kB/s 2023-10-08T16:08:06.422 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.422 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.478 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1548e-05 s, 0.0 kB/s 2023-10-08T16:08:06.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.478 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.534 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5219e-05 s, 0.0 kB/s 2023-10-08T16:08:06.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.590 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.591 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8628e-05 s, 0.0 kB/s 2023-10-08T16:08:06.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.591 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.646 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.647 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8662e-05 s, 0.0 kB/s 2023-10-08T16:08:06.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.647 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.702 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.703 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7305e-05 s, 0.0 kB/s 2023-10-08T16:08:06.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.703 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.758 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.758 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.758 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8721e-05 s, 0.0 kB/s 2023-10-08T16:08:06.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.759 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.813 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.813 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.813 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8285e-05 s, 0.0 kB/s 2023-10-08T16:08:06.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.814 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.869 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.870 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3457e-05 s, 0.0 kB/s 2023-10-08T16:08:06.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.870 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.983e-05 s, 0.0 kB/s 2023-10-08T16:08:06.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:06.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:06.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:06.982 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8452e-05 s, 0.0 kB/s 2023-10-08T16:08:06.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:06.982 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9482e-05 s, 0.0 kB/s 2023-10-08T16:08:07.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.038 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.093 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8728e-05 s, 0.0 kB/s 2023-10-08T16:08:07.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9375e-05 s, 0.0 kB/s 2023-10-08T16:08:07.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.149 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8253e-05 s, 0.0 kB/s 2023-10-08T16:08:07.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.204 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.259 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0873e-05 s, 0.0 kB/s 2023-10-08T16:08:07.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.315 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3752e-05 s, 0.0 kB/s 2023-10-08T16:08:07.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1666e-05 s, 0.0 kB/s 2023-10-08T16:08:07.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.371 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.426 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1507e-05 s, 0.0 kB/s 2023-10-08T16:08:07.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.427 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.482 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.482 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.482 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6448e-05 s, 0.0 kB/s 2023-10-08T16:08:07.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.483 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.538 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.538 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.538 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1039e-05 s, 0.0 kB/s 2023-10-08T16:08:07.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.539 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.594 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.594 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1892e-05 s, 0.0 kB/s 2023-10-08T16:08:07.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.594 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.650 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0524e-05 s, 0.0 kB/s 2023-10-08T16:08:07.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.705 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0597e-05 s, 0.0 kB/s 2023-10-08T16:08:07.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.706 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.761 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9711e-05 s, 0.0 kB/s 2023-10-08T16:08:07.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.761 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.817 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0172e-05 s, 0.0 kB/s 2023-10-08T16:08:07.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.817 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.873 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.873 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.873 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6633e-05 s, 0.0 kB/s 2023-10-08T16:08:07.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.873 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1623e-05 s, 0.0 kB/s 2023-10-08T16:08:07.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.929 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:07.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:07.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:07.986 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0576e-05 s, 0.0 kB/s 2023-10-08T16:08:07.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:07.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.041 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8612e-05 s, 0.0 kB/s 2023-10-08T16:08:08.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.042 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.097 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1286e-05 s, 0.0 kB/s 2023-10-08T16:08:08.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.097 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.153 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7758e-05 s, 0.0 kB/s 2023-10-08T16:08:08.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.153 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.209 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1782e-05 s, 0.0 kB/s 2023-10-08T16:08:08.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.209 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.264 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.264 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.265 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1431e-05 s, 0.0 kB/s 2023-10-08T16:08:08.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.265 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.320 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.320 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.320 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9999e-05 s, 0.0 kB/s 2023-10-08T16:08:08.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.321 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.376 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.376 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.376 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0261e-05 s, 0.0 kB/s 2023-10-08T16:08:08.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.377 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5305e-05 s, 0.0 kB/s 2023-10-08T16:08:08.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.434 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.489 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.489 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.490 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7289e-05 s, 0.0 kB/s 2023-10-08T16:08:08.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.490 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.545 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.545 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9468e-05 s, 0.0 kB/s 2023-10-08T16:08:08.546 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.546 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.602 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9059e-05 s, 0.0 kB/s 2023-10-08T16:08:08.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.603 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8258e-05 s, 0.0 kB/s 2023-10-08T16:08:08.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.658 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.714 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0225e-05 s, 0.0 kB/s 2023-10-08T16:08:08.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.770 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5321e-05 s, 0.0 kB/s 2023-10-08T16:08:08.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.770 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.825 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.825 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4185e-05 s, 0.0 kB/s 2023-10-08T16:08:08.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.826 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4991e-05 s, 0.0 kB/s 2023-10-08T16:08:08.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.882 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.937 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0706e-05 s, 0.0 kB/s 2023-10-08T16:08:08.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.938 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:08.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:08.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:08.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7615e-05 s, 0.0 kB/s 2023-10-08T16:08:08.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:08.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0177e-05 s, 0.0 kB/s 2023-10-08T16:08:09.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.048 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.102 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9215e-05 s, 0.0 kB/s 2023-10-08T16:08:09.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.103 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2475e-05 s, 0.0 kB/s 2023-10-08T16:08:09.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.159 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.214 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1795e-05 s, 0.0 kB/s 2023-10-08T16:08:09.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.215 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.270 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.270 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7591e-05 s, 0.0 kB/s 2023-10-08T16:08:09.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.271 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.325 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.325 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.326 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000103281 s, 0.0 kB/s 2023-10-08T16:08:09.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.326 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.411 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9577e-05 s, 0.0 kB/s 2023-10-08T16:08:09.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.411 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8444e-05 s, 0.0 kB/s 2023-10-08T16:08:09.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.427 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.482 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.482 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.483 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1566e-05 s, 0.0 kB/s 2023-10-08T16:08:09.483 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.484 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.539 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.882e-05 s, 0.0 kB/s 2023-10-08T16:08:09.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0979e-05 s, 0.0 kB/s 2023-10-08T16:08:09.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0202e-05 s, 0.0 kB/s 2023-10-08T16:08:09.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.652 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.708 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5622e-05 s, 0.0 kB/s 2023-10-08T16:08:09.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.709 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.765 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1231e-05 s, 0.0 kB/s 2023-10-08T16:08:09.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.766 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9849e-05 s, 0.0 kB/s 2023-10-08T16:08:09.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.821 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1085e-05 s, 0.0 kB/s 2023-10-08T16:08:09.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.932 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0376e-05 s, 0.0 kB/s 2023-10-08T16:08:09.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:09.987 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2894e-05 s, 0.0 kB/s 2023-10-08T16:08:09.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:09.988 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.044 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.044 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.044 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0226e-05 s, 0.0 kB/s 2023-10-08T16:08:10.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.045 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.100 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0798e-05 s, 0.0 kB/s 2023-10-08T16:08:10.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.100 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.156 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7607e-05 s, 0.0 kB/s 2023-10-08T16:08:10.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1626e-05 s, 0.0 kB/s 2023-10-08T16:08:10.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.268 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.031e-05 s, 0.0 kB/s 2023-10-08T16:08:10.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2691e-05 s, 0.0 kB/s 2023-10-08T16:08:10.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.324 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.380 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3811e-05 s, 0.0 kB/s 2023-10-08T16:08:10.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.437 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.438 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8666e-05 s, 0.0 kB/s 2023-10-08T16:08:10.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.438 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3035e-05 s, 0.0 kB/s 2023-10-08T16:08:10.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.495 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1141e-05 s, 0.0 kB/s 2023-10-08T16:08:10.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1565e-05 s, 0.0 kB/s 2023-10-08T16:08:10.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.663 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.663 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.663 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.028e-05 s, 0.0 kB/s 2023-10-08T16:08:10.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.664 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.718 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4872e-05 s, 0.0 kB/s 2023-10-08T16:08:10.719 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.719 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.775 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8115e-05 s, 0.0 kB/s 2023-10-08T16:08:10.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.775 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.831 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.22e-05 s, 0.0 kB/s 2023-10-08T16:08:10.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.927 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8352e-05 s, 0.0 kB/s 2023-10-08T16:08:10.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.928 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.944 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.944 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.944 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7877e-05 s, 0.0 kB/s 2023-10-08T16:08:10.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:10.944 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:10.999 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:10.999 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:10.999 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3905e-05 s, 0.0 kB/s 2023-10-08T16:08:11.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.000 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.056 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2857e-05 s, 0.0 kB/s 2023-10-08T16:08:11.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.056 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.111 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.112 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3718e-05 s, 0.0 kB/s 2023-10-08T16:08:11.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.112 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.168 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.168 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.168 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9411e-05 s, 0.0 kB/s 2023-10-08T16:08:11.169 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.169 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.224 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9958e-05 s, 0.0 kB/s 2023-10-08T16:08:11.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.224 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3908e-05 s, 0.0 kB/s 2023-10-08T16:08:11.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.280 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.336 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.336 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3355e-05 s, 0.0 kB/s 2023-10-08T16:08:11.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0157e-05 s, 0.0 kB/s 2023-10-08T16:08:11.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.392 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.446 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2988e-05 s, 0.0 kB/s 2023-10-08T16:08:11.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.447 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9975e-05 s, 0.0 kB/s 2023-10-08T16:08:11.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.505 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.560 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6093e-05 s, 0.0 kB/s 2023-10-08T16:08:11.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.560 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.615 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3882e-05 s, 0.0 kB/s 2023-10-08T16:08:11.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.616 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.673 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.673 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.673 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9885e-05 s, 0.0 kB/s 2023-10-08T16:08:11.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.674 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.729 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.729 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.729 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1769e-05 s, 0.0 kB/s 2023-10-08T16:08:11.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.730 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.785 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.785 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.718e-05 s, 0.0 kB/s 2023-10-08T16:08:11.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.785 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.841 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.841 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.841 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.00010179 s, 0.0 kB/s 2023-10-08T16:08:11.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.841 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.896 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.896 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7848e-05 s, 0.0 kB/s 2023-10-08T16:08:11.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.896 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:11.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:11.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:11.953 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9993e-05 s, 0.0 kB/s 2023-10-08T16:08:11.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:11.953 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1458e-05 s, 0.0 kB/s 2023-10-08T16:08:12.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.009 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.065 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1315e-05 s, 0.0 kB/s 2023-10-08T16:08:12.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.065 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.121 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.121 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8821e-05 s, 0.0 kB/s 2023-10-08T16:08:12.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.121 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9968e-05 s, 0.0 kB/s 2023-10-08T16:08:12.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.177 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.233 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.235 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0377e-05 s, 0.0 kB/s 2023-10-08T16:08:12.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.235 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.289 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8131e-05 s, 0.0 kB/s 2023-10-08T16:08:12.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.290 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.345 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.345 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1801e-05 s, 0.0 kB/s 2023-10-08T16:08:12.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.345 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.401 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9251e-05 s, 0.0 kB/s 2023-10-08T16:08:12.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.457 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1003e-05 s, 0.0 kB/s 2023-10-08T16:08:12.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.457 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.513 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0129e-05 s, 0.0 kB/s 2023-10-08T16:08:12.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.569 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.569 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9241e-05 s, 0.0 kB/s 2023-10-08T16:08:12.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.569 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.625 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5699e-05 s, 0.0 kB/s 2023-10-08T16:08:12.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.681 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5428e-05 s, 0.0 kB/s 2023-10-08T16:08:12.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.682 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.737 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2442e-05 s, 0.0 kB/s 2023-10-08T16:08:12.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.737 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3878e-05 s, 0.0 kB/s 2023-10-08T16:08:12.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.793 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.849 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7826e-05 s, 0.0 kB/s 2023-10-08T16:08:12.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.850 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3493e-05 s, 0.0 kB/s 2023-10-08T16:08:12.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.906 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:12.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.536e-05 s, 0.0 kB/s 2023-10-08T16:08:12.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:12.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.018 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3248e-05 s, 0.0 kB/s 2023-10-08T16:08:13.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.074 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2441e-05 s, 0.0 kB/s 2023-10-08T16:08:13.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.074 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.129 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7072e-05 s, 0.0 kB/s 2023-10-08T16:08:13.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.130 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.186 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7455e-05 s, 0.0 kB/s 2023-10-08T16:08:13.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.186 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.242 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4728e-05 s, 0.0 kB/s 2023-10-08T16:08:13.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.242 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.297 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1392e-05 s, 0.0 kB/s 2023-10-08T16:08:13.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.297 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0778e-05 s, 0.0 kB/s 2023-10-08T16:08:13.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.353 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.399 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:08:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.408 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.409 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9728e-05 s, 0.0 kB/s 2023-10-08T16:08:13.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.409 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.409 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:08:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.465 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.237e-05 s, 0.0 kB/s 2023-10-08T16:08:13.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.466 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.521 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.048e-05 s, 0.0 kB/s 2023-10-08T16:08:13.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.521 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.577 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6787e-05 s, 0.0 kB/s 2023-10-08T16:08:13.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.577 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.632 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.633 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1476e-05 s, 0.0 kB/s 2023-10-08T16:08:13.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.633 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.689 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1005e-05 s, 0.0 kB/s 2023-10-08T16:08:13.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.689 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.745 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.745 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.745 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-10-08T16:08:13.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.746 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.801 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9955e-05 s, 0.0 kB/s 2023-10-08T16:08:13.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.801 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.857 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1882e-05 s, 0.0 kB/s 2023-10-08T16:08:13.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.858 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.914 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0044e-05 s, 0.0 kB/s 2023-10-08T16:08:13.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.914 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:13.969 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8255e-05 s, 0.0 kB/s 2023-10-08T16:08:13.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:13.970 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.024 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.025 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.025 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2341e-05 s, 0.0 kB/s 2023-10-08T16:08:14.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.025 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.080 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.080 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.080 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1041e-05 s, 0.0 kB/s 2023-10-08T16:08:14.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.081 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.136 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.136 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.136 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6595e-05 s, 0.0 kB/s 2023-10-08T16:08:14.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.137 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.192 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.193 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9466e-05 s, 0.0 kB/s 2023-10-08T16:08:14.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.193 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.248 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9297e-05 s, 0.0 kB/s 2023-10-08T16:08:14.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.249 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.305 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.305 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9581e-05 s, 0.0 kB/s 2023-10-08T16:08:14.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.305 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.360 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.361 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.009e-05 s, 0.0 kB/s 2023-10-08T16:08:14.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.361 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8556e-05 s, 0.0 kB/s 2023-10-08T16:08:14.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.417 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.474 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9426e-05 s, 0.0 kB/s 2023-10-08T16:08:14.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.474 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.530 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0615e-05 s, 0.0 kB/s 2023-10-08T16:08:14.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.531 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1618e-05 s, 0.0 kB/s 2023-10-08T16:08:14.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.643 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.169e-05 s, 0.0 kB/s 2023-10-08T16:08:14.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2101e-05 s, 0.0 kB/s 2023-10-08T16:08:14.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.700 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9709e-05 s, 0.0 kB/s 2023-10-08T16:08:14.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.755 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1352e-05 s, 0.0 kB/s 2023-10-08T16:08:14.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0335e-05 s, 0.0 kB/s 2023-10-08T16:08:14.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9997e-05 s, 0.0 kB/s 2023-10-08T16:08:14.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:14.978 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1321e-05 s, 0.0 kB/s 2023-10-08T16:08:14.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:14.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.035 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0657e-05 s, 0.0 kB/s 2023-10-08T16:08:15.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.035 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0377e-05 s, 0.0 kB/s 2023-10-08T16:08:15.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.091 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.146 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7961e-05 s, 0.0 kB/s 2023-10-08T16:08:15.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.147 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.203 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.252e-05 s, 0.0 kB/s 2023-10-08T16:08:15.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.258 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8764e-05 s, 0.0 kB/s 2023-10-08T16:08:15.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6013e-05 s, 0.0 kB/s 2023-10-08T16:08:15.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.314 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.2387e-05 s, 0.0 kB/s 2023-10-08T16:08:15.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.425 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6174e-05 s, 0.0 kB/s 2023-10-08T16:08:15.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.425 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8496e-05 s, 0.0 kB/s 2023-10-08T16:08:15.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.481 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.537 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.537 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1025e-05 s, 0.0 kB/s 2023-10-08T16:08:15.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.593 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8167e-05 s, 0.0 kB/s 2023-10-08T16:08:15.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.593 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0655e-05 s, 0.0 kB/s 2023-10-08T16:08:15.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.649 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.705 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.705 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9005e-05 s, 0.0 kB/s 2023-10-08T16:08:15.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.705 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.761 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.761 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.761 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1219e-05 s, 0.0 kB/s 2023-10-08T16:08:15.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.761 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.816 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1655e-05 s, 0.0 kB/s 2023-10-08T16:08:15.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.817 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.874 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0101e-05 s, 0.0 kB/s 2023-10-08T16:08:15.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.875 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.929 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.930 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.598e-05 s, 0.0 kB/s 2023-10-08T16:08:15.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.930 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:15.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:15.986 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7929e-05 s, 0.0 kB/s 2023-10-08T16:08:15.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:15.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8319e-05 s, 0.0 kB/s 2023-10-08T16:08:16.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.096 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8808e-05 s, 0.0 kB/s 2023-10-08T16:08:16.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.097 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.153 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0232e-05 s, 0.0 kB/s 2023-10-08T16:08:16.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.153 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.208 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8046e-05 s, 0.0 kB/s 2023-10-08T16:08:16.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.209 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.264 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8913e-05 s, 0.0 kB/s 2023-10-08T16:08:16.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.264 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8202e-05 s, 0.0 kB/s 2023-10-08T16:08:16.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.320 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.375 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0652e-05 s, 0.0 kB/s 2023-10-08T16:08:16.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.376 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0634e-05 s, 0.0 kB/s 2023-10-08T16:08:16.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.431 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0969e-05 s, 0.0 kB/s 2023-10-08T16:08:16.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.486 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.542 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1511e-05 s, 0.0 kB/s 2023-10-08T16:08:16.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.543 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.597 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2114e-05 s, 0.0 kB/s 2023-10-08T16:08:16.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.599 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.654 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.655 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9701e-05 s, 0.0 kB/s 2023-10-08T16:08:16.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.655 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.077e-05 s, 0.0 kB/s 2023-10-08T16:08:16.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.766 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.767 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9923e-05 s, 0.0 kB/s 2023-10-08T16:08:16.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.767 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.823 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8604e-05 s, 0.0 kB/s 2023-10-08T16:08:16.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.823 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.879 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2271e-05 s, 0.0 kB/s 2023-10-08T16:08:16.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.879 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.934 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.934 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.934 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.149e-05 s, 0.0 kB/s 2023-10-08T16:08:16.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.935 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:16.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:16.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:16.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0232e-05 s, 0.0 kB/s 2023-10-08T16:08:16.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:16.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:17.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:17.048 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9442e-05 s, 0.0 kB/s 2023-10-08T16:08:17.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:17.048 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:17.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:17.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:17.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8801e-05 s, 0.0 kB/s 2023-10-08T16:08:17.877 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:17.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:17.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:17.930 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:17.930 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1428e-05 s, 0.0 kB/s 2023-10-08T16:08:17.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:17.931 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:17.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:17.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:17.953 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6743e-05 s, 0.0 kB/s 2023-10-08T16:08:17.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:17.953 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0308e-05 s, 0.0 kB/s 2023-10-08T16:08:18.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.010 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.066 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5281e-05 s, 0.0 kB/s 2023-10-08T16:08:18.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.067 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1439e-05 s, 0.0 kB/s 2023-10-08T16:08:18.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.178 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.178 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.5402e-05 s, 0.0 kB/s 2023-10-08T16:08:18.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.178 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.234 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9309e-05 s, 0.0 kB/s 2023-10-08T16:08:18.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.234 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.290 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.290 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.290 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9773e-05 s, 0.0 kB/s 2023-10-08T16:08:18.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.291 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.346 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.346 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.346 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0149e-05 s, 0.0 kB/s 2023-10-08T16:08:18.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.347 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.401 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.402 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6969e-05 s, 0.0 kB/s 2023-10-08T16:08:18.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.402 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.458 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8967e-05 s, 0.0 kB/s 2023-10-08T16:08:18.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.459 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.514 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.515 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4445e-05 s, 0.0 kB/s 2023-10-08T16:08:18.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.515 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1384e-05 s, 0.0 kB/s 2023-10-08T16:08:18.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.627 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6469e-05 s, 0.0 kB/s 2023-10-08T16:08:18.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.627 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8048e-05 s, 0.0 kB/s 2023-10-08T16:08:18.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.683 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.739 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9641e-05 s, 0.0 kB/s 2023-10-08T16:08:18.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.739 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.795 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.795 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7616e-05 s, 0.0 kB/s 2023-10-08T16:08:18.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.981e-05 s, 0.0 kB/s 2023-10-08T16:08:18.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.907 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.907 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.907 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2089e-05 s, 0.0 kB/s 2023-10-08T16:08:18.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:18.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:18.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:18.963 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1189e-05 s, 0.0 kB/s 2023-10-08T16:08:18.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:18.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.018 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-08T16:08:19.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.019 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.073 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-10-08T16:08:19.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.074 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.128 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.129 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4454e-05 s, 0.0 kB/s 2023-10-08T16:08:19.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.129 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.185 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0394e-05 s, 0.0 kB/s 2023-10-08T16:08:19.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.185 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.240 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.241 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9851e-05 s, 0.0 kB/s 2023-10-08T16:08:19.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.241 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.296 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.296 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.296 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1602e-05 s, 0.0 kB/s 2023-10-08T16:08:19.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.297 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.352 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7482e-05 s, 0.0 kB/s 2023-10-08T16:08:19.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.352 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.408 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0054e-05 s, 0.0 kB/s 2023-10-08T16:08:19.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.408 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.464 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9229e-05 s, 0.0 kB/s 2023-10-08T16:08:19.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.464 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.520 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.520 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.520 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.98e-05 s, 0.0 kB/s 2023-10-08T16:08:19.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.521 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.576 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8693e-05 s, 0.0 kB/s 2023-10-08T16:08:19.576 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.576 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.632 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.632 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.632 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8132e-05 s, 0.0 kB/s 2023-10-08T16:08:19.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.632 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.688 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.688 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.688 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.253e-05 s, 0.0 kB/s 2023-10-08T16:08:19.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.688 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.744 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2321e-05 s, 0.0 kB/s 2023-10-08T16:08:19.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.744 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.799 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.799 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.799 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9599e-05 s, 0.0 kB/s 2023-10-08T16:08:19.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.800 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.855 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.856 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8972e-05 s, 0.0 kB/s 2023-10-08T16:08:19.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.856 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.911 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9588e-05 s, 0.0 kB/s 2023-10-08T16:08:19.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.912 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:19.968 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:19.968 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:19.969 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0172e-05 s, 0.0 kB/s 2023-10-08T16:08:19.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:19.969 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.024 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4452e-05 s, 0.0 kB/s 2023-10-08T16:08:20.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.025 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.081 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8676e-05 s, 0.0 kB/s 2023-10-08T16:08:20.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.081 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.136 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.137 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.137 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8534e-05 s, 0.0 kB/s 2023-10-08T16:08:20.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.137 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.193 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.193 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1072e-05 s, 0.0 kB/s 2023-10-08T16:08:20.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.194 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.248 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.248 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6514e-05 s, 0.0 kB/s 2023-10-08T16:08:20.249 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.249 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.305 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1875e-05 s, 0.0 kB/s 2023-10-08T16:08:20.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.305 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.360 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.361 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.361 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0296e-05 s, 0.0 kB/s 2023-10-08T16:08:20.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.362 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.416 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6298e-05 s, 0.0 kB/s 2023-10-08T16:08:20.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.417 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.472 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.472 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.473 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9708e-05 s, 0.0 kB/s 2023-10-08T16:08:20.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.473 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.529 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.152e-05 s, 0.0 kB/s 2023-10-08T16:08:20.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.530 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.584 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.585 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4028e-05 s, 0.0 kB/s 2023-10-08T16:08:20.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.585 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.032e-05 s, 0.0 kB/s 2023-10-08T16:08:20.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.684 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9439e-05 s, 0.0 kB/s 2023-10-08T16:08:20.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.700 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9882e-05 s, 0.0 kB/s 2023-10-08T16:08:20.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.756 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.810 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1571e-05 s, 0.0 kB/s 2023-10-08T16:08:20.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.870 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.032e-05 s, 0.0 kB/s 2023-10-08T16:08:20.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.871 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.926 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0035e-05 s, 0.0 kB/s 2023-10-08T16:08:20.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:20.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:20.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:20.982 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4421e-05 s, 0.0 kB/s 2023-10-08T16:08:20.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:20.983 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.1999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.038 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1582e-05 s, 0.0 kB/s 2023-10-08T16:08:21.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.039 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2000 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.094 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.094 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.094 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-10-08T16:08:21.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2001 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.148 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.149 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2871e-05 s, 0.0 kB/s 2023-10-08T16:08:21.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.150 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2002 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.205 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0034e-05 s, 0.0 kB/s 2023-10-08T16:08:21.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.206 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2003 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1294e-05 s, 0.0 kB/s 2023-10-08T16:08:21.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.260 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2004 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.316 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.5849e-05 s, 0.0 kB/s 2023-10-08T16:08:21.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2005 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1747e-05 s, 0.0 kB/s 2023-10-08T16:08:21.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.372 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2006 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6285e-05 s, 0.0 kB/s 2023-10-08T16:08:21.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.428 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2007 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0471e-05 s, 0.0 kB/s 2023-10-08T16:08:21.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.484 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2008 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5255e-05 s, 0.0 kB/s 2023-10-08T16:08:21.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.540 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2009 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.596 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9918e-05 s, 0.0 kB/s 2023-10-08T16:08:21.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2010 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0711e-05 s, 0.0 kB/s 2023-10-08T16:08:21.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.652 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2011 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.708 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.708 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0497e-05 s, 0.0 kB/s 2023-10-08T16:08:21.709 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.709 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2012 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.763 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.764 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.764 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4448e-05 s, 0.0 kB/s 2023-10-08T16:08:21.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.765 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2013 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.820 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2785e-05 s, 0.0 kB/s 2023-10-08T16:08:21.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.821 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2014 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.876 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.876 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.876 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7249e-05 s, 0.0 kB/s 2023-10-08T16:08:21.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2015 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.932 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3533e-05 s, 0.0 kB/s 2023-10-08T16:08:21.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.933 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2016 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:21.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:21.987 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:21.988 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9245e-05 s, 0.0 kB/s 2023-10-08T16:08:21.988 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:21.988 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2017 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.043 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.043 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.043 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.016e-05 s, 0.0 kB/s 2023-10-08T16:08:22.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.044 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2018 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.100 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7515e-05 s, 0.0 kB/s 2023-10-08T16:08:22.100 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2019 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.155 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.156 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9111e-05 s, 0.0 kB/s 2023-10-08T16:08:22.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.156 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2020 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.211 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.211 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7961e-05 s, 0.0 kB/s 2023-10-08T16:08:22.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2021 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.451e-05 s, 0.0 kB/s 2023-10-08T16:08:22.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2022 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0452e-05 s, 0.0 kB/s 2023-10-08T16:08:22.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2023 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.380 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.380 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6015e-05 s, 0.0 kB/s 2023-10-08T16:08:22.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.381 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2024 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.436 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9381e-05 s, 0.0 kB/s 2023-10-08T16:08:22.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.437 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2025 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.492 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1203e-05 s, 0.0 kB/s 2023-10-08T16:08:22.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.493 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2026 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2537e-05 s, 0.0 kB/s 2023-10-08T16:08:22.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.549 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2027 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0285e-05 s, 0.0 kB/s 2023-10-08T16:08:22.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2028 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.661 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8078e-05 s, 0.0 kB/s 2023-10-08T16:08:22.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2029 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.716 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8996e-05 s, 0.0 kB/s 2023-10-08T16:08:22.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.717 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2030 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.772 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9265e-05 s, 0.0 kB/s 2023-10-08T16:08:22.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.772 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2031 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.827 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1676e-05 s, 0.0 kB/s 2023-10-08T16:08:22.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.828 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2032 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3731e-05 s, 0.0 kB/s 2023-10-08T16:08:22.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.883 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2033 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9776e-05 s, 0.0 kB/s 2023-10-08T16:08:22.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.940 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2034 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:22.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:22.995 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:22.995 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.989e-05 s, 0.0 kB/s 2023-10-08T16:08:22.996 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:22.996 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2035 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.050 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.051 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9185e-05 s, 0.0 kB/s 2023-10-08T16:08:23.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2036 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.107 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3483e-05 s, 0.0 kB/s 2023-10-08T16:08:23.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2037 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.163 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1204e-05 s, 0.0 kB/s 2023-10-08T16:08:23.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.163 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2038 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7471e-05 s, 0.0 kB/s 2023-10-08T16:08:23.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.219 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2039 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.273 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1882e-05 s, 0.0 kB/s 2023-10-08T16:08:23.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.274 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2040 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.330 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8543e-05 s, 0.0 kB/s 2023-10-08T16:08:23.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2041 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.386 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0018e-05 s, 0.0 kB/s 2023-10-08T16:08:23.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.386 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2042 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8651e-05 s, 0.0 kB/s 2023-10-08T16:08:23.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2043 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.497 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-10-08T16:08:23.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.498 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2044 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.552 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.553 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.066e-05 s, 0.0 kB/s 2023-10-08T16:08:23.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.553 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2045 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.608 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.608 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.608 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9197e-05 s, 0.0 kB/s 2023-10-08T16:08:23.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.609 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2046 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0534e-05 s, 0.0 kB/s 2023-10-08T16:08:23.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.665 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2047 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.720 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0411e-05 s, 0.0 kB/s 2023-10-08T16:08:23.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.721 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2048 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.775 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0202e-05 s, 0.0 kB/s 2023-10-08T16:08:23.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.776 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2049 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.832 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3808e-05 s, 0.0 kB/s 2023-10-08T16:08:23.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.832 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2050 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0553e-05 s, 0.0 kB/s 2023-10-08T16:08:23.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.888 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2051 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.943 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.943 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.943 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.559e-05 s, 0.0 kB/s 2023-10-08T16:08:23.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.943 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2052 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:23.999 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:23.999 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:23.999 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9002e-05 s, 0.0 kB/s 2023-10-08T16:08:23.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:23.999 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2053 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.055 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9825e-05 s, 0.0 kB/s 2023-10-08T16:08:24.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.055 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2054 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0895e-05 s, 0.0 kB/s 2023-10-08T16:08:24.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.111 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2055 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.165 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0353e-05 s, 0.0 kB/s 2023-10-08T16:08:24.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2056 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.222 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9874e-05 s, 0.0 kB/s 2023-10-08T16:08:24.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.223 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2057 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2357e-05 s, 0.0 kB/s 2023-10-08T16:08:24.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2058 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.335 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0124e-05 s, 0.0 kB/s 2023-10-08T16:08:24.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2059 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2755e-05 s, 0.0 kB/s 2023-10-08T16:08:24.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.391 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2060 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.447 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9013e-05 s, 0.0 kB/s 2023-10-08T16:08:24.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.447 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2061 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9493e-05 s, 0.0 kB/s 2023-10-08T16:08:24.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.503 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2062 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.559 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.559 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3078e-05 s, 0.0 kB/s 2023-10-08T16:08:24.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.559 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2063 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.615 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.615 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9213e-05 s, 0.0 kB/s 2023-10-08T16:08:24.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.616 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2064 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.671 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.671 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9784e-05 s, 0.0 kB/s 2023-10-08T16:08:24.671 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.671 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2065 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.727 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.727 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3778e-05 s, 0.0 kB/s 2023-10-08T16:08:24.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.728 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2066 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.782 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.783 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7638e-05 s, 0.0 kB/s 2023-10-08T16:08:24.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.783 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2067 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9242e-05 s, 0.0 kB/s 2023-10-08T16:08:24.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.839 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2068 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2118e-05 s, 0.0 kB/s 2023-10-08T16:08:24.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.896 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2069 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1611e-05 s, 0.0 kB/s 2023-10-08T16:08:24.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:24.952 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2070 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.008 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4001e-05 s, 0.0 kB/s 2023-10-08T16:08:25.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.009 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2071 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7771e-05 s, 0.0 kB/s 2023-10-08T16:08:25.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.064 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2072 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.120 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8202e-05 s, 0.0 kB/s 2023-10-08T16:08:25.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.120 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2073 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.5937e-05 s, 0.0 kB/s 2023-10-08T16:08:25.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2074 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.232 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9991e-05 s, 0.0 kB/s 2023-10-08T16:08:25.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.233 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2075 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.287 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0502e-05 s, 0.0 kB/s 2023-10-08T16:08:25.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.288 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2076 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4352e-05 s, 0.0 kB/s 2023-10-08T16:08:25.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.344 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2077 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.399 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.400 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9731e-05 s, 0.0 kB/s 2023-10-08T16:08:25.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.400 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2078 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.455 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.456 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3595e-05 s, 0.0 kB/s 2023-10-08T16:08:25.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.456 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2079 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.512 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1597e-05 s, 0.0 kB/s 2023-10-08T16:08:25.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2080 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.567 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.568 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.568 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6551e-05 s, 0.0 kB/s 2023-10-08T16:08:25.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.568 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2081 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.624 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0191e-05 s, 0.0 kB/s 2023-10-08T16:08:25.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.625 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2082 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.679 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0561e-05 s, 0.0 kB/s 2023-10-08T16:08:25.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.680 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2083 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.736 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.736 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.678e-05 s, 0.0 kB/s 2023-10-08T16:08:25.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.736 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2084 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.791 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7749e-05 s, 0.0 kB/s 2023-10-08T16:08:25.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.792 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2085 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.849 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.849 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3698e-05 s, 0.0 kB/s 2023-10-08T16:08:25.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2086 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.905 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1088e-05 s, 0.0 kB/s 2023-10-08T16:08:25.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.905 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2087 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1168e-05 s, 0.0 kB/s 2023-10-08T16:08:25.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:25.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2088 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.018 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7109e-05 s, 0.0 kB/s 2023-10-08T16:08:26.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2089 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.074 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7541e-05 s, 0.0 kB/s 2023-10-08T16:08:26.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.074 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2090 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.129 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-08T16:08:26.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.130 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2091 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.185 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8362e-05 s, 0.0 kB/s 2023-10-08T16:08:26.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.185 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2092 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.240 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.240 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.241 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4429e-05 s, 0.0 kB/s 2023-10-08T16:08:26.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.242 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2093 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.317 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7358e-05 s, 0.0 kB/s 2023-10-08T16:08:26.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.318 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2094 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.372 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0468e-05 s, 0.0 kB/s 2023-10-08T16:08:26.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.374 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2095 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1594e-05 s, 0.0 kB/s 2023-10-08T16:08:26.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.430 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2096 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9244e-05 s, 0.0 kB/s 2023-10-08T16:08:26.485 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.485 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2097 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0187e-05 s, 0.0 kB/s 2023-10-08T16:08:26.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.541 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2098 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.599 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.599 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1387e-05 s, 0.0 kB/s 2023-10-08T16:08:26.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.600 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2099 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.655 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0301e-05 s, 0.0 kB/s 2023-10-08T16:08:26.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.655 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2100 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.710 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4366e-05 s, 0.0 kB/s 2023-10-08T16:08:26.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2101 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0078e-05 s, 0.0 kB/s 2023-10-08T16:08:26.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2102 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.824 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7498e-05 s, 0.0 kB/s 2023-10-08T16:08:26.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.824 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2103 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1401e-05 s, 0.0 kB/s 2023-10-08T16:08:26.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.880 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2104 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.936 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7784e-05 s, 0.0 kB/s 2023-10-08T16:08:26.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2105 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:26.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:26.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:26.992 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0132e-05 s, 0.0 kB/s 2023-10-08T16:08:26.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:26.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2106 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8269e-05 s, 0.0 kB/s 2023-10-08T16:08:27.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.049 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2107 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.105 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.105 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8278e-05 s, 0.0 kB/s 2023-10-08T16:08:27.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2108 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.161 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.161 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-10-08T16:08:27.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.161 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2109 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.217 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.217 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8948e-05 s, 0.0 kB/s 2023-10-08T16:08:27.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.218 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2110 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.273 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9233e-05 s, 0.0 kB/s 2023-10-08T16:08:27.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.273 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2111 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.328 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.328 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1613e-05 s, 0.0 kB/s 2023-10-08T16:08:27.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.329 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2112 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.384 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.384 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8334e-05 s, 0.0 kB/s 2023-10-08T16:08:27.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2113 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.439 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.439 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3228e-05 s, 0.0 kB/s 2023-10-08T16:08:27.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2114 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.497 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1193e-05 s, 0.0 kB/s 2023-10-08T16:08:27.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.497 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2115 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.553 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8182e-05 s, 0.0 kB/s 2023-10-08T16:08:27.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.553 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2116 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.608 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.608 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.609 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3036e-05 s, 0.0 kB/s 2023-10-08T16:08:27.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.609 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2117 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9163e-05 s, 0.0 kB/s 2023-10-08T16:08:27.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.665 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2118 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9365e-05 s, 0.0 kB/s 2023-10-08T16:08:27.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.720 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2119 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.775 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6584e-05 s, 0.0 kB/s 2023-10-08T16:08:27.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.776 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2120 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.832 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2097e-05 s, 0.0 kB/s 2023-10-08T16:08:27.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.832 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2121 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.886 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2191e-05 s, 0.0 kB/s 2023-10-08T16:08:27.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.887 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2122 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9159e-05 s, 0.0 kB/s 2023-10-08T16:08:27.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2123 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:27.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:27.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:27.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1421e-05 s, 0.0 kB/s 2023-10-08T16:08:27.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:27.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2124 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.051 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.051 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5595e-05 s, 0.0 kB/s 2023-10-08T16:08:28.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.052 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2125 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.107 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.107 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9365e-05 s, 0.0 kB/s 2023-10-08T16:08:28.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2126 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.163 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5978e-05 s, 0.0 kB/s 2023-10-08T16:08:28.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.163 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2127 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.219 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6541e-05 s, 0.0 kB/s 2023-10-08T16:08:28.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.219 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2128 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.274 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.274 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.4245e-05 s, 0.0 kB/s 2023-10-08T16:08:28.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.275 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2129 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.330 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1646e-05 s, 0.0 kB/s 2023-10-08T16:08:28.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2130 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.385 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.386 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5194e-05 s, 0.0 kB/s 2023-10-08T16:08:28.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.386 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2131 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.441 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.442 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5322e-05 s, 0.0 kB/s 2023-10-08T16:08:28.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.442 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2132 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.498 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.724e-05 s, 0.0 kB/s 2023-10-08T16:08:28.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.499 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2133 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.554 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.234e-05 s, 0.0 kB/s 2023-10-08T16:08:28.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.554 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2134 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.609 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9749e-05 s, 0.0 kB/s 2023-10-08T16:08:28.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.610 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2135 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.664 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.047e-05 s, 0.0 kB/s 2023-10-08T16:08:28.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.665 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2136 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6899e-05 s, 0.0 kB/s 2023-10-08T16:08:28.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.720 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2137 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.775 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.776 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9776e-05 s, 0.0 kB/s 2023-10-08T16:08:28.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.776 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2138 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.831 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7068e-05 s, 0.0 kB/s 2023-10-08T16:08:28.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.833 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2139 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.888 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.888 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.888 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0272e-05 s, 0.0 kB/s 2023-10-08T16:08:28.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.889 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2140 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:28.944 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:28.944 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:28.945 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2037e-05 s, 0.0 kB/s 2023-10-08T16:08:28.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:28.945 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2141 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.000 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.001 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3929e-05 s, 0.0 kB/s 2023-10-08T16:08:29.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.001 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2142 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.057 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3234e-05 s, 0.0 kB/s 2023-10-08T16:08:29.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.057 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2143 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.112 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.112 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4261e-05 s, 0.0 kB/s 2023-10-08T16:08:29.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.113 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2144 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.170 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.985e-05 s, 0.0 kB/s 2023-10-08T16:08:29.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.170 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2145 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.226 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7706e-05 s, 0.0 kB/s 2023-10-08T16:08:29.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.226 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2146 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.281 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.282 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9103e-05 s, 0.0 kB/s 2023-10-08T16:08:29.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.282 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2147 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.338 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0262e-05 s, 0.0 kB/s 2023-10-08T16:08:29.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.338 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2148 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.394 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.394 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0643e-05 s, 0.0 kB/s 2023-10-08T16:08:29.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.394 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2149 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.450 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.450 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8694e-05 s, 0.0 kB/s 2023-10-08T16:08:29.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.450 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2150 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.506 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7298e-05 s, 0.0 kB/s 2023-10-08T16:08:29.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.507 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2151 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.562 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.562 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9702e-05 s, 0.0 kB/s 2023-10-08T16:08:29.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.563 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2152 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.619 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.619 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7674e-05 s, 0.0 kB/s 2023-10-08T16:08:29.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.619 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2153 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.675 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8884e-05 s, 0.0 kB/s 2023-10-08T16:08:29.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.675 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2154 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4145e-05 s, 0.0 kB/s 2023-10-08T16:08:29.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.731 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2155 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1752e-05 s, 0.0 kB/s 2023-10-08T16:08:29.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2156 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.842 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.842 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.842 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1502e-05 s, 0.0 kB/s 2023-10-08T16:08:29.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2157 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.897 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.898 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8621e-05 s, 0.0 kB/s 2023-10-08T16:08:29.898 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.898 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2158 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:29.953 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:29.953 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:29.953 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1608e-05 s, 0.0 kB/s 2023-10-08T16:08:29.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:29.954 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2159 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.010 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.767e-05 s, 0.0 kB/s 2023-10-08T16:08:30.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.011 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2160 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.065 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.066 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2104e-05 s, 0.0 kB/s 2023-10-08T16:08:30.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.066 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2161 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9646e-05 s, 0.0 kB/s 2023-10-08T16:08:30.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.123 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2162 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.179 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8478e-05 s, 0.0 kB/s 2023-10-08T16:08:30.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.179 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2163 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9777e-05 s, 0.0 kB/s 2023-10-08T16:08:30.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2164 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1811e-05 s, 0.0 kB/s 2023-10-08T16:08:30.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.293 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2165 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.347 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2021e-05 s, 0.0 kB/s 2023-10-08T16:08:30.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.348 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2166 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.403 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9849e-05 s, 0.0 kB/s 2023-10-08T16:08:30.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.404 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2167 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.458 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1303e-05 s, 0.0 kB/s 2023-10-08T16:08:30.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.459 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2168 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.516 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0871e-05 s, 0.0 kB/s 2023-10-08T16:08:30.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.516 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2169 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.572 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.572 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.572 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0323e-05 s, 0.0 kB/s 2023-10-08T16:08:30.573 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.573 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2170 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.628 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.628 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.628 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.4735e-05 s, 0.0 kB/s 2023-10-08T16:08:30.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.628 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2171 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1914e-05 s, 0.0 kB/s 2023-10-08T16:08:30.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.684 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2172 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.739 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9745e-05 s, 0.0 kB/s 2023-10-08T16:08:30.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.739 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2173 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.794 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7649e-05 s, 0.0 kB/s 2023-10-08T16:08:30.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2174 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.851 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9301e-05 s, 0.0 kB/s 2023-10-08T16:08:30.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2175 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.907 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.907 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.907 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5072e-05 s, 0.0 kB/s 2023-10-08T16:08:30.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.908 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2176 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:30.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0428e-05 s, 0.0 kB/s 2023-10-08T16:08:30.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:30.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2177 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.019 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1484e-05 s, 0.0 kB/s 2023-10-08T16:08:31.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.020 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2178 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.075 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0371e-05 s, 0.0 kB/s 2023-10-08T16:08:31.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.075 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2179 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.131 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.131 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9087e-05 s, 0.0 kB/s 2023-10-08T16:08:31.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.131 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2180 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.187 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4072e-05 s, 0.0 kB/s 2023-10-08T16:08:31.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.187 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2181 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.242 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8671e-05 s, 0.0 kB/s 2023-10-08T16:08:31.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.243 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2182 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.298 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9737e-05 s, 0.0 kB/s 2023-10-08T16:08:31.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.298 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2183 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8366e-05 s, 0.0 kB/s 2023-10-08T16:08:31.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.354 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2184 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8634e-05 s, 0.0 kB/s 2023-10-08T16:08:31.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.431 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2185 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-08T16:08:31.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.486 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2186 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1302e-05 s, 0.0 kB/s 2023-10-08T16:08:31.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.542 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2187 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.597 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.597 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.597 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0132e-05 s, 0.0 kB/s 2023-10-08T16:08:31.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.598 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2188 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.652 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.653 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.653 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.9703e-05 s, 0.0 kB/s 2023-10-08T16:08:31.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.653 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2189 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.709 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.709 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1665e-05 s, 0.0 kB/s 2023-10-08T16:08:31.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.710 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2190 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.765 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.765 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9095e-05 s, 0.0 kB/s 2023-10-08T16:08:31.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.766 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2191 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9913e-05 s, 0.0 kB/s 2023-10-08T16:08:31.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.822 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2192 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000101561 s, 0.0 kB/s 2023-10-08T16:08:31.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.878 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2193 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1728e-05 s, 0.0 kB/s 2023-10-08T16:08:31.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.934 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2194 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:31.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:31.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:31.988 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1392e-05 s, 0.0 kB/s 2023-10-08T16:08:31.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:31.989 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2195 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9681e-05 s, 0.0 kB/s 2023-10-08T16:08:32.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.046 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2196 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.101 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-10-08T16:08:32.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2197 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.157 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.157 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0189e-05 s, 0.0 kB/s 2023-10-08T16:08:32.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.158 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2198 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000101661 s, 0.0 kB/s 2023-10-08T16:08:32.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.213 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2199 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0111e-05 s, 0.0 kB/s 2023-10-08T16:08:32.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2200 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4298e-05 s, 0.0 kB/s 2023-10-08T16:08:32.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2201 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.381 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.381 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.381 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9641e-05 s, 0.0 kB/s 2023-10-08T16:08:32.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.381 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2202 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.436 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.436 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1258e-05 s, 0.0 kB/s 2023-10-08T16:08:32.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.437 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2203 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.492 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3843e-05 s, 0.0 kB/s 2023-10-08T16:08:32.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.493 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2204 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.549 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2532e-05 s, 0.0 kB/s 2023-10-08T16:08:32.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.549 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2205 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.605 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.605 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7333e-05 s, 0.0 kB/s 2023-10-08T16:08:32.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2206 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.661 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2526e-05 s, 0.0 kB/s 2023-10-08T16:08:32.661 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2207 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.717 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7041e-05 s, 0.0 kB/s 2023-10-08T16:08:32.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2208 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.773 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0606e-05 s, 0.0 kB/s 2023-10-08T16:08:32.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.773 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2209 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.829 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4141e-05 s, 0.0 kB/s 2023-10-08T16:08:32.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.830 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2210 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.885 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6838e-05 s, 0.0 kB/s 2023-10-08T16:08:32.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2211 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.942 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.942 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.942 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8709e-05 s, 0.0 kB/s 2023-10-08T16:08:32.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.943 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2212 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:32.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:32.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:32.997 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9905e-05 s, 0.0 kB/s 2023-10-08T16:08:32.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:32.998 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2213 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.054 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6849e-05 s, 0.0 kB/s 2023-10-08T16:08:33.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.055 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2214 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.975e-05 s, 0.0 kB/s 2023-10-08T16:08:33.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.111 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2215 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.167 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.168 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.168 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1094e-05 s, 0.0 kB/s 2023-10-08T16:08:33.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.168 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2216 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.224 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9616e-05 s, 0.0 kB/s 2023-10-08T16:08:33.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.224 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2217 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8651e-05 s, 0.0 kB/s 2023-10-08T16:08:33.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.280 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2218 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.336 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0004e-05 s, 0.0 kB/s 2023-10-08T16:08:33.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.336 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2219 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.391 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0768e-05 s, 0.0 kB/s 2023-10-08T16:08:33.392 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.392 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2220 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.448 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1851e-05 s, 0.0 kB/s 2023-10-08T16:08:33.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.448 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2221 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0128e-05 s, 0.0 kB/s 2023-10-08T16:08:33.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.505 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2222 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.561 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.561 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.561 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0807e-05 s, 0.0 kB/s 2023-10-08T16:08:33.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.562 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2223 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.616 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.617 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-08T16:08:33.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.617 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2224 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.672 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.673 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8332e-05 s, 0.0 kB/s 2023-10-08T16:08:33.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.674 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2225 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.730 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.364e-05 s, 0.0 kB/s 2023-10-08T16:08:33.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.730 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2226 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.786 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1678e-05 s, 0.0 kB/s 2023-10-08T16:08:33.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2227 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.843 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3058e-05 s, 0.0 kB/s 2023-10-08T16:08:33.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.844 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2228 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6405e-05 s, 0.0 kB/s 2023-10-08T16:08:33.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.900 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2229 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:33.954 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:33.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:33.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8542e-05 s, 0.0 kB/s 2023-10-08T16:08:33.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:33.955 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2230 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.011 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8399e-05 s, 0.0 kB/s 2023-10-08T16:08:34.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.012 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2231 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.067 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.068 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9837e-05 s, 0.0 kB/s 2023-10-08T16:08:34.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2232 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.124 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.124 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.124 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4341e-05 s, 0.0 kB/s 2023-10-08T16:08:34.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.126 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2233 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9074e-05 s, 0.0 kB/s 2023-10-08T16:08:34.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.180 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2234 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.0821e-05 s, 0.0 kB/s 2023-10-08T16:08:34.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2235 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.291 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1695e-05 s, 0.0 kB/s 2023-10-08T16:08:34.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.292 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2236 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.348 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9319e-05 s, 0.0 kB/s 2023-10-08T16:08:34.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.348 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2237 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.403 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9462e-05 s, 0.0 kB/s 2023-10-08T16:08:34.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.404 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2238 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.459 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.127e-05 s, 0.0 kB/s 2023-10-08T16:08:34.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.460 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2239 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.514 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.515 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.515 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1588e-05 s, 0.0 kB/s 2023-10-08T16:08:34.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.516 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2240 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.572 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.572 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.572 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0418e-05 s, 0.0 kB/s 2023-10-08T16:08:34.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.573 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2241 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0014e-05 s, 0.0 kB/s 2023-10-08T16:08:34.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.628 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2242 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2061e-05 s, 0.0 kB/s 2023-10-08T16:08:34.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.684 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2243 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.739 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8284e-05 s, 0.0 kB/s 2023-10-08T16:08:34.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.740 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2244 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.796 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.796 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.796 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1521e-05 s, 0.0 kB/s 2023-10-08T16:08:34.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.797 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2245 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.851 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0327e-05 s, 0.0 kB/s 2023-10-08T16:08:34.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.852 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2246 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.972e-05 s, 0.0 kB/s 2023-10-08T16:08:34.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2247 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:34.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:34.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:34.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0555e-05 s, 0.0 kB/s 2023-10-08T16:08:34.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:34.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2248 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.017 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9286e-05 s, 0.0 kB/s 2023-10-08T16:08:35.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2249 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.074 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9591e-05 s, 0.0 kB/s 2023-10-08T16:08:35.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.075 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2250 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.130 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0676e-05 s, 0.0 kB/s 2023-10-08T16:08:35.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.130 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2251 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.185 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6818e-05 s, 0.0 kB/s 2023-10-08T16:08:35.186 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.186 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2252 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000113961 s, 0.0 kB/s 2023-10-08T16:08:35.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.243 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2253 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.299 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.299 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.299 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6347e-05 s, 0.0 kB/s 2023-10-08T16:08:35.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.300 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2254 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.354 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.354 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7496e-05 s, 0.0 kB/s 2023-10-08T16:08:35.355 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.355 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2255 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8959e-05 s, 0.0 kB/s 2023-10-08T16:08:35.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.411 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2256 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.465 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9953e-05 s, 0.0 kB/s 2023-10-08T16:08:35.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.466 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2257 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.521 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0476e-05 s, 0.0 kB/s 2023-10-08T16:08:35.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.521 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2258 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.576 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.576 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.576 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000103631 s, 0.0 kB/s 2023-10-08T16:08:35.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.577 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2259 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.632 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.632 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.632 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000103415 s, 0.0 kB/s 2023-10-08T16:08:35.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.633 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2260 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.688 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.688 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.689 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2855e-05 s, 0.0 kB/s 2023-10-08T16:08:35.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.689 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2261 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.744 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.745 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7795e-05 s, 0.0 kB/s 2023-10-08T16:08:35.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.745 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2262 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.800 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.800 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2016e-05 s, 0.0 kB/s 2023-10-08T16:08:35.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.801 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2263 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.857 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8678e-05 s, 0.0 kB/s 2023-10-08T16:08:35.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.857 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2264 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.912 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.913 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.913 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3049e-05 s, 0.0 kB/s 2023-10-08T16:08:35.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.913 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2265 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:35.968 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6502e-05 s, 0.0 kB/s 2023-10-08T16:08:35.968 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:35.969 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2266 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.024 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.024 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.024 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9628e-05 s, 0.0 kB/s 2023-10-08T16:08:36.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.025 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2267 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0314e-05 s, 0.0 kB/s 2023-10-08T16:08:36.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.080 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2268 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.135 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.135 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.135 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3852e-05 s, 0.0 kB/s 2023-10-08T16:08:36.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.136 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2269 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.191 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5792e-05 s, 0.0 kB/s 2023-10-08T16:08:36.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.191 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2270 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.246 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.247 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.247 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1461e-05 s, 0.0 kB/s 2023-10-08T16:08:36.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.247 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2271 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.303 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.303 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.303 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0462e-05 s, 0.0 kB/s 2023-10-08T16:08:36.303 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.303 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2272 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.358 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.358 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.358 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8972e-05 s, 0.0 kB/s 2023-10-08T16:08:36.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.359 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2273 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.413 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.414 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.414 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9405e-05 s, 0.0 kB/s 2023-10-08T16:08:36.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.415 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2274 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.471 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.471 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.471 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8532e-05 s, 0.0 kB/s 2023-10-08T16:08:36.471 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.471 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2275 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.551 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.551 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.9386e-05 s, 0.0 kB/s 2023-10-08T16:08:36.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.551 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2276 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.607 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.607 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0327e-05 s, 0.0 kB/s 2023-10-08T16:08:36.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2277 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.663 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3761e-05 s, 0.0 kB/s 2023-10-08T16:08:36.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.664 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2278 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.719 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1884e-05 s, 0.0 kB/s 2023-10-08T16:08:36.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.720 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2279 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.658e-05 s, 0.0 kB/s 2023-10-08T16:08:36.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.775 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2280 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.831 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3098e-05 s, 0.0 kB/s 2023-10-08T16:08:36.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2281 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.887 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.887 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9143e-05 s, 0.0 kB/s 2023-10-08T16:08:36.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.887 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2282 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.942 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.943 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.943 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9705e-05 s, 0.0 kB/s 2023-10-08T16:08:36.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.943 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2283 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:36.998 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:36.998 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:36.998 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3988e-05 s, 0.0 kB/s 2023-10-08T16:08:36.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:36.999 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2284 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.055 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.055 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1488e-05 s, 0.0 kB/s 2023-10-08T16:08:37.055 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.055 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2285 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.111 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.111 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.111 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9288e-05 s, 0.0 kB/s 2023-10-08T16:08:37.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.111 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2286 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8131e-05 s, 0.0 kB/s 2023-10-08T16:08:37.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2287 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.222 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.123e-05 s, 0.0 kB/s 2023-10-08T16:08:37.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.222 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2288 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.279 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.279 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.999e-05 s, 0.0 kB/s 2023-10-08T16:08:37.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2289 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.335 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.335 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0851e-05 s, 0.0 kB/s 2023-10-08T16:08:37.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.335 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2290 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.391 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.176e-05 s, 0.0 kB/s 2023-10-08T16:08:37.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.391 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2291 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.449 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.449 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2923e-05 s, 0.0 kB/s 2023-10-08T16:08:37.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.450 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2292 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.505 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2975e-05 s, 0.0 kB/s 2023-10-08T16:08:37.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.505 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2293 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3256e-05 s, 0.0 kB/s 2023-10-08T16:08:37.561 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.562 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2294 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.617 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8042e-05 s, 0.0 kB/s 2023-10-08T16:08:37.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.617 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2295 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1217e-05 s, 0.0 kB/s 2023-10-08T16:08:37.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.674 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2296 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-10-08T16:08:37.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.730 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2297 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.785 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3579e-05 s, 0.0 kB/s 2023-10-08T16:08:37.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.785 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2298 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.841 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.841 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.841 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8122e-05 s, 0.0 kB/s 2023-10-08T16:08:37.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.842 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2299 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.897 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.897 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.897 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.7279e-05 s, 0.0 kB/s 2023-10-08T16:08:37.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.897 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2300 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:37.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:37.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:37.952 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8954e-05 s, 0.0 kB/s 2023-10-08T16:08:37.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:37.953 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2301 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.009 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0965e-05 s, 0.0 kB/s 2023-10-08T16:08:38.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.009 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2302 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8389e-05 s, 0.0 kB/s 2023-10-08T16:08:38.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.065 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2303 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.120 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6425e-05 s, 0.0 kB/s 2023-10-08T16:08:38.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.121 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2304 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.175 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.175 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8295e-05 s, 0.0 kB/s 2023-10-08T16:08:38.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.176 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2305 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.230 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.231 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0345e-05 s, 0.0 kB/s 2023-10-08T16:08:38.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.231 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2306 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.415 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.415 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1068e-05 s, 0.0 kB/s 2023-10-08T16:08:38.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.416 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2307 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.431 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.432 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8524e-05 s, 0.0 kB/s 2023-10-08T16:08:38.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.432 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2308 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.491 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1802e-05 s, 0.0 kB/s 2023-10-08T16:08:38.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.491 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2309 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.548 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2828e-05 s, 0.0 kB/s 2023-10-08T16:08:38.548 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.548 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2310 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.938e-05 s, 0.0 kB/s 2023-10-08T16:08:38.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2311 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.418e-05 s, 0.0 kB/s 2023-10-08T16:08:38.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2312 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.713 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0665e-05 s, 0.0 kB/s 2023-10-08T16:08:38.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.714 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2313 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.770 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8792e-05 s, 0.0 kB/s 2023-10-08T16:08:38.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2314 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.824 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.824 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1202e-05 s, 0.0 kB/s 2023-10-08T16:08:38.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.825 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2315 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1621e-05 s, 0.0 kB/s 2023-10-08T16:08:38.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.881 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2316 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.936 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.937 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8491e-05 s, 0.0 kB/s 2023-10-08T16:08:38.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.937 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2317 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:38.992 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:38.993 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-10-08T16:08:38.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:38.993 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2318 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.048 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0809e-05 s, 0.0 kB/s 2023-10-08T16:08:39.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.049 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2319 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.104 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.104 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0619e-05 s, 0.0 kB/s 2023-10-08T16:08:39.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.105 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2320 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.160 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9201e-05 s, 0.0 kB/s 2023-10-08T16:08:39.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.161 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2321 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.258e-05 s, 0.0 kB/s 2023-10-08T16:08:39.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2322 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0943e-05 s, 0.0 kB/s 2023-10-08T16:08:39.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.272 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2323 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7443e-05 s, 0.0 kB/s 2023-10-08T16:08:39.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2324 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.383 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9123e-05 s, 0.0 kB/s 2023-10-08T16:08:39.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2325 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.438 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.438 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1639e-05 s, 0.0 kB/s 2023-10-08T16:08:39.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.439 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2326 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.494 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1786e-05 s, 0.0 kB/s 2023-10-08T16:08:39.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.494 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2327 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.550 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.088e-05 s, 0.0 kB/s 2023-10-08T16:08:39.551 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.551 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2328 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.606 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0959e-05 s, 0.0 kB/s 2023-10-08T16:08:39.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.607 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2329 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.683 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.683 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9458e-05 s, 0.0 kB/s 2023-10-08T16:08:39.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.683 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2330 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.718 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7946e-05 s, 0.0 kB/s 2023-10-08T16:08:39.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2331 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.774 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9754e-05 s, 0.0 kB/s 2023-10-08T16:08:39.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.775 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2332 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7014e-05 s, 0.0 kB/s 2023-10-08T16:08:39.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2333 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.963 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.963 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2575e-05 s, 0.0 kB/s 2023-10-08T16:08:39.963 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.963 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2334 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:39.979 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:39.980 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8875e-05 s, 0.0 kB/s 2023-10-08T16:08:39.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:39.980 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2335 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.036 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.049 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.049 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9705e-05 s, 0.0 kB/s 2023-10-08T16:08:40.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.050 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2336 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.106 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2163e-05 s, 0.0 kB/s 2023-10-08T16:08:40.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.107 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2337 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.164 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0581e-05 s, 0.0 kB/s 2023-10-08T16:08:40.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.164 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2338 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.220 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9706e-05 s, 0.0 kB/s 2023-10-08T16:08:40.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.220 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2339 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.278 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0705e-05 s, 0.0 kB/s 2023-10-08T16:08:40.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.279 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2340 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.334 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9989e-05 s, 0.0 kB/s 2023-10-08T16:08:40.431 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.431 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2341 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.447 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.449 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2172e-05 s, 0.0 kB/s 2023-10-08T16:08:40.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.450 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2342 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.513 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.513 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7178e-05 s, 0.0 kB/s 2023-10-08T16:08:40.514 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.514 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2343 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.571 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.571 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.571 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2103e-05 s, 0.0 kB/s 2023-10-08T16:08:40.572 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.572 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2344 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.627 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.831 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.831 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000101119 s, 0.0 kB/s 2023-10-08T16:08:40.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.831 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2345 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.847 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.847 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.113e-05 s, 0.0 kB/s 2023-10-08T16:08:40.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.848 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2346 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.902 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.903 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.903 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.171e-05 s, 0.0 kB/s 2023-10-08T16:08:40.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.904 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2347 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:40.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:40.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:40.960 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1051e-05 s, 0.0 kB/s 2023-10-08T16:08:40.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:40.961 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2348 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.016 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.016 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.016 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.976e-05 s, 0.0 kB/s 2023-10-08T16:08:41.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.017 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2349 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.071 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.072 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.072 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5522e-05 s, 0.0 kB/s 2023-10-08T16:08:41.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.072 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2350 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.127 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.128 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.128 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7021e-05 s, 0.0 kB/s 2023-10-08T16:08:41.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.129 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2351 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.198 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9195e-05 s, 0.0 kB/s 2023-10-08T16:08:41.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.198 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2352 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.402 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.403 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3791e-05 s, 0.0 kB/s 2023-10-08T16:08:41.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.403 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2353 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.457 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1515e-05 s, 0.0 kB/s 2023-10-08T16:08:41.458 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.458 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2354 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.474 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9687e-05 s, 0.0 kB/s 2023-10-08T16:08:41.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.475 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2355 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.530 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9396e-05 s, 0.0 kB/s 2023-10-08T16:08:41.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.531 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2356 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.664 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.664 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4109e-05 s, 0.0 kB/s 2023-10-08T16:08:41.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.665 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2357 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6466e-05 s, 0.0 kB/s 2023-10-08T16:08:41.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.681 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2358 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.735 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.735 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0325e-05 s, 0.0 kB/s 2023-10-08T16:08:41.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.736 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2359 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3448e-05 s, 0.0 kB/s 2023-10-08T16:08:41.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.793 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2360 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.848 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.848 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9825e-05 s, 0.0 kB/s 2023-10-08T16:08:41.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.849 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2361 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.904 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.904 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6365e-05 s, 0.0 kB/s 2023-10-08T16:08:41.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.905 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2362 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:41.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:41.960 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:41.960 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8205e-05 s, 0.0 kB/s 2023-10-08T16:08:41.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:41.961 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2363 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8855e-05 s, 0.0 kB/s 2023-10-08T16:08:42.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2364 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.074 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.944e-05 s, 0.0 kB/s 2023-10-08T16:08:42.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.074 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2365 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.130 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1058e-05 s, 0.0 kB/s 2023-10-08T16:08:42.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.131 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2366 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.187 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.188 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0476e-05 s, 0.0 kB/s 2023-10-08T16:08:42.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.188 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2367 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.244 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.244 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.244 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8649e-05 s, 0.0 kB/s 2023-10-08T16:08:42.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.245 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2368 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.300 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6101e-05 s, 0.0 kB/s 2023-10-08T16:08:42.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.300 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2369 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0199e-05 s, 0.0 kB/s 2023-10-08T16:08:42.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2370 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.412 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1528e-05 s, 0.0 kB/s 2023-10-08T16:08:42.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.413 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2371 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.468 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0956e-05 s, 0.0 kB/s 2023-10-08T16:08:42.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.469 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2372 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.525 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.525 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7291e-05 s, 0.0 kB/s 2023-10-08T16:08:42.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.525 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2373 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.580 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1522e-05 s, 0.0 kB/s 2023-10-08T16:08:42.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.581 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2374 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.637 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9683e-05 s, 0.0 kB/s 2023-10-08T16:08:42.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.637 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2375 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.698 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.698 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0821e-05 s, 0.0 kB/s 2023-10-08T16:08:42.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.698 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2376 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.753 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.754 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.754 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1829e-05 s, 0.0 kB/s 2023-10-08T16:08:42.754 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.754 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2377 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.809 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.809 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4278e-05 s, 0.0 kB/s 2023-10-08T16:08:42.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.810 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2378 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.866 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.866 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000107451 s, 0.0 kB/s 2023-10-08T16:08:42.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.867 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2379 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.921 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.922 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7568e-05 s, 0.0 kB/s 2023-10-08T16:08:42.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.922 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2380 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:42.978 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3118e-05 s, 0.0 kB/s 2023-10-08T16:08:42.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:42.978 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2381 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.034 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9055e-05 s, 0.0 kB/s 2023-10-08T16:08:43.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.034 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2382 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.088 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.089 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9005e-05 s, 0.0 kB/s 2023-10-08T16:08:43.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.090 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2383 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.146 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.863e-05 s, 0.0 kB/s 2023-10-08T16:08:43.146 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.146 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2384 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.201 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.201 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.202 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3565e-05 s, 0.0 kB/s 2023-10-08T16:08:43.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2385 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.258 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.258 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6788e-05 s, 0.0 kB/s 2023-10-08T16:08:43.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.259 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2386 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.315 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4221e-05 s, 0.0 kB/s 2023-10-08T16:08:43.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2387 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9839e-05 s, 0.0 kB/s 2023-10-08T16:08:43.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.371 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2388 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.426 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1746e-05 s, 0.0 kB/s 2023-10-08T16:08:43.427 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.427 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2389 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.477 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:08:43.483 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5351e-05 s, 0.0 kB/s 2023-10-08T16:08:43.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.484 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2390 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.485 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:08:43.541 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.541 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.541 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3581e-05 s, 0.0 kB/s 2023-10-08T16:08:43.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.542 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2391 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.604 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.604 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.78e-05 s, 0.0 kB/s 2023-10-08T16:08:43.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.605 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2392 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.659 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6804e-05 s, 0.0 kB/s 2023-10-08T16:08:43.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.660 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2393 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7038e-05 s, 0.0 kB/s 2023-10-08T16:08:43.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2394 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.772 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0236e-05 s, 0.0 kB/s 2023-10-08T16:08:43.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.772 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2395 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.828 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.6169e-05 s, 0.0 kB/s 2023-10-08T16:08:43.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.828 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2396 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2074e-05 s, 0.0 kB/s 2023-10-08T16:08:43.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:43.884 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2397 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:43.939 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.082 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8725e-05 s, 0.0 kB/s 2023-10-08T16:08:44.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.082 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2398 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.098 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.167 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.167 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8871e-05 s, 0.0 kB/s 2023-10-08T16:08:44.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.168 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2399 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.183 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.183 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0098e-05 s, 0.0 kB/s 2023-10-08T16:08:44.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.184 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2400 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.239 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.239 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9803e-05 s, 0.0 kB/s 2023-10-08T16:08:44.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.240 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2401 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.294 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.295 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.295 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8861e-05 s, 0.0 kB/s 2023-10-08T16:08:44.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.295 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2402 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.350 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.350 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2216e-05 s, 0.0 kB/s 2023-10-08T16:08:44.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.351 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2403 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0568e-05 s, 0.0 kB/s 2023-10-08T16:08:44.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.407 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2404 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.462 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.463 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.463 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8032e-05 s, 0.0 kB/s 2023-10-08T16:08:44.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.463 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2405 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.519 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.519 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.519 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3924e-05 s, 0.0 kB/s 2023-10-08T16:08:44.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.520 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2406 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6688e-05 s, 0.0 kB/s 2023-10-08T16:08:44.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.575 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2407 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.630 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.3051e-05 s, 0.0 kB/s 2023-10-08T16:08:44.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.631 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2408 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.685 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.686 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.686 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1984e-05 s, 0.0 kB/s 2023-10-08T16:08:44.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.686 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2409 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.742 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.742 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.742 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9e-05 s, 0.0 kB/s 2023-10-08T16:08:44.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.743 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2410 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.798 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.801 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.801 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.651e-05 s, 0.0 kB/s 2023-10-08T16:08:44.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.802 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2411 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.854 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.854 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.855 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9161e-05 s, 0.0 kB/s 2023-10-08T16:08:44.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.855 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2412 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.909 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9397e-05 s, 0.0 kB/s 2023-10-08T16:08:44.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.910 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2413 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:44.965 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1716e-05 s, 0.0 kB/s 2023-10-08T16:08:44.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:44.966 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2414 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.021 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.021 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.021 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9135e-05 s, 0.0 kB/s 2023-10-08T16:08:45.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.021 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2415 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.076 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.077 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4994e-05 s, 0.0 kB/s 2023-10-08T16:08:45.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.077 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2416 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.132 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.133 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5015e-05 s, 0.0 kB/s 2023-10-08T16:08:45.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.133 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2417 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.189 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0738e-05 s, 0.0 kB/s 2023-10-08T16:08:45.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.190 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2418 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.245 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1602e-05 s, 0.0 kB/s 2023-10-08T16:08:45.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.246 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2419 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.301 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.301 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0015e-05 s, 0.0 kB/s 2023-10-08T16:08:45.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.301 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2420 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1405e-05 s, 0.0 kB/s 2023-10-08T16:08:45.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2421 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.412 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.413 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1546e-05 s, 0.0 kB/s 2023-10-08T16:08:45.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.413 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2422 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.469 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5788e-05 s, 0.0 kB/s 2023-10-08T16:08:45.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.470 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2423 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.526 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.526 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2171e-05 s, 0.0 kB/s 2023-10-08T16:08:45.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.527 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2424 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.582 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.582 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.582 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9075e-05 s, 0.0 kB/s 2023-10-08T16:08:45.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.583 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2425 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.638 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.638 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.638 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9736e-05 s, 0.0 kB/s 2023-10-08T16:08:45.638 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.638 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2426 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7895e-05 s, 0.0 kB/s 2023-10-08T16:08:45.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.695 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2427 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.749 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.750 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.750 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0292e-05 s, 0.0 kB/s 2023-10-08T16:08:45.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.751 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2428 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.806 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.806 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.806 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1942e-05 s, 0.0 kB/s 2023-10-08T16:08:45.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.807 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2429 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.863 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.863 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.863 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9504e-05 s, 0.0 kB/s 2023-10-08T16:08:45.863 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.863 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2430 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.919 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.919 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9733e-05 s, 0.0 kB/s 2023-10-08T16:08:45.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.920 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2431 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:45.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:45.975 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:45.975 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9231e-05 s, 0.0 kB/s 2023-10-08T16:08:45.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:45.975 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2432 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.030 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.031 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.031 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3016e-05 s, 0.0 kB/s 2023-10-08T16:08:46.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.031 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2433 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.087 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.087 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0639e-05 s, 0.0 kB/s 2023-10-08T16:08:46.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.087 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2434 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.143 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.143 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7089e-05 s, 0.0 kB/s 2023-10-08T16:08:46.144 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.144 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2435 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.199 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.199 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1941e-05 s, 0.0 kB/s 2023-10-08T16:08:46.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.199 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2436 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.255 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.255 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1425e-05 s, 0.0 kB/s 2023-10-08T16:08:46.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.256 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2437 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.310 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.311 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6933e-05 s, 0.0 kB/s 2023-10-08T16:08:46.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.311 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2438 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.366 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8e-05 s, 0.0 kB/s 2023-10-08T16:08:46.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.367 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2439 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.422 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.422 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1385e-05 s, 0.0 kB/s 2023-10-08T16:08:46.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.423 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2440 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.478 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.478 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.818e-05 s, 0.0 kB/s 2023-10-08T16:08:46.478 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.479 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2441 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.534 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.535 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.535 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2014e-05 s, 0.0 kB/s 2023-10-08T16:08:46.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.535 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2442 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.591 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.592 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3252e-05 s, 0.0 kB/s 2023-10-08T16:08:46.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.592 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2443 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3005e-05 s, 0.0 kB/s 2023-10-08T16:08:46.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.649 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2444 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.703 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.703 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.703 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8505e-05 s, 0.0 kB/s 2023-10-08T16:08:46.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.704 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2445 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.760 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8625e-05 s, 0.0 kB/s 2023-10-08T16:08:46.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.760 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2446 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.815 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.816 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1452e-05 s, 0.0 kB/s 2023-10-08T16:08:46.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.816 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2447 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.872 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8798e-05 s, 0.0 kB/s 2023-10-08T16:08:46.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.872 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2448 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1357e-05 s, 0.0 kB/s 2023-10-08T16:08:46.929 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.929 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2449 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:46.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:46.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:46.985 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8111e-05 s, 0.0 kB/s 2023-10-08T16:08:46.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:46.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2450 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.041 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.84e-05 s, 0.0 kB/s 2023-10-08T16:08:47.042 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.042 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2451 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.097 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.097 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.097 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7188e-05 s, 0.0 kB/s 2023-10-08T16:08:47.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.098 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2452 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.154 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.154 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7963e-05 s, 0.0 kB/s 2023-10-08T16:08:47.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.154 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2453 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.210 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.210 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.210 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8902e-05 s, 0.0 kB/s 2023-10-08T16:08:47.211 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.211 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2454 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.266 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6009e-05 s, 0.0 kB/s 2023-10-08T16:08:47.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.267 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2455 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.323 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6664e-05 s, 0.0 kB/s 2023-10-08T16:08:47.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.324 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2456 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.378 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1731e-05 s, 0.0 kB/s 2023-10-08T16:08:47.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.379 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2457 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.435 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.755e-05 s, 0.0 kB/s 2023-10-08T16:08:47.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.436 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2458 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.491 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.492 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3689e-05 s, 0.0 kB/s 2023-10-08T16:08:47.492 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.492 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2459 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.548 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2089e-05 s, 0.0 kB/s 2023-10-08T16:08:47.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.549 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2460 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.604 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0061e-05 s, 0.0 kB/s 2023-10-08T16:08:47.604 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.604 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2461 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.660 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1529e-05 s, 0.0 kB/s 2023-10-08T16:08:47.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.661 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2462 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.715 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.716 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.716 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.017e-05 s, 0.0 kB/s 2023-10-08T16:08:47.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.716 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2463 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.772 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.772 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7274e-05 s, 0.0 kB/s 2023-10-08T16:08:47.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.773 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2464 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.828 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4155e-05 s, 0.0 kB/s 2023-10-08T16:08:47.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.829 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2465 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1203e-05 s, 0.0 kB/s 2023-10-08T16:08:47.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.885 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2466 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.940 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8393e-05 s, 0.0 kB/s 2023-10-08T16:08:47.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2467 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:47.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:47.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:47.997 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.242e-05 s, 0.0 kB/s 2023-10-08T16:08:47.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:47.998 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2468 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.053 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.053 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0091e-05 s, 0.0 kB/s 2023-10-08T16:08:48.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.054 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2469 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.109 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7989e-05 s, 0.0 kB/s 2023-10-08T16:08:48.109 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.109 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2470 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.165 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.165 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.165 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.856e-05 s, 0.0 kB/s 2023-10-08T16:08:48.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.165 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2471 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.220 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.221 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.3953e-05 s, 0.0 kB/s 2023-10-08T16:08:48.221 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.221 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2472 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.277 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.277 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.277 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.085e-05 s, 0.0 kB/s 2023-10-08T16:08:48.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.278 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2473 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.352 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9428e-05 s, 0.0 kB/s 2023-10-08T16:08:48.352 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.353 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2474 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.407 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.407 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.407 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8818e-05 s, 0.0 kB/s 2023-10-08T16:08:48.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.408 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2475 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9931e-05 s, 0.0 kB/s 2023-10-08T16:08:48.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.465 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2476 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.522 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0484e-05 s, 0.0 kB/s 2023-10-08T16:08:48.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.522 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2477 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.578 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-08T16:08:48.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.578 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2478 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.633 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.634 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2119e-05 s, 0.0 kB/s 2023-10-08T16:08:48.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.634 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2479 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.690 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.690 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.128e-05 s, 0.0 kB/s 2023-10-08T16:08:48.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.690 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2480 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.745 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.745 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.745 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3859e-05 s, 0.0 kB/s 2023-10-08T16:08:48.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.746 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2481 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.801 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.802 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.802 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2127e-05 s, 0.0 kB/s 2023-10-08T16:08:48.802 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.802 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2482 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0888e-05 s, 0.0 kB/s 2023-10-08T16:08:48.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.859 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2483 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.914 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1462e-05 s, 0.0 kB/s 2023-10-08T16:08:48.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.915 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2484 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:48.969 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9001e-05 s, 0.0 kB/s 2023-10-08T16:08:48.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:48.970 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2485 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.026 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9765e-05 s, 0.0 kB/s 2023-10-08T16:08:49.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.027 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2486 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.081 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.082 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3622e-05 s, 0.0 kB/s 2023-10-08T16:08:49.082 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.082 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2487 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.138 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8591e-05 s, 0.0 kB/s 2023-10-08T16:08:49.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.138 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2488 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.194 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.194 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.194 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.89e-05 s, 0.0 kB/s 2023-10-08T16:08:49.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.195 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2489 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.249 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.249 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9461e-05 s, 0.0 kB/s 2023-10-08T16:08:49.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.250 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2490 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.304 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.305 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8126e-05 s, 0.0 kB/s 2023-10-08T16:08:49.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.306 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2491 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5195e-05 s, 0.0 kB/s 2023-10-08T16:08:49.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.362 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2492 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.417 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.417 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3638e-05 s, 0.0 kB/s 2023-10-08T16:08:49.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.418 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2493 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.474 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.474 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8136e-05 s, 0.0 kB/s 2023-10-08T16:08:49.475 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.475 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2494 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.530 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.531 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0784e-05 s, 0.0 kB/s 2023-10-08T16:08:49.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.531 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2495 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.586 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.587 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9816e-05 s, 0.0 kB/s 2023-10-08T16:08:49.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.587 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2496 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.643 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.643 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0436e-05 s, 0.0 kB/s 2023-10-08T16:08:49.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.643 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2497 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.699 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9041e-05 s, 0.0 kB/s 2023-10-08T16:08:49.699 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.699 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2498 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.755 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.755 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1793e-05 s, 0.0 kB/s 2023-10-08T16:08:49.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.755 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2499 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.811 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.811 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7425e-05 s, 0.0 kB/s 2023-10-08T16:08:49.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.811 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2500 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.867 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.867 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3425e-05 s, 0.0 kB/s 2023-10-08T16:08:49.867 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.868 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2501 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.922 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.923 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.923 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1929e-05 s, 0.0 kB/s 2023-10-08T16:08:49.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.923 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2502 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:49.978 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:49.979 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.8411e-05 s, 0.0 kB/s 2023-10-08T16:08:49.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:49.979 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2503 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.035 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.035 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3965e-05 s, 0.0 kB/s 2023-10-08T16:08:50.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.035 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2504 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.090 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.090 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.091 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9228e-05 s, 0.0 kB/s 2023-10-08T16:08:50.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.091 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2505 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9048e-05 s, 0.0 kB/s 2023-10-08T16:08:50.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.147 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2506 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.202 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.203 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8952e-05 s, 0.0 kB/s 2023-10-08T16:08:50.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.203 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2507 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.259 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.259 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1214e-05 s, 0.0 kB/s 2023-10-08T16:08:50.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.260 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2508 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.315 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.315 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1038e-05 s, 0.0 kB/s 2023-10-08T16:08:50.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.316 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2509 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.371 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8935e-05 s, 0.0 kB/s 2023-10-08T16:08:50.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.372 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2510 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.426 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.427 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8578e-05 s, 0.0 kB/s 2023-10-08T16:08:50.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.428 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2511 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.484 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.484 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9427e-05 s, 0.0 kB/s 2023-10-08T16:08:50.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.485 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2512 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.540 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.540 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0023e-05 s, 0.0 kB/s 2023-10-08T16:08:50.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.541 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2513 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.596 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.596 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0596e-05 s, 0.0 kB/s 2023-10-08T16:08:50.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2514 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.652 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9548e-05 s, 0.0 kB/s 2023-10-08T16:08:50.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.652 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2515 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.707 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0388e-05 s, 0.0 kB/s 2023-10-08T16:08:50.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.708 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2516 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.763 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.764 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2322e-05 s, 0.0 kB/s 2023-10-08T16:08:50.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.764 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2517 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.820 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0156e-05 s, 0.0 kB/s 2023-10-08T16:08:50.820 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.820 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2518 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.875 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.876 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9381e-05 s, 0.0 kB/s 2023-10-08T16:08:50.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.877 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2519 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.933 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.933 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1501e-05 s, 0.0 kB/s 2023-10-08T16:08:50.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.934 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2520 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:50.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:50.988 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:50.988 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6644e-05 s, 0.0 kB/s 2023-10-08T16:08:50.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:50.989 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2521 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.045 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.045 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9445e-05 s, 0.0 kB/s 2023-10-08T16:08:51.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.046 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2522 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.100 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.101 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.101 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3581e-05 s, 0.0 kB/s 2023-10-08T16:08:51.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.101 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2523 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.156 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9944e-05 s, 0.0 kB/s 2023-10-08T16:08:51.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.157 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2524 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.212 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.212 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0495e-05 s, 0.0 kB/s 2023-10-08T16:08:51.212 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2525 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.267 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.268 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9811e-05 s, 0.0 kB/s 2023-10-08T16:08:51.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.268 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2526 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.324 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.324 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0098e-05 s, 0.0 kB/s 2023-10-08T16:08:51.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.325 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2527 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.379 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.379 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6016e-05 s, 0.0 kB/s 2023-10-08T16:08:51.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.380 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2528 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.434 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.435 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8598e-05 s, 0.0 kB/s 2023-10-08T16:08:51.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.435 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2529 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.490 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.491 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9916e-05 s, 0.0 kB/s 2023-10-08T16:08:51.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.491 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2530 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0843e-05 s, 0.0 kB/s 2023-10-08T16:08:51.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.547 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2531 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1692e-05 s, 0.0 kB/s 2023-10-08T16:08:51.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.603 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2532 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3115e-05 s, 0.0 kB/s 2023-10-08T16:08:51.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2533 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.714 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.014e-05 s, 0.0 kB/s 2023-10-08T16:08:51.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.715 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2534 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.769 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.769 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8488e-05 s, 0.0 kB/s 2023-10-08T16:08:51.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.770 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2535 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.826 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.826 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0683e-05 s, 0.0 kB/s 2023-10-08T16:08:51.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.826 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2536 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.881 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.882 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.882 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9128e-05 s, 0.0 kB/s 2023-10-08T16:08:51.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.882 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2537 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.757e-05 s, 0.0 kB/s 2023-10-08T16:08:51.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.938 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2538 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:51.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:51.993 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:51.993 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9518e-05 s, 0.0 kB/s 2023-10-08T16:08:51.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:51.994 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2539 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.050 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.050 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.050 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9549e-05 s, 0.0 kB/s 2023-10-08T16:08:52.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.051 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2540 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.105 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.106 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.106 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7686e-05 s, 0.0 kB/s 2023-10-08T16:08:52.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.106 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2541 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.162 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.162 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.985e-05 s, 0.0 kB/s 2023-10-08T16:08:52.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.163 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2542 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.218 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1189e-05 s, 0.0 kB/s 2023-10-08T16:08:52.218 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.219 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2543 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.273 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.273 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8428e-05 s, 0.0 kB/s 2023-10-08T16:08:52.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.274 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2544 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.329 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.330 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.330 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0742e-05 s, 0.0 kB/s 2023-10-08T16:08:52.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.330 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2545 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.386 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.386 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1035e-05 s, 0.0 kB/s 2023-10-08T16:08:52.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.386 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2546 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.441 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.442 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.442 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.3599e-05 s, 0.0 kB/s 2023-10-08T16:08:52.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.442 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2547 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0872e-05 s, 0.0 kB/s 2023-10-08T16:08:52.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.498 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2548 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.554 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1343e-05 s, 0.0 kB/s 2023-10-08T16:08:52.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.554 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2549 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.609 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.610 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.511e-05 s, 0.0 kB/s 2023-10-08T16:08:52.613 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.613 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2550 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.666 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.666 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.666 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9872e-05 s, 0.0 kB/s 2023-10-08T16:08:52.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.666 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2551 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.721 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9862e-05 s, 0.0 kB/s 2023-10-08T16:08:52.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.722 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2552 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.777 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.778 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.778 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8288e-05 s, 0.0 kB/s 2023-10-08T16:08:52.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.778 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2553 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8825e-05 s, 0.0 kB/s 2023-10-08T16:08:52.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.835 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2554 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.889 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0605e-05 s, 0.0 kB/s 2023-10-08T16:08:52.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.890 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2555 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:52.945 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:52.945 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:52.945 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4698e-05 s, 0.0 kB/s 2023-10-08T16:08:52.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:52.946 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2556 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.002 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.002 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.002 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5526e-05 s, 0.0 kB/s 2023-10-08T16:08:53.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.002 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2557 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.058 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.058 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.058 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9878e-05 s, 0.0 kB/s 2023-10-08T16:08:53.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.059 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2558 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.172e-05 s, 0.0 kB/s 2023-10-08T16:08:53.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.114 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2559 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.169 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.169 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0524e-05 s, 0.0 kB/s 2023-10-08T16:08:53.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.170 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2560 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.225 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.226 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6252e-05 s, 0.0 kB/s 2023-10-08T16:08:53.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.226 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2561 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.281 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.281 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.281 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.636e-05 s, 0.0 kB/s 2023-10-08T16:08:53.282 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.282 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2562 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.337 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0814e-05 s, 0.0 kB/s 2023-10-08T16:08:53.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.338 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2563 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.393 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9025e-05 s, 0.0 kB/s 2023-10-08T16:08:53.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.394 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2564 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.448 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.449 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.449 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7932e-05 s, 0.0 kB/s 2023-10-08T16:08:53.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.449 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2565 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.504 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.504 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5028e-05 s, 0.0 kB/s 2023-10-08T16:08:53.505 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.505 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2566 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7792e-05 s, 0.0 kB/s 2023-10-08T16:08:53.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.562 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2567 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9998e-05 s, 0.0 kB/s 2023-10-08T16:08:53.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.619 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2568 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.674 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.675 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2679e-05 s, 0.0 kB/s 2023-10-08T16:08:53.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.675 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2569 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.731 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7541e-05 s, 0.0 kB/s 2023-10-08T16:08:53.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.731 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2570 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3663e-05 s, 0.0 kB/s 2023-10-08T16:08:53.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2571 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.843 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1185e-05 s, 0.0 kB/s 2023-10-08T16:08:53.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2572 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1431e-05 s, 0.0 kB/s 2023-10-08T16:08:53.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2573 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:53.954 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:53.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:53.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0467e-05 s, 0.0 kB/s 2023-10-08T16:08:53.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:53.955 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2574 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.010 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.011 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2192e-05 s, 0.0 kB/s 2023-10-08T16:08:54.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.012 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2575 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.068 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0064e-05 s, 0.0 kB/s 2023-10-08T16:08:54.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2576 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9121e-05 s, 0.0 kB/s 2023-10-08T16:08:54.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2577 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2242e-05 s, 0.0 kB/s 2023-10-08T16:08:54.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.180 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2578 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3741e-05 s, 0.0 kB/s 2023-10-08T16:08:54.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.237 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2579 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.291 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0743e-05 s, 0.0 kB/s 2023-10-08T16:08:54.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.292 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2580 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.349 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0518e-05 s, 0.0 kB/s 2023-10-08T16:08:54.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.349 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2581 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.404 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.405 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.405 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.271e-05 s, 0.0 kB/s 2023-10-08T16:08:54.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.405 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2582 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.461 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.461 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.461 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1673e-05 s, 0.0 kB/s 2023-10-08T16:08:54.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.461 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2583 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.516 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.517 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.517 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.2648e-05 s, 0.0 kB/s 2023-10-08T16:08:54.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.517 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2584 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.573 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9718e-05 s, 0.0 kB/s 2023-10-08T16:08:54.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.574 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2585 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.629 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.629 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.629 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9198e-05 s, 0.0 kB/s 2023-10-08T16:08:54.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.629 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2586 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.685 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.685 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.685 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9418e-05 s, 0.0 kB/s 2023-10-08T16:08:54.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.686 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2587 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.741 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.741 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8218e-05 s, 0.0 kB/s 2023-10-08T16:08:54.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.742 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2588 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.797 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.783e-05 s, 0.0 kB/s 2023-10-08T16:08:54.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.797 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2589 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.853 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8308e-05 s, 0.0 kB/s 2023-10-08T16:08:54.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.853 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2590 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.909 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0278e-05 s, 0.0 kB/s 2023-10-08T16:08:54.909 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.910 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2591 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:54.964 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:54.964 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:54.964 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5231e-05 s, 0.0 kB/s 2023-10-08T16:08:54.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:54.965 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2592 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.021 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.022 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.022 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8687e-05 s, 0.0 kB/s 2023-10-08T16:08:55.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.022 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2593 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.078 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6764e-05 s, 0.0 kB/s 2023-10-08T16:08:55.078 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.079 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2594 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.134 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.134 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.7696e-05 s, 0.0 kB/s 2023-10-08T16:08:55.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.134 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2595 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.189 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.190 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0972e-05 s, 0.0 kB/s 2023-10-08T16:08:55.190 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.190 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2596 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.245 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.631e-05 s, 0.0 kB/s 2023-10-08T16:08:55.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.246 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2597 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.300 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.925e-05 s, 0.0 kB/s 2023-10-08T16:08:55.301 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.301 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2598 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.356 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7466e-05 s, 0.0 kB/s 2023-10-08T16:08:55.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.357 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2599 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.411 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0695e-05 s, 0.0 kB/s 2023-10-08T16:08:55.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.413 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2600 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.468 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.468 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5265e-05 s, 0.0 kB/s 2023-10-08T16:08:55.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.468 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2601 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.524 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.524 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.6478e-05 s, 0.0 kB/s 2023-10-08T16:08:55.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2602 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.580 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.6444e-05 s, 0.0 kB/s 2023-10-08T16:08:55.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.581 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2603 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.636 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.636 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.636 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9422e-05 s, 0.0 kB/s 2023-10-08T16:08:55.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.637 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2604 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.691 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9435e-05 s, 0.0 kB/s 2023-10-08T16:08:55.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.692 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2605 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.748 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.748 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7402e-05 s, 0.0 kB/s 2023-10-08T16:08:55.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.749 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2606 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.804 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.00010091 s, 0.0 kB/s 2023-10-08T16:08:55.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.805 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2607 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.860 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7938e-05 s, 0.0 kB/s 2023-10-08T16:08:55.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.860 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2608 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1329e-05 s, 0.0 kB/s 2023-10-08T16:08:55.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2609 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:55.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.092e-05 s, 0.0 kB/s 2023-10-08T16:08:55.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:55.973 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2610 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.028 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.134e-05 s, 0.0 kB/s 2023-10-08T16:08:56.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2611 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.083 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1521e-05 s, 0.0 kB/s 2023-10-08T16:08:56.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.085 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2612 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.142 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7761e-05 s, 0.0 kB/s 2023-10-08T16:08:56.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.143 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2613 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.197 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.198 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.034e-05 s, 0.0 kB/s 2023-10-08T16:08:56.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.198 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2614 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.254 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.254 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.254 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.715e-05 s, 0.0 kB/s 2023-10-08T16:08:56.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.254 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2615 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.309 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.309 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.309 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4088e-05 s, 0.0 kB/s 2023-10-08T16:08:56.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.310 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2616 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.366 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.366 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.959e-05 s, 0.0 kB/s 2023-10-08T16:08:56.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.366 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2617 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.423 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.424 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.424 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7552e-05 s, 0.0 kB/s 2023-10-08T16:08:56.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.424 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2618 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.480 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7043e-05 s, 0.0 kB/s 2023-10-08T16:08:56.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.481 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2619 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.536 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.8129e-05 s, 0.0 kB/s 2023-10-08T16:08:56.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2620 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.592 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7905e-05 s, 0.0 kB/s 2023-10-08T16:08:56.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.592 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2621 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.648 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.648 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9537e-05 s, 0.0 kB/s 2023-10-08T16:08:56.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.649 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2622 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.704 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.704 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8725e-05 s, 0.0 kB/s 2023-10-08T16:08:56.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.705 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2623 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0817e-05 s, 0.0 kB/s 2023-10-08T16:08:56.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.761 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2624 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.816 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.816 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8555e-05 s, 0.0 kB/s 2023-10-08T16:08:56.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.817 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2625 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3522e-05 s, 0.0 kB/s 2023-10-08T16:08:56.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.873 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2626 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7224e-05 s, 0.0 kB/s 2023-10-08T16:08:56.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.928 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2627 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:56.983 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:56.983 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:56.983 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0519e-05 s, 0.0 kB/s 2023-10-08T16:08:56.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:56.985 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2628 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.041 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.815e-05 s, 0.0 kB/s 2023-10-08T16:08:57.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2629 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.097 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2335e-05 s, 0.0 kB/s 2023-10-08T16:08:57.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.097 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2630 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.153 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.153 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1845e-05 s, 0.0 kB/s 2023-10-08T16:08:57.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.154 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2631 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.209 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.209 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7714e-05 s, 0.0 kB/s 2023-10-08T16:08:57.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.209 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2632 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9958e-05 s, 0.0 kB/s 2023-10-08T16:08:57.265 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.265 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2633 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.321 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.321 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.321 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2855e-05 s, 0.0 kB/s 2023-10-08T16:08:57.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.321 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2634 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.377 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8773e-05 s, 0.0 kB/s 2023-10-08T16:08:57.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2635 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.435 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.436 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8571e-05 s, 0.0 kB/s 2023-10-08T16:08:57.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.436 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2636 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.492 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.493 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.908e-05 s, 0.0 kB/s 2023-10-08T16:08:57.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.493 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2637 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.549 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.550 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6872e-05 s, 0.0 kB/s 2023-10-08T16:08:57.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2638 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.617 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1378e-05 s, 0.0 kB/s 2023-10-08T16:08:57.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.617 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2639 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.662 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.662 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0235e-05 s, 0.0 kB/s 2023-10-08T16:08:57.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.662 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2640 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.718 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5531e-05 s, 0.0 kB/s 2023-10-08T16:08:57.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.719 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2641 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.774 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9375e-05 s, 0.0 kB/s 2023-10-08T16:08:57.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.774 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2642 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.830 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.830 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1292e-05 s, 0.0 kB/s 2023-10-08T16:08:57.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.830 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2643 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.886 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.886 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6183e-05 s, 0.0 kB/s 2023-10-08T16:08:57.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.886 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2644 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2578e-05 s, 0.0 kB/s 2023-10-08T16:08:57.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.942 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2645 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:57.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:57.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:57.997 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5472e-05 s, 0.0 kB/s 2023-10-08T16:08:57.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:57.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2646 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.052 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0784e-05 s, 0.0 kB/s 2023-10-08T16:08:58.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.053 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2647 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.108 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.108 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.108 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5411e-05 s, 0.0 kB/s 2023-10-08T16:08:58.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.108 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2648 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.163 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.164 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.164 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9299e-05 s, 0.0 kB/s 2023-10-08T16:08:58.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.164 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2649 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.219 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.219 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1624e-05 s, 0.0 kB/s 2023-10-08T16:08:58.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.220 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2650 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.275 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.275 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.275 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3575e-05 s, 0.0 kB/s 2023-10-08T16:08:58.276 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.276 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2651 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.332 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.332 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.332 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6483e-05 s, 0.0 kB/s 2023-10-08T16:08:58.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.332 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2652 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.388 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.388 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.388 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.987e-05 s, 0.0 kB/s 2023-10-08T16:08:58.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.389 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2653 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.443 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.443 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.443 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1477e-05 s, 0.0 kB/s 2023-10-08T16:08:58.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.444 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2654 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.499 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.500 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.500 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0133e-05 s, 0.0 kB/s 2023-10-08T16:08:58.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.500 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2655 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0335e-05 s, 0.0 kB/s 2023-10-08T16:08:58.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2656 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.626 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.626 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1459e-05 s, 0.0 kB/s 2023-10-08T16:08:58.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.626 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2657 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.682 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7421e-05 s, 0.0 kB/s 2023-10-08T16:08:58.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.682 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2658 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.738 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9661e-05 s, 0.0 kB/s 2023-10-08T16:08:58.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.738 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2659 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.794 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0386e-05 s, 0.0 kB/s 2023-10-08T16:08:58.794 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.794 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2660 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.850 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2084e-05 s, 0.0 kB/s 2023-10-08T16:08:58.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2661 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.7817e-05 s, 0.0 kB/s 2023-10-08T16:08:58.906 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2662 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0411e-05 s, 0.0 kB/s 2023-10-08T16:08:58.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:58.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2663 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.017 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8649e-05 s, 0.0 kB/s 2023-10-08T16:08:59.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.019 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2664 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.075 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.075 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.9503e-05 s, 0.0 kB/s 2023-10-08T16:08:59.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.076 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2665 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.130 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.131 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.131 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9024e-05 s, 0.0 kB/s 2023-10-08T16:08:59.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.131 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2666 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.186 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.186 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.187 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-08T16:08:59.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.187 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2667 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8644e-05 s, 0.0 kB/s 2023-10-08T16:08:59.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.244 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2668 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.298 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0384e-05 s, 0.0 kB/s 2023-10-08T16:08:59.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.299 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2669 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.7173e-05 s, 0.0 kB/s 2023-10-08T16:08:59.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.354 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2670 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1839e-05 s, 0.0 kB/s 2023-10-08T16:08:59.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.410 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2671 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.466 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.466 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0855e-05 s, 0.0 kB/s 2023-10-08T16:08:59.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.466 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2672 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.522 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.7534e-05 s, 0.0 kB/s 2023-10-08T16:08:59.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.523 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2673 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0548e-05 s, 0.0 kB/s 2023-10-08T16:08:59.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2674 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.633 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.634 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.634 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8099e-05 s, 0.0 kB/s 2023-10-08T16:08:59.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.634 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2675 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.689 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.068e-05 s, 0.0 kB/s 2023-10-08T16:08:59.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.690 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2676 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.746 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.746 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.746 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0161e-05 s, 0.0 kB/s 2023-10-08T16:08:59.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.747 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2677 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4459e-05 s, 0.0 kB/s 2023-10-08T16:08:59.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.803 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2678 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.858 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0535e-05 s, 0.0 kB/s 2023-10-08T16:08:59.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.859 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2679 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.914 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0635e-05 s, 0.0 kB/s 2023-10-08T16:08:59.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.915 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2680 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:08:59.970 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:08:59.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:08:59.971 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.1063e-05 s, 0.0 kB/s 2023-10-08T16:08:59.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:08:59.971 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2681 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8408e-05 s, 0.0 kB/s 2023-10-08T16:09:00.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.027 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2682 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.082 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.083 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0429e-05 s, 0.0 kB/s 2023-10-08T16:09:00.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.084 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2683 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.139 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.139 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.139 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0223e-05 s, 0.0 kB/s 2023-10-08T16:09:00.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.140 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2684 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.195 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.196 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0946e-05 s, 0.0 kB/s 2023-10-08T16:09:00.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.196 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2685 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.251 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.252 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.252 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4385e-05 s, 0.0 kB/s 2023-10-08T16:09:00.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.253 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2686 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.307 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.307 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.307 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5881e-05 s, 0.0 kB/s 2023-10-08T16:09:00.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.308 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2687 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.364 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0949e-05 s, 0.0 kB/s 2023-10-08T16:09:00.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.364 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2688 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.420 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.420 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.7312e-05 s, 0.0 kB/s 2023-10-08T16:09:00.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.421 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2689 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.477 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8174e-05 s, 0.0 kB/s 2023-10-08T16:09:00.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.477 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2690 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.532 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3088e-05 s, 0.0 kB/s 2023-10-08T16:09:00.533 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.533 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2691 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.589 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.589 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9428e-05 s, 0.0 kB/s 2023-10-08T16:09:00.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.590 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2692 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1345e-05 s, 0.0 kB/s 2023-10-08T16:09:00.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2693 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.700 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.701 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0892e-05 s, 0.0 kB/s 2023-10-08T16:09:00.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.701 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2694 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.757 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9337e-05 s, 0.0 kB/s 2023-10-08T16:09:00.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.757 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2695 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.813 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.796e-05 s, 0.0 kB/s 2023-10-08T16:09:00.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.813 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2696 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.869 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.869 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.869 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.7859e-05 s, 0.0 kB/s 2023-10-08T16:09:00.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.869 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2697 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.925 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0687e-05 s, 0.0 kB/s 2023-10-08T16:09:00.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2698 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:00.981 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9779e-05 s, 0.0 kB/s 2023-10-08T16:09:00.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:00.982 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2699 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.037 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.037 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1974e-05 s, 0.0 kB/s 2023-10-08T16:09:01.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.038 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2700 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.093 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.093 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.8745e-05 s, 0.0 kB/s 2023-10-08T16:09:01.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2701 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.149 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.149 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2835e-05 s, 0.0 kB/s 2023-10-08T16:09:01.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.149 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2702 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.8468e-05 s, 0.0 kB/s 2023-10-08T16:09:01.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.205 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2703 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.261 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.261 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8491e-05 s, 0.0 kB/s 2023-10-08T16:09:01.261 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.261 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2704 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.316 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.319 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9576e-05 s, 0.0 kB/s 2023-10-08T16:09:01.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.319 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2705 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.921e-05 s, 0.0 kB/s 2023-10-08T16:09:01.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.373 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2706 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.429 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3037e-05 s, 0.0 kB/s 2023-10-08T16:09:01.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.430 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2707 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8521e-05 s, 0.0 kB/s 2023-10-08T16:09:01.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.487 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2708 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.542 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.542 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9053e-05 s, 0.0 kB/s 2023-10-08T16:09:01.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.543 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2709 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.598 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.598 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9472e-05 s, 0.0 kB/s 2023-10-08T16:09:01.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.599 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2710 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.653 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.654 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.654 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8634e-05 s, 0.0 kB/s 2023-10-08T16:09:01.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.655 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2711 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.711 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.711 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1678e-05 s, 0.0 kB/s 2023-10-08T16:09:01.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.712 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2712 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.767 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.767 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8882e-05 s, 0.0 kB/s 2023-10-08T16:09:01.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.768 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2713 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.822 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.823 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.823 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9504e-05 s, 0.0 kB/s 2023-10-08T16:09:01.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.823 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2714 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0493e-05 s, 0.0 kB/s 2023-10-08T16:09:01.879 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.879 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2715 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.934 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9801e-05 s, 0.0 kB/s 2023-10-08T16:09:01.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.935 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2716 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:01.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:01.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:01.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8657e-05 s, 0.0 kB/s 2023-10-08T16:09:01.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:01.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2717 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5118e-05 s, 0.0 kB/s 2023-10-08T16:09:02.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2718 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0593e-05 s, 0.0 kB/s 2023-10-08T16:09:02.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2719 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.160 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1963e-05 s, 0.0 kB/s 2023-10-08T16:09:02.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.161 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2720 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.216 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.216 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-08T16:09:02.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2721 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.272 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.272 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0328e-05 s, 0.0 kB/s 2023-10-08T16:09:02.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.273 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2722 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.327 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.327 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5458e-05 s, 0.0 kB/s 2023-10-08T16:09:02.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.328 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2723 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.383 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.384 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.973e-05 s, 0.0 kB/s 2023-10-08T16:09:02.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.384 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2724 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.440 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7753e-05 s, 0.0 kB/s 2023-10-08T16:09:02.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.441 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2725 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.496 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.497 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.497 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3625e-05 s, 0.0 kB/s 2023-10-08T16:09:02.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.497 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2726 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.553 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.554 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.554 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6545e-05 s, 0.0 kB/s 2023-10-08T16:09:02.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.554 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2727 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.610 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.610 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0681e-05 s, 0.0 kB/s 2023-10-08T16:09:02.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.610 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2728 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.665 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.665 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.665 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1336e-05 s, 0.0 kB/s 2023-10-08T16:09:02.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.666 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2729 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0137e-05 s, 0.0 kB/s 2023-10-08T16:09:02.722 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.722 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2730 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.777 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.778 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9136e-05 s, 0.0 kB/s 2023-10-08T16:09:02.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.778 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2731 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.833 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1511e-05 s, 0.0 kB/s 2023-10-08T16:09:02.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.834 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2732 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.889 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.889 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0146e-05 s, 0.0 kB/s 2023-10-08T16:09:02.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.889 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2733 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:02.944 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:02.945 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:02.945 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0179e-05 s, 0.0 kB/s 2023-10-08T16:09:02.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:02.945 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2734 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.001 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.001 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6396e-05 s, 0.0 kB/s 2023-10-08T16:09:03.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.002 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2735 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.058 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.058 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.058 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.129e-05 s, 0.0 kB/s 2023-10-08T16:09:03.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.058 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2736 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.114 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.114 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1727e-05 s, 0.0 kB/s 2023-10-08T16:09:03.115 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.115 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2737 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0678e-05 s, 0.0 kB/s 2023-10-08T16:09:03.171 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.171 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2738 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0848e-05 s, 0.0 kB/s 2023-10-08T16:09:03.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.227 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2739 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.282 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8535e-05 s, 0.0 kB/s 2023-10-08T16:09:03.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.283 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2740 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.338 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.338 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.338 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1304e-05 s, 0.0 kB/s 2023-10-08T16:09:03.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.338 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2741 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.395 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9789e-05 s, 0.0 kB/s 2023-10-08T16:09:03.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.396 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2742 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.451 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.451 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.624e-05 s, 0.0 kB/s 2023-10-08T16:09:03.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.452 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2743 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.507 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.508 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.508 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9945e-05 s, 0.0 kB/s 2023-10-08T16:09:03.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.508 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2744 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.564 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2005e-05 s, 0.0 kB/s 2023-10-08T16:09:03.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.565 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2745 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.620 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.620 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1464e-05 s, 0.0 kB/s 2023-10-08T16:09:03.620 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.621 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2746 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.675 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.676 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.676 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0033e-05 s, 0.0 kB/s 2023-10-08T16:09:03.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.676 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2747 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.732 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.732 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9938e-05 s, 0.0 kB/s 2023-10-08T16:09:03.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.732 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2748 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.788 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.788 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.788 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8713e-05 s, 0.0 kB/s 2023-10-08T16:09:03.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.788 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2749 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.844 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.844 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5011e-05 s, 0.0 kB/s 2023-10-08T16:09:03.844 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.844 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2750 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.899 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3855e-05 s, 0.0 kB/s 2023-10-08T16:09:03.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.900 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2751 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:03.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:03.956 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:03.956 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8431e-05 s, 0.0 kB/s 2023-10-08T16:09:03.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:03.957 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2752 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.013 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.013 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4765e-05 s, 0.0 kB/s 2023-10-08T16:09:04.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.014 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2753 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.069 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.070 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0561e-05 s, 0.0 kB/s 2023-10-08T16:09:04.070 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.070 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2754 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.125 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8563e-05 s, 0.0 kB/s 2023-10-08T16:09:04.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.125 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2755 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.171e-05 s, 0.0 kB/s 2023-10-08T16:09:04.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.181 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2756 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.237 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.237 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.237 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.5495e-05 s, 0.0 kB/s 2023-10-08T16:09:04.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.238 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2757 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.293 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.294 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.147e-05 s, 0.0 kB/s 2023-10-08T16:09:04.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.294 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2758 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.349 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.349 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.349 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.2502e-05 s, 0.0 kB/s 2023-10-08T16:09:04.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.350 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2759 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.405 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.405 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.405 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.4575e-05 s, 0.0 kB/s 2023-10-08T16:09:04.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.406 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2760 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.461 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.462 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.462 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1301e-05 s, 0.0 kB/s 2023-10-08T16:09:04.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.463 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2761 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.518 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0465e-05 s, 0.0 kB/s 2023-10-08T16:09:04.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.519 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2762 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.575 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.575 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.575 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7899e-05 s, 0.0 kB/s 2023-10-08T16:09:04.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.576 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2763 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.631 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.631 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.631 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9823e-05 s, 0.0 kB/s 2023-10-08T16:09:04.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.631 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2764 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.722 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.723 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.723 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0894e-05 s, 0.0 kB/s 2023-10-08T16:09:04.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.723 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2765 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.739 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.739 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6175e-05 s, 0.0 kB/s 2023-10-08T16:09:04.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.740 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2766 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.795 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1829e-05 s, 0.0 kB/s 2023-10-08T16:09:04.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2767 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.851 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.851 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9579e-05 s, 0.0 kB/s 2023-10-08T16:09:04.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2768 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0238e-05 s, 0.0 kB/s 2023-10-08T16:09:04.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2769 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:04.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:04.962 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:04.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9608e-05 s, 0.0 kB/s 2023-10-08T16:09:04.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:04.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2770 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.017 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.018 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.021e-05 s, 0.0 kB/s 2023-10-08T16:09:05.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2771 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.073 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.073 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4514e-05 s, 0.0 kB/s 2023-10-08T16:09:05.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.073 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2772 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.128 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.129 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9436e-05 s, 0.0 kB/s 2023-10-08T16:09:05.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.129 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2773 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.185 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0996e-05 s, 0.0 kB/s 2023-10-08T16:09:05.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.185 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2774 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3768e-05 s, 0.0 kB/s 2023-10-08T16:09:05.242 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.242 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2775 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.297 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0007e-05 s, 0.0 kB/s 2023-10-08T16:09:05.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.298 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2776 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2615e-05 s, 0.0 kB/s 2023-10-08T16:09:05.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.353 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2777 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.409 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.5931e-05 s, 0.0 kB/s 2023-10-08T16:09:05.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.409 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2778 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.465 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.0687e-05 s, 0.0 kB/s 2023-10-08T16:09:05.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.466 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2779 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6971e-05 s, 0.0 kB/s 2023-10-08T16:09:05.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.522 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2780 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.825e-05 s, 0.0 kB/s 2023-10-08T16:09:05.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2781 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9575e-05 s, 0.0 kB/s 2023-10-08T16:09:05.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.635 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2782 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.691 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.95e-05 s, 0.0 kB/s 2023-10-08T16:09:05.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.691 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2783 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.747 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.747 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1342e-05 s, 0.0 kB/s 2023-10-08T16:09:05.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.747 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2784 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.802 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.802 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.802 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.086e-05 s, 0.0 kB/s 2023-10-08T16:09:05.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.803 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2785 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.4964e-05 s, 0.0 kB/s 2023-10-08T16:09:05.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.858 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2786 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.914 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.914 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3552e-05 s, 0.0 kB/s 2023-10-08T16:09:05.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.915 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2787 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:05.970 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:05.970 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:05.971 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000103149 s, 0.0 kB/s 2023-10-08T16:09:05.971 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:05.971 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2788 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1055e-05 s, 0.0 kB/s 2023-10-08T16:09:06.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2789 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.083 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.083 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6261e-05 s, 0.0 kB/s 2023-10-08T16:09:06.083 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.083 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2790 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.138 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.139 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8679e-05 s, 0.0 kB/s 2023-10-08T16:09:06.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.140 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2791 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.196 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.196 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.196 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9895e-05 s, 0.0 kB/s 2023-10-08T16:09:06.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.196 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2792 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.252 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.252 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.252 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1025e-05 s, 0.0 kB/s 2023-10-08T16:09:06.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.253 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2793 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.308 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.308 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.308 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0518e-05 s, 0.0 kB/s 2023-10-08T16:09:06.308 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.309 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2794 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.363 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.364 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.364 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1308e-05 s, 0.0 kB/s 2023-10-08T16:09:06.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.364 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2795 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.419 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.419 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1165e-05 s, 0.0 kB/s 2023-10-08T16:09:06.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.420 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2796 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.475 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.476 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 0.000104528 s, 0.0 kB/s 2023-10-08T16:09:06.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.476 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2797 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.531 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.531 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9517e-05 s, 0.0 kB/s 2023-10-08T16:09:06.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.532 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2798 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.587 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.588 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4052e-05 s, 0.0 kB/s 2023-10-08T16:09:06.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.588 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2799 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.644 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.644 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.8582e-05 s, 0.0 kB/s 2023-10-08T16:09:06.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.645 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2800 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.699 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.700 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.700 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.082e-05 s, 0.0 kB/s 2023-10-08T16:09:06.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.700 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2801 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.756 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.756 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7874e-05 s, 0.0 kB/s 2023-10-08T16:09:06.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.756 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2802 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.812 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.812 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.812 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0496e-05 s, 0.0 kB/s 2023-10-08T16:09:06.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.813 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2803 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.868 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.869 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.869 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0256e-05 s, 0.0 kB/s 2023-10-08T16:09:06.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.869 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2804 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.925 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.926 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8902e-05 s, 0.0 kB/s 2023-10-08T16:09:06.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.926 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2805 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:06.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:06.982 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:06.982 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6454e-05 s, 0.0 kB/s 2023-10-08T16:09:06.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:06.983 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2806 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.038 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5515e-05 s, 0.0 kB/s 2023-10-08T16:09:07.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.038 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2807 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.094 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.094 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.094 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9461e-05 s, 0.0 kB/s 2023-10-08T16:09:07.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.094 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2808 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1217e-05 s, 0.0 kB/s 2023-10-08T16:09:07.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.151 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2809 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.205 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.205 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9697e-05 s, 0.0 kB/s 2023-10-08T16:09:07.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.206 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2810 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.262 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.262 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.262 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0788e-05 s, 0.0 kB/s 2023-10-08T16:09:07.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.263 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2811 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.318 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.318 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.318 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1878e-05 s, 0.0 kB/s 2023-10-08T16:09:07.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.318 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2812 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.373 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.374 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.374 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0885e-05 s, 0.0 kB/s 2023-10-08T16:09:07.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.374 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2813 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.430 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.430 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0238e-05 s, 0.0 kB/s 2023-10-08T16:09:07.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.431 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2814 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.486 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.486 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3991e-05 s, 0.0 kB/s 2023-10-08T16:09:07.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.487 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2815 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.543 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.544 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.544 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8808e-05 s, 0.0 kB/s 2023-10-08T16:09:07.544 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.544 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2816 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.602 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.603 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.603 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5654e-05 s, 0.0 kB/s 2023-10-08T16:09:07.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.603 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2817 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.658 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.658 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3428e-05 s, 0.0 kB/s 2023-10-08T16:09:07.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.659 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2818 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.714 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.715 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1879e-05 s, 0.0 kB/s 2023-10-08T16:09:07.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.715 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2819 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.770 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.770 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0768e-05 s, 0.0 kB/s 2023-10-08T16:09:07.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.771 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2820 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.827 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.827 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9715e-05 s, 0.0 kB/s 2023-10-08T16:09:07.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.828 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2821 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.883 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.883 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9725e-05 s, 0.0 kB/s 2023-10-08T16:09:07.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.883 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2822 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.938 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.938 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.939 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3322e-05 s, 0.0 kB/s 2023-10-08T16:09:07.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.940 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2823 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:07.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:07.996 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:07.996 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1124e-05 s, 0.0 kB/s 2023-10-08T16:09:07.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:07.997 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2824 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.052 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.053 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4042e-05 s, 0.0 kB/s 2023-10-08T16:09:08.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.053 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2825 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.109 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.086e-05 s, 0.0 kB/s 2023-10-08T16:09:08.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.110 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2826 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1901e-05 s, 0.0 kB/s 2023-10-08T16:09:08.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.167 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2827 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.224 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.225 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9465e-05 s, 0.0 kB/s 2023-10-08T16:09:08.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.225 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2828 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.281 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.281 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.281 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3685e-05 s, 0.0 kB/s 2023-10-08T16:09:08.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.281 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2829 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.337 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.337 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1935e-05 s, 0.0 kB/s 2023-10-08T16:09:08.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.338 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2830 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.393 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.393 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.5699e-05 s, 0.0 kB/s 2023-10-08T16:09:08.394 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.394 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2831 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.449 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.449 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.449 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8743e-05 s, 0.0 kB/s 2023-10-08T16:09:08.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.449 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2832 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.506 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.506 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0788e-05 s, 0.0 kB/s 2023-10-08T16:09:08.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.506 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2833 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.5081e-05 s, 0.0 kB/s 2023-10-08T16:09:08.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.563 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2834 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.618 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.618 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.618 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9331e-05 s, 0.0 kB/s 2023-10-08T16:09:08.619 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.619 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2835 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.674 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.674 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.675 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7948e-05 s, 0.0 kB/s 2023-10-08T16:09:08.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.675 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2836 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.730 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.731 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.731 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0263e-05 s, 0.0 kB/s 2023-10-08T16:09:08.731 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.731 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2837 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.786 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.787 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.787 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0338e-05 s, 0.0 kB/s 2023-10-08T16:09:08.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.787 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2838 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.843 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.843 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0672e-05 s, 0.0 kB/s 2023-10-08T16:09:08.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.843 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2839 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.898 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.899 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0486e-05 s, 0.0 kB/s 2023-10-08T16:09:08.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.899 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2840 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9518e-05 s, 0.0 kB/s 2023-10-08T16:09:08.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:08.956 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2841 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.011 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.011 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9538e-05 s, 0.0 kB/s 2023-10-08T16:09:09.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.011 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2842 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5229e-05 s, 0.0 kB/s 2023-10-08T16:09:09.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.068 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2843 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1271e-05 s, 0.0 kB/s 2023-10-08T16:09:09.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2844 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.180 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.180 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0476e-05 s, 0.0 kB/s 2023-10-08T16:09:09.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.180 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2845 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.235 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.236 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.236 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8698e-05 s, 0.0 kB/s 2023-10-08T16:09:09.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.236 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2846 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.7195e-05 s, 0.0 kB/s 2023-10-08T16:09:09.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.293 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2847 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.347 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.348 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.348 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7922e-05 s, 0.0 kB/s 2023-10-08T16:09:09.348 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.348 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2848 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.404 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.404 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.404 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1458e-05 s, 0.0 kB/s 2023-10-08T16:09:09.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.404 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2849 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.2286e-05 s, 0.0 kB/s 2023-10-08T16:09:09.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.460 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2850 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.516 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.516 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.517 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0722e-05 s, 0.0 kB/s 2023-10-08T16:09:09.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.517 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2851 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.574 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.575 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5798e-05 s, 0.0 kB/s 2023-10-08T16:09:09.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.575 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2852 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.630 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.630 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1345e-05 s, 0.0 kB/s 2023-10-08T16:09:09.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.631 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2853 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.685 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.686 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.686 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.6855e-05 s, 0.0 kB/s 2023-10-08T16:09:09.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.686 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2854 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.742 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.742 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.742 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0577e-05 s, 0.0 kB/s 2023-10-08T16:09:09.742 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.743 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2855 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.0046e-05 s, 0.0 kB/s 2023-10-08T16:09:09.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.798 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2856 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.853 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.853 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9699e-05 s, 0.0 kB/s 2023-10-08T16:09:09.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.853 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2857 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.909 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.910 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.910 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9572e-05 s, 0.0 kB/s 2023-10-08T16:09:09.910 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.910 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2858 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:09.966 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:09.966 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:09.966 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2118e-05 s, 0.0 kB/s 2023-10-08T16:09:09.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:09.967 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2859 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.022 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:10.022 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:10.022 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8945e-05 s, 0.0 kB/s 2023-10-08T16:09:10.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:10.023 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2860 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:10.078 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:10.078 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0089e-05 s, 0.0 kB/s 2023-10-08T16:09:10.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:10.079 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2861 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.133 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:10.134 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.8317e-05 s, 0.0 kB/s 2023-10-08T16:09:10.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:10.134 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2862 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.190 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:10.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:10.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0629e-05 s, 0.0 kB/s 2023-10-08T16:09:10.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:10.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2863 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:10.245 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:10.245 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6789e-05 s, 0.0 kB/s 2023-10-08T16:09:10.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:10.247 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2864 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:10.356 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:11.298 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:11.298 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0335e-05 s, 0.0 kB/s 2023-10-08T16:09:11.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:11.298 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2865 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:11.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:11.317 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:11.317 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4695e-05 s, 0.0 kB/s 2023-10-08T16:09:11.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:11.318 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2866 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:11.375 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:13.243 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.3196e-05 s, 0.0 kB/s 2023-10-08T16:09:13.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:13.244 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2867 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:13.264 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:13.403 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.8709e-05 s, 0.0 kB/s 2023-10-08T16:09:13.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:13.405 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2868 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:13.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:13.464 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:13.464 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.9434e-05 s, 0.0 kB/s 2023-10-08T16:09:13.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:13.464 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2869 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:13.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:13.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:13.523 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4352e-05 s, 0.0 kB/s 2023-10-08T16:09:13.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:13.523 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2870 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:13.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:09:13.582 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:13.582 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:13.582 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.7096e-05 s, 0.0 kB/s 2023-10-08T16:09:13.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:13.583 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2871 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:13.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:09:14.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.065 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.4113e-05 s, 0.0 kB/s 2023-10-08T16:09:14.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.065 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2872 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.122 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.123 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.123 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2586e-05 s, 0.0 kB/s 2023-10-08T16:09:14.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.124 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2873 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.181 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3277e-05 s, 0.0 kB/s 2023-10-08T16:09:14.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.182 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2874 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.238 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.238 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.238 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3597e-05 s, 0.0 kB/s 2023-10-08T16:09:14.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.238 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2875 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.295 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.295 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.296 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3545e-05 s, 0.0 kB/s 2023-10-08T16:09:14.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.296 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2876 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2614e-05 s, 0.0 kB/s 2023-10-08T16:09:14.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.354 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2877 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.409 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.3841e-05 s, 0.0 kB/s 2023-10-08T16:09:14.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.410 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2878 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.466 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.467 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.467 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9665e-05 s, 0.0 kB/s 2023-10-08T16:09:14.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.467 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2879 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7042e-05 s, 0.0 kB/s 2023-10-08T16:09:14.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.524 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2880 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.580 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.581 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9367e-05 s, 0.0 kB/s 2023-10-08T16:09:14.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.582 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2881 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.638 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.638 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.639 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2688e-05 s, 0.0 kB/s 2023-10-08T16:09:14.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.639 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2882 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.695 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.695 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6388e-05 s, 0.0 kB/s 2023-10-08T16:09:14.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.696 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2883 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.751 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.752 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.752 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8762e-05 s, 0.0 kB/s 2023-10-08T16:09:14.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.752 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2884 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.807 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.808 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.4199e-05 s, 0.0 kB/s 2023-10-08T16:09:14.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.808 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2885 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.864 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.864 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9214e-05 s, 0.0 kB/s 2023-10-08T16:09:14.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.865 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2886 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.920 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.920 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.920 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9224e-05 s, 0.0 kB/s 2023-10-08T16:09:14.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.920 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2887 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:14.976 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1495e-05 s, 0.0 kB/s 2023-10-08T16:09:14.976 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:14.977 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2888 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.033 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.034 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.034 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4509e-05 s, 0.0 kB/s 2023-10-08T16:09:15.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.034 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2889 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.089 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.089 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9259e-05 s, 0.0 kB/s 2023-10-08T16:09:15.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.090 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2890 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.145 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.5976e-05 s, 0.0 kB/s 2023-10-08T16:09:15.145 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.145 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2891 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.201 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.201 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2355e-05 s, 0.0 kB/s 2023-10-08T16:09:15.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.201 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2892 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.257 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2308e-05 s, 0.0 kB/s 2023-10-08T16:09:15.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.258 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2893 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.997e-05 s, 0.0 kB/s 2023-10-08T16:09:15.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.315 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2894 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.370 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.371 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.2878e-05 s, 0.0 kB/s 2023-10-08T16:09:15.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.371 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2895 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2184e-05 s, 0.0 kB/s 2023-10-08T16:09:15.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.478 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2896 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.493 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.494 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0806e-05 s, 0.0 kB/s 2023-10-08T16:09:15.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.494 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2897 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.550 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.6776e-05 s, 0.0 kB/s 2023-10-08T16:09:15.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.550 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2898 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.606 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.606 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8459e-05 s, 0.0 kB/s 2023-10-08T16:09:15.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.606 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2899 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.661 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1507e-05 s, 0.0 kB/s 2023-10-08T16:09:15.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.662 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2900 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.717 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.718 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9502e-05 s, 0.0 kB/s 2023-10-08T16:09:15.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.718 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2901 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.773 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.773 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.3834e-05 s, 0.0 kB/s 2023-10-08T16:09:15.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.774 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2902 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.828 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.829 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9492e-05 s, 0.0 kB/s 2023-10-08T16:09:15.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.829 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2903 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.884 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.885 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9727e-05 s, 0.0 kB/s 2023-10-08T16:09:15.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.885 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2904 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.941 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6294e-05 s, 0.0 kB/s 2023-10-08T16:09:15.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.941 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2905 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:15.997 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:15.998 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9983e-05 s, 0.0 kB/s 2023-10-08T16:09:15.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:15.998 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2906 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.054 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8451e-05 s, 0.0 kB/s 2023-10-08T16:09:16.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.054 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2907 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.110 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8246e-05 s, 0.0 kB/s 2023-10-08T16:09:16.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.110 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2908 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.166 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.166 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2478e-05 s, 0.0 kB/s 2023-10-08T16:09:16.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.166 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2909 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.221 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.222 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.985e-05 s, 0.0 kB/s 2023-10-08T16:09:16.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.222 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2910 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.277 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.277 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.278 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.0476e-05 s, 0.0 kB/s 2023-10-08T16:09:16.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.278 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2911 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.334 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.334 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.334 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9423e-05 s, 0.0 kB/s 2023-10-08T16:09:16.335 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.335 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2912 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0465e-05 s, 0.0 kB/s 2023-10-08T16:09:16.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.391 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2913 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.445 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.446 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.4988e-05 s, 0.0 kB/s 2023-10-08T16:09:16.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.446 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2914 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.799e-05 s, 0.0 kB/s 2023-10-08T16:09:16.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.502 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2915 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.558 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1068e-05 s, 0.0 kB/s 2023-10-08T16:09:16.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.559 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2916 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.614 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.614 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0279e-05 s, 0.0 kB/s 2023-10-08T16:09:16.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.615 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2917 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.670 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.1691e-05 s, 0.0 kB/s 2023-10-08T16:09:16.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.671 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2918 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.726 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9425e-05 s, 0.0 kB/s 2023-10-08T16:09:16.727 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.727 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2919 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.782 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.783 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.783 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8892e-05 s, 0.0 kB/s 2023-10-08T16:09:16.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.784 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2920 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.839 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.9922e-05 s, 0.0 kB/s 2023-10-08T16:09:16.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.840 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2921 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.895 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.896 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7155e-05 s, 0.0 kB/s 2023-10-08T16:09:16.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.896 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2922 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:16.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:16.952 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:16.952 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6751e-05 s, 0.0 kB/s 2023-10-08T16:09:16.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:16.953 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2923 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.008 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.009 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9709e-05 s, 0.0 kB/s 2023-10-08T16:09:17.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.009 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2924 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7978e-05 s, 0.0 kB/s 2023-10-08T16:09:17.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.064 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2925 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.119 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.119 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4528e-05 s, 0.0 kB/s 2023-10-08T16:09:17.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.120 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2926 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.176 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.176 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9662e-05 s, 0.0 kB/s 2023-10-08T16:09:17.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.177 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2927 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.232 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7355e-05 s, 0.0 kB/s 2023-10-08T16:09:17.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.233 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2928 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.287 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0925e-05 s, 0.0 kB/s 2023-10-08T16:09:17.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.288 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2929 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.344 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.344 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1017e-05 s, 0.0 kB/s 2023-10-08T16:09:17.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.345 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2930 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2185e-05 s, 0.0 kB/s 2023-10-08T16:09:17.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.401 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2931 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.455 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.456 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.456 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.397e-05 s, 0.0 kB/s 2023-10-08T16:09:17.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.456 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2932 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 6.9676e-05 s, 0.0 kB/s 2023-10-08T16:09:17.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.513 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2933 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.570 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.571 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.182e-05 s, 0.0 kB/s 2023-10-08T16:09:17.571 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.571 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2934 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.017e-05 s, 0.0 kB/s 2023-10-08T16:09:17.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.626 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2935 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.682 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.682 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7987e-05 s, 0.0 kB/s 2023-10-08T16:09:17.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.682 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2936 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.738 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.738 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4731e-05 s, 0.0 kB/s 2023-10-08T16:09:17.738 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.739 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2937 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.794 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.794 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7901e-05 s, 0.0 kB/s 2023-10-08T16:09:17.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.795 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2938 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.850 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.850 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4738e-05 s, 0.0 kB/s 2023-10-08T16:09:17.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.851 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2939 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.906 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.906 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8837e-05 s, 0.0 kB/s 2023-10-08T16:09:17.907 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.907 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2940 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:17.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:17.961 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:17.962 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1761e-05 s, 0.0 kB/s 2023-10-08T16:09:17.962 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:17.962 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2941 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.018 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.018 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0191e-05 s, 0.0 kB/s 2023-10-08T16:09:18.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.018 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2942 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.074 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.075 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2903e-05 s, 0.0 kB/s 2023-10-08T16:09:18.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.075 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2943 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.129 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.130 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.9845e-05 s, 0.0 kB/s 2023-10-08T16:09:18.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.130 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2944 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.184 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.184 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1496e-05 s, 0.0 kB/s 2023-10-08T16:09:18.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.185 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2945 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.241 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.241 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.996e-05 s, 0.0 kB/s 2023-10-08T16:09:18.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.241 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2946 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.297 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.297 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4868e-05 s, 0.0 kB/s 2023-10-08T16:09:18.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.297 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2947 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.352 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.353 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.353 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0879e-05 s, 0.0 kB/s 2023-10-08T16:09:18.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.354 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2948 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.410 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0542e-05 s, 0.0 kB/s 2023-10-08T16:09:18.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.411 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2949 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.465 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.466 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.466 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.8693e-05 s, 0.0 kB/s 2023-10-08T16:09:18.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.466 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2950 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.522 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.522 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.5108e-05 s, 0.0 kB/s 2023-10-08T16:09:18.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.523 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2951 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.579 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.579 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0521e-05 s, 0.0 kB/s 2023-10-08T16:09:18.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.579 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2952 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 4.6862e-05 s, 0.0 kB/s 2023-10-08T16:09:18.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.635 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2953 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0802e-05 s, 0.0 kB/s 2023-10-08T16:09:18.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.692 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2954 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.746 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.746 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.747 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.7582e-05 s, 0.0 kB/s 2023-10-08T16:09:18.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.748 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2955 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.804 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 9.1394e-05 s, 0.0 kB/s 2023-10-08T16:09:18.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.805 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2956 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.860 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.860 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8874e-05 s, 0.0 kB/s 2023-10-08T16:09:18.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.860 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2957 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.916 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.916 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9066e-05 s, 0.0 kB/s 2023-10-08T16:09:18.917 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.917 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2958 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:18.971 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:18.972 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.4581e-05 s, 0.0 kB/s 2023-10-08T16:09:18.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:18.972 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2959 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.028 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.7383e-05 s, 0.0 kB/s 2023-10-08T16:09:19.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.028 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2960 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.085 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0611e-05 s, 0.0 kB/s 2023-10-08T16:09:19.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.086 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2961 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.141 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.142 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9613e-05 s, 0.0 kB/s 2023-10-08T16:09:19.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.142 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2962 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.198 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.198 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9598e-05 s, 0.0 kB/s 2023-10-08T16:09:19.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.198 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2963 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.254 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4391e-05 s, 0.0 kB/s 2023-10-08T16:09:19.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.255 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2964 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0505e-05 s, 0.0 kB/s 2023-10-08T16:09:19.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.313 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2965 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.369 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.369 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.6968e-05 s, 0.0 kB/s 2023-10-08T16:09:19.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.370 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2966 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.425 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.5698e-05 s, 0.0 kB/s 2023-10-08T16:09:19.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.425 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2967 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.481 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9603e-05 s, 0.0 kB/s 2023-10-08T16:09:19.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.481 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2968 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.536 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.536 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8659e-05 s, 0.0 kB/s 2023-10-08T16:09:19.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.537 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2969 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.593 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.2479e-05 s, 0.0 kB/s 2023-10-08T16:09:19.593 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.593 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2970 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.649 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9635e-05 s, 0.0 kB/s 2023-10-08T16:09:19.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.650 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2971 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.3935e-05 s, 0.0 kB/s 2023-10-08T16:09:19.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.707 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2972 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.762 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.762 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.762 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9417e-05 s, 0.0 kB/s 2023-10-08T16:09:19.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.762 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2973 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.9058e-05 s, 0.0 kB/s 2023-10-08T16:09:19.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.819 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2974 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.874 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0175e-05 s, 0.0 kB/s 2023-10-08T16:09:19.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.875 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2975 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.929 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.929 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.929 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0771e-05 s, 0.0 kB/s 2023-10-08T16:09:19.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.930 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2976 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.2983e-05 s, 0.0 kB/s 2023-10-08T16:09:19.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:19.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2977 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.040 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.041 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.9971e-05 s, 0.0 kB/s 2023-10-08T16:09:20.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.041 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2978 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.096 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.096 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0268e-05 s, 0.0 kB/s 2023-10-08T16:09:20.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.097 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2979 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.152 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.152 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.1492e-05 s, 0.0 kB/s 2023-10-08T16:09:20.153 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.153 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2980 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.208 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0787e-05 s, 0.0 kB/s 2023-10-08T16:09:20.209 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.209 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2981 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.263 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.263 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0291e-05 s, 0.0 kB/s 2023-10-08T16:09:20.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.264 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2982 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.319 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.320 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4398e-05 s, 0.0 kB/s 2023-10-08T16:09:20.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.320 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2983 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.376 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8654e-05 s, 0.0 kB/s 2023-10-08T16:09:20.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.376 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2984 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.431 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.432 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.432 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8339e-05 s, 0.0 kB/s 2023-10-08T16:09:20.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.432 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2985 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.487 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8875e-05 s, 0.0 kB/s 2023-10-08T16:09:20.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.488 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2986 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.578 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.578 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.5938e-05 s, 0.0 kB/s 2023-10-08T16:09:20.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.578 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2987 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.595 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1288e-05 s, 0.0 kB/s 2023-10-08T16:09:20.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2988 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.651 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.2196e-05 s, 0.0 kB/s 2023-10-08T16:09:20.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.651 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2989 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.707 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.4964e-05 s, 0.0 kB/s 2023-10-08T16:09:20.708 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.708 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2990 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.764 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.764 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 2.8385e-05 s, 0.0 kB/s 2023-10-08T16:09:20.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.765 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2991 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.820 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.821 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.821 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.0328e-05 s, 0.0 kB/s 2023-10-08T16:09:20.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.822 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2992 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.877 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0733e-05 s, 0.0 kB/s 2023-10-08T16:09:20.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.878 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2993 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.935 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.935 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.2302e-05 s, 0.0 kB/s 2023-10-08T16:09:20.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.936 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2994 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:20.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:20.991 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:20.991 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.0176e-05 s, 0.0 kB/s 2023-10-08T16:09:20.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:20.992 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2995 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:21.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:21.047 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:21.047 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 5.1863e-05 s, 0.0 kB/s 2023-10-08T16:09:21.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:21.047 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2996 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:21.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:21.103 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:21.103 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 7.8276e-05 s, 0.0 kB/s 2023-10-08T16:09:21.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:21.104 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2997 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:21.159 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.9571e-05 s, 0.0 kB/s 2023-10-08T16:09:21.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:21.160 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2998 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:21.215 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 8.6918e-05 s, 0.0 kB/s 2023-10-08T16:09:21.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:21.216 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvolume_0000000000439977/497c5d7c-72c5-4629-bac1-cd6002fc8c37/subvolume_file.2999 bs=1M conv=fdatasync count=0 seek=0) 2023-10-08T16:09:21.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records in 2023-10-08T16:09:21.271 INFO:teuthology.orchestra.run.smithi170.stderr:0+0 records out 2023-10-08T16:09:21.271 INFO:teuthology.orchestra.run.smithi170.stderr:0 bytes copied, 3.1453e-05 s, 0.0 kB/s 2023-10-08T16:09:21.271 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvolume_0000000000439977 2023-10-08T16:09:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.427+0000 7fcdb0010700 1 Processor -- start 2023-10-08T16:09:21.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 -- start start 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda815a160 con 0x7fcda8159890 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda815df50 con 0x7fcda8159c60 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.428+0000 7fcdb0010700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda80d13a0 con 0x7fcda815db10 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48758/0 (socket says 172.21.15.170:48758) 2023-10-08T16:09:21.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3986687078 learned_addr learned my addr 172.21.15.170:0/3986687078 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3986687078 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 msgr2=0x7fcda8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 --2- 172.21.15.170:0/3986687078 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3986687078 >> v1:172.21.15.191:6789/0 conn(0x7fcda8159890 legacy=0x7fcda815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3986687078 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda815e140 con 0x7fcda8159c60 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdad5ab700 1 --2- 172.21.15.170:0/3986687078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda815d5d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcda806a890 tx=0x7fcd9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d166d7f2284a751 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdacdaa700 1 -- 172.21.15.170:0/3986687078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00b820 con 0x7fcda8159c60 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdacdaa700 1 -- 172.21.15.170:0/3986687078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c00b980 con 0x7fcda8159c60 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.429+0000 7fcdacdaa700 1 -- 172.21.15.170:0/3986687078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c00bd20 con 0x7fcda8159c60 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 -- 172.21.15.170:0/3986687078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 msgr2=0x7fcda815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3986687078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda815d5d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcda806a890 tx=0x7fcd9c009250 comp rx=0 tx=0).stop 2023-10-08T16:09:21.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 -- 172.21.15.170:0/3986687078 shutdown_connections 2023-10-08T16:09:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3986687078 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3986687078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 -- 172.21.15.170:0/3986687078 >> 172.21.15.170:0/3986687078 conn(0x7fcda8068e20 msgr2=0x7fcda80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 -- 172.21.15.170:0/3986687078 shutdown_connections 2023-10-08T16:09:21.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.430+0000 7fcdb0010700 1 -- 172.21.15.170:0/3986687078 wait complete. 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.431+0000 7fcdb0010700 1 Processor -- start 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.431+0000 7fcdb0010700 1 -- start start 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda815db10 0x7fcda8154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda814ce60 con 0x7fcda8159c60 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c8e80 con 0x7fcda815db10 2023-10-08T16:09:21.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c7610 con 0x7fcda8159890 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48770/0 (socket says 172.21.15.170:48770) 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3760903195 learned_addr learned my addr 172.21.15.170:0/3760903195 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 msgr2=0x7fcda80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda815db10 msgr2=0x7fcda8154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda815db10 0x7fcda8154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 -- 172.21.15.170:0/3760903195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0094f0 con 0x7fcda8159c60 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdad5ab700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003080 tx=0x7fcd9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c011e20 con 0x7fcda8159c60 2023-10-08T16:09:21.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c01f070 con 0x7fcda8159c60 2023-10-08T16:09:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c01a8a0 con 0x7fcda8159c60 2023-10-08T16:09:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.432+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda80c9140 con 0x7fcda8159c60 2023-10-08T16:09:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.433+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda814aed0 con 0x7fcda8159c60 2023-10-08T16:09:21.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.434+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda804c3c0 con 0x7fcda8159c60 2023-10-08T16:09:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.434+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7fcd9c01ac90 con 0x7fcda8159c60 2023-10-08T16:09:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.434+0000 7fcda5ffb700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 0x7fcd94069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.434+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fcd9c021070 con 0x7fcda8159c60 2023-10-08T16:09:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.434+0000 7fcdaddac700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 0x7fcd94069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.435+0000 7fcdaddac700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 0x7fcd94069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd98002d90 tx=0x7fcd98007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.439+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd9c053b40 con 0x7fcda8159c60 2023-10-08T16:09:21.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.656+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvolume_0000000000439977", "target": ["mon-mgr", ""]}) v1 -- 0x7fcda806a5b0 con 0x7fcd94066fc0 2023-10-08T16:09:21.660 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.660+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.660 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.660+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.660 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.660+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.660 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.660+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.660 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.660+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.716+0000 7fcda5ffb700 1 -- 172.21.15.170:0/3760903195 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcda806a5b0 con 0x7fcd94066fc0 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 msgr2=0x7fcd94069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 0x7fcd94069470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcd98002d90 tx=0x7fcd98007ea0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 msgr2=0x7fcda814c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c003080 tx=0x7fcd9c016000 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 shutdown_connections 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda815db10 0x7fcda8154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd94066fc0 0x7fcd94069470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 --2- 172.21.15.170:0/3760903195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159c60 0x7fcda814c920 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 >> 172.21.15.170:0/3760903195 conn(0x7fcda8068e20 msgr2=0x7fcda80cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 shutdown_connections 2023-10-08T16:09:21.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.718+0000 7fcdb0010700 1 -- 172.21.15.170:0/3760903195 wait complete. 2023-10-08T16:09:21.720 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.719+0000 7f60a5035700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.720 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.719+0000 7f60a5035700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.720 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.719+0000 7f60a5035700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.720 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.719+0000 7f60a5035700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.721 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.719+0000 7f60a5035700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-08T16:09:21.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.732+0000 7f60a4834700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.732+0000 7f60a4834700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.732+0000 7f60a4834700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.732+0000 7f60a4834700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.732 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:21.732+0000 7f60a4834700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:21.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.855+0000 7f5030055700 1 Processor -- start 2023-10-08T16:09:21.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.855+0000 7f5030055700 1 -- start start 2023-10-08T16:09:21.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 0x7f502815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5028159c60 0x7f502815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f502815db10 0x7f5028069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502815a160 con 0x7f502815db10 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280d13a0 con 0x7f5028159890 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f5030055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502815e140 con 0x7f5028159c60 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f502e5f2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f502815db10 0x7f5028069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f502e5f2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f502815db10 0x7f5028069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48846/0 (socket says 172.21.15.170:48846) 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f502e5f2700 1 -- 172.21.15.170:0/4041670283 learned_addr learned my addr 172.21.15.170:0/4041670283 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f502ddf1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 0x7f502815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.856+0000 7f502d5f0700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5028159c60 0x7f502815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 -- 172.21.15.170:0/4041670283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5028159c60 msgr2=0x7f502815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5028159c60 0x7f502815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 -- 172.21.15.170:0/4041670283 >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 msgr2=0x7f502815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:21.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 --2- 172.21.15.170:0/4041670283 >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 0x7f502815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 -- 172.21.15.170:0/4041670283 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502815a3d0 con 0x7f502815db10 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502ddf1700 1 --2- 172.21.15.170:0/4041670283 >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 0x7f502815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502e5f2700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f502815db10 0x7f5028069560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f50280d1090 tx=0x7f5018009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36646dd51d3d805 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502cdef700 1 -- 172.21.15.170:0/4041670283 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800e070 con 0x7f502815db10 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502cdef700 1 -- 172.21.15.170:0/4041670283 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800b970 con 0x7f502815db10 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.857+0000 7f502cdef700 1 -- 172.21.15.170:0/4041670283 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50180092e0 con 0x7f502815db10 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 -- 172.21.15.170:0/4041670283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f502815db10 msgr2=0x7f5028069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f502815db10 0x7f5028069560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f50280d1090 tx=0x7f5018009840 comp rx=0 tx=0).stop 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 -- 172.21.15.170:0/4041670283 shutdown_connections 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f502815db10 0x7f5028069560 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 --2- 172.21.15.170:0/4041670283 >> v2:172.21.15.191:3300/0 conn(0x7f5028159890 0x7f502815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 --2- 172.21.15.170:0/4041670283 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5028159c60 0x7f502815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 -- 172.21.15.170:0/4041670283 >> 172.21.15.170:0/4041670283 conn(0x7f5028068e20 msgr2=0x7f50280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:21.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 -- 172.21.15.170:0/4041670283 shutdown_connections 2023-10-08T16:09:21.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.858+0000 7f5030055700 1 -- 172.21.15.170:0/4041670283 wait complete. 2023-10-08T16:09:21.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.859+0000 7f5030055700 1 Processor -- start 2023-10-08T16:09:21.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.859+0000 7f5030055700 1 -- start start 2023-10-08T16:09:21.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5028159890 0x7f50280b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f502815db10 0x7f5028149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280cd890 con 0x7f5028159c60 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50280b4ad0 con 0x7f5028159890 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50280b47f0 con 0x7f502815db10 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48860/0 (socket says 172.21.15.170:48860) 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 -- 172.21.15.170:0/125040010 learned_addr learned my addr 172.21.15.170:0/125040010 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502ddf1700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5028159890 0x7f50280b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502e5f2700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f502815db10 0x7f5028149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 -- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f502815db10 msgr2=0x7f5028149170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f502815db10 0x7f5028149170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 -- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5028159890 msgr2=0x7f50280b5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5028159890 0x7f50280b5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 -- 172.21.15.170:0/125040010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5020009420 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f502d5f0700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f501c00ec50 tx=0x7f501c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00c850 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00c9b0 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5018009ae0 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.860+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00cd70 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.861+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50280cd5a0 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.862+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50280b45b0 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.862+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 8) v1 ==== 84791+0+0 (secure 0 0 0) 0x7f501c013680 con 0x7f5028159c60 2023-10-08T16:09:21.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.862+0000 7f5025ffb700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 0x7f5014069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:21.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.862+0000 7f502ddf1700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 0x7f5014069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:21.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.862+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f501c0847f0 con 0x7f5028159c60 2023-10-08T16:09:21.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.863+0000 7f502ddf1700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 0x7f5014069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5020009f40 tx=0x7f50200091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:21.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:21.867+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50280b45b0 con 0x7f5028159c60 2023-10-08T16:09:22.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.093+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f50280cd0c0 con 0x7f5014066fc0 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.103+0000 7f5025ffb700 1 -- 172.21.15.170:0/125040010 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+539 (secure 0 0 0) 0x7f502001a030 con 0x7f5014066fc0 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2023-10-08T16:09:22.104 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "pending_subvolume_deletions": 1, 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "pools": { 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "data": [ 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365956726784, 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_data", 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 8192 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:22.105 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": [ 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365956726784, 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_metadata", 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 21086208 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout: "used_size": 145 2023-10-08T16:09:22.106 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:09:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 msgr2=0x7f5014069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:22.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 0x7f5014069470 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5020009f40 tx=0x7f50200091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 msgr2=0x7f50280b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f501c00ec50 tx=0x7f501c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 shutdown_connections 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f502815db10 0x7f5028149170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5028159890 0x7f50280b5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5014066fc0 0x7f5014069470 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 --2- 172.21.15.170:0/125040010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5028159c60 0x7f50280b3f50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 >> 172.21.15.170:0/125040010 conn(0x7f5028068e20 msgr2=0x7f5028149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 shutdown_connections 2023-10-08T16:09:22.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:22.106+0000 7f5030055700 1 -- 172.21.15.170:0/125040010 wait complete. 2023-10-08T16:09:22.119 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:09:22.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:22.157 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:22.193 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:09:22.193 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:09:22.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:22.193 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:22.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:09:22.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:22.707 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:09:22.723 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:09:22.723 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:22.724 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:23.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:09:23.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:23.215 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:09:23.231 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:23.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.351+0000 7fdc03e5a700 1 Processor -- start 2023-10-08T16:09:23.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.351+0000 7fdc03e5a700 1 -- start start 2023-10-08T16:09:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdbfc155c10 0x7fdbfc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc0ac230 0x7fdbfc0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdbfc1568a0 0x7fdbfc156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0ad930 con 0x7fdbfc155c10 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc0acd20 con 0x7fdbfc1568a0 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc03e5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0ad000 con 0x7fdbfc0ac230 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc01bf6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdbfc155c10 0x7fdbfc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc01bf6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdbfc155c10 0x7fdbfc155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48874/0 (socket says 172.21.15.170:48874) 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3547783027 learned_addr learned my addr 172.21.15.170:0/3547783027 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc023f7700 1 --2- 172.21.15.170:0/3547783027 >> v2:172.21.15.191:3300/0 conn(0x7fdbfc1568a0 0x7fdbfc156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.352+0000 7fdc013f5700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc0ac230 0x7fdbfc0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3547783027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc0ac230 msgr2=0x7fdbfc0ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:23.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc0ac230 0x7fdbfc0ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3547783027 >> v2:172.21.15.191:3300/0 conn(0x7fdbfc1568a0 msgr2=0x7fdbfc156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3547783027 >> v2:172.21.15.191:3300/0 conn(0x7fdbfc1568a0 0x7fdbfc156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3547783027 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0ad2e0 con 0x7fdbfc155c10 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc155c10 0x7fdbfc155fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0c8c60 tx=0x7fdbf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6889edb2cf2844 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc00bf4700 1 -- 172.21.15.170:0/3547783027 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800b640 con 0x7fdbfc155c10 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc00bf4700 1 -- 172.21.15.170:0/3547783027 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800b7a0 con 0x7fdbfc155c10 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc00bf4700 1 -- 172.21.15.170:0/3547783027 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800bb60 con 0x7fdbfc155c10 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3547783027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc155c10 msgr2=0x7fdbfc155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.353+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc155c10 0x7fdbfc155fe0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0c8c60 tx=0x7fdbf8009250 comp rx=0 tx=0).stop 2023-10-08T16:09:23.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3547783027 shutdown_connections 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3547783027 >> v2:172.21.15.191:3300/0 conn(0x7fdbfc1568a0 0x7fdbfc156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc155c10 0x7fdbfc155fe0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3547783027 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc0ac230 0x7fdbfc0ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3547783027 >> 172.21.15.170:0/3547783027 conn(0x7fdbfc06a320 msgr2=0x7fdbfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3547783027 shutdown_connections 2023-10-08T16:09:23.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.354+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3547783027 wait complete. 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.355+0000 7fdc03e5a700 1 Processor -- start 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.355+0000 7fdc03e5a700 1 -- start start 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbfc155c10 0x7fdbfc14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc1568a0 0x7fdbfc142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc14d920 con 0x7fdbfc0ac230 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc140450 con 0x7fdbfc155c10 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc14d650 con 0x7fdbfc1568a0 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48888/0 (socket says 172.21.15.170:48888) 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3518334227 learned_addr learned my addr 172.21.15.170:0/3518334227 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc023f7700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc1568a0 0x7fdbfc142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc013f5700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbfc155c10 0x7fdbfc14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc1568a0 msgr2=0x7fdbfc142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc1568a0 0x7fdbfc142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbfc155c10 msgr2=0x7fdbfc14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbfc155c10 0x7fdbfc14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 -- 172.21.15.170:0/3518334227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80094f0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc01bf6700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800b910 con 0x7fdbfc0ac230 2023-10-08T16:09:23.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800ba70 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc13fb20 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.356+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8011650 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.357+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc1400e0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.357+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc04c3c0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.358+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fdbf80119c0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.358+0000 7fdbf1ffb700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 0x7fdbe0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.358+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fdbf8085fc0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.358+0000 7fdc013f5700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 0x7fdbe0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.359+0000 7fdc013f5700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 0x7fdbe0069620 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc1489d0 tx=0x7fdbec0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.363+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbf80b75a0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.626+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbfc0ad5c0 con 0x7fdbfc0ac230 2023-10-08T16:09:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.626+0000 7fdbf1ffb700 1 -- 172.21.15.170:0/3518334227 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fdbf8052f30 con 0x7fdbfc0ac230 2023-10-08T16:09:23.627 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:23.627 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 msgr2=0x7fdbe0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 0x7fdbe0069620 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc1489d0 tx=0x7fdbec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 msgr2=0x7fdbfc14cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8017000 comp rx=0 tx=0).stop 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 shutdown_connections 2023-10-08T16:09:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdbfc1568a0 0x7fdbfc142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdbe0067170 0x7fdbe0069620 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdbfc0ac230 0x7fdbfc14cbd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 --2- 172.21.15.170:0/3518334227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdbfc155c10 0x7fdbfc14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 >> 172.21.15.170:0/3518334227 conn(0x7fdbfc06a320 msgr2=0x7fdbfc0ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 shutdown_connections 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.629+0000 7fdc03e5a700 1 -- 172.21.15.170:0/3518334227 wait complete. 2023-10-08T16:09:23.630 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 39 2023-10-08T16:09:23.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:23.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.759+0000 7f75f1a6b700 1 Processor -- start 2023-10-08T16:09:23.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.759+0000 7f75f1a6b700 1 -- start start 2023-10-08T16:09:23.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75f1a6b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75ec159c60 0x7f75ec15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75f1a6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75f1a6b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec15a160 con 0x7f75ec159890 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75f1a6b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec15df50 con 0x7f75ec159c60 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75f1a6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0d13a0 con 0x7f75ec15db10 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75eaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75ec159c60 0x7f75ec15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75eaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75ec159c60 0x7f75ec15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48900/0 (socket says 172.21.15.170:48900) 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75eaffd700 1 -- 172.21.15.170:0/2784060355 learned_addr learned my addr 172.21.15.170:0/2784060355 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.760+0000 7f75ebfff700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75eaffd700 1 -- 172.21.15.170:0/2784060355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 msgr2=0x7f75ec069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75ea7fc700 1 -- 172.21.15.170:0/2784060355 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728521472 0 0) 0x7f75ec15a160 con 0x7f75ec159890 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75eaffd700 1 -- 172.21.15.170:0/2784060355 >> v1:172.21.15.191:6789/0 conn(0x7f75ec159890 legacy=0x7f75ec15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75eaffd700 1 -- 172.21.15.170:0/2784060355 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec15e140 con 0x7f75ec159c60 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159c60 0x7f75ec15d5d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f75e00099b0 tx=0x7f75e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7563143421fc33af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75ea7fc700 1 -- 172.21.15.170:0/2784060355 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e0009ea0 con 0x7f75ec159c60 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75ea7fc700 1 -- 172.21.15.170:0/2784060355 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e000b860 con 0x7f75ec159c60 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.761+0000 7f75ea7fc700 1 -- 172.21.15.170:0/2784060355 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e000bc00 con 0x7f75ec159c60 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2784060355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159c60 msgr2=0x7f75ec15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159c60 0x7f75ec15d5d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f75e00099b0 tx=0x7f75e0009250 comp rx=0 tx=0).stop 2023-10-08T16:09:23.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2784060355 shutdown_connections 2023-10-08T16:09:23.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2784060355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159c60 0x7f75ec15d5d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2784060355 >> 172.21.15.170:0/2784060355 conn(0x7f75ec068e20 msgr2=0x7f75ec0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:23.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2784060355 shutdown_connections 2023-10-08T16:09:23.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.762+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2784060355 wait complete. 2023-10-08T16:09:23.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.763+0000 7f75f1a6b700 1 Processor -- start 2023-10-08T16:09:23.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.763+0000 7f75f1a6b700 1 -- start start 2023-10-08T16:09:23.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 0x7f75ec0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1537e0 con 0x7f75ec159890 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec154080 con 0x7f75ec159c60 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0cfd90 con 0x7f75ec15db10 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48910/0 (socket says 172.21.15.170:48910) 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 -- 172.21.15.170:0/2707883114 learned_addr learned my addr 172.21.15.170:0/2707883114 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 0x7f75ec0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75ebfff700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 -- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 msgr2=0x7f75ec154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 -- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 msgr2=0x7f75ec0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 0x7f75ec0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 -- 172.21.15.170:0/2707883114 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00094f0 con 0x7f75ec159890 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 0x7f75ec0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75eb7fe700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7e0 tx=0x7f75dc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0105c0 con 0x7f75ec159890 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc010740 con 0x7f75ec159890 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec153a30 con 0x7f75ec159890 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc010b00 con 0x7f75ec159890 2023-10-08T16:09:23.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.764+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec0b27d0 con 0x7f75ec159890 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.765+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec04c3c0 con 0x7f75ec159890 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.766+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f75dc01b3b0 con 0x7f75ec159890 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.766+0000 7f75d37fe700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 0x7f75d8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.766+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 0x7f75d8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.766+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f75dc083ea0 con 0x7f75ec159890 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.767+0000 7f75eaffd700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 0x7f75d8069620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f75e00099b0 tx=0x7f75e0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:23.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.771+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75dc050e10 con 0x7f75ec159890 2023-10-08T16:09:23.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.980+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75ec15ae60 con 0x7f75ec159890 2023-10-08T16:09:23.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.981+0000 7f75d37fe700 1 -- 172.21.15.170:0/2707883114 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15594 (secure 0 0 0) 0x7f75dc017070 con 0x7f75ec159890 2023-10-08T16:09:23.981 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:23.981 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":65,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:25.587130+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":5}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:23.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 msgr2=0x7f75d8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 0x7f75d8069620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f75e00099b0 tx=0x7f75e0017000 comp rx=0 tx=0).stop 2023-10-08T16:09:23.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 msgr2=0x7f75ec0c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00c7e0 tx=0x7f75dc007df0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 shutdown_connections 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75ec15db10 0x7f75ec154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75d8067170 0x7f75d8069620 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75ec159890 0x7f75ec0c78a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 --2- 172.21.15.170:0/2707883114 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75ec159c60 0x7f75ec0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 >> 172.21.15.170:0/2707883114 conn(0x7f75ec068e20 msgr2=0x7f75ec152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 shutdown_connections 2023-10-08T16:09:23.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:23.983+0000 7f75f1a6b700 1 -- 172.21.15.170:0/2707883114 wait complete. 2023-10-08T16:09:23.996 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:06:15.257380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:06:15.899239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:09:23.996 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:09:24.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.120+0000 7f5ca1779700 1 Processor -- start 2023-10-08T16:09:24.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.120+0000 7f5ca1779700 1 -- start start 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5ca1779700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5ca1779700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0cfb00 con 0x7f5c9c0b16d0 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5ca1779700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c0b5c40 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5ca1779700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0b5f20 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5c9b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5c9c0b16d0 0x7f5c9c0b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39574/0 (socket says 172.21.15.170:39574) 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5c9b7fe700 1 -- 172.21.15.170:0/3623736549 learned_addr learned my addr 172.21.15.170:0/3623736549 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.121+0000 7f5ca0f78700 1 --2- 172.21.15.170:0/3623736549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933479570 0 0) 0x7f5c9c0cfb00 con 0x7f5c9c0b16d0 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c84003570 con 0x7f5c9c0b16d0 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134594908 0 0) 0x7f5c9c0b5c40 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c0cfb00 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5ca0f78700 1 -- 172.21.15.170:0/3623736549 >> v1:172.21.15.191:6789/0 conn(0x7f5c9c0b4e70 legacy=0x7f5c9c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5ca0f78700 1 -- 172.21.15.170:0/3623736549 >> v1:172.21.15.170:6789/0 conn(0x7f5c9c0b16d0 legacy=0x7f5c9c0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5ca0f78700 1 -- 172.21.15.170:0/3623736549 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0b6200 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5ca0f78700 1 --2- 172.21.15.170:0/3623736549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0b3ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009e90 tx=0x7f5c94009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80ade5bc2f02491 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400e070 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400bb80 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.122+0000 7f5c9affd700 1 -- 172.21.15.170:0/3623736549 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94012430 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 -- 172.21.15.170:0/3623736549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 msgr2=0x7f5c9c0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 --2- 172.21.15.170:0/3623736549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0b3ac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009e90 tx=0x7f5c94009780 comp rx=0 tx=0).stop 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 -- 172.21.15.170:0/3623736549 shutdown_connections 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 --2- 172.21.15.170:0/3623736549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0b3ac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 -- 172.21.15.170:0/3623736549 >> 172.21.15.170:0/3623736549 conn(0x7f5c9c0d10c0 msgr2=0x7f5c9c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:24.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 -- 172.21.15.170:0/3623736549 shutdown_connections 2023-10-08T16:09:24.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.123+0000 7f5ca1779700 1 -- 172.21.15.170:0/3623736549 wait complete. 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.124+0000 7f5ca1779700 1 Processor -- start 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.124+0000 7f5ca1779700 1 -- start start 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 0x7f5c9c0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c6820 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c7300 con 0x7f5c9c0b16d0 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca1779700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c0c6a90 con 0x7f5c9c0b1b40 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5c9bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 0x7f5c9c0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5c9b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48914/0 (socket says 172.21.15.170:48914) 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5c9b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:42064/0 (socket says 172.21.15.170:42064) 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5c9bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 0x7f5c9c0c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:57480/0 (socket says 172.21.15.170:57480) 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 -- 172.21.15.170:0/1314732956 learned_addr learned my addr 172.21.15.170:0/1314732956 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 -- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 msgr2=0x7f5c9c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 -- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 msgr2=0x7f5c9c0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 0x7f5c9c0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 -- 172.21.15.170:0/1314732956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c94009a20 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.125+0000 7f5ca0f78700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400b610 tx=0x7f5c94019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.126+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94007000 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.126+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c940263f0 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.126+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c0c6d70 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.126+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940267d0 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.126+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c0c7880 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.127+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c088990 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.127+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5c94026c00 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.127+0000 7f5c83fff700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 0x7f5c88069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.127+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f5c94086930 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.128+0000 7f5c9bfff700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 0x7f5c88069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.128+0000 7f5c9bfff700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 0x7f5c88069620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007fd0 tx=0x7f5c8c007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.132+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c94053790 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.326+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c9c04bb80 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.326+0000 7f5c83fff700 1 -- 172.21.15.170:0/1314732956 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c94056db0 con 0x7f5c9c0b4e70 2023-10-08T16:09:24.327 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 msgr2=0x7f5c88069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 0x7f5c88069620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c007fd0 tx=0x7f5c8c007d10 comp rx=0 tx=0).stop 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 msgr2=0x7f5c9c0c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400b610 tx=0x7f5c94019000 comp rx=0 tx=0).stop 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 shutdown_connections 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c88067170 0x7f5c88069620 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c9c0b4e70 0x7f5c9c0c62e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c9c0b16d0 0x7f5c9c0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 --2- 172.21.15.170:0/1314732956 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c9c0b1b40 0x7f5c9c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 >> 172.21.15.170:0/1314732956 conn(0x7f5c9c0d10c0 msgr2=0x7f5c9c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:24.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 shutdown_connections 2023-10-08T16:09:24.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.329+0000 7f5ca1779700 1 -- 172.21.15.170:0/1314732956 wait complete. 2023-10-08T16:09:24.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:09:24.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.464+0000 7f6bae79b700 1 Processor -- start 2023-10-08T16:09:24.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.464+0000 7f6bae79b700 1 -- start start 2023-10-08T16:09:24.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bae79b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6ba806bfa0 0x7f6ba8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bae79b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba8068d30 con 0x7f6ba80688d0 2023-10-08T16:09:24.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bae79b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba80ca920 con 0x7f6ba8159a70 2023-10-08T16:09:24.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bae79b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba806af00 con 0x7f6ba806bfa0 2023-10-08T16:09:24.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bacf98700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6ba806bfa0 0x7f6ba8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6badf9a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6ba80688d0 0x7f6ba80caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39576/0 (socket says 172.21.15.170:39576) 2023-10-08T16:09:24.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6bacf98700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6ba806bfa0 0x7f6ba8068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48930/0 (socket says 172.21.15.170:48930) 2023-10-08T16:09:24.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.465+0000 7f6badf9a700 1 -- 172.21.15.170:0/3141871820 learned_addr learned my addr 172.21.15.170:0/3141871820 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bacf98700 1 -- 172.21.15.170:0/3141871820 >> v1:172.21.15.191:6789/0 conn(0x7f6ba8159a70 legacy=0x7f6ba80c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bacf98700 1 -- 172.21.15.170:0/3141871820 >> v1:172.21.15.170:6789/0 conn(0x7f6ba80688d0 legacy=0x7f6ba80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bacf98700 1 -- 172.21.15.170:0/3141871820 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba806b1e0 con 0x7f6ba806bfa0 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6b9ffff700 1 -- 172.21.15.170:0/3141871820 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963513496 0 0) 0x7f6ba0001490 con 0x7f6ba80688d0 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bacf98700 1 --2- 172.21.15.170:0/3141871820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba806bfa0 0x7f6ba8068390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b98009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a692913d26d33a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6b9ffff700 1 -- 172.21.15.170:0/3141871820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98009e70 con 0x7f6ba806bfa0 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6b9ffff700 1 -- 172.21.15.170:0/3141871820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b98005c10 con 0x7f6ba806bfa0 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6b9ffff700 1 -- 172.21.15.170:0/3141871820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800b650 con 0x7f6ba806bfa0 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bae79b700 1 -- 172.21.15.170:0/3141871820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba806bfa0 msgr2=0x7f6ba8068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3141871820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba806bfa0 0x7f6ba8068390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6b980098d0 tx=0x7f6b98009120 comp rx=0 tx=0).stop 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bae79b700 1 -- 172.21.15.170:0/3141871820 shutdown_connections 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.466+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3141871820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba806bfa0 0x7f6ba8068390 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.467+0000 7f6bae79b700 1 -- 172.21.15.170:0/3141871820 >> 172.21.15.170:0/3141871820 conn(0x7f6ba80cddc0 msgr2=0x7f6ba80c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:24.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.467+0000 7f6bae79b700 1 -- 172.21.15.170:0/3141871820 shutdown_connections 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.467+0000 7f6bae79b700 1 -- 172.21.15.170:0/3141871820 wait complete. 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.468+0000 7f6bae79b700 1 Processor -- start 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.468+0000 7f6bae79b700 1 -- start start 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ba806bfa0 0x7f6ba814f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 0x7f6ba80c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c6960 con 0x7f6ba80688d0 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c56f0 con 0x7f6ba806bfa0 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bae79b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba80c7330 con 0x7f6ba8159a70 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48936/0 (socket says 172.21.15.170:48936) 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 -- 172.21.15.170:0/3248102540 learned_addr learned my addr 172.21.15.170:0/3248102540 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bacf98700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ba806bfa0 0x7f6ba814f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6badf9a700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 0x7f6ba80c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 -- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 msgr2=0x7f6ba80c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 0x7f6ba80c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 -- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ba806bfa0 msgr2=0x7f6ba814f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ba806bfa0 0x7f6ba814f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 -- 172.21.15.170:0/3248102540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980093c0 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6badf9a700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 0x7f6ba80c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.469+0000 7f6bad799700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c5c0 tx=0x7f6ba400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.470+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011590 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.470+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4011710 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.470+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba80c5130 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.470+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba80c65f0 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.470+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4011ad0 con 0x7f6ba80688d0 2023-10-08T16:09:24.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.471+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b880052b0 con 0x7f6ba80688d0 2023-10-08T16:09:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.471+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f6ba401b3b0 con 0x7f6ba80688d0 2023-10-08T16:09:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.471+0000 7f6b9d7fa700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 0x7f6b8c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.471+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f6ba4083d80 con 0x7f6ba80688d0 2023-10-08T16:09:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.471+0000 7f6bacf98700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 0x7f6b8c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.472+0000 7f6bacf98700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 0x7f6b8c069620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009160 tx=0x7f6b98016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.476+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ba4050cf0 con 0x7f6ba80688d0 2023-10-08T16:09:24.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.685+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b88005ae0 con 0x7f6ba80688d0 2023-10-08T16:09:24.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.685+0000 7f6b9d7fa700 1 -- 172.21.15.170:0/3248102540 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6ba4017020 con 0x7f6ba80688d0 2023-10-08T16:09:24.686 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:09:24.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 msgr2=0x7f6b8c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 0x7f6b8c069620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009160 tx=0x7f6b98016000 comp rx=0 tx=0).stop 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 msgr2=0x7f6ba80c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400c5c0 tx=0x7f6ba400c8d0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 shutdown_connections 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ba8159a70 0x7f6ba80c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b8c067170 0x7f6b8c069620 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ba80688d0 0x7f6ba80c6df0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 --2- 172.21.15.170:0/3248102540 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ba806bfa0 0x7f6ba814f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 >> 172.21.15.170:0/3248102540 conn(0x7f6ba80cddc0 msgr2=0x7f6ba8157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:24.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 shutdown_connections 2023-10-08T16:09:24.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.687+0000 7f6bae79b700 1 -- 172.21.15.170:0/3248102540 wait complete. 2023-10-08T16:09:24.700 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:09:24.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.822+0000 7fc418e57700 1 Processor -- start 2023-10-08T16:09:24.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.822+0000 7fc418e57700 1 -- start start 2023-10-08T16:09:24.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc41406bcd0 0x7fc4140caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406c0a0 0x7fc41415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc41415df50 0x7fc41415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41415d7e0 con 0x7fc41415df50 2023-10-08T16:09:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41415dac0 con 0x7fc41406bcd0 2023-10-08T16:09:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc418e57700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41415aa30 con 0x7fc41406c0a0 2023-10-08T16:09:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc412d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc41415df50 0x7fc41415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc412d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc41415df50 0x7fc41415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48950/0 (socket says 172.21.15.170:48950) 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc412d9d700 1 -- 172.21.15.170:0/1854833680 learned_addr learned my addr 172.21.15.170:0/1854833680 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc411d9b700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406c0a0 0x7fc41415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.823+0000 7fc41259c700 1 --2- 172.21.15.170:0/1854833680 >> v2:172.21.15.191:3300/0 conn(0x7fc41406bcd0 0x7fc4140caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 -- 172.21.15.170:0/1854833680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406c0a0 msgr2=0x7fc41415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406c0a0 0x7fc41415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 -- 172.21.15.170:0/1854833680 >> v2:172.21.15.191:3300/0 conn(0x7fc41406bcd0 msgr2=0x7fc4140caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 --2- 172.21.15.170:0/1854833680 >> v2:172.21.15.191:3300/0 conn(0x7fc41406bcd0 0x7fc4140caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 -- 172.21.15.170:0/1854833680 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415ad10 con 0x7fc41415df50 2023-10-08T16:09:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc412d9d700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc41415e3c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc4140691a0 tx=0x7fc40c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9726a9da7a1a4f3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc41159a700 1 -- 172.21.15.170:0/1854833680 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c00e070 con 0x7fc41415df50 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc41159a700 1 -- 172.21.15.170:0/1854833680 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c00b970 con 0x7fc41415df50 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc41159a700 1 -- 172.21.15.170:0/1854833680 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0092e0 con 0x7fc41415df50 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 -- 172.21.15.170:0/1854833680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 msgr2=0x7fc41415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc41415e3c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc4140691a0 tx=0x7fc40c009840 comp rx=0 tx=0).stop 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 -- 172.21.15.170:0/1854833680 shutdown_connections 2023-10-08T16:09:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc41415e3c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 --2- 172.21.15.170:0/1854833680 >> v2:172.21.15.191:3300/0 conn(0x7fc41406bcd0 0x7fc4140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 --2- 172.21.15.170:0/1854833680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406c0a0 0x7fc41415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.824+0000 7fc418e57700 1 -- 172.21.15.170:0/1854833680 >> 172.21.15.170:0/1854833680 conn(0x7fc4140ce600 msgr2=0x7fc4140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.825+0000 7fc418e57700 1 -- 172.21.15.170:0/1854833680 shutdown_connections 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.825+0000 7fc418e57700 1 -- 172.21.15.170:0/1854833680 wait complete. 2023-10-08T16:09:24.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 Processor -- start 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 -- start start 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406bcd0 0x7fc4140b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41406c0a0 0x7fc414156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140b4120 con 0x7fc41415df50 2023-10-08T16:09:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c6b90 con 0x7fc41406c0a0 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc418e57700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c6e70 con 0x7fc41406bcd0 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc412d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48958/0 (socket says 172.21.15.170:48958) 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc411d9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41406c0a0 0x7fc414156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc412d9d700 1 -- 172.21.15.170:0/2277447923 learned_addr learned my addr 172.21.15.170:0/2277447923 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.826+0000 7fc41259c700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406bcd0 0x7fc4140b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 -- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406bcd0 msgr2=0x7fc4140b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406bcd0 0x7fc4140b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 -- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41406c0a0 msgr2=0x7fc414156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41406c0a0 0x7fc414156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 -- 172.21.15.170:0/2277447923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c009ae0 con 0x7fc41415df50 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc412d9d700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b610 tx=0x7fc40c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0092e0 con 0x7fc41415df50 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c007b00 con 0x7fc41415df50 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4140c7150 con 0x7fc41415df50 2023-10-08T16:09:24.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c01d420 con 0x7fc41415df50 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.827+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4140c7710 con 0x7fc41415df50 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.828+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41404c3c0 con 0x7fc41415df50 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.828+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc40c003050 con 0x7fc41415df50 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.829+0000 7fc4027fc700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 0x7fc3f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.829+0000 7fc41259c700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 0x7fc3f4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.829+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fc40c0864f0 con 0x7fc41415df50 2023-10-08T16:09:24.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.829+0000 7fc41259c700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 0x7fc3f4069620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc404009d10 tx=0x7fc4040091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:24.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:24.833+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc40c053350 con 0x7fc41415df50 2023-10-08T16:09:25.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.050+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc414071280 con 0x7fc41415df50 2023-10-08T16:09:25.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.051+0000 7fc4027fc700 1 -- 172.21.15.170:0/2277447923 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc40c056970 con 0x7fc41415df50 2023-10-08T16:09:25.051 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 msgr2=0x7fc3f4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 0x7fc3f4069620 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc404009d10 tx=0x7fc4040091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 msgr2=0x7fc414169ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b610 tx=0x7fc40c018000 comp rx=0 tx=0).stop 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 shutdown_connections 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3f4067170 0x7fc3f4069620 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc41415df50 0x7fc414169ce0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc41406bcd0 0x7fc4140b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 --2- 172.21.15.170:0/2277447923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc41406c0a0 0x7fc414156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 >> 172.21.15.170:0/2277447923 conn(0x7fc4140ce600 msgr2=0x7fc414157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 shutdown_connections 2023-10-08T16:09:25.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.053+0000 7fc418e57700 1 -- 172.21.15.170:0/2277447923 wait complete. 2023-10-08T16:09:25.066 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:25.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.186+0000 7f808a637700 1 Processor -- start 2023-10-08T16:09:25.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.186+0000 7f808a637700 1 -- start start 2023-10-08T16:09:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f808a637700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80840ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f808a637700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840ca860 con 0x7f8084156310 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f808a637700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cab40 con 0x7f80840cb900 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f808a637700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cae20 con 0x7f80841566e0 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f8089635700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8084156310 0x7f80840c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58088/0 (socket says 172.21.15.170:58088) 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f8089635700 1 -- 172.21.15.170:0/3623711539 learned_addr learned my addr 172.21.15.170:0/3623711539 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.187+0000 7f8088e34700 1 --2- 172.21.15.170:0/3623711539 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80840ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886990272 0 0) 0x7f80840ca860 con 0x7f8084156310 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8070003570 con 0x7f8084156310 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f8088e34700 1 -- 172.21.15.170:0/3623711539 >> v1:172.21.15.191:6789/0 conn(0x7f80840cb900 legacy=0x7f80840c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795679653 0 0) 0x7f80840cab40 con 0x7f80840cb900 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f8088e34700 1 -- 172.21.15.170:0/3623711539 >> v1:172.21.15.170:6789/0 conn(0x7f8084156310 legacy=0x7f80840c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f8088e34700 1 -- 172.21.15.170:0/3623711539 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cb100 con 0x7f80841566e0 2023-10-08T16:09:25.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f8088e34700 1 --2- 172.21.15.170:0/3623711539 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80840ae000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8074009910 tx=0x7f80740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7748ad10abcd2f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400b680 con 0x7f80841566e0 2023-10-08T16:09:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807400b7e0 con 0x7f80841566e0 2023-10-08T16:09:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.188+0000 7f807bfff700 1 -- 172.21.15.170:0/3623711539 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400bba0 con 0x7f80841566e0 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 -- 172.21.15.170:0/3623711539 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 msgr2=0x7f80840ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 --2- 172.21.15.170:0/3623711539 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80840ae000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8074009910 tx=0x7f80740091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 -- 172.21.15.170:0/3623711539 shutdown_connections 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 --2- 172.21.15.170:0/3623711539 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80840ae000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 -- 172.21.15.170:0/3623711539 >> 172.21.15.170:0/3623711539 conn(0x7f80840cc1f0 msgr2=0x7f80840cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 -- 172.21.15.170:0/3623711539 shutdown_connections 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.189+0000 7f808a637700 1 -- 172.21.15.170:0/3623711539 wait complete. 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.190+0000 7f808a637700 1 Processor -- start 2023-10-08T16:09:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.190+0000 7f808a637700 1 -- start start 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8084156310 0x7f8084167db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80841682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084153d70 con 0x7f80840cb900 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8084154050 con 0x7f8084156310 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f808a637700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084154330 con 0x7f80841566e0 2023-10-08T16:09:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51900/0 (socket says 172.21.15.170:51900) 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 -- 172.21.15.170:0/3556868942 learned_addr learned my addr 172.21.15.170:0/3556868942 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8088e34700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8084156310 0x7f8084167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089e36700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80841682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 -- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 msgr2=0x7f80841682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80841682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 -- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8084156310 msgr2=0x7f8084167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8084156310 0x7f8084167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.191+0000 7f8089635700 1 -- 172.21.15.170:0/3556868942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8074009420 con 0x7f80840cb900 2023-10-08T16:09:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f8089635700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f80800011e0 tx=0x7f8080007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080005b30 con 0x7f80840cb900 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8080005c90 con 0x7f80840cb900 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8080007300 con 0x7f80840cb900 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8084154610 con 0x7f80840cb900 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.192+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8084155330 con 0x7f80840cb900 2023-10-08T16:09:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.193+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404c3c0 con 0x7f80840cb900 2023-10-08T16:09:25.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.193+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8080007670 con 0x7f80840cb900 2023-10-08T16:09:25.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.194+0000 7f80797fa700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 0x7f8068069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.194+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f8080083de0 con 0x7f80840cb900 2023-10-08T16:09:25.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.194+0000 7f8088e34700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 0x7f8068069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.194+0000 7f8088e34700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 0x7f8068069620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8074009910 tx=0x7f8074017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.198+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8080050c80 con 0x7f80840cb900 2023-10-08T16:09:25.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.471+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f808406e900 con 0x7f80840cb900 2023-10-08T16:09:25.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.471+0000 7f80797fa700 1 -- 172.21.15.170:0/3556868942 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f8080017020 con 0x7f80840cb900 2023-10-08T16:09:25.472 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:25.472 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 msgr2=0x7f8068069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 0x7f8068069620 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8074009910 tx=0x7f8074017000 comp rx=0 tx=0).stop 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 msgr2=0x7f8084154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f80800011e0 tx=0x7f8080007df0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 shutdown_connections 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80841566e0 0x7f80841682f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8068067170 0x7f8068069620 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80840cb900 0x7f8084154df0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 --2- 172.21.15.170:0/3556868942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8084156310 0x7f8084167db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 >> 172.21.15.170:0/3556868942 conn(0x7f80840cc1f0 msgr2=0x7f80840c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 shutdown_connections 2023-10-08T16:09:25.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.473+0000 7f808a637700 1 -- 172.21.15.170:0/3556868942 wait complete. 2023-10-08T16:09:25.475 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 39 2023-10-08T16:09:25.486 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:25.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.608+0000 7f0d3100f700 1 Processor -- start 2023-10-08T16:09:25.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.608+0000 7f0d3100f700 1 -- start start 2023-10-08T16:09:25.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3100f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3100f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0d2c15d230 0x7f0d2c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3100f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2c15dbd0 con 0x7f0d2c159420 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3100f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b11d0 con 0x7f0d2c15d230 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3100f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c0b14b0 con 0x7f0d2c1597f0 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d2bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0d2c159420 0x7f0d2c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58096/0 (socket says 172.21.15.170:58096) 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d2bfff700 1 -- 172.21.15.170:0/2646796074 learned_addr learned my addr 172.21.15.170:0/2646796074 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d3080e700 1 --2- 172.21.15.170:0/2646796074 >> v2:172.21.15.191:3300/0 conn(0x7f0d2c15d230 0x7f0d2c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.609+0000 7f0d2b7fe700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2affd700 1 -- 172.21.15.170:0/2646796074 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819625353 0 0) 0x7f0d1c001110 con 0x7f0d2c159420 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2affd700 1 -- 172.21.15.170:0/2646796074 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d14003570 con 0x7f0d2c159420 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d3080e700 1 -- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 msgr2=0x7f0d2c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:25.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d3080e700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d3080e700 1 -- 172.21.15.170:0/2646796074 >> v1:172.21.15.170:6789/0 conn(0x7f0d2c159420 legacy=0x7f0d2c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d3080e700 1 -- 172.21.15.170:0/2646796074 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d2c0b1790 con 0x7f0d2c15d230 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2b7fe700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d3080e700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c15d690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009e60 tx=0x7f0d24009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90802111d5f824f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2affd700 1 -- 172.21.15.170:0/2646796074 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400e070 con 0x7f0d2c15d230 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2affd700 1 -- 172.21.15.170:0/2646796074 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2400bab0 con 0x7f0d2c15d230 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.610+0000 7f0d2affd700 1 -- 172.21.15.170:0/2646796074 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2400be30 con 0x7f0d2c15d230 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 -- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 msgr2=0x7f0d2c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c15d690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009e60 tx=0x7f0d24009780 comp rx=0 tx=0).stop 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 -- 172.21.15.170:0/2646796074 shutdown_connections 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c15d690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 --2- 172.21.15.170:0/2646796074 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 -- 172.21.15.170:0/2646796074 >> 172.21.15.170:0/2646796074 conn(0x7f0d2c0d0660 msgr2=0x7f0d2c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 -- 172.21.15.170:0/2646796074 shutdown_connections 2023-10-08T16:09:25.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.611+0000 7f0d3100f700 1 -- 172.21.15.170:0/2646796074 wait complete. 2023-10-08T16:09:25.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.612+0000 7f0d3100f700 1 Processor -- start 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.612+0000 7f0d3100f700 1 -- start start 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c15b040 con 0x7f0d2c159420 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c15a710 con 0x7f0d2c15d230 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3100f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2c15a9f0 con 0x7f0d2c1597f0 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51910/0 (socket says 172.21.15.170:51910) 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 -- 172.21.15.170:0/1601350814 learned_addr learned my addr 172.21.15.170:0/1601350814 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3080e700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2b7fe700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 -- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 msgr2=0x7f0d2c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c1566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 -- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 msgr2=0x7f0d2c0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 -- 172.21.15.170:0/1601350814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20009420 con 0x7f0d2c159420 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d3080e700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.613+0000 7f0d2bfff700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c7c0 tx=0x7f0d1c007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.614+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c005b30 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.614+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c005c90 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.614+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d24009a20 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.614+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c007300 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.614+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d2c15ca20 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.615+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d2c04c3c0 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.615+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0d1c0076b0 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.616+0000 7f0d13fff700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 0x7f0d18069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.616+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f0d1c083c30 con 0x7f0d2c159420 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.616+0000 7f0d2b7fe700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 0x7f0d18069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.616+0000 7f0d2b7fe700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 0x7f0d18069620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c16ce50 tx=0x7f0d200091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.620+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d1c050b00 con 0x7f0d2c159420 2023-10-08T16:09:25.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.818+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d2c15dbd0 con 0x7f0d2c159420 2023-10-08T16:09:25.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.819+0000 7f0d13fff700 1 -- 172.21.15.170:0/1601350814 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15594 (secure 0 0 0) 0x7f0d2c15dbd0 con 0x7f0d2c159420 2023-10-08T16:09:25.820 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:25.820 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":65,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:25.587130+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":5}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 msgr2=0x7f0d18069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 0x7f0d18069620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0d2c16ce50 tx=0x7f0d200091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 msgr2=0x7f0d2c156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c00c7c0 tx=0x7f0d1c007c30 comp rx=0 tx=0).stop 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 shutdown_connections 2023-10-08T16:09:25.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d2c15d230 0x7f0d2c0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d18067170 0x7f0d18069620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d2c159420 0x7f0d2c156ef0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 --2- 172.21.15.170:0/1601350814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d2c1597f0 0x7f0d2c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.821+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 >> 172.21.15.170:0/1601350814 conn(0x7f0d2c0d0660 msgr2=0x7f0d2c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.822+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 shutdown_connections 2023-10-08T16:09:25.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.822+0000 7f0d3100f700 1 -- 172.21.15.170:0/1601350814 wait complete. 2023-10-08T16:09:25.834 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:09:25.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:25.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.953+0000 7fbe863df700 1 Processor -- start 2023-10-08T16:09:25.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.953+0000 7fbe863df700 1 -- start start 2023-10-08T16:09:25.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe863df700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbe80143050 0x7fbe80159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe863df700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe800b3f20 0x7fbe8006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe863df700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8006c390 con 0x7fbe8006b9e0 2023-10-08T16:09:25.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe863df700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe80143420 con 0x7fbe800b3f20 2023-10-08T16:09:25.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe863df700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8006a1e0 con 0x7fbe80143050 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe85bde700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe800b3f20 0x7fbe8006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe84bdc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbe8006b9e0 0x7fbe800b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58100/0 (socket says 172.21.15.170:58100) 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe84bdc700 1 -- 172.21.15.170:0/3533929695 learned_addr learned my addr 172.21.15.170:0/3533929695 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe853dd700 1 --2- 172.21.15.170:0/3533929695 >> v2:172.21.15.191:3300/0 conn(0x7fbe80143050 0x7fbe80159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe85bde700 1 -- 172.21.15.170:0/3533929695 >> v2:172.21.15.191:3300/0 conn(0x7fbe80143050 msgr2=0x7fbe80159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.954+0000 7fbe85bde700 1 --2- 172.21.15.170:0/3533929695 >> v2:172.21.15.191:3300/0 conn(0x7fbe80143050 0x7fbe80159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe85bde700 1 -- 172.21.15.170:0/3533929695 >> v1:172.21.15.170:6789/0 conn(0x7fbe8006b9e0 legacy=0x7fbe800b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe85bde700 1 -- 172.21.15.170:0/3533929695 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800695d0 con 0x7fbe800b3f20 2023-10-08T16:09:25.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe77fff700 1 -- 172.21.15.170:0/3533929695 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991075922 0 0) 0x7fbe8006c390 con 0x7fbe8006b9e0 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe85bde700 1 --2- 172.21.15.170:0/3533929695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8006be50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbe8006eec0 tx=0x7fbe78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22441c8242476a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe77fff700 1 -- 172.21.15.170:0/3533929695 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800e070 con 0x7fbe800b3f20 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe77fff700 1 -- 172.21.15.170:0/3533929695 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7800bb60 con 0x7fbe800b3f20 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe77fff700 1 -- 172.21.15.170:0/3533929695 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78012430 con 0x7fbe800b3f20 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 -- 172.21.15.170:0/3533929695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 msgr2=0x7fbe8006be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 --2- 172.21.15.170:0/3533929695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8006be50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fbe8006eec0 tx=0x7fbe78009780 comp rx=0 tx=0).stop 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 -- 172.21.15.170:0/3533929695 shutdown_connections 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 --2- 172.21.15.170:0/3533929695 >> v2:172.21.15.191:3300/0 conn(0x7fbe80143050 0x7fbe80159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 --2- 172.21.15.170:0/3533929695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8006be50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.955+0000 7fbe863df700 1 -- 172.21.15.170:0/3533929695 >> 172.21.15.170:0/3533929695 conn(0x7fbe800ce430 msgr2=0x7fbe800ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:25.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.956+0000 7fbe863df700 1 -- 172.21.15.170:0/3533929695 shutdown_connections 2023-10-08T16:09:25.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.956+0000 7fbe863df700 1 -- 172.21.15.170:0/3533929695 wait complete. 2023-10-08T16:09:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.957+0000 7fbe863df700 1 Processor -- start 2023-10-08T16:09:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.957+0000 7fbe863df700 1 -- start start 2023-10-08T16:09:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe8006b9e0 0x7fbe8015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe80143050 0x7fbe80141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015d050 con 0x7fbe800b3f20 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015c440 con 0x7fbe80143050 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe863df700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8015c720 con 0x7fbe8006b9e0 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51936/0 (socket says 172.21.15.170:51936) 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 -- 172.21.15.170:0/3303376469 learned_addr learned my addr 172.21.15.170:0/3303376469 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe853dd700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe8006b9e0 0x7fbe8015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe85bde700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe80143050 0x7fbe80141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 -- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe8006b9e0 msgr2=0x7fbe8015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe8006b9e0 0x7fbe8015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 -- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe80143050 msgr2=0x7fbe80141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe80143050 0x7fbe80141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 -- 172.21.15.170:0/3303376469 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe78009a20 con 0x7fbe800b3f20 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe84bdc700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000c5b0 tx=0x7fbe70001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70005b30 con 0x7fbe800b3f20 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.958+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70005c90 con 0x7fbe800b3f20 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.959+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8015ca00 con 0x7fbe800b3f20 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.959+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70007320 con 0x7fbe800b3f20 2023-10-08T16:09:25.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.959+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8015c130 con 0x7fbe800b3f20 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.959+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8004c3c0 con 0x7fbe800b3f20 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.960+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fbe7001b3b0 con 0x7fbe800b3f20 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.960+0000 7fbe757fa700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 0x7fbe64069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.960+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fbe70083ca0 con 0x7fbe800b3f20 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.960+0000 7fbe853dd700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 0x7fbe64069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:25.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.961+0000 7fbe853dd700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 0x7fbe64069620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009da0 tx=0x7fbe7c009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:25.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:25.965+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe70050c10 con 0x7fbe800b3f20 2023-10-08T16:09:26.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.214+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe8015d190 con 0x7fbe800b3f20 2023-10-08T16:09:26.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.214+0000 7fbe757fa700 1 -- 172.21.15.170:0/3303376469 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fbe70017090 con 0x7fbe800b3f20 2023-10-08T16:09:26.215 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:26.215 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:09:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.216+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 msgr2=0x7fbe64069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 0x7fbe64069620 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009da0 tx=0x7fbe7c009120 comp rx=0 tx=0).stop 2023-10-08T16:09:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 msgr2=0x7fbe8013c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fbe7000c5b0 tx=0x7fbe70001070 comp rx=0 tx=0).stop 2023-10-08T16:09:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 shutdown_connections 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe80143050 0x7fbe80141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe8006b9e0 0x7fbe8015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe64067170 0x7fbe64069620 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 --2- 172.21.15.170:0/3303376469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe800b3f20 0x7fbe8013c160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 >> 172.21.15.170:0/3303376469 conn(0x7fbe800ce430 msgr2=0x7fbe8006cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 shutdown_connections 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.217+0000 7fbe863df700 1 -- 172.21.15.170:0/3303376469 wait complete. 2023-10-08T16:09:26.218 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 39 2023-10-08T16:09:26.229 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:26.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.358+0000 7f62634ae700 1 Processor -- start 2023-10-08T16:09:26.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 -- start start 2023-10-08T16:09:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 0x7f625c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 0x7f625c06bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625c0b1500 con 0x7f625c0ccd10 2023-10-08T16:09:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625c0cda20 con 0x7f625c140cd0 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.359+0000 7f62634ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0cdd00 con 0x7f625c1410a0 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 0x7f625c06bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62624ac700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 0x7f625c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6262cad700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f625c0ccd10 0x7f625c0cd170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:44280/0 (socket says 172.21.15.170:44280) 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6262cad700 1 -- 172.21.15.170:0/3778497505 learned_addr learned my addr 172.21.15.170:0/3778497505 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62614aa700 1 -- 172.21.15.170:0/3778497505 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593711619 0 0) 0x7f625c0b1500 con 0x7f625c0ccd10 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62614aa700 1 -- 172.21.15.170:0/3778497505 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6248003570 con 0x7f625c0ccd10 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 -- 172.21.15.170:0/3778497505 >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 msgr2=0x7f625c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 --2- 172.21.15.170:0/3778497505 >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 0x7f625c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 -- 172.21.15.170:0/3778497505 >> v1:172.21.15.191:6789/0 conn(0x7f625c0ccd10 legacy=0x7f625c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 -- 172.21.15.170:0/3778497505 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0cdfe0 con 0x7f625c1410a0 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62624ac700 1 --2- 172.21.15.170:0/3778497505 >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 0x7f625c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f6261cab700 1 --2- 172.21.15.170:0/3778497505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 0x7f625c06bf20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f624c0098e0 tx=0x7f624c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731af94d93607737 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62614aa700 1 -- 172.21.15.170:0/3778497505 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c011630 con 0x7f625c1410a0 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62614aa700 1 -- 172.21.15.170:0/3778497505 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c011790 con 0x7f625c1410a0 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.360+0000 7f62614aa700 1 -- 172.21.15.170:0/3778497505 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c011b30 con 0x7f625c1410a0 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 -- 172.21.15.170:0/3778497505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 msgr2=0x7f625c06bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 --2- 172.21.15.170:0/3778497505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 0x7f625c06bf20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f624c0098e0 tx=0x7f624c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 -- 172.21.15.170:0/3778497505 shutdown_connections 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 --2- 172.21.15.170:0/3778497505 >> v2:172.21.15.191:3300/0 conn(0x7f625c140cd0 0x7f625c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 --2- 172.21.15.170:0/3778497505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c1410a0 0x7f625c06bf20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 -- 172.21.15.170:0/3778497505 >> 172.21.15.170:0/3778497505 conn(0x7f625c0ce430 msgr2=0x7f625c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:26.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 -- 172.21.15.170:0/3778497505 shutdown_connections 2023-10-08T16:09:26.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.361+0000 7f62634ae700 1 -- 172.21.15.170:0/3778497505 wait complete. 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.362+0000 7f62634ae700 1 Processor -- start 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.362+0000 7f62634ae700 1 -- start start 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f625c140cd0 0x7f625c139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 0x7f625c0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0c9660 con 0x7f625c1410a0 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0c9940 con 0x7f625c140cd0 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62634ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0c9c20 con 0x7f625c0ccd10 2023-10-08T16:09:26.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f6262cad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 0x7f625c0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:43922/0 (socket says 172.21.15.170:43922) 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 -- 172.21.15.170:0/215134199 learned_addr learned my addr 172.21.15.170:0/215134199 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f6261cab700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f625c140cd0 0x7f625c139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 -- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f625c140cd0 msgr2=0x7f625c139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f625c140cd0 0x7f625c139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 -- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 msgr2=0x7f625c0c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 0x7f625c0c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f62624ac700 1 -- 172.21.15.170:0/215134199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6258009a20 con 0x7f625c0ccd10 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.363+0000 7f6262cad700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 0x7f625c0c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62624ac700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f625800c170 tx=0x7f625800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625800ccb0 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625800f040 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c009420 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258014480 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.364+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c0ca1e0 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.365+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c169670 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.365+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f62580147f0 con 0x7f625c0ccd10 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.365+0000 7f62527fc700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 0x7f6240069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.366+0000 7f6261cab700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 0x7f6240069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.366+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7f6258085820 con 0x7f625c0ccd10 2023-10-08T16:09:26.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.366+0000 7f6261cab700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 0x7f6240069620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f624c0091e0 tx=0x7f624c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.370+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62580525e0 con 0x7f625c0ccd10 2023-10-08T16:09:26.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.626+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f625c04c3c0 con 0x7f625c0ccd10 2023-10-08T16:09:26.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.626+0000 7f62527fc700 1 -- 172.21.15.170:0/215134199 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f6258055c00 con 0x7f625c0ccd10 2023-10-08T16:09:26.627 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:26.627 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:06:16.968284+0000","modified":"2023-10-08T16:06:20.570046+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5667":{"gid":5667,"name":"b","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/1489757560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1489757560},{"type":"v1","addr":"172.21.15.191:6834","nonce":1489757560}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T16:09:26.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 msgr2=0x7f6240069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 0x7f6240069620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f624c0091e0 tx=0x7f624c017000 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 msgr2=0x7f625c0c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f625800c170 tx=0x7f625800b040 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 shutdown_connections 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6240067170 0x7f6240069620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f625c1410a0 0x7f625c0c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f625c0ccd10 0x7f625c0c8d60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 --2- 172.21.15.170:0/215134199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f625c140cd0 0x7f625c139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 >> 172.21.15.170:0/215134199 conn(0x7f625c0ce430 msgr2=0x7f625c069720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 shutdown_connections 2023-10-08T16:09:26.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.629+0000 7f62634ae700 1 -- 172.21.15.170:0/215134199 wait complete. 2023-10-08T16:09:26.631 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 39 2023-10-08T16:09:26.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:26.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.759+0000 7fad48f99700 1 Processor -- start 2023-10-08T16:09:26.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.759+0000 7fad48f99700 1 -- start start 2023-10-08T16:09:26.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad48f99700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fad44159890 0x7fad4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad48f99700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fad44159c60 0x7fad4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad48f99700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad4415a160 con 0x7fad4415db10 2023-10-08T16:09:26.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad48f99700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad4415df50 con 0x7fad44159890 2023-10-08T16:09:26.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad48f99700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad440d13a0 con 0x7fad44159c60 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad4259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fad44159890 0x7fad4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad4259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fad44159890 0x7fad4415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51952/0 (socket says 172.21.15.170:51952) 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.760+0000 7fad4259c700 1 -- 172.21.15.170:0/220162488 learned_addr learned my addr 172.21.15.170:0/220162488 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad41d9b700 1 --2- 172.21.15.170:0/220162488 >> v2:172.21.15.191:3300/0 conn(0x7fad44159c60 0x7fad4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4259c700 1 -- 172.21.15.170:0/220162488 >> v2:172.21.15.191:3300/0 conn(0x7fad44159c60 msgr2=0x7fad4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4259c700 1 --2- 172.21.15.170:0/220162488 >> v2:172.21.15.191:3300/0 conn(0x7fad44159c60 0x7fad4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4259c700 1 -- 172.21.15.170:0/220162488 >> v1:172.21.15.191:6789/0 conn(0x7fad4415db10 legacy=0x7fad44069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4259c700 1 -- 172.21.15.170:0/220162488 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4415e140 con 0x7fad44159890 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4259c700 1 --2- 172.21.15.170:0/220162488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159890 0x7fad4415abd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad440b67f0 tx=0x7fad34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d497f1695ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4159a700 1 -- 172.21.15.170:0/220162488 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400b820 con 0x7fad44159890 2023-10-08T16:09:26.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4159a700 1 -- 172.21.15.170:0/220162488 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3400b980 con 0x7fad44159890 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad4159a700 1 -- 172.21.15.170:0/220162488 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400bd20 con 0x7fad44159890 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad48f99700 1 -- 172.21.15.170:0/220162488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159890 msgr2=0x7fad4415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.761+0000 7fad48f99700 1 --2- 172.21.15.170:0/220162488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159890 0x7fad4415abd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fad440b67f0 tx=0x7fad34009250 comp rx=0 tx=0).stop 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 -- 172.21.15.170:0/220162488 shutdown_connections 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 --2- 172.21.15.170:0/220162488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159890 0x7fad4415abd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 --2- 172.21.15.170:0/220162488 >> v2:172.21.15.191:3300/0 conn(0x7fad44159c60 0x7fad4415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 -- 172.21.15.170:0/220162488 >> 172.21.15.170:0/220162488 conn(0x7fad44068e20 msgr2=0x7fad440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 -- 172.21.15.170:0/220162488 shutdown_connections 2023-10-08T16:09:26.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.762+0000 7fad48f99700 1 -- 172.21.15.170:0/220162488 wait complete. 2023-10-08T16:09:26.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.763+0000 7fad48f99700 1 Processor -- start 2023-10-08T16:09:26.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.763+0000 7fad48f99700 1 -- start start 2023-10-08T16:09:26.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad44159890 0x7fad44157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad4415db10 0x7fad440ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4416b0d0 con 0x7fad44159c60 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4416a600 con 0x7fad44159890 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4416a830 con 0x7fad4415db10 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51968/0 (socket says 172.21.15.170:51968) 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 -- 172.21.15.170:0/4133680152 learned_addr learned my addr 172.21.15.170:0/4133680152 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad4259c700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad44159890 0x7fad44157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad42d9d700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad4415db10 0x7fad440ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 -- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad4415db10 msgr2=0x7fad440ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad4415db10 0x7fad440ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 -- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad44159890 msgr2=0x7fad44157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad44159890 0x7fad44157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 -- 172.21.15.170:0/4133680152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340094f0 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad41d9b700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad3800d8c0 tx=0x7fad38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800de60 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3800b7e0 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800bba0 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4416ab10 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.765+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4416b430 con 0x7fad44159c60 2023-10-08T16:09:26.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.766+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4416b290 con 0x7fad44159c60 2023-10-08T16:09:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.767+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fad3801e460 con 0x7fad44159c60 2023-10-08T16:09:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.767+0000 7fad327fc700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.767+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fad380850a0 con 0x7fad44159c60 2023-10-08T16:09:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.767+0000 7fad4259c700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:26.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.768+0000 7fad4259c700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 0x7fad24069620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad440b67f0 tx=0x7fad34016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:26.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.772+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad4416b290 con 0x7fad44159c60 2023-10-08T16:09:26.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.970+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad4416b290 con 0x7fad44159c60 2023-10-08T16:09:26.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.971+0000 7fad327fc700 1 -- 172.21.15.170:0/4133680152 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+15594 (secure 0 0 0) 0x7fad4416b290 con 0x7fad44159c60 2023-10-08T16:09:26.971 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:26.971 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":65,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:06:25.587130+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:06:15.257380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T16:06:15.899239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":5}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.30","mappings":[{"from":6,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:26.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 msgr2=0x7fad24069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 0x7fad24069620 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fad440b67f0 tx=0x7fad34016000 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 msgr2=0x7fad4416a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fad3800d8c0 tx=0x7fad38008040 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 shutdown_connections 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fad4415db10 0x7fad440ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fad44159890 0x7fad44157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fad24067170 0x7fad24069620 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 --2- 172.21.15.170:0/4133680152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fad44159c60 0x7fad4416a0c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 >> 172.21.15.170:0/4133680152 conn(0x7fad44068e20 msgr2=0x7fad440b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 shutdown_connections 2023-10-08T16:09:26.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:26.973+0000 7fad48f99700 1 -- 172.21.15.170:0/4133680152 wait complete. 2023-10-08T16:09:26.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:09:27.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.105+0000 7fe357f69700 1 Processor -- start 2023-10-08T16:09:27.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.105+0000 7fe357f69700 1 -- start start 2023-10-08T16:09:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe357f69700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe350159a40 0x7fe3500c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe357f69700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe35015d7b0 0x7fe3500c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe357f69700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35015dc40 con 0x7fe350159e10 2023-10-08T16:09:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe357f69700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35015df20 con 0x7fe35015d7b0 2023-10-08T16:09:27.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe357f69700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35015e200 con 0x7fe350159a40 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe35015d7b0 0x7fe3500c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355d05700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe350159a40 0x7fe3500c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe35015d7b0 0x7fe3500c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51976/0 (socket says 172.21.15.170:51976) 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 -- 172.21.15.170:0/1334946064 learned_addr learned my addr 172.21.15.170:0/1334946064 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe356506700 1 --1- 172.21.15.170:0/1334946064 >> v1:172.21.15.191:6789/0 conn(0x7fe350159e10 0x7fe3500c82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:44290/0 (socket says 172.21.15.170:44290) 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe354d03700 1 -- 172.21.15.170:0/1334946064 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158845047 0 0) 0x7fe35015dc40 con 0x7fe350159e10 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 -- 172.21.15.170:0/1334946064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe350159a40 msgr2=0x7fe3500c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 --2- 172.21.15.170:0/1334946064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe350159a40 0x7fe3500c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.106+0000 7fe355504700 1 -- 172.21.15.170:0/1334946064 >> v1:172.21.15.191:6789/0 conn(0x7fe350159e10 legacy=0x7fe3500c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe355504700 1 -- 172.21.15.170:0/1334946064 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35015e4e0 con 0x7fe35015d7b0 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe355504700 1 --2- 172.21.15.170:0/1334946064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe35015d7b0 0x7fe3500c9a50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe3400099b0 tx=0x7fe340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8291b64a4e9123b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe354d03700 1 -- 172.21.15.170:0/1334946064 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340009ea0 con 0x7fe35015d7b0 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe354d03700 1 -- 172.21.15.170:0/1334946064 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34000b8f0 con 0x7fe35015d7b0 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe354d03700 1 -- 172.21.15.170:0/1334946064 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34000bc90 con 0x7fe35015d7b0 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 -- 172.21.15.170:0/1334946064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe35015d7b0 msgr2=0x7fe3500c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 --2- 172.21.15.170:0/1334946064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe35015d7b0 0x7fe3500c9a50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe3400099b0 tx=0x7fe340009250 comp rx=0 tx=0).stop 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 -- 172.21.15.170:0/1334946064 shutdown_connections 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 --2- 172.21.15.170:0/1334946064 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe350159a40 0x7fe3500c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 --2- 172.21.15.170:0/1334946064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe35015d7b0 0x7fe3500c9a50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 -- 172.21.15.170:0/1334946064 >> 172.21.15.170:0/1334946064 conn(0x7fe35006b450 msgr2=0x7fe3500cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 -- 172.21.15.170:0/1334946064 shutdown_connections 2023-10-08T16:09:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.107+0000 7fe357f69700 1 -- 172.21.15.170:0/1334946064 wait complete. 2023-10-08T16:09:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 Processor -- start 2023-10-08T16:09:27.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 -- start start 2023-10-08T16:09:27.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe350159e10 0x7fe350154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe35015d7b0 0x7fe3501512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350154740 con 0x7fe350159a40 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350155b50 con 0x7fe350159e10 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe357f69700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35015b5f0 con 0x7fe35015d7b0 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe355d05700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe355d05700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51978/0 (socket says 172.21.15.170:51978) 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe355d05700 1 -- 172.21.15.170:0/1004231250 learned_addr learned my addr 172.21.15.170:0/1004231250 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe355504700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe350159e10 0x7fe350154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.109+0000 7fe356506700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe35015d7b0 0x7fe3501512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 -- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe35015d7b0 msgr2=0x7fe3501512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe35015d7b0 0x7fe3501512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 -- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe350159e10 msgr2=0x7fe350154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe350159e10 0x7fe350154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 -- 172.21.15.170:0/1004231250 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400094f0 con 0x7fe350159a40 2023-10-08T16:09:27.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe355d05700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00e930 tx=0x7fe34c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c013680 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0137e0 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35015b8d0 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c013b80 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.110+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35015be90 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.111+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35015c200 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.112+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe34c00c3e0 con 0x7fe350159a40 2023-10-08T16:09:27.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.112+0000 7fe345ffb700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 0x7fe334069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.112+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fe34c084990 con 0x7fe350159a40 2023-10-08T16:09:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.112+0000 7fe355504700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 0x7fe334069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.112+0000 7fe355504700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 0x7fe334069620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3400099b0 tx=0x7fe340017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.116+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe34c051900 con 0x7fe350159a40 2023-10-08T16:09:27.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.373+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe35004c3c0 con 0x7fe350159a40 2023-10-08T16:09:27.373 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:09:27.373+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:09:27.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.377+0000 7fe345ffb700 1 -- 172.21.15.170:0/1004231250 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe34c054f20 con 0x7fe350159a40 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 msgr2=0x7fe334069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 0x7fe334069620 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3400099b0 tx=0x7fe340017000 comp rx=0 tx=0).stop 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 msgr2=0x7fe350155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe34c00e930 tx=0x7fe34c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 shutdown_connections 2023-10-08T16:09:27.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe35015d7b0 0x7fe3501512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe334067170 0x7fe334069620 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe350159a40 0x7fe350155450 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 --2- 172.21.15.170:0/1004231250 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe350159e10 0x7fe350154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 >> 172.21.15.170:0/1004231250 conn(0x7fe35006b450 msgr2=0x7fe3500ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 shutdown_connections 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.379+0000 7fe357f69700 1 -- 172.21.15.170:0/1004231250 wait complete. 2023-10-08T16:09:27.381 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:09:27.391 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:09:27.412 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:09:27.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.508+0000 7f98b3fff700 1 Processor -- start 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.508+0000 7f98b3fff700 1 -- start start 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f98b40b16d0 0x7f98b40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b3fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f98b40b33b0 0x7f98b40b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b3fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b40b2080 con 0x7f98b40b4e70 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b3fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40cfb00 con 0x7f98b40b33b0 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b3fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b40b5c40 con 0x7f98b40b16d0 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f98b40b33b0 0x7f98b40b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f98b40b33b0 0x7f98b40b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51980/0 (socket says 172.21.15.170:51980) 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 -- 172.21.15.170:0/358936932 learned_addr learned my addr 172.21.15.170:0/358936932 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98aa7fc700 1 --2- 172.21.15.170:0/358936932 >> v2:172.21.15.191:3300/0 conn(0x7f98b40b16d0 0x7f98b40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 -- 172.21.15.170:0/358936932 >> v2:172.21.15.191:3300/0 conn(0x7f98b40b16d0 msgr2=0x7f98b40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 --2- 172.21.15.170:0/358936932 >> v2:172.21.15.191:3300/0 conn(0x7f98b40b16d0 0x7f98b40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b37fe700 1 -- 172.21.15.170:0/358936932 >> v1:172.21.15.191:6789/0 conn(0x7f98b40b4e70 legacy=0x7f98b40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.509+0000 7f98b27fc700 1 -- 172.21.15.170:0/358936932 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864797482 0 0) 0x7f98b40b2080 con 0x7f98b40b4e70 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b37fe700 1 -- 172.21.15.170:0/358936932 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b40b5f20 con 0x7f98b40b33b0 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b37fe700 1 --2- 172.21.15.170:0/358936932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b33b0 0x7f98b40b37d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f98b4089830 tx=0x7f98ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de69edb07f552cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b27fc700 1 -- 172.21.15.170:0/358936932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00e070 con 0x7f98b40b33b0 2023-10-08T16:09:27.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b27fc700 1 -- 172.21.15.170:0/358936932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac00ba20 con 0x7f98b40b33b0 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b27fc700 1 -- 172.21.15.170:0/358936932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0092e0 con 0x7f98b40b33b0 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 -- 172.21.15.170:0/358936932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b33b0 msgr2=0x7f98b40b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 --2- 172.21.15.170:0/358936932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b33b0 0x7f98b40b37d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f98b4089830 tx=0x7f98ac009840 comp rx=0 tx=0).stop 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 -- 172.21.15.170:0/358936932 shutdown_connections 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 --2- 172.21.15.170:0/358936932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b33b0 0x7f98b40b37d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 --2- 172.21.15.170:0/358936932 >> v2:172.21.15.191:3300/0 conn(0x7f98b40b16d0 0x7f98b40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 -- 172.21.15.170:0/358936932 >> 172.21.15.170:0/358936932 conn(0x7f98b40d10c0 msgr2=0x7f98b40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 -- 172.21.15.170:0/358936932 shutdown_connections 2023-10-08T16:09:27.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.510+0000 7f98b3fff700 1 -- 172.21.15.170:0/358936932 wait complete. 2023-10-08T16:09:27.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 Processor -- start 2023-10-08T16:09:27.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 -- start start 2023-10-08T16:09:27.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b40b16d0 0x7f98b4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b40b33b0 0x7f98b40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b40cc410 con 0x7f98b40b4e70 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b416a320 con 0x7f98b40b16d0 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b3fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b4169a60 con 0x7f98b40b33b0 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.512+0000 7f98b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51992/0 (socket says 172.21.15.170:51992) 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 -- 172.21.15.170:0/2594371391 learned_addr learned my addr 172.21.15.170:0/2594371391 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b2ffd700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b40b16d0 0x7f98b4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98aa7fc700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b40b33b0 0x7f98b40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 -- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b40b33b0 msgr2=0x7f98b40affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b40b33b0 0x7f98b40affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 -- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b40b16d0 msgr2=0x7f98b4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b40b16d0 0x7f98b4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 -- 172.21.15.170:0/2594371391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac009ae0 con 0x7f98b40b4e70 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b37fe700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f98ac003290 tx=0x7f98ac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac007000 con 0x7f98b40b4e70 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac007970 con 0x7f98b40b4e70 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b40cc130 con 0x7f98b40b4e70 2023-10-08T16:09:27.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac007d10 con 0x7f98b40b4e70 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.513+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b4169fb0 con 0x7f98b40b4e70 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.514+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b4088990 con 0x7f98b40b4e70 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.514+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f98ac01d420 con 0x7f98b40b4e70 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.515+0000 7f98ab7fe700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 0x7f9894069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.515+0000 7f98b2ffd700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 0x7f9894069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.515+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f98ac086880 con 0x7f98b40b4e70 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.515+0000 7f98b2ffd700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 0x7f9894069620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98a4007fd0 tx=0x7f98a4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:27.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.519+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98ac053680 con 0x7f98b40b4e70 2023-10-08T16:09:27.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:27.744+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98b40b48c0 con 0x7f98b40b4e70 2023-10-08T16:09:28.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.377+0000 7f98ab7fe700 1 -- 172.21.15.170:0/2594371391 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f98ac056ca0 con 0x7f98b40b4e70 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 msgr2=0x7f9894069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 0x7f9894069620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f98a4007fd0 tx=0x7f98a4007d30 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 msgr2=0x7f98b40b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f98ac003290 tx=0x7f98ac018000 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 shutdown_connections 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9894067170 0x7f9894069620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98b40b4e70 0x7f98b40b04f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98b40b16d0 0x7f98b4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 --2- 172.21.15.170:0/2594371391 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98b40b33b0 0x7f98b40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 >> 172.21.15.170:0/2594371391 conn(0x7f98b40d10c0 msgr2=0x7f98b4069820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:28.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 shutdown_connections 2023-10-08T16:09:28.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.380+0000 7f98b3fff700 1 -- 172.21.15.170:0/2594371391 wait complete. 2023-10-08T16:09:28.393 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:09:28.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.512+0000 7f5ac1143700 1 Processor -- start 2023-10-08T16:09:28.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.512+0000 7f5ac1143700 1 -- start start 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ac1143700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5abc0cd910 0x7f5abc0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ac1143700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 0x7f5abc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ac1143700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0b5b00 con 0x7f5abc159ec0 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ac1143700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0b5da0 con 0x7f5abc0cd910 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ac1143700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0b6080 con 0x7f5abc0b55a0 2023-10-08T16:09:28.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5abc0cd910 0x7f5abc0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5abc0cd910 0x7f5abc0ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52000/0 (socket says 172.21.15.170:52000) 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 -- 172.21.15.170:0/2778108087 learned_addr learned my addr 172.21.15.170:0/2778108087 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5abb59e700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 0x7f5abc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 -- 172.21.15.170:0/2778108087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 msgr2=0x7f5abc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 0x7f5abc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 -- 172.21.15.170:0/2778108087 >> v1:172.21.15.191:6789/0 conn(0x7f5abc159ec0 legacy=0x7f5abc0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.513+0000 7f5ab259c700 1 -- 172.21.15.170:0/2778108087 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0cc100 con 0x7f5abc0cd910 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5abb59e700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 0x7f5abc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ab259c700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0cd910 0x7f5abc0ce340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099e0 tx=0x7f5aa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc27ebe511441328 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5aba59c700 1 -- 172.21.15.170:0/2778108087 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400b820 con 0x7f5abc0cd910 2023-10-08T16:09:28.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5aba59c700 1 -- 172.21.15.170:0/2778108087 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa400b980 con 0x7f5abc0cd910 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5aba59c700 1 -- 172.21.15.170:0/2778108087 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400bd20 con 0x7f5abc0cd910 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 -- 172.21.15.170:0/2778108087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0cd910 msgr2=0x7f5abc0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0cd910 0x7f5abc0ce340 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099e0 tx=0x7f5aa4009250 comp rx=0 tx=0).stop 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 -- 172.21.15.170:0/2778108087 shutdown_connections 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc0b55a0 0x7f5abc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2778108087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0cd910 0x7f5abc0ce340 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 -- 172.21.15.170:0/2778108087 >> 172.21.15.170:0/2778108087 conn(0x7f5abc0999f0 msgr2=0x7f5abc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 -- 172.21.15.170:0/2778108087 shutdown_connections 2023-10-08T16:09:28.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.514+0000 7f5ac1143700 1 -- 172.21.15.170:0/2778108087 wait complete. 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 Processor -- start 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 -- start start 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5abc159ec0 0x7f5abc06aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc1535b0 0x7f5abc14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0689e0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc153d80 con 0x7f5abc159ec0 2023-10-08T16:09:28.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ac1143700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc154060 con 0x7f5abc1535b0 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5abad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5abad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52008/0 (socket says 172.21.15.170:52008) 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5abad9d700 1 -- 172.21.15.170:0/2368671324 learned_addr learned my addr 172.21.15.170:0/2368671324 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5abb59e700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc1535b0 0x7f5abc14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.516+0000 7f5ab259c700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5abc159ec0 0x7f5abc06aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 -- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc1535b0 msgr2=0x7f5abc14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc1535b0 0x7f5abc14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 -- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5abc159ec0 msgr2=0x7f5abc06aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5abc159ec0 0x7f5abc06aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 -- 172.21.15.170:0/2368671324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa40094f0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5abad9d700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c7c0 tx=0x7f5aac007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac011590 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac0116f0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5abc068390 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac011ab0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.517+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc153a00 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.518+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc063c90 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.518+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f5aac01b3b0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.519+0000 7f5ab37fe700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 0x7f5a9c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.519+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5907+0+0 (secure 0 0 0) 0x7f5aac083cf0 con 0x7f5abc0b55a0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.519+0000 7f5ab259c700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 0x7f5a9c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:28.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.519+0000 7f5ab259c700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 0x7f5a9c069620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099e0 tx=0x7f5aa4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:28.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.523+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5aac088070 con 0x7f5abc0b55a0 2023-10-08T16:09:28.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:28.708+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5abc0718b0 con 0x7f5abc0b55a0 2023-10-08T16:09:29.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.585+0000 7f5ab37fe700 1 -- 172.21.15.170:0/2368671324 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5aac017070 con 0x7f5abc0b55a0 2023-10-08T16:09:29.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.589+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 msgr2=0x7f5a9c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.589+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 0x7f5a9c069620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40099e0 tx=0x7f5aa4017000 comp rx=0 tx=0).stop 2023-10-08T16:09:29.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.589+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 msgr2=0x7f5abc156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.589+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00c7c0 tx=0x7f5aac007f70 comp rx=0 tx=0).stop 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 shutdown_connections 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5abc1535b0 0x7f5abc14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5a9c067170 0x7f5a9c069620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5abc0b55a0 0x7f5abc156d80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 --2- 172.21.15.170:0/2368671324 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5abc159ec0 0x7f5abc06aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.590+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 >> 172.21.15.170:0/2368671324 conn(0x7f5abc0999f0 msgr2=0x7f5abc0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.593+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 shutdown_connections 2023-10-08T16:09:29.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.593+0000 7f5ac1143700 1 -- 172.21.15.170:0/2368671324 wait complete. 2023-10-08T16:09:29.597 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:09:29.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:09:29.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.726+0000 7f65d4231700 1 Processor -- start 2023-10-08T16:09:29.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.726+0000 7f65d4231700 1 -- start start 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d4231700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f65cc159ec0 0x7f65cc0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d4231700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc0b5b00 con 0x7f65cc0cf350 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d4231700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc0b5850 con 0x7f65cc0b53d0 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d4231700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc0b5da0 con 0x7f65cc159ec0 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d1fcd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f65cc159ec0 0x7f65cc0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d27ce700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f65cc0cf350 0x7f65cc0cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58112/0 (socket says 172.21.15.170:58112) 2023-10-08T16:09:29.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.727+0000 7f65d27ce700 1 -- 172.21.15.170:0/1793502781 learned_addr learned my addr 172.21.15.170:0/1793502781 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d1fcd700 1 -- 172.21.15.170:0/1793502781 >> v1:172.21.15.191:6789/0 conn(0x7f65cc0b53d0 legacy=0x7f65cc0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d1fcd700 1 -- 172.21.15.170:0/1793502781 >> v1:172.21.15.170:6789/0 conn(0x7f65cc0cf350 legacy=0x7f65cc0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d1fcd700 1 -- 172.21.15.170:0/1793502781 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0b6080 con 0x7f65cc159ec0 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d0fcb700 1 -- 172.21.15.170:0/1793502781 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074805717 0 0) 0x7f65cc0b5b00 con 0x7f65cc0cf350 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d1fcd700 1 --2- 172.21.15.170:0/1793502781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc0b4e90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009eb0 tx=0x7f65c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1b0e036dff9e08 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d0fcb700 1 -- 172.21.15.170:0/1793502781 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800e070 con 0x7f65cc159ec0 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d0fcb700 1 -- 172.21.15.170:0/1793502781 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800bc30 con 0x7f65cc159ec0 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d0fcb700 1 -- 172.21.15.170:0/1793502781 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8012430 con 0x7f65cc159ec0 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d4231700 1 -- 172.21.15.170:0/1793502781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 msgr2=0x7f65cc0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d4231700 1 --2- 172.21.15.170:0/1793502781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc0b4e90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009eb0 tx=0x7f65c8009780 comp rx=0 tx=0).stop 2023-10-08T16:09:29.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d4231700 1 -- 172.21.15.170:0/1793502781 shutdown_connections 2023-10-08T16:09:29.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d4231700 1 --2- 172.21.15.170:0/1793502781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc0b4e90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.728+0000 7f65d4231700 1 -- 172.21.15.170:0/1793502781 >> 172.21.15.170:0/1793502781 conn(0x7f65cc0999f0 msgr2=0x7f65cc0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:29.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.729+0000 7f65d4231700 1 -- 172.21.15.170:0/1793502781 shutdown_connections 2023-10-08T16:09:29.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.729+0000 7f65d4231700 1 -- 172.21.15.170:0/1793502781 wait complete. 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.730+0000 7f65d4231700 1 Processor -- start 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.730+0000 7f65d4231700 1 -- start start 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.730+0000 7f65d4231700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 0x7f65cc09b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.730+0000 7f65d4231700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 0x7f65cc169ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc16dad0 con 0x7f65cc159ec0 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc16d4a0 con 0x7f65cc0cf350 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc16a9d0 con 0x7f65cc0b53d0 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:29.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52032/0 (socket says 172.21.15.170:52032) 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 -- 172.21.15.170:0/823634231 learned_addr learned my addr 172.21.15.170:0/823634231 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d1fcd700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 0x7f65cc09b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d17cc700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 0x7f65cc169ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 -- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 msgr2=0x7f65cc09b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 0x7f65cc09b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 -- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 msgr2=0x7f65cc169ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 0x7f65cc169ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 -- 172.21.15.170:0/823634231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009a20 con 0x7f65cc159ec0 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d17cc700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 0x7f65cc169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d1fcd700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 0x7f65cc09b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d27ce700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c7e0 tx=0x7f65c4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:29.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4005c00 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4005d60 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc16acb0 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4004a80 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.731+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc1574f0 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.732+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc063c90 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.733+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f65c4005030 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.733+0000 7f65c1ffb700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 0x7f65b0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.733+0000 7f65d1fcd700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 0x7f65b0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.733+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5469+0+0 (secure 0 0 0) 0x7f65c4083bd0 con 0x7f65cc159ec0 2023-10-08T16:09:29.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.733+0000 7f65d1fcd700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 0x7f65b0069620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009eb0 tx=0x7f65c8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:29.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.737+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65c404b5a0 con 0x7f65cc159ec0 2023-10-08T16:09:29.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:29.923+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f65cc0718b0 con 0x7f65cc159ec0 2023-10-08T16:09:30.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.590+0000 7f65c1ffb700 1 -- 172.21.15.170:0/823634231 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f65c4050ca0 con 0x7f65cc159ec0 2023-10-08T16:09:30.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 msgr2=0x7f65b0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 0x7f65b0069620 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009eb0 tx=0x7f65c8017000 comp rx=0 tx=0).stop 2023-10-08T16:09:30.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 msgr2=0x7f65cc16a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c7e0 tx=0x7f65c4007d90 comp rx=0 tx=0).stop 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 shutdown_connections 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b0067170 0x7f65b0069620 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65cc159ec0 0x7f65cc16a3e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65cc0b53d0 0x7f65cc09b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 --2- 172.21.15.170:0/823634231 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65cc0cf350 0x7f65cc169ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.593+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 >> 172.21.15.170:0/823634231 conn(0x7f65cc0999f0 msgr2=0x7f65cc1590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:30.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.594+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 shutdown_connections 2023-10-08T16:09:30.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.594+0000 7f65d4231700 1 -- 172.21.15.170:0/823634231 wait complete. 2023-10-08T16:09:30.595 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:09:30.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:09:30.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.724+0000 7fb1e499d700 1 Processor -- start 2023-10-08T16:09:30.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.724+0000 7fb1e499d700 1 -- start start 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1e499d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1e00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1e499d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb1e00ccc00 0x7fb1e0069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1e499d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e00c9cb0 con 0x7fb1e006a020 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1e499d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e006dfc0 con 0x7fb1e00ccc00 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1e499d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c9f30 con 0x7fb1e0159a60 2023-10-08T16:09:30.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1dffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb1e006a020 0x7fb1e00cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58126/0 (socket says 172.21.15.170:58126) 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.725+0000 7fb1dffff700 1 -- 172.21.15.170:0/2395322061 learned_addr learned my addr 172.21.15.170:0/2395322061 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132771164 0 0) 0x7fb1e00c9cb0 con 0x7fb1e006a020 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c8003570 con 0x7fb1e006a020 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1df7fe700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1e00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2395322061 >> v2:172.21.15.191:3300/0 conn(0x7fb1e00ccc00 0x7fb1e0069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394252954 0 0) 0x7fb1c8003570 con 0x7fb1e006a020 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 msgr2=0x7fb1e00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1e00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 >> v2:172.21.15.191:3300/0 conn(0x7fb1e00ccc00 msgr2=0x7fb1e0069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 --2- 172.21.15.170:0/2395322061 >> v2:172.21.15.191:3300/0 conn(0x7fb1e00ccc00 0x7fb1e0069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e00ca470 con 0x7fb1e006a020 2023-10-08T16:09:30.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb1d80017f0 con 0x7fb1e006a020 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 >> v1:172.21.15.170:6789/0 conn(0x7fb1e006a020 legacy=0x7fb1e00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8003a60 0x7fb1c8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8004810 0x7fb1c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e006dfc0 con 0x7fb1c8004810 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c9f30 con 0x7fb1c8003a60 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c800ba60 con 0x7fb1e0159a60 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8004810 0x7fb1c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1dffff700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1df7fe700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8003a60 0x7fb1c8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 -- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 msgr2=0x7fb1c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:30.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e0159a60 0x7fb1c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 -- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8003a60 msgr2=0x7fb1c8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8003a60 0x7fb1c8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.726+0000 7fb1deffd700 1 -- 172.21.15.170:0/2395322061 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c8003570 con 0x7fb1c8004810 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8004810 0x7fb1c8004c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009c30 tx=0x7fb1d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3759d17af689a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400e710 con 0x7fb1c8004810 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d400e870 con 0x7fb1c8004810 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1de7fc700 1 -- 172.21.15.170:0/2395322061 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d400ec30 con 0x7fb1c8004810 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 -- 172.21.15.170:0/2395322061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8004810 msgr2=0x7fb1c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2395322061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8004810 0x7fb1c8004c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009c30 tx=0x7fb1d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1dffff700 1 -- 172.21.15.170:0/2395322061 reap_dead start 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 -- 172.21.15.170:0/2395322061 shutdown_connections 2023-10-08T16:09:30.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 -- 172.21.15.170:0/2395322061 >> 172.21.15.170:0/2395322061 conn(0x7fb1e006baa0 msgr2=0x7fb1e006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:30.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 -- 172.21.15.170:0/2395322061 shutdown_connections 2023-10-08T16:09:30.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.727+0000 7fb1e499d700 1 -- 172.21.15.170:0/2395322061 wait complete. 2023-10-08T16:09:30.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.728+0000 7fb1e499d700 1 Processor -- start 2023-10-08T16:09:30.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 -- start start 2023-10-08T16:09:30.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8004810 0x7fb1e0159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e00b3760 0x7fb1e013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e015b590 con 0x7fb1e01583a0 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e015a370 con 0x7fb1c8004810 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1e499d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e00c65b0 con 0x7fb1e00b3760 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1dffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52060/0 (socket says 172.21.15.170:52060) 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1dffff700 1 -- 172.21.15.170:0/2949652072 learned_addr learned my addr 172.21.15.170:0/2949652072 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1df7fe700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8004810 0x7fb1e0159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.729+0000 7fb1deffd700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e00b3760 0x7fb1e013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 -- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e00b3760 msgr2=0x7fb1e013ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e00b3760 0x7fb1e013ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 -- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8004810 msgr2=0x7fb1e0159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8004810 0x7fb1e0159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 -- 172.21.15.170:0/2949652072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e0158d30 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1dffff700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb1e006f4b0 tx=0x7fb1d8002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d800acd0 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d800ae50 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e00c6280 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8013640 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.730+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e015a880 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.731+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e00c6740 con 0x7fb1e01583a0 2023-10-08T16:09:30.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.731+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb1d80139b0 con 0x7fb1e01583a0 2023-10-08T16:09:30.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.732+0000 7fb1c77fe700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 0x7fb1cc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:30.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.732+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb1d80850d0 con 0x7fb1e01583a0 2023-10-08T16:09:30.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.732+0000 7fb1df7fe700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 0x7fb1cc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:30.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.732+0000 7fb1df7fe700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 0x7fb1cc069620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0006f00 tx=0x7fb1d00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:30.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.736+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1d80523b0 con 0x7fb1e01583a0 2023-10-08T16:09:30.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.920+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb1e00ca750 con 0x7fb1e01583a0 2023-10-08T16:09:30.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.924+0000 7fb1c77fe700 1 -- 172.21.15.170:0/2949652072 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb1d80559d0 con 0x7fb1e01583a0 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 msgr2=0x7fb1cc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 0x7fb1cc069620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0006f00 tx=0x7fb1d00091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 msgr2=0x7fb1e01587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb1e006f4b0 tx=0x7fb1d8002210 comp rx=0 tx=0).stop 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 shutdown_connections 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb1cc067170 0x7fb1cc069620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1e01583a0 0x7fb1e01587f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1c8004810 0x7fb1e0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 --2- 172.21.15.170:0/2949652072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1e00b3760 0x7fb1e013ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:30.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 >> 172.21.15.170:0/2949652072 conn(0x7fb1e006baa0 msgr2=0x7fb1e0156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:30.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 shutdown_connections 2023-10-08T16:09:30.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:30.926+0000 7fb1e499d700 1 -- 172.21.15.170:0/2949652072 wait complete. 2023-10-08T16:09:30.938 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions' 2023-10-08T16:09:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.052+0000 7fabc483b700 1 Processor -- start 2023-10-08T16:09:31.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.052+0000 7fabc483b700 1 -- start start 2023-10-08T16:09:31.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.053+0000 7fabc483b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.053+0000 7fabc483b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fabc0069560 0x7fabc00699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.053+0000 7fabc483b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc015a160 con 0x7fabc0159c60 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.053+0000 7fabc483b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc0069f20 con 0x7fabc0069560 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.053+0000 7fabc483b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00d13a0 con 0x7fabc0159890 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbe59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbe59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:44006/0 (socket says 172.21.15.170:44006) 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbe59c700 1 -- 172.21.15.170:0/2234195571 learned_addr learned my addr 172.21.15.170:0/2234195571 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 --2- 172.21.15.170:0/2234195571 >> v2:172.21.15.191:3300/0 conn(0x7fabc0069560 0x7fabc00699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbdd9b700 1 --1- 172.21.15.170:0/2234195571 >> v1:172.21.15.191:6789/0 conn(0x7fabc0159c60 0x7fabc015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:44312/0 (socket says 172.21.15.170:44312) 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbd59a700 1 -- 172.21.15.170:0/2234195571 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303938615 0 0) 0x7fabc015a160 con 0x7fabc0159c60 2023-10-08T16:09:31.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbd59a700 1 -- 172.21.15.170:0/2234195571 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba8003570 con 0x7fabc0159c60 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 -- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 msgr2=0x7fabc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 -- 172.21.15.170:0/2234195571 >> v1:172.21.15.191:6789/0 conn(0x7fabc0159c60 legacy=0x7fabc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 -- 172.21.15.170:0/2234195571 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc015e140 con 0x7fabc0069560 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbe59c700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbed9d700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0069560 0x7fabc00699e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabb80098e0 tx=0x7fabb80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1b674c5bca6c42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.054+0000 7fabbd59a700 1 -- 172.21.15.170:0/2234195571 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8011630 con 0x7fabc0069560 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabbd59a700 1 -- 172.21.15.170:0/2234195571 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb8011790 con 0x7fabc0069560 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabbd59a700 1 -- 172.21.15.170:0/2234195571 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8011b30 con 0x7fabc0069560 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 -- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0069560 msgr2=0x7fabc00699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0069560 0x7fabc00699e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabb80098e0 tx=0x7fabb80091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 -- 172.21.15.170:0/2234195571 shutdown_connections 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0069560 0x7fabc00699e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 --2- 172.21.15.170:0/2234195571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159890 0x7fabc015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 -- 172.21.15.170:0/2234195571 >> 172.21.15.170:0/2234195571 conn(0x7fabc0068e20 msgr2=0x7fabc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 -- 172.21.15.170:0/2234195571 shutdown_connections 2023-10-08T16:09:31.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.055+0000 7fabc483b700 1 -- 172.21.15.170:0/2234195571 wait complete. 2023-10-08T16:09:31.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.056+0000 7fabc483b700 1 Processor -- start 2023-10-08T16:09:31.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.056+0000 7fabc483b700 1 -- start start 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0159890 0x7fabc00c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159c60 0x7fabc00c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00b42a0 con 0x7fabc0069560 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cdea0 con 0x7fabc0159890 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabc483b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cd570 con 0x7fabc0159c60 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52072/0 (socket says 172.21.15.170:52072) 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 -- 172.21.15.170:0/1873682932 learned_addr learned my addr 172.21.15.170:0/1873682932 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbdd9b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0159890 0x7fabc00c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbed9d700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159c60 0x7fabc00c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 -- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159c60 msgr2=0x7fabc00c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159c60 0x7fabc00c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 -- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0159890 msgr2=0x7fabc00c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0159890 0x7fabc00c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 -- 172.21.15.170:0/1873682932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009ae0 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.057+0000 7fabbe59c700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c170 tx=0x7fabb000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb0009dc0 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb000f040 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb8009420 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb000ccd0 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc00cdb30 con 0x7fabc0069560 2023-10-08T16:09:31.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.058+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc004c3c0 con 0x7fabc0069560 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.059+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fabb0014690 con 0x7fabc0069560 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.059+0000 7fabae7fc700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 0x7faba0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.059+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fabb0084cd0 con 0x7fabc0069560 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.059+0000 7fabbdd9b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 0x7faba0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.060+0000 7fabbdd9b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 0x7faba0069620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc0157190 tx=0x7fabb4007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.063+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabb0051e00 con 0x7fabc0069560 2023-10-08T16:09:31.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.326+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]} v 0) v1 -- 0x7fabc006a5b0 con 0x7fabc0069560 2023-10-08T16:09:31.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.591+0000 7fabae7fc700 1 -- 172.21.15.170:0/1873682932 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_pending_subvol_deletions"]}]=0 v225) v1 ==== 158+0+0 (secure 0 0 0) 0x7fabb0055420 con 0x7fabc0069560 2023-10-08T16:09:31.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 msgr2=0x7faba0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 0x7faba0069620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc0157190 tx=0x7fabb4007d40 comp rx=0 tx=0).stop 2023-10-08T16:09:31.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 msgr2=0x7fabc01443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fabb000c170 tx=0x7fabb000b040 comp rx=0 tx=0).stop 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 shutdown_connections 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fabc0159c60 0x7fabc00c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faba0067170 0x7faba0069620 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabc0069560 0x7fabc01443f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 --2- 172.21.15.170:0/1873682932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fabc0159890 0x7fabc00c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 >> 172.21.15.170:0/1873682932 conn(0x7fabc0068e20 msgr2=0x7fabc00b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 shutdown_connections 2023-10-08T16:09:31.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.593+0000 7fabc483b700 1 -- 172.21.15.170:0/1873682932 wait complete. 2023-10-08T16:09:31.605 INFO:tasks.cephfs_test_runner:Tests the pending_subvolume_deletions in 'fs volume info' command ... ok 2023-10-08T16:09:31.606 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command 2023-10-08T16:09:31.606 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:09:31.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-08T16:09:31.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 Processor -- start 2023-10-08T16:09:31.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 -- start start 2023-10-08T16:09:31.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f142c069d60 0x7f142c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c0c81f0 0x7f142c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0c8650 con 0x7f142c0aca80 2023-10-08T16:09:31.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142c06bdb0 con 0x7f142c069d60 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.721+0000 7f143100d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0cb370 con 0x7f142c0c81f0 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f142c069d60 0x7f142c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f142c069d60 0x7f142c0c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52086/0 (socket says 172.21.15.170:52086) 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 -- 172.21.15.170:0/2131107993 learned_addr learned my addr 172.21.15.170:0/2131107993 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142b59e700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c0c81f0 0x7f142c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 -- 172.21.15.170:0/2131107993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c0c81f0 msgr2=0x7f142c0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c0c81f0 0x7f142c0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 -- 172.21.15.170:0/2131107993 >> v1:172.21.15.191:6789/0 conn(0x7f142c0aca80 legacy=0x7f142c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 -- 172.21.15.170:0/2131107993 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c159f60 con 0x7f142c069d60 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f142a59c700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c069d60 0x7f142c0c7cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f14200099b0 tx=0x7f1420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76411485fc524247 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f1429d9b700 1 -- 172.21.15.170:0/2131107993 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142000b860 con 0x7f142c069d60 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f1429d9b700 1 -- 172.21.15.170:0/2131107993 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142000b9c0 con 0x7f142c069d60 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.722+0000 7f1429d9b700 1 -- 172.21.15.170:0/2131107993 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142000bd60 con 0x7f142c069d60 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 -- 172.21.15.170:0/2131107993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c069d60 msgr2=0x7f142c0c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c069d60 0x7f142c0c7cb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f14200099b0 tx=0x7f1420009250 comp rx=0 tx=0).stop 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 -- 172.21.15.170:0/2131107993 shutdown_connections 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c0c81f0 0x7f142c0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 --2- 172.21.15.170:0/2131107993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c069d60 0x7f142c0c7cb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 -- 172.21.15.170:0/2131107993 >> 172.21.15.170:0/2131107993 conn(0x7f142c0cd670 msgr2=0x7f142c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:31.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 -- 172.21.15.170:0/2131107993 shutdown_connections 2023-10-08T16:09:31.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.723+0000 7f143100d700 1 -- 172.21.15.170:0/2131107993 wait complete. 2023-10-08T16:09:31.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.724+0000 7f143100d700 1 Processor -- start 2023-10-08T16:09:31.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.724+0000 7f143100d700 1 -- start start 2023-10-08T16:09:31.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 0x7f142c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 0x7f142c15a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0af280 con 0x7f142c0aca80 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0b0410 con 0x7f142c0c81f0 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f143100d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c157ba0 con 0x7f142c069d60 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52092/0 (socket says 172.21.15.170:52092) 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 -- 172.21.15.170:0/2516555822 learned_addr learned my addr 172.21.15.170:0/2516555822 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142b59e700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 0x7f142c15a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142ad9d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 0x7f142c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 -- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 msgr2=0x7f142c0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 0x7f142c0aed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 -- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 msgr2=0x7f142c15a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 0x7f142c15a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 -- 172.21.15.170:0/2516555822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14200094f0 con 0x7f142c0aca80 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142b59e700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 0x7f142c15a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142ad9d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 0x7f142c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f142a59c700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f142c06c6a0 tx=0x7f1420017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.725+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420024440 con 0x7f142c0aca80 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.726+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14200245c0 con 0x7f142c0aca80 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.726+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c15a8d0 con 0x7f142c0aca80 2023-10-08T16:09:31.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.726+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420024980 con 0x7f142c0aca80 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.726+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c0b0990 con 0x7f142c0aca80 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.726+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c04c3c0 con 0x7f142c0aca80 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.727+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f142001a420 con 0x7f142c0aca80 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.727+0000 7f1412ffd700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 0x7f1418069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.727+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1420086410 con 0x7f142c0aca80 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.727+0000 7f142ad9d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 0x7f1418069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.728+0000 7f142ad9d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 0x7f1418069620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f141c0018f0 tx=0x7f141c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:31.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.731+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14200536f0 con 0x7f142c0aca80 2023-10-08T16:09:31.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:31.987+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7f142c0678d0 con 0x7f142c0aca80 2023-10-08T16:09:32.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.594+0000 7f1412ffd700 1 -- 172.21.15.170:0/2516555822 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v226) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1420056d10 con 0x7f142c0aca80 2023-10-08T16:09:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 msgr2=0x7f1418069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 0x7f1418069620 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f141c0018f0 tx=0x7f141c009040 comp rx=0 tx=0).stop 2023-10-08T16:09:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 msgr2=0x7f142c157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f142c06c6a0 tx=0x7f1420017000 comp rx=0 tx=0).stop 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 shutdown_connections 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f142c0c81f0 0x7f142c15a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f142c069d60 0x7f142c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1418067170 0x7f1418069620 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 --2- 172.21.15.170:0/2516555822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f142c0aca80 0x7f142c157660 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 >> 172.21.15.170:0/2516555822 conn(0x7f142c0cd670 msgr2=0x7f142c157dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 shutdown_connections 2023-10-08T16:09:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.596+0000 7f143100d700 1 -- 172.21.15.170:0/2516555822 wait complete. 2023-10-08T16:09:32.609 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:09:32.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.726+0000 7f8815054700 1 Processor -- start 2023-10-08T16:09:32.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.726+0000 7f8815054700 1 -- start start 2023-10-08T16:09:32.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8815054700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f88100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8815054700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88100cfb00 con 0x7f88100b1b40 2023-10-08T16:09:32.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8815054700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881015e060 con 0x7f88100b16d0 2023-10-08T16:09:32.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8815054700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88100b5c40 con 0x7f88100b4e70 2023-10-08T16:09:32.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8814853700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f88100b1b40 0x7f881015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58138/0 (socket says 172.21.15.170:58138) 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f8814853700 1 -- 172.21.15.170:0/4009948100 learned_addr learned my addr 172.21.15.170:0/4009948100 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508677141 0 0) 0x7f88100cfb00 con 0x7f88100b1b40 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f8003570 con 0x7f88100b1b40 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880ffff700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f88100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58299871 0 0) 0x7f87f8003570 con 0x7f88100b1b40 2023-10-08T16:09:32.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 msgr2=0x7f88100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f88100ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 >> v1:172.21.15.191:6789/0 conn(0x7f88100b16d0 legacy=0x7f881015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100b5f20 con 0x7f88100b1b40 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f88080017f0 con 0x7f88100b1b40 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.727+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 >> v1:172.21.15.170:6789/0 conn(0x7f88100b1b40 legacy=0x7f881015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f87f8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8004010 0x7f87f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004d50 0x7f87f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88100b5c40 con 0x7f87f8004010 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f800bbf0 con 0x7f87f8004d50 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f800bdb0 con 0x7f88100b4e70 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8004010 0x7f87f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880ffff700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f87f8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f8814853700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004d50 0x7f87f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 -- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 msgr2=0x7f87f8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f88100b4e70 0x7f87f8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 -- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004d50 msgr2=0x7f87f80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004d50 0x7f87f80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 -- 172.21.15.170:0/4009948100 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8003570 con 0x7f87f8004010 2023-10-08T16:09:32.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f8814853700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004d50 0x7f87f80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:32.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880f7fe700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8004010 0x7f87f8004810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8804001880 tx=0x7f8804007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363120c94dfb51a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:32.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804012420 con 0x7f87f8004010 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8804012580 con 0x7f87f8004010 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.728+0000 7f880effd700 1 -- 172.21.15.170:0/4009948100 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804012920 con 0x7f87f8004010 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 -- 172.21.15.170:0/4009948100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8004010 msgr2=0x7f87f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 --2- 172.21.15.170:0/4009948100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87f8004010 0x7f87f8004810 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8804001880 tx=0x7f8804007d30 comp rx=0 tx=0).stop 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8814853700 1 -- 172.21.15.170:0/4009948100 reap_dead start 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 -- 172.21.15.170:0/4009948100 shutdown_connections 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 -- 172.21.15.170:0/4009948100 >> 172.21.15.170:0/4009948100 conn(0x7f88100d10c0 msgr2=0x7f88100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 -- 172.21.15.170:0/4009948100 shutdown_connections 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.729+0000 7f8815054700 1 -- 172.21.15.170:0/4009948100 wait complete. 2023-10-08T16:09:32.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.730+0000 7f8815054700 1 Processor -- start 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.730+0000 7f8815054700 1 -- start start 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004010 0x7f881015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8810151bc0 0x7f881016a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881016aa40 con 0x7f8810151750 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881016a580 con 0x7f87f8004010 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88101549b0 con 0x7f8810151bc0 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52110/0 (socket says 172.21.15.170:52110) 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 -- 172.21.15.170:0/3858220798 learned_addr learned my addr 172.21.15.170:0/3858220798 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:32.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 -- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8810151bc0 msgr2=0x7f881016a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8810151bc0 0x7f881016a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880ffff700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004010 0x7f881015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 -- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004010 msgr2=0x7f881015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004010 0x7f881015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 -- 172.21.15.170:0/3858220798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881015da80 con 0x7f8810151750 2023-10-08T16:09:32.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f880f7fe700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8804001e70 tx=0x7f8804015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:32.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8804001200 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8804011b50 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8810153da0 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880401e400 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.731+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8810154360 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.732+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8810088990 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.733+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8804019070 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.733+0000 7f87f7fff700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 0x7f87fc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.733+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8804085c60 con 0x7f8810151750 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.733+0000 7f880ffff700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 0x7f87fc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.734+0000 7f880ffff700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 0x7f87fc069620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88000093f0 tx=0x7f8800008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.737+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8804052f40 con 0x7f8810151750 2023-10-08T16:09:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.921+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8810154af0 con 0x7f8810151750 2023-10-08T16:09:32.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.924+0000 7f87f7fff700 1 -- 172.21.15.170:0/3858220798 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88040170c0 con 0x7f8810151750 2023-10-08T16:09:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 msgr2=0x7f87fc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 0x7f87fc069620 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88000093f0 tx=0x7f8800008040 comp rx=0 tx=0).stop 2023-10-08T16:09:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 msgr2=0x7f8810169b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8804001e70 tx=0x7f8804015000 comp rx=0 tx=0).stop 2023-10-08T16:09:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 shutdown_connections 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8810151bc0 0x7f881016a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87f8004010 0x7f881015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87fc067170 0x7f87fc069620 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 --2- 172.21.15.170:0/3858220798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8810151750 0x7f8810169b00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 >> 172.21.15.170:0/3858220798 conn(0x7f88100d10c0 msgr2=0x7f8810158150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 shutdown_connections 2023-10-08T16:09:32.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:32.926+0000 7f8815054700 1 -- 172.21.15.170:0/3858220798 wait complete. 2023-10-08T16:09:32.938 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:09:32.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:32.991 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:33.036 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:09:33.036 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:09:33.036 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:33.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.150+0000 7f991d405700 1 Processor -- start 2023-10-08T16:09:33.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.150+0000 7f991d405700 1 -- start start 2023-10-08T16:09:33.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f991d405700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9918159890 0x7f991815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f991d405700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9918069560 0x7f99180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f991d405700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9918069ee0 con 0x7f9918159c60 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f991d405700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991815a160 con 0x7f9918159890 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f991d405700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99180d13a0 con 0x7f9918069560 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9918159890 0x7f991815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f990e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9918159c60 0x7f991815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58144/0 (socket says 172.21.15.170:58144) 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9918159890 0x7f991815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52118/0 (socket says 172.21.15.170:52118) 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f990e7fc700 1 -- 172.21.15.170:0/513400843 learned_addr learned my addr 172.21.15.170:0/513400843 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f99177fe700 1 --2- 172.21.15.170:0/513400843 >> v2:172.21.15.191:3300/0 conn(0x7f9918069560 0x7f99180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 -- 172.21.15.170:0/513400843 >> v2:172.21.15.191:3300/0 conn(0x7f9918069560 msgr2=0x7f99180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 --2- 172.21.15.170:0/513400843 >> v2:172.21.15.191:3300/0 conn(0x7f9918069560 0x7f99180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 -- 172.21.15.170:0/513400843 >> v1:172.21.15.170:6789/0 conn(0x7f9918159c60 legacy=0x7f991815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f99167fc700 1 -- 172.21.15.170:0/513400843 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732162865 0 0) 0x7f9918069ee0 con 0x7f9918159c60 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.151+0000 7f9916ffd700 1 -- 172.21.15.170:0/513400843 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991815e140 con 0x7f9918159890 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f9916ffd700 1 --2- 172.21.15.170:0/513400843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918159890 0x7f991815abd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f991806a890 tx=0x7f99080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854964e295d9653c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f99167fc700 1 -- 172.21.15.170:0/513400843 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800b6b0 con 0x7f9918159890 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f99167fc700 1 -- 172.21.15.170:0/513400843 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800b810 con 0x7f9918159890 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f99167fc700 1 -- 172.21.15.170:0/513400843 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800bbb0 con 0x7f9918159890 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 -- 172.21.15.170:0/513400843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918159890 msgr2=0x7f991815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 --2- 172.21.15.170:0/513400843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918159890 0x7f991815abd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f991806a890 tx=0x7f99080091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 -- 172.21.15.170:0/513400843 shutdown_connections 2023-10-08T16:09:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 --2- 172.21.15.170:0/513400843 >> v2:172.21.15.191:3300/0 conn(0x7f9918069560 0x7f99180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 --2- 172.21.15.170:0/513400843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918159890 0x7f991815abd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 -- 172.21.15.170:0/513400843 >> 172.21.15.170:0/513400843 conn(0x7f9918068e20 msgr2=0x7f99180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 -- 172.21.15.170:0/513400843 shutdown_connections 2023-10-08T16:09:33.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.152+0000 7f991d405700 1 -- 172.21.15.170:0/513400843 wait complete. 2023-10-08T16:09:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.153+0000 7f991d405700 1 Processor -- start 2023-10-08T16:09:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.153+0000 7f991d405700 1 -- start start 2023-10-08T16:09:33.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9918159890 0x7f99180af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9918159c60 0x7f99180c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180ce090 con 0x7f9918069560 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180ce930 con 0x7f9918159c60 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f991d405700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991816b5b0 con 0x7f9918159890 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f9916ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52124/0 (socket says 172.21.15.170:52124) 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f9916ffd700 1 -- 172.21.15.170:0/3986985376 learned_addr learned my addr 172.21.15.170:0/3986985376 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f990e7fc700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9918159890 0x7f99180af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.154+0000 7f99177fe700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9918159c60 0x7f99180c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 -- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9918159890 msgr2=0x7f99180af830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9918159890 0x7f99180af830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 -- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9918159c60 msgr2=0x7f99180c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9918159c60 0x7f99180c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 -- 172.21.15.170:0/3986985376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908009420 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f9916ffd700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f990800b290 tx=0x7f9908017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908000d10 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800bb90 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99180caa50 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908011670 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.155+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99180ce5c0 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.156+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991804c3c0 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.156+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9908011a60 con 0x7f9918069560 2023-10-08T16:09:33.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.157+0000 7f990f7fe700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 0x7f98f80696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.157+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9908085db0 con 0x7f9918069560 2023-10-08T16:09:33.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.157+0000 7f990e7fc700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 0x7f98f80696a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.157+0000 7f990e7fc700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 0x7f98f80696a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f991816b020 tx=0x7f9900007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.161+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9908052ff0 con 0x7f9918069560 2023-10-08T16:09:33.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.386+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991806a5b0 con 0x7f9918069560 2023-10-08T16:09:33.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.387+0000 7f990f7fe700 1 -- 172.21.15.170:0/3986985376 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f9908015df0 con 0x7f9918069560 2023-10-08T16:09:33.387 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:33.387 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/1135013687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1135013687},{"type":"v1","addr":"172.21.15.191:6835","nonce":1135013687}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5641,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4024377779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4024377779},{"type":"v1","addr":"172.21.15.191:6834","nonce":4024377779}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/624691903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":624691903},{"type":"v1","addr":"172.21.15.170:6836","nonce":624691903}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":5649,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2096905460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2096905460},{"type":"v1","addr":"172.21.15.170:6837","nonce":2096905460}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-10-08T16:09:33.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 msgr2=0x7f98f80696a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 0x7f98f80696a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f991816b020 tx=0x7f9900007f30 comp rx=0 tx=0).stop 2023-10-08T16:09:33.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 msgr2=0x7f991816b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f990800b290 tx=0x7f9908017000 comp rx=0 tx=0).stop 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 shutdown_connections 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9918159c60 0x7f99180c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98f80671f0 0x7f98f80696a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9918069560 0x7f991816b070 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 --2- 172.21.15.170:0/3986985376 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9918159890 0x7f99180af830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 >> 172.21.15.170:0/3986985376 conn(0x7f9918068e20 msgr2=0x7f99180b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 shutdown_connections 2023-10-08T16:09:33.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.389+0000 7f991d405700 1 -- 172.21.15.170:0/3986985376 wait complete. 2023-10-08T16:09:33.391 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 41 2023-10-08T16:09:33.401 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:09:33.402 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:09:33.402 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:09:33.402 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:33.402 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:09:33.402 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:09:33.403 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:09:33.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:33.403 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:09:33.403 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:09:33.403 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:09:33.403 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:33.404 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:09:33.404 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:09:33.404 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:09:33.404 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:33.411 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:09:33.411 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:09:33.412 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:09:33.412 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:09:33.416 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:09:33.417 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:09:33.417 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:09:33.418 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:09:33.453 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:09:33.454 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:33.454 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:09:33.455 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:33.459 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:09:33.459 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:33.460 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:09:33.460 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:33.461 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:09:33.536 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:09:33.536 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:09:33.536 INFO:tasks.ceph.mds.a.smithi170.stderr:2023-10-08T16:09:33.536+0000 7f70b83b0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:09:33.541 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:09:33.541 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:09:33.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.583+0000 7f902934d700 1 Processor -- start 2023-10-08T16:09:33.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.583+0000 7f902934d700 1 -- start start 2023-10-08T16:09:33.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.583+0000 7f902934d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 0x7f902415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.583+0000 7f902934d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9024159c60 0x7f902415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f902934d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902415a160 con 0x7f902415db10 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f902934d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90240d13a0 con 0x7f9024159c60 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f902934d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902415e140 con 0x7f9024159890 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90237fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f902415db10 0x7f9024069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58170/0 (socket says 172.21.15.170:58170) 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90237fe700 1 -- 172.21.15.170:0/1335287814 learned_addr learned my addr 172.21.15.170:0/1335287814 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 --2- 172.21.15.170:0/1335287814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 0x7f902415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f901bfff700 1 --2- 172.21.15.170:0/1335287814 >> v2:172.21.15.191:3300/0 conn(0x7f9024159c60 0x7f902415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90227fc700 1 -- 172.21.15.170:0/1335287814 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652389468 0 0) 0x7f902415a160 con 0x7f902415db10 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90227fc700 1 -- 172.21.15.170:0/1335287814 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9010003570 con 0x7f902415db10 2023-10-08T16:09:33.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 -- 172.21.15.170:0/1335287814 >> v2:172.21.15.191:3300/0 conn(0x7f9024159c60 msgr2=0x7f902415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 --2- 172.21.15.170:0/1335287814 >> v2:172.21.15.191:3300/0 conn(0x7f9024159c60 0x7f902415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 -- 172.21.15.170:0/1335287814 >> v1:172.21.15.170:6789/0 conn(0x7f902415db10 legacy=0x7f9024069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 -- 172.21.15.170:0/1335287814 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902415a3d0 con 0x7f9024159890 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f9022ffd700 1 --2- 172.21.15.170:0/1335287814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 0x7f902415abd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90140098e0 tx=0x7f90140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ecf85c1196f0fb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90227fc700 1 -- 172.21.15.170:0/1335287814 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014011630 con 0x7f9024159890 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90227fc700 1 -- 172.21.15.170:0/1335287814 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9014011790 con 0x7f9024159890 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.584+0000 7f90227fc700 1 -- 172.21.15.170:0/1335287814 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9014011b30 con 0x7f9024159890 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 -- 172.21.15.170:0/1335287814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 msgr2=0x7f902415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 --2- 172.21.15.170:0/1335287814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 0x7f902415abd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f90140098e0 tx=0x7f90140091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 -- 172.21.15.170:0/1335287814 shutdown_connections 2023-10-08T16:09:33.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 --2- 172.21.15.170:0/1335287814 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9024159890 0x7f902415abd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 --2- 172.21.15.170:0/1335287814 >> v2:172.21.15.191:3300/0 conn(0x7f9024159c60 0x7f902415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 -- 172.21.15.170:0/1335287814 >> 172.21.15.170:0/1335287814 conn(0x7f9024068e20 msgr2=0x7f90240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:33.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 -- 172.21.15.170:0/1335287814 shutdown_connections 2023-10-08T16:09:33.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.585+0000 7f902934d700 1 -- 172.21.15.170:0/1335287814 wait complete. 2023-10-08T16:09:33.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.586+0000 7f902934d700 1 Processor -- start 2023-10-08T16:09:33.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.586+0000 7f902934d700 1 -- start start 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9024159890 0x7f902416abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f902415db10 0x7f90241698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902416b100 con 0x7f9024159c60 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902416b880 con 0x7f9024159890 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f902934d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90240cd680 con 0x7f902415db10 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52148/0 (socket says 172.21.15.170:52148) 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 -- 172.21.15.170:0/183703899 learned_addr learned my addr 172.21.15.170:0/183703899 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:33.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f90237fe700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f902415db10 0x7f90241698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f9022ffd700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9024159890 0x7f902416abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 -- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f902415db10 msgr2=0x7f90241698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f902415db10 0x7f90241698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 -- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9024159890 msgr2=0x7f902416abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9024159890 0x7f902416abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 -- 172.21.15.170:0/183703899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014009420 con 0x7f9024159c60 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901bfff700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f900c00ec50 tx=0x7f900c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.587+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00c850 con 0x7f9024159c60 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.588+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c00c9b0 con 0x7f9024159c60 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.588+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90240cd960 con 0x7f9024159c60 2023-10-08T16:09:33.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.588+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c00cd70 con 0x7f9024159c60 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.588+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90240cdf20 con 0x7f9024159c60 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.588+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902404c3c0 con 0x7f9024159c60 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.589+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f900c013680 con 0x7f9024159c60 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.589+0000 7f901affd700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 0x7f9004069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.589+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f900c0847f0 con 0x7f9024159c60 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.589+0000 7f9022ffd700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 0x7f9004069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.590+0000 7f9022ffd700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 0x7f9004069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90140091e0 tx=0x7f9014017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:33.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.594+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f900c051ad0 con 0x7f9024159c60 2023-10-08T16:09:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:33.794+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f90240cbfb0 con 0x7f9024159c60 2023-10-08T16:09:34.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.624+0000 7f901affd700 1 -- 172.21.15.170:0/183703899 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7f900c0550f0 con 0x7f9024159c60 2023-10-08T16:09:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 msgr2=0x7f9004069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 0x7f9004069620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f90140091e0 tx=0x7f9014017000 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 msgr2=0x7f902416b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f900c00ec50 tx=0x7f900c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 shutdown_connections 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f902415db10 0x7f90241698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9024159890 0x7f902416abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9004067170 0x7f9004069620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 --2- 172.21.15.170:0/183703899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9024159c60 0x7f902416b340 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 >> 172.21.15.170:0/183703899 conn(0x7f9024068e20 msgr2=0x7f9024153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 shutdown_connections 2023-10-08T16:09:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.627+0000 7f902934d700 1 -- 172.21.15.170:0/183703899 wait complete. 2023-10-08T16:09:34.629 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:09:34.640 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:09:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.755+0000 7f1d45887700 1 Processor -- start 2023-10-08T16:09:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.755+0000 7f1d45887700 1 -- start start 2023-10-08T16:09:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.755+0000 7f1d45887700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d40159890 0x7f1d4015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.755+0000 7f1d45887700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1d40159c60 0x7f1d4015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.755+0000 7f1d45887700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4015a160 con 0x7f1d4015db10 2023-10-08T16:09:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d45887700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4015df50 con 0x7f1d40159890 2023-10-08T16:09:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d45887700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400d13a0 con 0x7f1d40159c60 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d40159890 0x7f1d4015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d40159890 0x7f1d4015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52158/0 (socket says 172.21.15.170:52158) 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 -- 172.21.15.170:0/4209319127 learned_addr learned my addr 172.21.15.170:0/4209319127 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 -- 172.21.15.170:0/4209319127 >> v2:172.21.15.191:3300/0 conn(0x7f1d40159c60 msgr2=0x7f1d4015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 --2- 172.21.15.170:0/4209319127 >> v2:172.21.15.191:3300/0 conn(0x7f1d40159c60 0x7f1d4015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 -- 172.21.15.170:0/4209319127 >> v1:172.21.15.191:6789/0 conn(0x7f1d4015db10 legacy=0x7f1d40069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 -- 172.21.15.170:0/4209319127 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4015e140 con 0x7f1d40159890 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3effd700 1 --2- 172.21.15.170:0/4209319127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d4015abd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1d400b67f0 tx=0x7f1d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b4f24f5c5038ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3dffb700 1 -- 172.21.15.170:0/4209319127 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000b820 con 0x7f1d40159890 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3dffb700 1 -- 172.21.15.170:0/4209319127 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000b980 con 0x7f1d40159890 2023-10-08T16:09:34.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.756+0000 7f1d3dffb700 1 -- 172.21.15.170:0/4209319127 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000bd20 con 0x7f1d40159890 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 -- 172.21.15.170:0/4209319127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 msgr2=0x7f1d4015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 --2- 172.21.15.170:0/4209319127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d4015abd0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1d400b67f0 tx=0x7f1d30009250 comp rx=0 tx=0).stop 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 -- 172.21.15.170:0/4209319127 shutdown_connections 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 --2- 172.21.15.170:0/4209319127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d4015abd0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 --2- 172.21.15.170:0/4209319127 >> v2:172.21.15.191:3300/0 conn(0x7f1d40159c60 0x7f1d4015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 -- 172.21.15.170:0/4209319127 >> 172.21.15.170:0/4209319127 conn(0x7f1d40068e20 msgr2=0x7f1d400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 -- 172.21.15.170:0/4209319127 shutdown_connections 2023-10-08T16:09:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.757+0000 7f1d45887700 1 -- 172.21.15.170:0/4209319127 wait complete. 2023-10-08T16:09:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.758+0000 7f1d45887700 1 Processor -- start 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.758+0000 7f1d45887700 1 -- start start 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d40159c60 0x7f1d400ce140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 0x7f1d400b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cf7d0 con 0x7f1d40159890 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400cebc0 con 0x7f1d4015db10 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400ceea0 con 0x7f1d40159c60 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52164/0 (socket says 172.21.15.170:52164) 2023-10-08T16:09:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 -- 172.21.15.170:0/3621699664 learned_addr learned my addr 172.21.15.170:0/3621699664 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3e7fc700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d40159c60 0x7f1d400ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3f7fe700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 0x7f1d400b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 -- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d40159c60 msgr2=0x7f1d400ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d40159c60 0x7f1d400ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 -- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 msgr2=0x7f1d400b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 0x7f1d400b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 -- 172.21.15.170:0/3621699664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300094f0 con 0x7f1d40159890 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3f7fe700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 0x7f1d400b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d3effd700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1d30003080 tx=0x7f1d30016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30026480 con 0x7f1d40159890 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3001f070 con 0x7f1d40159890 2023-10-08T16:09:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30026830 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.759+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400cf180 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.760+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401478c0 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.760+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4004c3c0 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.761+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f1d3001a440 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.761+0000 7f1d26ffd700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 0x7f1d2c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.761+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1d30021070 con 0x7f1d40159890 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.761+0000 7f1d3e7fc700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 0x7f1d2c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:34.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.762+0000 7f1d3e7fc700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 0x7f1d2c069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d40069fc0 tx=0x7f1d340091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:34.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:34.765+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d30053f20 con 0x7f1d40159890 2023-10-08T16:09:35.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.024+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1d4006a5b0 con 0x7f1d40159890 2023-10-08T16:09:35.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.025+0000 7f1d26ffd700 1 -- 172.21.15.170:0/3621699664 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1d30057540 con 0x7f1d40159890 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:09:35.026 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:09:35.027 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.028 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.029 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:09:35.030 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.031 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.032 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.033 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:09:35.034 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:09:35.036 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.037 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:09:35.038 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:09:35.039 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.040 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.041 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:09:35.042 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:09:35.043 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 msgr2=0x7f1d2c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 0x7f1d2c069620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d40069fc0 tx=0x7f1d340091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 msgr2=0x7f1d40147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1d30003080 tx=0x7f1d30016000 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 shutdown_connections 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d4015db10 0x7f1d400b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d2c067170 0x7f1d2c069620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d40159890 0x7f1d40147380 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 --2- 172.21.15.170:0/3621699664 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d40159c60 0x7f1d400ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 >> 172.21.15.170:0/3621699664 conn(0x7f1d40068e20 msgr2=0x7f1d400b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 shutdown_connections 2023-10-08T16:09:35.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.027+0000 7f1d45887700 1 -- 172.21.15.170:0/3621699664 wait complete. 2023-10-08T16:09:35.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:35.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.161+0000 7fc9ec45f700 1 Processor -- start 2023-10-08T16:09:35.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.161+0000 7fc9ec45f700 1 -- start start 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ec45f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 0x7fc9e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ec45f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc9e4159c60 0x7fc9e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ec45f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e415a160 con 0x7fc9e415db10 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ec45f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e415df50 con 0x7fc9e4159c60 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ec45f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40d13a0 con 0x7fc9e4159890 2023-10-08T16:09:35.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9e99fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc9e4159c60 0x7fc9e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ea9fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc9e415db10 0x7fc9e4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58184/0 (socket says 172.21.15.170:58184) 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9e99fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc9e4159c60 0x7fc9e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52178/0 (socket says 172.21.15.170:52178) 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ea9fc700 1 -- 172.21.15.170:0/2718337086 learned_addr learned my addr 172.21.15.170:0/2718337086 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.162+0000 7fc9ea1fb700 1 --2- 172.21.15.170:0/2718337086 >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 0x7fc9e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e99fa700 1 -- 172.21.15.170:0/2718337086 >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 msgr2=0x7fc9e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e99fa700 1 --2- 172.21.15.170:0/2718337086 >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 0x7fc9e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e99fa700 1 -- 172.21.15.170:0/2718337086 >> v1:172.21.15.170:6789/0 conn(0x7fc9e415db10 legacy=0x7fc9e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e99fa700 1 -- 172.21.15.170:0/2718337086 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e415e140 con 0x7fc9e4159c60 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e91f9700 1 -- 172.21.15.170:0/2718337086 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465430511 0 0) 0x7fc9e415a160 con 0x7fc9e415db10 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ea1fb700 1 --2- 172.21.15.170:0/2718337086 >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 0x7fc9e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e99fa700 1 --2- 172.21.15.170:0/2718337086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e4159c60 0x7fc9e415d5d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40d1090 tx=0x7fc9d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46493795c89ce068 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e91f9700 1 -- 172.21.15.170:0/2718337086 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400b6b0 con 0x7fc9e4159c60 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e91f9700 1 -- 172.21.15.170:0/2718337086 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400b810 con 0x7fc9e4159c60 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9e91f9700 1 -- 172.21.15.170:0/2718337086 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400bbb0 con 0x7fc9e4159c60 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 -- 172.21.15.170:0/2718337086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e4159c60 msgr2=0x7fc9e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/2718337086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e4159c60 0x7fc9e415d5d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40d1090 tx=0x7fc9d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 -- 172.21.15.170:0/2718337086 shutdown_connections 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/2718337086 >> v2:172.21.15.191:3300/0 conn(0x7fc9e4159890 0x7fc9e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/2718337086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e4159c60 0x7fc9e415d5d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 -- 172.21.15.170:0/2718337086 >> 172.21.15.170:0/2718337086 conn(0x7fc9e4068e20 msgr2=0x7fc9e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.163+0000 7fc9ec45f700 1 -- 172.21.15.170:0/2718337086 shutdown_connections 2023-10-08T16:09:35.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.164+0000 7fc9ec45f700 1 -- 172.21.15.170:0/2718337086 wait complete. 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 Processor -- start 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 -- start start 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4159890 0x7fc9e40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 0x7fc9e40ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40cf9d0 con 0x7fc9e415db10 2023-10-08T16:09:35.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40cfc70 con 0x7fc9e4159890 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.165+0000 7fc9ec45f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40cde20 con 0x7fc9e4159c60 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52180/0 (socket says 172.21.15.170:52180) 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 -- 172.21.15.170:0/264705789 learned_addr learned my addr 172.21.15.170:0/264705789 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea1fb700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4159890 0x7fc9e40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9e99fa700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 0x7fc9e40ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 -- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 msgr2=0x7fc9e40ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 0x7fc9e40ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 -- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4159890 msgr2=0x7fc9e40b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4159890 0x7fc9e40b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 -- 172.21.15.170:0/264705789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4009420 con 0x7fc9e415db10 2023-10-08T16:09:35.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9e99fa700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 0x7fc9e40ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ea9fc700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c530 tx=0x7fc9dc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc005b30 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc005c90 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40ce100 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc007320 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.166+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e40ceb60 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.167+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40cbf20 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.167+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc9dc01b3b0 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.168+0000 7fc9da7fc700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 0x7fc9c8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.168+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc9dc083a60 con 0x7fc9e415db10 2023-10-08T16:09:35.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.168+0000 7fc9ea1fb700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 0x7fc9c8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.168+0000 7fc9ea1fb700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 0x7fc9c8069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00076a0 tx=0x7fc9e00098f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.172+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9dc050d90 con 0x7fc9e415db10 2023-10-08T16:09:35.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.358+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9e40cdc50 con 0x7fc9e415db10 2023-10-08T16:09:35.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.359+0000 7fc9da7fc700 1 -- 172.21.15.170:0/264705789 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12183 (secure 0 0 0) 0x7fc9dc017020 con 0x7fc9e415db10 2023-10-08T16:09:35.359 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:35.360 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":73,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:34.621149+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 msgr2=0x7fc9c8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 0x7fc9c8069620 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00076a0 tx=0x7fc9e00098f0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 msgr2=0x7fc9e416d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00c530 tx=0x7fc9dc007f90 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 shutdown_connections 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc9c8067170 0x7fc9c8069620 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9e415db10 0x7fc9e416d080 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc9e4159890 0x7fc9e40b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 --2- 172.21.15.170:0/264705789 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc9e4159c60 0x7fc9e40ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 >> 172.21.15.170:0/264705789 conn(0x7fc9e4068e20 msgr2=0x7fc9e4153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 shutdown_connections 2023-10-08T16:09:35.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.361+0000 7fc9ec45f700 1 -- 172.21.15.170:0/264705789 wait complete. 2023-10-08T16:09:35.373 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:09:35.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:09:35.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.490+0000 7fcaa6fc6700 1 Processor -- start 2023-10-08T16:09:35.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.490+0000 7fcaa6fc6700 1 -- start start 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa6fc6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fcaa015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa6fc6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa015a160 con 0x7fcaa0159890 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa6fc6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00d13a0 con 0x7fcaa015db10 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa6fc6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015e140 con 0x7fcaa0159c60 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa4d62700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcaa0159890 0x7fcaa015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46838/0 (socket says 172.21.15.170:46838) 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.491+0000 7fcaa4d62700 1 -- 172.21.15.170:0/65930491 learned_addr learned my addr 172.21.15.170:0/65930491 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca97fff700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fcaa015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118854338 0 0) 0x7fcaa015a160 con 0x7fcaa0159890 2023-10-08T16:09:35.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca88003570 con 0x7fcaa0159890 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915731497 0 0) 0x7fca88003570 con 0x7fcaa0159890 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 msgr2=0x7fcaa015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fcaa015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 >> v1:172.21.15.191:6789/0 conn(0x7fcaa015db10 legacy=0x7fcaa0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa015a3d0 con 0x7fcaa0159890 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fca9c0017f0 con 0x7fcaa0159890 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 >> v1:172.21.15.170:6789/0 conn(0x7fcaa0159890 legacy=0x7fcaa015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca88003a90 0x7fca88003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fca88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca88004d50 0x7fca880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015e140 con 0x7fca88004d50 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8800bb70 con 0x7fca88003a90 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8800bd30 con 0x7fcaa0159c60 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca88004d50 0x7fca880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa4d62700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca88003a90 0x7fca88003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fca97fff700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fca88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 -- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 msgr2=0x7fca88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:35.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa0159c60 0x7fca88004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 -- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca88003a90 msgr2=0x7fca88003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:35.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca88003a90 0x7fca88003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.492+0000 7fcaa5563700 1 -- 172.21.15.170:0/65930491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88003570 con 0x7fca88004d50 2023-10-08T16:09:35.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa5563700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca88004d50 0x7fca880051c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fca980018b0 tx=0x7fca98007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac548cbe9478e886 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012420 con 0x7fca88004d50 2023-10-08T16:09:35.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98012580 con 0x7fca88004d50 2023-10-08T16:09:35.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fca977fe700 1 -- 172.21.15.170:0/65930491 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98012920 con 0x7fca88004d50 2023-10-08T16:09:35.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/65930491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca88004d50 msgr2=0x7fca880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/65930491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca88004d50 0x7fca880051c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fca980018b0 tx=0x7fca98007d30 comp rx=0 tx=0).stop 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa5563700 1 -- 172.21.15.170:0/65930491 reap_dead start 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/65930491 shutdown_connections 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/65930491 >> 172.21.15.170:0/65930491 conn(0x7fcaa0068e20 msgr2=0x7fcaa00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/65930491 shutdown_connections 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.493+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/65930491 wait complete. 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.494+0000 7fcaa6fc6700 1 Processor -- start 2023-10-08T16:09:35.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.494+0000 7fcaa6fc6700 1 -- start start 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa016d120 0x7fcaa015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 0x7fcaa00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa015b9e0 con 0x7fcaa014c0a0 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa016d630 con 0x7fcaa015c3b0 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa6fc6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa016d910 con 0x7fcaa016d120 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa4d62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa4d62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47548/0 (socket says 172.21.15.170:47548) 2023-10-08T16:09:35.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa4d62700 1 -- 172.21.15.170:0/823175784 learned_addr learned my addr 172.21.15.170:0/823175784 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fca97fff700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa016d120 0x7fcaa015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa4d62700 1 -- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa016d120 msgr2=0x7fcaa015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.495+0000 7fcaa5563700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 0x7fcaa00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa4d62700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa016d120 0x7fcaa015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa4d62700 1 -- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 msgr2=0x7fcaa00cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa4d62700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 0x7fcaa00cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa4d62700 1 -- 172.21.15.170:0/823175784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa015b670 con 0x7fcaa014c0a0 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa5563700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 0x7fcaa00cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa4d62700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001920 tx=0x7fca9c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c013600 con 0x7fcaa014c0a0 2023-10-08T16:09:35.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c013780 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa0151400 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c013b40 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.496+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa014c630 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.497+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa016db50 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.497+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fca9c01c400 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.498+0000 7fca94ff9700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 0x7fca8c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.498+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fca9c085810 con 0x7fcaa014c0a0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.498+0000 7fca97fff700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 0x7fca8c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.498+0000 7fca97fff700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 0x7fca8c069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fca90006f00 tx=0x7fca90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.502+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcaa016db50 con 0x7fcaa014c0a0 2023-10-08T16:09:35.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.684+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaa015b4d0 con 0x7fcaa014c0a0 2023-10-08T16:09:35.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.684+0000 7fca94ff9700 1 -- 172.21.15.170:0/823175784 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcaa015b4d0 con 0x7fcaa014c0a0 2023-10-08T16:09:35.685 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 msgr2=0x7fca8c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 0x7fca8c069620 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fca90006f00 tx=0x7fca90008040 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 msgr2=0x7fcaa016cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001920 tx=0x7fca9c0021d0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 shutdown_connections 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcaa015c3b0 0x7fcaa00cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca8c067170 0x7fca8c069620 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaa014c0a0 0x7fcaa016cbe0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 --2- 172.21.15.170:0/823175784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcaa016d120 0x7fcaa015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 >> 172.21.15.170:0/823175784 conn(0x7fcaa0068e20 msgr2=0x7fcaa0153690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 shutdown_connections 2023-10-08T16:09:35.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.686+0000 7fcaa6fc6700 1 -- 172.21.15.170:0/823175784 wait complete. 2023-10-08T16:09:35.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:09:35.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.813+0000 7f8225ced700 1 Processor -- start 2023-10-08T16:09:35.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.813+0000 7f8225ced700 1 -- start start 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8225ced700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8220159420 0x7f82200b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8225ced700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f822015d230 0x7f822015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8225ced700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82200b2980 con 0x7f82201597f0 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8225ced700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822015db70 con 0x7f822015d230 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8225ced700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82200b11d0 con 0x7f8220159420 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f822015d230 0x7f822015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f822015d230 0x7f822015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47552/0 (socket says 172.21.15.170:47552) 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 -- 172.21.15.170:0/3351539909 learned_addr learned my addr 172.21.15.170:0/3351539909 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:35.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f8224ceb700 1 --2- 172.21.15.170:0/3351539909 >> v2:172.21.15.191:3300/0 conn(0x7f8220159420 0x7f82200b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 -- 172.21.15.170:0/3351539909 >> v2:172.21.15.191:3300/0 conn(0x7f8220159420 msgr2=0x7f82200b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 --2- 172.21.15.170:0/3351539909 >> v2:172.21.15.191:3300/0 conn(0x7f8220159420 0x7f82200b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 -- 172.21.15.170:0/3351539909 >> v1:172.21.15.191:6789/0 conn(0x7f82201597f0 legacy=0x7f82200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.814+0000 7f82254ec700 1 -- 172.21.15.170:0/3351539909 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200b14b0 con 0x7f822015d230 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f82254ec700 1 --2- 172.21.15.170:0/3351539909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f822015d230 0x7f822015d630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f82200b2f20 tx=0x7f8218009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120c1cf06a4fc32f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f82177fe700 1 -- 172.21.15.170:0/3351539909 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821800e070 con 0x7f822015d230 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f82177fe700 1 -- 172.21.15.170:0/3351539909 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821800ba20 con 0x7f822015d230 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f82177fe700 1 -- 172.21.15.170:0/3351539909 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180092e0 con 0x7f822015d230 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 -- 172.21.15.170:0/3351539909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f822015d230 msgr2=0x7f822015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 --2- 172.21.15.170:0/3351539909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f822015d230 0x7f822015d630 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f82200b2f20 tx=0x7f8218009840 comp rx=0 tx=0).stop 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 -- 172.21.15.170:0/3351539909 shutdown_connections 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 --2- 172.21.15.170:0/3351539909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f822015d230 0x7f822015d630 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 --2- 172.21.15.170:0/3351539909 >> v2:172.21.15.191:3300/0 conn(0x7f8220159420 0x7f82200b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 -- 172.21.15.170:0/3351539909 >> 172.21.15.170:0/3351539909 conn(0x7f82200d0660 msgr2=0x7f82200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 -- 172.21.15.170:0/3351539909 shutdown_connections 2023-10-08T16:09:35.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.815+0000 7f8225ced700 1 -- 172.21.15.170:0/3351539909 wait complete. 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.816+0000 7f8225ced700 1 Processor -- start 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 -- start start 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8220159420 0x7f8220155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f822015d230 0x7f82200c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c9740 con 0x7f82201597f0 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c5400 con 0x7f8220159420 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8225ced700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82200c5650 con 0x7f822015d230 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8217fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8217fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47558/0 (socket says 172.21.15.170:47558) 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8217fff700 1 -- 172.21.15.170:0/3575973377 learned_addr learned my addr 172.21.15.170:0/3575973377 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f8224ceb700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8220159420 0x7f8220155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.817+0000 7f82254ec700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f822015d230 0x7f82200c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 -- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f822015d230 msgr2=0x7f82200c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f822015d230 0x7f82200c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 -- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8220159420 msgr2=0x7f8220155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8220159420 0x7f8220155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 -- 172.21.15.170:0/3575973377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8218009ae0 con 0x7f82201597f0 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8217fff700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f8210007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210011590 con 0x7f82201597f0 2023-10-08T16:09:35.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82100116f0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8210011ab0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82200c5930 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.818+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82200c5ef0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.819+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822004c3c0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.819+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f821001b3b0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.820+0000 7f8214ff9700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 0x7f820c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.820+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f82100838e0 con 0x7f82201597f0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.820+0000 7f8224ceb700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 0x7f820c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:35.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.820+0000 7f8224ceb700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 0x7f820c069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f821c009db0 tx=0x7f821c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:35.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:35.824+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8210050c10 con 0x7f82201597f0 2023-10-08T16:09:36.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.010+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f82200b1790 con 0x7f82201597f0 2023-10-08T16:09:36.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.010+0000 7f8214ff9700 1 -- 172.21.15.170:0/3575973377 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8210017070 con 0x7f82201597f0 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:36.011 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 msgr2=0x7f820c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 0x7f820c069620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f821c009db0 tx=0x7f821c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 msgr2=0x7f82200c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f821000c7c0 tx=0x7f8210007f50 comp rx=0 tx=0).stop 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 shutdown_connections 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f822015d230 0x7f82200c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8220159420 0x7f8220155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f820c067170 0x7f820c069620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 --2- 172.21.15.170:0/3575973377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82201597f0 0x7f82200c66f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 >> 172.21.15.170:0/3575973377 conn(0x7f82200d0660 msgr2=0x7f82201586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:36.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 shutdown_connections 2023-10-08T16:09:36.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.012+0000 7f8225ced700 1 -- 172.21.15.170:0/3575973377 wait complete. 2023-10-08T16:09:36.025 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:09:36.025 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:09:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.144+0000 7fb7389aa700 1 Processor -- start 2023-10-08T16:09:36.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.144+0000 7fb7389aa700 1 -- start start 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb7389aa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb734159ec0 0x7fb7340b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb7389aa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7340cf350 0x7fb7340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb7389aa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340b5b00 con 0x7fb7340b53d0 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb7389aa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340b5850 con 0x7fb7340cf350 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb7389aa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340b5da0 con 0x7fb734159ec0 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7340cf350 0x7fb7340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb7340cf350 0x7fb7340cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47566/0 (socket says 172.21.15.170:47566) 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 -- 172.21.15.170:0/1413965547 learned_addr learned my addr 172.21.15.170:0/1413965547 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb73259c700 1 --2- 172.21.15.170:0/1413965547 >> v2:172.21.15.191:3300/0 conn(0x7fb734159ec0 0x7fb7340b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 -- 172.21.15.170:0/1413965547 >> v2:172.21.15.191:3300/0 conn(0x7fb734159ec0 msgr2=0x7fb7340b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 --2- 172.21.15.170:0/1413965547 >> v2:172.21.15.191:3300/0 conn(0x7fb734159ec0 0x7fb7340b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 -- 172.21.15.170:0/1413965547 >> v1:172.21.15.191:6789/0 conn(0x7fb7340b53d0 legacy=0x7fb7340cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.145+0000 7fb732d9d700 1 -- 172.21.15.170:0/1413965547 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340b6080 con 0x7fb7340cf350 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb732d9d700 1 --2- 172.21.15.170:0/1413965547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb7340cd010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb734089120 tx=0x7fb724009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccd5ec2cadd47b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb73159a700 1 -- 172.21.15.170:0/1413965547 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400e070 con 0x7fb7340cf350 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb73159a700 1 -- 172.21.15.170:0/1413965547 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400ba20 con 0x7fb7340cf350 2023-10-08T16:09:36.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb73159a700 1 -- 172.21.15.170:0/1413965547 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240092e0 con 0x7fb7340cf350 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 -- 172.21.15.170:0/1413965547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 msgr2=0x7fb7340cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 --2- 172.21.15.170:0/1413965547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb7340cd010 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb734089120 tx=0x7fb724009840 comp rx=0 tx=0).stop 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 -- 172.21.15.170:0/1413965547 shutdown_connections 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 --2- 172.21.15.170:0/1413965547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb7340cd010 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 --2- 172.21.15.170:0/1413965547 >> v2:172.21.15.191:3300/0 conn(0x7fb734159ec0 0x7fb7340b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 -- 172.21.15.170:0/1413965547 >> 172.21.15.170:0/1413965547 conn(0x7fb7340999f0 msgr2=0x7fb7340b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 -- 172.21.15.170:0/1413965547 shutdown_connections 2023-10-08T16:09:36.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.146+0000 7fb7389aa700 1 -- 172.21.15.170:0/1413965547 wait complete. 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.147+0000 7fb7389aa700 1 Processor -- start 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.147+0000 7fb7389aa700 1 -- start start 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7340b53d0 0x7fb7341477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb734159ec0 0x7fb7340c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734147330 con 0x7fb7340cf350 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb734146720 con 0x7fb7340b53d0 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb7389aa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb734146a00 con 0x7fb734159ec0 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb731d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb731d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47580/0 (socket says 172.21.15.170:47580) 2023-10-08T16:09:36.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb731d9b700 1 -- 172.21.15.170:0/434652322 learned_addr learned my addr 172.21.15.170:0/434652322 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb73259c700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7340b53d0 0x7fb7341477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.148+0000 7fb732d9d700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb734159ec0 0x7fb7340c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 -- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb734159ec0 msgr2=0x7fb7340c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb734159ec0 0x7fb7340c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 -- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7340b53d0 msgr2=0x7fb7341477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7340b53d0 0x7fb7341477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 -- 172.21.15.170:0/434652322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb724009ae0 con 0x7fb7340cf350 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb731d9b700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c4e0 tx=0x7fb71c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00cea0 con 0x7fb7340cf350 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c011590 con 0x7fb7340cf350 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb734146ce0 con 0x7fb7340cf350 2023-10-08T16:09:36.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c011930 con 0x7fb7340cf350 2023-10-08T16:09:36.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.149+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb734157000 con 0x7fb7340cf350 2023-10-08T16:09:36.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.150+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7341474e0 con 0x7fb7340cf350 2023-10-08T16:09:36.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.150+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fb71c011d20 con 0x7fb7340cf350 2023-10-08T16:09:36.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.151+0000 7fb72a7fc700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 0x7fb714069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.151+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb71c083810 con 0x7fb7340cf350 2023-10-08T16:09:36.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.151+0000 7fb73259c700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 0x7fb714069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.151+0000 7fb73259c700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 0x7fb714069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009da0 tx=0x7fb72c009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.155+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7341474e0 con 0x7fb7340cf350 2023-10-08T16:09:36.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.339+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb734063c90 con 0x7fb7340cf350 2023-10-08T16:09:36.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.630+0000 7fb72a7fc700 1 -- 172.21.15.170:0/434652322 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb71c017070 con 0x7fb7340cf350 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 msgr2=0x7fb714069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 0x7fb714069620 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009da0 tx=0x7fb72c009120 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 msgr2=0x7fb734156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00c4e0 tx=0x7fb71c00c7f0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 shutdown_connections 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb734159ec0 0x7fb7340c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb7340b53d0 0x7fb7341477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb714067170 0x7fb714069620 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 --2- 172.21.15.170:0/434652322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7340cf350 0x7fb734156ac0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.634+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 >> 172.21.15.170:0/434652322 conn(0x7fb7340999f0 msgr2=0x7fb7340cdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:36.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.635+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 shutdown_connections 2023-10-08T16:09:36.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.635+0000 7fb7389aa700 1 -- 172.21.15.170:0/434652322 wait complete. 2023-10-08T16:09:36.637 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:09:36.649 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:09:36.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.770+0000 7fe0338cc700 1 Processor -- start 2023-10-08T16:09:36.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.770+0000 7fe0338cc700 1 -- start start 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe0338cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe02c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe0338cc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c0ce380 con 0x7fe02c0af320 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe0338cc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c0cec20 con 0x7fe02c159950 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe0338cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0afa90 con 0x7fe02c069c70 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe031e69700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe02c0af320 0x7fe02c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46842/0 (socket says 172.21.15.170:46842) 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.771+0000 7fe031e69700 1 -- 172.21.15.170:0/1475648098 learned_addr learned my addr 172.21.15.170:0/1475648098 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:36.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe02c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640560242 0 0) 0x7fe02c0ce380 con 0x7fe02c0af320 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe018003570 con 0x7fe02c0af320 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1349600401 0 0) 0x7fe018003570 con 0x7fe02c0af320 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 msgr2=0x7fe02c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe02c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 >> v1:172.21.15.191:6789/0 conn(0x7fe02c159950 legacy=0x7fe02c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0ce5d0 con 0x7fe02c0af320 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fe0240017f0 con 0x7fe02c0af320 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 >> v1:172.21.15.170:6789/0 conn(0x7fe02c0af320 legacy=0x7fe02c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018003a90 0x7fe018003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe018004810 0x7fe018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe0180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0afa90 con 0x7fe018004810 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01800bb10 con 0x7fe018003a90 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01800bcd0 con 0x7fe02c069c70 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe018004810 0x7fe018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe031668700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018003a90 0x7fe018003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe031e69700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe0180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 -- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 msgr2=0x7fe0180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c069c70 0x7fe0180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 -- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018003a90 msgr2=0x7fe018003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018003a90 0x7fe018003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.772+0000 7fe030e67700 1 -- 172.21.15.170:0/1475648098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018003570 con 0x7fe018004810 2023-10-08T16:09:36.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe030e67700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe018004810 0x7fe018004c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009c00 tx=0x7fe01c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f9acdca5a21fb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00e710 con 0x7fe018004810 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00e870 con 0x7fe018004810 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe023fff700 1 -- 172.21.15.170:0/1475648098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00ec30 con 0x7fe018004810 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 -- 172.21.15.170:0/1475648098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe018004810 msgr2=0x7fe018004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 --2- 172.21.15.170:0/1475648098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe018004810 0x7fe018004c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009c00 tx=0x7fe01c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe031e69700 1 -- 172.21.15.170:0/1475648098 reap_dead start 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 -- 172.21.15.170:0/1475648098 shutdown_connections 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 -- 172.21.15.170:0/1475648098 >> 172.21.15.170:0/1475648098 conn(0x7fe02c068e00 msgr2=0x7fe02c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 -- 172.21.15.170:0/1475648098 shutdown_connections 2023-10-08T16:09:36.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.773+0000 7fe0338cc700 1 -- 172.21.15.170:0/1475648098 wait complete. 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.774+0000 7fe0338cc700 1 Processor -- start 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.774+0000 7fe0338cc700 1 -- start start 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018004810 0x7fe02c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c0c94c0 0x7fe02c0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c158b00 con 0x7fe02c158150 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c155f60 con 0x7fe018004810 2023-10-08T16:09:36.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe0338cc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c1545a0 con 0x7fe02c0c94c0 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe030e67700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe030e67700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47592/0 (socket says 172.21.15.170:47592) 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe030e67700 1 -- 172.21.15.170:0/2103942878 learned_addr learned my addr 172.21.15.170:0/2103942878 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.775+0000 7fe031668700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018004810 0x7fe02c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe031e69700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c0c94c0 0x7fe02c0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 -- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c0c94c0 msgr2=0x7fe02c0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c0c94c0 0x7fe02c0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 -- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018004810 msgr2=0x7fe02c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018004810 0x7fe02c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 -- 172.21.15.170:0/2103942878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0ca210 con 0x7fe02c158150 2023-10-08T16:09:36.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe030e67700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e4b0 tx=0x7fe01c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c023070 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00ec10 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c0ca4f0 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00e710 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.776+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c0c9e50 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.777+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02c158b00 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.777+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe01c013680 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.778+0000 7fe0217fa700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 0x7fe010069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.778+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe01c085e20 con 0x7fe02c158150 2023-10-08T16:09:36.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.778+0000 7fe031668700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 0x7fe010069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.778+0000 7fe031668700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 0x7fe010069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe028001050 tx=0x7fe028007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:36.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.782+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe01c0530a0 con 0x7fe02c158150 2023-10-08T16:09:36.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:36.964+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe02c0caa10 con 0x7fe02c158150 2023-10-08T16:09:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.636+0000 7fe0217fa700 1 -- 172.21.15.170:0/2103942878 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe02c0caa10 con 0x7fe02c158150 2023-10-08T16:09:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.641+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 msgr2=0x7fe010069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.641+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 0x7fe010069620 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe028001050 tx=0x7fe028007d30 comp rx=0 tx=0).stop 2023-10-08T16:09:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.641+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 msgr2=0x7fe02c1585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.641+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00e4b0 tx=0x7fe01c01a000 comp rx=0 tx=0).stop 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 shutdown_connections 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe02c0c94c0 0x7fe02c0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe018004810 0x7fe02c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe010067170 0x7fe010069620 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 --2- 172.21.15.170:0/2103942878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe02c158150 0x7fe02c1585c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.643+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 >> 172.21.15.170:0/2103942878 conn(0x7fe02c068e00 msgr2=0x7fe02c0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.647+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 shutdown_connections 2023-10-08T16:09:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.650+0000 7fe0338cc700 1 -- 172.21.15.170:0/2103942878 wait complete. 2023-10-08T16:09:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:09:37.665 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:09:37.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.783+0000 7fdcd5daa700 1 Processor -- start 2023-10-08T16:09:37.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.783+0000 7fdcd5daa700 1 -- start start 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 0x7fdcd00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdcd00b16d0 0x7fdcd00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 0x7fdcd015d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd00b2080 con 0x7fdcd00b16d0 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd00cfb00 con 0x7fdcd00b4e70 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd5daa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd00b5c40 con 0x7fdcd015d240 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdcd00b16d0 0x7fdcd00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdcd00b16d0 0x7fdcd00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47600/0 (socket says 172.21.15.170:47600) 2023-10-08T16:09:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 -- 172.21.15.170:0/3670979667 learned_addr learned my addr 172.21.15.170:0/3670979667 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/3670979667 >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 0x7fdcd00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd55a9700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 0x7fdcd015d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 -- 172.21.15.170:0/3670979667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 msgr2=0x7fdcd015d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 0x7fdcd015d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 -- 172.21.15.170:0/3670979667 >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 msgr2=0x7fdcd00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 --2- 172.21.15.170:0/3670979667 >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 0x7fdcd00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 -- 172.21.15.170:0/3670979667 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd00b5f20 con 0x7fdcd00b16d0 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/3670979667 >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 0x7fdcd00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdcd55a9700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 0x7fdcd015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccffff700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd00b16d0 0x7fdcd00b1b40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0064910 tx=0x7fdcb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e92a826c272ac6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.784+0000 7fdccf7fe700 1 -- 172.21.15.170:0/3670979667 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800b860 con 0x7fdcd00b16d0 2023-10-08T16:09:37.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdccf7fe700 1 -- 172.21.15.170:0/3670979667 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb800b9c0 con 0x7fdcd00b16d0 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdccf7fe700 1 -- 172.21.15.170:0/3670979667 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb800bd60 con 0x7fdcd00b16d0 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 -- 172.21.15.170:0/3670979667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd00b16d0 msgr2=0x7fdcd00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd00b16d0 0x7fdcd00b1b40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0064910 tx=0x7fdcb8009250 comp rx=0 tx=0).stop 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 -- 172.21.15.170:0/3670979667 shutdown_connections 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd015d240 0x7fdcd015d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/3670979667 >> v2:172.21.15.191:3300/0 conn(0x7fdcd00b4e70 0x7fdcd00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/3670979667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd00b16d0 0x7fdcd00b1b40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 -- 172.21.15.170:0/3670979667 >> 172.21.15.170:0/3670979667 conn(0x7fdcd00d10c0 msgr2=0x7fdcd00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 -- 172.21.15.170:0/3670979667 shutdown_connections 2023-10-08T16:09:37.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.785+0000 7fdcd5daa700 1 -- 172.21.15.170:0/3670979667 wait complete. 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.786+0000 7fdcd5daa700 1 Processor -- start 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.786+0000 7fdcd5daa700 1 -- start start 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 0x7fdcd01459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 0x7fdcd00cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd014c6e0 con 0x7fdcd015d240 2023-10-08T16:09:37.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0145f20 con 0x7fdcd00b16d0 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd5daa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd0147b40 con 0x7fdcd00b4e70 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47602/0 (socket says 172.21.15.170:47602) 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 -- 172.21.15.170:0/4268525157 learned_addr learned my addr 172.21.15.170:0/4268525157 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 0x7fdcd01459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdccffff700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 0x7fdcd00cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 -- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 msgr2=0x7fdcd00cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 0x7fdcd00cba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 -- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 msgr2=0x7fdcd01459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.787+0000 7fdcd55a9700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 0x7fdcd01459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcd55a9700 1 -- 172.21.15.170:0/4268525157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80094f0 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 0x7fdcd01459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdccffff700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 0x7fdcd00cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcd55a9700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000e8a0 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0013410 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc0013570 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd0149680 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc00138f0 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.788+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd0147290 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.789+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd0147cd0 con 0x7fdcd015d240 2023-10-08T16:09:37.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.789+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fdcc0013ce0 con 0x7fdcd015d240 2023-10-08T16:09:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.790+0000 7fdcccff9700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 0x7fdcb0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.790+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fdcc0084700 con 0x7fdcd015d240 2023-10-08T16:09:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.790+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 0x7fdcb0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.790+0000 7fdcd4da8700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 0x7fdcb0069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80076e0 tx=0x7fdcc80098f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:37.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:37.794+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcc00550c0 con 0x7fdcd015d240 2023-10-08T16:09:38.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.019+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdcd0069b90 con 0x7fdcd015d240 2023-10-08T16:09:38.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.640+0000 7fdcccff9700 1 -- 172.21.15.170:0/4268525157 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdcc0054c50 con 0x7fdcd015d240 2023-10-08T16:09:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.643+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 msgr2=0x7fdcb0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.643+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 0x7fdcb0069620 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80076e0 tx=0x7fdcc80098f0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 msgr2=0x7fdcd0158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000e8a0 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2023-10-08T16:09:38.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 shutdown_connections 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdcb0067170 0x7fdcb0069620 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80076e0 tx=0x7fdcc80098f0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdcd015d240 0x7fdcd0158cb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdcd00b16d0 0x7fdcd01459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 --2- 172.21.15.170:0/4268525157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdcd00b4e70 0x7fdcd00cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 >> 172.21.15.170:0/4268525157 conn(0x7fdcd00d10c0 msgr2=0x7fdcd0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 shutdown_connections 2023-10-08T16:09:38.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.644+0000 7fdcd5daa700 1 -- 172.21.15.170:0/4268525157 wait complete. 2023-10-08T16:09:38.649 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 12 and data pool 13 2023-10-08T16:09:38.659 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.773+0000 7f00533bb700 1 Processor -- start 2023-10-08T16:09:38.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.773+0000 7f00533bb700 1 -- start start 2023-10-08T16:09:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f00533bb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f004c06c0a0 0x7f004c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f00533bb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f004c15df50 0x7f004c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f00533bb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c15d7e0 con 0x7f004c06bcd0 2023-10-08T16:09:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f00533bb700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c15dac0 con 0x7f004c06c0a0 2023-10-08T16:09:38.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f00533bb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c15aa30 con 0x7f004c15df50 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f0050956700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f004c06c0a0 0x7f004c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f0050956700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f004c06c0a0 0x7f004c0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47612/0 (socket says 172.21.15.170:47612) 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.774+0000 7f0050956700 1 -- 172.21.15.170:0/698352916 learned_addr learned my addr 172.21.15.170:0/698352916 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0051958700 1 --2- 172.21.15.170:0/698352916 >> v2:172.21.15.191:3300/0 conn(0x7f004c15df50 0x7f004c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0050956700 1 -- 172.21.15.170:0/698352916 >> v2:172.21.15.191:3300/0 conn(0x7f004c15df50 msgr2=0x7f004c15e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0050956700 1 --2- 172.21.15.170:0/698352916 >> v2:172.21.15.191:3300/0 conn(0x7f004c15df50 0x7f004c15e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0050956700 1 -- 172.21.15.170:0/698352916 >> v1:172.21.15.191:6789/0 conn(0x7f004c06bcd0 legacy=0x7f004c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0050956700 1 -- 172.21.15.170:0/698352916 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c15ad10 con 0x7f004c06c0a0 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0050956700 1 --2- 172.21.15.170:0/698352916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06c0a0 0x7f004c0caa90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f004c15d720 tx=0x7f003c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b63f39210c7d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:38.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0043fff700 1 -- 172.21.15.170:0/698352916 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00b860 con 0x7f004c06c0a0 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0043fff700 1 -- 172.21.15.170:0/698352916 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c00b9c0 con 0x7f004c06c0a0 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.775+0000 7f0043fff700 1 -- 172.21.15.170:0/698352916 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c00bd80 con 0x7f004c06c0a0 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 -- 172.21.15.170:0/698352916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06c0a0 msgr2=0x7f004c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 --2- 172.21.15.170:0/698352916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06c0a0 0x7f004c0caa90 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f004c15d720 tx=0x7f003c009250 comp rx=0 tx=0).stop 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 -- 172.21.15.170:0/698352916 shutdown_connections 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 --2- 172.21.15.170:0/698352916 >> v2:172.21.15.191:3300/0 conn(0x7f004c15df50 0x7f004c15e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 --2- 172.21.15.170:0/698352916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06c0a0 0x7f004c0caa90 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 -- 172.21.15.170:0/698352916 >> 172.21.15.170:0/698352916 conn(0x7f004c0ce600 msgr2=0x7f004c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 -- 172.21.15.170:0/698352916 shutdown_connections 2023-10-08T16:09:38.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.776+0000 7f00533bb700 1 -- 172.21.15.170:0/698352916 wait complete. 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.777+0000 7f00533bb700 1 Processor -- start 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.777+0000 7f00533bb700 1 -- start start 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004c06c0a0 0x7f004c0c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f004c15df50 0x7f004c0c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c16b170 con 0x7f004c06bcd0 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0c83b0 con 0x7f004c06c0a0 2023-10-08T16:09:38.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c16dfc0 con 0x7f004c15df50 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47622/0 (socket says 172.21.15.170:47622) 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 -- 172.21.15.170:0/1120012505 learned_addr learned my addr 172.21.15.170:0/1120012505 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051958700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f004c15df50 0x7f004c0c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0050956700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004c06c0a0 0x7f004c0c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 -- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f004c15df50 msgr2=0x7f004c0c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f004c15df50 0x7f004c0c38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 -- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004c06c0a0 msgr2=0x7f004c0c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004c06c0a0 0x7f004c0c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 -- 172.21.15.170:0/1120012505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0094f0 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f0051157700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f004800c4e0 tx=0x7f004800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048011590 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0048011710 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c16ae90 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048011ad0 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.778+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c16dc50 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.779+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c16d7d0 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.780+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f004801b3b0 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.780+0000 7f00417fa700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 0x7f0030069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.780+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0048083c40 con 0x7f004c06bcd0 2023-10-08T16:09:38.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.780+0000 7f0050956700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 0x7f0030069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:38.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.781+0000 7f0050956700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 0x7f0030069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f004c15d720 tx=0x7f003c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:38.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.784+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f004c16d7d0 con 0x7f004c06bcd0 2023-10-08T16:09:38.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.968+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f004c16d7d0 con 0x7f004c06bcd0 2023-10-08T16:09:38.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.969+0000 7f00417fa700 1 -- 172.21.15.170:0/1120012505 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f004c16d7d0 con 0x7f004c06bcd0 2023-10-08T16:09:38.970 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:38.970 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":76,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:38.020706+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.971+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 msgr2=0x7f0030069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.971+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 0x7f0030069620 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f004c15d720 tx=0x7f003c017000 comp rx=0 tx=0).stop 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.971+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 msgr2=0x7f004c0b3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.971+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f004800c4e0 tx=0x7f004800c7f0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 shutdown_connections 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f004c15df50 0x7f004c0c38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0030067170 0x7f0030069620 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f004c06bcd0 0x7f004c0b3c40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 --2- 172.21.15.170:0/1120012505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f004c06c0a0 0x7f004c0c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:38.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 >> 172.21.15.170:0/1120012505 conn(0x7f004c0ce600 msgr2=0x7f004c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:38.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 shutdown_connections 2023-10-08T16:09:38.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:38.972+0000 7f00533bb700 1 -- 172.21.15.170:0/1120012505 wait complete. 2023-10-08T16:09:38.984 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:39.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.098+0000 7fc0a20e7700 1 Processor -- start 2023-10-08T16:09:39.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.098+0000 7fc0a20e7700 1 -- start start 2023-10-08T16:09:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc0a20e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc0a20e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc09c069560 0x7fc09c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc0a20e7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c15a160 con 0x7fc09c159c60 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc0a20e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c069f20 con 0x7fc09c069560 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc0a20e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c0d13a0 con 0x7fc09c159890 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc09bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc09c069560 0x7fc09c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc09affd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc09c159c60 0x7fc09c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46854/0 (socket says 172.21.15.170:46854) 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.099+0000 7fc09affd700 1 -- 172.21.15.170:0/4082629046 learned_addr learned my addr 172.21.15.170:0/4082629046 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09b7fe700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09bfff700 1 -- 172.21.15.170:0/4082629046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 msgr2=0x7fc09c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09bfff700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09bfff700 1 -- 172.21.15.170:0/4082629046 >> v1:172.21.15.170:6789/0 conn(0x7fc09c159c60 legacy=0x7fc09c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09a7fc700 1 -- 172.21.15.170:0/4082629046 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673629928 0 0) 0x7fc09c15a160 con 0x7fc09c159c60 2023-10-08T16:09:39.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09bfff700 1 -- 172.21.15.170:0/4082629046 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c15e140 con 0x7fc09c069560 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09b7fe700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09bfff700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c069560 0x7fc09c0699e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098e0 tx=0x7fc0940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed59960e73b2914a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09a7fc700 1 -- 172.21.15.170:0/4082629046 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094009e80 con 0x7fc09c069560 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09a7fc700 1 -- 172.21.15.170:0/4082629046 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09400b860 con 0x7fc09c069560 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc09a7fc700 1 -- 172.21.15.170:0/4082629046 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09400bbe0 con 0x7fc09c069560 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc0a20e7700 1 -- 172.21.15.170:0/4082629046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c069560 msgr2=0x7fc09c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.100+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c069560 0x7fc09c0699e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc0940098e0 tx=0x7fc0940091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 -- 172.21.15.170:0/4082629046 shutdown_connections 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/4082629046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c069560 0x7fc09c0699e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 -- 172.21.15.170:0/4082629046 >> 172.21.15.170:0/4082629046 conn(0x7fc09c068e20 msgr2=0x7fc09c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:39.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 -- 172.21.15.170:0/4082629046 shutdown_connections 2023-10-08T16:09:39.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.101+0000 7fc0a20e7700 1 -- 172.21.15.170:0/4082629046 wait complete. 2023-10-08T16:09:39.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 Processor -- start 2023-10-08T16:09:39.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 -- start start 2023-10-08T16:09:39.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc09c069560 0x7fc09c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c15ce20 con 0x7fc09c159c60 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c15c210 con 0x7fc09c069560 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.102+0000 7fc0a20e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c15c4f0 con 0x7fc09c159890 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47634/0 (socket says 172.21.15.170:47634) 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 -- 172.21.15.170:0/846049695 learned_addr learned my addr 172.21.15.170:0/846049695 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09affd700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09b7fe700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc09c069560 0x7fc09c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 -- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 msgr2=0x7fc09c169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 -- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc09c069560 msgr2=0x7fc09c148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc09c069560 0x7fc09c148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 -- 172.21.15.170:0/846049695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094009420 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc09bfff700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b3c0 tx=0x7fc094017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094011420 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc094011580 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc094011940 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c15c7d0 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.103+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c1486a0 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.104+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c15cf90 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.105+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fc0940253f0 con 0x7fc09c159c60 2023-10-08T16:09:39.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.105+0000 7fc0837fe700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 0x7fc088069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.105+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc094086520 con 0x7fc09c159c60 2023-10-08T16:09:39.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.105+0000 7fc09b7fe700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 0x7fc088069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.105+0000 7fc09b7fe700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 0x7fc088069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00d6e0 tx=0x7fc08c007850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.109+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc094053500 con 0x7fc09c159c60 2023-10-08T16:09:39.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.292+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc09c04c3c0 con 0x7fc09c159c60 2023-10-08T16:09:39.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.293+0000 7fc0837fe700 1 -- 172.21.15.170:0/846049695 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+15414 (secure 0 0 0) 0x7fc094084e60 con 0x7fc09c159c60 2023-10-08T16:09:39.293 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:39.294 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":76,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:38.020706+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 msgr2=0x7fc088069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 0x7fc088069620 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00d6e0 tx=0x7fc08c007850 comp rx=0 tx=0).stop 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 msgr2=0x7fc09c0b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc09400b3c0 tx=0x7fc094017000 comp rx=0 tx=0).stop 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 shutdown_connections 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc088067170 0x7fc088069620 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc09c159c60 0x7fc09c0b5d30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc09c069560 0x7fc09c148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 --2- 172.21.15.170:0/846049695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc09c159890 0x7fc09c169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 >> 172.21.15.170:0/846049695 conn(0x7fc09c068e20 msgr2=0x7fc09c157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 shutdown_connections 2023-10-08T16:09:39.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.295+0000 7fc0a20e7700 1 -- 172.21.15.170:0/846049695 wait complete. 2023-10-08T16:09:39.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:09:39.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.424+0000 7f206cb18700 1 Processor -- start 2023-10-08T16:09:39.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.424+0000 7f206cb18700 1 -- start start 2023-10-08T16:09:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 0x7f206815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159c60 0x7f206815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f206815db10 0x7f2068069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206815a160 con 0x7f206815db10 2023-10-08T16:09:39.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20680d13a0 con 0x7f2068159890 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206cb18700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206815e140 con 0x7f2068159c60 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f206815db10 0x7f2068069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f206815db10 0x7f2068069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47638/0 (socket says 172.21.15.170:47638) 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 -- 172.21.15.170:0/3275342253 learned_addr learned my addr 172.21.15.170:0/3275342253 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f205dd9b700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159c60 0x7f206815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 -- 172.21.15.170:0/3275342253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159c60 msgr2=0x7f206815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159c60 0x7f206815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f206659c700 1 --2- 172.21.15.170:0/3275342253 >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 0x7f206815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 -- 172.21.15.170:0/3275342253 >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 msgr2=0x7f206815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.425+0000 7f2066d9d700 1 --2- 172.21.15.170:0/3275342253 >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 0x7f206815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f2066d9d700 1 -- 172.21.15.170:0/3275342253 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206815a3d0 con 0x7f206815db10 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206659c700 1 --2- 172.21.15.170:0/3275342253 >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 0x7f206815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f2066d9d700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f206815db10 0x7f2068069560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f20680d1090 tx=0x7f2060009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec0a11a3629203 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f2065d9b700 1 -- 172.21.15.170:0/3275342253 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206000e070 con 0x7f206815db10 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f2065d9b700 1 -- 172.21.15.170:0/3275342253 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206000ba50 con 0x7f206815db10 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f2065d9b700 1 -- 172.21.15.170:0/3275342253 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20600092e0 con 0x7f206815db10 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 -- 172.21.15.170:0/3275342253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f206815db10 msgr2=0x7f2068069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f206815db10 0x7f2068069560 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f20680d1090 tx=0x7f2060009840 comp rx=0 tx=0).stop 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 -- 172.21.15.170:0/3275342253 shutdown_connections 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f206815db10 0x7f2068069560 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 --2- 172.21.15.170:0/3275342253 >> v2:172.21.15.191:3300/0 conn(0x7f2068159890 0x7f206815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 --2- 172.21.15.170:0/3275342253 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159c60 0x7f206815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 -- 172.21.15.170:0/3275342253 >> 172.21.15.170:0/3275342253 conn(0x7f2068068e20 msgr2=0x7f20680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 -- 172.21.15.170:0/3275342253 shutdown_connections 2023-10-08T16:09:39.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.426+0000 7f206cb18700 1 -- 172.21.15.170:0/3275342253 wait complete. 2023-10-08T16:09:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 Processor -- start 2023-10-08T16:09:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 -- start start 2023-10-08T16:09:39.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159890 0x7f206816ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 0x7f206814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2068152320 con 0x7f2068159c60 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206816b180 con 0x7f206815db10 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206cb18700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2068169b60 con 0x7f2068159890 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f205dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f205dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47644/0 (socket says 172.21.15.170:47644) 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f205dd9b700 1 -- 172.21.15.170:0/2764640128 learned_addr learned my addr 172.21.15.170:0/2764640128 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f2066d9d700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 0x7f206814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.428+0000 7f206659c700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159890 0x7f206816ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 -- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159890 msgr2=0x7f206816ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159890 0x7f206816ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 -- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 msgr2=0x7f206814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 0x7f206814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 -- 172.21.15.170:0/2764640128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060009ae0 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f2066d9d700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 0x7f206814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205dd9b700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f205000c880 tx=0x7f2050008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011600 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050011780 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011b40 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2068169de0 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.429+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206816a3a0 con 0x7f2068159c60 2023-10-08T16:09:39.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.430+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206804c3c0 con 0x7f2068159c60 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.430+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2050010350 con 0x7f2068159c60 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.430+0000 7f205effd700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 0x7f2048069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.431+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2050084710 con 0x7f2068159c60 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.431+0000 7f206659c700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 0x7f2048069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.431+0000 7f206659c700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 0x7f2048069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2068155c00 tx=0x7f2058006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:39.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.435+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f20500515e0 con 0x7f2068159c60 2023-10-08T16:09:39.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:39.668+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f206806a5b0 con 0x7f2068159c60 2023-10-08T16:09:40.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.651+0000 7f205effd700 1 -- 172.21.15.170:0/2764640128 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2050054c00 con 0x7f2068159c60 2023-10-08T16:09:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 msgr2=0x7f2048069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 0x7f2048069620 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2068155c00 tx=0x7f2058006f90 comp rx=0 tx=0).stop 2023-10-08T16:09:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 msgr2=0x7f206816c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f205000c880 tx=0x7f2050008040 comp rx=0 tx=0).stop 2023-10-08T16:09:40.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 shutdown_connections 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f206815db10 0x7f206814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2068159890 0x7f206816ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2048067170 0x7f2048069620 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 --2- 172.21.15.170:0/2764640128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2068159c60 0x7f206816c940 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 >> 172.21.15.170:0/2764640128 conn(0x7f2068068e20 msgr2=0x7f206815b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 shutdown_connections 2023-10-08T16:09:40.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.653+0000 7f206cb18700 1 -- 172.21.15.170:0/2764640128 wait complete. 2023-10-08T16:09:40.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:09:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.785+0000 7f0558915700 1 Processor -- start 2023-10-08T16:09:40.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.785+0000 7f0558915700 1 -- start start 2023-10-08T16:09:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0558915700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0554159c60 0x7f055415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0558915700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f0554069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0558915700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055415a160 con 0x7f0554159890 2023-10-08T16:09:40.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0558915700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055415df50 con 0x7f0554159c60 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0558915700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540d13a0 con 0x7f055415db10 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f055259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0554159890 0x7f055415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46858/0 (socket says 172.21.15.170:46858) 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f055259c700 1 -- 172.21.15.170:0/2767074359 learned_addr learned my addr 172.21.15.170:0/2767074359 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0549d9b700 1 --2- 172.21.15.170:0/2767074359 >> v2:172.21.15.170:3300/0 conn(0x7f0554159c60 0x7f055415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0551d9b700 1 -- 172.21.15.170:0/2767074359 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261608693 0 0) 0x7f055415a160 con 0x7f0554159890 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0551d9b700 1 -- 172.21.15.170:0/2767074359 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0540003570 con 0x7f0554159890 2023-10-08T16:09:40.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0552d9d700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f0554069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.786+0000 7f0549d9b700 1 -- 172.21.15.170:0/2767074359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 msgr2=0x7f0554069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0549d9b700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f0554069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0549d9b700 1 -- 172.21.15.170:0/2767074359 >> v1:172.21.15.170:6789/0 conn(0x7f0554159890 legacy=0x7f055415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0549d9b700 1 -- 172.21.15.170:0/2767074359 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055415e140 con 0x7f0554159c60 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0549d9b700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159c60 0x7f055415d5d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f053c0098e0 tx=0x7f053c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a190ece5dd45ccd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0551d9b700 1 -- 172.21.15.170:0/2767074359 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00b640 con 0x7f0554159c60 2023-10-08T16:09:40.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0551d9b700 1 -- 172.21.15.170:0/2767074359 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00b7a0 con 0x7f0554159c60 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0551d9b700 1 -- 172.21.15.170:0/2767074359 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00bb20 con 0x7f0554159c60 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 -- 172.21.15.170:0/2767074359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159c60 msgr2=0x7f055415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159c60 0x7f055415d5d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f053c0098e0 tx=0x7f053c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 -- 172.21.15.170:0/2767074359 shutdown_connections 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f0554069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 --2- 172.21.15.170:0/2767074359 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159c60 0x7f055415d5d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 -- 172.21.15.170:0/2767074359 >> 172.21.15.170:0/2767074359 conn(0x7f0554068e20 msgr2=0x7f05540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 -- 172.21.15.170:0/2767074359 shutdown_connections 2023-10-08T16:09:40.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.787+0000 7f0558915700 1 -- 172.21.15.170:0/2767074359 wait complete. 2023-10-08T16:09:40.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 Processor -- start 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 -- start start 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0554159c60 0x7f05540cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f05540ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055414d0a0 con 0x7f0554159890 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0554151dd0 con 0x7f0554159c60 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f0558915700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05540b24e0 con 0x7f055415db10 2023-10-08T16:09:40.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.789+0000 7f055259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47666/0 (socket says 172.21.15.170:47666) 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 -- 172.21.15.170:0/3274842699 learned_addr learned my addr 172.21.15.170:0/3274842699 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 -- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 msgr2=0x7f05540ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f05540ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 -- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0554159c60 msgr2=0x7f05540cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0554159c60 0x7f05540cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 -- 172.21.15.170:0/3274842699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c009420 con 0x7f0554159890 2023-10-08T16:09:40.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f055259c700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f05440011e0 tx=0x7f0544007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544005b30 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544005c90 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541514a0 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440072e0 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.790+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0554151a60 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.791+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f055404c3c0 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.791+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0544007650 con 0x7f0554159890 2023-10-08T16:09:40.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.792+0000 7f054affd700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 0x7f0534069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:40.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.792+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0544083db0 con 0x7f0554159890 2023-10-08T16:09:40.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.792+0000 7f0549d9b700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 0x7f0534069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:40.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.792+0000 7f0549d9b700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 0x7f0534069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f055414d810 tx=0x7f053c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:40.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:40.796+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f054404b5a0 con 0x7f0554159890 2023-10-08T16:09:41.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.025+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f05540d0d40 con 0x7f0554159890 2023-10-08T16:09:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.652+0000 7f054affd700 1 -- 172.21.15.170:0/3274842699 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0544017020 con 0x7f0554159890 2023-10-08T16:09:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 msgr2=0x7f0534069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 0x7f0534069620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f055414d810 tx=0x7f053c017000 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 msgr2=0x7f055414cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f05440011e0 tx=0x7f0544007df0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 shutdown_connections 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f055415db10 0x7f05540ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0534067170 0x7f0534069620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0554159890 0x7f055414cb60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 --2- 172.21.15.170:0/3274842699 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0554159c60 0x7f05540cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 >> 172.21.15.170:0/3274842699 conn(0x7f0554068e20 msgr2=0x7f0554158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 shutdown_connections 2023-10-08T16:09:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.654+0000 7f0558915700 1 -- 172.21.15.170:0/3274842699 wait complete. 2023-10-08T16:09:41.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:41.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.782+0000 7f96f7528700 1 Processor -- start 2023-10-08T16:09:41.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.782+0000 7f96f7528700 1 -- start start 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96f0159420 0x7f96f00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96f015d230 0x7f96f015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f015dbd0 con 0x7f96f0159420 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00b11d0 con 0x7f96f015d230 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f7528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00b14b0 con 0x7f96f01597f0 2023-10-08T16:09:41.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96f0159420 0x7f96f00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96f0159420 0x7f96f00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47678/0 (socket says 172.21.15.170:47678) 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 -- 172.21.15.170:0/1609118061 learned_addr learned my addr 172.21.15.170:0/1609118061 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f5d25700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 -- 172.21.15.170:0/1609118061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 msgr2=0x7f96f00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6d27700 1 --2- 172.21.15.170:0/1609118061 >> v2:172.21.15.191:3300/0 conn(0x7f96f015d230 0x7f96f015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 -- 172.21.15.170:0/1609118061 >> v2:172.21.15.191:3300/0 conn(0x7f96f015d230 msgr2=0x7f96f015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 --2- 172.21.15.170:0/1609118061 >> v2:172.21.15.191:3300/0 conn(0x7f96f015d230 0x7f96f015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f6526700 1 -- 172.21.15.170:0/1609118061 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00b1790 con 0x7f96f0159420 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.783+0000 7f96f5d25700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f6526700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f00b5f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f96f00d1720 tx=0x7f96ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde2eb1281e53ad0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f5524700 1 -- 172.21.15.170:0/1609118061 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00b860 con 0x7f96f0159420 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f5524700 1 -- 172.21.15.170:0/1609118061 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec00b9c0 con 0x7f96f0159420 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f5524700 1 -- 172.21.15.170:0/1609118061 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00bd60 con 0x7f96f0159420 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 -- 172.21.15.170:0/1609118061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 msgr2=0x7f96f00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f00b5f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f96f00d1720 tx=0x7f96ec009250 comp rx=0 tx=0).stop 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 -- 172.21.15.170:0/1609118061 shutdown_connections 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 --2- 172.21.15.170:0/1609118061 >> v2:172.21.15.191:3300/0 conn(0x7f96f015d230 0x7f96f015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f00b5f40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 --2- 172.21.15.170:0/1609118061 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 -- 172.21.15.170:0/1609118061 >> 172.21.15.170:0/1609118061 conn(0x7f96f00d0660 msgr2=0x7f96f00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:41.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 -- 172.21.15.170:0/1609118061 shutdown_connections 2023-10-08T16:09:41.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.784+0000 7f96f7528700 1 -- 172.21.15.170:0/1609118061 wait complete. 2023-10-08T16:09:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.785+0000 7f96f7528700 1 Processor -- start 2023-10-08T16:09:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.785+0000 7f96f7528700 1 -- start start 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f0169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 0x7f96f01521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f01415d0 con 0x7f96f0159420 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f015aa80 con 0x7f96f015d230 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f7528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f015ad60 con 0x7f96f01597f0 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f6526700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f6526700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47692/0 (socket says 172.21.15.170:47692) 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f6526700 1 -- 172.21.15.170:0/1283491500 learned_addr learned my addr 172.21.15.170:0/1283491500 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f5d25700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f0169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.786+0000 7f96f6d27700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 0x7f96f01521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 -- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 msgr2=0x7f96f0169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f0169af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 -- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 msgr2=0x7f96f01521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 0x7f96f01521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 -- 172.21.15.170:0/1283491500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0094f0 con 0x7f96f0159420 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6d27700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 0x7f96f01521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f6526700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f96f0089340 tx=0x7f96ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec024440 con 0x7f96f0159420 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec0245a0 con 0x7f96f0159420 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f015b040 con 0x7f96f0159420 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec024960 con 0x7f96f0159420 2023-10-08T16:09:41.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.787+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f015b600 con 0x7f96f0159420 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.788+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f015b7f0 con 0x7f96f0159420 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.788+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f96ec01a420 con 0x7f96f0159420 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.789+0000 7f96e67fc700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 0x7f96d4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.789+0000 7f96f5d25700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 0x7f96d4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.789+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96ec086740 con 0x7f96f0159420 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.789+0000 7f96f5d25700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 0x7f96d4069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96f015a630 tx=0x7f96e0006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:41.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:41.793+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96ec053720 con 0x7f96f0159420 2023-10-08T16:09:42.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.016+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96f004c3c0 con 0x7f96f0159420 2023-10-08T16:09:42.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.017+0000 7f96e67fc700 1 -- 172.21.15.170:0/1283491500 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f96ec056d40 con 0x7f96f0159420 2023-10-08T16:09:42.017 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:42.017 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 msgr2=0x7f96d4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 0x7f96d4069620 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96f015a630 tx=0x7f96e0006f90 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 msgr2=0x7f96f0141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f96f0089340 tx=0x7f96ec017000 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 shutdown_connections 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96f015d230 0x7f96f01521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96d4067170 0x7f96d4069620 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96f0159420 0x7f96f0141090 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 --2- 172.21.15.170:0/1283491500 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96f01597f0 0x7f96f0169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 >> 172.21.15.170:0/1283491500 conn(0x7f96f00d0660 msgr2=0x7f96f00698d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 shutdown_connections 2023-10-08T16:09:42.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.019+0000 7f96f7528700 1 -- 172.21.15.170:0/1283491500 wait complete. 2023-10-08T16:09:42.021 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 47 2023-10-08T16:09:42.031 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:42.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.146+0000 7f0f0ffff700 1 Processor -- start 2023-10-08T16:09:42.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.146+0000 7f0f0ffff700 1 -- start start 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0f10159420 0x7f0f100b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0ffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100b2980 con 0x7f0f1015d230 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0ffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100b11d0 con 0x7f0f101597f0 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0ffff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f100b14b0 con 0x7f0f10159420 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0f10159420 0x7f0f100b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0f1015d230 0x7f0f1015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46868/0 (socket says 172.21.15.170:46868) 2023-10-08T16:09:42.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.147+0000 7f0f0f7fe700 1 -- 172.21.15.170:0/4052278671 learned_addr learned my addr 172.21.15.170:0/4052278671 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0effd700 1 -- 172.21.15.170:0/4052278671 >> v1:172.21.15.191:6789/0 conn(0x7f0f101597f0 legacy=0x7f0f100b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0effd700 1 -- 172.21.15.170:0/4052278671 >> v1:172.21.15.170:6789/0 conn(0x7f0f1015d230 legacy=0x7f0f1015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0effd700 1 -- 172.21.15.170:0/4052278671 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100b1790 con 0x7f0f10159420 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0effd700 1 --2- 172.21.15.170:0/4052278671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f100b5f40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098e0 tx=0x7f0f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc414353f054d449 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0e7fc700 1 -- 172.21.15.170:0/4052278671 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f00009e80 con 0x7f0f10159420 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0e7fc700 1 -- 172.21.15.170:0/4052278671 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0000b820 con 0x7f0f10159420 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0e7fc700 1 -- 172.21.15.170:0/4052278671 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000bba0 con 0x7f0f10159420 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 -- 172.21.15.170:0/4052278671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 msgr2=0x7f0f100b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/4052278671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f100b5f40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098e0 tx=0x7f0f000091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 -- 172.21.15.170:0/4052278671 shutdown_connections 2023-10-08T16:09:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/4052278671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f100b5f40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 -- 172.21.15.170:0/4052278671 >> 172.21.15.170:0/4052278671 conn(0x7f0f100d0660 msgr2=0x7f0f100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 -- 172.21.15.170:0/4052278671 shutdown_connections 2023-10-08T16:09:42.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.148+0000 7f0f0ffff700 1 -- 172.21.15.170:0/4052278671 wait complete. 2023-10-08T16:09:42.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 Processor -- start 2023-10-08T16:09:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 -- start start 2023-10-08T16:09:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 0x7f0f100c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f1015d230 0x7f0f100c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100687a0 con 0x7f0f10159420 2023-10-08T16:09:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f10144830 con 0x7f0f1015d230 2023-10-08T16:09:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1016b3f0 con 0x7f0f101597f0 2023-10-08T16:09:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47708/0 (socket says 172.21.15.170:47708) 2023-10-08T16:09:42.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0effd700 1 -- 172.21.15.170:0/1627260434 learned_addr learned my addr 172.21.15.170:0/1627260434 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.150+0000 7f0f0f7fe700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f1015d230 0x7f0f100c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f067fc700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 0x7f0f100c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 -- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 msgr2=0x7f0f100c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 0x7f0f100c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 -- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f1015d230 msgr2=0x7f0f100c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f1015d230 0x7f0f100c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 -- 172.21.15.170:0/1627260434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f00009420 con 0x7f0f10159420 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f067fc700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 0x7f0f100c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0effd700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0f000030c0 tx=0x7f0f00016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ba60 con 0x7f0f10159420 2023-10-08T16:09:42.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f00020070 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000bbe0 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f100cbde0 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.151+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f100cc3a0 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.152+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1004c3c0 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.152+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f0f00022030 con 0x7f0f10159420 2023-10-08T16:09:42.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.153+0000 7f0f077fe700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 0x7f0ef0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.153+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0f00086850 con 0x7f0f10159420 2023-10-08T16:09:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.153+0000 7f0f067fc700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 0x7f0ef0069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.153+0000 7f0f067fc700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 0x7f0ef0069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8007fd0 tx=0x7f0ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.157+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f00053830 con 0x7f0f10159420 2023-10-08T16:09:42.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.342+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f1004c3c0 con 0x7f0f10159420 2023-10-08T16:09:42.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.343+0000 7f0f077fe700 1 -- 172.21.15.170:0/1627260434 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f0f00056e50 con 0x7f0f10159420 2023-10-08T16:09:42.343 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:42.343 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":77,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:39.474771+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 msgr2=0x7f0ef0069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 0x7f0ef0069620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8007fd0 tx=0x7f0ef8009040 comp rx=0 tx=0).stop 2023-10-08T16:09:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 msgr2=0x7f0f101442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0f000030c0 tx=0x7f0f00016000 comp rx=0 tx=0).stop 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 shutdown_connections 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0f1015d230 0x7f0f100c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0ef0067170 0x7f0ef0069620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0f10159420 0x7f0f101442f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 --2- 172.21.15.170:0/1627260434 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0f101597f0 0x7f0f100c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 >> 172.21.15.170:0/1627260434 conn(0x7f0f100d0660 msgr2=0x7f0f10156330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 shutdown_connections 2023-10-08T16:09:42.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.345+0000 7f0f0ffff700 1 -- 172.21.15.170:0/1627260434 wait complete. 2023-10-08T16:09:42.357 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:09:42.358 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:09:42.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.471+0000 7fa1ec8b6700 1 Processor -- start 2023-10-08T16:09:42.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.471+0000 7fa1ec8b6700 1 -- start start 2023-10-08T16:09:42.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ec8b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 0x7fa1e8140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ec8b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa1e806c0a0 0x7fa1e8141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ec8b6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e815c2a0 con 0x7fa1e80c8e00 2023-10-08T16:09:42.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ec8b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e815b690 con 0x7fa1e806c0a0 2023-10-08T16:09:42.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ec8b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e815b970 con 0x7fa1e806bcd0 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1e6d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa1e80c8e00 0x7fa1e80c9260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46870/0 (socket says 172.21.15.170:46870) 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1e6d9d700 1 -- 172.21.15.170:0/4112585164 learned_addr learned my addr 172.21.15.170:0/4112585164 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1e659c700 1 --2- 172.21.15.170:0/4112585164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 0x7fa1e8140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1ddd9b700 1 --2- 172.21.15.170:0/4112585164 >> v2:172.21.15.191:3300/0 conn(0x7fa1e806c0a0 0x7fa1e8141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1e5d9b700 1 -- 172.21.15.170:0/4112585164 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289061880 0 0) 0x7fa1e815c2a0 con 0x7fa1e80c8e00 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.472+0000 7fa1e5d9b700 1 -- 172.21.15.170:0/4112585164 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003570 con 0x7fa1e80c8e00 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e659c700 1 -- 172.21.15.170:0/4112585164 >> v2:172.21.15.191:3300/0 conn(0x7fa1e806c0a0 msgr2=0x7fa1e8141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e659c700 1 --2- 172.21.15.170:0/4112585164 >> v2:172.21.15.191:3300/0 conn(0x7fa1e806c0a0 0x7fa1e8141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e659c700 1 -- 172.21.15.170:0/4112585164 >> v1:172.21.15.170:6789/0 conn(0x7fa1e80c8e00 legacy=0x7fa1e80c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e659c700 1 -- 172.21.15.170:0/4112585164 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e815bc50 con 0x7fa1e806bcd0 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e659c700 1 --2- 172.21.15.170:0/4112585164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 0x7fa1e8140df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098e0 tx=0x7fa1d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8148bbd99a9ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e5d9b700 1 -- 172.21.15.170:0/4112585164 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011630 con 0x7fa1e806bcd0 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e5d9b700 1 -- 172.21.15.170:0/4112585164 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8011790 con 0x7fa1e806bcd0 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1e5d9b700 1 -- 172.21.15.170:0/4112585164 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011b30 con 0x7fa1e806bcd0 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/4112585164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 msgr2=0x7fa1e8140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.473+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/4112585164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 0x7fa1e8140df0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80098e0 tx=0x7fa1d80091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/4112585164 shutdown_connections 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/4112585164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e806bcd0 0x7fa1e8140df0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/4112585164 >> v2:172.21.15.191:3300/0 conn(0x7fa1e806c0a0 0x7fa1e8141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/4112585164 >> 172.21.15.170:0/4112585164 conn(0x7fa1e80ce600 msgr2=0x7fa1e80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/4112585164 shutdown_connections 2023-10-08T16:09:42.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.474+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/4112585164 wait complete. 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.475+0000 7fa1ec8b6700 1 Processor -- start 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.475+0000 7fa1ec8b6700 1 -- start start 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.475+0000 7fa1ec8b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.475+0000 7fa1ec8b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e806c0a0 0x7fa1e816b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.475+0000 7fa1ec8b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e80c8e00 0x7fa1e816b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ec8b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b3140 con 0x7fa1e806bcd0 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ec8b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b2530 con 0x7fa1e806c0a0 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ec8b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80b2810 con 0x7fa1e80c8e00 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47722/0 (socket says 172.21.15.170:47722) 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 -- 172.21.15.170:0/3969089190 learned_addr learned my addr 172.21.15.170:0/3969089190 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ddd9b700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e806c0a0 0x7fa1e816b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e6d9d700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e80c8e00 0x7fa1e816b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 -- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e80c8e00 msgr2=0x7fa1e816b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e80c8e00 0x7fa1e816b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 -- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e806c0a0 msgr2=0x7fa1e816b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e806c0a0 0x7fa1e816b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 -- 172.21.15.170:0/3969089190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009420 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1e659c700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8069220 tx=0x7fa1d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8011b10 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d80253f0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80b2af0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80257b0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.476+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8152ec0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.477+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e804c3c0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.478+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fa1d8025ba0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.478+0000 7fa1deffd700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 0x7fa1c8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.478+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1d80860f0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.478+0000 7fa1ddd9b700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 0x7fa1c8069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.478+0000 7fa1ddd9b700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 0x7fa1c8069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00093f0 tx=0x7fa1d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.482+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1d80530d0 con 0x7fa1e806bcd0 2023-10-08T16:09:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.659+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1e80b3280 con 0x7fa1c8067170 2023-10-08T16:09:42.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.667+0000 7fa1deffd700 1 -- 172.21.15.170:0/3969089190 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289284 (secure 0 0 0) 0x7fa1d0009dc0 con 0x7fa1c8067170 2023-10-08T16:09:42.671 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:42.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 msgr2=0x7fa1c8069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 0x7fa1c8069620 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00093f0 tx=0x7fa1d0008040 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 msgr2=0x7fa1e80b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8069220 tx=0x7fa1d8017000 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 shutdown_connections 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1e80c8e00 0x7fa1e816b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa1c8067170 0x7fa1c8069620 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1e806bcd0 0x7fa1e80b1a70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 --2- 172.21.15.170:0/3969089190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1e806c0a0 0x7fa1e816b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 >> 172.21.15.170:0/3969089190 conn(0x7fa1e80ce600 msgr2=0x7fa1e80c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 shutdown_connections 2023-10-08T16:09:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.671+0000 7fa1ec8b6700 1 -- 172.21.15.170:0/3969089190 wait complete. 2023-10-08T16:09:42.675 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:09:42.684 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-10-08T16:09:42.288027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2403208,"kb_used_data":8520,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752571512,"statfs":{"total":773094113280,"available":770633228288,"internally_reserved":0,"allocated":8724480,"data_stored":4382372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.658269"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495408+0000","last_change":"2023-10-08T16:09:38.495408+0000","last_active":"2023-10-08T16:09:38.495408+0000","last_peered":"2023-10-08T16:09:38.495408+0000","last_clean":"2023-10-08T16:09:38.495408+0000","last_became_active":"2023-10-08T16:09:38.495154+0000","last_became_peered":"2023-10-08T16:09:38.495154+0000","last_unstale":"2023-10-08T16:09:38.495408+0000","last_undegraded":"2023-10-08T16:09:38.495408+0000","last_fullsized":"2023-10-08T16:09:38.495408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.919970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488706+0000","last_change":"2023-10-08T16:09:38.488706+0000","last_active":"2023-10-08T16:09:38.488706+0000","last_peered":"2023-10-08T16:09:38.488706+0000","last_clean":"2023-10-08T16:09:38.488706+0000","last_became_active":"2023-10-08T16:09:38.488460+0000","last_became_peered":"2023-10-08T16:09:38.488460+0000","last_unstale":"2023-10-08T16:09:38.488706+0000","last_undegraded":"2023-10-08T16:09:38.488706+0000","last_fullsized":"2023-10-08T16:09:38.488706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:20:39.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.501141+0000","last_change":"2023-10-08T16:09:38.501141+0000","last_active":"2023-10-08T16:09:38.501141+0000","last_peered":"2023-10-08T16:09:38.501141+0000","last_clean":"2023-10-08T16:09:38.501141+0000","last_became_active":"2023-10-08T16:09:38.500894+0000","last_became_peered":"2023-10-08T16:09:38.500894+0000","last_unstale":"2023-10-08T16:09:38.501141+0000","last_undegraded":"2023-10-08T16:09:38.501141+0000","last_fullsized":"2023-10-08T16:09:38.501141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:05.398877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.492081+0000","last_change":"2023-10-08T16:09:38.492081+0000","last_active":"2023-10-08T16:09:38.492081+0000","last_peered":"2023-10-08T16:09:38.492081+0000","last_clean":"2023-10-08T16:09:38.492081+0000","last_became_active":"2023-10-08T16:09:38.491482+0000","last_became_peered":"2023-10-08T16:09:38.491482+0000","last_unstale":"2023-10-08T16:09:38.492081+0000","last_undegraded":"2023-10-08T16:09:38.492081+0000","last_fullsized":"2023-10-08T16:09:38.492081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:58.912361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.497164+0000","last_change":"2023-10-08T16:09:38.497164+0000","last_active":"2023-10-08T16:09:38.497164+0000","last_peered":"2023-10-08T16:09:38.497164+0000","last_clean":"2023-10-08T16:09:38.497164+0000","last_became_active":"2023-10-08T16:09:38.496680+0000","last_became_peered":"2023-10-08T16:09:38.496680+0000","last_unstale":"2023-10-08T16:09:38.497164+0000","last_undegraded":"2023-10-08T16:09:38.497164+0000","last_fullsized":"2023-10-08T16:09:38.497164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:50.417249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490002+0000","last_change":"2023-10-08T16:09:38.490002+0000","last_active":"2023-10-08T16:09:38.490002+0000","last_peered":"2023-10-08T16:09:38.490002+0000","last_clean":"2023-10-08T16:09:38.490002+0000","last_became_active":"2023-10-08T16:09:38.489725+0000","last_became_peered":"2023-10-08T16:09:38.489725+0000","last_unstale":"2023-10-08T16:09:38.490002+0000","last_undegraded":"2023-10-08T16:09:38.490002+0000","last_fullsized":"2023-10-08T16:09:38.490002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:24.509666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498791+0000","last_change":"2023-10-08T16:09:38.498791+0000","last_active":"2023-10-08T16:09:38.498791+0000","last_peered":"2023-10-08T16:09:38.498791+0000","last_clean":"2023-10-08T16:09:38.498791+0000","last_became_active":"2023-10-08T16:09:38.498517+0000","last_became_peered":"2023-10-08T16:09:38.498517+0000","last_unstale":"2023-10-08T16:09:38.498791+0000","last_undegraded":"2023-10-08T16:09:38.498791+0000","last_fullsized":"2023-10-08T16:09:38.498791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:51.956068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485583+0000","last_change":"2023-10-08T16:09:38.485583+0000","last_active":"2023-10-08T16:09:38.485583+0000","last_peered":"2023-10-08T16:09:38.485583+0000","last_clean":"2023-10-08T16:09:38.485583+0000","last_became_active":"2023-10-08T16:09:38.484866+0000","last_became_peered":"2023-10-08T16:09:38.484866+0000","last_unstale":"2023-10-08T16:09:38.485583+0000","last_undegraded":"2023-10-08T16:09:38.485583+0000","last_fullsized":"2023-10-08T16:09:38.485583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.919970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485046+0000","last_change":"2023-10-08T16:09:38.485046+0000","last_active":"2023-10-08T16:09:38.485046+0000","last_peered":"2023-10-08T16:09:38.485046+0000","last_clean":"2023-10-08T16:09:38.485046+0000","last_became_active":"2023-10-08T16:09:38.484771+0000","last_became_peered":"2023-10-08T16:09:38.484771+0000","last_unstale":"2023-10-08T16:09:38.485046+0000","last_undegraded":"2023-10-08T16:09:38.485046+0000","last_fullsized":"2023-10-08T16:09:38.485046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:22.524101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487959+0000","last_change":"2023-10-08T16:09:38.487959+0000","last_active":"2023-10-08T16:09:38.487959+0000","last_peered":"2023-10-08T16:09:38.487959+0000","last_clean":"2023-10-08T16:09:38.487959+0000","last_became_active":"2023-10-08T16:09:38.487503+0000","last_became_peered":"2023-10-08T16:09:38.487503+0000","last_unstale":"2023-10-08T16:09:38.487959+0000","last_undegraded":"2023-10-08T16:09:38.487959+0000","last_fullsized":"2023-10-08T16:09:38.487959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:11.593957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.497822+0000","last_change":"2023-10-08T16:09:38.497822+0000","last_active":"2023-10-08T16:09:38.497822+0000","last_peered":"2023-10-08T16:09:38.497822+0000","last_clean":"2023-10-08T16:09:38.497822+0000","last_became_active":"2023-10-08T16:09:38.497575+0000","last_became_peered":"2023-10-08T16:09:38.497575+0000","last_unstale":"2023-10-08T16:09:38.497822+0000","last_undegraded":"2023-10-08T16:09:38.497822+0000","last_fullsized":"2023-10-08T16:09:38.497822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:30.837089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493776+0000","last_change":"2023-10-08T16:09:38.493776+0000","last_active":"2023-10-08T16:09:38.493776+0000","last_peered":"2023-10-08T16:09:38.493776+0000","last_clean":"2023-10-08T16:09:38.493776+0000","last_became_active":"2023-10-08T16:09:38.492998+0000","last_became_peered":"2023-10-08T16:09:38.492998+0000","last_unstale":"2023-10-08T16:09:38.493776+0000","last_undegraded":"2023-10-08T16:09:38.493776+0000","last_fullsized":"2023-10-08T16:09:38.493776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:20:43.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498663+0000","last_change":"2023-10-08T16:09:38.498663+0000","last_active":"2023-10-08T16:09:38.498663+0000","last_peered":"2023-10-08T16:09:38.498663+0000","last_clean":"2023-10-08T16:09:38.498663+0000","last_became_active":"2023-10-08T16:09:38.498304+0000","last_became_peered":"2023-10-08T16:09:38.498304+0000","last_unstale":"2023-10-08T16:09:38.498663+0000","last_undegraded":"2023-10-08T16:09:38.498663+0000","last_fullsized":"2023-10-08T16:09:38.498663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:57.588827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493730+0000","last_change":"2023-10-08T16:09:38.493730+0000","last_active":"2023-10-08T16:09:38.493730+0000","last_peered":"2023-10-08T16:09:38.493730+0000","last_clean":"2023-10-08T16:09:38.493730+0000","last_became_active":"2023-10-08T16:09:38.490622+0000","last_became_peered":"2023-10-08T16:09:38.490622+0000","last_unstale":"2023-10-08T16:09:38.493730+0000","last_undegraded":"2023-10-08T16:09:38.493730+0000","last_fullsized":"2023-10-08T16:09:38.493730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:51.051576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499123+0000","last_change":"2023-10-08T16:09:38.499123+0000","last_active":"2023-10-08T16:09:38.499123+0000","last_peered":"2023-10-08T16:09:38.499123+0000","last_clean":"2023-10-08T16:09:38.499123+0000","last_became_active":"2023-10-08T16:09:38.498872+0000","last_became_peered":"2023-10-08T16:09:38.498872+0000","last_unstale":"2023-10-08T16:09:38.499123+0000","last_undegraded":"2023-10-08T16:09:38.499123+0000","last_fullsized":"2023-10-08T16:09:38.499123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:52.601103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493525+0000","last_change":"2023-10-08T16:09:38.493525+0000","last_active":"2023-10-08T16:09:38.493525+0000","last_peered":"2023-10-08T16:09:38.493525+0000","last_clean":"2023-10-08T16:09:38.493525+0000","last_became_active":"2023-10-08T16:09:38.493155+0000","last_became_peered":"2023-10-08T16:09:38.493155+0000","last_unstale":"2023-10-08T16:09:38.493525+0000","last_undegraded":"2023-10-08T16:09:38.493525+0000","last_fullsized":"2023-10-08T16:09:38.493525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:02:42.762068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491414+0000","last_change":"2023-10-08T16:09:38.491414+0000","last_active":"2023-10-08T16:09:38.491414+0000","last_peered":"2023-10-08T16:09:38.491414+0000","last_clean":"2023-10-08T16:09:38.491414+0000","last_became_active":"2023-10-08T16:09:38.490646+0000","last_became_peered":"2023-10-08T16:09:38.490646+0000","last_unstale":"2023-10-08T16:09:38.491414+0000","last_undegraded":"2023-10-08T16:09:38.491414+0000","last_fullsized":"2023-10-08T16:09:38.491414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:08.609700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494072+0000","last_change":"2023-10-08T16:09:38.494072+0000","last_active":"2023-10-08T16:09:38.494072+0000","last_peered":"2023-10-08T16:09:38.494072+0000","last_clean":"2023-10-08T16:09:38.494072+0000","last_became_active":"2023-10-08T16:09:38.493798+0000","last_became_peered":"2023-10-08T16:09:38.493798+0000","last_unstale":"2023-10-08T16:09:38.494072+0000","last_undegraded":"2023-10-08T16:09:38.494072+0000","last_fullsized":"2023-10-08T16:09:38.494072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:53.054694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.481708+0000","last_change":"2023-10-08T16:09:38.481708+0000","last_active":"2023-10-08T16:09:38.481708+0000","last_peered":"2023-10-08T16:09:38.481708+0000","last_clean":"2023-10-08T16:09:38.481708+0000","last_became_active":"2023-10-08T16:09:38.481376+0000","last_became_peered":"2023-10-08T16:09:38.481376+0000","last_unstale":"2023-10-08T16:09:38.481708+0000","last_undegraded":"2023-10-08T16:09:38.481708+0000","last_fullsized":"2023-10-08T16:09:38.481708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:49.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490850+0000","last_change":"2023-10-08T16:09:38.490850+0000","last_active":"2023-10-08T16:09:38.490850+0000","last_peered":"2023-10-08T16:09:38.490850+0000","last_clean":"2023-10-08T16:09:38.490850+0000","last_became_active":"2023-10-08T16:09:38.490598+0000","last_became_peered":"2023-10-08T16:09:38.490598+0000","last_unstale":"2023-10-08T16:09:38.490850+0000","last_undegraded":"2023-10-08T16:09:38.490850+0000","last_fullsized":"2023-10-08T16:09:38.490850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:55.340672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491568+0000","last_change":"2023-10-08T16:09:38.491568+0000","last_active":"2023-10-08T16:09:38.491568+0000","last_peered":"2023-10-08T16:09:38.491568+0000","last_clean":"2023-10-08T16:09:38.491568+0000","last_became_active":"2023-10-08T16:09:38.491178+0000","last_became_peered":"2023-10-08T16:09:38.491178+0000","last_unstale":"2023-10-08T16:09:38.491568+0000","last_undegraded":"2023-10-08T16:09:38.491568+0000","last_fullsized":"2023-10-08T16:09:38.491568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:57.376194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.481489+0000","last_change":"2023-10-08T16:09:38.481489+0000","last_active":"2023-10-08T16:09:38.481489+0000","last_peered":"2023-10-08T16:09:38.481489+0000","last_clean":"2023-10-08T16:09:38.481489+0000","last_became_active":"2023-10-08T16:09:38.481226+0000","last_became_peered":"2023-10-08T16:09:38.481226+0000","last_unstale":"2023-10-08T16:09:38.481489+0000","last_undegraded":"2023-10-08T16:09:38.481489+0000","last_fullsized":"2023-10-08T16:09:38.481489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:15:44.366084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487663+0000","last_change":"2023-10-08T16:09:38.487663+0000","last_active":"2023-10-08T16:09:38.487663+0000","last_peered":"2023-10-08T16:09:38.487663+0000","last_clean":"2023-10-08T16:09:38.487663+0000","last_became_active":"2023-10-08T16:09:38.486857+0000","last_became_peered":"2023-10-08T16:09:38.486857+0000","last_unstale":"2023-10-08T16:09:38.487663+0000","last_undegraded":"2023-10-08T16:09:38.487663+0000","last_fullsized":"2023-10-08T16:09:38.487663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:54.159859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493913+0000","last_change":"2023-10-08T16:09:38.493913+0000","last_active":"2023-10-08T16:09:38.493913+0000","last_peered":"2023-10-08T16:09:38.493913+0000","last_clean":"2023-10-08T16:09:38.493913+0000","last_became_active":"2023-10-08T16:09:38.493652+0000","last_became_peered":"2023-10-08T16:09:38.493652+0000","last_unstale":"2023-10-08T16:09:38.493913+0000","last_undegraded":"2023-10-08T16:09:38.493913+0000","last_fullsized":"2023-10-08T16:09:38.493913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:52:00.184659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495647+0000","last_change":"2023-10-08T16:09:38.495647+0000","last_active":"2023-10-08T16:09:38.495647+0000","last_peered":"2023-10-08T16:09:38.495647+0000","last_clean":"2023-10-08T16:09:38.495647+0000","last_became_active":"2023-10-08T16:09:38.495389+0000","last_became_peered":"2023-10-08T16:09:38.495389+0000","last_unstale":"2023-10-08T16:09:38.495647+0000","last_undegraded":"2023-10-08T16:09:38.495647+0000","last_fullsized":"2023-10-08T16:09:38.495647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:40.495764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498626+0000","last_change":"2023-10-08T16:09:38.498626+0000","last_active":"2023-10-08T16:09:38.498626+0000","last_peered":"2023-10-08T16:09:38.498626+0000","last_clean":"2023-10-08T16:09:38.498626+0000","last_became_active":"2023-10-08T16:09:38.498320+0000","last_became_peered":"2023-10-08T16:09:38.498320+0000","last_unstale":"2023-10-08T16:09:38.498626+0000","last_undegraded":"2023-10-08T16:09:38.498626+0000","last_fullsized":"2023-10-08T16:09:38.498626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:52.601103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491808+0000","last_change":"2023-10-08T16:09:38.491808+0000","last_active":"2023-10-08T16:09:38.491808+0000","last_peered":"2023-10-08T16:09:38.491808+0000","last_clean":"2023-10-08T16:09:38.491808+0000","last_became_active":"2023-10-08T16:09:38.491163+0000","last_became_peered":"2023-10-08T16:09:38.491163+0000","last_unstale":"2023-10-08T16:09:38.491808+0000","last_undegraded":"2023-10-08T16:09:38.491808+0000","last_fullsized":"2023-10-08T16:09:38.491808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:37.656010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494758+0000","last_change":"2023-10-08T16:09:38.494758+0000","last_active":"2023-10-08T16:09:38.494758+0000","last_peered":"2023-10-08T16:09:38.494758+0000","last_clean":"2023-10-08T16:09:38.494758+0000","last_became_active":"2023-10-08T16:09:38.494047+0000","last_became_peered":"2023-10-08T16:09:38.494047+0000","last_unstale":"2023-10-08T16:09:38.494758+0000","last_undegraded":"2023-10-08T16:09:38.494758+0000","last_fullsized":"2023-10-08T16:09:38.494758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:21.550265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.479586+0000","last_change":"2023-10-08T16:09:38.479586+0000","last_active":"2023-10-08T16:09:38.479586+0000","last_peered":"2023-10-08T16:09:38.479586+0000","last_clean":"2023-10-08T16:09:38.479586+0000","last_became_active":"2023-10-08T16:09:38.479321+0000","last_became_peered":"2023-10-08T16:09:38.479321+0000","last_unstale":"2023-10-08T16:09:38.479586+0000","last_undegraded":"2023-10-08T16:09:38.479586+0000","last_fullsized":"2023-10-08T16:09:38.479586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:11.593957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490332+0000","last_change":"2023-10-08T16:09:38.490332+0000","last_active":"2023-10-08T16:09:38.490332+0000","last_peered":"2023-10-08T16:09:38.490332+0000","last_clean":"2023-10-08T16:09:38.490332+0000","last_became_active":"2023-10-08T16:09:38.490064+0000","last_became_peered":"2023-10-08T16:09:38.490064+0000","last_unstale":"2023-10-08T16:09:38.490332+0000","last_undegraded":"2023-10-08T16:09:38.490332+0000","last_fullsized":"2023-10-08T16:09:38.490332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.033893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.492338+0000","last_change":"2023-10-08T16:09:38.492338+0000","last_active":"2023-10-08T16:09:38.492338+0000","last_peered":"2023-10-08T16:09:38.492338+0000","last_clean":"2023-10-08T16:09:38.492338+0000","last_became_active":"2023-10-08T16:09:38.492122+0000","last_became_peered":"2023-10-08T16:09:38.492122+0000","last_unstale":"2023-10-08T16:09:38.492338+0000","last_undegraded":"2023-10-08T16:09:38.492338+0000","last_fullsized":"2023-10-08T16:09:38.492338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:27.529093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487965+0000","last_change":"2023-10-08T16:09:38.487965+0000","last_active":"2023-10-08T16:09:38.487965+0000","last_peered":"2023-10-08T16:09:38.487965+0000","last_clean":"2023-10-08T16:09:38.487965+0000","last_became_active":"2023-10-08T16:09:38.487431+0000","last_became_peered":"2023-10-08T16:09:38.487431+0000","last_unstale":"2023-10-08T16:09:38.487965+0000","last_undegraded":"2023-10-08T16:09:38.487965+0000","last_fullsized":"2023-10-08T16:09:38.487965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:40.495764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493846+0000","last_change":"2023-10-08T16:09:38.493846+0000","last_active":"2023-10-08T16:09:38.493846+0000","last_peered":"2023-10-08T16:09:38.493846+0000","last_clean":"2023-10-08T16:09:38.493846+0000","last_became_active":"2023-10-08T16:09:38.493518+0000","last_became_peered":"2023-10-08T16:09:38.493518+0000","last_unstale":"2023-10-08T16:09:38.493846+0000","last_undegraded":"2023-10-08T16:09:38.493846+0000","last_fullsized":"2023-10-08T16:09:38.493846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:36:37.088615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"76'4","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648475+0000","last_change":"2023-10-08T16:09:37.648435+0000","last_active":"2023-10-08T16:09:38.648475+0000","last_peered":"2023-10-08T16:09:38.648475+0000","last_clean":"2023-10-08T16:09:38.648475+0000","last_became_active":"2023-10-08T16:09:37.648102+0000","last_became_peered":"2023-10-08T16:09:37.648102+0000","last_unstale":"2023-10-08T16:09:38.648475+0000","last_undegraded":"2023-10-08T16:09:38.648475+0000","last_fullsized":"2023-10-08T16:09:38.648475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:40.160762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493640+0000","last_change":"2023-10-08T16:09:38.493640+0000","last_active":"2023-10-08T16:09:38.493640+0000","last_peered":"2023-10-08T16:09:38.493640+0000","last_clean":"2023-10-08T16:09:38.493640+0000","last_became_active":"2023-10-08T16:09:38.493338+0000","last_became_peered":"2023-10-08T16:09:38.493338+0000","last_unstale":"2023-10-08T16:09:38.493640+0000","last_undegraded":"2023-10-08T16:09:38.493640+0000","last_fullsized":"2023-10-08T16:09:38.493640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:15:44.366084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487563+0000","last_change":"2023-10-08T16:09:37.654432+0000","last_active":"2023-10-08T16:09:38.487563+0000","last_peered":"2023-10-08T16:09:38.487563+0000","last_clean":"2023-10-08T16:09:38.487563+0000","last_became_active":"2023-10-08T16:09:37.654146+0000","last_became_peered":"2023-10-08T16:09:37.654146+0000","last_unstale":"2023-10-08T16:09:38.487563+0000","last_undegraded":"2023-10-08T16:09:38.487563+0000","last_fullsized":"2023-10-08T16:09:38.487563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:22.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494322+0000","last_change":"2023-10-08T16:09:38.494322+0000","last_active":"2023-10-08T16:09:38.494322+0000","last_peered":"2023-10-08T16:09:38.494322+0000","last_clean":"2023-10-08T16:09:38.494322+0000","last_became_active":"2023-10-08T16:09:38.493439+0000","last_became_peered":"2023-10-08T16:09:38.493439+0000","last_unstale":"2023-10-08T16:09:38.494322+0000","last_undegraded":"2023-10-08T16:09:38.494322+0000","last_fullsized":"2023-10-08T16:09:38.494322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.033893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.647197+0000","last_change":"2023-10-08T16:09:37.648072+0000","last_active":"2023-10-08T16:09:38.647197+0000","last_peered":"2023-10-08T16:09:38.647197+0000","last_clean":"2023-10-08T16:09:38.647197+0000","last_became_active":"2023-10-08T16:09:37.647796+0000","last_became_peered":"2023-10-08T16:09:37.647796+0000","last_unstale":"2023-10-08T16:09:38.647197+0000","last_undegraded":"2023-10-08T16:09:38.647197+0000","last_fullsized":"2023-10-08T16:09:38.647197+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:57.915707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.500300+0000","last_change":"2023-10-08T16:09:38.500300+0000","last_active":"2023-10-08T16:09:38.500300+0000","last_peered":"2023-10-08T16:09:38.500300+0000","last_clean":"2023-10-08T16:09:38.500300+0000","last_became_active":"2023-10-08T16:09:38.499005+0000","last_became_peered":"2023-10-08T16:09:38.499005+0000","last_unstale":"2023-10-08T16:09:38.500300+0000","last_undegraded":"2023-10-08T16:09:38.500300+0000","last_fullsized":"2023-10-08T16:09:38.500300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:31:45.808377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648989+0000","last_change":"2023-10-08T16:09:37.649091+0000","last_active":"2023-10-08T16:09:38.648989+0000","last_peered":"2023-10-08T16:09:38.648989+0000","last_clean":"2023-10-08T16:09:38.648989+0000","last_became_active":"2023-10-08T16:09:37.648832+0000","last_became_peered":"2023-10-08T16:09:37.648832+0000","last_unstale":"2023-10-08T16:09:38.648989+0000","last_undegraded":"2023-10-08T16:09:38.648989+0000","last_fullsized":"2023-10-08T16:09:38.648989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:05.279013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488092+0000","last_change":"2023-10-08T16:09:38.488092+0000","last_active":"2023-10-08T16:09:38.488092+0000","last_peered":"2023-10-08T16:09:38.488092+0000","last_clean":"2023-10-08T16:09:38.488092+0000","last_became_active":"2023-10-08T16:09:38.487841+0000","last_became_peered":"2023-10-08T16:09:38.487841+0000","last_unstale":"2023-10-08T16:09:38.488092+0000","last_undegraded":"2023-10-08T16:09:38.488092+0000","last_fullsized":"2023-10-08T16:09:38.488092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:54.159859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.486620+0000","last_change":"2023-10-08T16:09:37.652162+0000","last_active":"2023-10-08T16:09:38.486620+0000","last_peered":"2023-10-08T16:09:38.486620+0000","last_clean":"2023-10-08T16:09:38.486620+0000","last_became_active":"2023-10-08T16:09:37.651798+0000","last_became_peered":"2023-10-08T16:09:37.651798+0000","last_unstale":"2023-10-08T16:09:38.486620+0000","last_undegraded":"2023-10-08T16:09:38.486620+0000","last_fullsized":"2023-10-08T16:09:38.486620+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:56.263462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495519+0000","last_change":"2023-10-08T16:09:38.495519+0000","last_active":"2023-10-08T16:09:38.495519+0000","last_peered":"2023-10-08T16:09:38.495519+0000","last_clean":"2023-10-08T16:09:38.495519+0000","last_became_active":"2023-10-08T16:09:38.492459+0000","last_became_peered":"2023-10-08T16:09:38.492459+0000","last_unstale":"2023-10-08T16:09:38.495519+0000","last_undegraded":"2023-10-08T16:09:38.495519+0000","last_fullsized":"2023-10-08T16:09:38.495519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:55.584901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.486072+0000","last_change":"2023-10-08T16:09:37.654509+0000","last_active":"2023-10-08T16:09:38.486072+0000","last_peered":"2023-10-08T16:09:38.486072+0000","last_clean":"2023-10-08T16:09:38.486072+0000","last_became_active":"2023-10-08T16:09:37.654135+0000","last_became_peered":"2023-10-08T16:09:37.654135+0000","last_unstale":"2023-10-08T16:09:38.486072+0000","last_undegraded":"2023-10-08T16:09:38.486072+0000","last_fullsized":"2023-10-08T16:09:38.486072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:10.589694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496581+0000","last_change":"2023-10-08T16:09:38.496581+0000","last_active":"2023-10-08T16:09:38.496581+0000","last_peered":"2023-10-08T16:09:38.496581+0000","last_clean":"2023-10-08T16:09:38.496581+0000","last_became_active":"2023-10-08T16:09:38.496028+0000","last_became_peered":"2023-10-08T16:09:38.496028+0000","last_unstale":"2023-10-08T16:09:38.496581+0000","last_undegraded":"2023-10-08T16:09:38.496581+0000","last_fullsized":"2023-10-08T16:09:38.496581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:32.460601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652548+0000","last_change":"2023-10-08T16:09:37.648458+0000","last_active":"2023-10-08T16:09:37.652548+0000","last_peered":"2023-10-08T16:09:37.652548+0000","last_clean":"2023-10-08T16:09:37.652548+0000","last_became_active":"2023-10-08T16:09:37.647730+0000","last_became_peered":"2023-10-08T16:09:37.647730+0000","last_unstale":"2023-10-08T16:09:37.652548+0000","last_undegraded":"2023-10-08T16:09:37.652548+0000","last_fullsized":"2023-10-08T16:09:37.652548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:29.832826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496514+0000","last_change":"2023-10-08T16:09:38.496514+0000","last_active":"2023-10-08T16:09:38.496514+0000","last_peered":"2023-10-08T16:09:38.496514+0000","last_clean":"2023-10-08T16:09:38.496514+0000","last_became_active":"2023-10-08T16:09:38.496224+0000","last_became_peered":"2023-10-08T16:09:38.496224+0000","last_unstale":"2023-10-08T16:09:38.496514+0000","last_undegraded":"2023-10-08T16:09:38.496514+0000","last_fullsized":"2023-10-08T16:09:38.496514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:05.398877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655103+0000","last_change":"2023-10-08T16:09:37.654422+0000","last_active":"2023-10-08T16:09:37.655103+0000","last_peered":"2023-10-08T16:09:37.655103+0000","last_clean":"2023-10-08T16:09:37.655103+0000","last_became_active":"2023-10-08T16:09:37.654192+0000","last_became_peered":"2023-10-08T16:09:37.654192+0000","last_unstale":"2023-10-08T16:09:37.655103+0000","last_undegraded":"2023-10-08T16:09:37.655103+0000","last_fullsized":"2023-10-08T16:09:37.655103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.145714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.480100+0000","last_change":"2023-10-08T16:09:38.480100+0000","last_active":"2023-10-08T16:09:38.480100+0000","last_peered":"2023-10-08T16:09:38.480100+0000","last_clean":"2023-10-08T16:09:38.480100+0000","last_became_active":"2023-10-08T16:09:38.479853+0000","last_became_peered":"2023-10-08T16:09:38.479853+0000","last_unstale":"2023-10-08T16:09:38.480100+0000","last_undegraded":"2023-10-08T16:09:38.480100+0000","last_fullsized":"2023-10-08T16:09:38.480100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:32.460601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653353+0000","last_change":"2023-10-08T16:09:37.648553+0000","last_active":"2023-10-08T16:09:37.653353+0000","last_peered":"2023-10-08T16:09:37.653353+0000","last_clean":"2023-10-08T16:09:37.653353+0000","last_became_active":"2023-10-08T16:09:37.648167+0000","last_became_peered":"2023-10-08T16:09:37.648167+0000","last_unstale":"2023-10-08T16:09:37.653353+0000","last_undegraded":"2023-10-08T16:09:37.653353+0000","last_fullsized":"2023-10-08T16:09:37.653353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:03.895723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490155+0000","last_change":"2023-10-08T16:09:38.490155+0000","last_active":"2023-10-08T16:09:38.490155+0000","last_peered":"2023-10-08T16:09:38.490155+0000","last_clean":"2023-10-08T16:09:38.490155+0000","last_became_active":"2023-10-08T16:09:38.489241+0000","last_became_peered":"2023-10-08T16:09:38.489241+0000","last_unstale":"2023-10-08T16:09:38.490155+0000","last_undegraded":"2023-10-08T16:09:38.490155+0000","last_fullsized":"2023-10-08T16:09:38.490155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:33.553406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653085+0000","last_change":"2023-10-08T16:09:37.650879+0000","last_active":"2023-10-08T16:09:37.653085+0000","last_peered":"2023-10-08T16:09:37.653085+0000","last_clean":"2023-10-08T16:09:37.653085+0000","last_became_active":"2023-10-08T16:09:37.650623+0000","last_became_peered":"2023-10-08T16:09:37.650623+0000","last_unstale":"2023-10-08T16:09:37.653085+0000","last_undegraded":"2023-10-08T16:09:37.653085+0000","last_fullsized":"2023-10-08T16:09:37.653085+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:31.874956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494248+0000","last_change":"2023-10-08T16:09:38.494248+0000","last_active":"2023-10-08T16:09:38.494248+0000","last_peered":"2023-10-08T16:09:38.494248+0000","last_clean":"2023-10-08T16:09:38.494248+0000","last_became_active":"2023-10-08T16:09:38.494018+0000","last_became_peered":"2023-10-08T16:09:38.494018+0000","last_unstale":"2023-10-08T16:09:38.494248+0000","last_undegraded":"2023-10-08T16:09:38.494248+0000","last_fullsized":"2023-10-08T16:09:38.494248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:50.417249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"76'3","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.646022+0000","last_change":"2023-10-08T16:09:37.650502+0000","last_active":"2023-10-08T16:09:38.646022+0000","last_peered":"2023-10-08T16:09:38.646022+0000","last_clean":"2023-10-08T16:09:38.646022+0000","last_became_active":"2023-10-08T16:09:37.650237+0000","last_became_peered":"2023-10-08T16:09:37.650237+0000","last_unstale":"2023-10-08T16:09:38.646022+0000","last_undegraded":"2023-10-08T16:09:38.646022+0000","last_fullsized":"2023-10-08T16:09:38.646022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.145714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496311+0000","last_change":"2023-10-08T16:09:38.496311+0000","last_active":"2023-10-08T16:09:38.496311+0000","last_peered":"2023-10-08T16:09:38.496311+0000","last_clean":"2023-10-08T16:09:38.496311+0000","last_became_active":"2023-10-08T16:09:38.496064+0000","last_became_peered":"2023-10-08T16:09:38.496064+0000","last_unstale":"2023-10-08T16:09:38.496311+0000","last_undegraded":"2023-10-08T16:09:38.496311+0000","last_fullsized":"2023-10-08T16:09:38.496311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:29.659461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'6","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.647285+0000","last_change":"2023-10-08T16:09:37.645549+0000","last_active":"2023-10-08T16:09:38.647285+0000","last_peered":"2023-10-08T16:09:38.647285+0000","last_clean":"2023-10-08T16:09:38.647285+0000","last_became_active":"2023-10-08T16:09:37.645260+0000","last_became_peered":"2023-10-08T16:09:37.645260+0000","last_unstale":"2023-10-08T16:09:38.647285+0000","last_undegraded":"2023-10-08T16:09:38.647285+0000","last_fullsized":"2023-10-08T16:09:38.647285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:22.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491462+0000","last_change":"2023-10-08T16:09:38.491462+0000","last_active":"2023-10-08T16:09:38.491462+0000","last_peered":"2023-10-08T16:09:38.491462+0000","last_clean":"2023-10-08T16:09:38.491462+0000","last_became_active":"2023-10-08T16:09:38.491199+0000","last_became_peered":"2023-10-08T16:09:38.491199+0000","last_unstale":"2023-10-08T16:09:38.491462+0000","last_undegraded":"2023-10-08T16:09:38.491462+0000","last_fullsized":"2023-10-08T16:09:38.491462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:27.529093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648931+0000","last_change":"2023-10-08T16:09:37.651232+0000","last_active":"2023-10-08T16:09:38.648931+0000","last_peered":"2023-10-08T16:09:38.648931+0000","last_clean":"2023-10-08T16:09:38.648931+0000","last_became_active":"2023-10-08T16:09:37.650978+0000","last_became_peered":"2023-10-08T16:09:37.650978+0000","last_unstale":"2023-10-08T16:09:38.648931+0000","last_undegraded":"2023-10-08T16:09:38.648931+0000","last_fullsized":"2023-10-08T16:09:38.648931+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.877996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499508+0000","last_change":"2023-10-08T16:09:38.499508+0000","last_active":"2023-10-08T16:09:38.499508+0000","last_peered":"2023-10-08T16:09:38.499508+0000","last_clean":"2023-10-08T16:09:38.499508+0000","last_became_active":"2023-10-08T16:09:38.499216+0000","last_became_peered":"2023-10-08T16:09:38.499216+0000","last_unstale":"2023-10-08T16:09:38.499508+0000","last_undegraded":"2023-10-08T16:09:38.499508+0000","last_fullsized":"2023-10-08T16:09:38.499508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:06.424722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.651497+0000","last_change":"2023-10-08T16:09:37.644138+0000","last_active":"2023-10-08T16:09:37.651497+0000","last_peered":"2023-10-08T16:09:37.651497+0000","last_clean":"2023-10-08T16:09:37.651497+0000","last_became_active":"2023-10-08T16:09:37.643901+0000","last_became_peered":"2023-10-08T16:09:37.643901+0000","last_unstale":"2023-10-08T16:09:37.651497+0000","last_undegraded":"2023-10-08T16:09:37.651497+0000","last_fullsized":"2023-10-08T16:09:37.651497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:49.412986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490153+0000","last_change":"2023-10-08T16:09:38.490153+0000","last_active":"2023-10-08T16:09:38.490153+0000","last_peered":"2023-10-08T16:09:38.490153+0000","last_clean":"2023-10-08T16:09:38.490153+0000","last_became_active":"2023-10-08T16:09:38.489892+0000","last_became_peered":"2023-10-08T16:09:38.489892+0000","last_unstale":"2023-10-08T16:09:38.490153+0000","last_undegraded":"2023-10-08T16:09:38.490153+0000","last_fullsized":"2023-10-08T16:09:38.490153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:21.550265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652564+0000","last_change":"2023-10-08T16:09:37.650980+0000","last_active":"2023-10-08T16:09:37.652564+0000","last_peered":"2023-10-08T16:09:37.652564+0000","last_clean":"2023-10-08T16:09:37.652564+0000","last_became_active":"2023-10-08T16:09:37.650691+0000","last_became_peered":"2023-10-08T16:09:37.650691+0000","last_unstale":"2023-10-08T16:09:37.652564+0000","last_undegraded":"2023-10-08T16:09:37.652564+0000","last_fullsized":"2023-10-08T16:09:37.652564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:16.411106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.489475+0000","last_change":"2023-10-08T16:09:38.489475+0000","last_active":"2023-10-08T16:09:38.489475+0000","last_peered":"2023-10-08T16:09:38.489475+0000","last_clean":"2023-10-08T16:09:38.489475+0000","last_became_active":"2023-10-08T16:09:38.489208+0000","last_became_peered":"2023-10-08T16:09:38.489208+0000","last_unstale":"2023-10-08T16:09:38.489475+0000","last_undegraded":"2023-10-08T16:09:38.489475+0000","last_fullsized":"2023-10-08T16:09:38.489475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:20:43.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"76'8","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.649183+0000","last_change":"2023-10-08T16:09:37.647952+0000","last_active":"2023-10-08T16:09:38.649183+0000","last_peered":"2023-10-08T16:09:38.649183+0000","last_clean":"2023-10-08T16:09:38.649183+0000","last_became_active":"2023-10-08T16:09:37.647689+0000","last_became_peered":"2023-10-08T16:09:37.647689+0000","last_unstale":"2023-10-08T16:09:38.649183+0000","last_undegraded":"2023-10-08T16:09:38.649183+0000","last_fullsized":"2023-10-08T16:09:38.649183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:00:00.643719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499493+0000","last_change":"2023-10-08T16:09:38.499493+0000","last_active":"2023-10-08T16:09:38.499493+0000","last_peered":"2023-10-08T16:09:38.499493+0000","last_clean":"2023-10-08T16:09:38.499493+0000","last_became_active":"2023-10-08T16:09:38.499227+0000","last_became_peered":"2023-10-08T16:09:38.499227+0000","last_unstale":"2023-10-08T16:09:38.499493+0000","last_undegraded":"2023-10-08T16:09:38.499493+0000","last_fullsized":"2023-10-08T16:09:38.499493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:12.815590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484295+0000","last_change":"2023-10-08T16:09:37.648387+0000","last_active":"2023-10-08T16:09:38.484295+0000","last_peered":"2023-10-08T16:09:38.484295+0000","last_clean":"2023-10-08T16:09:38.484295+0000","last_became_active":"2023-10-08T16:09:37.647727+0000","last_became_peered":"2023-10-08T16:09:37.647727+0000","last_unstale":"2023-10-08T16:09:38.484295+0000","last_undegraded":"2023-10-08T16:09:38.484295+0000","last_fullsized":"2023-10-08T16:09:38.484295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.877996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498115+0000","last_change":"2023-10-08T16:09:38.498115+0000","last_active":"2023-10-08T16:09:38.498115+0000","last_peered":"2023-10-08T16:09:38.498115+0000","last_clean":"2023-10-08T16:09:38.498115+0000","last_became_active":"2023-10-08T16:09:38.497853+0000","last_became_peered":"2023-10-08T16:09:38.497853+0000","last_unstale":"2023-10-08T16:09:38.498115+0000","last_undegraded":"2023-10-08T16:09:38.498115+0000","last_fullsized":"2023-10-08T16:09:38.498115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:23.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.651102+0000","last_change":"2023-10-08T16:09:37.655061+0000","last_active":"2023-10-08T16:09:38.651102+0000","last_peered":"2023-10-08T16:09:38.651102+0000","last_clean":"2023-10-08T16:09:38.651102+0000","last_became_active":"2023-10-08T16:09:37.654736+0000","last_became_peered":"2023-10-08T16:09:37.654736+0000","last_unstale":"2023-10-08T16:09:38.651102+0000","last_undegraded":"2023-10-08T16:09:38.651102+0000","last_fullsized":"2023-10-08T16:09:38.651102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:20.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491437+0000","last_change":"2023-10-08T16:09:38.491437+0000","last_active":"2023-10-08T16:09:38.491437+0000","last_peered":"2023-10-08T16:09:38.491437+0000","last_clean":"2023-10-08T16:09:38.491437+0000","last_became_active":"2023-10-08T16:09:38.490823+0000","last_became_peered":"2023-10-08T16:09:38.490823+0000","last_unstale":"2023-10-08T16:09:38.491437+0000","last_undegraded":"2023-10-08T16:09:38.491437+0000","last_fullsized":"2023-10-08T16:09:38.491437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:28.044242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652659+0000","last_change":"2023-10-08T16:09:37.648637+0000","last_active":"2023-10-08T16:09:37.652659+0000","last_peered":"2023-10-08T16:09:37.652659+0000","last_clean":"2023-10-08T16:09:37.652659+0000","last_became_active":"2023-10-08T16:09:37.648368+0000","last_became_peered":"2023-10-08T16:09:37.648368+0000","last_unstale":"2023-10-08T16:09:37.652659+0000","last_undegraded":"2023-10-08T16:09:37.652659+0000","last_fullsized":"2023-10-08T16:09:37.652659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:10.029630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494592+0000","last_change":"2023-10-08T16:09:38.494592+0000","last_active":"2023-10-08T16:09:38.494592+0000","last_peered":"2023-10-08T16:09:38.494592+0000","last_clean":"2023-10-08T16:09:38.494592+0000","last_became_active":"2023-10-08T16:09:38.493943+0000","last_became_peered":"2023-10-08T16:09:38.493943+0000","last_unstale":"2023-10-08T16:09:38.494592+0000","last_undegraded":"2023-10-08T16:09:38.494592+0000","last_fullsized":"2023-10-08T16:09:38.494592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:05.175830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.646561+0000","last_change":"2023-10-08T16:09:37.646528+0000","last_active":"2023-10-08T16:09:38.646561+0000","last_peered":"2023-10-08T16:09:38.646561+0000","last_clean":"2023-10-08T16:09:38.646561+0000","last_became_active":"2023-10-08T16:09:37.646269+0000","last_became_peered":"2023-10-08T16:09:37.646269+0000","last_unstale":"2023-10-08T16:09:38.646561+0000","last_undegraded":"2023-10-08T16:09:38.646561+0000","last_fullsized":"2023-10-08T16:09:38.646561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:28.655198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484168+0000","last_change":"2023-10-08T16:09:38.484168+0000","last_active":"2023-10-08T16:09:38.484168+0000","last_peered":"2023-10-08T16:09:38.484168+0000","last_clean":"2023-10-08T16:09:38.484168+0000","last_became_active":"2023-10-08T16:09:38.483896+0000","last_became_peered":"2023-10-08T16:09:38.483896+0000","last_unstale":"2023-10-08T16:09:38.484168+0000","last_undegraded":"2023-10-08T16:09:38.484168+0000","last_fullsized":"2023-10-08T16:09:38.484168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:06.283276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653678+0000","last_change":"2023-10-08T16:09:37.652625+0000","last_active":"2023-10-08T16:09:37.653678+0000","last_peered":"2023-10-08T16:09:37.653678+0000","last_clean":"2023-10-08T16:09:37.653678+0000","last_became_active":"2023-10-08T16:09:37.652366+0000","last_became_peered":"2023-10-08T16:09:37.652366+0000","last_unstale":"2023-10-08T16:09:37.653678+0000","last_undegraded":"2023-10-08T16:09:37.653678+0000","last_fullsized":"2023-10-08T16:09:37.653678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:53:18.541603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488834+0000","last_change":"2023-10-08T16:09:38.488834+0000","last_active":"2023-10-08T16:09:38.488834+0000","last_peered":"2023-10-08T16:09:38.488834+0000","last_clean":"2023-10-08T16:09:38.488834+0000","last_became_active":"2023-10-08T16:09:38.488580+0000","last_became_peered":"2023-10-08T16:09:38.488580+0000","last_unstale":"2023-10-08T16:09:38.488834+0000","last_undegraded":"2023-10-08T16:09:38.488834+0000","last_fullsized":"2023-10-08T16:09:38.488834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:50.484122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653701+0000","last_change":"2023-10-08T16:09:37.653080+0000","last_active":"2023-10-08T16:09:37.653701+0000","last_peered":"2023-10-08T16:09:37.653701+0000","last_clean":"2023-10-08T16:09:37.653701+0000","last_became_active":"2023-10-08T16:09:37.652801+0000","last_became_peered":"2023-10-08T16:09:37.652801+0000","last_unstale":"2023-10-08T16:09:37.653701+0000","last_undegraded":"2023-10-08T16:09:37.653701+0000","last_fullsized":"2023-10-08T16:09:37.653701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:12.890633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485601+0000","last_change":"2023-10-08T16:09:38.485601+0000","last_active":"2023-10-08T16:09:38.485601+0000","last_peered":"2023-10-08T16:09:38.485601+0000","last_clean":"2023-10-08T16:09:38.485601+0000","last_became_active":"2023-10-08T16:09:38.485299+0000","last_became_peered":"2023-10-08T16:09:38.485299+0000","last_unstale":"2023-10-08T16:09:38.485601+0000","last_undegraded":"2023-10-08T16:09:38.485601+0000","last_fullsized":"2023-10-08T16:09:38.485601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:28.044242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.651315+0000","last_change":"2023-10-08T16:09:37.647068+0000","last_active":"2023-10-08T16:09:37.651315+0000","last_peered":"2023-10-08T16:09:37.651315+0000","last_clean":"2023-10-08T16:09:37.651315+0000","last_became_active":"2023-10-08T16:09:37.646521+0000","last_became_peered":"2023-10-08T16:09:37.646521+0000","last_unstale":"2023-10-08T16:09:37.651315+0000","last_undegraded":"2023-10-08T16:09:37.651315+0000","last_fullsized":"2023-10-08T16:09:37.651315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:26.524830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495701+0000","last_change":"2023-10-08T16:09:38.495701+0000","last_active":"2023-10-08T16:09:38.495701+0000","last_peered":"2023-10-08T16:09:38.495701+0000","last_clean":"2023-10-08T16:09:38.495701+0000","last_became_active":"2023-10-08T16:09:38.495370+0000","last_became_peered":"2023-10-08T16:09:38.495370+0000","last_unstale":"2023-10-08T16:09:38.495701+0000","last_undegraded":"2023-10-08T16:09:38.495701+0000","last_fullsized":"2023-10-08T16:09:38.495701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:49.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.645884+0000","last_change":"2023-10-08T16:09:37.644030+0000","last_active":"2023-10-08T16:09:38.645884+0000","last_peered":"2023-10-08T16:09:38.645884+0000","last_clean":"2023-10-08T16:09:38.645884+0000","last_became_active":"2023-10-08T16:09:37.643771+0000","last_became_peered":"2023-10-08T16:09:37.643771+0000","last_unstale":"2023-10-08T16:09:38.645884+0000","last_undegraded":"2023-10-08T16:09:38.645884+0000","last_fullsized":"2023-10-08T16:09:38.645884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:57.831073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.501228+0000","last_change":"2023-10-08T16:09:38.501228+0000","last_active":"2023-10-08T16:09:38.501228+0000","last_peered":"2023-10-08T16:09:38.501228+0000","last_clean":"2023-10-08T16:09:38.501228+0000","last_became_active":"2023-10-08T16:09:38.500954+0000","last_became_peered":"2023-10-08T16:09:38.500954+0000","last_unstale":"2023-10-08T16:09:38.501228+0000","last_undegraded":"2023-10-08T16:09:38.501228+0000","last_fullsized":"2023-10-08T16:09:38.501228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:12.822712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653402+0000","last_change":"2023-10-08T16:09:37.651108+0000","last_active":"2023-10-08T16:09:37.653402+0000","last_peered":"2023-10-08T16:09:37.653402+0000","last_clean":"2023-10-08T16:09:37.653402+0000","last_became_active":"2023-10-08T16:09:37.650863+0000","last_became_peered":"2023-10-08T16:09:37.650863+0000","last_unstale":"2023-10-08T16:09:37.653402+0000","last_undegraded":"2023-10-08T16:09:37.653402+0000","last_fullsized":"2023-10-08T16:09:37.653402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:40.160762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494545+0000","last_change":"2023-10-08T16:09:38.494545+0000","last_active":"2023-10-08T16:09:38.494545+0000","last_peered":"2023-10-08T16:09:38.494545+0000","last_clean":"2023-10-08T16:09:38.494545+0000","last_became_active":"2023-10-08T16:09:38.493741+0000","last_became_peered":"2023-10-08T16:09:38.493741+0000","last_unstale":"2023-10-08T16:09:38.494545+0000","last_undegraded":"2023-10-08T16:09:38.494545+0000","last_fullsized":"2023-10-08T16:09:38.494545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:30.837089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652702+0000","last_change":"2023-10-08T16:09:37.648127+0000","last_active":"2023-10-08T16:09:37.652702+0000","last_peered":"2023-10-08T16:09:37.652702+0000","last_clean":"2023-10-08T16:09:37.652702+0000","last_became_active":"2023-10-08T16:09:37.647693+0000","last_became_peered":"2023-10-08T16:09:37.647693+0000","last_unstale":"2023-10-08T16:09:37.652702+0000","last_undegraded":"2023-10-08T16:09:37.652702+0000","last_fullsized":"2023-10-08T16:09:37.652702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:43:16.723311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491937+0000","last_change":"2023-10-08T16:09:38.491937+0000","last_active":"2023-10-08T16:09:38.491937+0000","last_peered":"2023-10-08T16:09:38.491937+0000","last_clean":"2023-10-08T16:09:38.491937+0000","last_became_active":"2023-10-08T16:09:38.491620+0000","last_became_peered":"2023-10-08T16:09:38.491620+0000","last_unstale":"2023-10-08T16:09:38.491937+0000","last_undegraded":"2023-10-08T16:09:38.491937+0000","last_fullsized":"2023-10-08T16:09:38.491937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:33.553406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655295+0000","last_change":"2023-10-08T16:09:37.653427+0000","last_active":"2023-10-08T16:09:37.655295+0000","last_peered":"2023-10-08T16:09:37.655295+0000","last_clean":"2023-10-08T16:09:37.655295+0000","last_became_active":"2023-10-08T16:09:37.653157+0000","last_became_peered":"2023-10-08T16:09:37.653157+0000","last_unstale":"2023-10-08T16:09:37.655295+0000","last_undegraded":"2023-10-08T16:09:37.655295+0000","last_fullsized":"2023-10-08T16:09:37.655295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:00.229704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484278+0000","last_change":"2023-10-08T16:09:38.484278+0000","last_active":"2023-10-08T16:09:38.484278+0000","last_peered":"2023-10-08T16:09:38.484278+0000","last_clean":"2023-10-08T16:09:38.484278+0000","last_became_active":"2023-10-08T16:09:38.483819+0000","last_became_peered":"2023-10-08T16:09:38.483819+0000","last_unstale":"2023-10-08T16:09:38.484278+0000","last_undegraded":"2023-10-08T16:09:38.484278+0000","last_fullsized":"2023-10-08T16:09:38.484278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:00:01.647982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652642+0000","last_change":"2023-10-08T16:09:37.648809+0000","last_active":"2023-10-08T16:09:37.652642+0000","last_peered":"2023-10-08T16:09:37.652642+0000","last_clean":"2023-10-08T16:09:37.652642+0000","last_became_active":"2023-10-08T16:09:37.648393+0000","last_became_peered":"2023-10-08T16:09:37.648393+0000","last_unstale":"2023-10-08T16:09:37.652642+0000","last_undegraded":"2023-10-08T16:09:37.652642+0000","last_fullsized":"2023-10-08T16:09:37.652642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:21.519838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496295+0000","last_change":"2023-10-08T16:09:38.496295+0000","last_active":"2023-10-08T16:09:38.496295+0000","last_peered":"2023-10-08T16:09:38.496295+0000","last_clean":"2023-10-08T16:09:38.496295+0000","last_became_active":"2023-10-08T16:09:38.496040+0000","last_became_peered":"2023-10-08T16:09:38.496040+0000","last_unstale":"2023-10-08T16:09:38.496295+0000","last_undegraded":"2023-10-08T16:09:38.496295+0000","last_fullsized":"2023-10-08T16:09:38.496295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:29.659461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655897+0000","last_change":"2023-10-08T16:09:37.654516+0000","last_active":"2023-10-08T16:09:37.655897+0000","last_peered":"2023-10-08T16:09:37.655897+0000","last_clean":"2023-10-08T16:09:37.655897+0000","last_became_active":"2023-10-08T16:09:37.653225+0000","last_became_peered":"2023-10-08T16:09:37.653225+0000","last_unstale":"2023-10-08T16:09:37.655897+0000","last_undegraded":"2023-10-08T16:09:37.655897+0000","last_fullsized":"2023-10-08T16:09:37.655897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:16.411106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496751+0000","last_change":"2023-10-08T16:09:38.496751+0000","last_active":"2023-10-08T16:09:38.496751+0000","last_peered":"2023-10-08T16:09:38.496751+0000","last_clean":"2023-10-08T16:09:38.496751+0000","last_became_active":"2023-10-08T16:09:38.496327+0000","last_became_peered":"2023-10-08T16:09:38.496327+0000","last_unstale":"2023-10-08T16:09:38.496751+0000","last_undegraded":"2023-10-08T16:09:38.496751+0000","last_fullsized":"2023-10-08T16:09:38.496751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:50.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653475+0000","last_change":"2023-10-08T16:09:37.651318+0000","last_active":"2023-10-08T16:09:37.653475+0000","last_peered":"2023-10-08T16:09:37.653475+0000","last_clean":"2023-10-08T16:09:37.653475+0000","last_became_active":"2023-10-08T16:09:37.651084+0000","last_became_peered":"2023-10-08T16:09:37.651084+0000","last_unstale":"2023-10-08T16:09:37.653475+0000","last_undegraded":"2023-10-08T16:09:37.653475+0000","last_fullsized":"2023-10-08T16:09:37.653475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:06:59.487257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487932+0000","last_change":"2023-10-08T16:09:38.487932+0000","last_active":"2023-10-08T16:09:38.487932+0000","last_peered":"2023-10-08T16:09:38.487932+0000","last_clean":"2023-10-08T16:09:38.487932+0000","last_became_active":"2023-10-08T16:09:38.487324+0000","last_became_peered":"2023-10-08T16:09:38.487324+0000","last_unstale":"2023-10-08T16:09:38.487932+0000","last_undegraded":"2023-10-08T16:09:38.487932+0000","last_fullsized":"2023-10-08T16:09:38.487932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:20:39.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655880+0000","last_change":"2023-10-08T16:09:37.654448+0000","last_active":"2023-10-08T16:09:37.655880+0000","last_peered":"2023-10-08T16:09:37.655880+0000","last_clean":"2023-10-08T16:09:37.655880+0000","last_became_active":"2023-10-08T16:09:37.654173+0000","last_became_peered":"2023-10-08T16:09:37.654173+0000","last_unstale":"2023-10-08T16:09:37.655880+0000","last_undegraded":"2023-10-08T16:09:37.655880+0000","last_fullsized":"2023-10-08T16:09:37.655880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:29.832826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.483118+0000","last_change":"2023-10-08T16:09:38.483118+0000","last_active":"2023-10-08T16:09:38.483118+0000","last_peered":"2023-10-08T16:09:38.483118+0000","last_clean":"2023-10-08T16:09:38.483118+0000","last_became_active":"2023-10-08T16:09:38.482869+0000","last_became_peered":"2023-10-08T16:09:38.482869+0000","last_unstale":"2023-10-08T16:09:38.483118+0000","last_undegraded":"2023-10-08T16:09:38.483118+0000","last_fullsized":"2023-10-08T16:09:38.483118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:23.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490411+0000","last_change":"2023-10-08T16:09:37.651788+0000","last_active":"2023-10-08T16:09:38.490411+0000","last_peered":"2023-10-08T16:09:38.490411+0000","last_clean":"2023-10-08T16:09:38.490411+0000","last_became_active":"2023-10-08T16:09:37.651457+0000","last_became_peered":"2023-10-08T16:09:37.651457+0000","last_unstale":"2023-10-08T16:09:38.490411+0000","last_undegraded":"2023-10-08T16:09:38.490411+0000","last_fullsized":"2023-10-08T16:09:38.490411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:23.505403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":210,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.640097+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:09:37.640097+0000","last_peered":"2023-10-08T16:09:37.640097+0000","last_clean":"2023-10-08T16:09:37.640097+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:09:37.640097+0000","last_undegraded":"2023-10-08T16:09:37.640097+0000","last_fullsized":"2023-10-08T16:09:37.640097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298784,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073056,"statfs":{"total":96636764160,"available":96330809344,"internally_reserved":0,"allocated":1531904,"data_stored":990066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.74099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.497},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.61099999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":400294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.313,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.629,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66700000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48299999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67300000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.503}]},{"osd":3,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65800000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73699999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.378}]},{"osd":4,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.502,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77100000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.50600000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79400000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76200000000000001,"5min":0.874,"15min":0.874},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81899999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":399822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.309,"15min":0.311},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.377,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.215},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.41799999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.34000000000000002,"15min":0.36199999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.39500000000000002,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.373},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.43099999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36499999999999999,"15min":0.39100000000000001},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.36699999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.34200000000000003,"15min":0.36299999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46200000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66900000000000004,"5min":1.077,"15min":1.077},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79300000000000004,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.696}]},{"osd":5,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55100000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.302,"15min":1.302},"last":1.657},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":1.099,"15min":1.099},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":1.268,"15min":1.268},"last":1.7310000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76400000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302880,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068960,"statfs":{"total":96636764160,"available":96326615040,"internally_reserved":0,"allocated":1531904,"data_stored":990030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.44700000000000001,"15min":0.57699999999999996},"min":{"1min":0.21299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45500000000000002,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.41099999999999998,"15min":0.497},"min":{"1min":0.22700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51200000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.47199999999999998,"15min":0.61199999999999999},"min":{"1min":0.22900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53000000000000003,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.41599999999999998,"15min":0.504},"min":{"1min":0.22600000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.48499999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.372},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.50800000000000001,"15min":0.64600000000000002},"min":{"1min":0.28100000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58999999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.68400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61499999999999999,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.68899999999999995,"15min":0.82099999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83399999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.48399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.84299999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86299999999999999,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.84699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68000000000000005,"15min":0.76000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":937984,"data_stored":400182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.379},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.34699999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.435},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":401164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.26800000000000002,"5min":0.33400000000000002,"15min":0.35899999999999999},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.379,"5min":1.238,"15min":1.238},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.36399999999999999,"15min":0.39900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40000000000000002,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.34499999999999997,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45000000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38200000000000001,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.40500000000000003,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.38200000000000001,"15min":0.40200000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.42899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.39900000000000002,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.432,"5min":1.327,"15min":1.327},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.61199999999999999},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":1.089,"15min":1.089},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56200000000000006,"15min":0.58999999999999997},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":1.27,"15min":1.27},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.755,"5min":1.879,"15min":1.879},"last":0.71999999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61799999999999999,"15min":0.65000000000000002},"min":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":1.825,"15min":1.825},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":1.115},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":399628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.379}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.497}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":401186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.436},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.30099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.375},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.502}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:09:42.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:09:42.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.836+0000 7feea6a34700 1 Processor -- start 2023-10-08T16:09:42.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.836+0000 7feea6a34700 1 -- start start 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7feea6a34700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feea0159890 0x7feea015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7feea6a34700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea015a160 con 0x7feea0159c60 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7feea6a34700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00d13a0 con 0x7feea0069560 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7feea6a34700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea015e140 con 0x7feea0159890 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7fee9ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feea0159890 0x7feea015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7fee9f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7feea0159c60 0x7feea015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46880/0 (socket says 172.21.15.170:46880) 2023-10-08T16:09:42.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.837+0000 7fee9f7fe700 1 -- 172.21.15.170:0/541303999 learned_addr learned my addr 172.21.15.170:0/541303999 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9ffff700 1 -- 172.21.15.170:0/541303999 >> v1:172.21.15.191:6789/0 conn(0x7feea0069560 legacy=0x7feea00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9ffff700 1 -- 172.21.15.170:0/541303999 >> v1:172.21.15.170:6789/0 conn(0x7feea0159c60 legacy=0x7feea015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9ffff700 1 -- 172.21.15.170:0/541303999 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea015a3d0 con 0x7feea0159890 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9ffff700 1 --2- 172.21.15.170:0/541303999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159890 0x7feea015abd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fee900098e0 tx=0x7fee900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf290729e0b1730 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9effd700 1 -- 172.21.15.170:0/541303999 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90009e80 con 0x7feea0159890 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9effd700 1 -- 172.21.15.170:0/541303999 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9000b860 con 0x7feea0159890 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7fee9effd700 1 -- 172.21.15.170:0/541303999 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000bbe0 con 0x7feea0159890 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 -- 172.21.15.170:0/541303999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159890 msgr2=0x7feea015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 --2- 172.21.15.170:0/541303999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159890 0x7feea015abd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fee900098e0 tx=0x7fee900091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 -- 172.21.15.170:0/541303999 shutdown_connections 2023-10-08T16:09:42.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 --2- 172.21.15.170:0/541303999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159890 0x7feea015abd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 -- 172.21.15.170:0/541303999 >> 172.21.15.170:0/541303999 conn(0x7feea0068e20 msgr2=0x7feea00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:42.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 -- 172.21.15.170:0/541303999 shutdown_connections 2023-10-08T16:09:42.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.838+0000 7feea6a34700 1 -- 172.21.15.170:0/541303999 wait complete. 2023-10-08T16:09:42.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.839+0000 7feea6a34700 1 Processor -- start 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 -- start start 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feea0069560 0x7feea00b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 0x7feea014d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00b3f50 con 0x7feea0159c60 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00b4140 con 0x7feea0069560 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.840+0000 7feea6a34700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00b4420 con 0x7feea0159890 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47740/0 (socket says 172.21.15.170:47740) 2023-10-08T16:09:42.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 -- 172.21.15.170:0/175782575 learned_addr learned my addr 172.21.15.170:0/175782575 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee9ffff700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feea0069560 0x7feea00b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee9f7fe700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 0x7feea014d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 -- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 msgr2=0x7feea014d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 0x7feea014d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 -- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feea0069560 msgr2=0x7feea00b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feea0069560 0x7feea00b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 -- 172.21.15.170:0/175782575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee90009420 con 0x7feea0159c60 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee9f7fe700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 0x7feea014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea4fd1700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c8e0 tx=0x7fee98001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ce80 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9800a7e0 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea00b4700 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ab80 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.841+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea00b5720 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.842+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea00b4f00 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.842+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fee98012420 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.843+0000 7fee87fff700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 0x7fee8c069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.843+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee98084b50 con 0x7feea0159c60 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.843+0000 7fee9ffff700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 0x7fee8c069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:42.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.843+0000 7fee9ffff700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 0x7fee8c069650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee900093f0 tx=0x7fee90017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:42.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:42.847+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feea00b4f00 con 0x7feea0159c60 2023-10-08T16:09:43.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.025+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feea00b4f00 con 0x7fee8c0671a0 2023-10-08T16:09:43.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.033+0000 7fee87fff700 1 -- 172.21.15.170:0/175782575 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289284 (secure 0 0 0) 0x7feea00b4f00 con 0x7fee8c0671a0 2023-10-08T16:09:43.037 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 msgr2=0x7fee8c069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 0x7fee8c069650 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fee900093f0 tx=0x7fee90017000 comp rx=0 tx=0).stop 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 msgr2=0x7feea00cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c8e0 tx=0x7fee98001710 comp rx=0 tx=0).stop 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 shutdown_connections 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fee8c0671a0 0x7fee8c069650 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feea0159c60 0x7feea00cea00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feea0069560 0x7feea00b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 --2- 172.21.15.170:0/175782575 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feea0159890 0x7feea014d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 >> 172.21.15.170:0/175782575 conn(0x7feea0068e20 msgr2=0x7feea00b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:43.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 shutdown_connections 2023-10-08T16:09:43.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.036+0000 7feea6a34700 1 -- 172.21.15.170:0/175782575 wait complete. 2023-10-08T16:09:43.041 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:09:43.050 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2023-10-08T16:09:42.288027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2403208,"kb_used_data":8520,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752571512,"statfs":{"total":773094113280,"available":770633228288,"internally_reserved":0,"allocated":8724480,"data_stored":4382372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.658269"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495408+0000","last_change":"2023-10-08T16:09:38.495408+0000","last_active":"2023-10-08T16:09:38.495408+0000","last_peered":"2023-10-08T16:09:38.495408+0000","last_clean":"2023-10-08T16:09:38.495408+0000","last_became_active":"2023-10-08T16:09:38.495154+0000","last_became_peered":"2023-10-08T16:09:38.495154+0000","last_unstale":"2023-10-08T16:09:38.495408+0000","last_undegraded":"2023-10-08T16:09:38.495408+0000","last_fullsized":"2023-10-08T16:09:38.495408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.919970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488706+0000","last_change":"2023-10-08T16:09:38.488706+0000","last_active":"2023-10-08T16:09:38.488706+0000","last_peered":"2023-10-08T16:09:38.488706+0000","last_clean":"2023-10-08T16:09:38.488706+0000","last_became_active":"2023-10-08T16:09:38.488460+0000","last_became_peered":"2023-10-08T16:09:38.488460+0000","last_unstale":"2023-10-08T16:09:38.488706+0000","last_undegraded":"2023-10-08T16:09:38.488706+0000","last_fullsized":"2023-10-08T16:09:38.488706+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:20:39.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.501141+0000","last_change":"2023-10-08T16:09:38.501141+0000","last_active":"2023-10-08T16:09:38.501141+0000","last_peered":"2023-10-08T16:09:38.501141+0000","last_clean":"2023-10-08T16:09:38.501141+0000","last_became_active":"2023-10-08T16:09:38.500894+0000","last_became_peered":"2023-10-08T16:09:38.500894+0000","last_unstale":"2023-10-08T16:09:38.501141+0000","last_undegraded":"2023-10-08T16:09:38.501141+0000","last_fullsized":"2023-10-08T16:09:38.501141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:05.398877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.492081+0000","last_change":"2023-10-08T16:09:38.492081+0000","last_active":"2023-10-08T16:09:38.492081+0000","last_peered":"2023-10-08T16:09:38.492081+0000","last_clean":"2023-10-08T16:09:38.492081+0000","last_became_active":"2023-10-08T16:09:38.491482+0000","last_became_peered":"2023-10-08T16:09:38.491482+0000","last_unstale":"2023-10-08T16:09:38.492081+0000","last_undegraded":"2023-10-08T16:09:38.492081+0000","last_fullsized":"2023-10-08T16:09:38.492081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:58.912361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.497164+0000","last_change":"2023-10-08T16:09:38.497164+0000","last_active":"2023-10-08T16:09:38.497164+0000","last_peered":"2023-10-08T16:09:38.497164+0000","last_clean":"2023-10-08T16:09:38.497164+0000","last_became_active":"2023-10-08T16:09:38.496680+0000","last_became_peered":"2023-10-08T16:09:38.496680+0000","last_unstale":"2023-10-08T16:09:38.497164+0000","last_undegraded":"2023-10-08T16:09:38.497164+0000","last_fullsized":"2023-10-08T16:09:38.497164+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:50.417249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490002+0000","last_change":"2023-10-08T16:09:38.490002+0000","last_active":"2023-10-08T16:09:38.490002+0000","last_peered":"2023-10-08T16:09:38.490002+0000","last_clean":"2023-10-08T16:09:38.490002+0000","last_became_active":"2023-10-08T16:09:38.489725+0000","last_became_peered":"2023-10-08T16:09:38.489725+0000","last_unstale":"2023-10-08T16:09:38.490002+0000","last_undegraded":"2023-10-08T16:09:38.490002+0000","last_fullsized":"2023-10-08T16:09:38.490002+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:24.509666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498791+0000","last_change":"2023-10-08T16:09:38.498791+0000","last_active":"2023-10-08T16:09:38.498791+0000","last_peered":"2023-10-08T16:09:38.498791+0000","last_clean":"2023-10-08T16:09:38.498791+0000","last_became_active":"2023-10-08T16:09:38.498517+0000","last_became_peered":"2023-10-08T16:09:38.498517+0000","last_unstale":"2023-10-08T16:09:38.498791+0000","last_undegraded":"2023-10-08T16:09:38.498791+0000","last_fullsized":"2023-10-08T16:09:38.498791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:51.956068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485583+0000","last_change":"2023-10-08T16:09:38.485583+0000","last_active":"2023-10-08T16:09:38.485583+0000","last_peered":"2023-10-08T16:09:38.485583+0000","last_clean":"2023-10-08T16:09:38.485583+0000","last_became_active":"2023-10-08T16:09:38.484866+0000","last_became_peered":"2023-10-08T16:09:38.484866+0000","last_unstale":"2023-10-08T16:09:38.485583+0000","last_undegraded":"2023-10-08T16:09:38.485583+0000","last_fullsized":"2023-10-08T16:09:38.485583+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:58.919970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485046+0000","last_change":"2023-10-08T16:09:38.485046+0000","last_active":"2023-10-08T16:09:38.485046+0000","last_peered":"2023-10-08T16:09:38.485046+0000","last_clean":"2023-10-08T16:09:38.485046+0000","last_became_active":"2023-10-08T16:09:38.484771+0000","last_became_peered":"2023-10-08T16:09:38.484771+0000","last_unstale":"2023-10-08T16:09:38.485046+0000","last_undegraded":"2023-10-08T16:09:38.485046+0000","last_fullsized":"2023-10-08T16:09:38.485046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:22.524101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487959+0000","last_change":"2023-10-08T16:09:38.487959+0000","last_active":"2023-10-08T16:09:38.487959+0000","last_peered":"2023-10-08T16:09:38.487959+0000","last_clean":"2023-10-08T16:09:38.487959+0000","last_became_active":"2023-10-08T16:09:38.487503+0000","last_became_peered":"2023-10-08T16:09:38.487503+0000","last_unstale":"2023-10-08T16:09:38.487959+0000","last_undegraded":"2023-10-08T16:09:38.487959+0000","last_fullsized":"2023-10-08T16:09:38.487959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:11.593957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.497822+0000","last_change":"2023-10-08T16:09:38.497822+0000","last_active":"2023-10-08T16:09:38.497822+0000","last_peered":"2023-10-08T16:09:38.497822+0000","last_clean":"2023-10-08T16:09:38.497822+0000","last_became_active":"2023-10-08T16:09:38.497575+0000","last_became_peered":"2023-10-08T16:09:38.497575+0000","last_unstale":"2023-10-08T16:09:38.497822+0000","last_undegraded":"2023-10-08T16:09:38.497822+0000","last_fullsized":"2023-10-08T16:09:38.497822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:30.837089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493776+0000","last_change":"2023-10-08T16:09:38.493776+0000","last_active":"2023-10-08T16:09:38.493776+0000","last_peered":"2023-10-08T16:09:38.493776+0000","last_clean":"2023-10-08T16:09:38.493776+0000","last_became_active":"2023-10-08T16:09:38.492998+0000","last_became_peered":"2023-10-08T16:09:38.492998+0000","last_unstale":"2023-10-08T16:09:38.493776+0000","last_undegraded":"2023-10-08T16:09:38.493776+0000","last_fullsized":"2023-10-08T16:09:38.493776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:20:43.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498663+0000","last_change":"2023-10-08T16:09:38.498663+0000","last_active":"2023-10-08T16:09:38.498663+0000","last_peered":"2023-10-08T16:09:38.498663+0000","last_clean":"2023-10-08T16:09:38.498663+0000","last_became_active":"2023-10-08T16:09:38.498304+0000","last_became_peered":"2023-10-08T16:09:38.498304+0000","last_unstale":"2023-10-08T16:09:38.498663+0000","last_undegraded":"2023-10-08T16:09:38.498663+0000","last_fullsized":"2023-10-08T16:09:38.498663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:57.588827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493730+0000","last_change":"2023-10-08T16:09:38.493730+0000","last_active":"2023-10-08T16:09:38.493730+0000","last_peered":"2023-10-08T16:09:38.493730+0000","last_clean":"2023-10-08T16:09:38.493730+0000","last_became_active":"2023-10-08T16:09:38.490622+0000","last_became_peered":"2023-10-08T16:09:38.490622+0000","last_unstale":"2023-10-08T16:09:38.493730+0000","last_undegraded":"2023-10-08T16:09:38.493730+0000","last_fullsized":"2023-10-08T16:09:38.493730+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:51.051576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499123+0000","last_change":"2023-10-08T16:09:38.499123+0000","last_active":"2023-10-08T16:09:38.499123+0000","last_peered":"2023-10-08T16:09:38.499123+0000","last_clean":"2023-10-08T16:09:38.499123+0000","last_became_active":"2023-10-08T16:09:38.498872+0000","last_became_peered":"2023-10-08T16:09:38.498872+0000","last_unstale":"2023-10-08T16:09:38.499123+0000","last_undegraded":"2023-10-08T16:09:38.499123+0000","last_fullsized":"2023-10-08T16:09:38.499123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:52.601103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493525+0000","last_change":"2023-10-08T16:09:38.493525+0000","last_active":"2023-10-08T16:09:38.493525+0000","last_peered":"2023-10-08T16:09:38.493525+0000","last_clean":"2023-10-08T16:09:38.493525+0000","last_became_active":"2023-10-08T16:09:38.493155+0000","last_became_peered":"2023-10-08T16:09:38.493155+0000","last_unstale":"2023-10-08T16:09:38.493525+0000","last_undegraded":"2023-10-08T16:09:38.493525+0000","last_fullsized":"2023-10-08T16:09:38.493525+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:02:42.762068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491414+0000","last_change":"2023-10-08T16:09:38.491414+0000","last_active":"2023-10-08T16:09:38.491414+0000","last_peered":"2023-10-08T16:09:38.491414+0000","last_clean":"2023-10-08T16:09:38.491414+0000","last_became_active":"2023-10-08T16:09:38.490646+0000","last_became_peered":"2023-10-08T16:09:38.490646+0000","last_unstale":"2023-10-08T16:09:38.491414+0000","last_undegraded":"2023-10-08T16:09:38.491414+0000","last_fullsized":"2023-10-08T16:09:38.491414+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:08.609700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494072+0000","last_change":"2023-10-08T16:09:38.494072+0000","last_active":"2023-10-08T16:09:38.494072+0000","last_peered":"2023-10-08T16:09:38.494072+0000","last_clean":"2023-10-08T16:09:38.494072+0000","last_became_active":"2023-10-08T16:09:38.493798+0000","last_became_peered":"2023-10-08T16:09:38.493798+0000","last_unstale":"2023-10-08T16:09:38.494072+0000","last_undegraded":"2023-10-08T16:09:38.494072+0000","last_fullsized":"2023-10-08T16:09:38.494072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:53.054694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.481708+0000","last_change":"2023-10-08T16:09:38.481708+0000","last_active":"2023-10-08T16:09:38.481708+0000","last_peered":"2023-10-08T16:09:38.481708+0000","last_clean":"2023-10-08T16:09:38.481708+0000","last_became_active":"2023-10-08T16:09:38.481376+0000","last_became_peered":"2023-10-08T16:09:38.481376+0000","last_unstale":"2023-10-08T16:09:38.481708+0000","last_undegraded":"2023-10-08T16:09:38.481708+0000","last_fullsized":"2023-10-08T16:09:38.481708+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:49.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490850+0000","last_change":"2023-10-08T16:09:38.490850+0000","last_active":"2023-10-08T16:09:38.490850+0000","last_peered":"2023-10-08T16:09:38.490850+0000","last_clean":"2023-10-08T16:09:38.490850+0000","last_became_active":"2023-10-08T16:09:38.490598+0000","last_became_peered":"2023-10-08T16:09:38.490598+0000","last_unstale":"2023-10-08T16:09:38.490850+0000","last_undegraded":"2023-10-08T16:09:38.490850+0000","last_fullsized":"2023-10-08T16:09:38.490850+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:59:55.340672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491568+0000","last_change":"2023-10-08T16:09:38.491568+0000","last_active":"2023-10-08T16:09:38.491568+0000","last_peered":"2023-10-08T16:09:38.491568+0000","last_clean":"2023-10-08T16:09:38.491568+0000","last_became_active":"2023-10-08T16:09:38.491178+0000","last_became_peered":"2023-10-08T16:09:38.491178+0000","last_unstale":"2023-10-08T16:09:38.491568+0000","last_undegraded":"2023-10-08T16:09:38.491568+0000","last_fullsized":"2023-10-08T16:09:38.491568+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:32:57.376194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.481489+0000","last_change":"2023-10-08T16:09:38.481489+0000","last_active":"2023-10-08T16:09:38.481489+0000","last_peered":"2023-10-08T16:09:38.481489+0000","last_clean":"2023-10-08T16:09:38.481489+0000","last_became_active":"2023-10-08T16:09:38.481226+0000","last_became_peered":"2023-10-08T16:09:38.481226+0000","last_unstale":"2023-10-08T16:09:38.481489+0000","last_undegraded":"2023-10-08T16:09:38.481489+0000","last_fullsized":"2023-10-08T16:09:38.481489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:15:44.366084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487663+0000","last_change":"2023-10-08T16:09:38.487663+0000","last_active":"2023-10-08T16:09:38.487663+0000","last_peered":"2023-10-08T16:09:38.487663+0000","last_clean":"2023-10-08T16:09:38.487663+0000","last_became_active":"2023-10-08T16:09:38.486857+0000","last_became_peered":"2023-10-08T16:09:38.486857+0000","last_unstale":"2023-10-08T16:09:38.487663+0000","last_undegraded":"2023-10-08T16:09:38.487663+0000","last_fullsized":"2023-10-08T16:09:38.487663+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:54.159859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493913+0000","last_change":"2023-10-08T16:09:38.493913+0000","last_active":"2023-10-08T16:09:38.493913+0000","last_peered":"2023-10-08T16:09:38.493913+0000","last_clean":"2023-10-08T16:09:38.493913+0000","last_became_active":"2023-10-08T16:09:38.493652+0000","last_became_peered":"2023-10-08T16:09:38.493652+0000","last_unstale":"2023-10-08T16:09:38.493913+0000","last_undegraded":"2023-10-08T16:09:38.493913+0000","last_fullsized":"2023-10-08T16:09:38.493913+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:52:00.184659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495647+0000","last_change":"2023-10-08T16:09:38.495647+0000","last_active":"2023-10-08T16:09:38.495647+0000","last_peered":"2023-10-08T16:09:38.495647+0000","last_clean":"2023-10-08T16:09:38.495647+0000","last_became_active":"2023-10-08T16:09:38.495389+0000","last_became_peered":"2023-10-08T16:09:38.495389+0000","last_unstale":"2023-10-08T16:09:38.495647+0000","last_undegraded":"2023-10-08T16:09:38.495647+0000","last_fullsized":"2023-10-08T16:09:38.495647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:40.495764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498626+0000","last_change":"2023-10-08T16:09:38.498626+0000","last_active":"2023-10-08T16:09:38.498626+0000","last_peered":"2023-10-08T16:09:38.498626+0000","last_clean":"2023-10-08T16:09:38.498626+0000","last_became_active":"2023-10-08T16:09:38.498320+0000","last_became_peered":"2023-10-08T16:09:38.498320+0000","last_unstale":"2023-10-08T16:09:38.498626+0000","last_undegraded":"2023-10-08T16:09:38.498626+0000","last_fullsized":"2023-10-08T16:09:38.498626+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:52.601103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491808+0000","last_change":"2023-10-08T16:09:38.491808+0000","last_active":"2023-10-08T16:09:38.491808+0000","last_peered":"2023-10-08T16:09:38.491808+0000","last_clean":"2023-10-08T16:09:38.491808+0000","last_became_active":"2023-10-08T16:09:38.491163+0000","last_became_peered":"2023-10-08T16:09:38.491163+0000","last_unstale":"2023-10-08T16:09:38.491808+0000","last_undegraded":"2023-10-08T16:09:38.491808+0000","last_fullsized":"2023-10-08T16:09:38.491808+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:37.656010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494758+0000","last_change":"2023-10-08T16:09:38.494758+0000","last_active":"2023-10-08T16:09:38.494758+0000","last_peered":"2023-10-08T16:09:38.494758+0000","last_clean":"2023-10-08T16:09:38.494758+0000","last_became_active":"2023-10-08T16:09:38.494047+0000","last_became_peered":"2023-10-08T16:09:38.494047+0000","last_unstale":"2023-10-08T16:09:38.494758+0000","last_undegraded":"2023-10-08T16:09:38.494758+0000","last_fullsized":"2023-10-08T16:09:38.494758+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:21.550265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.479586+0000","last_change":"2023-10-08T16:09:38.479586+0000","last_active":"2023-10-08T16:09:38.479586+0000","last_peered":"2023-10-08T16:09:38.479586+0000","last_clean":"2023-10-08T16:09:38.479586+0000","last_became_active":"2023-10-08T16:09:38.479321+0000","last_became_peered":"2023-10-08T16:09:38.479321+0000","last_unstale":"2023-10-08T16:09:38.479586+0000","last_undegraded":"2023-10-08T16:09:38.479586+0000","last_fullsized":"2023-10-08T16:09:38.479586+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:11.593957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490332+0000","last_change":"2023-10-08T16:09:38.490332+0000","last_active":"2023-10-08T16:09:38.490332+0000","last_peered":"2023-10-08T16:09:38.490332+0000","last_clean":"2023-10-08T16:09:38.490332+0000","last_became_active":"2023-10-08T16:09:38.490064+0000","last_became_peered":"2023-10-08T16:09:38.490064+0000","last_unstale":"2023-10-08T16:09:38.490332+0000","last_undegraded":"2023-10-08T16:09:38.490332+0000","last_fullsized":"2023-10-08T16:09:38.490332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.033893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.492338+0000","last_change":"2023-10-08T16:09:38.492338+0000","last_active":"2023-10-08T16:09:38.492338+0000","last_peered":"2023-10-08T16:09:38.492338+0000","last_clean":"2023-10-08T16:09:38.492338+0000","last_became_active":"2023-10-08T16:09:38.492122+0000","last_became_peered":"2023-10-08T16:09:38.492122+0000","last_unstale":"2023-10-08T16:09:38.492338+0000","last_undegraded":"2023-10-08T16:09:38.492338+0000","last_fullsized":"2023-10-08T16:09:38.492338+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:27.529093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487965+0000","last_change":"2023-10-08T16:09:38.487965+0000","last_active":"2023-10-08T16:09:38.487965+0000","last_peered":"2023-10-08T16:09:38.487965+0000","last_clean":"2023-10-08T16:09:38.487965+0000","last_became_active":"2023-10-08T16:09:38.487431+0000","last_became_peered":"2023-10-08T16:09:38.487431+0000","last_unstale":"2023-10-08T16:09:38.487965+0000","last_undegraded":"2023-10-08T16:09:38.487965+0000","last_fullsized":"2023-10-08T16:09:38.487965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:40.495764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493846+0000","last_change":"2023-10-08T16:09:38.493846+0000","last_active":"2023-10-08T16:09:38.493846+0000","last_peered":"2023-10-08T16:09:38.493846+0000","last_clean":"2023-10-08T16:09:38.493846+0000","last_became_active":"2023-10-08T16:09:38.493518+0000","last_became_peered":"2023-10-08T16:09:38.493518+0000","last_unstale":"2023-10-08T16:09:38.493846+0000","last_undegraded":"2023-10-08T16:09:38.493846+0000","last_fullsized":"2023-10-08T16:09:38.493846+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:36:37.088615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"76'4","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648475+0000","last_change":"2023-10-08T16:09:37.648435+0000","last_active":"2023-10-08T16:09:38.648475+0000","last_peered":"2023-10-08T16:09:38.648475+0000","last_clean":"2023-10-08T16:09:38.648475+0000","last_became_active":"2023-10-08T16:09:37.648102+0000","last_became_peered":"2023-10-08T16:09:37.648102+0000","last_unstale":"2023-10-08T16:09:38.648475+0000","last_undegraded":"2023-10-08T16:09:38.648475+0000","last_fullsized":"2023-10-08T16:09:38.648475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:40.160762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.493640+0000","last_change":"2023-10-08T16:09:38.493640+0000","last_active":"2023-10-08T16:09:38.493640+0000","last_peered":"2023-10-08T16:09:38.493640+0000","last_clean":"2023-10-08T16:09:38.493640+0000","last_became_active":"2023-10-08T16:09:38.493338+0000","last_became_peered":"2023-10-08T16:09:38.493338+0000","last_unstale":"2023-10-08T16:09:38.493640+0000","last_undegraded":"2023-10-08T16:09:38.493640+0000","last_fullsized":"2023-10-08T16:09:38.493640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:15:44.366084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487563+0000","last_change":"2023-10-08T16:09:37.654432+0000","last_active":"2023-10-08T16:09:38.487563+0000","last_peered":"2023-10-08T16:09:38.487563+0000","last_clean":"2023-10-08T16:09:38.487563+0000","last_became_active":"2023-10-08T16:09:37.654146+0000","last_became_peered":"2023-10-08T16:09:37.654146+0000","last_unstale":"2023-10-08T16:09:38.487563+0000","last_undegraded":"2023-10-08T16:09:38.487563+0000","last_fullsized":"2023-10-08T16:09:38.487563+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:22.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494322+0000","last_change":"2023-10-08T16:09:38.494322+0000","last_active":"2023-10-08T16:09:38.494322+0000","last_peered":"2023-10-08T16:09:38.494322+0000","last_clean":"2023-10-08T16:09:38.494322+0000","last_became_active":"2023-10-08T16:09:38.493439+0000","last_became_peered":"2023-10-08T16:09:38.493439+0000","last_unstale":"2023-10-08T16:09:38.494322+0000","last_undegraded":"2023-10-08T16:09:38.494322+0000","last_fullsized":"2023-10-08T16:09:38.494322+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:11.033893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.647197+0000","last_change":"2023-10-08T16:09:37.648072+0000","last_active":"2023-10-08T16:09:38.647197+0000","last_peered":"2023-10-08T16:09:38.647197+0000","last_clean":"2023-10-08T16:09:38.647197+0000","last_became_active":"2023-10-08T16:09:37.647796+0000","last_became_peered":"2023-10-08T16:09:37.647796+0000","last_unstale":"2023-10-08T16:09:38.647197+0000","last_undegraded":"2023-10-08T16:09:38.647197+0000","last_fullsized":"2023-10-08T16:09:38.647197+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:57.915707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.500300+0000","last_change":"2023-10-08T16:09:38.500300+0000","last_active":"2023-10-08T16:09:38.500300+0000","last_peered":"2023-10-08T16:09:38.500300+0000","last_clean":"2023-10-08T16:09:38.500300+0000","last_became_active":"2023-10-08T16:09:38.499005+0000","last_became_peered":"2023-10-08T16:09:38.499005+0000","last_unstale":"2023-10-08T16:09:38.500300+0000","last_undegraded":"2023-10-08T16:09:38.500300+0000","last_fullsized":"2023-10-08T16:09:38.500300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:31:45.808377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648989+0000","last_change":"2023-10-08T16:09:37.649091+0000","last_active":"2023-10-08T16:09:38.648989+0000","last_peered":"2023-10-08T16:09:38.648989+0000","last_clean":"2023-10-08T16:09:38.648989+0000","last_became_active":"2023-10-08T16:09:37.648832+0000","last_became_peered":"2023-10-08T16:09:37.648832+0000","last_unstale":"2023-10-08T16:09:38.648989+0000","last_undegraded":"2023-10-08T16:09:38.648989+0000","last_fullsized":"2023-10-08T16:09:38.648989+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:05.279013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488092+0000","last_change":"2023-10-08T16:09:38.488092+0000","last_active":"2023-10-08T16:09:38.488092+0000","last_peered":"2023-10-08T16:09:38.488092+0000","last_clean":"2023-10-08T16:09:38.488092+0000","last_became_active":"2023-10-08T16:09:38.487841+0000","last_became_peered":"2023-10-08T16:09:38.487841+0000","last_unstale":"2023-10-08T16:09:38.488092+0000","last_undegraded":"2023-10-08T16:09:38.488092+0000","last_fullsized":"2023-10-08T16:09:38.488092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:54.159859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.486620+0000","last_change":"2023-10-08T16:09:37.652162+0000","last_active":"2023-10-08T16:09:38.486620+0000","last_peered":"2023-10-08T16:09:38.486620+0000","last_clean":"2023-10-08T16:09:38.486620+0000","last_became_active":"2023-10-08T16:09:37.651798+0000","last_became_peered":"2023-10-08T16:09:37.651798+0000","last_unstale":"2023-10-08T16:09:38.486620+0000","last_undegraded":"2023-10-08T16:09:38.486620+0000","last_fullsized":"2023-10-08T16:09:38.486620+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:56.263462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495519+0000","last_change":"2023-10-08T16:09:38.495519+0000","last_active":"2023-10-08T16:09:38.495519+0000","last_peered":"2023-10-08T16:09:38.495519+0000","last_clean":"2023-10-08T16:09:38.495519+0000","last_became_active":"2023-10-08T16:09:38.492459+0000","last_became_peered":"2023-10-08T16:09:38.492459+0000","last_unstale":"2023-10-08T16:09:38.495519+0000","last_undegraded":"2023-10-08T16:09:38.495519+0000","last_fullsized":"2023-10-08T16:09:38.495519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:55.584901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.486072+0000","last_change":"2023-10-08T16:09:37.654509+0000","last_active":"2023-10-08T16:09:38.486072+0000","last_peered":"2023-10-08T16:09:38.486072+0000","last_clean":"2023-10-08T16:09:38.486072+0000","last_became_active":"2023-10-08T16:09:37.654135+0000","last_became_peered":"2023-10-08T16:09:37.654135+0000","last_unstale":"2023-10-08T16:09:38.486072+0000","last_undegraded":"2023-10-08T16:09:38.486072+0000","last_fullsized":"2023-10-08T16:09:38.486072+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:10.589694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496581+0000","last_change":"2023-10-08T16:09:38.496581+0000","last_active":"2023-10-08T16:09:38.496581+0000","last_peered":"2023-10-08T16:09:38.496581+0000","last_clean":"2023-10-08T16:09:38.496581+0000","last_became_active":"2023-10-08T16:09:38.496028+0000","last_became_peered":"2023-10-08T16:09:38.496028+0000","last_unstale":"2023-10-08T16:09:38.496581+0000","last_undegraded":"2023-10-08T16:09:38.496581+0000","last_fullsized":"2023-10-08T16:09:38.496581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:32.460601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652548+0000","last_change":"2023-10-08T16:09:37.648458+0000","last_active":"2023-10-08T16:09:37.652548+0000","last_peered":"2023-10-08T16:09:37.652548+0000","last_clean":"2023-10-08T16:09:37.652548+0000","last_became_active":"2023-10-08T16:09:37.647730+0000","last_became_peered":"2023-10-08T16:09:37.647730+0000","last_unstale":"2023-10-08T16:09:37.652548+0000","last_undegraded":"2023-10-08T16:09:37.652548+0000","last_fullsized":"2023-10-08T16:09:37.652548+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:29.832826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496514+0000","last_change":"2023-10-08T16:09:38.496514+0000","last_active":"2023-10-08T16:09:38.496514+0000","last_peered":"2023-10-08T16:09:38.496514+0000","last_clean":"2023-10-08T16:09:38.496514+0000","last_became_active":"2023-10-08T16:09:38.496224+0000","last_became_peered":"2023-10-08T16:09:38.496224+0000","last_unstale":"2023-10-08T16:09:38.496514+0000","last_undegraded":"2023-10-08T16:09:38.496514+0000","last_fullsized":"2023-10-08T16:09:38.496514+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:05.398877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655103+0000","last_change":"2023-10-08T16:09:37.654422+0000","last_active":"2023-10-08T16:09:37.655103+0000","last_peered":"2023-10-08T16:09:37.655103+0000","last_clean":"2023-10-08T16:09:37.655103+0000","last_became_active":"2023-10-08T16:09:37.654192+0000","last_became_peered":"2023-10-08T16:09:37.654192+0000","last_unstale":"2023-10-08T16:09:37.655103+0000","last_undegraded":"2023-10-08T16:09:37.655103+0000","last_fullsized":"2023-10-08T16:09:37.655103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.145714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.480100+0000","last_change":"2023-10-08T16:09:38.480100+0000","last_active":"2023-10-08T16:09:38.480100+0000","last_peered":"2023-10-08T16:09:38.480100+0000","last_clean":"2023-10-08T16:09:38.480100+0000","last_became_active":"2023-10-08T16:09:38.479853+0000","last_became_peered":"2023-10-08T16:09:38.479853+0000","last_unstale":"2023-10-08T16:09:38.480100+0000","last_undegraded":"2023-10-08T16:09:38.480100+0000","last_fullsized":"2023-10-08T16:09:38.480100+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:32.460601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653353+0000","last_change":"2023-10-08T16:09:37.648553+0000","last_active":"2023-10-08T16:09:37.653353+0000","last_peered":"2023-10-08T16:09:37.653353+0000","last_clean":"2023-10-08T16:09:37.653353+0000","last_became_active":"2023-10-08T16:09:37.648167+0000","last_became_peered":"2023-10-08T16:09:37.648167+0000","last_unstale":"2023-10-08T16:09:37.653353+0000","last_undegraded":"2023-10-08T16:09:37.653353+0000","last_fullsized":"2023-10-08T16:09:37.653353+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:03.895723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490155+0000","last_change":"2023-10-08T16:09:38.490155+0000","last_active":"2023-10-08T16:09:38.490155+0000","last_peered":"2023-10-08T16:09:38.490155+0000","last_clean":"2023-10-08T16:09:38.490155+0000","last_became_active":"2023-10-08T16:09:38.489241+0000","last_became_peered":"2023-10-08T16:09:38.489241+0000","last_unstale":"2023-10-08T16:09:38.490155+0000","last_undegraded":"2023-10-08T16:09:38.490155+0000","last_fullsized":"2023-10-08T16:09:38.490155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:33.553406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653085+0000","last_change":"2023-10-08T16:09:37.650879+0000","last_active":"2023-10-08T16:09:37.653085+0000","last_peered":"2023-10-08T16:09:37.653085+0000","last_clean":"2023-10-08T16:09:37.653085+0000","last_became_active":"2023-10-08T16:09:37.650623+0000","last_became_peered":"2023-10-08T16:09:37.650623+0000","last_unstale":"2023-10-08T16:09:37.653085+0000","last_undegraded":"2023-10-08T16:09:37.653085+0000","last_fullsized":"2023-10-08T16:09:37.653085+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:54:31.874956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494248+0000","last_change":"2023-10-08T16:09:38.494248+0000","last_active":"2023-10-08T16:09:38.494248+0000","last_peered":"2023-10-08T16:09:38.494248+0000","last_clean":"2023-10-08T16:09:38.494248+0000","last_became_active":"2023-10-08T16:09:38.494018+0000","last_became_peered":"2023-10-08T16:09:38.494018+0000","last_unstale":"2023-10-08T16:09:38.494248+0000","last_undegraded":"2023-10-08T16:09:38.494248+0000","last_fullsized":"2023-10-08T16:09:38.494248+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:50.417249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"76'3","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.646022+0000","last_change":"2023-10-08T16:09:37.650502+0000","last_active":"2023-10-08T16:09:38.646022+0000","last_peered":"2023-10-08T16:09:38.646022+0000","last_clean":"2023-10-08T16:09:38.646022+0000","last_became_active":"2023-10-08T16:09:37.650237+0000","last_became_peered":"2023-10-08T16:09:37.650237+0000","last_unstale":"2023-10-08T16:09:38.646022+0000","last_undegraded":"2023-10-08T16:09:38.646022+0000","last_fullsized":"2023-10-08T16:09:38.646022+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.145714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496311+0000","last_change":"2023-10-08T16:09:38.496311+0000","last_active":"2023-10-08T16:09:38.496311+0000","last_peered":"2023-10-08T16:09:38.496311+0000","last_clean":"2023-10-08T16:09:38.496311+0000","last_became_active":"2023-10-08T16:09:38.496064+0000","last_became_peered":"2023-10-08T16:09:38.496064+0000","last_unstale":"2023-10-08T16:09:38.496311+0000","last_undegraded":"2023-10-08T16:09:38.496311+0000","last_fullsized":"2023-10-08T16:09:38.496311+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:29.659461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"76'6","reported_seq":15,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.647285+0000","last_change":"2023-10-08T16:09:37.645549+0000","last_active":"2023-10-08T16:09:38.647285+0000","last_peered":"2023-10-08T16:09:38.647285+0000","last_clean":"2023-10-08T16:09:38.647285+0000","last_became_active":"2023-10-08T16:09:37.645260+0000","last_became_peered":"2023-10-08T16:09:37.645260+0000","last_unstale":"2023-10-08T16:09:38.647285+0000","last_undegraded":"2023-10-08T16:09:38.647285+0000","last_fullsized":"2023-10-08T16:09:38.647285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:22.079276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491462+0000","last_change":"2023-10-08T16:09:38.491462+0000","last_active":"2023-10-08T16:09:38.491462+0000","last_peered":"2023-10-08T16:09:38.491462+0000","last_clean":"2023-10-08T16:09:38.491462+0000","last_became_active":"2023-10-08T16:09:38.491199+0000","last_became_peered":"2023-10-08T16:09:38.491199+0000","last_unstale":"2023-10-08T16:09:38.491462+0000","last_undegraded":"2023-10-08T16:09:38.491462+0000","last_fullsized":"2023-10-08T16:09:38.491462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:27.529093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.648931+0000","last_change":"2023-10-08T16:09:37.651232+0000","last_active":"2023-10-08T16:09:38.648931+0000","last_peered":"2023-10-08T16:09:38.648931+0000","last_clean":"2023-10-08T16:09:38.648931+0000","last_became_active":"2023-10-08T16:09:37.650978+0000","last_became_peered":"2023-10-08T16:09:37.650978+0000","last_unstale":"2023-10-08T16:09:38.648931+0000","last_undegraded":"2023-10-08T16:09:38.648931+0000","last_fullsized":"2023-10-08T16:09:38.648931+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.877996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499508+0000","last_change":"2023-10-08T16:09:38.499508+0000","last_active":"2023-10-08T16:09:38.499508+0000","last_peered":"2023-10-08T16:09:38.499508+0000","last_clean":"2023-10-08T16:09:38.499508+0000","last_became_active":"2023-10-08T16:09:38.499216+0000","last_became_peered":"2023-10-08T16:09:38.499216+0000","last_unstale":"2023-10-08T16:09:38.499508+0000","last_undegraded":"2023-10-08T16:09:38.499508+0000","last_fullsized":"2023-10-08T16:09:38.499508+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:06.424722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.651497+0000","last_change":"2023-10-08T16:09:37.644138+0000","last_active":"2023-10-08T16:09:37.651497+0000","last_peered":"2023-10-08T16:09:37.651497+0000","last_clean":"2023-10-08T16:09:37.651497+0000","last_became_active":"2023-10-08T16:09:37.643901+0000","last_became_peered":"2023-10-08T16:09:37.643901+0000","last_unstale":"2023-10-08T16:09:37.651497+0000","last_undegraded":"2023-10-08T16:09:37.651497+0000","last_fullsized":"2023-10-08T16:09:37.651497+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:49.412986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490153+0000","last_change":"2023-10-08T16:09:38.490153+0000","last_active":"2023-10-08T16:09:38.490153+0000","last_peered":"2023-10-08T16:09:38.490153+0000","last_clean":"2023-10-08T16:09:38.490153+0000","last_became_active":"2023-10-08T16:09:38.489892+0000","last_became_peered":"2023-10-08T16:09:38.489892+0000","last_unstale":"2023-10-08T16:09:38.490153+0000","last_undegraded":"2023-10-08T16:09:38.490153+0000","last_fullsized":"2023-10-08T16:09:38.490153+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:21.550265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652564+0000","last_change":"2023-10-08T16:09:37.650980+0000","last_active":"2023-10-08T16:09:37.652564+0000","last_peered":"2023-10-08T16:09:37.652564+0000","last_clean":"2023-10-08T16:09:37.652564+0000","last_became_active":"2023-10-08T16:09:37.650691+0000","last_became_peered":"2023-10-08T16:09:37.650691+0000","last_unstale":"2023-10-08T16:09:37.652564+0000","last_undegraded":"2023-10-08T16:09:37.652564+0000","last_fullsized":"2023-10-08T16:09:37.652564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:16.411106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.489475+0000","last_change":"2023-10-08T16:09:38.489475+0000","last_active":"2023-10-08T16:09:38.489475+0000","last_peered":"2023-10-08T16:09:38.489475+0000","last_clean":"2023-10-08T16:09:38.489475+0000","last_became_active":"2023-10-08T16:09:38.489208+0000","last_became_peered":"2023-10-08T16:09:38.489208+0000","last_unstale":"2023-10-08T16:09:38.489475+0000","last_undegraded":"2023-10-08T16:09:38.489475+0000","last_fullsized":"2023-10-08T16:09:38.489475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:20:43.691375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"76'8","reported_seq":17,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.649183+0000","last_change":"2023-10-08T16:09:37.647952+0000","last_active":"2023-10-08T16:09:38.649183+0000","last_peered":"2023-10-08T16:09:38.649183+0000","last_clean":"2023-10-08T16:09:38.649183+0000","last_became_active":"2023-10-08T16:09:37.647689+0000","last_became_peered":"2023-10-08T16:09:37.647689+0000","last_unstale":"2023-10-08T16:09:38.649183+0000","last_undegraded":"2023-10-08T16:09:38.649183+0000","last_fullsized":"2023-10-08T16:09:38.649183+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:00:00.643719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.499493+0000","last_change":"2023-10-08T16:09:38.499493+0000","last_active":"2023-10-08T16:09:38.499493+0000","last_peered":"2023-10-08T16:09:38.499493+0000","last_clean":"2023-10-08T16:09:38.499493+0000","last_became_active":"2023-10-08T16:09:38.499227+0000","last_became_peered":"2023-10-08T16:09:38.499227+0000","last_unstale":"2023-10-08T16:09:38.499493+0000","last_undegraded":"2023-10-08T16:09:38.499493+0000","last_fullsized":"2023-10-08T16:09:38.499493+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:12.815590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484295+0000","last_change":"2023-10-08T16:09:37.648387+0000","last_active":"2023-10-08T16:09:38.484295+0000","last_peered":"2023-10-08T16:09:38.484295+0000","last_clean":"2023-10-08T16:09:38.484295+0000","last_became_active":"2023-10-08T16:09:37.647727+0000","last_became_peered":"2023-10-08T16:09:37.647727+0000","last_unstale":"2023-10-08T16:09:38.484295+0000","last_undegraded":"2023-10-08T16:09:38.484295+0000","last_fullsized":"2023-10-08T16:09:38.484295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:14:54.877996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.498115+0000","last_change":"2023-10-08T16:09:38.498115+0000","last_active":"2023-10-08T16:09:38.498115+0000","last_peered":"2023-10-08T16:09:38.498115+0000","last_clean":"2023-10-08T16:09:38.498115+0000","last_became_active":"2023-10-08T16:09:38.497853+0000","last_became_peered":"2023-10-08T16:09:38.497853+0000","last_unstale":"2023-10-08T16:09:38.498115+0000","last_undegraded":"2023-10-08T16:09:38.498115+0000","last_fullsized":"2023-10-08T16:09:38.498115+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:23.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.651102+0000","last_change":"2023-10-08T16:09:37.655061+0000","last_active":"2023-10-08T16:09:38.651102+0000","last_peered":"2023-10-08T16:09:38.651102+0000","last_clean":"2023-10-08T16:09:38.651102+0000","last_became_active":"2023-10-08T16:09:37.654736+0000","last_became_peered":"2023-10-08T16:09:37.654736+0000","last_unstale":"2023-10-08T16:09:38.651102+0000","last_undegraded":"2023-10-08T16:09:38.651102+0000","last_fullsized":"2023-10-08T16:09:38.651102+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:20.546002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491437+0000","last_change":"2023-10-08T16:09:38.491437+0000","last_active":"2023-10-08T16:09:38.491437+0000","last_peered":"2023-10-08T16:09:38.491437+0000","last_clean":"2023-10-08T16:09:38.491437+0000","last_became_active":"2023-10-08T16:09:38.490823+0000","last_became_peered":"2023-10-08T16:09:38.490823+0000","last_unstale":"2023-10-08T16:09:38.491437+0000","last_undegraded":"2023-10-08T16:09:38.491437+0000","last_fullsized":"2023-10-08T16:09:38.491437+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:28.044242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652659+0000","last_change":"2023-10-08T16:09:37.648637+0000","last_active":"2023-10-08T16:09:37.652659+0000","last_peered":"2023-10-08T16:09:37.652659+0000","last_clean":"2023-10-08T16:09:37.652659+0000","last_became_active":"2023-10-08T16:09:37.648368+0000","last_became_peered":"2023-10-08T16:09:37.648368+0000","last_unstale":"2023-10-08T16:09:37.652659+0000","last_undegraded":"2023-10-08T16:09:37.652659+0000","last_fullsized":"2023-10-08T16:09:37.652659+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:10.029630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494592+0000","last_change":"2023-10-08T16:09:38.494592+0000","last_active":"2023-10-08T16:09:38.494592+0000","last_peered":"2023-10-08T16:09:38.494592+0000","last_clean":"2023-10-08T16:09:38.494592+0000","last_became_active":"2023-10-08T16:09:38.493943+0000","last_became_peered":"2023-10-08T16:09:38.493943+0000","last_unstale":"2023-10-08T16:09:38.494592+0000","last_undegraded":"2023-10-08T16:09:38.494592+0000","last_fullsized":"2023-10-08T16:09:38.494592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:41:05.175830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.646561+0000","last_change":"2023-10-08T16:09:37.646528+0000","last_active":"2023-10-08T16:09:38.646561+0000","last_peered":"2023-10-08T16:09:38.646561+0000","last_clean":"2023-10-08T16:09:38.646561+0000","last_became_active":"2023-10-08T16:09:37.646269+0000","last_became_peered":"2023-10-08T16:09:37.646269+0000","last_unstale":"2023-10-08T16:09:38.646561+0000","last_undegraded":"2023-10-08T16:09:38.646561+0000","last_fullsized":"2023-10-08T16:09:38.646561+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:28.655198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484168+0000","last_change":"2023-10-08T16:09:38.484168+0000","last_active":"2023-10-08T16:09:38.484168+0000","last_peered":"2023-10-08T16:09:38.484168+0000","last_clean":"2023-10-08T16:09:38.484168+0000","last_became_active":"2023-10-08T16:09:38.483896+0000","last_became_peered":"2023-10-08T16:09:38.483896+0000","last_unstale":"2023-10-08T16:09:38.484168+0000","last_undegraded":"2023-10-08T16:09:38.484168+0000","last_fullsized":"2023-10-08T16:09:38.484168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:06.283276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653678+0000","last_change":"2023-10-08T16:09:37.652625+0000","last_active":"2023-10-08T16:09:37.653678+0000","last_peered":"2023-10-08T16:09:37.653678+0000","last_clean":"2023-10-08T16:09:37.653678+0000","last_became_active":"2023-10-08T16:09:37.652366+0000","last_became_peered":"2023-10-08T16:09:37.652366+0000","last_unstale":"2023-10-08T16:09:37.653678+0000","last_undegraded":"2023-10-08T16:09:37.653678+0000","last_fullsized":"2023-10-08T16:09:37.653678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:53:18.541603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.488834+0000","last_change":"2023-10-08T16:09:38.488834+0000","last_active":"2023-10-08T16:09:38.488834+0000","last_peered":"2023-10-08T16:09:38.488834+0000","last_clean":"2023-10-08T16:09:38.488834+0000","last_became_active":"2023-10-08T16:09:38.488580+0000","last_became_peered":"2023-10-08T16:09:38.488580+0000","last_unstale":"2023-10-08T16:09:38.488834+0000","last_undegraded":"2023-10-08T16:09:38.488834+0000","last_fullsized":"2023-10-08T16:09:38.488834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:50.484122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653701+0000","last_change":"2023-10-08T16:09:37.653080+0000","last_active":"2023-10-08T16:09:37.653701+0000","last_peered":"2023-10-08T16:09:37.653701+0000","last_clean":"2023-10-08T16:09:37.653701+0000","last_became_active":"2023-10-08T16:09:37.652801+0000","last_became_peered":"2023-10-08T16:09:37.652801+0000","last_unstale":"2023-10-08T16:09:37.653701+0000","last_undegraded":"2023-10-08T16:09:37.653701+0000","last_fullsized":"2023-10-08T16:09:37.653701+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:12.890633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.485601+0000","last_change":"2023-10-08T16:09:38.485601+0000","last_active":"2023-10-08T16:09:38.485601+0000","last_peered":"2023-10-08T16:09:38.485601+0000","last_clean":"2023-10-08T16:09:38.485601+0000","last_became_active":"2023-10-08T16:09:38.485299+0000","last_became_peered":"2023-10-08T16:09:38.485299+0000","last_unstale":"2023-10-08T16:09:38.485601+0000","last_undegraded":"2023-10-08T16:09:38.485601+0000","last_fullsized":"2023-10-08T16:09:38.485601+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:28.044242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.651315+0000","last_change":"2023-10-08T16:09:37.647068+0000","last_active":"2023-10-08T16:09:37.651315+0000","last_peered":"2023-10-08T16:09:37.651315+0000","last_clean":"2023-10-08T16:09:37.651315+0000","last_became_active":"2023-10-08T16:09:37.646521+0000","last_became_peered":"2023-10-08T16:09:37.646521+0000","last_unstale":"2023-10-08T16:09:37.651315+0000","last_undegraded":"2023-10-08T16:09:37.651315+0000","last_fullsized":"2023-10-08T16:09:37.651315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:26.524830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.495701+0000","last_change":"2023-10-08T16:09:38.495701+0000","last_active":"2023-10-08T16:09:38.495701+0000","last_peered":"2023-10-08T16:09:38.495701+0000","last_clean":"2023-10-08T16:09:38.495701+0000","last_became_active":"2023-10-08T16:09:38.495370+0000","last_became_peered":"2023-10-08T16:09:38.495370+0000","last_unstale":"2023-10-08T16:09:38.495701+0000","last_undegraded":"2023-10-08T16:09:38.495701+0000","last_fullsized":"2023-10-08T16:09:38.495701+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:49.249762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"76'2","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.645884+0000","last_change":"2023-10-08T16:09:37.644030+0000","last_active":"2023-10-08T16:09:38.645884+0000","last_peered":"2023-10-08T16:09:38.645884+0000","last_clean":"2023-10-08T16:09:38.645884+0000","last_became_active":"2023-10-08T16:09:37.643771+0000","last_became_peered":"2023-10-08T16:09:37.643771+0000","last_unstale":"2023-10-08T16:09:38.645884+0000","last_undegraded":"2023-10-08T16:09:38.645884+0000","last_fullsized":"2023-10-08T16:09:38.645884+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:56:57.831073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.501228+0000","last_change":"2023-10-08T16:09:38.501228+0000","last_active":"2023-10-08T16:09:38.501228+0000","last_peered":"2023-10-08T16:09:38.501228+0000","last_clean":"2023-10-08T16:09:38.501228+0000","last_became_active":"2023-10-08T16:09:38.500954+0000","last_became_peered":"2023-10-08T16:09:38.500954+0000","last_unstale":"2023-10-08T16:09:38.501228+0000","last_undegraded":"2023-10-08T16:09:38.501228+0000","last_fullsized":"2023-10-08T16:09:38.501228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:12.822712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653402+0000","last_change":"2023-10-08T16:09:37.651108+0000","last_active":"2023-10-08T16:09:37.653402+0000","last_peered":"2023-10-08T16:09:37.653402+0000","last_clean":"2023-10-08T16:09:37.653402+0000","last_became_active":"2023-10-08T16:09:37.650863+0000","last_became_peered":"2023-10-08T16:09:37.650863+0000","last_unstale":"2023-10-08T16:09:37.653402+0000","last_undegraded":"2023-10-08T16:09:37.653402+0000","last_fullsized":"2023-10-08T16:09:37.653402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:40.160762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.494545+0000","last_change":"2023-10-08T16:09:38.494545+0000","last_active":"2023-10-08T16:09:38.494545+0000","last_peered":"2023-10-08T16:09:38.494545+0000","last_clean":"2023-10-08T16:09:38.494545+0000","last_became_active":"2023-10-08T16:09:38.493741+0000","last_became_peered":"2023-10-08T16:09:38.493741+0000","last_unstale":"2023-10-08T16:09:38.494545+0000","last_undegraded":"2023-10-08T16:09:38.494545+0000","last_fullsized":"2023-10-08T16:09:38.494545+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:30.837089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652702+0000","last_change":"2023-10-08T16:09:37.648127+0000","last_active":"2023-10-08T16:09:37.652702+0000","last_peered":"2023-10-08T16:09:37.652702+0000","last_clean":"2023-10-08T16:09:37.652702+0000","last_became_active":"2023-10-08T16:09:37.647693+0000","last_became_peered":"2023-10-08T16:09:37.647693+0000","last_unstale":"2023-10-08T16:09:37.652702+0000","last_undegraded":"2023-10-08T16:09:37.652702+0000","last_fullsized":"2023-10-08T16:09:37.652702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:43:16.723311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.491937+0000","last_change":"2023-10-08T16:09:38.491937+0000","last_active":"2023-10-08T16:09:38.491937+0000","last_peered":"2023-10-08T16:09:38.491937+0000","last_clean":"2023-10-08T16:09:38.491937+0000","last_became_active":"2023-10-08T16:09:38.491620+0000","last_became_peered":"2023-10-08T16:09:38.491620+0000","last_unstale":"2023-10-08T16:09:38.491937+0000","last_undegraded":"2023-10-08T16:09:38.491937+0000","last_fullsized":"2023-10-08T16:09:38.491937+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:33.553406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655295+0000","last_change":"2023-10-08T16:09:37.653427+0000","last_active":"2023-10-08T16:09:37.655295+0000","last_peered":"2023-10-08T16:09:37.655295+0000","last_clean":"2023-10-08T16:09:37.655295+0000","last_became_active":"2023-10-08T16:09:37.653157+0000","last_became_peered":"2023-10-08T16:09:37.653157+0000","last_unstale":"2023-10-08T16:09:37.655295+0000","last_undegraded":"2023-10-08T16:09:37.655295+0000","last_fullsized":"2023-10-08T16:09:37.655295+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:00.229704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.484278+0000","last_change":"2023-10-08T16:09:38.484278+0000","last_active":"2023-10-08T16:09:38.484278+0000","last_peered":"2023-10-08T16:09:38.484278+0000","last_clean":"2023-10-08T16:09:38.484278+0000","last_became_active":"2023-10-08T16:09:38.483819+0000","last_became_peered":"2023-10-08T16:09:38.483819+0000","last_unstale":"2023-10-08T16:09:38.484278+0000","last_undegraded":"2023-10-08T16:09:38.484278+0000","last_fullsized":"2023-10-08T16:09:38.484278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:00:01.647982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.652642+0000","last_change":"2023-10-08T16:09:37.648809+0000","last_active":"2023-10-08T16:09:37.652642+0000","last_peered":"2023-10-08T16:09:37.652642+0000","last_clean":"2023-10-08T16:09:37.652642+0000","last_became_active":"2023-10-08T16:09:37.648393+0000","last_became_peered":"2023-10-08T16:09:37.648393+0000","last_unstale":"2023-10-08T16:09:37.652642+0000","last_undegraded":"2023-10-08T16:09:37.652642+0000","last_fullsized":"2023-10-08T16:09:37.652642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:21.519838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496295+0000","last_change":"2023-10-08T16:09:38.496295+0000","last_active":"2023-10-08T16:09:38.496295+0000","last_peered":"2023-10-08T16:09:38.496295+0000","last_clean":"2023-10-08T16:09:38.496295+0000","last_became_active":"2023-10-08T16:09:38.496040+0000","last_became_peered":"2023-10-08T16:09:38.496040+0000","last_unstale":"2023-10-08T16:09:38.496295+0000","last_undegraded":"2023-10-08T16:09:38.496295+0000","last_fullsized":"2023-10-08T16:09:38.496295+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:29.659461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655897+0000","last_change":"2023-10-08T16:09:37.654516+0000","last_active":"2023-10-08T16:09:37.655897+0000","last_peered":"2023-10-08T16:09:37.655897+0000","last_clean":"2023-10-08T16:09:37.655897+0000","last_became_active":"2023-10-08T16:09:37.653225+0000","last_became_peered":"2023-10-08T16:09:37.653225+0000","last_unstale":"2023-10-08T16:09:37.655897+0000","last_undegraded":"2023-10-08T16:09:37.655897+0000","last_fullsized":"2023-10-08T16:09:37.655897+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:28:16.411106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.496751+0000","last_change":"2023-10-08T16:09:38.496751+0000","last_active":"2023-10-08T16:09:38.496751+0000","last_peered":"2023-10-08T16:09:38.496751+0000","last_clean":"2023-10-08T16:09:38.496751+0000","last_became_active":"2023-10-08T16:09:38.496327+0000","last_became_peered":"2023-10-08T16:09:38.496327+0000","last_unstale":"2023-10-08T16:09:38.496751+0000","last_undegraded":"2023-10-08T16:09:38.496751+0000","last_fullsized":"2023-10-08T16:09:38.496751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:50.852600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.653475+0000","last_change":"2023-10-08T16:09:37.651318+0000","last_active":"2023-10-08T16:09:37.653475+0000","last_peered":"2023-10-08T16:09:37.653475+0000","last_clean":"2023-10-08T16:09:37.653475+0000","last_became_active":"2023-10-08T16:09:37.651084+0000","last_became_peered":"2023-10-08T16:09:37.651084+0000","last_unstale":"2023-10-08T16:09:37.653475+0000","last_undegraded":"2023-10-08T16:09:37.653475+0000","last_fullsized":"2023-10-08T16:09:37.653475+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:06:59.487257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.487932+0000","last_change":"2023-10-08T16:09:38.487932+0000","last_active":"2023-10-08T16:09:38.487932+0000","last_peered":"2023-10-08T16:09:38.487932+0000","last_clean":"2023-10-08T16:09:38.487932+0000","last_became_active":"2023-10-08T16:09:38.487324+0000","last_became_peered":"2023-10-08T16:09:38.487324+0000","last_unstale":"2023-10-08T16:09:38.487932+0000","last_undegraded":"2023-10-08T16:09:38.487932+0000","last_fullsized":"2023-10-08T16:09:38.487932+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:20:39.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.655880+0000","last_change":"2023-10-08T16:09:37.654448+0000","last_active":"2023-10-08T16:09:37.655880+0000","last_peered":"2023-10-08T16:09:37.655880+0000","last_clean":"2023-10-08T16:09:37.655880+0000","last_became_active":"2023-10-08T16:09:37.654173+0000","last_became_peered":"2023-10-08T16:09:37.654173+0000","last_unstale":"2023-10-08T16:09:37.655880+0000","last_undegraded":"2023-10-08T16:09:37.655880+0000","last_fullsized":"2023-10-08T16:09:37.655880+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:03:29.832826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":8,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.483118+0000","last_change":"2023-10-08T16:09:38.483118+0000","last_active":"2023-10-08T16:09:38.483118+0000","last_peered":"2023-10-08T16:09:38.483118+0000","last_clean":"2023-10-08T16:09:38.483118+0000","last_became_active":"2023-10-08T16:09:38.482869+0000","last_became_peered":"2023-10-08T16:09:38.482869+0000","last_unstale":"2023-10-08T16:09:38.483118+0000","last_undegraded":"2023-10-08T16:09:38.483118+0000","last_fullsized":"2023-10-08T16:09:38.483118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:37.629836+0000","last_clean_scrub_stamp":"2023-10-08T16:09:37.629836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:23.083539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"76'1","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T16:09:38.490411+0000","last_change":"2023-10-08T16:09:37.651788+0000","last_active":"2023-10-08T16:09:38.490411+0000","last_peered":"2023-10-08T16:09:38.490411+0000","last_clean":"2023-10-08T16:09:38.490411+0000","last_became_active":"2023-10-08T16:09:37.651457+0000","last_became_peered":"2023-10-08T16:09:37.651457+0000","last_unstale":"2023-10-08T16:09:38.490411+0000","last_undegraded":"2023-10-08T16:09:38.490411+0000","last_fullsized":"2023-10-08T16:09:38.490411+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:09:36.625573+0000","last_clean_scrub_stamp":"2023-10-08T16:09:36.625573+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:23.505403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":210,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-10-08T16:09:37.640097+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:09:37.640097+0000","last_peered":"2023-10-08T16:09:37.640097+0000","last_clean":"2023-10-08T16:09:37.640097+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:09:37.640097+0000","last_undegraded":"2023-10-08T16:09:37.640097+0000","last_fullsized":"2023-10-08T16:09:37.640097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298784,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073056,"statfs":{"total":96636764160,"available":96330809344,"internally_reserved":0,"allocated":1531904,"data_stored":990066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.74099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.497},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.89000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.61099999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.66600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298212,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073628,"statfs":{"total":96636764160,"available":96331395072,"internally_reserved":0,"allocated":946176,"data_stored":400294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48799999999999999,"15min":0.48399999999999999},"min":{"1min":0.313,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.629,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72899999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.38800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66700000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.48299999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.45500000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53000000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67300000000000004,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.503}]},{"osd":3,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.41499999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.65800000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.73699999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.378}]},{"osd":4,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.502,"15min":0.52100000000000002},"min":{"1min":0.27400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68200000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.314,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77100000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.50600000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.79400000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.56499999999999995,"15min":0.58699999999999997},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76200000000000001,"5min":0.874,"15min":0.874},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80700000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.61099999999999999},"min":{"1min":0.39000000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81899999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":399822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.27200000000000002,"5min":0.309,"15min":0.311},"min":{"1min":0.16900000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.377,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.215},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.32800000000000001,"15min":0.33600000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.41799999999999998,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58199999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.34000000000000002,"15min":0.36199999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.39500000000000002,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.373},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.374},"min":{"1min":0.23699999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.43099999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.36499999999999999,"15min":0.39100000000000001},"min":{"1min":0.248,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.36699999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.308,"5min":0.34200000000000003,"15min":0.36299999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.46200000000000002,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40100000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66900000000000004,"5min":1.077,"15min":1.077},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.375,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.79300000000000004,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":1.696}]},{"osd":5,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.55100000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.752,"5min":1.302,"15min":1.302},"last":1.657},{"interface":"front","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.68999999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.377,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.71399999999999997,"5min":1.099,"15min":1.099},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.44900000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81699999999999995,"5min":1.268,"15min":1.268},"last":1.7310000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:08:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56899999999999995,"15min":0.58499999999999996},"min":{"1min":0.39600000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56999999999999995,"15min":0.59799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76400000000000001,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302880,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068960,"statfs":{"total":96636764160,"available":96326615040,"internally_reserved":0,"allocated":1531904,"data_stored":990030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.44700000000000001,"15min":0.57699999999999996},"min":{"1min":0.21299999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.45500000000000002,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.41099999999999998,"15min":0.497},"min":{"1min":0.22700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.51200000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.47199999999999998,"15min":0.61199999999999999},"min":{"1min":0.22900000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.53000000000000003,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.36499999999999999,"15min":0.38500000000000001},"min":{"1min":0.255,"5min":0.219,"15min":0.219},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.41599999999999998,"15min":0.504},"min":{"1min":0.22600000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.48499999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.372},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.50800000000000001,"15min":0.64600000000000002},"min":{"1min":0.28100000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58999999999999997,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.35399999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.61599999999999999,"15min":0.68400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80800000000000005,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.40400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.61499999999999999,"15min":0.67900000000000005},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.438},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.68899999999999995,"15min":0.82099999999999995},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.83399999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.48399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.84299999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86299999999999999,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81799999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 16:08:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.84699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.68000000000000005,"15min":0.76000000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82799999999999996,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.46500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":937984,"data_stored":400182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.379},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.34699999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.435},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.48899999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302308,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069532,"statfs":{"total":96636764160,"available":96327200768,"internally_reserved":0,"allocated":946176,"data_stored":401164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.26800000000000002,"5min":0.33400000000000002,"15min":0.35899999999999999},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.379,"5min":1.238,"15min":1.238},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.36399999999999999,"15min":0.39900000000000002},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40000000000000002,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.46700000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.34499999999999997,"15min":0.36099999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.45000000000000001,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.32100000000000001,"5min":0.38200000000000001,"15min":0.41599999999999998},"min":{"1min":0.25700000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.40500000000000003,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.38200000000000001,"15min":0.40200000000000002},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.42899999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.39900000000000002,"15min":0.42599999999999999},"min":{"1min":0.25900000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.432,"5min":1.327,"15min":1.327},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57199999999999995,"15min":0.61199999999999999},"min":{"1min":0.35299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.72999999999999998,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.57299999999999995},"min":{"1min":0.379,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":1.089,"15min":1.089},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56200000000000006,"15min":0.58999999999999997},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":1.27,"15min":1.27},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58299999999999996,"15min":0.625},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.755,"5min":1.879,"15min":1.879},"last":0.71999999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.746,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81200000000000006,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.61799999999999999,"15min":0.65000000000000002},"min":{"1min":0.38500000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78700000000000003,"5min":1.825,"15min":1.825},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76900000000000002,"5min":1.115,"15min":1.115},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":399628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.379}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.47199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.43099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.497}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59899999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.60799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298216,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073624,"statfs":{"total":96636764160,"available":96331390976,"internally_reserved":0,"allocated":950272,"data_stored":401186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.436},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.30099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.375},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.502}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:09:43.056 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:09:43.056 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:09:43.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.210+0000 7fef6ffff700 1 Processor -- start 2023-10-08T16:09:43.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.210+0000 7fef6ffff700 1 -- start start 2023-10-08T16:09:43.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fef701595d0 0x7fef700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef700cf480 0x7fef7015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6ffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7015dbc0 con 0x7fef7015d760 2023-10-08T16:09:43.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6ffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700b10f0 con 0x7fef701595d0 2023-10-08T16:09:43.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700cd530 con 0x7fef700cf480 2023-10-08T16:09:43.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6effd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fef701595d0 0x7fef700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6e7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef700cf480 0x7fef7015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6f7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fef7015d760 0x7fef701599a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:39732/0 (socket says 172.21.15.170:39732) 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6f7fe700 1 -- 172.21.15.170:0/2643863479 learned_addr learned my addr 172.21.15.170:0/2643863479 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6effd700 1 -- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef700cf480 msgr2=0x7fef7015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6effd700 1 --2- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef700cf480 0x7fef7015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6effd700 1 -- 172.21.15.170:0/2643863479 >> v1:172.21.15.191:6789/0 conn(0x7fef7015d760 legacy=0x7fef701599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6dffb700 1 -- 172.21.15.170:0/2643863479 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380674308 0 0) 0x7fef68001490 con 0x7fef7015d760 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.211+0000 7fef6effd700 1 -- 172.21.15.170:0/2643863479 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7015e000 con 0x7fef701595d0 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6effd700 1 --2- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef701595d0 0x7fef700ccff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef600098d0 tx=0x7fef60009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb22515927bcc2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6dffb700 1 -- 172.21.15.170:0/2643863479 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60005c10 con 0x7fef701595d0 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6dffb700 1 -- 172.21.15.170:0/2643863479 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60005d70 con 0x7fef701595d0 2023-10-08T16:09:43.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6dffb700 1 -- 172.21.15.170:0/2643863479 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60006280 con 0x7fef701595d0 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 -- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef701595d0 msgr2=0x7fef700ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 --2- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef701595d0 0x7fef700ccff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef600098d0 tx=0x7fef60009120 comp rx=0 tx=0).stop 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 -- 172.21.15.170:0/2643863479 shutdown_connections 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 --2- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef701595d0 0x7fef700ccff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 --2- 172.21.15.170:0/2643863479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef700cf480 0x7fef7015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 -- 172.21.15.170:0/2643863479 >> 172.21.15.170:0/2643863479 conn(0x7fef700d05f0 msgr2=0x7fef700d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:43.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 -- 172.21.15.170:0/2643863479 shutdown_connections 2023-10-08T16:09:43.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.212+0000 7fef6ffff700 1 -- 172.21.15.170:0/2643863479 wait complete. 2023-10-08T16:09:43.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 Processor -- start 2023-10-08T16:09:43.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 -- start start 2023-10-08T16:09:43.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 0x7fef70151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef701595d0 0x7fef70154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef70156ee0 con 0x7fef7015d760 2023-10-08T16:09:43.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef701557b0 con 0x7fef700cf480 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef70155a90 con 0x7fef701595d0 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6effd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 0x7fef70151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6e7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef701595d0 0x7fef70154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47748/0 (socket says 172.21.15.170:47748) 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.214+0000 7fef6f7fe700 1 -- 172.21.15.170:0/3484636896 learned_addr learned my addr 172.21.15.170:0/3484636896 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 -- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef701595d0 msgr2=0x7fef70154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef701595d0 0x7fef70154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 -- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 msgr2=0x7fef700b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 -- 172.21.15.170:0/3484636896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef600093c0 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6f7fe700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6effd700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 0x7fef70151f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef60015000 tx=0x7fef60016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6001a630 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6001a7b0 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6001ab70 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef70155d70 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.215+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef700cc990 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.216+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7004c3c0 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.217+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fef60011420 con 0x7fef700cf480 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.217+0000 7fef56ffd700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 0x7fef5c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.217+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fef60086090 con 0x7fef700cf480 2023-10-08T16:09:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.217+0000 7fef6e7fc700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 0x7fef5c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.217+0000 7fef6e7fc700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 0x7fef5c069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef700b2e90 tx=0x7fef640091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.222+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef60053070 con 0x7fef700cf480 2023-10-08T16:09:43.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.483+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fef70088360 con 0x7fef700cf480 2023-10-08T16:09:43.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.487+0000 7fef56ffd700 1 -- 172.21.15.170:0/3484636896 <== mon.1 v2:172.21.15.191:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fef6002b0e0 con 0x7fef700cf480 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 msgr2=0x7fef5c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 0x7fef5c069620 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fef700b2e90 tx=0x7fef640091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 msgr2=0x7fef70151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 0x7fef70151f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef60015000 tx=0x7fef60016000 comp rx=0 tx=0).stop 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 shutdown_connections 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fef5c067170 0x7fef5c069620 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7015d760 0x7fef700b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fef700cf480 0x7fef70151f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 --2- 172.21.15.170:0/3484636896 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fef701595d0 0x7fef70154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 >> 172.21.15.170:0/3484636896 conn(0x7fef700d05f0 msgr2=0x7fef700af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.489+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 shutdown_connections 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.490+0000 7fef6ffff700 1 -- 172.21.15.170:0/3484636896 wait complete. 2023-10-08T16:09:43.492 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:09:43.503 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.616+0000 7f8cc6e42700 1 Processor -- start 2023-10-08T16:09:43.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.616+0000 7f8cc6e42700 1 -- start start 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc6e42700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc0159890 0x7f8cc015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc6e42700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 0x7f8cc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc6e42700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc015a160 con 0x7f8cc015db10 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc6e42700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc015df50 con 0x7f8cc0159890 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc6e42700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00d13a0 con 0x7f8cc0159c60 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc0159890 0x7f8cc015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc53df700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8cc015db10 0x7f8cc0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46890/0 (socket says 172.21.15.170:46890) 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc53df700 1 -- 172.21.15.170:0/3549992224 learned_addr learned my addr 172.21.15.170:0/3549992224 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:43.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cbffff700 1 --2- 172.21.15.170:0/3549992224 >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 0x7f8cc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 -- 172.21.15.170:0/3549992224 >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 msgr2=0x7f8cc015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 --2- 172.21.15.170:0/3549992224 >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 0x7f8cc015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 -- 172.21.15.170:0/3549992224 >> v1:172.21.15.170:6789/0 conn(0x7f8cc015db10 legacy=0x7f8cc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 -- 172.21.15.170:0/3549992224 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc015e140 con 0x7f8cc0159890 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3549992224 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357292161 0 0) 0x7f8cc015a160 con 0x7f8cc015db10 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cbffff700 1 --2- 172.21.15.170:0/3549992224 >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 0x7f8cc015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cc4bde700 1 --2- 172.21.15.170:0/3549992224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159890 0x7f8cc015abd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b67f0 tx=0x7f8cb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a4bb41cd71a167 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.617+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3549992224 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400b6b0 con 0x7f8cc0159890 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3549992224 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb400b810 con 0x7f8cc0159890 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3549992224 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400bbb0 con 0x7f8cc0159890 2023-10-08T16:09:43.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 -- 172.21.15.170:0/3549992224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159890 msgr2=0x7f8cc015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/3549992224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159890 0x7f8cc015abd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00b67f0 tx=0x7f8cb40091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 -- 172.21.15.170:0/3549992224 shutdown_connections 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/3549992224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159890 0x7f8cc015abd0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/3549992224 >> v2:172.21.15.191:3300/0 conn(0x7f8cc0159c60 0x7f8cc015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 -- 172.21.15.170:0/3549992224 >> 172.21.15.170:0/3549992224 conn(0x7f8cc0068e20 msgr2=0x7f8cc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 -- 172.21.15.170:0/3549992224 shutdown_connections 2023-10-08T16:09:43.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.618+0000 7f8cc6e42700 1 -- 172.21.15.170:0/3549992224 wait complete. 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.619+0000 7f8cc6e42700 1 Processor -- start 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.619+0000 7f8cc6e42700 1 -- start start 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc0159890 0x7f8cc00b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc015db10 0x7f8cc00ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00b4df0 con 0x7f8cc0159c60 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01477a0 con 0x7f8cc015db10 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc6e42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0169f90 con 0x7f8cc0159890 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cbffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cbffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47768/0 (socket says 172.21.15.170:47768) 2023-10-08T16:09:43.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cbffff700 1 -- 172.21.15.170:0/1871108377 learned_addr learned my addr 172.21.15.170:0/1871108377 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc53df700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc015db10 0x7f8cc00ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.620+0000 7f8cc4bde700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc0159890 0x7f8cc00b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 -- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc0159890 msgr2=0x7f8cc00b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc0159890 0x7f8cc00b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 -- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc015db10 msgr2=0x7f8cc00ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc015db10 0x7f8cc00ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 -- 172.21.15.170:0/1871108377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4009420 con 0x7f8cc0159c60 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbffff700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00c830 tx=0x7f8cac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac011600 con 0x7f8cc0159c60 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac011760 con 0x7f8cc0159c60 2023-10-08T16:09:43.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc016a270 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac011b20 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.621+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc016a830 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.622+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc00b3590 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.622+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f8cac00a3e0 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.623+0000 7f8cbcff9700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 0x7f8ca4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.623+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8cac084de0 con 0x7f8cc0159c60 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.623+0000 7f8cc4bde700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 0x7f8ca4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:43.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.623+0000 7f8cc4bde700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 0x7f8ca4069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40093f0 tx=0x7f8cb4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:43.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.627+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cc00b3590 con 0x7f8cc0159c60 2023-10-08T16:09:43.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.856+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc00b3590 con 0x7f8cc0159c60 2023-10-08T16:09:43.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.856+0000 7f8cbcff9700 1 -- 172.21.15.170:0/1871108377 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f8cc00b3590 con 0x7f8cc0159c60 2023-10-08T16:09:43.858 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:43.858 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":47,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 msgr2=0x7f8ca4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 0x7f8ca4069620 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8cb40093f0 tx=0x7f8cb4017000 comp rx=0 tx=0).stop 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 msgr2=0x7f8cc0147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f8cac00c830 tx=0x7f8cac008040 comp rx=0 tx=0).stop 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 shutdown_connections 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc015db10 0x7f8cc00ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc0159890 0x7f8cc00b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca4067170 0x7f8ca4069620 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 --2- 172.21.15.170:0/1871108377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0159c60 0x7f8cc0147260 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:43.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 >> 172.21.15.170:0/1871108377 conn(0x7f8cc0068e20 msgr2=0x7f8cc00b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:43.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 shutdown_connections 2023-10-08T16:09:43.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:43.858+0000 7f8cc6e42700 1 -- 172.21.15.170:0/1871108377 wait complete. 2023-10-08T16:09:43.861 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 47 2023-10-08T16:09:44.102 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:09:44.104 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:09:44.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:09:38.021095+0000', 'modified': '2023-10-08T16:09:41.650491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6036}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6036': {'gid': 6036, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/3424487232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 3424487232}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3424487232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:09:44.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:09:44.274 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T16:09:44.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.388+0000 7f3517abb700 1 Processor -- start 2023-10-08T16:09:44.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.388+0000 7f3517abb700 1 -- start start 2023-10-08T16:09:44.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3517abb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3510159890 0x7f351015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3517abb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351015a160 con 0x7f3510069560 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3517abb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35100d13a0 con 0x7f3510159c60 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3517abb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351015e140 con 0x7f3510159890 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3516058700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3510069560 0x7f35100ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46902/0 (socket says 172.21.15.170:46902) 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3516058700 1 -- 172.21.15.170:0/2848640959 learned_addr learned my addr 172.21.15.170:0/2848640959 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3515857700 1 --2- 172.21.15.170:0/2848640959 >> v2:172.21.15.191:3300/0 conn(0x7f3510159890 0x7f351015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158547620 0 0) 0x7f351015a160 con 0x7f3510069560 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34fc003570 con 0x7f3510069560 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198454398 0 0) 0x7f35100d13a0 con 0x7f3510159c60 2023-10-08T16:09:44.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f351015a160 con 0x7f3510159c60 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799633026 0 0) 0x7f34fc003570 con 0x7f3510069560 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3515857700 1 -- 172.21.15.170:0/2848640959 >> v1:172.21.15.191:6789/0 conn(0x7f3510159c60 legacy=0x7f351015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3515857700 1 -- 172.21.15.170:0/2848640959 >> v1:172.21.15.170:6789/0 conn(0x7f3510069560 legacy=0x7f35100ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3515857700 1 -- 172.21.15.170:0/2848640959 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351015a3d0 con 0x7f3510159890 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.389+0000 7f3515857700 1 --2- 172.21.15.170:0/2848640959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510159890 0x7f351015abd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f350c0098e0 tx=0x7f350c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d13981d4e8266b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080017f0 con 0x7f3510159890 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c011420 con 0x7f3510159890 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3514855700 1 -- 172.21.15.170:0/2848640959 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c0117c0 con 0x7f3510159890 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 -- 172.21.15.170:0/2848640959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510159890 msgr2=0x7f351015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 --2- 172.21.15.170:0/2848640959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510159890 0x7f351015abd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f350c0098e0 tx=0x7f350c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 -- 172.21.15.170:0/2848640959 shutdown_connections 2023-10-08T16:09:44.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 --2- 172.21.15.170:0/2848640959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510159890 0x7f351015abd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 -- 172.21.15.170:0/2848640959 >> 172.21.15.170:0/2848640959 conn(0x7f3510068e20 msgr2=0x7f35100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 -- 172.21.15.170:0/2848640959 shutdown_connections 2023-10-08T16:09:44.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.390+0000 7f3517abb700 1 -- 172.21.15.170:0/2848640959 wait complete. 2023-10-08T16:09:44.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.391+0000 7f3517abb700 1 Processor -- start 2023-10-08T16:09:44.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.391+0000 7f3517abb700 1 -- start start 2023-10-08T16:09:44.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510069560 0x7f35101522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3510159c60 0x7f35100b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35100cdab0 con 0x7f3510159890 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351015ca40 con 0x7f3510069560 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3517abb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351015cd20 con 0x7f3510159c60 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47772/0 (socket says 172.21.15.170:47772) 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 -- 172.21.15.170:0/760050697 learned_addr learned my addr 172.21.15.170:0/760050697 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515857700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510069560 0x7f35101522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3516058700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3510159c60 0x7f35100b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 -- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3510159c60 msgr2=0x7f35100b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3510159c60 0x7f35100b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 -- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510069560 msgr2=0x7f35101522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510069560 0x7f35101522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.392+0000 7f3515056700 1 -- 172.21.15.170:0/760050697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c009420 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3515056700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f350000c580 tx=0x7f3500007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3505ffb700 1 -- 172.21.15.170:0/760050697 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500010510 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3505ffb700 1 -- 172.21.15.170:0/760050697 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500010670 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351015d000 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3505ffb700 1 -- 172.21.15.170:0/760050697 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500010a30 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.393+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35100b37c0 con 0x7f3510159890 2023-10-08T16:09:44.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.394+0000 7f3505ffb700 1 -- 172.21.15.170:0/760050697 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f350001b400 con 0x7f3510159890 2023-10-08T16:09:44.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.395+0000 7f3505ffb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 0x7f34f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.395+0000 7f3505ffb700 1 -- 172.21.15.170:0/760050697 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3500084500 con 0x7f3510159890 2023-10-08T16:09:44.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.395+0000 7f3515857700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 0x7f34f4069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.395+0000 7f3515857700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 0x7f34f4069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f350c0093f0 tx=0x7f350c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 Processor -- start 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 -- start start 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b36e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37a8da70 con 0x564a37b80d00 2023-10-08T16:09:44.405 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37a8b270 con 0x564a37b36a50 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.397+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a37a87060 con 0x564a37b5ff60 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47786/0 (socket says 172.21.15.170:47786) 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 -- 172.21.15.170:0/3569505515 learned_addr learned my addr 172.21.15.170:0/3569505515 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515857700 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b36e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3516058700 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 -- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 msgr2=0x564a37b32cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b32cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 -- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 msgr2=0x564a37b36e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b36e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 -- 172.21.15.170:0/3569505515 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a37921270 con 0x564a37b80d00 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f3515056700 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3500051600 tx=0x7f3500054f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1840b94ca18851ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f34faffd700 1 -- 172.21.15.170:0/3569505515 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500055100 con 0x564a37b80d00 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f34faffd700 1 -- 172.21.15.170:0/3569505515 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500058350 con 0x564a37b80d00 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.398+0000 7f34faffd700 1 -- 172.21.15.170:0/3569505515 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350005c0a0 con 0x564a37b80d00 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- 172.21.15.170:0/3569505515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 msgr2=0x564a37b5fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3500051600 tx=0x7f3500054f80 comp rx=0 tx=0).stop 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- 172.21.15.170:0/3569505515 shutdown_connections 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b32cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b36e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3569505515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b5fa20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- 172.21.15.170:0/3569505515 >> 172.21.15.170:0/3569505515 conn(0x564a37b38370 msgr2=0x564a37b38740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- 172.21.15.170:0/3569505515 shutdown_connections 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- 172.21.15.170:0/3569505515 wait complete. 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 -- start start 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.399+0000 7f3520db3b80 1 Processor -- start 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b5ff60 0x564a37b76e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b80d00 0x564a37b773b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37a32890 con 0x564a37b36a50 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37a4dea0 con 0x564a37b5ff60 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a37909e30 con 0x564a37b80d00 2023-10-08T16:09:44.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3520db3b80 10 client.0 _ll_register_callbacks cb 0x564a37afa110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3515857700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3515857700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47788/0 (socket says 172.21.15.170:47788) 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3515857700 1 -- 172.21.15.170:0/3589001693 learned_addr learned my addr 172.21.15.170:0/3589001693 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3515056700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b5ff60 0x564a37b76e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.400+0000 7f3516058700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b80d00 0x564a37b773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 -- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b80d00 msgr2=0x564a37b773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b80d00 0x564a37b773b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 -- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b5ff60 msgr2=0x564a37b76e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b5ff60 0x564a37b76e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 -- 172.21.15.170:0/3589001693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35100b37c0 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3515857700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f350c009ed0 tx=0x7f350c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a573ce3095ff226c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c02f070 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f350c01caf0 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a37b2bb70 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f350c022070 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 10 client.6183 fetch_fsmap learned FSMap version 47 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x564a378c3fe0 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a37b2bb70 con 0x564a37b36a50 2023-10-08T16:09:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f350c033410 con 0x564a37b36a50 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 10 client.6183 fetch_fsmap finished waiting for FSMap version 47 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 10 client.6183 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 20 client.6183 populate_metadata read hostname 'smithi170' 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b7d830 0x564a37b7dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 4 client.6183 mds_command: new command op to 6036 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.401+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 --> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a37b08aa0 con 0x564a37b7d830 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3515056700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b7d830 0x564a37b7dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f34effff700 10 client.6183 ms_handle_connect on v2:172.21.15.170:6835/3424487232 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3515056700 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b7d830 0x564a37b7dc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77d388a7e24f71f server_cookie=ddc739cf8e2f9a1b in_seq=0 out_seq=0 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f34effff700 1 -- 172.21.15.170:0/3589001693 <== mds.0 v2:172.21.15.170:6835/3424487232 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564a37b08aa0 con 0x564a37b7d830 2023-10-08T16:09:44.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f34effff700 10 client.6183 handle_command_reply: tid=0 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 client.6183 shutdown 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 msgr2=0x564a37b78390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f350c009ed0 tx=0x7f350c022000 comp rx=0 tx=0).stop 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 shutdown_connections 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b7d830 0x564a37b7dc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b80d00 0x564a37b773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b36a50 0x564a37b78390 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f3520db3b80 1 --2- 172.21.15.170:0/3589001693 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b5ff60 0x564a37b76e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.402+0000 7f34effff700 0 client.6183 ms_handle_reset on v2:172.21.15.170:6835/3424487232 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.403+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 >> 172.21.15.170:0/3589001693 conn(0x564a37b38370 msgr2=0x564a37b31c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.403+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 shutdown_connections 2023-10-08T16:09:44.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.403+0000 7f3520db3b80 1 -- 172.21.15.170:0/3589001693 wait complete. 2023-10-08T16:09:44.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.403+0000 7f3520db3b80 20 client.6183 trim_cache size 0 max 16384 2023-10-08T16:09:44.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.419+0000 7f3520db3b80 1 Processor -- start 2023-10-08T16:09:44.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.419+0000 7f3520db3b80 1 -- start start 2023-10-08T16:09:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37b32950 con 0x564a37b80d00 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37b5f890 con 0x564a37b36a50 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a37b7ed50 con 0x564a37b5ff60 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47792/0 (socket says 172.21.15.170:47792) 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 -- 172.21.15.170:0/1139284879 learned_addr learned my addr 172.21.15.170:0/1139284879 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3515056700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3515857700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 -- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 msgr2=0x564a37b339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 -- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 msgr2=0x564a37b781d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b781d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3516058700 1 -- 172.21.15.170:0/1139284879 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a37a55780 con 0x564a37b80d00 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.420+0000 7f3515056700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3516058700 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3508014370 tx=0x7f3508007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d9d6b809d8f3b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f34fb7fe700 1 -- 172.21.15.170:0/1139284879 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508014e50 con 0x564a37b80d00 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f34fb7fe700 1 -- 172.21.15.170:0/1139284879 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508004dc0 con 0x564a37b80d00 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f34fb7fe700 1 -- 172.21.15.170:0/1139284879 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508016670 con 0x564a37b80d00 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 -- 172.21.15.170:0/1139284879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 msgr2=0x564a37b81130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3508014370 tx=0x7f3508007040 comp rx=0 tx=0).stop 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 -- 172.21.15.170:0/1139284879 shutdown_connections 2023-10-08T16:09:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b80d00 0x564a37b81130 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b36a50 0x564a37b781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 --2- 172.21.15.170:0/1139284879 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b5ff60 0x564a37b339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 -- 172.21.15.170:0/1139284879 >> 172.21.15.170:0/1139284879 conn(0x564a37b38370 msgr2=0x564a37b2f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 -- 172.21.15.170:0/1139284879 shutdown_connections 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.421+0000 7f3520db3b80 1 -- 172.21.15.170:0/1139284879 wait complete. 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.422+0000 7f3520db3b80 1 -- start start 2023-10-08T16:09:44.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.422+0000 7f3520db3b80 1 Processor -- start 2023-10-08T16:09:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b36a50 0x564a37b76c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 0x564a37b956f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37bdcbf0 con 0x564a37b5ff60 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a37b5b7f0 con 0x564a37b80d00 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a37bda350 con 0x564a37b36a50 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 10 client.0 _ll_register_callbacks cb 0x564a37afa110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47794/0 (socket says 172.21.15.170:47794) 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 -- 172.21.15.170:0/25739257 learned_addr learned my addr 172.21.15.170:0/25739257 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515857700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b36a50 0x564a37b76c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3516058700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 0x564a37b956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 -- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b36a50 msgr2=0x564a37b76c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b36a50 0x564a37b76c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 -- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 msgr2=0x564a37b956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 0x564a37b956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 -- 172.21.15.170:0/25739257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350800f040 con 0x564a37b5ff60 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3516058700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 0x564a37b956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3515056700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f350000a040 tx=0x7f350006d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820e9a65113ca125 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f34effff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500081040 con 0x564a37b5ff60 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f350000f260 con 0x564a37b5ff60 2023-10-08T16:09:44.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.423+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3500058350 con 0x564a37b5ff60 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35000610d0 con 0x564a37b5ff60 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 10 client.6189 fetch_fsmap learned FSMap version 47 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x564a37a3c470 con 0x564a37b5ff60 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3500058350 con 0x564a37b5ff60 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3500055360 con 0x564a37b5ff60 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 10 client.6189 fetch_fsmap finished waiting for FSMap version 47 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 10 client.6189 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 20 client.6189 populate_metadata read hostname 'smithi170' 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b81b20 0x564a37b95c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 4 client.6189 mds_command: new command op to 6036 tid=0[{"prefix": "status"}] 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 --> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564a37a29e00 con 0x564a37b81b20 2023-10-08T16:09:44.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3515857700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b81b20 0x564a37b95c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 10 client.6189 ms_handle_connect on v2:172.21.15.170:6835/3424487232 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3515857700 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b81b20 0x564a37b95c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836ecd38310d4c8c server_cookie=f791cb36ab25b2fb in_seq=0 out_seq=0 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 1 -- 172.21.15.170:0/25739257 <== mds.0 v2:172.21.15.170:6835/3424487232 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564a37a29e00 con 0x564a37b81b20 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f34effff700 10 client.6189 handle_command_reply: tid=0 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 client.6189 shutdown 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 msgr2=0x564a37b77170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.424+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f350000a040 tx=0x7f350006d040 comp rx=0 tx=0).stop 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 shutdown_connections 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:6835/3424487232,v1:172.21.15.170:6837/3424487232] conn(0x564a37b81b20 0x564a37b95c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x564a37b80d00 0x564a37b956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x564a37b36a50 0x564a37b76c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f34effff700 0 client.6189 ms_handle_reset on v2:172.21.15.170:6835/3424487232 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 --2- 172.21.15.170:0/25739257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564a37b5ff60 0x564a37b77170 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 >> 172.21.15.170:0/25739257 conn(0x564a37b38370 msgr2=0x564a37b78080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 shutdown_connections 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 1 -- 172.21.15.170:0/25739257 wait complete. 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3520db3b80 20 client.6189 trim_cache size 0 max 16384 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:09:44.436 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6036, 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.7871690410000003, 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 47, 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 77, 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 77, 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.888235841 2023-10-08T16:09:44.437 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 msgr2=0x7f34f4069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 0x7f34f4069620 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f350c0093f0 tx=0x7f350c016000 comp rx=0 tx=0).stop 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 msgr2=0x7f3510145380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f350000c580 tx=0x7f3500007e10 comp rx=0 tx=0).stop 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 shutdown_connections 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3510159c60 0x7f35100b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3510069560 0x7f35101522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34f4067170 0x7f34f4069620 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 --2- 172.21.15.170:0/760050697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3510159890 0x7f3510145380 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 >> 172.21.15.170:0/760050697 conn(0x7f3510068e20 msgr2=0x7f351015fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 shutdown_connections 2023-10-08T16:09:44.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.425+0000 7f3517abb700 1 -- 172.21.15.170:0/760050697 wait complete. 2023-10-08T16:09:44.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:09:44.481 INFO:tasks.ceph.mds.a.smithi170.stderr:2023-10-08T16:09:44.479+0000 7f70b23a4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:09:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.553+0000 7f331ffff700 1 Processor -- start 2023-10-08T16:09:44.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.553+0000 7f331ffff700 1 -- start start 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f33201597f0 0x7f33200b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f332015d230 0x7f332015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331ffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332015dbb0 con 0x7f3320159420 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331ffff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33200b11d0 con 0x7f33201597f0 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200b14b0 con 0x7f332015d230 2023-10-08T16:09:44.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f33167fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f33201597f0 0x7f33200b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3320159420 0x7f33200b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46912/0 (socket says 172.21.15.170:46912) 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331effd700 1 -- 172.21.15.170:0/221573920 learned_addr learned my addr 172.21.15.170:0/221573920 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.554+0000 7f331f7fe700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f332015d230 0x7f332015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f33167fc700 1 -- 172.21.15.170:0/221573920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f332015d230 msgr2=0x7f332015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f33167fc700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f332015d230 0x7f332015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f33167fc700 1 -- 172.21.15.170:0/221573920 >> v1:172.21.15.170:6789/0 conn(0x7f3320159420 legacy=0x7f33200b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331e7fc700 1 -- 172.21.15.170:0/221573920 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870466920 0 0) 0x7f332015dbb0 con 0x7f3320159420 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f33167fc700 1 -- 172.21.15.170:0/221573920 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200b1790 con 0x7f33201597f0 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f33167fc700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33201597f0 0x7f33200b2270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f33080098e0 tx=0x7f33080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b5fdeef58f1411 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331e7fc700 1 -- 172.21.15.170:0/221573920 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800b640 con 0x7f33201597f0 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331e7fc700 1 -- 172.21.15.170:0/221573920 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330800b7a0 con 0x7f33201597f0 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331e7fc700 1 -- 172.21.15.170:0/221573920 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800bb40 con 0x7f33201597f0 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 -- 172.21.15.170:0/221573920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33201597f0 msgr2=0x7f33200b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33201597f0 0x7f33200b2270 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f33080098e0 tx=0x7f33080091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 -- 172.21.15.170:0/221573920 shutdown_connections 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f332015d230 0x7f332015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 --2- 172.21.15.170:0/221573920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33201597f0 0x7f33200b2270 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 -- 172.21.15.170:0/221573920 >> 172.21.15.170:0/221573920 conn(0x7f33200d0660 msgr2=0x7f33200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 -- 172.21.15.170:0/221573920 shutdown_connections 2023-10-08T16:09:44.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.555+0000 7f331ffff700 1 -- 172.21.15.170:0/221573920 wait complete. 2023-10-08T16:09:44.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 Processor -- start 2023-10-08T16:09:44.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 -- start start 2023-10-08T16:09:44.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 0x7f33201577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 0x7f3320157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332015bab0 con 0x7f3320159420 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332015a770 con 0x7f332015d230 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.557+0000 7f331ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3320159e40 con 0x7f33201597f0 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47810/0 (socket says 172.21.15.170:47810) 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 -- 172.21.15.170:0/3386901269 learned_addr learned my addr 172.21.15.170:0/3386901269 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f33167fc700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 0x7f33201577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331f7fe700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 0x7f3320157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 -- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 msgr2=0x7f33201577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 0x7f33201577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 -- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 msgr2=0x7f3320157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 0x7f3320157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 -- 172.21.15.170:0/3386901269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308009420 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f33167fc700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 0x7f33201577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331f7fe700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 0x7f3320157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331effd700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f331000c810 tx=0x7f3310007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310005b30 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3310005c90 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332015a120 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310007300 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.558+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3320169ae0 con 0x7f3320159420 2023-10-08T16:09:44.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.559+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332015cfe0 con 0x7f3320159420 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.559+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f331001b400 con 0x7f3320159420 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.560+0000 7f33177fe700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 0x7f3300069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.560+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3310084450 con 0x7f3320159420 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.560+0000 7f33167fc700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 0x7f3300069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.560+0000 7f33167fc700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 0x7f3300069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3320159c70 tx=0x7f3308017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:44.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.564+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33100513e0 con 0x7f3320159420 2023-10-08T16:09:44.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.792+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f332004c3c0 con 0x7f3320159420 2023-10-08T16:09:44.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.792+0000 7f33177fe700 1 -- 172.21.15.170:0/3386901269 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3310054a00 con 0x7f3320159420 2023-10-08T16:09:44.794 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 msgr2=0x7f3300069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 0x7f3300069620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3320159c70 tx=0x7f3308017000 comp rx=0 tx=0).stop 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 msgr2=0x7f33201542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f331000c810 tx=0x7f3310007df0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 shutdown_connections 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f332015d230 0x7f3320157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3300067170 0x7f3300069620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3320159420 0x7f33201542c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 --2- 172.21.15.170:0/3386901269 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33201597f0 0x7f33201577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:44.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 >> 172.21.15.170:0/3386901269 conn(0x7f33200d0660 msgr2=0x7f3320156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:44.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 shutdown_connections 2023-10-08T16:09:44.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:44.794+0000 7f331ffff700 1 -- 172.21.15.170:0/3386901269 wait complete. 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:09:44.808 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:09:44.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:44.809 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:09:44.864 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:44.865 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42166sec preferred_lft 42166sec 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:09:44.866 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:44.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:44.867 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:09:44.927 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:09:44.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:44.929 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:09:44.985 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:09:44.986 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:09:44.986 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:09:44.986 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:09:44.986 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:09:44.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:44.986 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:09:44.987 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:09:44.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:09:44.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:09:44.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:09:44.987 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:09:45.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.087 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:09:45.101 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:09:45.101 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:09:45.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:09:45.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.101 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:09:45.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:09:45.166 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:45.221 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:09:45.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:09:45.222 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:45.275 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:09:45.276 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:09:45.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:45.356 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:45.356 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:45.375 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:45.375 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:45.375 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:09:45.375 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:09:45.375 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:09:45.376 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:09:45.376 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:09:45.376 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:09:45.397 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:09:45.398 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:09:45.399 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:09:45.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:09:45.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:45.465 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:09:45.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.620+0000 7f90a93a3700 1 Processor -- start 2023-10-08T16:09:45.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.620+0000 7f90a93a3700 1 -- start start 2023-10-08T16:09:45.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159890 0x7f90a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f90a4159c60 0x7f90a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 0x7f90a4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a415a160 con 0x7f90a4159c60 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40d13a0 con 0x7f90a415db10 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a93a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a415e140 con 0x7f90a4159890 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f90a4159c60 0x7f90a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159890 0x7f90a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f90a4159c60 0x7f90a415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47658/0 (socket says 172.21.15.170:47658) 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 0x7f90a4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.621+0000 7f90a27fc700 1 -- 172.21.15.170:0/1815392991 learned_addr learned my addr 172.21.15.170:0/1815392991 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 -- 172.21.15.170:0/1815392991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159890 msgr2=0x7f90a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 --2- 172.21.15.170:0/1815392991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159890 0x7f90a415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 -- 172.21.15.170:0/1815392991 >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 msgr2=0x7f90a4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 --2- 172.21.15.170:0/1815392991 >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 0x7f90a4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 -- 172.21.15.170:0/1815392991 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a415a3d0 con 0x7f90a4159c60 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a37fe700 1 --2- 172.21.15.170:0/1815392991 >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 0x7f90a4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a27fc700 1 --2- 172.21.15.170:0/1815392991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159c60 0x7f90a415d5d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f90a4089b50 tx=0x7f9094009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1797f83d989159 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:45.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a1ffb700 1 -- 172.21.15.170:0/1815392991 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400b640 con 0x7f90a4159c60 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a1ffb700 1 -- 172.21.15.170:0/1815392991 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400b7a0 con 0x7f90a4159c60 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a1ffb700 1 -- 172.21.15.170:0/1815392991 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400bb60 con 0x7f90a4159c60 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a93a3700 1 -- 172.21.15.170:0/1815392991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159c60 msgr2=0x7f90a415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.622+0000 7f90a93a3700 1 --2- 172.21.15.170:0/1815392991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159c60 0x7f90a415d5d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f90a4089b50 tx=0x7f9094009250 comp rx=0 tx=0).stop 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 -- 172.21.15.170:0/1815392991 shutdown_connections 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 --2- 172.21.15.170:0/1815392991 >> v2:172.21.15.191:3300/0 conn(0x7f90a415db10 0x7f90a4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 --2- 172.21.15.170:0/1815392991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159890 0x7f90a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 --2- 172.21.15.170:0/1815392991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159c60 0x7f90a415d5d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 -- 172.21.15.170:0/1815392991 >> 172.21.15.170:0/1815392991 conn(0x7f90a4068e20 msgr2=0x7f90a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 -- 172.21.15.170:0/1815392991 shutdown_connections 2023-10-08T16:09:45.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.623+0000 7f90a93a3700 1 -- 172.21.15.170:0/1815392991 wait complete. 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.624+0000 7f90a93a3700 1 Processor -- start 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.624+0000 7f90a93a3700 1 -- start start 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159c60 0x7f90a415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a415db10 0x7f90a415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a416ad80 con 0x7f90a4159890 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40cead0 con 0x7f90a415db10 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a416aaa0 con 0x7f90a4159c60 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47668/0 (socket says 172.21.15.170:47668) 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 -- 172.21.15.170:0/2880909719 learned_addr learned my addr 172.21.15.170:0/2880909719 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a27fc700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159c60 0x7f90a415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a37fe700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a415db10 0x7f90a415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 -- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159c60 msgr2=0x7f90a415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159c60 0x7f90a415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 -- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a415db10 msgr2=0x7f90a415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:45.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a415db10 0x7f90a415ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 -- 172.21.15.170:0/2880909719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009a20 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a2ffd700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f909800ecb0 tx=0x7f90980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800c920 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800ca80 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90940094f0 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.625+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800ce20 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.626+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40ce760 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.627+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a404c3c0 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.627+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9098013680 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.627+0000 7f908affd700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 0x7f9090069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.628+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9098084b20 con 0x7f90a4159890 2023-10-08T16:09:45.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.628+0000 7f90a27fc700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 0x7f9090069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:45.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.628+0000 7f90a27fc700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 0x7f9090069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f90a4089b50 tx=0x7f9094017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:45.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.632+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9098051bc0 con 0x7f90a4159890 2023-10-08T16:09:45.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.865+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f90a415a990 con 0x7f90a4159890 2023-10-08T16:09:45.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.865+0000 7f908affd700 1 -- 172.21.15.170:0/2880909719 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9098017070 con 0x7f90a4159890 2023-10-08T16:09:45.867 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.867+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 msgr2=0x7f9090069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.867+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 0x7f9090069620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f90a4089b50 tx=0x7f9094017000 comp rx=0 tx=0).stop 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.867+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 msgr2=0x7f90a41472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.867+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f909800ecb0 tx=0x7f90980091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.867+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 shutdown_connections 2023-10-08T16:09:45.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f90a415db10 0x7f90a415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9090067170 0x7f9090069620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90a4159890 0x7f90a41472f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 --2- 172.21.15.170:0/2880909719 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f90a4159c60 0x7f90a415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 >> 172.21.15.170:0/2880909719 conn(0x7f90a4068e20 msgr2=0x7f90a4153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 shutdown_connections 2023-10-08T16:09:45.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:45.868+0000 7f90a93a3700 1 -- 172.21.15.170:0/2880909719 wait complete. 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:09:45.881 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:09:45.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.882 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:09:45.900 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:09:45.900 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:09:45.901 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:09:45.902 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42165sec preferred_lft 42165sec 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.903 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:09:45.904 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:09:45.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.905 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:09:45.968 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:09:45.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:45.970 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:09:46.028 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:09:46.028 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:09:46.028 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:09:46.028 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:09:46.028 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:09:46.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:09:46.029 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:09:46.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:46.134 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:09:46.149 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:09:46.149 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:09:46.149 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:09:46.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:46.150 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:09:46.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:09:46.215 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:46.270 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:09:46.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:09:46.270 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:46.324 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:09:46.324 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:09:46.325 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:46.405 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:46.405 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:46.423 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:46.423 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:46.423 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:09:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:09:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:09:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:09:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:09:46.424 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:09:46.446 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:09:46.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:09:46.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:09:46.447 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:09:46.448 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:09:46.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:46.448 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:46.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:09:46.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.637+0000 7fe375180700 1 Processor -- start 2023-10-08T16:09:46.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.637+0000 7fe375180700 1 -- start start 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe375180700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 0x7fe370164660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe375180700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3700af900 con 0x7fe3701597b0 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe375180700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe370069d80 con 0x7fe370159b80 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe375180700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe370068390 con 0x7fe37006a9c0 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe3701597b0 0x7fe3700af1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57310/0 (socket says 172.21.15.170:57310) 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36ed9d700 1 -- 172.21.15.170:0/172825818 learned_addr learned my addr 172.21.15.170:0/172825818 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36f59e700 1 --2- 172.21.15.170:0/172825818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 0x7fe370164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:46.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350182667 0 0) 0x7fe3700af900 con 0x7fe3701597b0 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe358003570 con 0x7fe3701597b0 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862556072 0 0) 0x7fe370069d80 con 0x7fe370159b80 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3700af900 con 0x7fe370159b80 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36f59e700 1 -- 172.21.15.170:0/172825818 >> v1:172.21.15.191:6789/0 conn(0x7fe370159b80 legacy=0x7fe37006a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36f59e700 1 -- 172.21.15.170:0/172825818 >> v1:172.21.15.170:6789/0 conn(0x7fe3701597b0 legacy=0x7fe3700af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36f59e700 1 -- 172.21.15.170:0/172825818 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3700aa680 con 0x7fe37006a9c0 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.638+0000 7fe36f59e700 1 --2- 172.21.15.170:0/172825818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 0x7fe370164660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ec0 tx=0x7fe368009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=199f8ff98ecca68e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36800e070 con 0x7fe37006a9c0 2023-10-08T16:09:46.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36800bb40 con 0x7fe37006a9c0 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe36dd9b700 1 -- 172.21.15.170:0/172825818 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368012430 con 0x7fe37006a9c0 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 -- 172.21.15.170:0/172825818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 msgr2=0x7fe370164660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 --2- 172.21.15.170:0/172825818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 0x7fe370164660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe368009ec0 tx=0x7fe368009780 comp rx=0 tx=0).stop 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 -- 172.21.15.170:0/172825818 shutdown_connections 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 --2- 172.21.15.170:0/172825818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe37006a9c0 0x7fe370164660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:46.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 -- 172.21.15.170:0/172825818 >> 172.21.15.170:0/172825818 conn(0x7fe3700cd880 msgr2=0x7fe3700ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 -- 172.21.15.170:0/172825818 shutdown_connections 2023-10-08T16:09:46.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.639+0000 7fe375180700 1 -- 172.21.15.170:0/172825818 wait complete. 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.640+0000 7fe375180700 1 Processor -- start 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.640+0000 7fe375180700 1 -- start start 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37006a9c0 0x7fe37015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3701597b0 0x7fe370154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370169e40 con 0x7fe370159b80 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370169af0 con 0x7fe37006a9c0 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe375180700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37015f410 con 0x7fe3701597b0 2023-10-08T16:09:46.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe36f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe36f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47684/0 (socket says 172.21.15.170:47684) 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe36f59e700 1 -- 172.21.15.170:0/2395482716 learned_addr learned my addr 172.21.15.170:0/2395482716 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe36ed9d700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37006a9c0 0x7fe37015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.641+0000 7fe36e59c700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3701597b0 0x7fe370154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 -- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3701597b0 msgr2=0x7fe370154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3701597b0 0x7fe370154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 -- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37006a9c0 msgr2=0x7fe37015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37006a9c0 0x7fe37015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 -- 172.21.15.170:0/2395482716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368009a20 con 0x7fe370159b80 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe36f59e700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe368013000 tx=0x7fe368019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3680032f0 con 0x7fe370159b80 2023-10-08T16:09:46.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36801b420 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36801b7c0 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37015f6f0 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.642+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe370156c10 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.643+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3700b2960 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.644+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7fe36801bbb0 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.644+0000 7fe356ffd700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 0x7fe35c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.644+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe368086180 con 0x7fe370159b80 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.644+0000 7fe36ed9d700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 0x7fe35c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:46.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.644+0000 7fe36ed9d700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 0x7fe35c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe360007fd0 tx=0x7fe360009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:46.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.652+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe368053050 con 0x7fe370159b80 2023-10-08T16:09:46.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:46.892+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3700718b0 con 0x7fe370159b80 2023-10-08T16:09:47.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.667+0000 7fe356ffd700 1 -- 172.21.15.170:0/2395482716 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe36801be60 con 0x7fe370159b80 2023-10-08T16:09:47.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 msgr2=0x7fe35c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 0x7fe35c069620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe360007fd0 tx=0x7fe360009040 comp rx=0 tx=0).stop 2023-10-08T16:09:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 msgr2=0x7fe37015ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe368013000 tx=0x7fe368019000 comp rx=0 tx=0).stop 2023-10-08T16:09:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 shutdown_connections 2023-10-08T16:09:47.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe35c067170 0x7fe35c069620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe370159b80 0x7fe37015ed50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe37006a9c0 0x7fe37015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 --2- 172.21.15.170:0/2395482716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe3701597b0 0x7fe370154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 >> 172.21.15.170:0/2395482716 conn(0x7fe3700cd880 msgr2=0x7fe370158190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 shutdown_connections 2023-10-08T16:09:47.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.669+0000 7fe375180700 1 -- 172.21.15.170:0/2395482716 wait complete. 2023-10-08T16:09:47.683 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:09:47.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.798+0000 7f46fd92b700 1 Processor -- start 2023-10-08T16:09:47.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.798+0000 7f46fd92b700 1 -- start start 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46fd92b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46f80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46fd92b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f80ca860 con 0x7f46f81566e0 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46fd92b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f80cab40 con 0x7f46f80ae710 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46fd92b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80cae20 con 0x7f46f8156310 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f7fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f46f81566e0 0x7f46f80ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57320/0 (socket says 172.21.15.170:57320) 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f7fff700 1 -- 172.21.15.170:0/2714364703 learned_addr learned my addr 172.21.15.170:0/2714364703 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945425019 0 0) 0x7f46f80ca860 con 0x7f46f81566e0 2023-10-08T16:09:47.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46fc929700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46f80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e8003570 con 0x7f46f81566e0 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 123257918 0 0) 0x7f46e8003570 con 0x7f46f81566e0 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 msgr2=0x7f46f80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.799+0000 7f46f77fe700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46f80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 >> v1:172.21.15.191:6789/0 conn(0x7f46f80ae710 legacy=0x7f46f80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80cb100 con 0x7f46f81566e0 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f46e4001470 con 0x7f46f81566e0 2023-10-08T16:09:47.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 >> v1:172.21.15.170:6789/0 conn(0x7f46f81566e0 legacy=0x7f46f80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46e8003a90 0x7f46e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46e8004d50 0x7f46e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80cae20 con 0x7f46e8004d50 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e800bb70 con 0x7f46e8003a90 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e800bd30 con 0x7f46f8156310 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46e8004d50 0x7f46e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fc929700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46e8003a90 0x7f46e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f7fff700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 -- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 msgr2=0x7f46e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:47.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8156310 0x7f46e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 -- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46e8003a90 msgr2=0x7f46e8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46e8003a90 0x7f46e8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 -- 172.21.15.170:0/2714364703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8003570 con 0x7f46e8004d50 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46fd12a700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46e8004d50 0x7f46e80051c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001f90 tx=0x7f46f0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b80ad1fcbff61b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0001950 con 0x7f46e8004d50 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000c8b0 con 0x7f46e8004d50 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.800+0000 7f46f77fe700 1 -- 172.21.15.170:0/2714364703 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000cc50 con 0x7f46e8004d50 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 -- 172.21.15.170:0/2714364703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46e8004d50 msgr2=0x7f46e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 --2- 172.21.15.170:0/2714364703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46e8004d50 0x7f46e80051c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001f90 tx=0x7f46f0001640 comp rx=0 tx=0).stop 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd12a700 1 -- 172.21.15.170:0/2714364703 reap_dead start 2023-10-08T16:09:47.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 -- 172.21.15.170:0/2714364703 shutdown_connections 2023-10-08T16:09:47.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 -- 172.21.15.170:0/2714364703 >> 172.21.15.170:0/2714364703 conn(0x7f46f80cc1f0 msgr2=0x7f46f80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:47.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 -- 172.21.15.170:0/2714364703 shutdown_connections 2023-10-08T16:09:47.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.801+0000 7f46fd92b700 1 -- 172.21.15.170:0/2714364703 wait complete. 2023-10-08T16:09:47.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.802+0000 7f46fd92b700 1 Processor -- start 2023-10-08T16:09:47.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.802+0000 7f46fd92b700 1 -- start start 2023-10-08T16:09:47.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f815b170 0x7f46f80c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8144dd0 0x7f46f80c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8145270 con 0x7f46f815b540 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c4630 con 0x7f46f815b170 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd92b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80c4910 con 0x7f46f8144dd0 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47702/0 (socket says 172.21.15.170:47702) 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 -- 172.21.15.170:0/3550825846 learned_addr learned my addr 172.21.15.170:0/3550825846 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 -- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8144dd0 msgr2=0x7f46f80c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fd12a700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8144dd0 0x7f46f80c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46fc929700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f815b170 0x7f46f80c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8144dd0 0x7f46f80c3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 -- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f815b170 msgr2=0x7f46f80c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f815b170 0x7f46f80c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 -- 172.21.15.170:0/3550825846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f80c4bf0 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.803+0000 7f46f7fff700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f46e40010e0 tx=0x7f46e4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.804+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4018030 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.804+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400f870 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.804+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f80c4ed0 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.804+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400fc10 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.804+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f80c3e40 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.805+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f80c53f0 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.805+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f46e401b070 con 0x7f46f815b540 2023-10-08T16:09:47.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.805+0000 7f46f4ff9700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 0x7f46dc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:47.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.805+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f46e4019070 con 0x7f46f815b540 2023-10-08T16:09:47.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.805+0000 7f46fc929700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 0x7f46dc069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:47.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.806+0000 7f46fc929700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 0x7f46dc069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006f00 tx=0x7f46ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:47.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:47.810+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46f80c53f0 con 0x7f46f815b540 2023-10-08T16:09:48.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.017+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f46f80c53f0 con 0x7f46dc067170 2023-10-08T16:09:48.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.018+0000 7f46f4ff9700 1 -- 172.21.15.170:0/3550825846 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f46f80c53f0 con 0x7f46dc067170 2023-10-08T16:09:48.019 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:09:48.019 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:48.020 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:09:48.020 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:48.020 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 msgr2=0x7f46dc069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 0x7f46dc069620 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46ec006f00 tx=0x7f46ec008040 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 msgr2=0x7f46f8144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f46e40010e0 tx=0x7f46e4001e50 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 shutdown_connections 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f8144dd0 0x7f46f80c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f815b170 0x7f46f80c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46dc067170 0x7f46dc069620 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 --2- 172.21.15.170:0/3550825846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f815b540 0x7f46f8144890 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 >> 172.21.15.170:0/3550825846 conn(0x7f46f80cc1f0 msgr2=0x7f46f80af360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 shutdown_connections 2023-10-08T16:09:48.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.020+0000 7f46fd92b700 1 -- 172.21.15.170:0/3550825846 wait complete. 2023-10-08T16:09:48.033 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:09:48.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.148+0000 7f423112d700 1 Processor -- start 2023-10-08T16:09:48.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.148+0000 7f423112d700 1 -- start start 2023-10-08T16:09:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f423112d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c159890 0x7f422c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f423112d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f422c069560 0x7f422c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f423112d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422c15a160 con 0x7f422c159c60 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f423112d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f422c069f20 con 0x7f422c069560 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f423112d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c0d13a0 con 0x7f422c159890 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f422c069560 0x7f422c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f422c159c60 0x7f422c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57322/0 (socket says 172.21.15.170:57322) 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422259c700 1 -- 172.21.15.170:0/874535355 learned_addr learned my addr 172.21.15.170:0/874535355 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422ad9d700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c159890 0x7f422c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 -- 172.21.15.170:0/874535355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c159890 msgr2=0x7f422c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c159890 0x7f422c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 -- 172.21.15.170:0/874535355 >> v1:172.21.15.170:6789/0 conn(0x7f422c159c60 legacy=0x7f422c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 -- 172.21.15.170:0/874535355 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c15e140 con 0x7f422c069560 2023-10-08T16:09:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422a59c700 1 -- 172.21.15.170:0/874535355 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562513454 0 0) 0x7f422c15a160 con 0x7f422c159c60 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.149+0000 7f422b59e700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c069560 0x7f422c0699e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4224009e60 tx=0x7f4224009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e69390f229ed46c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f422a59c700 1 -- 172.21.15.170:0/874535355 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422400e070 con 0x7f422c069560 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f422a59c700 1 -- 172.21.15.170:0/874535355 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422400bb60 con 0x7f422c069560 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f422a59c700 1 -- 172.21.15.170:0/874535355 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224012430 con 0x7f422c069560 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 -- 172.21.15.170:0/874535355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c069560 msgr2=0x7f422c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c069560 0x7f422c0699e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4224009e60 tx=0x7f4224009780 comp rx=0 tx=0).stop 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 -- 172.21.15.170:0/874535355 shutdown_connections 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c159890 0x7f422c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 --2- 172.21.15.170:0/874535355 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c069560 0x7f422c0699e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 -- 172.21.15.170:0/874535355 >> 172.21.15.170:0/874535355 conn(0x7f422c068e20 msgr2=0x7f422c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 -- 172.21.15.170:0/874535355 shutdown_connections 2023-10-08T16:09:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.150+0000 7f423112d700 1 -- 172.21.15.170:0/874535355 wait complete. 2023-10-08T16:09:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.151+0000 7f423112d700 1 Processor -- start 2023-10-08T16:09:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.151+0000 7f423112d700 1 -- start start 2023-10-08T16:09:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 0x7f422c152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 0x7f422c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c169ef0 con 0x7f422c159890 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422c16a1d0 con 0x7f422c159c60 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f423112d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422c16a4b0 con 0x7f422c069560 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47710/0 (socket says 172.21.15.170:47710) 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 -- 172.21.15.170:0/1949262648 learned_addr learned my addr 172.21.15.170:0/1949262648 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422ad9d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 0x7f422c152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422b59e700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 0x7f422c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 -- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 msgr2=0x7f422c152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 0x7f422c152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 -- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 msgr2=0x7f422c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 0x7f422c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422259c700 1 -- 172.21.15.170:0/1949262648 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224009a20 con 0x7f422c159890 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422ad9d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 0x7f422c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.152+0000 7f422b59e700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 0x7f422c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f422259c700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f421400c5e0 tx=0x7f4214000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214005b30 con 0x7f422c159890 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4214005cb0 con 0x7f422c159890 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4214007320 con 0x7f422c159890 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f422c16a790 con 0x7f422c159890 2023-10-08T16:09:48.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.153+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f422c169ae0 con 0x7f422c159890 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.154+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422c16ad00 con 0x7f422c159890 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.154+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f421401b3b0 con 0x7f422c159890 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.154+0000 7f42237fe700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 0x7f420c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.154+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4214083d20 con 0x7f422c159890 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.154+0000 7f422ad9d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 0x7f420c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.155+0000 7f422ad9d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 0x7f420c069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f421c006f00 tx=0x7f421c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.159+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f422c16ad00 con 0x7f422c159890 2023-10-08T16:09:48.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.340+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f422c16ad00 con 0x7f422c159890 2023-10-08T16:09:48.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.340+0000 7f42237fe700 1 -- 172.21.15.170:0/1949262648 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f422c16ad00 con 0x7f422c159890 2023-10-08T16:09:48.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.342+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 msgr2=0x7f420c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.342+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 0x7f420c069620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f421c006f00 tx=0x7f421c008040 comp rx=0 tx=0).stop 2023-10-08T16:09:48.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.342+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 msgr2=0x7f422c0b5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.342+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f421400c5e0 tx=0x7f4214000fe0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 shutdown_connections 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f422c159c60 0x7f422c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f422c069560 0x7f422c152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f420c067170 0x7f420c069620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 --2- 172.21.15.170:0/1949262648 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f422c159890 0x7f422c0b5c10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 >> 172.21.15.170:0/1949262648 conn(0x7f422c068e20 msgr2=0x7f422c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 shutdown_connections 2023-10-08T16:09:48.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.343+0000 7f423112d700 1 -- 172.21.15.170:0/1949262648 wait complete. 2023-10-08T16:09:48.356 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs group_0000000000460347 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.472+0000 7f679ab56700 1 Processor -- start 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.472+0000 7f679ab56700 1 -- start start 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6794159890 0x7f679415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6794159c60 0x7f679415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f6794069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415df50 con 0x7f6794159890 2023-10-08T16:09:48.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415a160 con 0x7f6794159c60 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f679ab56700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67940d13a0 con 0x7f679415db10 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6794159890 0x7f679415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6794159890 0x7f679415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47718/0 (socket says 172.21.15.170:47718) 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 -- 172.21.15.170:0/1087126774 learned_addr learned my addr 172.21.15.170:0/1087126774 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67990f3700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f6794069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f6793fff700 1 --2- 172.21.15.170:0/1087126774 >> v2:172.21.15.191:3300/0 conn(0x7f6794159c60 0x7f679415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 -- 172.21.15.170:0/1087126774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 msgr2=0x7f6794069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f6794069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 -- 172.21.15.170:0/1087126774 >> v2:172.21.15.191:3300/0 conn(0x7f6794159c60 msgr2=0x7f679415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 --2- 172.21.15.170:0/1087126774 >> v2:172.21.15.191:3300/0 conn(0x7f6794159c60 0x7f679415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.473+0000 7f67988f2700 1 -- 172.21.15.170:0/1087126774 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415e140 con 0x7f6794159890 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f67988f2700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159890 0x7f679415abd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67940b67f0 tx=0x7f6788009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c448565dc49530 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f67937fe700 1 -- 172.21.15.170:0/1087126774 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800b860 con 0x7f6794159890 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f67937fe700 1 -- 172.21.15.170:0/1087126774 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678800b9c0 con 0x7f6794159890 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f67937fe700 1 -- 172.21.15.170:0/1087126774 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800bd60 con 0x7f6794159890 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 -- 172.21.15.170:0/1087126774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159890 msgr2=0x7f679415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159890 0x7f679415abd0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67940b67f0 tx=0x7f6788009250 comp rx=0 tx=0).stop 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 -- 172.21.15.170:0/1087126774 shutdown_connections 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f6794069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 --2- 172.21.15.170:0/1087126774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159890 0x7f679415abd0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 --2- 172.21.15.170:0/1087126774 >> v2:172.21.15.191:3300/0 conn(0x7f6794159c60 0x7f679415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 -- 172.21.15.170:0/1087126774 >> 172.21.15.170:0/1087126774 conn(0x7f6794068e20 msgr2=0x7f67940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 -- 172.21.15.170:0/1087126774 shutdown_connections 2023-10-08T16:09:48.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.474+0000 7f679ab56700 1 -- 172.21.15.170:0/1087126774 wait complete. 2023-10-08T16:09:48.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.475+0000 7f679ab56700 1 Processor -- start 2023-10-08T16:09:48.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 -- start start 2023-10-08T16:09:48.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6794159890 0x7f67940b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f67941584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679416acb0 con 0x7f6794159c60 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67940c9980 con 0x7f6794159890 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f679ab56700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67940c9c60 con 0x7f679415db10 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f6793fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f6793fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47730/0 (socket says 172.21.15.170:47730) 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f6793fff700 1 -- 172.21.15.170:0/3247527609 learned_addr learned my addr 172.21.15.170:0/3247527609 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.476+0000 7f67988f2700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6794159890 0x7f67940b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f67990f3700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f67941584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 -- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 msgr2=0x7f67941584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f67941584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 -- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6794159890 msgr2=0x7f67940b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6794159890 0x7f67940b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 -- 172.21.15.170:0/3247527609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880094f0 con 0x7f6794159c60 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6793fff700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f678000c7d0 tx=0x7f6780008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780011600 con 0x7f6794159c60 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67940c9f40 con 0x7f6794159c60 2023-10-08T16:09:48.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780011760 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780011b20 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.477+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67940b42d0 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.478+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f679416ae20 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.479+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f67800193f0 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.479+0000 7f6790ff9700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 0x7f6778069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.480+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6780085260 con 0x7f6794159c60 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.480+0000 7f67988f2700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 0x7f6778069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.480+0000 7f67988f2700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 0x7f6778069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f67940b67f0 tx=0x7f6788017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.483+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67800515a0 con 0x7f6794159c60 2023-10-08T16:09:48.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.696+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "group_0000000000460347", "target": ["mon-mgr", ""]}) v1 -- 0x7f679404c3c0 con 0x7f6778067170 2023-10-08T16:09:48.702 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:48.701+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:48.702 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:48.701+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:48.702 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:48.701+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:48.702 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:48.701+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:48.702 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:09:48.701+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:09:48.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.713+0000 7f6790ff9700 1 -- 172.21.15.170:0/3247527609 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f679404c3c0 con 0x7f6778067170 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 msgr2=0x7f6778069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 0x7f6778069620 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f67940b67f0 tx=0x7f6788017000 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 msgr2=0x7f6794157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f678000c7d0 tx=0x7f6780008040 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 shutdown_connections 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f679415db10 0x7f67941584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6794159890 0x7f67940b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6778067170 0x7f6778069620 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 --2- 172.21.15.170:0/3247527609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6794159c60 0x7f6794157fb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 >> 172.21.15.170:0/3247527609 conn(0x7f6794068e20 msgr2=0x7f6794154400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 shutdown_connections 2023-10-08T16:09:48.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.715+0000 7f679ab56700 1 -- 172.21.15.170:0/3247527609 wait complete. 2023-10-08T16:09:48.728 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-08T16:09:48.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.844+0000 7f029ee7a700 1 Processor -- start 2023-10-08T16:09:48.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 -- start start 2023-10-08T16:09:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 0x7f02980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f02980b1b40 0x7f029815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980cfb00 con 0x7f02980b16d0 2023-10-08T16:09:48.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980b5c40 con 0x7f02980b1b40 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.845+0000 7f029ee7a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02980b5f20 con 0x7f02980b4e70 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f02980b1b40 0x7f029815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f02980b1b40 0x7f029815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47748/0 (socket says 172.21.15.170:47748) 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 -- 172.21.15.170:0/3889135000 learned_addr learned my addr 172.21.15.170:0/3889135000 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029de78700 1 --2- 172.21.15.170:0/3889135000 >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 0x7f02980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 -- 172.21.15.170:0/3889135000 >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 msgr2=0x7f02980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 --2- 172.21.15.170:0/3889135000 >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 0x7f02980ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 -- 172.21.15.170:0/3889135000 >> v1:172.21.15.191:6789/0 conn(0x7f02980b16d0 legacy=0x7f029815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 -- 172.21.15.170:0/3889135000 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02980b6200 con 0x7f02980b1b40 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029de78700 1 --2- 172.21.15.170:0/3889135000 >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 0x7f02980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:48.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029e679700 1 --2- 172.21.15.170:0/3889135000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f029815d950 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0298064910 tx=0x7f0290009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d8863449466e58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029ce76700 1 -- 172.21.15.170:0/3889135000 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000e070 con 0x7f02980b1b40 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029ce76700 1 -- 172.21.15.170:0/3889135000 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000ba10 con 0x7f02980b1b40 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.846+0000 7f029ce76700 1 -- 172.21.15.170:0/3889135000 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02900092e0 con 0x7f02980b1b40 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 -- 172.21.15.170:0/3889135000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 msgr2=0x7f029815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 --2- 172.21.15.170:0/3889135000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f029815d950 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0298064910 tx=0x7f0290009840 comp rx=0 tx=0).stop 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 -- 172.21.15.170:0/3889135000 shutdown_connections 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 --2- 172.21.15.170:0/3889135000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f029815d950 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 --2- 172.21.15.170:0/3889135000 >> v2:172.21.15.191:3300/0 conn(0x7f02980b4e70 0x7f02980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 -- 172.21.15.170:0/3889135000 >> 172.21.15.170:0/3889135000 conn(0x7f02980d10c0 msgr2=0x7f02980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 -- 172.21.15.170:0/3889135000 shutdown_connections 2023-10-08T16:09:48.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.847+0000 7f029ee7a700 1 -- 172.21.15.170:0/3889135000 wait complete. 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 Processor -- start 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 -- start start 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 0x7f0298156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 0x7f029815e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980cc240 con 0x7f02980b1b40 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980cb9a0 con 0x7f02980b4e70 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.848+0000 7f029ee7a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980cbc80 con 0x7f02980b16d0 2023-10-08T16:09:48.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47752/0 (socket says 172.21.15.170:47752) 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 -- 172.21.15.170:0/1892938505 learned_addr learned my addr 172.21.15.170:0/1892938505 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029de78700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 0x7f0298156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029e679700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 0x7f029815e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 -- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 msgr2=0x7f0298156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 0x7f0298156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 -- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 msgr2=0x7f029815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 0x7f029815e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 -- 172.21.15.170:0/1892938505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290009ae0 con 0x7f02980b1b40 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029de78700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 0x7f0298156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029e679700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 0x7f029815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029d677700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f028800c5d0 tx=0x7f0288007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288011590 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288011710 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288011ad0 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02980cbf60 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.849+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0298156ac0 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.850+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02980c6990 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.851+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f028801c460 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.851+0000 7f028dffb700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 0x7f027c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.851+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0288084610 con 0x7f02980b1b40 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.851+0000 7f029de78700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 0x7f027c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.851+0000 7f029de78700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 0x7f027c069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f0294008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:48.855+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02980c6990 con 0x7f02980b1b40 2023-10-08T16:09:49.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.063+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f02980c6990 con 0x7f027c067170 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.066+0000 7f028dffb700 1 -- 172.21.15.170:0/1892938505 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+535 (secure 0 0 0) 0x7f02980c6990 con 0x7f027c067170 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2023-10-08T16:09:49.067 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "pending_subvolume_deletions": 0, 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "pools": { 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "data": [ 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": " 340G", 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_data", 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "used": " 0 " 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": [ 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:09:49.068 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": " 340G", 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_metadata", 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: "used": "64.0k" 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout: "used_size": " 0 " 2023-10-08T16:09:49.069 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:09:49.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 msgr2=0x7f027c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 0x7f027c069620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f0294008040 comp rx=0 tx=0).stop 2023-10-08T16:09:49.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 msgr2=0x7f02980cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f028800c5d0 tx=0x7f0288007f60 comp rx=0 tx=0).stop 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 shutdown_connections 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02980b4e70 0x7f029815e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02980b16d0 0x7f0298156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f027c067170 0x7f027c069620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 --2- 172.21.15.170:0/1892938505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02980b1b40 0x7f02980cb050 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 >> 172.21.15.170:0/1892938505 conn(0x7f02980d10c0 msgr2=0x7f0298158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 shutdown_connections 2023-10-08T16:09:49.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.068+0000 7f029ee7a700 1 -- 172.21.15.170:0/1892938505 wait complete. 2023-10-08T16:09:49.081 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:09:49.081 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:49.129 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:49.172 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:09:49.173 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:09:49.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:49.173 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:09:49.212 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:09:49.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:49.212 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:09:49.267 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:09:49.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:09:49.267 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:09:49.313 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:09:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:09:49.313 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:09:49.329 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:49.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.443+0000 7f76562ed700 1 Processor -- start 2023-10-08T16:09:49.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.443+0000 7f76562ed700 1 -- start start 2023-10-08T16:09:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76562ed700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7650159420 0x7f76500b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76562ed700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76562ed700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76500b2980 con 0x7f76501597f0 2023-10-08T16:09:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76562ed700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765015dbb0 con 0x7f7650159420 2023-10-08T16:09:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76562ed700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500b11d0 con 0x7f765015d230 2023-10-08T16:09:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f7655aec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f76552eb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7650159420 0x7f76500b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f7655aec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:56480/0 (socket says 172.21.15.170:56480) 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f7655aec700 1 -- 172.21.15.170:0/3131295377 learned_addr learned my addr 172.21.15.170:0/3131295377 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.444+0000 7f7654aea700 1 --1- 172.21.15.170:0/3131295377 >> v1:172.21.15.191:6789/0 conn(0x7f76501597f0 0x7f76500b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:42468/0 (socket says 172.21.15.170:42468) 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f76552eb700 1 -- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 msgr2=0x7f765015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f76552eb700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f76552eb700 1 -- 172.21.15.170:0/3131295377 >> v1:172.21.15.191:6789/0 conn(0x7f76501597f0 legacy=0x7f76500b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f76552eb700 1 -- 172.21.15.170:0/3131295377 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500b14b0 con 0x7f7650159420 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f7655aec700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f76552eb700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7650159420 0x7f76500b5f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f764c009e60 tx=0x7f764c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c115ff7a1231b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f7647fff700 1 -- 172.21.15.170:0/3131295377 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00e070 con 0x7f7650159420 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f7647fff700 1 -- 172.21.15.170:0/3131295377 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00bb60 con 0x7f7650159420 2023-10-08T16:09:49.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.445+0000 7f7647fff700 1 -- 172.21.15.170:0/3131295377 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c012430 con 0x7f7650159420 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 -- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7650159420 msgr2=0x7f76500b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7650159420 0x7f76500b5f40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f764c009e60 tx=0x7f764c009780 comp rx=0 tx=0).stop 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 -- 172.21.15.170:0/3131295377 shutdown_connections 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f765015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 --2- 172.21.15.170:0/3131295377 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7650159420 0x7f76500b5f40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 -- 172.21.15.170:0/3131295377 >> 172.21.15.170:0/3131295377 conn(0x7f76500d0660 msgr2=0x7f76500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 -- 172.21.15.170:0/3131295377 shutdown_connections 2023-10-08T16:09:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.446+0000 7f76562ed700 1 -- 172.21.15.170:0/3131295377 wait complete. 2023-10-08T16:09:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.447+0000 7f76562ed700 1 Processor -- start 2023-10-08T16:09:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.447+0000 7f76562ed700 1 -- start start 2023-10-08T16:09:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76501597f0 0x7f76500ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f7650155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cace0 con 0x7f7650159420 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cafc0 con 0x7f76501597f0 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76562ed700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76500cb2a0 con 0x7f765015d230 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47766/0 (socket says 172.21.15.170:47766) 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 -- 172.21.15.170:0/4062659405 learned_addr learned my addr 172.21.15.170:0/4062659405 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f7655aec700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f7650155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f7654aea700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76501597f0 0x7f76500ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 -- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 msgr2=0x7f7650155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f7650155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 -- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76501597f0 msgr2=0x7f76500ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76501597f0 0x7f76500ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f76552eb700 1 -- 172.21.15.170:0/4062659405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480094f0 con 0x7f7650159420 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.448+0000 7f7655aec700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f7650155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76552eb700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c012c40 con 0x7f7650159420 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c012dc0 con 0x7f7650159420 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0276c0 con 0x7f7650159420 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c009a20 con 0x7f7650159420 2023-10-08T16:09:49.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.449+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76500cb860 con 0x7f7650159420 2023-10-08T16:09:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.450+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76500cba80 con 0x7f7650159420 2023-10-08T16:09:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.450+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f764c024070 con 0x7f7650159420 2023-10-08T16:09:49.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.451+0000 7f76457fa700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 0x7f7634069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.451+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f764c086d60 con 0x7f7650159420 2023-10-08T16:09:49.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.451+0000 7f7654aea700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 0x7f7634069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.451+0000 7f7654aea700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 0x7f7634069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76500cff80 tx=0x7f7640007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.455+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f764c053d40 con 0x7f7650159420 2023-10-08T16:09:49.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.681+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765004c3c0 con 0x7f7650159420 2023-10-08T16:09:49.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.682+0000 7f76457fa700 1 -- 172.21.15.170:0/4062659405 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f764c057360 con 0x7f7650159420 2023-10-08T16:09:49.683 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:49.684 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:49.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 msgr2=0x7f7634069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 0x7f7634069620 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f76500cff80 tx=0x7f7640007e70 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 msgr2=0x7f76500cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f764c0097c0 tx=0x7f764c019000 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 shutdown_connections 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f765015d230 0x7f7650155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7634067170 0x7f7634069620 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7650159420 0x7f76500cf800 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 --2- 172.21.15.170:0/4062659405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76501597f0 0x7f76500ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 >> 172.21.15.170:0/4062659405 conn(0x7f76500d0660 msgr2=0x7f76500b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:49.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 shutdown_connections 2023-10-08T16:09:49.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.684+0000 7f76562ed700 1 -- 172.21.15.170:0/4062659405 wait complete. 2023-10-08T16:09:49.687 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2023-10-08T16:09:49.697 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:49.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.811+0000 7f6239888700 1 Processor -- start 2023-10-08T16:09:49.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.811+0000 7f6239888700 1 -- start start 2023-10-08T16:09:49.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6239888700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f62341597f0 0x7f62340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6239888700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62340b11d0 con 0x7f6234159420 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6239888700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62340b14b0 con 0x7f623415d230 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6239888700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340b1790 con 0x7f62341597f0 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6238886700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6234159420 0x7f62340b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57334/0 (socket says 172.21.15.170:57334) 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6238886700 1 -- 172.21.15.170:0/3193933834 learned_addr learned my addr 172.21.15.170:0/3193933834 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990898621 0 0) 0x7f62340b11d0 con 0x7f6234159420 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6224003570 con 0x7f6234159420 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6233fff700 1 --2- 172.21.15.170:0/3193933834 >> v2:172.21.15.191:3300/0 conn(0x7f62341597f0 0x7f62340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3140663965 0 0) 0x7f6224003570 con 0x7f6234159420 2023-10-08T16:09:49.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 >> v2:172.21.15.191:3300/0 conn(0x7f62341597f0 msgr2=0x7f62340b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 --2- 172.21.15.170:0/3193933834 >> v2:172.21.15.191:3300/0 conn(0x7f62341597f0 0x7f62340b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 >> v1:172.21.15.191:6789/0 conn(0x7f623415d230 legacy=0x7f623415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340b1a70 con 0x7f6234159420 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6228001470 con 0x7f6234159420 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 >> v1:172.21.15.170:6789/0 conn(0x7f6234159420 legacy=0x7f62340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 0x7f6224003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6224004810 0x7f6224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240052e0 0x7f622400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340b1790 con 0x7f6224003a60 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622400c190 con 0x7f6224004810 2023-10-08T16:09:49.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622400c350 con 0x7f62240052e0 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.812+0000 7f6238886700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 0x7f6224003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6233fff700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6224004810 0x7f6224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6239087700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240052e0 0x7f622400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 -- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240052e0 msgr2=0x7f622400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f62240052e0 0x7f622400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 -- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6224004810 msgr2=0x7f6224004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6224004810 0x7f6224004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 -- 172.21.15.170:0/3193933834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6224003570 con 0x7f6224003a60 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6239087700 1 -- 172.21.15.170:0/3193933834 reap_dead start 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f6238886700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 0x7f6224003e70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f622800c3c0 tx=0x7f6228001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1408f39462e3be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800cc90 con 0x7f6224003a60 2023-10-08T16:09:49.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622800cdf0 con 0x7f6224003a60 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.813+0000 7f62337fe700 1 -- 172.21.15.170:0/3193933834 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228005b50 con 0x7f6224003a60 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 -- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 msgr2=0x7f6224003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 0x7f6224003e70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f622800c3c0 tx=0x7f6228001e50 comp rx=0 tx=0).stop 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 -- 172.21.15.170:0/3193933834 shutdown_connections 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 --2- 172.21.15.170:0/3193933834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6224003a60 0x7f6224003e70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 -- 172.21.15.170:0/3193933834 >> 172.21.15.170:0/3193933834 conn(0x7f62340d0660 msgr2=0x7f62340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 -- 172.21.15.170:0/3193933834 shutdown_connections 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.814+0000 7f6239888700 1 -- 172.21.15.170:0/3193933834 wait complete. 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.815+0000 7f6239888700 1 Processor -- start 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.815+0000 7f6239888700 1 -- start start 2023-10-08T16:09:49.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6224003a60 0x7f6234148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62340c6ce0 0x7f62340c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340c8690 con 0x7f623415bf30 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340c7690 con 0x7f62340c6ce0 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623415c8e0 con 0x7f6224003a60 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47778/0 (socket says 172.21.15.170:47778) 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 -- 172.21.15.170:0/3001171164 learned_addr learned my addr 172.21.15.170:0/3001171164 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6238886700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6224003a60 0x7f6234148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 -- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6224003a60 msgr2=0x7f6234148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6224003a60 0x7f6234148f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 -- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62340c6ce0 msgr2=0x7f62340c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6233fff700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62340c6ce0 0x7f62340c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62340c6ce0 0x7f62340c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 -- 172.21.15.170:0/3001171164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340c5400 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239087700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f62340af930 tx=0x7f622c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c006db0 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c003070 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62340c5650 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c003430 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.816+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62340c5c10 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.817+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623404c3c0 con 0x7f623415bf30 2023-10-08T16:09:49.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.818+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f622c001790 con 0x7f623415bf30 2023-10-08T16:09:49.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.818+0000 7f6230ff9700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 0x7f6218069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:49.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.818+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f622c083f40 con 0x7f623415bf30 2023-10-08T16:09:49.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.818+0000 7f6238886700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 0x7f6218069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:49.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.818+0000 7f6238886700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 0x7f6218069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6228001ed0 tx=0x7f6228015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:49.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:49.822+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f622c04c5a0 con 0x7f623415bf30 2023-10-08T16:09:50.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.005+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62340b1d50 con 0x7f623415bf30 2023-10-08T16:09:50.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.006+0000 7f6230ff9700 1 -- 172.21.15.170:0/3001171164 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f622c0549b0 con 0x7f623415bf30 2023-10-08T16:09:50.007 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:50.007 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":77,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:39.474771+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:50.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 msgr2=0x7f6218069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 0x7f6218069620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6228001ed0 tx=0x7f6228015000 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 msgr2=0x7f623415c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f62340af930 tx=0x7f622c009040 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 shutdown_connections 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6218067170 0x7f6218069620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f623415bf30 0x7f623415c3a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6224003a60 0x7f6234148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 --2- 172.21.15.170:0/3001171164 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f62340c6ce0 0x7f62340c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 >> 172.21.15.170:0/3001171164 conn(0x7f62340d0660 msgr2=0x7f6234069710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 shutdown_connections 2023-10-08T16:09:50.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.008+0000 7f6239888700 1 -- 172.21.15.170:0/3001171164 wait complete. 2023-10-08T16:09:50.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:09:36.357276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:09:36.982584+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:09:50.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:09:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.139+0000 7f983cd62700 1 Processor -- start 2023-10-08T16:09:50.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.139+0000 7f983cd62700 1 -- start start 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983cd62700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9838069560 0x7f98380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983cd62700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9838069ee0 con 0x7f9838159890 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983cd62700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983815a160 con 0x7f9838159c60 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983cd62700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98380d13a0 con 0x7f9838069560 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9838159890 0x7f983815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57346/0 (socket says 172.21.15.170:57346) 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983659c700 1 -- 172.21.15.170:0/1303945272 learned_addr learned my addr 172.21.15.170:0/1303945272 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f9836d9d700 1 --2- 172.21.15.170:0/1303945272 >> v2:172.21.15.191:3300/0 conn(0x7f9838069560 0x7f98380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315842420 0 0) 0x7f9838069ee0 con 0x7f9838159890 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9824003570 con 0x7f9838159890 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455517737 0 0) 0x7f983815a160 con 0x7f9838159c60 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9838069ee0 con 0x7f9838159c60 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f9836d9d700 1 -- 172.21.15.170:0/1303945272 >> v1:172.21.15.191:6789/0 conn(0x7f9838159c60 legacy=0x7f983815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4277851754 0 0) 0x7f9824003570 con 0x7f9838159890 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f9836d9d700 1 -- 172.21.15.170:0/1303945272 >> v1:172.21.15.170:6789/0 conn(0x7f9838159890 legacy=0x7f983815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f9836d9d700 1 -- 172.21.15.170:0/1303945272 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983815e140 con 0x7f9838069560 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.140+0000 7f9836d9d700 1 --2- 172.21.15.170:0/1303945272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838069560 0x7f98380699a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98300098e0 tx=0x7f98300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d669ee69afa29511 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280017f0 con 0x7f9838069560 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9830011420 con 0x7f9838069560 2023-10-08T16:09:50.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983559a700 1 -- 172.21.15.170:0/1303945272 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300117c0 con 0x7f9838069560 2023-10-08T16:09:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 -- 172.21.15.170:0/1303945272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838069560 msgr2=0x7f98380699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 --2- 172.21.15.170:0/1303945272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838069560 0x7f98380699a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98300098e0 tx=0x7f98300091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 -- 172.21.15.170:0/1303945272 shutdown_connections 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 --2- 172.21.15.170:0/1303945272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838069560 0x7f98380699a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 -- 172.21.15.170:0/1303945272 >> 172.21.15.170:0/1303945272 conn(0x7f9838068e20 msgr2=0x7f98380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 -- 172.21.15.170:0/1303945272 shutdown_connections 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.141+0000 7f983cd62700 1 -- 172.21.15.170:0/1303945272 wait complete. 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.142+0000 7f983cd62700 1 Processor -- start 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.142+0000 7f983cd62700 1 -- start start 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 0x7f9838153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 0x7f9838169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380ce880 con 0x7f9838069560 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98380cf270 con 0x7f9838159890 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983cd62700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98380cdc70 con 0x7f9838159c60 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47784/0 (socket says 172.21.15.170:47784) 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 -- 172.21.15.170:0/2281898302 learned_addr learned my addr 172.21.15.170:0/2281898302 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f9836d9d700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 0x7f9838169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f9835d9b700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 0x7f9838153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 -- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 msgr2=0x7f9838169660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 0x7f9838169660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 -- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 msgr2=0x7f9838153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 0x7f9838153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f983659c700 1 -- 172.21.15.170:0/2281898302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009420 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f9836d9d700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 0x7f9838169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.143+0000 7f9835d9b700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 0x7f9838153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f983659c700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f982800ccc0 tx=0x7f9828007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9828004dc0 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982800a920 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98380cdf50 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982800ace0 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.144+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98380ce510 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.145+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98380ceb60 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.145+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f9828007360 con 0x7f9838069560 2023-10-08T16:09:50.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.145+0000 7f982e7fc700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 0x7f9818069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.145+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f98280843a0 con 0x7f9838069560 2023-10-08T16:09:50.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.145+0000 7f9835d9b700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 0x7f9818069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.146+0000 7f9835d9b700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 0x7f9818069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9820007fd0 tx=0x7f9820009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.149+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9828051400 con 0x7f9838069560 2023-10-08T16:09:50.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.331+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f983804c3c0 con 0x7f9838069560 2023-10-08T16:09:50.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.331+0000 7f982e7fc700 1 -- 172.21.15.170:0/2281898302 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9828018020 con 0x7f9838069560 2023-10-08T16:09:50.333 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 msgr2=0x7f9818069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 0x7f9818069620 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9820007fd0 tx=0x7f9820009040 comp rx=0 tx=0).stop 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 msgr2=0x7f98380ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f982800ccc0 tx=0x7f9828007da0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 shutdown_connections 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9838159c60 0x7f9838169660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9818067170 0x7f9818069620 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9838069560 0x7f98380ced30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 --2- 172.21.15.170:0/2281898302 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9838159890 0x7f9838153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 >> 172.21.15.170:0/2281898302 conn(0x7f9838068e20 msgr2=0x7f9838155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 shutdown_connections 2023-10-08T16:09:50.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.333+0000 7f983cd62700 1 -- 172.21.15.170:0/2281898302 wait complete. 2023-10-08T16:09:50.346 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:09:50.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.461+0000 7f2a9f95d700 1 Processor -- start 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.461+0000 7f2a9f95d700 1 -- start start 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9f95d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 0x7f2a9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9f95d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9f95d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9815a160 con 0x7f2a98159890 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9f95d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9815df50 con 0x7f2a98159c60 2023-10-08T16:09:50.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9f95d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a980d13a0 con 0x7f2a9815db10 2023-10-08T16:09:50.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9defa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9cef8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 0x7f2a9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9defa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:56536/0 (socket says 172.21.15.170:56536) 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9defa700 1 -- 172.21.15.170:0/3315390977 learned_addr learned my addr 172.21.15.170:0/3315390977 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.462+0000 7f2a9d6f9700 1 --1- 172.21.15.170:0/3315390977 >> v1:172.21.15.191:6789/0 conn(0x7f2a98159890 0x7f2a9815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:42488/0 (socket says 172.21.15.170:42488) 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9defa700 1 -- 172.21.15.170:0/3315390977 >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 msgr2=0x7f2a9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9defa700 1 --2- 172.21.15.170:0/3315390977 >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 0x7f2a9815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9defa700 1 -- 172.21.15.170:0/3315390977 >> v1:172.21.15.191:6789/0 conn(0x7f2a98159890 legacy=0x7f2a9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9defa700 1 -- 172.21.15.170:0/3315390977 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9815e140 con 0x7f2a9815db10 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9cef8700 1 --2- 172.21.15.170:0/3315390977 >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 0x7f2a9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a9defa700 1 --2- 172.21.15.170:0/3315390977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009f20 tx=0x7f2a90009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34bbb70cccef4a26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a8ffff700 1 -- 172.21.15.170:0/3315390977 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9000e070 con 0x7f2a9815db10 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a8ffff700 1 -- 172.21.15.170:0/3315390977 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9000b9d0 con 0x7f2a9815db10 2023-10-08T16:09:50.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.463+0000 7f2a8ffff700 1 -- 172.21.15.170:0/3315390977 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a900092e0 con 0x7f2a9815db10 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 -- 172.21.15.170:0/3315390977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 msgr2=0x7f2a98069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/3315390977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009f20 tx=0x7f2a90009840 comp rx=0 tx=0).stop 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 -- 172.21.15.170:0/3315390977 shutdown_connections 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/3315390977 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a9815db10 0x7f2a98069560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/3315390977 >> v2:172.21.15.191:3300/0 conn(0x7f2a98159c60 0x7f2a9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 -- 172.21.15.170:0/3315390977 >> 172.21.15.170:0/3315390977 conn(0x7f2a98068e20 msgr2=0x7f2a980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 -- 172.21.15.170:0/3315390977 shutdown_connections 2023-10-08T16:09:50.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.464+0000 7f2a9f95d700 1 -- 172.21.15.170:0/3315390977 wait complete. 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.465+0000 7f2a9f95d700 1 Processor -- start 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.465+0000 7f2a9f95d700 1 -- start start 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a98159890 0x7f2a980b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9815db10 0x7f2a98158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9816d890 con 0x7f2a98159c60 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9816db70 con 0x7f2a9815db10 2023-10-08T16:09:50.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9816de50 con 0x7f2a98159890 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47800/0 (socket says 172.21.15.170:47800) 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 -- 172.21.15.170:0/2243070647 learned_addr learned my addr 172.21.15.170:0/2243070647 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9d6f9700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a98159890 0x7f2a980b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9defa700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9815db10 0x7f2a98158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 -- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a98159890 msgr2=0x7f2a980b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a98159890 0x7f2a980b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 -- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9815db10 msgr2=0x7f2a98158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9815db10 0x7f2a98158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 -- 172.21.15.170:0/2243070647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88009420 con 0x7f2a98159c60 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9cef8700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800ec80 tx=0x7f2a880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800c920 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8800ca80 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a90009ae0 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800ce00 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.466+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9816e410 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.467+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9804c3c0 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.468+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 9) v1 ==== 84944+0+0 (secure 0 0 0) 0x7f2a88013680 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.468+0000 7f2a8d7fa700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 0x7f2a7c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.468+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2a88084c10 con 0x7f2a98159c60 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.468+0000 7f2a9d6f9700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 0x7f2a7c069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.469+0000 7f2a9d6f9700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 0x7f2a7c069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a94004fd0 tx=0x7f2a94007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.473+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a88051bf0 con 0x7f2a98159c60 2023-10-08T16:09:50.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.651+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a9806a5b0 con 0x7f2a98159c60 2023-10-08T16:09:50.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.652+0000 7f2a8d7fa700 1 -- 172.21.15.170:0/2243070647 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a88055210 con 0x7f2a98159c60 2023-10-08T16:09:50.653 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:09:50.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.653+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 msgr2=0x7f2a7c069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.653+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 0x7f2a7c069620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a94004fd0 tx=0x7f2a94007eb0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.653+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 msgr2=0x7f2a98169920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.653+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800ec80 tx=0x7f2a880091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 shutdown_connections 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2a9815db10 0x7f2a98158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2a98159890 0x7f2a980b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2a7c067170 0x7f2a7c069620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 --2- 172.21.15.170:0/2243070647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98159c60 0x7f2a98169920 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 >> 172.21.15.170:0/2243070647 conn(0x7f2a98068e20 msgr2=0x7f2a98152dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 shutdown_connections 2023-10-08T16:09:50.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.654+0000 7f2a9f95d700 1 -- 172.21.15.170:0/2243070647 wait complete. 2023-10-08T16:09:50.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:09:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.785+0000 7fd5b77cd700 1 Processor -- start 2023-10-08T16:09:50.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 -- start start 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006aa40 0x7fd5b00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd5b006bac0 0x7fd5b006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b00cc210 con 0x7fd5b006b350 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b00cc4f0 con 0x7fd5b006bac0 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.786+0000 7fd5b77cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00cc7d0 con 0x7fd5b006aa40 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd5b006bac0 0x7fd5b006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd5b006bac0 0x7fd5b006ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47804/0 (socket says 172.21.15.170:47804) 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 -- 172.21.15.170:0/3147500961 learned_addr learned my addr 172.21.15.170:0/3147500961 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b5569700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006aa40 0x7fd5b00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 -- 172.21.15.170:0/3147500961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006aa40 msgr2=0x7fd5b00cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006aa40 0x7fd5b00cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 -- 172.21.15.170:0/3147500961 >> v1:172.21.15.191:6789/0 conn(0x7fd5b006b350 legacy=0x7fd5b00c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 -- 172.21.15.170:0/3147500961 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b00ccab0 con 0x7fd5b006bac0 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006bac0 0x7fd5b006ae10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00099b0 tx=0x7fd5a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2543497434d0bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5a7fff700 1 -- 172.21.15.170:0/3147500961 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000b860 con 0x7fd5b006bac0 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5a7fff700 1 -- 172.21.15.170:0/3147500961 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a000b9c0 con 0x7fd5b006bac0 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.787+0000 7fd5a7fff700 1 -- 172.21.15.170:0/3147500961 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000bd60 con 0x7fd5b006bac0 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 -- 172.21.15.170:0/3147500961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006bac0 msgr2=0x7fd5b006ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006bac0 0x7fd5b006ae10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00099b0 tx=0x7fd5a0009250 comp rx=0 tx=0).stop 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 -- 172.21.15.170:0/3147500961 shutdown_connections 2023-10-08T16:09:50.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006aa40 0x7fd5b00cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/3147500961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006bac0 0x7fd5b006ae10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 -- 172.21.15.170:0/3147500961 >> 172.21.15.170:0/3147500961 conn(0x7fd5b00cd670 msgr2=0x7fd5b00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 -- 172.21.15.170:0/3147500961 shutdown_connections 2023-10-08T16:09:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.788+0000 7fd5b77cd700 1 -- 172.21.15.170:0/3147500961 wait complete. 2023-10-08T16:09:50.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.789+0000 7fd5b77cd700 1 Processor -- start 2023-10-08T16:09:50.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.789+0000 7fd5b77cd700 1 -- start start 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 0x7fd5b0168550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 0x7fd5b00b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b013b840 con 0x7fd5b006aa40 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0151a70 con 0x7fd5b006bac0 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0150b80 con 0x7fd5b006b350 2023-10-08T16:09:50.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47808/0 (socket says 172.21.15.170:47808) 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 -- 172.21.15.170:0/979992261 learned_addr learned my addr 172.21.15.170:0/979992261 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 0x7fd5b0168550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 -- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 msgr2=0x7fd5b0168550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 0x7fd5b0168550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 -- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 msgr2=0x7fd5b00b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5d6a700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 0x7fd5b00b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 0x7fd5b00b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 -- 172.21.15.170:0/979992261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00094f0 con 0x7fd5b006aa40 2023-10-08T16:09:50.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 0x7fd5b0168550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5d6a700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 0x7fd5b00b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b5569700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00d810 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0125f0 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac012750 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b0150e60 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.790+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac012b10 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.791+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b0151420 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.792+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b0099690 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.792+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd5ac01e460 con 0x7fd5b006aa40 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.792+0000 7fd5a57fa700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 0x7fd5940696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:50.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.792+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd5ac085010 con 0x7fd5b006aa40 2023-10-08T16:09:50.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.792+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 0x7fd5940696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:50.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.793+0000 7fd5b4d68700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 0x7fd5940696b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00099b0 tx=0x7fd5a0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:50.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.797+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5ac051f30 con 0x7fd5b006aa40 2023-10-08T16:09:50.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.978+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5b0099840 con 0x7fd5b006aa40 2023-10-08T16:09:51.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.979+0000 7fd5a57fa700 1 -- 172.21.15.170:0/979992261 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5ac055550 con 0x7fd5b006aa40 2023-10-08T16:09:51.568 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.980+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 msgr2=0x7fd5940696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.980+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 0x7fd5940696b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5a00099b0 tx=0x7fd5a0017000 comp rx=0 tx=0).stop 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.980+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 msgr2=0x7fd5b013b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.980+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00d810 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 shutdown_connections 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd5b006bac0 0x7fd5b00b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd594067200 0x7fd5940696b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5b006aa40 0x7fd5b013b300 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 --2- 172.21.15.170:0/979992261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd5b006b350 0x7fd5b0168550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 >> 172.21.15.170:0/979992261 conn(0x7fd5b00cd670 msgr2=0x7fd5b00cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 shutdown_connections 2023-10-08T16:09:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:50.981+0000 7fd5b77cd700 1 -- 172.21.15.170:0/979992261 wait complete. 2023-10-08T16:09:51.571 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:51.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.686+0000 7f6dbc7b8700 1 Processor -- start 2023-10-08T16:09:51.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.686+0000 7f6dbc7b8700 1 -- start start 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbc7b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbc7b8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6db415db10 0x7f6db4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbc7b8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db415a160 con 0x7f6db4159890 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbc7b8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40d13a0 con 0x7f6db415db10 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbc7b8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db415e140 con 0x7f6db4159c60 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbad55700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6db415db10 0x7f6db4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dba554700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6db4159890 0x7f6db415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57354/0 (socket says 172.21.15.170:57354) 2023-10-08T16:09:51.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dba554700 1 -- 172.21.15.170:0/1595252080 learned_addr learned my addr 172.21.15.170:0/1595252080 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbad55700 1 -- 172.21.15.170:0/1595252080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 msgr2=0x7f6db415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbad55700 1 --2- 172.21.15.170:0/1595252080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbad55700 1 -- 172.21.15.170:0/1595252080 >> v1:172.21.15.170:6789/0 conn(0x7f6db4159890 legacy=0x7f6db415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6dbad55700 1 -- 172.21.15.170:0/1595252080 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db415a3d0 con 0x7f6db415db10 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.687+0000 7f6db9552700 1 -- 172.21.15.170:0/1595252080 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954414413 0 0) 0x7f6db415a160 con 0x7f6db4159890 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbad55700 1 --2- 172.21.15.170:0/1595252080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4069560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098e0 tx=0x7f6dac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22748d8b9a52fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6db9552700 1 -- 172.21.15.170:0/1595252080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac009e80 con 0x7f6db415db10 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6db9552700 1 -- 172.21.15.170:0/1595252080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00b860 con 0x7f6db415db10 2023-10-08T16:09:51.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6db9552700 1 -- 172.21.15.170:0/1595252080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00bbe0 con 0x7f6db415db10 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/1595252080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 msgr2=0x7f6db4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/1595252080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4069560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f6dac0098e0 tx=0x7f6dac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/1595252080 shutdown_connections 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/1595252080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4069560 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/1595252080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/1595252080 >> 172.21.15.170:0/1595252080 conn(0x7f6db4068e20 msgr2=0x7f6db40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/1595252080 shutdown_connections 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.688+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/1595252080 wait complete. 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.689+0000 7f6dbc7b8700 1 Processor -- start 2023-10-08T16:09:51.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.689+0000 7f6dbc7b8700 1 -- start start 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 0x7f6db4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db40c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40c74b0 con 0x7f6db415db10 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40cfe40 con 0x7f6db4159890 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbc7b8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40cefb0 con 0x7f6db4159c60 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47828/0 (socket says 172.21.15.170:47828) 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 -- 172.21.15.170:0/2866125533 learned_addr learned my addr 172.21.15.170:0/2866125533 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6db9d53700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db40c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:51.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 -- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 msgr2=0x7f6db40c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dba554700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 0x7f6db4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db40c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 -- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 msgr2=0x7f6db4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 0x7f6db4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dbad55700 1 -- 172.21.15.170:0/2866125533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009420 con 0x7f6db415db10 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6db9d53700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db40c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.690+0000 7f6dba554700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 0x7f6db4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6dbad55700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dac01af80 tx=0x7f6dac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac01f070 con 0x7f6db415db10 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00b9d0 con 0x7f6db415db10 2023-10-08T16:09:51.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac025670 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db40cf230 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.691+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40cf7f0 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.692+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db404c3c0 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.692+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f6dac023070 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.692+0000 7f6daa7fc700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 0x7f6d980696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.692+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6dac086150 con 0x7f6db415db10 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.692+0000 7f6dba554700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 0x7f6d980696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:51.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.693+0000 7f6dba554700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 0x7f6d980696b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6db000d270 tx=0x7f6db0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:51.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.697+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dac053070 con 0x7f6db415db10 2023-10-08T16:09:51.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.920+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6db41537e0 con 0x7f6db415db10 2023-10-08T16:09:51.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.921+0000 7f6daa7fc700 1 -- 172.21.15.170:0/2866125533 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f6dac056690 con 0x7f6db415db10 2023-10-08T16:09:51.922 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:51.922 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.922+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 msgr2=0x7f6d980696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.922+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 0x7f6d980696b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6db000d270 tx=0x7f6db0009040 comp rx=0 tx=0).stop 2023-10-08T16:09:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.922+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 msgr2=0x7f6db4152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.922+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f6dac01af80 tx=0x7f6dac017000 comp rx=0 tx=0).stop 2023-10-08T16:09:51.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 shutdown_connections 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d98067200 0x7f6d980696b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6db415db10 0x7f6db4152c80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6db4159890 0x7f6db4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 --2- 172.21.15.170:0/2866125533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6db4159c60 0x7f6db40c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 >> 172.21.15.170:0/2866125533 conn(0x7f6db4068e20 msgr2=0x7f6db414d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 shutdown_connections 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:51.923+0000 7f6dbc7b8700 1 -- 172.21.15.170:0/2866125533 wait complete. 2023-10-08T16:09:51.925 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2023-10-08T16:09:51.936 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:52.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.049+0000 7f514b3f4700 1 Processor -- start 2023-10-08T16:09:52.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.049+0000 7f514b3f4700 1 -- start start 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514b3f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f514415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514b3f4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514415a160 con 0x7f5144159c60 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514b3f4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440d13a0 con 0x7f5144069560 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514b3f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514415e140 con 0x7f5144159890 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514898f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5144159c60 0x7f514415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57358/0 (socket says 172.21.15.170:57358) 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f514898f700 1 -- 172.21.15.170:0/3970937273 learned_addr learned my addr 172.21.15.170:0/3970937273 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:52.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f5149190700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f514415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266370731 0 0) 0x7f514415a160 con 0x7f5144159c60 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5130003570 con 0x7f5144159c60 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1090906895 0 0) 0x7f5130003570 con 0x7f5144159c60 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 msgr2=0x7f514415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f514415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 >> v1:172.21.15.191:6789/0 conn(0x7f5144069560 legacy=0x7f51440ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514415a3d0 con 0x7f5144159c60 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f5134001470 con 0x7f5144159c60 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 >> v1:172.21.15.170:6789/0 conn(0x7f5144159c60 legacy=0x7f514415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5130003a90 0x7f5130003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f5130004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f51300051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514415e140 con 0x7f5130004810 2023-10-08T16:09:52.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513000bb60 con 0x7f5130003a90 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.050+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513000bd20 con 0x7f5144159890 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f5130004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f5149190700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5130003a90 0x7f5130003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f5149991700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f51300051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 -- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 msgr2=0x7f51300051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159890 0x7f51300051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 -- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5130003a90 msgr2=0x7f5130003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5130003a90 0x7f5130003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 -- 172.21.15.170:0/3970937273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130003570 con 0x7f5130004810 2023-10-08T16:09:52.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f514898f700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f5130004c80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f513400c390 tx=0x7f5134001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bae2f2b82866e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400cc60 con 0x7f5130004810 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400cdc0 con 0x7f5130004810 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.051+0000 7f513bfff700 1 -- 172.21.15.170:0/3970937273 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134005b50 con 0x7f5130004810 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 -- 172.21.15.170:0/3970937273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 msgr2=0x7f5130004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 --2- 172.21.15.170:0/3970937273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f5130004c80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f513400c390 tx=0x7f5134001e50 comp rx=0 tx=0).stop 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f5149991700 1 -- 172.21.15.170:0/3970937273 reap_dead start 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 -- 172.21.15.170:0/3970937273 shutdown_connections 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 -- 172.21.15.170:0/3970937273 >> 172.21.15.170:0/3970937273 conn(0x7f5144068e20 msgr2=0x7f51440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 -- 172.21.15.170:0/3970937273 shutdown_connections 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.052+0000 7f514b3f4700 1 -- 172.21.15.170:0/3970937273 wait complete. 2023-10-08T16:09:52.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.053+0000 7f514b3f4700 1 Processor -- start 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.053+0000 7f514b3f4700 1 -- start start 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 0x7f5144141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144141750 0x7f5144141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514413ee80 con 0x7f5130004810 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440c6f90 con 0x7f51440c7de0 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440c8250 con 0x7f5144141750 2023-10-08T16:09:52.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47844/0 (socket says 172.21.15.170:47844) 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 -- 172.21.15.170:0/4040950606 learned_addr learned my addr 172.21.15.170:0/4040950606 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149991700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144141750 0x7f5144141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514898f700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 0x7f5144141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 -- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144141750 msgr2=0x7f5144141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144141750 0x7f5144141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 -- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 msgr2=0x7f5144141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 0x7f5144141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 -- 172.21.15.170:0/4040950606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440b4dd0 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514898f700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 0x7f5144141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f5149190700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5140011d40 tx=0x7f51400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514000fac0 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514000fc20 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51440b44a0 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140014440 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.054+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51440b4a60 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.055+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514404c3c0 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.056+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5140014870 con 0x7f5130004810 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.056+0000 7f51397fa700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 0x7f51280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.056+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5140018030 con 0x7f5130004810 2023-10-08T16:09:52.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.056+0000 7f514898f700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 0x7f51280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.056+0000 7f514898f700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 0x7f51280696b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5144159350 tx=0x7f5134015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.060+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5140052330 con 0x7f5130004810 2023-10-08T16:09:52.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.243+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514406a5b0 con 0x7f5130004810 2023-10-08T16:09:52.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.244+0000 7f51397fa700 1 -- 172.21.15.170:0/4040950606 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f5140055950 con 0x7f5130004810 2023-10-08T16:09:52.345 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:52.345 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":77,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:39.474771+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 msgr2=0x7f51280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 0x7f51280696b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5144159350 tx=0x7f5134015000 comp rx=0 tx=0).stop 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 msgr2=0x7f51440c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5140011d40 tx=0x7f51400091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 shutdown_connections 2023-10-08T16:09:52.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144141750 0x7f5144141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5128067200 0x7f51280696b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5130004810 0x7f51440c78a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 --2- 172.21.15.170:0/4040950606 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51440c7de0 0x7f5144141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 >> 172.21.15.170:0/4040950606 conn(0x7f5144068e20 msgr2=0x7f514415b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 shutdown_connections 2023-10-08T16:09:52.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.246+0000 7f514b3f4700 1 -- 172.21.15.170:0/4040950606 wait complete. 2023-10-08T16:09:52.349 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:09:52.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:52.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 Processor -- start 2023-10-08T16:09:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 -- start start 2023-10-08T16:09:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 0x7f136c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c15c2a0 con 0x7f136c141500 2023-10-08T16:09:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c15b690 con 0x7f136c06bcd0 2023-10-08T16:09:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f1371793700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c15b970 con 0x7f136c0cd1e0 2023-10-08T16:09:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.462+0000 7f13627fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f136c141500 0x7f136c0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57372/0 (socket says 172.21.15.170:57372) 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f13627fc700 1 -- 172.21.15.170:0/2639598383 learned_addr learned my addr 172.21.15.170:0/2639598383 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136b7fe700 1 --2- 172.21.15.170:0/2639598383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 0x7f136c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822649882 0 0) 0x7f136c15c2a0 con 0x7f136c141500 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1358003570 con 0x7f136c141500 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838933526 0 0) 0x7f136c15b690 con 0x7f136c06bcd0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f136c15c2a0 con 0x7f136c06bcd0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136b7fe700 1 -- 172.21.15.170:0/2639598383 >> v1:172.21.15.191:6789/0 conn(0x7f136c06bcd0 legacy=0x7f136c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136b7fe700 1 -- 172.21.15.170:0/2639598383 >> v1:172.21.15.170:6789/0 conn(0x7f136c141500 legacy=0x7f136c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136b7fe700 1 -- 172.21.15.170:0/2639598383 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c15bc50 con 0x7f136c0cd1e0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136b7fe700 1 --2- 172.21.15.170:0/2639598383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 0x7f136c0c9510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1364009e90 tx=0x7f1364009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97952f89289e05c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136400e070 con 0x7f136c0cd1e0 2023-10-08T16:09:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136400bb80 con 0x7f136c0cd1e0 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.463+0000 7f136a7fc700 1 -- 172.21.15.170:0/2639598383 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364012430 con 0x7f136c0cd1e0 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 -- 172.21.15.170:0/2639598383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 msgr2=0x7f136c0c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 --2- 172.21.15.170:0/2639598383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 0x7f136c0c9510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1364009e90 tx=0x7f1364009780 comp rx=0 tx=0).stop 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 -- 172.21.15.170:0/2639598383 shutdown_connections 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 --2- 172.21.15.170:0/2639598383 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c0cd1e0 0x7f136c0c9510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 -- 172.21.15.170:0/2639598383 >> 172.21.15.170:0/2639598383 conn(0x7f136c0ce600 msgr2=0x7f136c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 -- 172.21.15.170:0/2639598383 shutdown_connections 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.464+0000 7f1371793700 1 -- 172.21.15.170:0/2639598383 wait complete. 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.465+0000 7f1371793700 1 Processor -- start 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.465+0000 7f1371793700 1 -- start start 2023-10-08T16:09:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0cd1e0 0x7f136c13afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 0x7f136c13e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c13ab30 con 0x7f136c06bcd0 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c139c40 con 0x7f136c0cd1e0 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f1371793700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c139f20 con 0x7f136c141500 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47860/0 (socket says 172.21.15.170:47860) 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 -- 172.21.15.170:0/852616333 learned_addr learned my addr 172.21.15.170:0/852616333 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136b7fe700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 0x7f136c13e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f13627fc700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0cd1e0 0x7f136c13afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 -- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 msgr2=0x7f136c13e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 0x7f136c13e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 -- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0cd1e0 msgr2=0x7f136c13afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0cd1e0 0x7f136c13afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 -- 172.21.15.170:0/852616333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364009a20 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136b7fe700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 0x7f136c13e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.466+0000 7f136affd700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c7c0 tx=0x7f135c007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.467+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c010510 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.467+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c010670 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.467+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c13a200 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.467+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c010a30 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.467+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c13a7c0 con 0x7f136c06bcd0 2023-10-08T16:09:52.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.468+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c13ae10 con 0x7f136c06bcd0 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.468+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f135c01b400 con 0x7f136c06bcd0 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.468+0000 7f13637fe700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 0x7f134c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.468+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f135c084640 con 0x7f136c06bcd0 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.468+0000 7f13627fc700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 0x7f134c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.469+0000 7f13627fc700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 0x7f134c0696b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f136c13d180 tx=0x7f1354009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.473+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f135c051560 con 0x7f136c06bcd0 2023-10-08T16:09:52.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.699+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f136c04c3c0 con 0x7f136c06bcd0 2023-10-08T16:09:52.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.700+0000 7f13637fe700 1 -- 172.21.15.170:0/852616333 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f135c054b80 con 0x7f136c06bcd0 2023-10-08T16:09:52.701 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:52.701 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 msgr2=0x7f134c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 0x7f134c0696b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f136c13d180 tx=0x7f1354009040 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 msgr2=0x7f136c0b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f135c00c7c0 tx=0x7f135c007dc0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 shutdown_connections 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f136c141500 0x7f136c13e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f134c067200 0x7f134c0696b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f136c06bcd0 0x7f136c0b1d60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 --2- 172.21.15.170:0/852616333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f136c0cd1e0 0x7f136c13afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 >> 172.21.15.170:0/852616333 conn(0x7f136c0ce600 msgr2=0x7f136c0c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 shutdown_connections 2023-10-08T16:09:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.702+0000 7f1371793700 1 -- 172.21.15.170:0/852616333 wait complete. 2023-10-08T16:09:52.705 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2023-10-08T16:09:52.715 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:52.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.830+0000 7f75dd84c700 1 Processor -- start 2023-10-08T16:09:52.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.830+0000 7f75dd84c700 1 -- start start 2023-10-08T16:09:52.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.830+0000 7f75dd84c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75d8068550 0x7f75d8068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75dd84c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f75d806f810 0x7f75d8159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75dd84c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d806ec70 con 0x7f75d815e010 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75dd84c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d80b3430 con 0x7f75d8068550 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75dd84c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d806fc70 con 0x7f75d806f810 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75d8068550 0x7f75d8068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f75d8068550 0x7f75d8068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47876/0 (socket says 172.21.15.170:47876) 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 -- 172.21.15.170:0/2850234391 learned_addr learned my addr 172.21.15.170:0/2850234391 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:52.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 -- 172.21.15.170:0/2850234391 >> v2:172.21.15.191:3300/0 conn(0x7f75d806f810 msgr2=0x7f75d8159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d77fe700 1 --2- 172.21.15.170:0/2850234391 >> v2:172.21.15.191:3300/0 conn(0x7f75d806f810 0x7f75d8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 --2- 172.21.15.170:0/2850234391 >> v2:172.21.15.191:3300/0 conn(0x7f75d806f810 0x7f75d8159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 -- 172.21.15.170:0/2850234391 >> v1:172.21.15.191:6789/0 conn(0x7f75d815e010 legacy=0x7f75d806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 -- 172.21.15.170:0/2850234391 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80b2e70 con 0x7f75d8068550 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d6ffd700 1 --2- 172.21.15.170:0/2850234391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d8068550 0x7f75d8068920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aceb0 tx=0x7f75c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb6f000b3b55734 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d67fc700 1 -- 172.21.15.170:0/2850234391 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800b820 con 0x7f75d8068550 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d67fc700 1 -- 172.21.15.170:0/2850234391 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c800b980 con 0x7f75d8068550 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.831+0000 7f75d67fc700 1 -- 172.21.15.170:0/2850234391 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800bd20 con 0x7f75d8068550 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 -- 172.21.15.170:0/2850234391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d8068550 msgr2=0x7f75d8068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 --2- 172.21.15.170:0/2850234391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d8068550 0x7f75d8068920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aceb0 tx=0x7f75c8009250 comp rx=0 tx=0).stop 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 -- 172.21.15.170:0/2850234391 shutdown_connections 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 --2- 172.21.15.170:0/2850234391 >> v2:172.21.15.191:3300/0 conn(0x7f75d806f810 0x7f75d8159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 --2- 172.21.15.170:0/2850234391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d8068550 0x7f75d8068920 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 -- 172.21.15.170:0/2850234391 >> 172.21.15.170:0/2850234391 conn(0x7f75d80cd480 msgr2=0x7f75d80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 -- 172.21.15.170:0/2850234391 shutdown_connections 2023-10-08T16:09:52.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.832+0000 7f75dd84c700 1 -- 172.21.15.170:0/2850234391 wait complete. 2023-10-08T16:09:52.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.833+0000 7f75dd84c700 1 Processor -- start 2023-10-08T16:09:52.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.833+0000 7f75dd84c700 1 -- start start 2023-10-08T16:09:52.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d8068550 0x7f75d8141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 0x7f75d815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815ae30 con 0x7f75d815e010 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815b100 con 0x7f75d806f810 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75dd84c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815b3e0 con 0x7f75d8068550 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47886/0 (socket says 172.21.15.170:47886) 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 -- 172.21.15.170:0/1737931421 learned_addr learned my addr 172.21.15.170:0/1737931421 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d6ffd700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d8068550 0x7f75d8141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75ce7fc700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 0x7f75d815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 -- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d8068550 msgr2=0x7f75d8141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d8068550 0x7f75d8141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 -- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 msgr2=0x7f75d815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 0x7f75d815a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 -- 172.21.15.170:0/1737931421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80094f0 con 0x7f75d815e010 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75ce7fc700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 0x7f75d815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.834+0000 7f75d77fe700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f75d000ceb0 tx=0x7f75d0007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.835+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d000ae40 con 0x7f75d815e010 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.835+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d0016430 con 0x7f75d815e010 2023-10-08T16:09:52.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.835+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d815b6c0 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.835+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d00167d0 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.835+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d815bc80 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.836+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d815bea0 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.836+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f75d0003020 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.836+0000 7f75cf7fe700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 0x7f75b80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.836+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f75d0012070 con 0x7f75d815e010 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.836+0000 7f75d6ffd700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 0x7f75b80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:52.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.837+0000 7f75d6ffd700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 0x7f75b80696b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aceb0 tx=0x7f75c8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:52.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:52.841+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75d0052740 con 0x7f75d815e010 2023-10-08T16:09:53.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.063+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75d804c3c0 con 0x7f75d815e010 2023-10-08T16:09:53.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.063+0000 7f75cf7fe700 1 -- 172.21.15.170:0/1737931421 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f75d0055d60 con 0x7f75d815e010 2023-10-08T16:09:53.065 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:53.065 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:09:38.021095+0000","modified":"2023-10-08T16:09:41.650491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6036},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6036":{"gid":6036,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/3424487232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3424487232},{"type":"v1","addr":"172.21.15.170:6837","nonce":3424487232}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T16:09:53.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 msgr2=0x7f75b80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 0x7f75b80696b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aceb0 tx=0x7f75c8016000 comp rx=0 tx=0).stop 2023-10-08T16:09:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 msgr2=0x7f75d815a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f75d000ceb0 tx=0x7f75d0007870 comp rx=0 tx=0).stop 2023-10-08T16:09:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 shutdown_connections 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f75b8067200 0x7f75b80696b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75d815e010 0x7f75d815a8f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f75d8068550 0x7f75d8141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 --2- 172.21.15.170:0/1737931421 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f75d806f810 0x7f75d815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 >> 172.21.15.170:0/1737931421 conn(0x7f75d80cd480 msgr2=0x7f75d806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 shutdown_connections 2023-10-08T16:09:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.066+0000 7f75dd84c700 1 -- 172.21.15.170:0/1737931421 wait complete. 2023-10-08T16:09:53.117 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 48 2023-10-08T16:09:53.117 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:09:53.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.231+0000 7f9837485700 1 Processor -- start 2023-10-08T16:09:53.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 -- start start 2023-10-08T16:09:53.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 0x7f98300cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 0x7f983006ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f983006b350 0x7f98300c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300cc210 con 0x7f983006b350 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98300cc4f0 con 0x7f983006aa40 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9837485700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300cc7d0 con 0x7f983006bac0 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.232+0000 7f9835a22700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f983006b350 0x7f98300c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f983006b350 0x7f98300c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47894/0 (socket says 172.21.15.170:47894) 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 -- 172.21.15.170:0/3579913221 learned_addr learned my addr 172.21.15.170:0/3579913221 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9834a20700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 0x7f983006ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835221700 1 --2- 172.21.15.170:0/3579913221 >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 0x7f98300cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 -- 172.21.15.170:0/3579913221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 msgr2=0x7f983006ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 0x7f983006ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 -- 172.21.15.170:0/3579913221 >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 msgr2=0x7f98300cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 --2- 172.21.15.170:0/3579913221 >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 0x7f98300cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 -- 172.21.15.170:0/3579913221 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98300ccab0 con 0x7f983006b350 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835221700 1 --2- 172.21.15.170:0/3579913221 >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 0x7f98300cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9834a20700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 0x7f983006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9835a22700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f98300c81d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f983006e570 tx=0x7f9828009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e261070996a99f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9827fff700 1 -- 172.21.15.170:0/3579913221 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982800e070 con 0x7f983006b350 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9827fff700 1 -- 172.21.15.170:0/3579913221 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982800ba50 con 0x7f983006b350 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.233+0000 7f9827fff700 1 -- 172.21.15.170:0/3579913221 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98280092e0 con 0x7f983006b350 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 -- 172.21.15.170:0/3579913221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 msgr2=0x7f98300c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f98300c81d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f983006e570 tx=0x7f9828009840 comp rx=0 tx=0).stop 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 -- 172.21.15.170:0/3579913221 shutdown_connections 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f98300c81d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 --2- 172.21.15.170:0/3579913221 >> v2:172.21.15.191:3300/0 conn(0x7f983006aa40 0x7f98300cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 --2- 172.21.15.170:0/3579913221 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006bac0 0x7f983006ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 -- 172.21.15.170:0/3579913221 >> 172.21.15.170:0/3579913221 conn(0x7f98300cd670 msgr2=0x7f98300cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 -- 172.21.15.170:0/3579913221 shutdown_connections 2023-10-08T16:09:53.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.234+0000 7f9837485700 1 -- 172.21.15.170:0/3579913221 wait complete. 2023-10-08T16:09:53.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.235+0000 7f9837485700 1 Processor -- start 2023-10-08T16:09:53.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.235+0000 7f9837485700 1 -- start start 2023-10-08T16:09:53.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 0x7f98300b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 0x7f98300b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300b1670 con 0x7f983006b350 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98300b0a60 con 0x7f983006bac0 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9837485700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98300b0d40 con 0x7f983006aa40 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47906/0 (socket says 172.21.15.170:47906) 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 -- 172.21.15.170:0/3638436534 learned_addr learned my addr 172.21.15.170:0/3638436534 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9835221700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 0x7f98300b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9835a22700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 0x7f98300b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 -- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 msgr2=0x7f98300b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 0x7f98300b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 -- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 msgr2=0x7f98300b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 0x7f98300b2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 -- 172.21.15.170:0/3638436534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828009ae0 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9835a22700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 0x7f98300b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9835221700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 0x7f98300b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f9834a20700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f982000c880 tx=0x7f9820008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820011600 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.236+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9820011760 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.237+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98300b1020 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.237+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9820011b20 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.237+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9830152000 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.237+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98300c5860 con 0x7f983006b350 2023-10-08T16:09:53.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.238+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f98200193b0 con 0x7f983006b350 2023-10-08T16:09:53.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.238+0000 7f98257fa700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 0x7f98140696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.238+0000 7f9835221700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 0x7f98140696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.238+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9820084a80 con 0x7f983006b350 2023-10-08T16:09:53.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.239+0000 7f9835221700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 0x7f98140696b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f982c006fd0 tx=0x7f982c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.243+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9820086e60 con 0x7f983006b350 2023-10-08T16:09:53.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.427+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f983004c3c0 con 0x7f983006b350 2023-10-08T16:09:53.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.428+0000 7f98257fa700 1 -- 172.21.15.170:0/3638436534 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f9820051890 con 0x7f983006b350 2023-10-08T16:09:53.430 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":77,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:09:39.474771+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:09:36.357276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T16:09:36.982584+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.430+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 msgr2=0x7f98140696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.430+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 0x7f98140696b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f982c006fd0 tx=0x7f982c008040 comp rx=0 tx=0).stop 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 msgr2=0x7f9830151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f982000c880 tx=0x7f9820008040 comp rx=0 tx=0).stop 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 shutdown_connections 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f983006bac0 0x7f98300b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f983006aa40 0x7f98300b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9814067200 0x7f98140696b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 --2- 172.21.15.170:0/3638436534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983006b350 0x7f9830151ac0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 >> 172.21.15.170:0/3638436534 conn(0x7f98300cd670 msgr2=0x7f9830153940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:53.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 shutdown_connections 2023-10-08T16:09:53.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.431+0000 7f9837485700 1 -- 172.21.15.170:0/3638436534 wait complete. 2023-10-08T16:09:53.444 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:09:53.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.559+0000 7fa8ec1d1700 1 Processor -- start 2023-10-08T16:09:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.559+0000 7fa8ec1d1700 1 -- start start 2023-10-08T16:09:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ec1d1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa8e4159890 0x7fa8e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ec1d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8e40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ec1d1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e4069ee0 con 0x7fa8e4159c60 2023-10-08T16:09:53.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ec1d1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e415a160 con 0x7fa8e4159890 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ec1d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40d13a0 con 0x7fa8e4069560 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e976c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa8e4159c60 0x7fa8e415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57382/0 (socket says 172.21.15.170:57382) 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e976c700 1 -- 172.21.15.170:0/592287339 learned_addr learned my addr 172.21.15.170:0/592287339 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423315610 0 0) 0x7fa8e4069ee0 con 0x7fa8e4159c60 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d0003570 con 0x7fa8e4159c60 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8ea76e700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8e40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e9f6d700 1 --2- 172.21.15.170:0/592287339 >> v2:172.21.15.191:3300/0 conn(0x7fa8e4159890 0x7fa8e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325494648 0 0) 0x7fa8d0003570 con 0x7fa8e4159c60 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 msgr2=0x7fa8e40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8e40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 >> v2:172.21.15.191:3300/0 conn(0x7fa8e4159890 msgr2=0x7fa8e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 --2- 172.21.15.170:0/592287339 >> v2:172.21.15.191:3300/0 conn(0x7fa8e4159890 0x7fa8e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e415e140 con 0x7fa8e4159c60 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fa8d4001470 con 0x7fa8e4159c60 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 >> v1:172.21.15.170:6789/0 conn(0x7fa8e4159c60 legacy=0x7fa8e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8d0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0004d50 0x7fa8d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e415a160 con 0x7fa8d0003fe0 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40d13a0 con 0x7fa8d0004d50 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d000bb40 con 0x7fa8e4069560 2023-10-08T16:09:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e976c700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.560+0000 7fa8e9f6d700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8d0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ea76e700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0004d50 0x7fa8d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 -- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 msgr2=0x7fa8d0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4069560 0x7fa8d0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 -- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0004d50 msgr2=0x7fa8d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8d0004d50 0x7fa8d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 -- 172.21.15.170:0/592287339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0003570 con 0x7fa8d0003fe0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e976c700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8d0004810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c390 tx=0x7fa8d4001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d6c985e7373caa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d400cc60 con 0x7fa8d0003fe0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d400cde0 con 0x7fa8d0003fe0 2023-10-08T16:09:53.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8e8f6b700 1 -- 172.21.15.170:0/592287339 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4005b70 con 0x7fa8d0003fe0 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/592287339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 msgr2=0x7fa8d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/592287339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8d0004810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c390 tx=0x7fa8d4001e90 comp rx=0 tx=0).stop 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ea76e700 1 -- 172.21.15.170:0/592287339 reap_dead start 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/592287339 shutdown_connections 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.561+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/592287339 >> 172.21.15.170:0/592287339 conn(0x7fa8e4068e20 msgr2=0x7fa8e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.562+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/592287339 shutdown_connections 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.562+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/592287339 wait complete. 2023-10-08T16:09:53.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 Processor -- start 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 -- start start 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4169d60 0x7fa8e4150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e4151220 0x7fa8e40cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e416ad60 con 0x7fa8d0003fe0 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e4150850 con 0x7fa8e4151220 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ec1d1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e40ce730 con 0x7fa8e4169d60 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8e9f6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8e9f6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47932/0 (socket says 172.21.15.170:47932) 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8e9f6d700 1 -- 172.21.15.170:0/1801969121 learned_addr learned my addr 172.21.15.170:0/1801969121 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8ea76e700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e4151220 0x7fa8e40cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.563+0000 7fa8e976c700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4169d60 0x7fa8e4150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 -- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4169d60 msgr2=0x7fa8e4150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4169d60 0x7fa8e4150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 -- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e4151220 msgr2=0x7fa8e40cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e4151220 0x7fa8e40cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 -- 172.21.15.170:0/1801969121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e416a7a0 con 0x7fa8d0003fe0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8e9f6d700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0010d60 tx=0x7fa8e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000e920 con 0x7fa8d0003fe0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000ea80 con 0x7fa8d0003fe0 2023-10-08T16:09:53.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e416aa80 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000ee20 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.564+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e414ffa0 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.565+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e404c3c0 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.566+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa8e0014640 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.566+0000 7fa8d9ffb700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 0x7fa8c80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.566+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa8e0085070 con 0x7fa8d0003fe0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.566+0000 7fa8e976c700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 0x7fa8c80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.567+0000 7fa8e976c700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 0x7fa8c80696b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001ed0 tx=0x7fa8d4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:53.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.570+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8e0051f90 con 0x7fa8d0003fe0 2023-10-08T16:09:53.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.798+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8e406a5b0 con 0x7fa8d0003fe0 2023-10-08T16:09:53.800 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:09:53.799+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:09:53.805 INFO:tasks.ceph.mds.a.smithi170.stderr: -3042> 2023-10-08T16:09:33.536+0000 7f70b83b0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:09:54.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.802+0000 7fa8d9ffb700 1 -- 172.21.15.170:0/1801969121 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8e00555b0 con 0x7fa8d0003fe0 2023-10-08T16:09:54.114 INFO:tasks.ceph.mds.a.smithi170.stderr: -1795> 2023-10-08T16:09:44.479+0000 7f70b23a4700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 msgr2=0x7fa8c80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 0x7fa8c80696b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4001ed0 tx=0x7fa8d4015000 comp rx=0 tx=0).stop 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 msgr2=0x7fa8e40ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0010d60 tx=0x7fa8e0008040 comp rx=0 tx=0).stop 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 shutdown_connections 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa8e4151220 0x7fa8e40cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa8c8067200 0x7fa8c80696b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa8d0003fe0 0x7fa8e40ce1f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 --2- 172.21.15.170:0/1801969121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa8e4169d60 0x7fa8e4150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 >> 172.21.15.170:0/1801969121 conn(0x7fa8e4068e20 msgr2=0x7fa8e415ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:54.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 shutdown_connections 2023-10-08T16:09:54.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:53.805+0000 7fa8ec1d1700 1 -- 172.21.15.170:0/1801969121 wait complete. 2023-10-08T16:09:54.117 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:09:54.117 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:09:54.118 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:09:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.234+0000 7f1736269700 1 Processor -- start 2023-10-08T16:09:54.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.234+0000 7f1736269700 1 -- start start 2023-10-08T16:09:54.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1736269700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f173015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1736269700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f173015db10 0x7f1730069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1736269700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f173015a160 con 0x7f1730159890 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1736269700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17300d13a0 con 0x7f173015db10 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1736269700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173015e140 con 0x7f1730159c60 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1734806700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f173015db10 0x7f1730069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1734806700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f173015db10 0x7f1730069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47964/0 (socket says 172.21.15.170:47964) 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f1734806700 1 -- 172.21.15.170:0/1735248950 learned_addr learned my addr 172.21.15.170:0/1735248950 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.235+0000 7f172f7fe700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f173015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1734806700 1 -- 172.21.15.170:0/1735248950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 msgr2=0x7f173015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1734806700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f173015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1734806700 1 -- 172.21.15.170:0/1735248950 >> v1:172.21.15.191:6789/0 conn(0x7f1730159890 legacy=0x7f173015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1734806700 1 -- 172.21.15.170:0/1735248950 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173015a3d0 con 0x7f173015db10 2023-10-08T16:09:54.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1734806700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f173015db10 0x7f1730069560 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1728009f20 tx=0x7f1728009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a2a193a3b384c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f172effd700 1 -- 172.21.15.170:0/1735248950 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172800e070 con 0x7f173015db10 2023-10-08T16:09:54.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f172effd700 1 -- 172.21.15.170:0/1735248950 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172800b9d0 con 0x7f173015db10 2023-10-08T16:09:54.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f172effd700 1 -- 172.21.15.170:0/1735248950 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280092e0 con 0x7f173015db10 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 -- 172.21.15.170:0/1735248950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f173015db10 msgr2=0x7f1730069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f173015db10 0x7f1730069560 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f1728009f20 tx=0x7f1728009840 comp rx=0 tx=0).stop 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 -- 172.21.15.170:0/1735248950 shutdown_connections 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f173015db10 0x7f1730069560 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 --2- 172.21.15.170:0/1735248950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f173015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.236+0000 7f1736269700 1 -- 172.21.15.170:0/1735248950 >> 172.21.15.170:0/1735248950 conn(0x7f1730068e20 msgr2=0x7f17300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.237+0000 7f1736269700 1 -- 172.21.15.170:0/1735248950 shutdown_connections 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.237+0000 7f1736269700 1 -- 172.21.15.170:0/1735248950 wait complete. 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 Processor -- start 2023-10-08T16:09:54.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 -- start start 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f17300cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 0x7f17300cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17300b2770 con 0x7f1730159890 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1730169880 con 0x7f173015db10 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f1736269700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173016a120 con 0x7f1730159c60 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.238+0000 7f172ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47970/0 (socket says 172.21.15.170:47970) 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 -- 172.21.15.170:0/3573640101 learned_addr learned my addr 172.21.15.170:0/3573640101 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172f7fe700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f17300cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1734806700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 0x7f17300cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 -- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 msgr2=0x7f17300cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f17300cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 -- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 msgr2=0x7f17300cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 0x7f17300cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 -- 172.21.15.170:0/3573640101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009ae0 con 0x7f1730159890 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1734806700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 0x7f17300cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f172ffff700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f172000c5c0 tx=0x7f172000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011590 con 0x7f1730159890 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720011710 con 0x7f1730159890 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011ad0 con 0x7f1730159890 2023-10-08T16:09:54.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17300b2010 con 0x7f1730159890 2023-10-08T16:09:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.239+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1730169ad0 con 0x7f1730159890 2023-10-08T16:09:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.240+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173004c3c0 con 0x7f1730159890 2023-10-08T16:09:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.240+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f172001b3b0 con 0x7f1730159890 2023-10-08T16:09:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.241+0000 7f1717fff700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 0x7f171c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.241+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f1720083f00 con 0x7f1730159890 2023-10-08T16:09:54.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.241+0000 7f172f7fe700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 0x7f171c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.241+0000 7f172f7fe700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 0x7f171c0696b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1730152e30 tx=0x7f17240091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.245+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1720050dd0 con 0x7f1730159890 2023-10-08T16:09:54.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.473+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1730155b10 con 0x7f1730159890 2023-10-08T16:09:54.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.808+0000 7f1717fff700 1 -- 172.21.15.170:0/3573640101 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17200170a0 con 0x7f1730159890 2023-10-08T16:09:54.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 msgr2=0x7f171c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 0x7f171c0696b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1730152e30 tx=0x7f17240091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 msgr2=0x7f17300b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f172000c5c0 tx=0x7f172000c8d0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 shutdown_connections 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f173015db10 0x7f17300cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f171c067200 0x7f171c0696b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1730159890 0x7f17300b1ad0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 --2- 172.21.15.170:0/3573640101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1730159c60 0x7f17300cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 >> 172.21.15.170:0/3573640101 conn(0x7f1730068e20 msgr2=0x7f1730156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 shutdown_connections 2023-10-08T16:09:54.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.811+0000 7f1736269700 1 -- 172.21.15.170:0/3573640101 wait complete. 2023-10-08T16:09:54.824 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:09:54.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.941+0000 7fbe5b8f1700 1 Processor -- start 2023-10-08T16:09:54.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.942+0000 7fbe5b8f1700 1 -- start start 2023-10-08T16:09:54.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.942+0000 7fbe5b8f1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe540b55a0 0x7fbe54164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.942+0000 7fbe5b8f1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540cfa60 con 0x7fbe540cd910 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.942+0000 7fbe5b8f1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540b5b00 con 0x7fbe54159ec0 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.942+0000 7fbe5b8f1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe540b5da0 con 0x7fbe540b55a0 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe59e8e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe540b55a0 0x7fbe54164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe58e8c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbe540cd910 0x7fbe540ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57414/0 (socket says 172.21.15.170:57414) 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe58e8c700 1 -- 172.21.15.170:0/4024156424 learned_addr learned my addr 172.21.15.170:0/4024156424 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe59e8e700 1 -- 172.21.15.170:0/4024156424 >> v1:172.21.15.191:6789/0 conn(0x7fbe54159ec0 legacy=0x7fbe540b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe59e8e700 1 -- 172.21.15.170:0/4024156424 >> v1:172.21.15.170:6789/0 conn(0x7fbe540cd910 legacy=0x7fbe540ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe59e8e700 1 -- 172.21.15.170:0/4024156424 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540b6080 con 0x7fbe540b55a0 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe4bfff700 1 -- 172.21.15.170:0/4024156424 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978911938 0 0) 0x7fbe540cfa60 con 0x7fbe540cd910 2023-10-08T16:09:54.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe59e8e700 1 --2- 172.21.15.170:0/4024156424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540b55a0 0x7fbe54164400 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009910 tx=0x7fbe4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc8f15e859c835d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe4bfff700 1 -- 172.21.15.170:0/4024156424 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00b7e0 con 0x7fbe540b55a0 2023-10-08T16:09:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe4bfff700 1 -- 172.21.15.170:0/4024156424 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00b940 con 0x7fbe540b55a0 2023-10-08T16:09:54.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.943+0000 7fbe4bfff700 1 -- 172.21.15.170:0/4024156424 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00bce0 con 0x7fbe540b55a0 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/4024156424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540b55a0 msgr2=0x7fbe54164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/4024156424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540b55a0 0x7fbe54164400 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009910 tx=0x7fbe4c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/4024156424 shutdown_connections 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/4024156424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe540b55a0 0x7fbe54164400 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/4024156424 >> 172.21.15.170:0/4024156424 conn(0x7fbe540999f0 msgr2=0x7fbe540b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/4024156424 shutdown_connections 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.944+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/4024156424 wait complete. 2023-10-08T16:09:54.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.945+0000 7fbe5b8f1700 1 Processor -- start 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.945+0000 7fbe5b8f1700 1 -- start start 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 0x7fbe540c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 0x7fbe541699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5416de30 con 0x7fbe54169f30 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5409abe0 con 0x7fbe540b55a0 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5416db50 con 0x7fbe54159ec0 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47976/0 (socket says 172.21.15.170:47976) 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 -- 172.21.15.170:0/2689656432 learned_addr learned my addr 172.21.15.170:0/2689656432 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:54.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe58e8c700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 0x7fbe541699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 -- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 msgr2=0x7fbe541699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5968d700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 0x7fbe540c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 0x7fbe541699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 -- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 msgr2=0x7fbe540c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 0x7fbe540c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 -- 172.21.15.170:0/2689656432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c009420 con 0x7fbe54169f30 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5968d700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 0x7fbe540c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe58e8c700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 0x7fbe541699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe59e8e700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0030c0 tx=0x7fbe4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c01b520 con 0x7fbe54169f30 2023-10-08T16:09:54.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c020070 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5409a590 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c01b8b0 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.946+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe540c6de0 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.947+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe540c6fd0 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.948+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fbe4c022030 con 0x7fbe54169f30 2023-10-08T16:09:54.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.948+0000 7fbe497fa700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 0x7fbe380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:54.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.948+0000 7fbe5968d700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 0x7fbe380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:54.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.948+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbe4c086c20 con 0x7fbe54169f30 2023-10-08T16:09:54.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.949+0000 7fbe5968d700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 0x7fbe380696b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d220 tx=0x7fbe50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:54.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:54.953+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe4c053a10 con 0x7fbe54169f30 2023-10-08T16:09:55.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.134+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe540c6fd0 con 0x7fbe54169f30 2023-10-08T16:09:55.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.812+0000 7fbe497fa700 1 -- 172.21.15.170:0/2689656432 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbe4c057030 con 0x7fbe54169f30 2023-10-08T16:09:55.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.815+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 msgr2=0x7fbe380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.815+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 0x7fbe380696b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d220 tx=0x7fbe50009040 comp rx=0 tx=0).stop 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.815+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 msgr2=0x7fbe5416cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.815+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0030c0 tx=0x7fbe4c016000 comp rx=0 tx=0).stop 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 shutdown_connections 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbe38067200 0x7fbe380696b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:55.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe54169f30 0x7fbe5416cf70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:55.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe540b55a0 0x7fbe540c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:55.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 --2- 172.21.15.170:0/2689656432 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe54159ec0 0x7fbe541699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:55.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 >> 172.21.15.170:0/2689656432 conn(0x7fbe540999f0 msgr2=0x7fbe540c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:55.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 shutdown_connections 2023-10-08T16:09:55.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.816+0000 7fbe5b8f1700 1 -- 172.21.15.170:0/2689656432 wait complete. 2023-10-08T16:09:55.819 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:09:55.830 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:09:55.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.949+0000 7fe585a97700 1 Processor -- start 2023-10-08T16:09:55.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.949+0000 7fe585a97700 1 -- start start 2023-10-08T16:09:55.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585a97700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe5800b03c0 0x7fe5800cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585a97700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe5801591e0 0x7fe580159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585a97700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5800cfa00 con 0x7fe5800b0790 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585a97700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5800cb580 con 0x7fe5800b03c0 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585a97700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5800cf160 con 0x7fe5801591e0 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe584a95700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe5800b03c0 0x7fe5800cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe577fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe5800b0790 0x7fe5800b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35318/0 (socket says 172.21.15.170:35318) 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe577fff700 1 -- 172.21.15.170:0/2815792606 learned_addr learned my addr 172.21.15.170:0/2815792606 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe585296700 1 --2- 172.21.15.170:0/2815792606 >> v2:172.21.15.191:3300/0 conn(0x7fe5801591e0 0x7fe580159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe584a95700 1 -- 172.21.15.170:0/2815792606 >> v2:172.21.15.191:3300/0 conn(0x7fe5801591e0 msgr2=0x7fe580159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe584a95700 1 --2- 172.21.15.170:0/2815792606 >> v2:172.21.15.191:3300/0 conn(0x7fe5801591e0 0x7fe580159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe584a95700 1 -- 172.21.15.170:0/2815792606 >> v1:172.21.15.170:6789/0 conn(0x7fe5800b0790 legacy=0x7fe5800b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe5777fe700 1 -- 172.21.15.170:0/2815792606 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722588937 0 0) 0x7fe5800cfa00 con 0x7fe5800b0790 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.950+0000 7fe584a95700 1 -- 172.21.15.170:0/2815792606 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5800cf440 con 0x7fe5800b03c0 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe584a95700 1 --2- 172.21.15.170:0/2815792606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5800b03c0 0x7fe5800cea70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098e0 tx=0x7fe57c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4550efb877a9dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe5777fe700 1 -- 172.21.15.170:0/2815792606 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c009e80 con 0x7fe5800b03c0 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe5777fe700 1 -- 172.21.15.170:0/2815792606 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c00b820 con 0x7fe5800b03c0 2023-10-08T16:09:56.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe5777fe700 1 -- 172.21.15.170:0/2815792606 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c00bba0 con 0x7fe5800b03c0 2023-10-08T16:09:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 -- 172.21.15.170:0/2815792606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5800b03c0 msgr2=0x7fe5800cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 --2- 172.21.15.170:0/2815792606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5800b03c0 0x7fe5800cea70 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098e0 tx=0x7fe57c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 -- 172.21.15.170:0/2815792606 shutdown_connections 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 --2- 172.21.15.170:0/2815792606 >> v2:172.21.15.191:3300/0 conn(0x7fe5801591e0 0x7fe580159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 --2- 172.21.15.170:0/2815792606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5800b03c0 0x7fe5800cea70 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 -- 172.21.15.170:0/2815792606 >> 172.21.15.170:0/2815792606 conn(0x7fe580063230 msgr2=0x7fe580063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 -- 172.21.15.170:0/2815792606 shutdown_connections 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.951+0000 7fe585a97700 1 -- 172.21.15.170:0/2815792606 wait complete. 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.952+0000 7fe585a97700 1 Processor -- start 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 -- start start 2023-10-08T16:09:56.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5800b03c0 0x7fe5801585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5800b0790 0x7fe58015a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58015d500 con 0x7fe5801591e0 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58015c8f0 con 0x7fe5800b0790 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585a97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58015cbd0 con 0x7fe5800b03c0 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585296700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585296700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48650/0 (socket says 172.21.15.170:48650) 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe585296700 1 -- 172.21.15.170:0/2055018209 learned_addr learned my addr 172.21.15.170:0/2055018209 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe584a95700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5800b03c0 0x7fe5801585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.953+0000 7fe577fff700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5800b0790 0x7fe58015a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 -- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5800b03c0 msgr2=0x7fe5801585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5800b03c0 0x7fe5801585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 -- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5800b0790 msgr2=0x7fe58015a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5800b0790 0x7fe58015a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 -- 172.21.15.170:0/2055018209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009420 con 0x7fe5801591e0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585296700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe578009fa0 tx=0x7fe578009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57800ebc0 con 0x7fe5801591e0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57800ed20 con 0x7fe5801591e0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58015ceb0 con 0x7fe5801591e0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578014670 con 0x7fe5801591e0 2023-10-08T16:09:56.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.954+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58015d990 con 0x7fe5801591e0 2023-10-08T16:09:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.955+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58004c3c0 con 0x7fe5801591e0 2023-10-08T16:09:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.955+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe5780149e0 con 0x7fe5801591e0 2023-10-08T16:09:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.956+0000 7fe574ff9700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 0x7fe56c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.956+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fe578084ec0 con 0x7fe5801591e0 2023-10-08T16:09:56.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.956+0000 7fe584a95700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 0x7fe56c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.956+0000 7fe584a95700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 0x7fe56c0696b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0093f0 tx=0x7fe57c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:55.960+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe578051f40 con 0x7fe5801591e0 2023-10-08T16:09:56.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.142+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe58015d640 con 0x7fe5801591e0 2023-10-08T16:09:56.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.816+0000 7fe574ff9700 1 -- 172.21.15.170:0/2055018209 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe578055560 con 0x7fe5801591e0 2023-10-08T16:09:56.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.821+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 msgr2=0x7fe56c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.821+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 0x7fe56c0696b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0093f0 tx=0x7fe57c017000 comp rx=0 tx=0).stop 2023-10-08T16:09:56.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.821+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 msgr2=0x7fe58015acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.821+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe578009fa0 tx=0x7fe578009900 comp rx=0 tx=0).stop 2023-10-08T16:09:56.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 shutdown_connections 2023-10-08T16:09:56.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe56c067200 0x7fe56c0696b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5801591e0 0x7fe58015acd0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5800b03c0 0x7fe5801585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 --2- 172.21.15.170:0/2055018209 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5800b0790 0x7fe58015a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.823+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 >> 172.21.15.170:0/2055018209 conn(0x7fe580063230 msgr2=0x7fe5800b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:56.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.824+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 shutdown_connections 2023-10-08T16:09:56.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.824+0000 7fe585a97700 1 -- 172.21.15.170:0/2055018209 wait complete. 2023-10-08T16:09:56.827 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:09:56.839 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:09:56.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.959+0000 7f258576e700 1 Processor -- start 2023-10-08T16:09:56.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.959+0000 7f258576e700 1 -- start start 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f258576e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2580159890 0x7f258015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f258576e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 0x7f25800699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f258576e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2580069ee0 con 0x7f2580159c60 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f258576e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258015a160 con 0x7f2580159890 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f258576e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800d13a0 con 0x7f2580069560 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2580159890 0x7f258015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2580159890 0x7f258015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48664/0 (socket says 172.21.15.170:48664) 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 -- 172.21.15.170:0/1366618762 learned_addr learned my addr 172.21.15.170:0/1366618762 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:56.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257f7fe700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 0x7f25800699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 -- 172.21.15.170:0/1366618762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 msgr2=0x7f25800699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 0x7f25800699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 -- 172.21.15.170:0/1366618762 >> v1:172.21.15.191:6789/0 conn(0x7f2580159c60 legacy=0x7f258015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 -- 172.21.15.170:0/1366618762 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258015e140 con 0x7f2580159890 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257f7fe700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 0x7f25800699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.960+0000 7f257effd700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159890 0x7f258015abd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25700099b0 tx=0x7f2570009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8da58688a5dc219 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f257e7fc700 1 -- 172.21.15.170:0/1366618762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257000b820 con 0x7f2580159890 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f257e7fc700 1 -- 172.21.15.170:0/1366618762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257000b980 con 0x7f2580159890 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f257e7fc700 1 -- 172.21.15.170:0/1366618762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257000bd20 con 0x7f2580159890 2023-10-08T16:09:56.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 -- 172.21.15.170:0/1366618762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159890 msgr2=0x7f258015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159890 0x7f258015abd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25700099b0 tx=0x7f2570009250 comp rx=0 tx=0).stop 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 -- 172.21.15.170:0/1366618762 shutdown_connections 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580069560 0x7f25800699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 --2- 172.21.15.170:0/1366618762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159890 0x7f258015abd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 -- 172.21.15.170:0/1366618762 >> 172.21.15.170:0/1366618762 conn(0x7f2580068e20 msgr2=0x7f25800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 -- 172.21.15.170:0/1366618762 shutdown_connections 2023-10-08T16:09:56.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.961+0000 7f258576e700 1 -- 172.21.15.170:0/1366618762 wait complete. 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.962+0000 7f258576e700 1 Processor -- start 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.962+0000 7f258576e700 1 -- start start 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2580069560 0x7f25800cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580159890 0x7f258016a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800b3be0 con 0x7f2580159c60 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258016ae60 con 0x7f2580069560 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f258576e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2580158850 con 0x7f2580159890 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48676/0 (socket says 172.21.15.170:48676) 2023-10-08T16:09:56.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 -- 172.21.15.170:0/3676430737 learned_addr learned my addr 172.21.15.170:0/3676430737 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257effd700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2580069560 0x7f25800cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f25767fc700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580159890 0x7f258016a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 -- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580159890 msgr2=0x7f258016a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580159890 0x7f258016a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 -- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2580069560 msgr2=0x7f25800cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.963+0000 7f257f7fe700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2580069560 0x7f25800cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f257f7fe700 1 -- 172.21.15.170:0/3676430737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700094f0 con 0x7f2580159c60 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f257f7fe700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f257800cef0 tx=0x7f2578007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800ad20 con 0x7f2580159c60 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578016420 con 0x7f2580159c60 2023-10-08T16:09:56.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2580158b30 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800aea0 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.964+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800b3870 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.965+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258004c3c0 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.965+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2578016a70 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.965+0000 7f25777fe700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 0x7f25600696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.966+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2578012070 con 0x7f2580159c60 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.966+0000 7f257effd700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 0x7f25600696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:56.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.966+0000 7f257effd700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 0x7f25600696b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25700099b0 tx=0x7f2570016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:56.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:56.970+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f257804e5a0 con 0x7f2580159c60 2023-10-08T16:09:57.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.154+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f25800b3d20 con 0x7f2580159c60 2023-10-08T16:09:57.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.158+0000 7f25777fe700 1 -- 172.21.15.170:0/3676430737 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2578052110 con 0x7f2580159c60 2023-10-08T16:09:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 msgr2=0x7f25600696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 0x7f25600696b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f25700099b0 tx=0x7f2570016000 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 msgr2=0x7f25800b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f257800cef0 tx=0x7f2578007860 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 shutdown_connections 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2560067200 0x7f25600696b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2580159c60 0x7f25800b55f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2580069560 0x7f25800cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 --2- 172.21.15.170:0/3676430737 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2580159890 0x7f258016a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 >> 172.21.15.170:0/3676430737 conn(0x7f2580068e20 msgr2=0x7f2580153270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 shutdown_connections 2023-10-08T16:09:57.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.160+0000 7f258576e700 1 -- 172.21.15.170:0/3676430737 wait complete. 2023-10-08T16:09:57.172 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag' 2023-10-08T16:09:57.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.287+0000 7ff6fc773700 1 Processor -- start 2023-10-08T16:09:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.287+0000 7ff6fc773700 1 -- start start 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fc773700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6f4141580 0x7ff6f40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fc773700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40ccfb0 con 0x7ff6f41411b0 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fc773700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f40b3e50 con 0x7ff6f40caf80 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fc773700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f40b40d0 con 0x7ff6f4141580 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fa50f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff6f41411b0 0x7ff6f415bfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35324/0 (socket says 172.21.15.170:35324) 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6fa50f700 1 -- 172.21.15.170:0/3337027976 learned_addr learned my addr 172.21.15.170:0/3337027976 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:57.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4141580 0x7ff6f40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570089465 0 0) 0x7ff6f40ccfb0 con 0x7ff6f41411b0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0003570 con 0x7ff6f41411b0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641868506 0 0) 0x7ff6f40b3e50 con 0x7ff6f40caf80 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f40ccfb0 con 0x7ff6f40caf80 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395909081 0 0) 0x7ff6e0003570 con 0x7ff6f41411b0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4141580 msgr2=0x7ff6f40cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 --2- 172.21.15.170:0/3337027976 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4141580 0x7ff6f40cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 >> v1:172.21.15.191:6789/0 conn(0x7ff6f40caf80 legacy=0x7ff6f40b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40b43b0 con 0x7ff6f41411b0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7ff6f00017f0 con 0x7ff6f41411b0 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 >> v1:172.21.15.170:6789/0 conn(0x7ff6f41411b0 legacy=0x7ff6f415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003bf0 0x7ff6e0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f40b40d0 con 0x7ff6e0004810 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000c0b0 con 0x7ff6e00051c0 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e000c270 con 0x7ff6e0003bf0 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> v2:172.21.15.191:3300/0 conn(0x7ff6f4141580 0x7ff6f40cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.288+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6fa50f700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003bf0 0x7ff6e0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6fad10700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 -- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003bf0 msgr2=0x7ff6e0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:57.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0003bf0 0x7ff6e0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 -- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e00051c0 msgr2=0x7ff6e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6e00051c0 0x7ff6e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 -- 172.21.15.170:0/3337027976 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0003570 con 0x7ff6e0004810 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6fad10700 1 -- 172.21.15.170:0/3337027976 reap_dead start 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009db0 tx=0x7ff6e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d793c90ef1dc36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4009740 con 0x7ff6e0004810 2023-10-08T16:09:57.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400e920 con 0x7ff6e0004810 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6f950d700 1 -- 172.21.15.170:0/3337027976 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400eca0 con 0x7ff6e0004810 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6fc773700 1 -- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 msgr2=0x7ff6e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.289+0000 7ff6fc773700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009db0 tx=0x7ff6e40091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.290+0000 7ff6fc773700 1 -- 172.21.15.170:0/3337027976 shutdown_connections 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.290+0000 7ff6fc773700 1 --2- 172.21.15.170:0/3337027976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6e0004810 0x7ff6e0004c80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.290+0000 7ff6fc773700 1 -- 172.21.15.170:0/3337027976 >> 172.21.15.170:0/3337027976 conn(0x7ff6f406bb50 msgr2=0x7ff6f406bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.290+0000 7ff6fc773700 1 -- 172.21.15.170:0/3337027976 shutdown_connections 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.290+0000 7ff6fc773700 1 -- 172.21.15.170:0/3337027976 wait complete. 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 Processor -- start 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 -- start start 2023-10-08T16:09:57.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0004810 0x7ff6f41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f4158080 0x7ff6f41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4139950 con 0x7ff6f415d6b0 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f4139690 con 0x7ff6f4158080 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fc773700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f415e060 con 0x7ff6e0004810 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fad10700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.291+0000 7ff6fad10700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48698/0 (socket says 172.21.15.170:48698) 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 -- 172.21.15.170:0/1481940577 learned_addr learned my addr 172.21.15.170:0/1481940577 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fa50f700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0004810 0x7ff6f41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6f9d0e700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f4158080 0x7ff6f41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 -- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0004810 msgr2=0x7ff6f41574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0004810 0x7ff6f41574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 -- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f4158080 msgr2=0x7ff6f41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f4158080 0x7ff6f41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 -- 172.21.15.170:0/1481940577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f41593a0 con 0x7ff6f415d6b0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fad10700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec002a80 tx=0x7ff6ec007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec006a30 con 0x7ff6f415d6b0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ec006bb0 con 0x7ff6f415d6b0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4158b00 con 0x7ff6f415d6b0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ec007030 con 0x7ff6f415d6b0 2023-10-08T16:09:57.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.292+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f41590c0 con 0x7ff6f415d6b0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.293+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f404c3c0 con 0x7ff6f415d6b0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.293+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ff6ec007460 con 0x7ff6f415d6b0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.294+0000 7ff6ea7fc700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 0x7ff6d80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.294+0000 7ff6fa50f700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 0x7ff6d80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.294+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff6ec084230 con 0x7ff6f415d6b0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.294+0000 7ff6fa50f700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 0x7ff6d80696b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0001190 tx=0x7ff6f0006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.298+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6ec051450 con 0x7ff6f415d6b0 2023-10-08T16:09:57.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.555+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]} v 0) v1 -- 0x7ff6f40ccc40 con 0x7ff6f415d6b0 2023-10-08T16:09:57.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.816+0000 7ff6ea7fc700 1 -- 172.21.15.170:0/1481940577 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag"]}]=0 v251) v1 ==== 158+0+0 (secure 0 0 0) 0x7ff6ec0190e0 con 0x7ff6f415d6b0 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 msgr2=0x7ff6d80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 0x7ff6d80696b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0001190 tx=0x7ff6f0006040 comp rx=0 tx=0).stop 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 msgr2=0x7ff6f415db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec002a80 tx=0x7ff6ec007d30 comp rx=0 tx=0).stop 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 shutdown_connections 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6d8067200 0x7ff6d80696b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f415d6b0 0x7ff6f415db20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6e0004810 0x7ff6f41574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 --2- 172.21.15.170:0/1481940577 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f4158080 0x7ff6f41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 >> 172.21.15.170:0/1481940577 conn(0x7ff6f406bb50 msgr2=0x7ff6f40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:57.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 shutdown_connections 2023-10-08T16:09:57.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.818+0000 7ff6fc773700 1 -- 172.21.15.170:0/1481940577 wait complete. 2023-10-08T16:09:57.830 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command ... ok 2023-10-08T16:09:57.831 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info --human_readable' command without subvolume group 2023-10-08T16:09:57.831 INFO:tasks.cephfs_test_runner:test_volume_info_with_human_readable_flag_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:09:57.831 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-08T16:09:57.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.944+0000 7fa3a28ab700 1 Processor -- start 2023-10-08T16:09:57.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 -- start start 2023-10-08T16:09:57.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa39c0ccf90 0x7fa39c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39c0cc300 con 0x7fa39c0cbf30 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39c156bf0 con 0x7fa39c0ccf90 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a28ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0ae950 con 0x7fa39c0b1420 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a0e48700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa39c0ccf90 0x7fa39c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a0e48700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa39c0ccf90 0x7fa39c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48704/0 (socket says 172.21.15.170:48704) 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.945+0000 7fa3a0e48700 1 -- 172.21.15.170:0/2936623756 learned_addr learned my addr 172.21.15.170:0/2936623756 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa39b7fe700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a0e48700 1 -- 172.21.15.170:0/2936623756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 msgr2=0x7fa39c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a0e48700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a0e48700 1 -- 172.21.15.170:0/2936623756 >> v1:172.21.15.191:6789/0 conn(0x7fa39c0cbf30 legacy=0x7fa39c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a0e48700 1 -- 172.21.15.170:0/2936623756 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0ad370 con 0x7fa39c0ccf90 2023-10-08T16:09:57.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a0e48700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c0c9740 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d1d0 tx=0x7fa39400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eeaf93b2d77ec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa39affd700 1 -- 172.21.15.170:0/2936623756 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394010070 con 0x7fa39c0ccf90 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa39affd700 1 -- 172.21.15.170:0/2936623756 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39400de70 con 0x7fa39c0ccf90 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa39affd700 1 -- 172.21.15.170:0/2936623756 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39400b300 con 0x7fa39c0ccf90 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a28ab700 1 -- 172.21.15.170:0/2936623756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 msgr2=0x7fa39c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.946+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c0c9740 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d1d0 tx=0x7fa39400ba50 comp rx=0 tx=0).stop 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 -- 172.21.15.170:0/2936623756 shutdown_connections 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c0c9740 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/2936623756 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 -- 172.21.15.170:0/2936623756 >> 172.21.15.170:0/2936623756 conn(0x7fa39c06a320 msgr2=0x7fa39c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 -- 172.21.15.170:0/2936623756 shutdown_connections 2023-10-08T16:09:57.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.947+0000 7fa3a28ab700 1 -- 172.21.15.170:0/2936623756 wait complete. 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.948+0000 7fa3a28ab700 1 Processor -- start 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.948+0000 7fa3a28ab700 1 -- start start 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c0cbf30 0x7fa39c168c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0c2ed0 con 0x7fa39c0ccf90 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0c2250 con 0x7fa39c0cbf30 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c0c2530 con 0x7fa39c0b1420 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48712/0 (socket says 172.21.15.170:48712) 2023-10-08T16:09:57.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 -- 172.21.15.170:0/634683115 learned_addr learned my addr 172.21.15.170:0/634683115 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa39b7fe700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c0cbf30 0x7fa39c168c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa39bfff700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 -- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 msgr2=0x7fa39c15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c15a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 -- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c0cbf30 msgr2=0x7fa39c168c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c0cbf30 0x7fa39c168c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 -- 172.21.15.170:0/634683115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39400bcf0 con 0x7fa39c0ccf90 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a0e48700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa39c064290 tx=0x7fa394019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa394009d50 con 0x7fa39c0ccf90 2023-10-08T16:09:57.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa394014430 con 0x7fa39c0ccf90 2023-10-08T16:09:57.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c0c2810 con 0x7fa39c0ccf90 2023-10-08T16:09:57.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3940147f0 con 0x7fa39c0ccf90 2023-10-08T16:09:57.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.949+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c15aa30 con 0x7fa39c0ccf90 2023-10-08T16:09:57.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.950+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c0c3010 con 0x7fa39c0ccf90 2023-10-08T16:09:57.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.951+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fa394014be0 con 0x7fa39c0ccf90 2023-10-08T16:09:57.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.952+0000 7fa383fff700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 0x7fa3880696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:57.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.952+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fa394088750 con 0x7fa39c0ccf90 2023-10-08T16:09:57.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.952+0000 7fa39bfff700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 0x7fa3880696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:57.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.952+0000 7fa39bfff700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 0x7fa3880696b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0027e0 tx=0x7fa38c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:57.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:57.955+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3940557c0 con 0x7fa39c0ccf90 2023-10-08T16:09:58.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.213+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fa39c0c31f0 con 0x7fa39c0ccf90 2023-10-08T16:09:58.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.819+0000 7fa383fff700 1 -- 172.21.15.170:0/634683115 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v252) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa394058de0 con 0x7fa39c0ccf90 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 msgr2=0x7fa3880696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 0x7fa3880696b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0027e0 tx=0x7fa38c009040 comp rx=0 tx=0).stop 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 msgr2=0x7fa39c155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa39c064290 tx=0x7fa394019000 comp rx=0 tx=0).stop 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 shutdown_connections 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa388067200 0x7fa3880696b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa39c0ccf90 0x7fa39c155d00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa39c0b1420 0x7fa39c15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 --2- 172.21.15.170:0/634683115 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa39c0cbf30 0x7fa39c168c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 >> 172.21.15.170:0/634683115 conn(0x7fa39c06a320 msgr2=0x7fa39c0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:58.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 shutdown_connections 2023-10-08T16:09:58.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.821+0000 7fa3a28ab700 1 -- 172.21.15.170:0/634683115 wait complete. 2023-10-08T16:09:58.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:09:58.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.952+0000 7f533120b700 1 Processor -- start 2023-10-08T16:09:58.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.952+0000 7f533120b700 1 -- start start 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f532c0b4e70 0x7f532c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f532c15d240 0x7f532c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0b2080 con 0x7f532c0b4e70 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0cfb00 con 0x7f532c15d240 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f533120b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0b5c40 con 0x7f532c0b16d0 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f532c0b4e70 0x7f532c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f532c0b4e70 0x7f532c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48728/0 (socket says 172.21.15.170:48728) 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 -- 172.21.15.170:0/3841919553 learned_addr learned my addr 172.21.15.170:0/3841919553 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532b7fe700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f5330a0a700 1 --2- 172.21.15.170:0/3841919553 >> v2:172.21.15.191:3300/0 conn(0x7f532c15d240 0x7f532c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 -- 172.21.15.170:0/3841919553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 msgr2=0x7f532c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 -- 172.21.15.170:0/3841919553 >> v2:172.21.15.191:3300/0 conn(0x7f532c15d240 msgr2=0x7f532c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 --2- 172.21.15.170:0/3841919553 >> v2:172.21.15.191:3300/0 conn(0x7f532c15d240 0x7f532c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 -- 172.21.15.170:0/3841919553 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0b5f20 con 0x7f532c0b4e70 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532bfff700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c0b4e70 0x7f532c0ccff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f532c15d180 tx=0x7f5320009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cab13c254cad30 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.953+0000 7f532affd700 1 -- 172.21.15.170:0/3841919553 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000b860 con 0x7f532c0b4e70 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f532affd700 1 -- 172.21.15.170:0/3841919553 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532000b9c0 con 0x7f532c0b4e70 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f532affd700 1 -- 172.21.15.170:0/3841919553 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000bd60 con 0x7f532c0b4e70 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 -- 172.21.15.170:0/3841919553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c0b4e70 msgr2=0x7f532c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:58.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c0b4e70 0x7f532c0ccff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f532c15d180 tx=0x7f5320009250 comp rx=0 tx=0).stop 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 -- 172.21.15.170:0/3841919553 shutdown_connections 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 --2- 172.21.15.170:0/3841919553 >> v2:172.21.15.191:3300/0 conn(0x7f532c15d240 0x7f532c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c0b4e70 0x7f532c0ccff0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 --2- 172.21.15.170:0/3841919553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 -- 172.21.15.170:0/3841919553 >> 172.21.15.170:0/3841919553 conn(0x7f532c0d10c0 msgr2=0x7f532c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 -- 172.21.15.170:0/3841919553 shutdown_connections 2023-10-08T16:09:58.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.954+0000 7f533120b700 1 -- 172.21.15.170:0/3841919553 wait complete. 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.955+0000 7f533120b700 1 Processor -- start 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.955+0000 7f533120b700 1 -- start start 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c15c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f532c0b4e70 0x7f532c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0c5700 con 0x7f532c15d240 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0c59e0 con 0x7f532c0b4e70 2023-10-08T16:09:58.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f533120b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0c5cc0 con 0x7f532c0b16d0 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48734/0 (socket says 172.21.15.170:48734) 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 -- 172.21.15.170:0/1341531252 learned_addr learned my addr 172.21.15.170:0/1341531252 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f532bfff700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c15c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f532b7fe700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f532c0b4e70 0x7f532c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 -- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 msgr2=0x7f532c15c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c15c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 -- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f532c0b4e70 msgr2=0x7f532c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f532c0b4e70 0x7f532c0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.956+0000 7f5330a0a700 1 -- 172.21.15.170:0/1341531252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200094f0 con 0x7f532c15d240 2023-10-08T16:09:58.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f5330a0a700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5324007f20 tx=0x7f5324009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532400eb90 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532400ed10 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c0c5fa0 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324014670 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.957+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c0c5390 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.958+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c0c64c0 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.958+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f53240149e0 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.958+0000 7f5313fff700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 0x7f53180696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.958+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5324084c80 con 0x7f532c15d240 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.959+0000 7f532bfff700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 0x7f53180696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:58.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.959+0000 7f532bfff700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 0x7f53180696b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f532c15d180 tx=0x7f5320017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:58.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:58.963+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f532c0c64c0 con 0x7f532c15d240 2023-10-08T16:09:59.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.145+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f532c0c64c0 con 0x7f532c15d240 2023-10-08T16:09:59.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.149+0000 7f5313fff700 1 -- 172.21.15.170:0/1341531252 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f532c0c64c0 con 0x7f532c15d240 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 msgr2=0x7f53180696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 0x7f53180696b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f532c15d180 tx=0x7f5320017000 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 msgr2=0x7f532c14bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5324007f20 tx=0x7f5324009860 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 shutdown_connections 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5318067200 0x7f53180696b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f532c15d240 0x7f532c14bbf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f532c0b16d0 0x7f532c15c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 --2- 172.21.15.170:0/1341531252 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f532c0b4e70 0x7f532c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 >> 172.21.15.170:0/1341531252 conn(0x7f532c0d10c0 msgr2=0x7f532c158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:59.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 shutdown_connections 2023-10-08T16:09:59.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.151+0000 7f533120b700 1 -- 172.21.15.170:0/1341531252 wait complete. 2023-10-08T16:09:59.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:09:59.164 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:59.203 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:09:59.249 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:09:59.250 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:09:59.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:09:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.364+0000 7f2171f10700 1 Processor -- start 2023-10-08T16:09:59.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.364+0000 7f2171f10700 1 -- start start 2023-10-08T16:09:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2171f10700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 0x7f216c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2171f10700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 0x7f216c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2171f10700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0b11d0 con 0x7f216c15d230 2023-10-08T16:09:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2171f10700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0b14b0 con 0x7f216c159420 2023-10-08T16:09:59.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2171f10700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b1790 con 0x7f216c1597f0 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2163fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 0x7f216c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f2170f0e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 0x7f216c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f217170f700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f216c15d230 0x7f216c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:55704/0 (socket says 172.21.15.170:55704) 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.365+0000 7f217170f700 1 -- 172.21.15.170:0/194704817 learned_addr learned my addr 172.21.15.170:0/194704817 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f21637fe700 1 -- 172.21.15.170:0/194704817 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930910660 0 0) 0x7f216c0b11d0 con 0x7f216c15d230 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f21637fe700 1 -- 172.21.15.170:0/194704817 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154003570 con 0x7f216c15d230 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2163fff700 1 -- 172.21.15.170:0/194704817 >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 msgr2=0x7f216c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2163fff700 1 --2- 172.21.15.170:0/194704817 >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 0x7f216c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2163fff700 1 -- 172.21.15.170:0/194704817 >> v1:172.21.15.191:6789/0 conn(0x7f216c15d230 legacy=0x7f216c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2163fff700 1 -- 172.21.15.170:0/194704817 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0b1a70 con 0x7f216c1597f0 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2170f0e700 1 --2- 172.21.15.170:0/194704817 >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 0x7f216c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f2163fff700 1 --2- 172.21.15.170:0/194704817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 0x7f216c0b2270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f215c0098e0 tx=0x7f215c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb2908a1994b3898 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f21637fe700 1 -- 172.21.15.170:0/194704817 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011630 con 0x7f216c1597f0 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f21637fe700 1 -- 172.21.15.170:0/194704817 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c011790 con 0x7f216c1597f0 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.366+0000 7f21637fe700 1 -- 172.21.15.170:0/194704817 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011b30 con 0x7f216c1597f0 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 -- 172.21.15.170:0/194704817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 msgr2=0x7f216c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 --2- 172.21.15.170:0/194704817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 0x7f216c0b2270 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f215c0098e0 tx=0x7f215c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 -- 172.21.15.170:0/194704817 shutdown_connections 2023-10-08T16:09:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 --2- 172.21.15.170:0/194704817 >> v2:172.21.15.191:3300/0 conn(0x7f216c159420 0x7f216c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 --2- 172.21.15.170:0/194704817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c1597f0 0x7f216c0b2270 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 -- 172.21.15.170:0/194704817 >> 172.21.15.170:0/194704817 conn(0x7f216c0d0660 msgr2=0x7f216c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 -- 172.21.15.170:0/194704817 shutdown_connections 2023-10-08T16:09:59.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.367+0000 7f2171f10700 1 -- 172.21.15.170:0/194704817 wait complete. 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.368+0000 7f2171f10700 1 Processor -- start 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.368+0000 7f2171f10700 1 -- start start 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c159420 0x7f216c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 0x7f216c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0c6ef0 con 0x7f216c1597f0 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15f560 con 0x7f216c15d230 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2171f10700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15e950 con 0x7f216c159420 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48742/0 (socket says 172.21.15.170:48742) 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 -- 172.21.15.170:0/867015333 learned_addr learned my addr 172.21.15.170:0/867015333 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2170f0e700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c159420 0x7f216c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f217170f700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 0x7f216c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 -- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c159420 msgr2=0x7f216c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c159420 0x7f216c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 -- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 msgr2=0x7f216c0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 0x7f216c0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 -- 172.21.15.170:0/867015333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168009a20 con 0x7f216c1597f0 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f217170f700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 0x7f216c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:09:59.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.369+0000 7f2163fff700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f216c0b18d0 tx=0x7f215c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.370+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c011b10 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.370+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c0253f0 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.370+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c009420 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.370+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0257b0 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.370+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c15ef10 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.371+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c15f6f0 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.371+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f215c025ba0 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.371+0000 7f2160ff9700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 0x7f21580696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.371+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f215c085f30 con 0x7f216c1597f0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.371+0000 7f2170f0e700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 0x7f21580696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.372+0000 7f2170f0e700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 0x7f21580696b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f216c167100 tx=0x7f216800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.375+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f215c053110 con 0x7f216c1597f0 2023-10-08T16:09:59.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.602+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216c0b1d50 con 0x7f216c1597f0 2023-10-08T16:09:59.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.602+0000 7f2160ff9700 1 -- 172.21.15.170:0/867015333 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f215c015df0 con 0x7f216c1597f0 2023-10-08T16:09:59.603 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:09:59.603 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5342,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2354480178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2354480178},{"type":"v1","addr":"172.21.15.191:6835","nonce":2354480178}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3929649453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3929649453},{"type":"v1","addr":"172.21.15.191:6834","nonce":3929649453}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6033,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/2780354663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2780354663},{"type":"v1","addr":"172.21.15.170:6836","nonce":2780354663}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":6315,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2528456258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2528456258},{"type":"v1","addr":"172.21.15.170:6837","nonce":2528456258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 msgr2=0x7f21580696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 0x7f21580696b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f216c167100 tx=0x7f216800b040 comp rx=0 tx=0).stop 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 msgr2=0x7f216c0c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f216c0b18d0 tx=0x7f215c017000 comp rx=0 tx=0).stop 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 shutdown_connections 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f216c15d230 0x7f216c0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f216c159420 0x7f216c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2158067200 0x7f21580696b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 --2- 172.21.15.170:0/867015333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216c1597f0 0x7f216c0c5400 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 >> 172.21.15.170:0/867015333 conn(0x7f216c0d0660 msgr2=0x7f216c0af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 shutdown_connections 2023-10-08T16:09:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.604+0000 7f2171f10700 1 -- 172.21.15.170:0/867015333 wait complete. 2023-10-08T16:09:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 50 2023-10-08T16:09:59.617 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:09:59.617 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:09:59.617 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:09:59.617 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:59.618 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:09:59.618 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:09:59.618 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:09:59.618 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:59.618 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:09:59.619 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:09:59.619 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:09:59.619 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:59.619 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:09:59.619 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:09:59.619 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:09:59.619 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:09:59.626 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:09:59.626 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:09:59.629 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:09:59.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:09:59.631 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:09:59.631 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:09:59.632 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:09:59.632 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:09:59.668 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:09:59.669 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:59.669 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:09:59.669 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:59.673 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:09:59.674 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:59.675 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:09:59.675 DEBUG:teuthology.parallel:result is None 2023-10-08T16:09:59.676 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:09:59.741 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:09:59.741 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:09:59.753 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:09:59.753 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:09:59.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.796+0000 7efe5f33f700 1 Processor -- start 2023-10-08T16:09:59.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.796+0000 7efe5f33f700 1 -- start start 2023-10-08T16:09:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5f33f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efe5815d230 0x7efe5815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5f33f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe580b2980 con 0x7efe58159420 2023-10-08T16:09:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5f33f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5815db70 con 0x7efe581597f0 2023-10-08T16:09:59.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5f33f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe580b11d0 con 0x7efe5815d230 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5e33d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efe58159420 0x7efe580b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35352/0 (socket says 172.21.15.170:35352) 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5e33d700 1 -- 172.21.15.170:0/2652041957 learned_addr learned my addr 172.21.15.170:0/2652041957 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2652041957 >> v2:172.21.15.191:3300/0 conn(0x7efe5815d230 0x7efe5815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.797+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65397479 0 0) 0x7efe580b2980 con 0x7efe58159420 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe44003570 con 0x7efe58159420 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686752678 0 0) 0x7efe44003570 con 0x7efe58159420 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 >> v2:172.21.15.191:3300/0 conn(0x7efe5815d230 msgr2=0x7efe5815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 --2- 172.21.15.170:0/2652041957 >> v2:172.21.15.191:3300/0 conn(0x7efe5815d230 0x7efe5815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 >> v1:172.21.15.191:6789/0 conn(0x7efe581597f0 legacy=0x7efe580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580b14b0 con 0x7efe58159420 2023-10-08T16:09:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7efe540017f0 con 0x7efe58159420 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 >> v1:172.21.15.170:6789/0 conn(0x7efe58159420 legacy=0x7efe580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe44003a60 0x7efe44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe44004810 0x7efe44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 0x7efe4400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe580b11d0 con 0x7efe440051c0 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4400c0d0 con 0x7efe44004810 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4400c290 con 0x7efe44003a60 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 0x7efe4400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5e33d700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe44003a60 0x7efe44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5db3c700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe44004810 0x7efe44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 -- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe44003a60 msgr2=0x7efe44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe44003a60 0x7efe44003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 -- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe44004810 msgr2=0x7efe44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe44004810 0x7efe44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 -- 172.21.15.170:0/2652041957 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44003570 con 0x7efe440051c0 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 -- 172.21.15.170:0/2652041957 reap_dead start 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 0x7efe4400ba60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efe50009c00 tx=0x7efe500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fa3076751f0000 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5000e710 con 0x7efe440051c0 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5000e870 con 0x7efe440051c0 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.798+0000 7efe5d33b700 1 -- 172.21.15.170:0/2652041957 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5000ec30 con 0x7efe440051c0 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 -- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 msgr2=0x7efe4400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 0x7efe4400ba60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7efe50009c00 tx=0x7efe500091a0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 -- 172.21.15.170:0/2652041957 shutdown_connections 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2652041957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe440051c0 0x7efe4400ba60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 -- 172.21.15.170:0/2652041957 >> 172.21.15.170:0/2652041957 conn(0x7efe580d0660 msgr2=0x7efe580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 -- 172.21.15.170:0/2652041957 shutdown_connections 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.799+0000 7efe5f33f700 1 -- 172.21.15.170:0/2652041957 wait complete. 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.800+0000 7efe5f33f700 1 Processor -- start 2023-10-08T16:09:59.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.800+0000 7efe5f33f700 1 -- start start 2023-10-08T16:09:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe440051c0 0x7efe580c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe58155c30 0x7efe58156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe580c9a30 con 0x7efe580c56c0 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe581565c0 con 0x7efe440051c0 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5f33f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe580c5f10 con 0x7efe58155c30 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48796/0 (socket says 172.21.15.170:48796) 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 -- 172.21.15.170:0/2420032595 learned_addr learned my addr 172.21.15.170:0/2420032595 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5eb3e700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe58155c30 0x7efe58156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5e33d700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe440051c0 0x7efe580c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 -- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe58155c30 msgr2=0x7efe58156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe58155c30 0x7efe58156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 -- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe440051c0 msgr2=0x7efe580c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe440051c0 0x7efe580c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 -- 172.21.15.170:0/2420032595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe580c61f0 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.801+0000 7efe5db3c700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efe48002980 tx=0x7efe48007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe48006820 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe48006980 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe48006d20 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe580c64d0 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe58154d80 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.802+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe580c9800 con 0x7efe580c56c0 2023-10-08T16:09:59.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.803+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7efe48007230 con 0x7efe580c56c0 2023-10-08T16:09:59.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.803+0000 7efe4e7fc700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 0x7efe3c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:09:59.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.803+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efe48084420 con 0x7efe580c56c0 2023-10-08T16:09:59.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.803+0000 7efe5e33d700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 0x7efe3c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:09:59.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.804+0000 7efe5e33d700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 0x7efe3c0696b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe5400a0e0 tx=0x7efe5815dd00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:09:59.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:09:59.807+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe580c9800 con 0x7efe580c56c0 2023-10-08T16:10:00.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.007+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efe580c9800 con 0x7efe580c56c0 2023-10-08T16:10:00.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.831+0000 7efe4e7fc700 1 -- 172.21.15.170:0/2420032595 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7efe580c9800 con 0x7efe580c56c0 2023-10-08T16:10:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 msgr2=0x7efe3c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 0x7efe3c0696b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efe5400a0e0 tx=0x7efe5815dd00 comp rx=0 tx=0).stop 2023-10-08T16:10:00.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 msgr2=0x7efe58154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efe48002980 tx=0x7efe48007d30 comp rx=0 tx=0).stop 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 shutdown_connections 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe58155c30 0x7efe58156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe440051c0 0x7efe580c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe3c067200 0x7efe3c0696b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 --2- 172.21.15.170:0/2420032595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe580c56c0 0x7efe58154840 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 >> 172.21.15.170:0/2420032595 conn(0x7efe580d0660 msgr2=0x7efe581551b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 shutdown_connections 2023-10-08T16:10:00.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.834+0000 7efe5f33f700 1 -- 172.21.15.170:0/2420032595 wait complete. 2023-10-08T16:10:00.836 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:10:00.847 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:10:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.965+0000 7face4623700 1 Processor -- start 2023-10-08T16:10:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.965+0000 7face4623700 1 -- start start 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face4623700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7facdc159890 0x7facdc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face4623700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc15a160 con 0x7facdc159c60 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face4623700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facdc0d13a0 con 0x7facdc069560 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face4623700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc15e140 con 0x7facdc159890 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face1bbe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7facdc159c60 0x7facdc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35360/0 (socket says 172.21.15.170:35360) 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face1bbe700 1 -- 172.21.15.170:0/1820084989 learned_addr learned my addr 172.21.15.170:0/1820084989 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face23bf700 1 --2- 172.21.15.170:0/1820084989 >> v2:172.21.15.191:3300/0 conn(0x7facdc159890 0x7facdc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237595763 0 0) 0x7facdc15a160 con 0x7facdc159c60 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc8003570 con 0x7facdc159c60 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3391494549 0 0) 0x7facc8003570 con 0x7facdc159c60 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 >> v2:172.21.15.191:3300/0 conn(0x7facdc159890 msgr2=0x7facdc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 --2- 172.21.15.170:0/1820084989 >> v2:172.21.15.191:3300/0 conn(0x7facdc159890 0x7facdc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 >> v1:172.21.15.191:6789/0 conn(0x7facdc069560 legacy=0x7facdc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc15a3d0 con 0x7facdc159c60 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.966+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7faccc001470 con 0x7facdc159c60 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 >> v1:172.21.15.170:6789/0 conn(0x7facdc159c60 legacy=0x7facdc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 0x7facc8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8004810 0x7facc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facc80051c0 0x7facc800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc15e140 con 0x7facc8003a60 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc800c0d0 con 0x7facc8004810 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc800c290 con 0x7facc80051c0 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 0x7facc8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face1bbe700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8004810 0x7facc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face2bc0700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facc80051c0 0x7facc800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 -- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facc80051c0 msgr2=0x7facc800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facc80051c0 0x7facc800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 -- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8004810 msgr2=0x7facc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8004810 0x7facc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 -- 172.21.15.170:0/1820084989 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8003570 con 0x7facc8003a60 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face2bc0700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facc80051c0 0x7facc800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face2bc0700 1 -- 172.21.15.170:0/1820084989 reap_dead start 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face23bf700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 0x7facc8003e70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7facd8009c00 tx=0x7facd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a7b1781c4c7d3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e710 con 0x7facc8003a60 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800e870 con 0x7facc8003a60 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.967+0000 7face13bd700 1 -- 172.21.15.170:0/1820084989 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800ec30 con 0x7facc8003a60 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 -- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 msgr2=0x7facc8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 0x7facc8003e70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7facd8009c00 tx=0x7facd80091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 -- 172.21.15.170:0/1820084989 shutdown_connections 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 --2- 172.21.15.170:0/1820084989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facc8003a60 0x7facc8003e70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 -- 172.21.15.170:0/1820084989 >> 172.21.15.170:0/1820084989 conn(0x7facdc068e20 msgr2=0x7facdc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 -- 172.21.15.170:0/1820084989 shutdown_connections 2023-10-08T16:10:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.968+0000 7face4623700 1 -- 172.21.15.170:0/1820084989 wait complete. 2023-10-08T16:10:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.969+0000 7face4623700 1 Processor -- start 2023-10-08T16:10:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.969+0000 7face4623700 1 -- start start 2023-10-08T16:10:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8003a60 0x7facdc0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ca980 0x7facdc0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0cea70 con 0x7facdc0c9500 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0ced50 con 0x7facc8003a60 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face4623700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0c7de0 con 0x7facdc0ca980 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48820/0 (socket says 172.21.15.170:48820) 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 -- 172.21.15.170:0/4416465 learned_addr learned my addr 172.21.15.170:0/4416465 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face1bbe700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ca980 0x7facdc0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face23bf700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8003a60 0x7facdc0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 -- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ca980 msgr2=0x7facdc0cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ca980 0x7facdc0cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 -- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8003a60 msgr2=0x7facdc0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8003a60 0x7facdc0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.970+0000 7face2bc0700 1 -- 172.21.15.170:0/4416465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0cd9f0 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7face2bc0700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7facd400e810 tx=0x7facd4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4012590 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd40126f0 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7face4623700 1 -- 172.21.15.170:0/4416465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc0cdcd0 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4012ab0 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.971+0000 7face4623700 1 -- 172.21.15.170:0/4416465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc0cd600 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.972+0000 7face4623700 1 -- 172.21.15.170:0/4416465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc0ceee0 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.972+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7facd4003030 con 0x7facdc0c9500 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.972+0000 7facd27fc700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 0x7facc00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:00.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.973+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7facd4084400 con 0x7facdc0c9500 2023-10-08T16:10:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.973+0000 7face23bf700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 0x7facc00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.973+0000 7face23bf700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 0x7facc00696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7facd80091e0 tx=0x7facd801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:00.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:00.977+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7facd40550c0 con 0x7facdc0c9500 2023-10-08T16:10:01.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.235+0000 7face4623700 1 -- 172.21.15.170:0/4416465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7facdc04c3c0 con 0x7facdc0c9500 2023-10-08T16:10:01.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.236+0000 7facd27fc700 1 -- 172.21.15.170:0/4416465 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7facd4054c50 con 0x7facdc0c9500 2023-10-08T16:10:01.236 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:01.237 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.238 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.239 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.240 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:10:01.241 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.242 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.243 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.244 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:10:01.245 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.246 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:10:01.247 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.248 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.249 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:10:01.250 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.251 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:01.252 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.253 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:01.253 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:10:01.253 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:01.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 msgr2=0x7facc00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 0x7facc00696b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7facd80091e0 tx=0x7facd801a000 comp rx=0 tx=0).stop 2023-10-08T16:10:01.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 msgr2=0x7facdc0b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7facd400e810 tx=0x7facd4009040 comp rx=0 tx=0).stop 2023-10-08T16:10:01.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 shutdown_connections 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facc0067200 0x7facc00696b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facdc0c9500 0x7facdc0b1a40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facc8003a60 0x7facdc0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 --2- 172.21.15.170:0/4416465 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facdc0ca980 0x7facdc0cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 >> 172.21.15.170:0/4416465 conn(0x7facdc068e20 msgr2=0x7facdc149f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 shutdown_connections 2023-10-08T16:10:01.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.238+0000 7face4623700 1 -- 172.21.15.170:0/4416465 wait complete. 2023-10-08T16:10:01.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.371+0000 7f5dc126b700 1 Processor -- start 2023-10-08T16:10:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.371+0000 7f5dc126b700 1 -- start start 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.371+0000 7f5dc126b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5dbc156680 0x7f5dbc06c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.371+0000 7f5dc126b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5dbc156a50 0x7f5dbc0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.371+0000 7f5dc126b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dbc06be50 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dc126b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc0c88d0 con 0x7f5dbc156a50 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dc126b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dbc06c3f0 con 0x7f5dbc156680 2023-10-08T16:10:01.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5dbc156a50 0x7f5dbc0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dbb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5dbc0c7e70 0x7f5dbc0af1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35372/0 (socket says 172.21.15.170:35372) 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5dbc156a50 0x7f5dbc0c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48828/0 (socket says 172.21.15.170:48828) 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dbb7fe700 1 -- 172.21.15.170:0/831376600 learned_addr learned my addr 172.21.15.170:0/831376600 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dbaffd700 1 --2- 172.21.15.170:0/831376600 >> v2:172.21.15.191:3300/0 conn(0x7f5dbc156680 0x7f5dbc06c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 -- 172.21.15.170:0/831376600 >> v2:172.21.15.191:3300/0 conn(0x7f5dbc156680 msgr2=0x7f5dbc06c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 --2- 172.21.15.170:0/831376600 >> v2:172.21.15.191:3300/0 conn(0x7f5dbc156680 0x7f5dbc06c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 -- 172.21.15.170:0/831376600 >> v1:172.21.15.170:6789/0 conn(0x7f5dbc0c7e70 legacy=0x7f5dbc0af1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 -- 172.21.15.170:0/831376600 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc0c8310 con 0x7f5dbc156a50 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dba7fc700 1 -- 172.21.15.170:0/831376600 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858007562 0 0) 0x7f5dbc06be50 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5db3fff700 1 --2- 172.21.15.170:0/831376600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc156a50 0x7f5dbc0c7930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06e0b0 tx=0x7f5da40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9971b08a08360ed4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dba7fc700 1 -- 172.21.15.170:0/831376600 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4011630 con 0x7f5dbc156a50 2023-10-08T16:10:01.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dba7fc700 1 -- 172.21.15.170:0/831376600 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da4011790 con 0x7f5dbc156a50 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.372+0000 7f5dba7fc700 1 -- 172.21.15.170:0/831376600 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4011b30 con 0x7f5dbc156a50 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 -- 172.21.15.170:0/831376600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc156a50 msgr2=0x7f5dbc0c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 --2- 172.21.15.170:0/831376600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc156a50 0x7f5dbc0c7930 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc06e0b0 tx=0x7f5da40091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 -- 172.21.15.170:0/831376600 shutdown_connections 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 --2- 172.21.15.170:0/831376600 >> v2:172.21.15.191:3300/0 conn(0x7f5dbc156680 0x7f5dbc06c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 --2- 172.21.15.170:0/831376600 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc156a50 0x7f5dbc0c7930 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 -- 172.21.15.170:0/831376600 >> 172.21.15.170:0/831376600 conn(0x7f5dbc0cdd80 msgr2=0x7f5dbc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 -- 172.21.15.170:0/831376600 shutdown_connections 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.373+0000 7f5dc126b700 1 -- 172.21.15.170:0/831376600 wait complete. 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.374+0000 7f5dc126b700 1 Processor -- start 2023-10-08T16:10:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.374+0000 7f5dc126b700 1 -- start start 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dbc156680 0x7f5dbc150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5dbc156a50 0x7f5dbc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cc320 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc149950 con 0x7f5dbc156680 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc0cba30 con 0x7f5dbc156a50 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48840/0 (socket says 172.21.15.170:48840) 2023-10-08T16:10:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 -- 172.21.15.170:0/3373014852 learned_addr learned my addr 172.21.15.170:0/3373014852 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5db3fff700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dbc156680 0x7f5dbc150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbb7fe700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5dbc156a50 0x7f5dbc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 -- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5dbc156a50 msgr2=0x7f5dbc150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5dbc156a50 0x7f5dbc150840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 -- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dbc156680 msgr2=0x7f5dbc150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dbc156680 0x7f5dbc150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 -- 172.21.15.170:0/3373014852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da4009420 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dbaffd700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00ec80 tx=0x7f5dac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00c9d0 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac00cb50 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc0cc7b0 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac013470 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.375+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc0cbfb0 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.376+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dbc04c3c0 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.377+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5dac013860 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.377+0000 7f5db2ffd700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 0x7f5d9c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.377+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5dac084a00 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.377+0000 7f5db3fff700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 0x7f5d9c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.377+0000 7f5db3fff700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 0x7f5d9c0696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc156ea0 tx=0x7f5da4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.381+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dac051c70 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.565+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dbc0cc600 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.566+0000 7f5db2ffd700 1 -- 172.21.15.170:0/3373014852 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+12183 (secure 0 0 0) 0x7f5dac055290 con 0x7f5dbc0c7e70 2023-10-08T16:10:01.566 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:01.566 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":85,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:00.828050+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:01.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 msgr2=0x7f5d9c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 0x7f5d9c0696b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc156ea0 tx=0x7f5da4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 msgr2=0x7f5dbc149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00ec80 tx=0x7f5dac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 shutdown_connections 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5dbc156a50 0x7f5dbc150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d9c067200 0x7f5d9c0696b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5dbc0c7e70 0x7f5dbc149410 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 --2- 172.21.15.170:0/3373014852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5dbc156680 0x7f5dbc150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 >> 172.21.15.170:0/3373014852 conn(0x7f5dbc0cdd80 msgr2=0x7f5dbc14b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 shutdown_connections 2023-10-08T16:10:01.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.568+0000 7f5dc126b700 1 -- 172.21.15.170:0/3373014852 wait complete. 2023-10-08T16:10:01.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:10:01.580 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:10:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.698+0000 7f3fc8db9700 1 Processor -- start 2023-10-08T16:10:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.698+0000 7f3fc8db9700 1 -- start start 2023-10-08T16:10:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.698+0000 7f3fc8db9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3fc4159890 0x7f3fc415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc8db9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc8db9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc4069ee0 con 0x7f3fc4159c60 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc8db9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc415a160 con 0x7f3fc4159890 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc8db9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40d13a0 con 0x7f3fc4069560 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3fc4159890 0x7f3fc415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3fc4159890 0x7f3fc415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48854/0 (socket says 172.21.15.170:48854) 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc2d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 -- 172.21.15.170:0/2118000257 learned_addr learned my addr 172.21.15.170:0/2118000257 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 -- 172.21.15.170:0/2118000257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 msgr2=0x7f3fc40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 --2- 172.21.15.170:0/2118000257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 -- 172.21.15.170:0/2118000257 >> v1:172.21.15.191:6789/0 conn(0x7f3fc4159c60 legacy=0x7f3fc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 -- 172.21.15.170:0/2118000257 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc415e140 con 0x7f3fc4159890 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc259c700 1 --2- 172.21.15.170:0/2118000257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc415abd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40099b0 tx=0x7f3fb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c8473a36588dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc1d9b700 1 -- 172.21.15.170:0/2118000257 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4009ea0 con 0x7f3fc4159890 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc1d9b700 1 -- 172.21.15.170:0/2118000257 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb400b860 con 0x7f3fc4159890 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.699+0000 7f3fc1d9b700 1 -- 172.21.15.170:0/2118000257 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb400bc00 con 0x7f3fc4159890 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2118000257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 msgr2=0x7f3fc415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2118000257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc415abd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40099b0 tx=0x7f3fb4009250 comp rx=0 tx=0).stop 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2118000257 shutdown_connections 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2118000257 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2118000257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc415abd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2118000257 >> 172.21.15.170:0/2118000257 conn(0x7f3fc4068e20 msgr2=0x7f3fc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2118000257 shutdown_connections 2023-10-08T16:10:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.700+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2118000257 wait complete. 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.701+0000 7f3fc8db9700 1 Processor -- start 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.701+0000 7f3fc8db9700 1 -- start start 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc416e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc4159c60 0x7f3fc40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc416efa0 con 0x7f3fc4159890 2023-10-08T16:10:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc416f840 con 0x7f3fc4159c60 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc8db9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40b3b50 con 0x7f3fc4069560 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48870/0 (socket says 172.21.15.170:48870) 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 -- 172.21.15.170:0/2862235385 learned_addr learned my addr 172.21.15.170:0/2862235385 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc259c700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc416e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 -- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 msgr2=0x7f3fc416e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc416e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 -- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc4159c60 msgr2=0x7f3fc40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fc2d9d700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc4159c60 0x7f3fc40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc4159c60 0x7f3fc40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 -- 172.21.15.170:0/2862235385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb40094f0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.702+0000 7f3fb9d9b700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c590 tx=0x7f3fac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.703+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac011590 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.703+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fac0116f0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.703+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc416f1f0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.703+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fac011ab0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.703+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc416ed40 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.704+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc416f9b0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.704+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f3fac01b3b0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.704+0000 7f3fbaffd700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 0x7f3fa40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.704+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3fac083cc0 con 0x7f3fc4159890 2023-10-08T16:10:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.704+0000 7f3fc259c700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 0x7f3fa40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.705+0000 7f3fc259c700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 0x7f3fa40696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40099b0 tx=0x7f3fb4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.708+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fac050e50 con 0x7f3fc4159890 2023-10-08T16:10:01.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.891+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fc404c3c0 con 0x7f3fc4159890 2023-10-08T16:10:01.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.892+0000 7f3fbaffd700 1 -- 172.21.15.170:0/2862235385 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fac017070 con 0x7f3fc4159890 2023-10-08T16:10:01.892 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 msgr2=0x7f3fa40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 0x7f3fa40696b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40099b0 tx=0x7f3fb4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 msgr2=0x7f3fc4148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3fac00c590 tx=0x7f3fac00c950 comp rx=0 tx=0).stop 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 shutdown_connections 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3fc4159c60 0x7f3fc40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3fc4069560 0x7f3fc416e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3fa4067200 0x7f3fa40696b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 --2- 172.21.15.170:0/2862235385 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fc4159890 0x7f3fc4148190 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:01.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 >> 172.21.15.170:0/2862235385 conn(0x7f3fc4068e20 msgr2=0x7f3fc40b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:01.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 shutdown_connections 2023-10-08T16:10:01.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:01.894+0000 7f3fc8db9700 1 -- 172.21.15.170:0/2862235385 wait complete. 2023-10-08T16:10:01.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:10:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.021+0000 7f096d218700 1 Processor -- start 2023-10-08T16:10:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.021+0000 7f096d218700 1 -- start start 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968159420 0x7f09680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 0x7f09680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 0x7f096815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096815dbb0 con 0x7f0968159420 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680b11d0 con 0x7f09681597f0 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096d218700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680b14b0 con 0x7f096815d230 2023-10-08T16:10:02.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968159420 0x7f09680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0968159420 0x7f09680b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48876/0 (socket says 172.21.15.170:48876) 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 -- 172.21.15.170:0/1618383607 learned_addr learned my addr 172.21.15.170:0/1618383607 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096ca17700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 0x7f096815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f09677fe700 1 --2- 172.21.15.170:0/1618383607 >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 0x7f09680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 -- 172.21.15.170:0/1618383607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 msgr2=0x7f096815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 0x7f096815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 -- 172.21.15.170:0/1618383607 >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 msgr2=0x7f09680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 --2- 172.21.15.170:0/1618383607 >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 0x7f09680b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f0967fff700 1 -- 172.21.15.170:0/1618383607 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09680b1790 con 0x7f0968159420 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f096ca17700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 0x7f096815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.022+0000 7f09677fe700 1 --2- 172.21.15.170:0/1618383607 >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 0x7f09680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f0967fff700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680b5f40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0968089750 tx=0x7f0958009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fae918f54f2aaaf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f0966ffd700 1 -- 172.21.15.170:0/1618383607 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800b860 con 0x7f0968159420 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f0966ffd700 1 -- 172.21.15.170:0/1618383607 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095800b9c0 con 0x7f0968159420 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f0966ffd700 1 -- 172.21.15.170:0/1618383607 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800bd60 con 0x7f0968159420 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 -- 172.21.15.170:0/1618383607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 msgr2=0x7f09680b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680b5f40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0968089750 tx=0x7f0958009250 comp rx=0 tx=0).stop 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 -- 172.21.15.170:0/1618383607 shutdown_connections 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f096815d230 0x7f096815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 --2- 172.21.15.170:0/1618383607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680b5f40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 --2- 172.21.15.170:0/1618383607 >> v2:172.21.15.191:3300/0 conn(0x7f09681597f0 0x7f09680b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 -- 172.21.15.170:0/1618383607 >> 172.21.15.170:0/1618383607 conn(0x7f09680d0660 msgr2=0x7f09680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 -- 172.21.15.170:0/1618383607 shutdown_connections 2023-10-08T16:10:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.023+0000 7f096d218700 1 -- 172.21.15.170:0/1618383607 wait complete. 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.024+0000 7f096d218700 1 Processor -- start 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 -- start start 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 0x7f09680cf900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 0x7f0968148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680ca0c0 con 0x7f0968159420 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c96f0 con 0x7f096815d230 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f096d218700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c8ae0 con 0x7f09681597f0 2023-10-08T16:10:02.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f0967fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f0967fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48884/0 (socket says 172.21.15.170:48884) 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.025+0000 7f0967fff700 1 -- 172.21.15.170:0/577164365 learned_addr learned my addr 172.21.15.170:0/577164365 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f09677fe700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 0x7f09680cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f096ca17700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 0x7f0968148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 -- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 msgr2=0x7f09680cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 0x7f09680cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 -- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 msgr2=0x7f0968148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 0x7f0968148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 -- 172.21.15.170:0/577164365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09580094f0 con 0x7f0968159420 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f09677fe700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 0x7f09680cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f096ca17700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 0x7f0968148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:02.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f0967fff700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0958009290 tx=0x7f0958017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958026480 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095801f070 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680c8dc0 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958026830 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.026+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680c9380 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.027+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096804c3c0 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.027+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f0958026ba0 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.028+0000 7f094ffff700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 0x7f09540696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.028+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0958021070 con 0x7f0968159420 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.028+0000 7f09677fe700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 0x7f09540696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.028+0000 7f09677fe700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 0x7f09540696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f095c006f00 tx=0x7f095c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.032+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0958053810 con 0x7f0968159420 2023-10-08T16:10:02.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.218+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f09680b1a70 con 0x7f0968159420 2023-10-08T16:10:02.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.218+0000 7f094ffff700 1 -- 172.21.15.170:0/577164365 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0958056e30 con 0x7f0968159420 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:02.219 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 msgr2=0x7f09540696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 0x7f09540696b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f095c006f00 tx=0x7f095c008040 comp rx=0 tx=0).stop 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 msgr2=0x7f09680c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0958009290 tx=0x7f0958017000 comp rx=0 tx=0).stop 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 shutdown_connections 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f096815d230 0x7f0968148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0954067200 0x7f09540696b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0968159420 0x7f09680c9b80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 --2- 172.21.15.170:0/577164365 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09681597f0 0x7f09680cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 >> 172.21.15.170:0/577164365 conn(0x7f09680d0660 msgr2=0x7f0968158370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 shutdown_connections 2023-10-08T16:10:02.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.220+0000 7f096d218700 1 -- 172.21.15.170:0/577164365 wait complete. 2023-10-08T16:10:02.232 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:10:02.233 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:10:02.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.351+0000 7f76a5152700 1 Processor -- start 2023-10-08T16:10:02.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.351+0000 7f76a5152700 1 -- start start 2023-10-08T16:10:02.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f76a5152700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f76a5152700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f76a00b53d0 0x7f76a00cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f76a5152700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a00b5b00 con 0x7f76a00cce40 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f76a5152700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a00b5da0 con 0x7f76a00b53d0 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f76a5152700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00b6080 con 0x7f76a0159ec0 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f76a00b53d0 0x7f76a00cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f76a00b53d0 0x7f76a00cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48902/0 (socket says 172.21.15.170:48902) 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 -- 172.21.15.170:0/3036291093 learned_addr learned my addr 172.21.15.170:0/3036291093 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769ed9d700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 -- 172.21.15.170:0/3036291093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 msgr2=0x7f76a00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 -- 172.21.15.170:0/3036291093 >> v1:172.21.15.191:6789/0 conn(0x7f76a00cce40 legacy=0x7f76a00cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 -- 172.21.15.170:0/3036291093 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00cc100 con 0x7f76a00b53d0 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.352+0000 7f769659c700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00b53d0 0x7f76a00cc900 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76880099b0 tx=0x7f7688009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819267a3df5cf036 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f769e59c700 1 -- 172.21.15.170:0/3036291093 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688009ea0 con 0x7f76a00b53d0 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f769e59c700 1 -- 172.21.15.170:0/3036291093 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768800b860 con 0x7f76a00b53d0 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f769e59c700 1 -- 172.21.15.170:0/3036291093 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800bc00 con 0x7f76a00b53d0 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 -- 172.21.15.170:0/3036291093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00b53d0 msgr2=0x7f76a00cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00b53d0 0x7f76a00cc900 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f76880099b0 tx=0x7f7688009250 comp rx=0 tx=0).stop 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 -- 172.21.15.170:0/3036291093 shutdown_connections 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 --2- 172.21.15.170:0/3036291093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00b53d0 0x7f76a00cc900 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 -- 172.21.15.170:0/3036291093 >> 172.21.15.170:0/3036291093 conn(0x7f76a00999f0 msgr2=0x7f76a00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:02.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 -- 172.21.15.170:0/3036291093 shutdown_connections 2023-10-08T16:10:02.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.353+0000 7f76a5152700 1 -- 172.21.15.170:0/3036291093 wait complete. 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.354+0000 7f76a5152700 1 Processor -- start 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 -- start start 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a00b53d0 0x7f76a009a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a014b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a006b260 con 0x7f76a00cce40 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a009bbd0 con 0x7f76a00b53d0 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f76a5152700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a009afc0 con 0x7f76a0159ec0 2023-10-08T16:10:02.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f769659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f769659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48916/0 (socket says 172.21.15.170:48916) 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f769659c700 1 -- 172.21.15.170:0/724472150 learned_addr learned my addr 172.21.15.170:0/724472150 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f769ed9d700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a00b53d0 0x7f76a009a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.355+0000 7f769f59e700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a014b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 -- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 msgr2=0x7f76a014b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a014b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 -- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a00b53d0 msgr2=0x7f76a009a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a00b53d0 0x7f76a009a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 -- 172.21.15.170:0/724472150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76880094f0 con 0x7f76a00cce40 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f769659c700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f76a0089390 tx=0x7f7688017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688000d10 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768800bbe0 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a009b2a0 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7688011670 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.356+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a009b860 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.357+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a0063c90 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.357+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f7688011aa0 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.358+0000 7f76977fe700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 0x7f76800696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.358+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7688085df0 con 0x7f76a00cce40 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.358+0000 7f769ed9d700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 0x7f76800696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.358+0000 7f769ed9d700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 0x7f76800696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7690009da0 tx=0x7f7690009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.362+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7688053060 con 0x7f76a00cce40 2023-10-08T16:10:02.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.547+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76a00718b0 con 0x7f76a00cce40 2023-10-08T16:10:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.839+0000 7f76977fe700 1 -- 172.21.15.170:0/724472150 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7688056680 con 0x7f76a00cce40 2023-10-08T16:10:02.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 msgr2=0x7f76800696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 0x7f76800696b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7690009da0 tx=0x7f7690009120 comp rx=0 tx=0).stop 2023-10-08T16:10:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 msgr2=0x7f76a009a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f76a0089390 tx=0x7f7688017000 comp rx=0 tx=0).stop 2023-10-08T16:10:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 shutdown_connections 2023-10-08T16:10:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f76a0159ec0 0x7f76a014b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f76a00b53d0 0x7f76a009a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7680067200 0x7f76800696b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 --2- 172.21.15.170:0/724472150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76a00cce40 0x7f76a009a930 secure :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f76a0089390 tx=0x7f7688017000 comp rx=0 tx=0).stop 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.841+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 >> 172.21.15.170:0/724472150 conn(0x7f76a00999f0 msgr2=0x7f76a00d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.842+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 shutdown_connections 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.842+0000 7f76a5152700 1 -- 172.21.15.170:0/724472150 wait complete. 2023-10-08T16:10:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:10:02.857 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:10:02.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.978+0000 7f250ee5a700 1 Processor -- start 2023-10-08T16:10:02.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.978+0000 7f250ee5a700 1 -- start start 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250ee5a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25081595d0 0x7f25080ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250ee5a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080b10f0 con 0x7f25080cf480 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250ee5a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25080cd530 con 0x7f250815d930 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250ee5a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250815e000 con 0x7f25081595d0 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250de58700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25081595d0 0x7f25080ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250d657700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f25080cf480 0x7f250815d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35376/0 (socket says 172.21.15.170:35376) 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250d657700 1 -- 172.21.15.170:0/2753840212 learned_addr learned my addr 172.21.15.170:0/2753840212 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:02.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250de58700 1 -- 172.21.15.170:0/2753840212 >> v1:172.21.15.191:6789/0 conn(0x7f250815d930 legacy=0x7f25080b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250de58700 1 -- 172.21.15.170:0/2753840212 >> v1:172.21.15.170:6789/0 conn(0x7f25080cf480 legacy=0x7f250815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250de58700 1 -- 172.21.15.170:0/2753840212 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25080cf990 con 0x7f25081595d0 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.979+0000 7f250ce56700 1 -- 172.21.15.170:0/2753840212 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374923818 0 0) 0x7f25080b10f0 con 0x7f25080cf480 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250de58700 1 --2- 172.21.15.170:0/2753840212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25081595d0 0x7f25080ccff0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2504009eb0 tx=0x7f2504009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d8304c2936241b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ce56700 1 -- 172.21.15.170:0/2753840212 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250400e070 con 0x7f25081595d0 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ce56700 1 -- 172.21.15.170:0/2753840212 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250400bc30 con 0x7f25081595d0 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ce56700 1 -- 172.21.15.170:0/2753840212 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504012430 con 0x7f25081595d0 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 -- 172.21.15.170:0/2753840212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25081595d0 msgr2=0x7f25080ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 --2- 172.21.15.170:0/2753840212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25081595d0 0x7f25080ccff0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2504009eb0 tx=0x7f2504009780 comp rx=0 tx=0).stop 2023-10-08T16:10:02.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 -- 172.21.15.170:0/2753840212 shutdown_connections 2023-10-08T16:10:02.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 --2- 172.21.15.170:0/2753840212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25081595d0 0x7f25080ccff0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 -- 172.21.15.170:0/2753840212 >> 172.21.15.170:0/2753840212 conn(0x7f25080d05f0 msgr2=0x7f25080d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:02.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 -- 172.21.15.170:0/2753840212 shutdown_connections 2023-10-08T16:10:02.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.980+0000 7f250ee5a700 1 -- 172.21.15.170:0/2753840212 wait complete. 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.981+0000 7f250ee5a700 1 Processor -- start 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 -- start start 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25080cf480 0x7f25080c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 0x7f250815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25081576c0 con 0x7f250815d930 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2508156ab0 con 0x7f25081595d0 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250ee5a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2508156d90 con 0x7f25080cf480 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250e659700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250e659700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48938/0 (socket says 172.21.15.170:48938) 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.982+0000 7f250e659700 1 -- 172.21.15.170:0/323599943 learned_addr learned my addr 172.21.15.170:0/323599943 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250d657700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 0x7f250815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250de58700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25080cf480 0x7f25080c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 -- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25080cf480 msgr2=0x7f25080c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25080cf480 0x7f25080c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 -- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 msgr2=0x7f250815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 0x7f250815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 -- 172.21.15.170:0/323599943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2504009a20 con 0x7f250815d930 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250d657700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 0x7f250815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250e659700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f250000c560 tx=0x7f2500007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500011590 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000116f0 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2508157070 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500011ab0 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.983+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25080c5f00 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.984+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2508157870 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.984+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f250001b3b0 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.985+0000 7f24fdffb700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 0x7f24ec0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.985+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f2500083d80 con 0x7f250815d930 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.985+0000 7f250de58700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 0x7f24ec0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:02.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.985+0000 7f250de58700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 0x7f24ec0696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2504009eb0 tx=0x7f2504017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:02.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:02.989+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2500050da0 con 0x7f250815d930 2023-10-08T16:10:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.173+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f250804c3c0 con 0x7f250815d930 2023-10-08T16:10:03.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.843+0000 7f24fdffb700 1 -- 172.21.15.170:0/323599943 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2500017070 con 0x7f250815d930 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 msgr2=0x7f24ec0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 0x7f24ec0696b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2504009eb0 tx=0x7f2504017000 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 msgr2=0x7f250816d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f250000c560 tx=0x7f2500007ef0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 shutdown_connections 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f24ec067200 0x7f24ec0696b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f250815d930 0x7f250816d240 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25080cf480 0x7f25080c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 --2- 172.21.15.170:0/323599943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25081595d0 0x7f250815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.848+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 >> 172.21.15.170:0/323599943 conn(0x7f25080d05f0 msgr2=0x7f2508152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:03.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.851+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 shutdown_connections 2023-10-08T16:10:03.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.851+0000 7f250ee5a700 1 -- 172.21.15.170:0/323599943 wait complete. 2023-10-08T16:10:03.864 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:10:03.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:10:03.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.991+0000 7fb0caa9f700 1 Processor -- start 2023-10-08T16:10:03.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 -- start start 2023-10-08T16:10:03.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0c406bcd0 0x7fb0c40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:03.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:03.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb0c415df50 0x7fb0c415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c415d7e0 con 0x7fb0c415df50 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c415dac0 con 0x7fb0c406bcd0 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.992+0000 7fb0caa9f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415aa30 con 0x7fb0c406c0a0 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb0c415df50 0x7fb0c415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb0c415df50 0x7fb0c415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48942/0 (socket says 172.21.15.170:48942) 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 -- 172.21.15.170:0/1353020229 learned_addr learned my addr 172.21.15.170:0/1353020229 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c883b700 1 --2- 172.21.15.170:0/1353020229 >> v2:172.21.15.191:3300/0 conn(0x7fb0c406bcd0 0x7fb0c40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 -- 172.21.15.170:0/1353020229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 msgr2=0x7fb0c415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 -- 172.21.15.170:0/1353020229 >> v2:172.21.15.191:3300/0 conn(0x7fb0c406bcd0 msgr2=0x7fb0c40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:03.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 --2- 172.21.15.170:0/1353020229 >> v2:172.21.15.191:3300/0 conn(0x7fb0c406bcd0 0x7fb0c40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 -- 172.21.15.170:0/1353020229 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c415ad10 con 0x7fb0c415df50 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0c903c700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c415df50 0x7fb0c415e3c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40691a0 tx=0x7fb0bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2eb641aec4829a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0bb7fe700 1 -- 172.21.15.170:0/1353020229 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc009e70 con 0x7fb0c415df50 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0bb7fe700 1 -- 172.21.15.170:0/1353020229 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc00b860 con 0x7fb0c415df50 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.993+0000 7fb0bb7fe700 1 -- 172.21.15.170:0/1353020229 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00bc20 con 0x7fb0c415df50 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1353020229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c415df50 msgr2=0x7fb0c415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c415df50 0x7fb0c415e3c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40691a0 tx=0x7fb0bc009250 comp rx=0 tx=0).stop 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1353020229 shutdown_connections 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c415df50 0x7fb0c415e3c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1353020229 >> v2:172.21.15.191:3300/0 conn(0x7fb0c406bcd0 0x7fb0c40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1353020229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:03.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1353020229 >> 172.21.15.170:0/1353020229 conn(0x7fb0c40ce600 msgr2=0x7fb0c40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:03.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1353020229 shutdown_connections 2023-10-08T16:10:03.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.994+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1353020229 wait complete. 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.995+0000 7fb0caa9f700 1 Processor -- start 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.995+0000 7fb0caa9f700 1 -- start start 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0c415df50 0x7fb0c40c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40c47d0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c41585d0 con 0x7fb0c415df50 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40c44f0 con 0x7fb0c406c0a0 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:04.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48954/0 (socket says 172.21.15.170:48954) 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 -- 172.21.15.170:0/1055419784 learned_addr learned my addr 172.21.15.170:0/1055419784 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c903c700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0c415df50 0x7fb0c40c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 -- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 msgr2=0x7fb0c414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 -- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0c415df50 msgr2=0x7fb0c40c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0c415df50 0x7fb0c40c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 -- 172.21.15.170:0/1055419784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0094f0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0c883b700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009e50 tx=0x7fb0c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:04.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000cc20 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000f040 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4157f80 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0014480 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.996+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c4158fa0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.997+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c4157de0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.998+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb0c00147f0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.998+0000 7fb0b8ff9700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 0x7fb0b00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.998+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fb0c0085a60 con 0x7fb0c406bcd0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.998+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 0x7fb0b00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:04.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:03.999+0000 7fb0bbfff700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 0x7fb0b00696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4006f00 tx=0x7fb0b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:04.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.002+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0c0052910 con 0x7fb0c406bcd0 2023-10-08T16:10:04.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.226+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb0c4157de0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.856+0000 7fb0b8ff9700 1 -- 172.21.15.170:0/1055419784 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb0c4157de0 con 0x7fb0c406bcd0 2023-10-08T16:10:04.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.862+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 msgr2=0x7fb0b00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:04.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.862+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 0x7fb0b00696b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4006f00 tx=0x7fb0b4008040 comp rx=0 tx=0).stop 2023-10-08T16:10:04.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.862+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 msgr2=0x7fb0c4158a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:04.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.862+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009e50 tx=0x7fb0c000b040 comp rx=0 tx=0).stop 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 shutdown_connections 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0c415df50 0x7fb0c40c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0b0067200 0x7fb0b00696b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0c406bcd0 0x7fb0c4158a60 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 --2- 172.21.15.170:0/1055419784 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0c406c0a0 0x7fb0c414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:04.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 >> 172.21.15.170:0/1055419784 conn(0x7fb0c40ce600 msgr2=0x7fb0c406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:04.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 shutdown_connections 2023-10-08T16:10:04.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.864+0000 7fb0caa9f700 1 -- 172.21.15.170:0/1055419784 wait complete. 2023-10-08T16:10:04.867 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 14 and data pool 15 2023-10-08T16:10:04.882 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:04.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.996+0000 7f4f9759e700 1 Processor -- start 2023-10-08T16:10:04.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.996+0000 7f4f9759e700 1 -- start start 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f98159420 0x7f4f980b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4f981597f0 0x7f4f980b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 0x7f4f9815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9815dbb0 con 0x7f4f98159420 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f980b11d0 con 0x7f4f981597f0 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980b14b0 con 0x7f4f9815d230 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f98159420 0x7f4f980b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f98159420 0x7f4f980b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48964/0 (socket says 172.21.15.170:48964) 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 -- 172.21.15.170:0/4024375240 learned_addr learned my addr 172.21.15.170:0/4024375240 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:04.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f96d9d700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 0x7f4f9815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f8dd9b700 1 --2- 172.21.15.170:0/4024375240 >> v2:172.21.15.191:3300/0 conn(0x7f4f981597f0 0x7f4f980b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 -- 172.21.15.170:0/4024375240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 msgr2=0x7f4f9815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 0x7f4f9815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 -- 172.21.15.170:0/4024375240 >> v2:172.21.15.191:3300/0 conn(0x7f4f981597f0 msgr2=0x7f4f980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 --2- 172.21.15.170:0/4024375240 >> v2:172.21.15.191:3300/0 conn(0x7f4f981597f0 0x7f4f980b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 -- 172.21.15.170:0/4024375240 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f980b1790 con 0x7f4f98159420 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f96d9d700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 0x7f4f9815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.997+0000 7f4f9659c700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f980b5f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4f98089750 tx=0x7f4f88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc4372b16f6ab18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f95d9b700 1 -- 172.21.15.170:0/4024375240 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800b860 con 0x7f4f98159420 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f95d9b700 1 -- 172.21.15.170:0/4024375240 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8800b9c0 con 0x7f4f98159420 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f95d9b700 1 -- 172.21.15.170:0/4024375240 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800bd60 con 0x7f4f98159420 2023-10-08T16:10:05.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 -- 172.21.15.170:0/4024375240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 msgr2=0x7f4f980b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f980b5f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4f98089750 tx=0x7f4f88009250 comp rx=0 tx=0).stop 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 -- 172.21.15.170:0/4024375240 shutdown_connections 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f9815d230 0x7f4f9815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 --2- 172.21.15.170:0/4024375240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f980b5f40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 --2- 172.21.15.170:0/4024375240 >> v2:172.21.15.191:3300/0 conn(0x7f4f981597f0 0x7f4f980b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 -- 172.21.15.170:0/4024375240 >> 172.21.15.170:0/4024375240 conn(0x7f4f980d0660 msgr2=0x7f4f980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 -- 172.21.15.170:0/4024375240 shutdown_connections 2023-10-08T16:10:05.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.998+0000 7f4f9759e700 1 -- 172.21.15.170:0/4024375240 wait complete. 2023-10-08T16:10:05.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.999+0000 7f4f9759e700 1 Processor -- start 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:04.999+0000 7f4f9759e700 1 -- start start 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f981597f0 0x7f4f98156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f9815d230 0x7f4f980c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f980c6740 con 0x7f4f98159420 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f981569c0 con 0x7f4f9815d230 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9815cf40 con 0x7f4f981597f0 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48978/0 (socket says 172.21.15.170:48978) 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f9659c700 1 -- 172.21.15.170:0/1689318673 learned_addr learned my addr 172.21.15.170:0/1689318673 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:05.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f8dd9b700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f981597f0 0x7f4f98156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.000+0000 7f4f96d9d700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f9815d230 0x7f4f980c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 -- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f981597f0 msgr2=0x7f4f98156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f981597f0 0x7f4f98156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 -- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f9815d230 msgr2=0x7f4f980c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f9815d230 0x7f4f980c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 -- 172.21.15.170:0/1689318673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f880094f0 con 0x7f4f98159420 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9659c700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4f98089340 tx=0x7f4f88017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88011dc0 con 0x7f4f98159420 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88024440 con 0x7f4f98159420 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f980c60f0 con 0x7f4f98159420 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88024800 con 0x7f4f98159420 2023-10-08T16:10:05.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.001+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f980c5db0 con 0x7f4f98159420 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.002+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9804c3c0 con 0x7f4f98159420 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.002+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4f8801a420 con 0x7f4f98159420 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.003+0000 7f4f8effd700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 0x7f4f780696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.003+0000 7f4f8dd9b700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 0x7f4f780696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.003+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4f88086760 con 0x7f4f98159420 2023-10-08T16:10:05.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.003+0000 7f4f8dd9b700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 0x7f4f780696b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4f9816cda0 tx=0x7f4f80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.007+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f880536c0 con 0x7f4f98159420 2023-10-08T16:10:05.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.192+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f980cd120 con 0x7f4f98159420 2023-10-08T16:10:05.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.193+0000 7f4f8effd700 1 -- 172.21.15.170:0/1689318673 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f4f88056ce0 con 0x7f4f98159420 2023-10-08T16:10:05.194 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:05.194 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":88,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:04.228002+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:05.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 msgr2=0x7f4f780696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 0x7f4f780696b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4f9816cda0 tx=0x7f4f80008040 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 msgr2=0x7f4f98156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4f98089340 tx=0x7f4f88017000 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 shutdown_connections 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f9815d230 0x7f4f980c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f78067200 0x7f4f780696b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f98159420 0x7f4f98156bd0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 --2- 172.21.15.170:0/1689318673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f981597f0 0x7f4f98156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 >> 172.21.15.170:0/1689318673 conn(0x7f4f980d0660 msgr2=0x7f4f980ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 shutdown_connections 2023-10-08T16:10:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.196+0000 7f4f9759e700 1 -- 172.21.15.170:0/1689318673 wait complete. 2023-10-08T16:10:05.208 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:05.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.322+0000 7f9601027700 1 Processor -- start 2023-10-08T16:10:05.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.322+0000 7f9601027700 1 -- start start 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159890 0x7f95fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f95fc159c60 0x7f95fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f95fc15db10 0x7f95fc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc15a160 con 0x7f95fc159c60 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0d13a0 con 0x7f95fc15db10 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f9601027700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc15e140 con 0x7f95fc159890 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f95fc159c60 0x7f95fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f95fc159c60 0x7f95fc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49328/0 (socket says 172.21.15.170:49328) 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 -- 172.21.15.170:0/3179567728 learned_addr learned my addr 172.21.15.170:0/3179567728 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:05.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95fad9d700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159890 0x7f95fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95fb59e700 1 --2- 172.21.15.170:0/3179567728 >> v2:172.21.15.191:3300/0 conn(0x7f95fc15db10 0x7f95fc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 -- 172.21.15.170:0/3179567728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159890 msgr2=0x7f95fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159890 0x7f95fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 -- 172.21.15.170:0/3179567728 >> v2:172.21.15.191:3300/0 conn(0x7f95fc15db10 msgr2=0x7f95fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 --2- 172.21.15.170:0/3179567728 >> v2:172.21.15.191:3300/0 conn(0x7f95fc15db10 0x7f95fc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.323+0000 7f95f259c700 1 -- 172.21.15.170:0/3179567728 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc15a3d0 con 0x7f95fc159c60 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f95f259c700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159c60 0x7f95fc15d5d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f95fc089b50 tx=0x7f95e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64401337c487b89 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f95fa59c700 1 -- 172.21.15.170:0/3179567728 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400b640 con 0x7f95fc159c60 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f95fa59c700 1 -- 172.21.15.170:0/3179567728 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e400b7a0 con 0x7f95fc159c60 2023-10-08T16:10:05.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f95fa59c700 1 -- 172.21.15.170:0/3179567728 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400bb60 con 0x7f95fc159c60 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 -- 172.21.15.170:0/3179567728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159c60 msgr2=0x7f95fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159c60 0x7f95fc15d5d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f95fc089b50 tx=0x7f95e4009250 comp rx=0 tx=0).stop 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 -- 172.21.15.170:0/3179567728 shutdown_connections 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 --2- 172.21.15.170:0/3179567728 >> v2:172.21.15.191:3300/0 conn(0x7f95fc15db10 0x7f95fc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159890 0x7f95fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 --2- 172.21.15.170:0/3179567728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159c60 0x7f95fc15d5d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 -- 172.21.15.170:0/3179567728 >> 172.21.15.170:0/3179567728 conn(0x7f95fc068e20 msgr2=0x7f95fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 -- 172.21.15.170:0/3179567728 shutdown_connections 2023-10-08T16:10:05.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.324+0000 7f9601027700 1 -- 172.21.15.170:0/3179567728 wait complete. 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 Processor -- start 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 -- start start 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 0x7f95fc151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc15db10 0x7f95fc0cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0ce9d0 con 0x7f95fc159890 2023-10-08T16:10:05.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c9150 con 0x7f95fc15db10 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f9601027700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c7bf0 con 0x7f95fc159c60 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f95fad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f95fad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49340/0 (socket says 172.21.15.170:49340) 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f95fad9d700 1 -- 172.21.15.170:0/3294129287 learned_addr learned my addr 172.21.15.170:0/3294129287 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f95f259c700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 0x7f95fc151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.326+0000 7f95fb59e700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc15db10 0x7f95fc0cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 -- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 msgr2=0x7f95fc151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 0x7f95fc151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 -- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc15db10 msgr2=0x7f95fc0cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc15db10 0x7f95fc0cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 -- 172.21.15.170:0/3294129287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40094f0 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95f259c700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 0x7f95fc151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95fad9d700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00ecb0 tx=0x7f95ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00c9d0 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec00cb30 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0c96e0 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00ced0 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.327+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc0c9370 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.328+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc04c3c0 con 0x7f95fc159890 2023-10-08T16:10:05.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.328+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f95ec013680 con 0x7f95fc159890 2023-10-08T16:10:05.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.329+0000 7f95f37fe700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 0x7f95dc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.329+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f95ec084d40 con 0x7f95fc159890 2023-10-08T16:10:05.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.329+0000 7f95f259c700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 0x7f95dc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.329+0000 7f95f259c700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 0x7f95dc0696b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95fc089b50 tx=0x7f95e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.333+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95ec051c60 con 0x7f95fc159890 2023-10-08T16:10:05.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.517+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95fc06a5b0 con 0x7f95fc159890 2023-10-08T16:10:05.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.518+0000 7f95f37fe700 1 -- 172.21.15.170:0/3294129287 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+15414 (secure 0 0 0) 0x7f95ec055280 con 0x7f95fc159890 2023-10-08T16:10:05.518 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:05.518 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":88,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:04.228002+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:05.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 msgr2=0x7f95dc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 0x7f95dc0696b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95fc089b50 tx=0x7f95e4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 msgr2=0x7f95fc0b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00ecb0 tx=0x7f95ec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 shutdown_connections 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95fc15db10 0x7f95fc0cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f95dc067200 0x7f95dc0696b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95fc159890 0x7f95fc0b3590 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 --2- 172.21.15.170:0/3294129287 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95fc159c60 0x7f95fc151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 >> 172.21.15.170:0/3294129287 conn(0x7f95fc068e20 msgr2=0x7f95fc154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:05.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 shutdown_connections 2023-10-08T16:10:05.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.520+0000 7f9601027700 1 -- 172.21.15.170:0/3294129287 wait complete. 2023-10-08T16:10:05.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:10:05.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 Processor -- start 2023-10-08T16:10:05.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 -- start start 2023-10-08T16:10:05.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0b4e70 0x7f8bdc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bdc0b2080 con 0x7f8bdc15d240 2023-10-08T16:10:05.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0cfb00 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.653+0000 7f8bdb59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0b5c40 con 0x7f8bdc0b4e70 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49354/0 (socket says 172.21.15.170:49354) 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bda59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0b4e70 0x7f8bdc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 -- 172.21.15.170:0/881340420 learned_addr learned my addr 172.21.15.170:0/881340420 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 -- 172.21.15.170:0/881340420 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0b4e70 msgr2=0x7f8bdc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 --2- 172.21.15.170:0/881340420 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0b4e70 0x7f8bdc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 -- 172.21.15.170:0/881340420 >> v1:172.21.15.191:6789/0 conn(0x7f8bdc15d240 legacy=0x7f8bdc15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 -- 172.21.15.170:0/881340420 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0b5f20 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd1d9b700 1 --2- 172.21.15.170:0/881340420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc064910 tx=0x7f8bc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee856de33a01ae5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd9d9b700 1 -- 172.21.15.170:0/881340420 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009e70 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd9d9b700 1 -- 172.21.15.170:0/881340420 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc400b860 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.654+0000 7f8bd9d9b700 1 -- 172.21.15.170:0/881340420 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400bc00 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 -- 172.21.15.170:0/881340420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 msgr2=0x7f8bdc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/881340420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc064910 tx=0x7f8bc4009250 comp rx=0 tx=0).stop 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 -- 172.21.15.170:0/881340420 shutdown_connections 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/881340420 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc0b4e70 0x7f8bdc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/881340420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc0b1b40 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 -- 172.21.15.170:0/881340420 >> 172.21.15.170:0/881340420 conn(0x7f8bdc0d10c0 msgr2=0x7f8bdc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 -- 172.21.15.170:0/881340420 shutdown_connections 2023-10-08T16:10:05.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.655+0000 7f8bdb59e700 1 -- 172.21.15.170:0/881340420 wait complete. 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.656+0000 7f8bdb59e700 1 Processor -- start 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.656+0000 7f8bdb59e700 1 -- start start 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0b4e70 0x7f8bdc15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d240 0x7f8bdc15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0c6280 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc14b690 con 0x7f8bdc15d240 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc15c1c0 con 0x7f8bdc0b4e70 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49368/0 (socket says 172.21.15.170:49368) 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 -- 172.21.15.170:0/943027497 learned_addr learned my addr 172.21.15.170:0/943027497 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bd1d9b700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0b4e70 0x7f8bdc15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdad9d700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d240 0x7f8bdc15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 -- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0b4e70 msgr2=0x7f8bdc15c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0b4e70 0x7f8bdc15c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 -- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d240 msgr2=0x7f8bdc15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d240 0x7f8bdc15cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 -- 172.21.15.170:0/943027497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40094f0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bda59c700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00ecb0 tx=0x7f8bcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00c7f0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc00c950 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc15b5b0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00ccf0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.657+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc15b940 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.658+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc15c4a0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.659+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f8bcc013680 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.659+0000 7f8bd2ffd700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 0x7f8bbc0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.659+0000 7f8bd1d9b700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 0x7f8bbc0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.659+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bcc084b90 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.660+0000 7f8bd1d9b700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 0x7f8bbc0696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc064910 tx=0x7f8bc4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:05.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.663+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bcc051ab0 con 0x7f8bdc0b16d0 2023-10-08T16:10:05.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:05.892+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8bdc15bfa0 con 0x7f8bdc0b16d0 2023-10-08T16:10:06.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.861+0000 7f8bd2ffd700 1 -- 172.21.15.170:0/943027497 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8bcc0550d0 con 0x7f8bdc0b16d0 2023-10-08T16:10:06.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 msgr2=0x7f8bbc0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:06.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 0x7f8bbc0696b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc064910 tx=0x7f8bc4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:06.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 msgr2=0x7f8bdc14b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:06.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00ecb0 tx=0x7f8bcc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:06.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 shutdown_connections 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d240 0x7f8bdc15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067200 0x7f8bbc0696b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc0b16d0 0x7f8bdc14b150 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.863+0000 7f8bdb59e700 1 --2- 172.21.15.170:0/943027497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc0b4e70 0x7f8bdc15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.864+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 >> 172.21.15.170:0/943027497 conn(0x7f8bdc0d10c0 msgr2=0x7f8bdc158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.864+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 shutdown_connections 2023-10-08T16:10:06.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.864+0000 7f8bdb59e700 1 -- 172.21.15.170:0/943027497 wait complete. 2023-10-08T16:10:06.877 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:10:07.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.996+0000 7fb315b03700 1 Processor -- start 2023-10-08T16:10:07.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.996+0000 7fb315b03700 1 -- start start 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315b03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3100cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315b03700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb3100b1450 0x7fb3100c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315b03700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3100cd240 con 0x7fb310157a50 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315b03700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100cc700 con 0x7fb3100b1450 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315b03700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c9a20 con 0x7fb310157680 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb310157a50 0x7fb3100c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45580/0 (socket says 172.21.15.170:45580) 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30ffff700 1 -- 172.21.15.170:0/1978234746 learned_addr learned my addr 172.21.15.170:0/1978234746 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb315302700 1 --2- 172.21.15.170:0/1978234746 >> v2:172.21.15.191:3300/0 conn(0x7fb3100b1450 0x7fb3100c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb314b01700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3100cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246308101 0 0) 0x7fb3100cd240 con 0x7fb310157a50 2023-10-08T16:10:07.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb300003570 con 0x7fb310157a50 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43480245 0 0) 0x7fb300003570 con 0x7fb310157a50 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 msgr2=0x7fb3100cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3100cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 >> v2:172.21.15.191:3300/0 conn(0x7fb3100b1450 msgr2=0x7fb3100c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 --2- 172.21.15.170:0/1978234746 >> v2:172.21.15.191:3300/0 conn(0x7fb3100b1450 0x7fb3100c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100cc910 con 0x7fb310157a50 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb2fc001470 con 0x7fb310157a50 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 >> v1:172.21.15.170:6789/0 conn(0x7fb310157a50 legacy=0x7fb3100c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.997+0000 7fb30f7fe700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb300003a60 0x7fb300003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb300004810 0x7fb300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100cc700 con 0x7fb300003a60 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100c9a20 con 0x7fb300004810 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb30000ba60 con 0x7fb310157680 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb300003a60 0x7fb300003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30ffff700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb300004810 0x7fb300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb315302700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 -- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 msgr2=0x7fb3000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb310157680 0x7fb3000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 -- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb300004810 msgr2=0x7fb300004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb300004810 0x7fb300004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 -- 172.21.15.170:0/1978234746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300003570 con 0x7fb300003a60 2023-10-08T16:10:07.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30ffff700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb300004810 0x7fb300004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb314b01700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb300003a60 0x7fb300003e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb304009c30 tx=0x7fb3040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3125b536823da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400e710 con 0x7fb300003a60 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30400e870 con 0x7fb300003a60 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.998+0000 7fb30f7fe700 1 -- 172.21.15.170:0/1978234746 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30400ec30 con 0x7fb300003a60 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 -- 172.21.15.170:0/1978234746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb300003a60 msgr2=0x7fb300003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 --2- 172.21.15.170:0/1978234746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb300003a60 0x7fb300003e70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb304009c30 tx=0x7fb3040091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315302700 1 -- 172.21.15.170:0/1978234746 reap_dead start 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 -- 172.21.15.170:0/1978234746 shutdown_connections 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 -- 172.21.15.170:0/1978234746 >> 172.21.15.170:0/1978234746 conn(0x7fb31006baa0 msgr2=0x7fb31006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 -- 172.21.15.170:0/1978234746 shutdown_connections 2023-10-08T16:10:07.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:06.999+0000 7fb315b03700 1 -- 172.21.15.170:0/1978234746 wait complete. 2023-10-08T16:10:07.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.000+0000 7fb315b03700 1 Processor -- start 2023-10-08T16:10:07.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.000+0000 7fb315b03700 1 -- start start 2023-10-08T16:10:07.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 0x7fb310168ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31006d670 0x7fb3100b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31006db10 con 0x7fb31015a050 2023-10-08T16:10:07.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100b1950 con 0x7fb31006d670 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100b1c30 con 0x7fb300003a60 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49386/0 (socket says 172.21.15.170:49386) 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 -- 172.21.15.170:0/156093077 learned_addr learned my addr 172.21.15.170:0/156093077 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb314b01700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 0x7fb310168ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315302700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31006d670 0x7fb3100b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 -- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 msgr2=0x7fb310168ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 0x7fb310168ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 -- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31006d670 msgr2=0x7fb3100b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31006d670 0x7fb3100b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 -- 172.21.15.170:0/156093077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100b1f10 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb314b01700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 0x7fb310168ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30ffff700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00eb90 tx=0x7fb2fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00c700 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc00c860 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3100b21f0 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00cbe0 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.001+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3100b3ea0 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.002+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3100b0c50 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.003+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb2fc01d460 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.003+0000 7fb30cff9700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 0x7fb2f40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.003+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb2fc084a60 con 0x7fb31015a050 2023-10-08T16:10:07.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.003+0000 7fb314b01700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 0x7fb2f40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:07.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.004+0000 7fb314b01700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 0x7fb2f40696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3040091e0 tx=0x7fb30401a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:07.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.007+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2fc051980 con 0x7fb31015a050 2023-10-08T16:10:07.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.233+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3100ccbf0 con 0x7fb31015a050 2023-10-08T16:10:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.866+0000 7fb30cff9700 1 -- 172.21.15.170:0/156093077 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2fc054fa0 con 0x7fb31015a050 2023-10-08T16:10:07.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 msgr2=0x7fb2f40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 0x7fb2f40696b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3040091e0 tx=0x7fb30401a000 comp rx=0 tx=0).stop 2023-10-08T16:10:07.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 msgr2=0x7fb31006d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:07.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00eb90 tx=0x7fb2fc001e90 comp rx=0 tx=0).stop 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 shutdown_connections 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb31006d670 0x7fb3100b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb300003a60 0x7fb310168ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb2f4067200 0x7fb2f40696b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 --2- 172.21.15.170:0/156093077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb31015a050 0x7fb31006d130 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.868+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 >> 172.21.15.170:0/156093077 conn(0x7fb31006baa0 msgr2=0x7fb3100ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.869+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 shutdown_connections 2023-10-08T16:10:07.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.869+0000 7fb315b03700 1 -- 172.21.15.170:0/156093077 wait complete. 2023-10-08T16:10:07.882 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:07.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.997+0000 7fe0413da700 1 Processor -- start 2023-10-08T16:10:07.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.997+0000 7fe0413da700 1 -- start start 2023-10-08T16:10:08.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe0413da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159890 0x7fe03c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe0413da700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03c069560 0x7fe03c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe0413da700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c15a160 con 0x7fe03c159c60 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe0413da700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c069f20 con 0x7fe03c069560 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe0413da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0d13a0 con 0x7fe03c159890 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe03b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03c069560 0x7fe03c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.998+0000 7fe03b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe03c069560 0x7fe03c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49398/0 (socket says 172.21.15.170:49398) 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 -- 172.21.15.170:0/4237362483 learned_addr learned my addr 172.21.15.170:0/4237362483 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03affd700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159890 0x7fe03c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 -- 172.21.15.170:0/4237362483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159890 msgr2=0x7fe03c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159890 0x7fe03c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 -- 172.21.15.170:0/4237362483 >> v1:172.21.15.191:6789/0 conn(0x7fe03c159c60 legacy=0x7fe03c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 -- 172.21.15.170:0/4237362483 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c15e140 con 0x7fe03c069560 2023-10-08T16:10:08.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe03b7fe700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c069560 0x7fe03c0699e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099b0 tx=0x7fe034009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ae08097a6b8b3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe039ffb700 1 -- 172.21.15.170:0/4237362483 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034009ea0 con 0x7fe03c069560 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe039ffb700 1 -- 172.21.15.170:0/4237362483 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400b860 con 0x7fe03c069560 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:07.999+0000 7fe039ffb700 1 -- 172.21.15.170:0/4237362483 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400bc00 con 0x7fe03c069560 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 -- 172.21.15.170:0/4237362483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c069560 msgr2=0x7fe03c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c069560 0x7fe03c0699e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe0340099b0 tx=0x7fe034009250 comp rx=0 tx=0).stop 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 -- 172.21.15.170:0/4237362483 shutdown_connections 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c069560 0x7fe03c0699e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 --2- 172.21.15.170:0/4237362483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159890 0x7fe03c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 -- 172.21.15.170:0/4237362483 >> 172.21.15.170:0/4237362483 conn(0x7fe03c068e20 msgr2=0x7fe03c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 -- 172.21.15.170:0/4237362483 shutdown_connections 2023-10-08T16:10:08.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.000+0000 7fe0413da700 1 -- 172.21.15.170:0/4237362483 wait complete. 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.001+0000 7fe0413da700 1 Processor -- start 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.001+0000 7fe0413da700 1 -- start start 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 0x7fe03c14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 0x7fe03c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0cd510 con 0x7fe03c159890 2023-10-08T16:10:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0b1eb0 con 0x7fe03c069560 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0b2190 con 0x7fe03c159c60 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49400/0 (socket says 172.21.15.170:49400) 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 -- 172.21.15.170:0/2581145901 learned_addr learned my addr 172.21.15.170:0/2581145901 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03affd700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 0x7fe03c14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03b7fe700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 0x7fe03c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 -- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 msgr2=0x7fe03c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 0x7fe03c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 -- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 msgr2=0x7fe03c14d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 0x7fe03c14d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 -- 172.21.15.170:0/2581145901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0340094f0 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03affd700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 0x7fe03c14d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03b7fe700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 0x7fe03c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe03a7fc700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c7c0 tx=0x7fe030007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011590 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300116f0 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0b2470 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011ab0 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.002+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c14da60 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.003+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c0cd680 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.004+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fe03001b3b0 con 0x7fe03c159890 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.004+0000 7fe022ffd700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 0x7fe0280696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.004+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe030083ee0 con 0x7fe03c159890 2023-10-08T16:10:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.004+0000 7fe03affd700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 0x7fe0280696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.004+0000 7fe03affd700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 0x7fe0280696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0ccf90 tx=0x7fe02c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.008+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe030050e00 con 0x7fe03c159890 2023-10-08T16:10:08.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.234+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe03c04c3c0 con 0x7fe03c159890 2023-10-08T16:10:08.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.234+0000 7fe022ffd700 1 -- 172.21.15.170:0/2581145901 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fe030017070 con 0x7fe03c159890 2023-10-08T16:10:08.235 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:08.235 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 msgr2=0x7fe0280696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 0x7fe0280696b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0ccf90 tx=0x7fe02c00b040 comp rx=0 tx=0).stop 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 msgr2=0x7fe03c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe03000c7c0 tx=0x7fe030007f50 comp rx=0 tx=0).stop 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 shutdown_connections 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe03c159c60 0x7fe03c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe03c069560 0x7fe03c14d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe028067200 0x7fe0280696b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 --2- 172.21.15.170:0/2581145901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe03c159890 0x7fe03c153150 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 >> 172.21.15.170:0/2581145901 conn(0x7fe03c068e20 msgr2=0x7fe03c154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 shutdown_connections 2023-10-08T16:10:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.236+0000 7fe0413da700 1 -- 172.21.15.170:0/2581145901 wait complete. 2023-10-08T16:10:08.238 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 56 2023-10-08T16:10:08.249 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.364+0000 7f4425dcb700 1 Processor -- start 2023-10-08T16:10:08.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.364+0000 7f4425dcb700 1 -- start start 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.364+0000 7f4425dcb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 0x7f442015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f4425dcb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442015a160 con 0x7f4420069560 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f4425dcb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200d13a0 con 0x7f4420159c60 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f4425dcb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f442015e140 con 0x7f4420159890 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4420069560 0x7f44200ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45582/0 (socket says 172.21.15.170:45582) 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441ffff700 1 -- 172.21.15.170:0/4259005021 learned_addr learned my addr 172.21.15.170:0/4259005021 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441f7fe700 1 --2- 172.21.15.170:0/4259005021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 0x7f442015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990959082 0 0) 0x7f442015a160 con 0x7f4420069560 2023-10-08T16:10:08.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4408003570 con 0x7f4420069560 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286582367 0 0) 0x7f44200d13a0 con 0x7f4420159c60 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f442015a160 con 0x7f4420159c60 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441f7fe700 1 -- 172.21.15.170:0/4259005021 >> v1:172.21.15.191:6789/0 conn(0x7f4420159c60 legacy=0x7f442015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441f7fe700 1 -- 172.21.15.170:0/4259005021 >> v1:172.21.15.170:6789/0 conn(0x7f4420069560 legacy=0x7f44200ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441f7fe700 1 -- 172.21.15.170:0/4259005021 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442015a3d0 con 0x7f4420159890 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441f7fe700 1 --2- 172.21.15.170:0/4259005021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 0x7f442015abd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4410009910 tx=0x7f44100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff043bb8e1ccc632 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.365+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410011630 con 0x7f4420159890 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410011790 con 0x7f4420159890 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f441e7fc700 1 -- 172.21.15.170:0/4259005021 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410011b30 con 0x7f4420159890 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 -- 172.21.15.170:0/4259005021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 msgr2=0x7f442015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 --2- 172.21.15.170:0/4259005021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 0x7f442015abd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4410009910 tx=0x7f44100091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 -- 172.21.15.170:0/4259005021 shutdown_connections 2023-10-08T16:10:08.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 --2- 172.21.15.170:0/4259005021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159890 0x7f442015abd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 -- 172.21.15.170:0/4259005021 >> 172.21.15.170:0/4259005021 conn(0x7f4420068e20 msgr2=0x7f44200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 -- 172.21.15.170:0/4259005021 shutdown_connections 2023-10-08T16:10:08.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.366+0000 7f4425dcb700 1 -- 172.21.15.170:0/4259005021 wait complete. 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.367+0000 7f4425dcb700 1 Processor -- start 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.367+0000 7f4425dcb700 1 -- start start 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 0x7f44200b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 0x7f442015d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b62e0 con 0x7f4420069560 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b56d0 con 0x7f4420159890 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f4425dcb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b59b0 con 0x7f4420159c60 2023-10-08T16:10:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49414/0 (socket says 172.21.15.170:49414) 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 -- 172.21.15.170:0/2515174072 learned_addr learned my addr 172.21.15.170:0/2515174072 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441ffff700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 0x7f442015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441effd700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 0x7f44200b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 -- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 msgr2=0x7f442015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 0x7f442015d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 -- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 msgr2=0x7f44200b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 0x7f44200b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441f7fe700 1 -- 172.21.15.170:0/2515174072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4410009420 con 0x7f4420069560 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441ffff700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 0x7f442015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.368+0000 7f441effd700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 0x7f44200b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f441f7fe700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4410009f00 tx=0x7f4410017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441001f070 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410000bc0 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44200b5c90 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410025420 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.369+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44200b53c0 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.370+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442004c3c0 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.370+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4410023070 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.370+0000 7f44077fe700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 0x7f440c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.371+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4410085dd0 con 0x7f4420069560 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.371+0000 7f441effd700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 0x7f440c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.371+0000 7f441effd700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 0x7f440c0696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4414007fd0 tx=0x7f4414009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.375+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4410052cf0 con 0x7f4420069560 2023-10-08T16:10:08.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.559+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f442006a5b0 con 0x7f4420069560 2023-10-08T16:10:08.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.560+0000 7f44077fe700 1 -- 172.21.15.170:0/2515174072 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f441002a090 con 0x7f4420069560 2023-10-08T16:10:08.561 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:08.561 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":89,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:05.843576+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 msgr2=0x7f440c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 0x7f440c0696b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4414007fd0 tx=0x7f4414009040 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 msgr2=0x7f442014d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f4410009f00 tx=0x7f4410017000 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 shutdown_connections 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4420159c60 0x7f442015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f440c067200 0x7f440c0696b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4420069560 0x7f442014d400 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 --2- 172.21.15.170:0/2515174072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4420159890 0x7f44200b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 >> 172.21.15.170:0/2515174072 conn(0x7f4420068e20 msgr2=0x7f4420154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 shutdown_connections 2023-10-08T16:10:08.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.562+0000 7f4425dcb700 1 -- 172.21.15.170:0/2515174072 wait complete. 2023-10-08T16:10:08.575 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:10:08.575 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:08.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.688+0000 7f2fe50db700 1 Processor -- start 2023-10-08T16:10:08.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.688+0000 7f2fe50db700 1 -- start start 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fe50db700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 0x7f2fe015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fe50db700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2fe0159c60 0x7f2fe015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fe50db700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe015a160 con 0x7f2fe015db10 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fe50db700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe015df50 con 0x7f2fe0159c60 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fe50db700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe00d13a0 con 0x7f2fe0159890 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fd659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2fe0159c60 0x7f2fe015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fdf59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2fe015db10 0x7f2fe0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45586/0 (socket says 172.21.15.170:45586) 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.689+0000 7f2fdf59e700 1 -- 172.21.15.170:0/4000922673 learned_addr learned my addr 172.21.15.170:0/4000922673 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fd659c700 1 -- 172.21.15.170:0/4000922673 >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 msgr2=0x7f2fe015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fded9d700 1 --2- 172.21.15.170:0/4000922673 >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 0x7f2fe015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fd659c700 1 --2- 172.21.15.170:0/4000922673 >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 0x7f2fe015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fde59c700 1 -- 172.21.15.170:0/4000922673 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427783980 0 0) 0x7f2fe015a160 con 0x7f2fe015db10 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fd659c700 1 -- 172.21.15.170:0/4000922673 >> v1:172.21.15.170:6789/0 conn(0x7f2fe015db10 legacy=0x7f2fe0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fd659c700 1 -- 172.21.15.170:0/4000922673 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe015e140 con 0x7f2fe0159c60 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fded9d700 1 --2- 172.21.15.170:0/4000922673 >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 0x7f2fe015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fd659c700 1 --2- 172.21.15.170:0/4000922673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159c60 0x7f2fe015d5d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00d1090 tx=0x7f2fc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6552e1e2b77269 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fde59c700 1 -- 172.21.15.170:0/4000922673 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800b6b0 con 0x7f2fe0159c60 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fde59c700 1 -- 172.21.15.170:0/4000922673 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc800b810 con 0x7f2fe0159c60 2023-10-08T16:10:08.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fde59c700 1 -- 172.21.15.170:0/4000922673 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc800bbb0 con 0x7f2fe0159c60 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 -- 172.21.15.170:0/4000922673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159c60 msgr2=0x7f2fe015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 --2- 172.21.15.170:0/4000922673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159c60 0x7f2fe015d5d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2fe00d1090 tx=0x7f2fc80091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 -- 172.21.15.170:0/4000922673 shutdown_connections 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 --2- 172.21.15.170:0/4000922673 >> v2:172.21.15.191:3300/0 conn(0x7f2fe0159890 0x7f2fe015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 --2- 172.21.15.170:0/4000922673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159c60 0x7f2fe015d5d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 -- 172.21.15.170:0/4000922673 >> 172.21.15.170:0/4000922673 conn(0x7f2fe0068e20 msgr2=0x7f2fe00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 -- 172.21.15.170:0/4000922673 shutdown_connections 2023-10-08T16:10:08.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.690+0000 7f2fe50db700 1 -- 172.21.15.170:0/4000922673 wait complete. 2023-10-08T16:10:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 Processor -- start 2023-10-08T16:10:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 -- start start 2023-10-08T16:10:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 0x7f2fe0151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fe015db10 0x7f2fe014b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c7140 con 0x7f2fe0159890 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c6530 con 0x7f2fe0159c60 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fe50db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c6810 con 0x7f2fe015db10 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49436/0 (socket says 172.21.15.170:49436) 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.692+0000 7f2fded9d700 1 -- 172.21.15.170:0/162132297 learned_addr learned my addr 172.21.15.170:0/162132297 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fd659c700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 0x7f2fe0151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fdf59e700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fe015db10 0x7f2fe014b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 -- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fe015db10 msgr2=0x7f2fe014b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fe015db10 0x7f2fe014b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 -- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 msgr2=0x7f2fe0151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 0x7f2fe0151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 -- 172.21.15.170:0/162132297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009420 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fd659c700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 0x7f2fe0151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fded9d700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c830 tx=0x7f2fd0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0011600 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0011760 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe00c6af0 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0011b20 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.693+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe014aa90 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.694+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe014b240 con 0x7f2fe0159890 2023-10-08T16:10:08.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.694+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2fd00193b0 con 0x7f2fe0159890 2023-10-08T16:10:08.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.695+0000 7f2fd77fe700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 0x7f2fc00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:08.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.695+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2fd0084a70 con 0x7f2fe0159890 2023-10-08T16:10:08.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.695+0000 7f2fd659c700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 0x7f2fc00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:08.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.695+0000 7f2fd659c700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 0x7f2fc00696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80093f0 tx=0x7f2fc8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:08.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.699+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd0051880 con 0x7f2fe0159890 2023-10-08T16:10:08.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.880+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2fe00c7280 con 0x7f2fc0067200 2023-10-08T16:10:08.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.888+0000 7f2fd77fe700 1 -- 172.21.15.170:0/162132297 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286362 (secure 0 0 0) 0x7f2fc8000ce0 con 0x7f2fc0067200 2023-10-08T16:10:08.892 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 msgr2=0x7f2fc00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 0x7f2fc00696b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80093f0 tx=0x7f2fc8017000 comp rx=0 tx=0).stop 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 msgr2=0x7f2fe014a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2fd000c830 tx=0x7f2fd0008040 comp rx=0 tx=0).stop 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 shutdown_connections 2023-10-08T16:10:08.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fe015db10 0x7f2fe014b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2fc0067200 0x7f2fc00696b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fe0159890 0x7f2fe014a550 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 --2- 172.21.15.170:0/162132297 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fe0159c60 0x7f2fe0151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 >> 172.21.15.170:0/162132297 conn(0x7f2fe0068e20 msgr2=0x7f2fe014d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.891+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 shutdown_connections 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:08.892+0000 7f2fe50db700 1 -- 172.21.15.170:0/162132297 wait complete. 2023-10-08T16:10:08.895 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:08.904 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-10-08T16:10:08.292693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2404240,"kb_used_data":9552,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752570480,"statfs":{"total":773094113280,"available":770632171520,"internally_reserved":0,"allocated":9781248,"data_stored":4858098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454632"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.846576+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855906+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845596+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847904+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849710+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849731+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849180+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854318+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847899+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854692+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847882+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851768+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:34:03.864214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848862+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852017+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848718+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848600+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850460+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:08.970273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854164+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851237+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:55.867665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854404+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851540+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849101+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854806+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850963+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848987+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848807+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851334+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:34.817905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848424+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845597+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852366+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:39.030916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859417+0000","last_change":"2023-10-08T16:10:03.859417+0000","last_active":"2023-10-08T16:10:03.859417+0000","last_peered":"2023-10-08T16:10:03.859417+0000","last_clean":"2023-10-08T16:10:03.859417+0000","last_became_active":"2023-10-08T16:10:03.859153+0000","last_became_peered":"2023-10-08T16:10:03.859153+0000","last_unstale":"2023-10-08T16:10:03.859417+0000","last_undegraded":"2023-10-08T16:10:03.859417+0000","last_fullsized":"2023-10-08T16:10:03.859417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848401+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859490+0000","last_change":"2023-10-08T16:10:03.859490+0000","last_active":"2023-10-08T16:10:03.859490+0000","last_peered":"2023-10-08T16:10:03.859490+0000","last_clean":"2023-10-08T16:10:03.859490+0000","last_became_active":"2023-10-08T16:10:03.859218+0000","last_became_peered":"2023-10-08T16:10:03.859218+0000","last_unstale":"2023-10-08T16:10:03.859490+0000","last_undegraded":"2023-10-08T16:10:03.859490+0000","last_fullsized":"2023-10-08T16:10:03.859490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845567+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862941+0000","last_change":"2023-10-08T16:10:03.862941+0000","last_active":"2023-10-08T16:10:03.862941+0000","last_peered":"2023-10-08T16:10:03.862941+0000","last_clean":"2023-10-08T16:10:03.862941+0000","last_became_active":"2023-10-08T16:10:03.862732+0000","last_became_peered":"2023-10-08T16:10:03.862732+0000","last_unstale":"2023-10-08T16:10:03.862941+0000","last_undegraded":"2023-10-08T16:10:03.862941+0000","last_fullsized":"2023-10-08T16:10:03.862941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852544+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862782+0000","last_change":"2023-10-08T16:10:03.862782+0000","last_active":"2023-10-08T16:10:03.862782+0000","last_peered":"2023-10-08T16:10:03.862782+0000","last_clean":"2023-10-08T16:10:03.862782+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.862782+0000","last_undegraded":"2023-10-08T16:10:03.862782+0000","last_fullsized":"2023-10-08T16:10:03.862782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:13.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849013+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857135+0000","last_change":"2023-10-08T16:10:03.857135+0000","last_active":"2023-10-08T16:10:03.857135+0000","last_peered":"2023-10-08T16:10:03.857135+0000","last_clean":"2023-10-08T16:10:03.857135+0000","last_became_active":"2023-10-08T16:10:03.856911+0000","last_became_peered":"2023-10-08T16:10:03.856911+0000","last_unstale":"2023-10-08T16:10:03.857135+0000","last_undegraded":"2023-10-08T16:10:03.857135+0000","last_fullsized":"2023-10-08T16:10:03.857135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850231+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859382+0000","last_change":"2023-10-08T16:10:03.859382+0000","last_active":"2023-10-08T16:10:03.859382+0000","last_peered":"2023-10-08T16:10:03.859382+0000","last_clean":"2023-10-08T16:10:03.859382+0000","last_became_active":"2023-10-08T16:10:03.859137+0000","last_became_peered":"2023-10-08T16:10:03.859137+0000","last_unstale":"2023-10-08T16:10:03.859382+0000","last_undegraded":"2023-10-08T16:10:03.859382+0000","last_fullsized":"2023-10-08T16:10:03.859382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847828+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863315+0000","last_change":"2023-10-08T16:10:03.863315+0000","last_active":"2023-10-08T16:10:03.863315+0000","last_peered":"2023-10-08T16:10:03.863315+0000","last_clean":"2023-10-08T16:10:03.863315+0000","last_became_active":"2023-10-08T16:10:03.863021+0000","last_became_peered":"2023-10-08T16:10:03.863021+0000","last_unstale":"2023-10-08T16:10:03.863315+0000","last_undegraded":"2023-10-08T16:10:03.863315+0000","last_fullsized":"2023-10-08T16:10:03.863315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853269+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.850832+0000","last_change":"2023-10-08T16:10:03.850832+0000","last_active":"2023-10-08T16:10:03.850832+0000","last_peered":"2023-10-08T16:10:03.850832+0000","last_clean":"2023-10-08T16:10:03.850832+0000","last_became_active":"2023-10-08T16:10:03.850090+0000","last_became_peered":"2023-10-08T16:10:03.850090+0000","last_unstale":"2023-10-08T16:10:03.850832+0000","last_undegraded":"2023-10-08T16:10:03.850832+0000","last_fullsized":"2023-10-08T16:10:03.850832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:14.511584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.853913+0000","last_change":"2023-10-08T16:10:03.853913+0000","last_active":"2023-10-08T16:10:03.853913+0000","last_peered":"2023-10-08T16:10:03.853913+0000","last_clean":"2023-10-08T16:10:03.853913+0000","last_became_active":"2023-10-08T16:10:03.853638+0000","last_became_peered":"2023-10-08T16:10:03.853638+0000","last_unstale":"2023-10-08T16:10:03.853913+0000","last_undegraded":"2023-10-08T16:10:03.853913+0000","last_fullsized":"2023-10-08T16:10:03.853913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853636+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867602+0000","last_change":"2023-10-08T16:10:03.867602+0000","last_active":"2023-10-08T16:10:03.867602+0000","last_peered":"2023-10-08T16:10:03.867602+0000","last_clean":"2023-10-08T16:10:03.867602+0000","last_became_active":"2023-10-08T16:10:03.867359+0000","last_became_peered":"2023-10-08T16:10:03.867359+0000","last_unstale":"2023-10-08T16:10:03.867602+0000","last_undegraded":"2023-10-08T16:10:03.867602+0000","last_fullsized":"2023-10-08T16:10:03.867602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848490+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862435+0000","last_change":"2023-10-08T16:10:03.862435+0000","last_active":"2023-10-08T16:10:03.862435+0000","last_peered":"2023-10-08T16:10:03.862435+0000","last_clean":"2023-10-08T16:10:03.862435+0000","last_became_active":"2023-10-08T16:10:03.862150+0000","last_became_peered":"2023-10-08T16:10:03.862150+0000","last_unstale":"2023-10-08T16:10:03.862435+0000","last_undegraded":"2023-10-08T16:10:03.862435+0000","last_fullsized":"2023-10-08T16:10:03.862435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850485+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859113+0000","last_change":"2023-10-08T16:10:03.859113+0000","last_active":"2023-10-08T16:10:03.859113+0000","last_peered":"2023-10-08T16:10:03.859113+0000","last_clean":"2023-10-08T16:10:03.859113+0000","last_became_active":"2023-10-08T16:10:03.858920+0000","last_became_peered":"2023-10-08T16:10:03.858920+0000","last_unstale":"2023-10-08T16:10:03.859113+0000","last_undegraded":"2023-10-08T16:10:03.859113+0000","last_fullsized":"2023-10-08T16:10:03.859113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:23.618518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847341+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.860299+0000","last_change":"2023-10-08T16:10:03.860299+0000","last_active":"2023-10-08T16:10:03.860299+0000","last_peered":"2023-10-08T16:10:03.860299+0000","last_clean":"2023-10-08T16:10:03.860299+0000","last_became_active":"2023-10-08T16:10:03.860082+0000","last_became_peered":"2023-10-08T16:10:03.860082+0000","last_unstale":"2023-10-08T16:10:03.860299+0000","last_undegraded":"2023-10-08T16:10:03.860299+0000","last_fullsized":"2023-10-08T16:10:03.860299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:24.121795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844997+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857314+0000","last_change":"2023-10-08T16:10:03.857314+0000","last_active":"2023-10-08T16:10:03.857314+0000","last_peered":"2023-10-08T16:10:03.857314+0000","last_clean":"2023-10-08T16:10:03.857314+0000","last_became_active":"2023-10-08T16:10:03.857042+0000","last_became_peered":"2023-10-08T16:10:03.857042+0000","last_unstale":"2023-10-08T16:10:03.857314+0000","last_undegraded":"2023-10-08T16:10:03.857314+0000","last_fullsized":"2023-10-08T16:10:03.857314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:46.752089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851825+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862333+0000","last_change":"2023-10-08T16:10:03.862333+0000","last_active":"2023-10-08T16:10:03.862333+0000","last_peered":"2023-10-08T16:10:03.862333+0000","last_clean":"2023-10-08T16:10:03.862333+0000","last_became_active":"2023-10-08T16:10:03.862068+0000","last_became_peered":"2023-10-08T16:10:03.862068+0000","last_unstale":"2023-10-08T16:10:03.862333+0000","last_undegraded":"2023-10-08T16:10:03.862333+0000","last_fullsized":"2023-10-08T16:10:03.862333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:30.932977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851551+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:21.793105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855130+0000","last_change":"2023-10-08T16:10:03.855130+0000","last_active":"2023-10-08T16:10:03.855130+0000","last_peered":"2023-10-08T16:10:03.855130+0000","last_clean":"2023-10-08T16:10:03.855130+0000","last_became_active":"2023-10-08T16:10:03.854858+0000","last_became_peered":"2023-10-08T16:10:03.854858+0000","last_unstale":"2023-10-08T16:10:03.855130+0000","last_undegraded":"2023-10-08T16:10:03.855130+0000","last_fullsized":"2023-10-08T16:10:03.855130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848684+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859396+0000","last_change":"2023-10-08T16:10:03.859396+0000","last_active":"2023-10-08T16:10:03.859396+0000","last_peered":"2023-10-08T16:10:03.859396+0000","last_clean":"2023-10-08T16:10:03.859396+0000","last_became_active":"2023-10-08T16:10:03.859124+0000","last_became_peered":"2023-10-08T16:10:03.859124+0000","last_unstale":"2023-10-08T16:10:03.859396+0000","last_undegraded":"2023-10-08T16:10:03.859396+0000","last_fullsized":"2023-10-08T16:10:03.859396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848654+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.854991+0000","last_change":"2023-10-08T16:10:03.854991+0000","last_active":"2023-10-08T16:10:03.854991+0000","last_peered":"2023-10-08T16:10:03.854991+0000","last_clean":"2023-10-08T16:10:03.854991+0000","last_became_active":"2023-10-08T16:10:03.854728+0000","last_became_peered":"2023-10-08T16:10:03.854728+0000","last_unstale":"2023-10-08T16:10:03.854991+0000","last_undegraded":"2023-10-08T16:10:03.854991+0000","last_fullsized":"2023-10-08T16:10:03.854991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850001+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867125+0000","last_change":"2023-10-08T16:10:03.867125+0000","last_active":"2023-10-08T16:10:03.867125+0000","last_peered":"2023-10-08T16:10:03.867125+0000","last_clean":"2023-10-08T16:10:03.867125+0000","last_became_active":"2023-10-08T16:10:03.866922+0000","last_became_peered":"2023-10-08T16:10:03.866922+0000","last_unstale":"2023-10-08T16:10:03.867125+0000","last_undegraded":"2023-10-08T16:10:03.867125+0000","last_fullsized":"2023-10-08T16:10:03.867125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849640+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:50.717870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862486+0000","last_change":"2023-10-08T16:10:03.862486+0000","last_active":"2023-10-08T16:10:03.862486+0000","last_peered":"2023-10-08T16:10:03.862486+0000","last_clean":"2023-10-08T16:10:03.862486+0000","last_became_active":"2023-10-08T16:10:03.862165+0000","last_became_peered":"2023-10-08T16:10:03.862165+0000","last_unstale":"2023-10-08T16:10:03.862486+0000","last_undegraded":"2023-10-08T16:10:03.862486+0000","last_fullsized":"2023-10-08T16:10:03.862486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850795+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:16.625454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855666+0000","last_change":"2023-10-08T16:10:03.855666+0000","last_active":"2023-10-08T16:10:03.855666+0000","last_peered":"2023-10-08T16:10:03.855666+0000","last_clean":"2023-10-08T16:10:03.855666+0000","last_became_active":"2023-10-08T16:10:03.855286+0000","last_became_peered":"2023-10-08T16:10:03.855286+0000","last_unstale":"2023-10-08T16:10:03.855666+0000","last_undegraded":"2023-10-08T16:10:03.855666+0000","last_fullsized":"2023-10-08T16:10:03.855666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850693+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:55:15.457966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857899+0000","last_change":"2023-10-08T16:10:03.857899+0000","last_active":"2023-10-08T16:10:03.857899+0000","last_peered":"2023-10-08T16:10:03.857899+0000","last_clean":"2023-10-08T16:10:03.857899+0000","last_became_active":"2023-10-08T16:10:03.857621+0000","last_became_peered":"2023-10-08T16:10:03.857621+0000","last_unstale":"2023-10-08T16:10:03.857899+0000","last_undegraded":"2023-10-08T16:10:03.857899+0000","last_fullsized":"2023-10-08T16:10:03.857899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:49.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847733+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855883+0000","last_change":"2023-10-08T16:10:03.855883+0000","last_active":"2023-10-08T16:10:03.855883+0000","last_peered":"2023-10-08T16:10:03.855883+0000","last_clean":"2023-10-08T16:10:03.855883+0000","last_became_active":"2023-10-08T16:10:03.855617+0000","last_became_peered":"2023-10-08T16:10:03.855617+0000","last_unstale":"2023-10-08T16:10:03.855883+0000","last_undegraded":"2023-10-08T16:10:03.855883+0000","last_fullsized":"2023-10-08T16:10:03.855883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:14.237332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855191+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859539+0000","last_change":"2023-10-08T16:10:03.859539+0000","last_active":"2023-10-08T16:10:03.859539+0000","last_peered":"2023-10-08T16:10:03.859539+0000","last_clean":"2023-10-08T16:10:03.859539+0000","last_became_active":"2023-10-08T16:10:03.859277+0000","last_became_peered":"2023-10-08T16:10:03.859277+0000","last_unstale":"2023-10-08T16:10:03.859539+0000","last_undegraded":"2023-10-08T16:10:03.859539+0000","last_fullsized":"2023-10-08T16:10:03.859539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:58.278821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844983+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:59.285201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867454+0000","last_change":"2023-10-08T16:10:03.867454+0000","last_active":"2023-10-08T16:10:03.867454+0000","last_peered":"2023-10-08T16:10:03.867454+0000","last_clean":"2023-10-08T16:10:03.867454+0000","last_became_active":"2023-10-08T16:10:03.867150+0000","last_became_peered":"2023-10-08T16:10:03.867150+0000","last_unstale":"2023-10-08T16:10:03.867454+0000","last_undegraded":"2023-10-08T16:10:03.867454+0000","last_fullsized":"2023-10-08T16:10:03.867454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:49.978809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845004+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863109+0000","last_change":"2023-10-08T16:10:03.863109+0000","last_active":"2023-10-08T16:10:03.863109+0000","last_peered":"2023-10-08T16:10:03.863109+0000","last_clean":"2023-10-08T16:10:03.863109+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.863109+0000","last_undegraded":"2023-10-08T16:10:03.863109+0000","last_fullsized":"2023-10-08T16:10:03.863109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:46.037098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848039+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857051+0000","last_change":"2023-10-08T16:10:03.857051+0000","last_active":"2023-10-08T16:10:03.857051+0000","last_peered":"2023-10-08T16:10:03.857051+0000","last_clean":"2023-10-08T16:10:03.857051+0000","last_became_active":"2023-10-08T16:10:03.856770+0000","last_became_peered":"2023-10-08T16:10:03.856770+0000","last_unstale":"2023-10-08T16:10:03.857051+0000","last_undegraded":"2023-10-08T16:10:03.857051+0000","last_fullsized":"2023-10-08T16:10:03.857051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849203+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:59.761610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859409+0000","last_change":"2023-10-08T16:10:03.859409+0000","last_active":"2023-10-08T16:10:03.859409+0000","last_peered":"2023-10-08T16:10:03.859409+0000","last_clean":"2023-10-08T16:10:03.859409+0000","last_became_active":"2023-10-08T16:10:03.859104+0000","last_became_peered":"2023-10-08T16:10:03.859104+0000","last_unstale":"2023-10-08T16:10:03.859409+0000","last_undegraded":"2023-10-08T16:10:03.859409+0000","last_fullsized":"2023-10-08T16:10:03.859409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851897+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857623+0000","last_change":"2023-10-08T16:10:03.857623+0000","last_active":"2023-10-08T16:10:03.857623+0000","last_peered":"2023-10-08T16:10:03.857623+0000","last_clean":"2023-10-08T16:10:03.857623+0000","last_became_active":"2023-10-08T16:10:03.857351+0000","last_became_peered":"2023-10-08T16:10:03.857351+0000","last_unstale":"2023-10-08T16:10:03.857623+0000","last_undegraded":"2023-10-08T16:10:03.857623+0000","last_fullsized":"2023-10-08T16:10:03.857623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850225+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:54.252446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856881+0000","last_change":"2023-10-08T16:10:03.856881+0000","last_active":"2023-10-08T16:10:03.856881+0000","last_peered":"2023-10-08T16:10:03.856881+0000","last_clean":"2023-10-08T16:10:03.856881+0000","last_became_active":"2023-10-08T16:10:03.856607+0000","last_became_peered":"2023-10-08T16:10:03.856607+0000","last_unstale":"2023-10-08T16:10:03.856881+0000","last_undegraded":"2023-10-08T16:10:03.856881+0000","last_fullsized":"2023-10-08T16:10:03.856881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848298+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863035+0000","last_change":"2023-10-08T16:10:03.863035+0000","last_active":"2023-10-08T16:10:03.863035+0000","last_peered":"2023-10-08T16:10:03.863035+0000","last_clean":"2023-10-08T16:10:03.863035+0000","last_became_active":"2023-10-08T16:10:03.862801+0000","last_became_peered":"2023-10-08T16:10:03.862801+0000","last_unstale":"2023-10-08T16:10:03.863035+0000","last_undegraded":"2023-10-08T16:10:03.863035+0000","last_fullsized":"2023-10-08T16:10:03.863035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:31.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850391+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:03.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856920+0000","last_change":"2023-10-08T16:10:03.856920+0000","last_active":"2023-10-08T16:10:03.856920+0000","last_peered":"2023-10-08T16:10:03.856920+0000","last_clean":"2023-10-08T16:10:03.856920+0000","last_became_active":"2023-10-08T16:10:03.856673+0000","last_became_peered":"2023-10-08T16:10:03.856673+0000","last_unstale":"2023-10-08T16:10:03.856920+0000","last_undegraded":"2023-10-08T16:10:03.856920+0000","last_fullsized":"2023-10-08T16:10:03.856920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":219,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T16:10:02.842902+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:02.842902+0000","last_peered":"2023-10-08T16:10:02.842902+0000","last_clean":"2023-10-08T16:10:02.842902+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:02.842902+0000","last_undegraded":"2023-10-08T16:10:02.842902+0000","last_fullsized":"2023-10-08T16:10:02.842902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298920,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072920,"statfs":{"total":96636764160,"available":96330670080,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.43099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.219}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28199999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.312},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.27000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068824,"statfs":{"total":96636764160,"available":96326475776,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.313},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.378},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.314},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.81499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.505},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.432}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298340,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073500,"statfs":{"total":96636764160,"available":96331264000,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.503},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.439},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.624}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64200000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:08.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:09.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.057+0000 7f11529b6700 1 Processor -- start 2023-10-08T16:10:09.058 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.057+0000 7f11529b6700 1 -- start start 2023-10-08T16:10:09.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f11529b6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f114c141500 0x7f114c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f11529b6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f114c06c0a0 0x7f114c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f11529b6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c15c2a0 con 0x7f114c06bcd0 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f11529b6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0c9880 con 0x7f114c141500 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f11529b6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c15b690 con 0x7f114c06c0a0 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f114b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f114c141500 0x7f114c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f114b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f114c141500 0x7f114c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49444/0 (socket says 172.21.15.170:49444) 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f114b7fe700 1 -- 172.21.15.170:0/2522396030 learned_addr learned my addr 172.21.15.170:0/2522396030 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.058+0000 7f114bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f114c06bcd0 0x7f114c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45600/0 (socket says 172.21.15.170:45600) 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f1150f53700 1 --2- 172.21.15.170:0/2522396030 >> v2:172.21.15.191:3300/0 conn(0x7f114c06c0a0 0x7f114c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114b7fe700 1 -- 172.21.15.170:0/2522396030 >> v2:172.21.15.191:3300/0 conn(0x7f114c06c0a0 msgr2=0x7f114c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114b7fe700 1 --2- 172.21.15.170:0/2522396030 >> v2:172.21.15.191:3300/0 conn(0x7f114c06c0a0 0x7f114c0c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114b7fe700 1 -- 172.21.15.170:0/2522396030 >> v1:172.21.15.170:6789/0 conn(0x7f114c06bcd0 legacy=0x7f114c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114b7fe700 1 -- 172.21.15.170:0/2522396030 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c15b970 con 0x7f114c141500 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114affd700 1 -- 172.21.15.170:0/2522396030 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383686932 0 0) 0x7f114c15c2a0 con 0x7f114c06bcd0 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114b7fe700 1 --2- 172.21.15.170:0/2522396030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c141500 0x7f114c0c8e00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cf210 tx=0x7f1140009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320c1f282b733032 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114affd700 1 -- 172.21.15.170:0/2522396030 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140009e70 con 0x7f114c141500 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114affd700 1 -- 172.21.15.170:0/2522396030 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114000b860 con 0x7f114c141500 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.059+0000 7f114affd700 1 -- 172.21.15.170:0/2522396030 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000bc00 con 0x7f114c141500 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 -- 172.21.15.170:0/2522396030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c141500 msgr2=0x7f114c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 --2- 172.21.15.170:0/2522396030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c141500 0x7f114c0c8e00 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f114c0cf210 tx=0x7f1140009250 comp rx=0 tx=0).stop 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 -- 172.21.15.170:0/2522396030 shutdown_connections 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 --2- 172.21.15.170:0/2522396030 >> v2:172.21.15.191:3300/0 conn(0x7f114c06c0a0 0x7f114c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 --2- 172.21.15.170:0/2522396030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c141500 0x7f114c0c8e00 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 -- 172.21.15.170:0/2522396030 >> 172.21.15.170:0/2522396030 conn(0x7f114c0ce600 msgr2=0x7f114c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 -- 172.21.15.170:0/2522396030 shutdown_connections 2023-10-08T16:10:09.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.060+0000 7f11529b6700 1 -- 172.21.15.170:0/2522396030 wait complete. 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.061+0000 7f11529b6700 1 Processor -- start 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.061+0000 7f11529b6700 1 -- start start 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 0x7f114c13c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f114c141500 0x7f114c0c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c13cc20 con 0x7f114c06c0a0 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c168aa0 con 0x7f114c06bcd0 2023-10-08T16:10:09.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c167eb0 con 0x7f114c141500 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49448/0 (socket says 172.21.15.170:49448) 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 -- 172.21.15.170:0/3618094907 learned_addr learned my addr 172.21.15.170:0/3618094907 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114bfff700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 0x7f114c13c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f1150f53700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f114c141500 0x7f114c0c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 -- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f114c141500 msgr2=0x7f114c0c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f114c141500 0x7f114c0c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 -- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 msgr2=0x7f114c13c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 0x7f114c13c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 -- 172.21.15.170:0/3618094907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400094f0 con 0x7f114c06c0a0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114bfff700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 0x7f114c13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f114b7fe700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f11400094c0 tx=0x7f1140017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000bbe0 con 0x7f114c06c0a0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114001a430 con 0x7f114c06c0a0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c168170 con 0x7f114c06c0a0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114001a7f0 con 0x7f114c06c0a0 2023-10-08T16:10:09.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.062+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c168730 con 0x7f114c06c0a0 2023-10-08T16:10:09.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.063+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c168e30 con 0x7f114c06c0a0 2023-10-08T16:10:09.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.064+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f114001ab60 con 0x7f114c06c0a0 2023-10-08T16:10:09.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.064+0000 7f1133fff700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 0x7f11380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.064+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1140086660 con 0x7f114c06c0a0 2023-10-08T16:10:09.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.064+0000 7f114bfff700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 0x7f11380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.064+0000 7f114bfff700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 0x7f11380696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f113c0011e0 tx=0x7f113c007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.068+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1140053580 con 0x7f114c06c0a0 2023-10-08T16:10:09.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.246+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f114c04c3c0 con 0x7f1138067200 2023-10-08T16:10:09.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.253+0000 7f1133fff700 1 -- 172.21.15.170:0/3618094907 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286362 (secure 0 0 0) 0x7f114c04c3c0 con 0x7f1138067200 2023-10-08T16:10:09.257 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:09.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 msgr2=0x7f11380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 0x7f11380696b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f113c0011e0 tx=0x7f113c007c90 comp rx=0 tx=0).stop 2023-10-08T16:10:09.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 msgr2=0x7f114c0c4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f11400094c0 tx=0x7f1140017000 comp rx=0 tx=0).stop 2023-10-08T16:10:09.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 shutdown_connections 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f114c141500 0x7f114c0c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f114c06bcd0 0x7f114c13c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1138067200 0x7f11380696b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 --2- 172.21.15.170:0/3618094907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f114c06c0a0 0x7f114c0c4950 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 >> 172.21.15.170:0/3618094907 conn(0x7f114c0ce600 msgr2=0x7f114c158710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 shutdown_connections 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.257+0000 7f11529b6700 1 -- 172.21.15.170:0/3618094907 wait complete. 2023-10-08T16:10:09.261 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:09.269 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-10-08T16:10:08.292693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2404240,"kb_used_data":9552,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752570480,"statfs":{"total":773094113280,"available":770632171520,"internally_reserved":0,"allocated":9781248,"data_stored":4858098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454632"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.846576+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855906+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845596+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847904+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849710+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849731+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849180+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854318+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847899+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854692+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847882+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851768+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:34:03.864214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848862+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852017+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848718+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848600+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850460+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:08.970273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854164+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851237+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:55.867665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854404+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851540+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849101+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854806+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850963+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848987+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848807+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851334+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:34.817905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848424+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845597+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852366+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:39.030916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859417+0000","last_change":"2023-10-08T16:10:03.859417+0000","last_active":"2023-10-08T16:10:03.859417+0000","last_peered":"2023-10-08T16:10:03.859417+0000","last_clean":"2023-10-08T16:10:03.859417+0000","last_became_active":"2023-10-08T16:10:03.859153+0000","last_became_peered":"2023-10-08T16:10:03.859153+0000","last_unstale":"2023-10-08T16:10:03.859417+0000","last_undegraded":"2023-10-08T16:10:03.859417+0000","last_fullsized":"2023-10-08T16:10:03.859417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848401+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859490+0000","last_change":"2023-10-08T16:10:03.859490+0000","last_active":"2023-10-08T16:10:03.859490+0000","last_peered":"2023-10-08T16:10:03.859490+0000","last_clean":"2023-10-08T16:10:03.859490+0000","last_became_active":"2023-10-08T16:10:03.859218+0000","last_became_peered":"2023-10-08T16:10:03.859218+0000","last_unstale":"2023-10-08T16:10:03.859490+0000","last_undegraded":"2023-10-08T16:10:03.859490+0000","last_fullsized":"2023-10-08T16:10:03.859490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845567+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862941+0000","last_change":"2023-10-08T16:10:03.862941+0000","last_active":"2023-10-08T16:10:03.862941+0000","last_peered":"2023-10-08T16:10:03.862941+0000","last_clean":"2023-10-08T16:10:03.862941+0000","last_became_active":"2023-10-08T16:10:03.862732+0000","last_became_peered":"2023-10-08T16:10:03.862732+0000","last_unstale":"2023-10-08T16:10:03.862941+0000","last_undegraded":"2023-10-08T16:10:03.862941+0000","last_fullsized":"2023-10-08T16:10:03.862941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852544+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862782+0000","last_change":"2023-10-08T16:10:03.862782+0000","last_active":"2023-10-08T16:10:03.862782+0000","last_peered":"2023-10-08T16:10:03.862782+0000","last_clean":"2023-10-08T16:10:03.862782+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.862782+0000","last_undegraded":"2023-10-08T16:10:03.862782+0000","last_fullsized":"2023-10-08T16:10:03.862782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:13.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849013+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857135+0000","last_change":"2023-10-08T16:10:03.857135+0000","last_active":"2023-10-08T16:10:03.857135+0000","last_peered":"2023-10-08T16:10:03.857135+0000","last_clean":"2023-10-08T16:10:03.857135+0000","last_became_active":"2023-10-08T16:10:03.856911+0000","last_became_peered":"2023-10-08T16:10:03.856911+0000","last_unstale":"2023-10-08T16:10:03.857135+0000","last_undegraded":"2023-10-08T16:10:03.857135+0000","last_fullsized":"2023-10-08T16:10:03.857135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850231+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859382+0000","last_change":"2023-10-08T16:10:03.859382+0000","last_active":"2023-10-08T16:10:03.859382+0000","last_peered":"2023-10-08T16:10:03.859382+0000","last_clean":"2023-10-08T16:10:03.859382+0000","last_became_active":"2023-10-08T16:10:03.859137+0000","last_became_peered":"2023-10-08T16:10:03.859137+0000","last_unstale":"2023-10-08T16:10:03.859382+0000","last_undegraded":"2023-10-08T16:10:03.859382+0000","last_fullsized":"2023-10-08T16:10:03.859382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847828+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863315+0000","last_change":"2023-10-08T16:10:03.863315+0000","last_active":"2023-10-08T16:10:03.863315+0000","last_peered":"2023-10-08T16:10:03.863315+0000","last_clean":"2023-10-08T16:10:03.863315+0000","last_became_active":"2023-10-08T16:10:03.863021+0000","last_became_peered":"2023-10-08T16:10:03.863021+0000","last_unstale":"2023-10-08T16:10:03.863315+0000","last_undegraded":"2023-10-08T16:10:03.863315+0000","last_fullsized":"2023-10-08T16:10:03.863315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853269+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.850832+0000","last_change":"2023-10-08T16:10:03.850832+0000","last_active":"2023-10-08T16:10:03.850832+0000","last_peered":"2023-10-08T16:10:03.850832+0000","last_clean":"2023-10-08T16:10:03.850832+0000","last_became_active":"2023-10-08T16:10:03.850090+0000","last_became_peered":"2023-10-08T16:10:03.850090+0000","last_unstale":"2023-10-08T16:10:03.850832+0000","last_undegraded":"2023-10-08T16:10:03.850832+0000","last_fullsized":"2023-10-08T16:10:03.850832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:14.511584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.853913+0000","last_change":"2023-10-08T16:10:03.853913+0000","last_active":"2023-10-08T16:10:03.853913+0000","last_peered":"2023-10-08T16:10:03.853913+0000","last_clean":"2023-10-08T16:10:03.853913+0000","last_became_active":"2023-10-08T16:10:03.853638+0000","last_became_peered":"2023-10-08T16:10:03.853638+0000","last_unstale":"2023-10-08T16:10:03.853913+0000","last_undegraded":"2023-10-08T16:10:03.853913+0000","last_fullsized":"2023-10-08T16:10:03.853913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853636+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867602+0000","last_change":"2023-10-08T16:10:03.867602+0000","last_active":"2023-10-08T16:10:03.867602+0000","last_peered":"2023-10-08T16:10:03.867602+0000","last_clean":"2023-10-08T16:10:03.867602+0000","last_became_active":"2023-10-08T16:10:03.867359+0000","last_became_peered":"2023-10-08T16:10:03.867359+0000","last_unstale":"2023-10-08T16:10:03.867602+0000","last_undegraded":"2023-10-08T16:10:03.867602+0000","last_fullsized":"2023-10-08T16:10:03.867602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848490+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862435+0000","last_change":"2023-10-08T16:10:03.862435+0000","last_active":"2023-10-08T16:10:03.862435+0000","last_peered":"2023-10-08T16:10:03.862435+0000","last_clean":"2023-10-08T16:10:03.862435+0000","last_became_active":"2023-10-08T16:10:03.862150+0000","last_became_peered":"2023-10-08T16:10:03.862150+0000","last_unstale":"2023-10-08T16:10:03.862435+0000","last_undegraded":"2023-10-08T16:10:03.862435+0000","last_fullsized":"2023-10-08T16:10:03.862435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850485+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859113+0000","last_change":"2023-10-08T16:10:03.859113+0000","last_active":"2023-10-08T16:10:03.859113+0000","last_peered":"2023-10-08T16:10:03.859113+0000","last_clean":"2023-10-08T16:10:03.859113+0000","last_became_active":"2023-10-08T16:10:03.858920+0000","last_became_peered":"2023-10-08T16:10:03.858920+0000","last_unstale":"2023-10-08T16:10:03.859113+0000","last_undegraded":"2023-10-08T16:10:03.859113+0000","last_fullsized":"2023-10-08T16:10:03.859113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:23.618518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847341+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.860299+0000","last_change":"2023-10-08T16:10:03.860299+0000","last_active":"2023-10-08T16:10:03.860299+0000","last_peered":"2023-10-08T16:10:03.860299+0000","last_clean":"2023-10-08T16:10:03.860299+0000","last_became_active":"2023-10-08T16:10:03.860082+0000","last_became_peered":"2023-10-08T16:10:03.860082+0000","last_unstale":"2023-10-08T16:10:03.860299+0000","last_undegraded":"2023-10-08T16:10:03.860299+0000","last_fullsized":"2023-10-08T16:10:03.860299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:24.121795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844997+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857314+0000","last_change":"2023-10-08T16:10:03.857314+0000","last_active":"2023-10-08T16:10:03.857314+0000","last_peered":"2023-10-08T16:10:03.857314+0000","last_clean":"2023-10-08T16:10:03.857314+0000","last_became_active":"2023-10-08T16:10:03.857042+0000","last_became_peered":"2023-10-08T16:10:03.857042+0000","last_unstale":"2023-10-08T16:10:03.857314+0000","last_undegraded":"2023-10-08T16:10:03.857314+0000","last_fullsized":"2023-10-08T16:10:03.857314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:46.752089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851825+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862333+0000","last_change":"2023-10-08T16:10:03.862333+0000","last_active":"2023-10-08T16:10:03.862333+0000","last_peered":"2023-10-08T16:10:03.862333+0000","last_clean":"2023-10-08T16:10:03.862333+0000","last_became_active":"2023-10-08T16:10:03.862068+0000","last_became_peered":"2023-10-08T16:10:03.862068+0000","last_unstale":"2023-10-08T16:10:03.862333+0000","last_undegraded":"2023-10-08T16:10:03.862333+0000","last_fullsized":"2023-10-08T16:10:03.862333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:30.932977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851551+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:21.793105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855130+0000","last_change":"2023-10-08T16:10:03.855130+0000","last_active":"2023-10-08T16:10:03.855130+0000","last_peered":"2023-10-08T16:10:03.855130+0000","last_clean":"2023-10-08T16:10:03.855130+0000","last_became_active":"2023-10-08T16:10:03.854858+0000","last_became_peered":"2023-10-08T16:10:03.854858+0000","last_unstale":"2023-10-08T16:10:03.855130+0000","last_undegraded":"2023-10-08T16:10:03.855130+0000","last_fullsized":"2023-10-08T16:10:03.855130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848684+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859396+0000","last_change":"2023-10-08T16:10:03.859396+0000","last_active":"2023-10-08T16:10:03.859396+0000","last_peered":"2023-10-08T16:10:03.859396+0000","last_clean":"2023-10-08T16:10:03.859396+0000","last_became_active":"2023-10-08T16:10:03.859124+0000","last_became_peered":"2023-10-08T16:10:03.859124+0000","last_unstale":"2023-10-08T16:10:03.859396+0000","last_undegraded":"2023-10-08T16:10:03.859396+0000","last_fullsized":"2023-10-08T16:10:03.859396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848654+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.854991+0000","last_change":"2023-10-08T16:10:03.854991+0000","last_active":"2023-10-08T16:10:03.854991+0000","last_peered":"2023-10-08T16:10:03.854991+0000","last_clean":"2023-10-08T16:10:03.854991+0000","last_became_active":"2023-10-08T16:10:03.854728+0000","last_became_peered":"2023-10-08T16:10:03.854728+0000","last_unstale":"2023-10-08T16:10:03.854991+0000","last_undegraded":"2023-10-08T16:10:03.854991+0000","last_fullsized":"2023-10-08T16:10:03.854991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850001+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867125+0000","last_change":"2023-10-08T16:10:03.867125+0000","last_active":"2023-10-08T16:10:03.867125+0000","last_peered":"2023-10-08T16:10:03.867125+0000","last_clean":"2023-10-08T16:10:03.867125+0000","last_became_active":"2023-10-08T16:10:03.866922+0000","last_became_peered":"2023-10-08T16:10:03.866922+0000","last_unstale":"2023-10-08T16:10:03.867125+0000","last_undegraded":"2023-10-08T16:10:03.867125+0000","last_fullsized":"2023-10-08T16:10:03.867125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849640+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:50.717870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862486+0000","last_change":"2023-10-08T16:10:03.862486+0000","last_active":"2023-10-08T16:10:03.862486+0000","last_peered":"2023-10-08T16:10:03.862486+0000","last_clean":"2023-10-08T16:10:03.862486+0000","last_became_active":"2023-10-08T16:10:03.862165+0000","last_became_peered":"2023-10-08T16:10:03.862165+0000","last_unstale":"2023-10-08T16:10:03.862486+0000","last_undegraded":"2023-10-08T16:10:03.862486+0000","last_fullsized":"2023-10-08T16:10:03.862486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850795+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:16.625454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855666+0000","last_change":"2023-10-08T16:10:03.855666+0000","last_active":"2023-10-08T16:10:03.855666+0000","last_peered":"2023-10-08T16:10:03.855666+0000","last_clean":"2023-10-08T16:10:03.855666+0000","last_became_active":"2023-10-08T16:10:03.855286+0000","last_became_peered":"2023-10-08T16:10:03.855286+0000","last_unstale":"2023-10-08T16:10:03.855666+0000","last_undegraded":"2023-10-08T16:10:03.855666+0000","last_fullsized":"2023-10-08T16:10:03.855666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850693+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:55:15.457966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857899+0000","last_change":"2023-10-08T16:10:03.857899+0000","last_active":"2023-10-08T16:10:03.857899+0000","last_peered":"2023-10-08T16:10:03.857899+0000","last_clean":"2023-10-08T16:10:03.857899+0000","last_became_active":"2023-10-08T16:10:03.857621+0000","last_became_peered":"2023-10-08T16:10:03.857621+0000","last_unstale":"2023-10-08T16:10:03.857899+0000","last_undegraded":"2023-10-08T16:10:03.857899+0000","last_fullsized":"2023-10-08T16:10:03.857899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:49.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847733+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855883+0000","last_change":"2023-10-08T16:10:03.855883+0000","last_active":"2023-10-08T16:10:03.855883+0000","last_peered":"2023-10-08T16:10:03.855883+0000","last_clean":"2023-10-08T16:10:03.855883+0000","last_became_active":"2023-10-08T16:10:03.855617+0000","last_became_peered":"2023-10-08T16:10:03.855617+0000","last_unstale":"2023-10-08T16:10:03.855883+0000","last_undegraded":"2023-10-08T16:10:03.855883+0000","last_fullsized":"2023-10-08T16:10:03.855883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:14.237332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855191+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859539+0000","last_change":"2023-10-08T16:10:03.859539+0000","last_active":"2023-10-08T16:10:03.859539+0000","last_peered":"2023-10-08T16:10:03.859539+0000","last_clean":"2023-10-08T16:10:03.859539+0000","last_became_active":"2023-10-08T16:10:03.859277+0000","last_became_peered":"2023-10-08T16:10:03.859277+0000","last_unstale":"2023-10-08T16:10:03.859539+0000","last_undegraded":"2023-10-08T16:10:03.859539+0000","last_fullsized":"2023-10-08T16:10:03.859539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:58.278821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844983+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:59.285201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867454+0000","last_change":"2023-10-08T16:10:03.867454+0000","last_active":"2023-10-08T16:10:03.867454+0000","last_peered":"2023-10-08T16:10:03.867454+0000","last_clean":"2023-10-08T16:10:03.867454+0000","last_became_active":"2023-10-08T16:10:03.867150+0000","last_became_peered":"2023-10-08T16:10:03.867150+0000","last_unstale":"2023-10-08T16:10:03.867454+0000","last_undegraded":"2023-10-08T16:10:03.867454+0000","last_fullsized":"2023-10-08T16:10:03.867454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:49.978809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845004+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863109+0000","last_change":"2023-10-08T16:10:03.863109+0000","last_active":"2023-10-08T16:10:03.863109+0000","last_peered":"2023-10-08T16:10:03.863109+0000","last_clean":"2023-10-08T16:10:03.863109+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.863109+0000","last_undegraded":"2023-10-08T16:10:03.863109+0000","last_fullsized":"2023-10-08T16:10:03.863109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:46.037098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848039+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857051+0000","last_change":"2023-10-08T16:10:03.857051+0000","last_active":"2023-10-08T16:10:03.857051+0000","last_peered":"2023-10-08T16:10:03.857051+0000","last_clean":"2023-10-08T16:10:03.857051+0000","last_became_active":"2023-10-08T16:10:03.856770+0000","last_became_peered":"2023-10-08T16:10:03.856770+0000","last_unstale":"2023-10-08T16:10:03.857051+0000","last_undegraded":"2023-10-08T16:10:03.857051+0000","last_fullsized":"2023-10-08T16:10:03.857051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849203+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:59.761610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859409+0000","last_change":"2023-10-08T16:10:03.859409+0000","last_active":"2023-10-08T16:10:03.859409+0000","last_peered":"2023-10-08T16:10:03.859409+0000","last_clean":"2023-10-08T16:10:03.859409+0000","last_became_active":"2023-10-08T16:10:03.859104+0000","last_became_peered":"2023-10-08T16:10:03.859104+0000","last_unstale":"2023-10-08T16:10:03.859409+0000","last_undegraded":"2023-10-08T16:10:03.859409+0000","last_fullsized":"2023-10-08T16:10:03.859409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851897+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857623+0000","last_change":"2023-10-08T16:10:03.857623+0000","last_active":"2023-10-08T16:10:03.857623+0000","last_peered":"2023-10-08T16:10:03.857623+0000","last_clean":"2023-10-08T16:10:03.857623+0000","last_became_active":"2023-10-08T16:10:03.857351+0000","last_became_peered":"2023-10-08T16:10:03.857351+0000","last_unstale":"2023-10-08T16:10:03.857623+0000","last_undegraded":"2023-10-08T16:10:03.857623+0000","last_fullsized":"2023-10-08T16:10:03.857623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850225+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:54.252446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856881+0000","last_change":"2023-10-08T16:10:03.856881+0000","last_active":"2023-10-08T16:10:03.856881+0000","last_peered":"2023-10-08T16:10:03.856881+0000","last_clean":"2023-10-08T16:10:03.856881+0000","last_became_active":"2023-10-08T16:10:03.856607+0000","last_became_peered":"2023-10-08T16:10:03.856607+0000","last_unstale":"2023-10-08T16:10:03.856881+0000","last_undegraded":"2023-10-08T16:10:03.856881+0000","last_fullsized":"2023-10-08T16:10:03.856881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848298+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863035+0000","last_change":"2023-10-08T16:10:03.863035+0000","last_active":"2023-10-08T16:10:03.863035+0000","last_peered":"2023-10-08T16:10:03.863035+0000","last_clean":"2023-10-08T16:10:03.863035+0000","last_became_active":"2023-10-08T16:10:03.862801+0000","last_became_peered":"2023-10-08T16:10:03.862801+0000","last_unstale":"2023-10-08T16:10:03.863035+0000","last_undegraded":"2023-10-08T16:10:03.863035+0000","last_fullsized":"2023-10-08T16:10:03.863035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:31.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850391+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:03.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856920+0000","last_change":"2023-10-08T16:10:03.856920+0000","last_active":"2023-10-08T16:10:03.856920+0000","last_peered":"2023-10-08T16:10:03.856920+0000","last_clean":"2023-10-08T16:10:03.856920+0000","last_became_active":"2023-10-08T16:10:03.856673+0000","last_became_peered":"2023-10-08T16:10:03.856673+0000","last_unstale":"2023-10-08T16:10:03.856920+0000","last_undegraded":"2023-10-08T16:10:03.856920+0000","last_fullsized":"2023-10-08T16:10:03.856920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":219,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T16:10:02.842902+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:02.842902+0000","last_peered":"2023-10-08T16:10:02.842902+0000","last_clean":"2023-10-08T16:10:02.842902+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:02.842902+0000","last_undegraded":"2023-10-08T16:10:02.842902+0000","last_fullsized":"2023-10-08T16:10:02.842902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298920,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072920,"statfs":{"total":96636764160,"available":96330670080,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.43099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.219}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28199999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.312},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.27000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068824,"statfs":{"total":96636764160,"available":96326475776,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.313},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.378},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.314},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.81499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.505},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.432}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298340,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073500,"statfs":{"total":96636764160,"available":96331264000,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.503},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.439},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.624}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64200000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:09.276 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:09.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.423+0000 7f47cdc9e700 1 Processor -- start 2023-10-08T16:10:09.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.423+0000 7f47cdc9e700 1 -- start start 2023-10-08T16:10:09.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cdc9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c806c0f0 0x7f47c8159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cdc9e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f47c80b3810 0x7f47c80b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cdc9e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c8143420 con 0x7f47c8143050 2023-10-08T16:10:09.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cdc9e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c806a1e0 con 0x7f47c80b3810 2023-10-08T16:10:09.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cdc9e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c80695d0 con 0x7f47c806c0f0 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c806c0f0 0x7f47c8159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47cd49d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f47c80b3810 0x7f47c80b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47ccc9c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f47c8143050 0x7f47c806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:39240/0 (socket says 172.21.15.170:39240) 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.424+0000 7f47ccc9c700 1 -- 172.21.15.170:0/1099445992 learned_addr learned my addr 172.21.15.170:0/1099445992 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cd49d700 1 -- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c806c0f0 msgr2=0x7f47c8159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cd49d700 1 --2- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c806c0f0 0x7f47c8159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cd49d700 1 -- 172.21.15.170:0/1099445992 >> v1:172.21.15.191:6789/0 conn(0x7f47c8143050 legacy=0x7f47c806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cd49d700 1 -- 172.21.15.170:0/1099445992 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80698b0 con 0x7f47c80b3810 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47bf7fe700 1 -- 172.21.15.170:0/1099445992 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437661853 0 0) 0x7f47c8143420 con 0x7f47c8143050 2023-10-08T16:10:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cd49d700 1 --2- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c80b3810 0x7f47c80b3c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f47c00098e0 tx=0x7f47c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f680ae369a5c0e3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47bf7fe700 1 -- 172.21.15.170:0/1099445992 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000b640 con 0x7f47c80b3810 2023-10-08T16:10:09.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47bf7fe700 1 -- 172.21.15.170:0/1099445992 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c000b7a0 con 0x7f47c80b3810 2023-10-08T16:10:09.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47bf7fe700 1 -- 172.21.15.170:0/1099445992 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c000bb40 con 0x7f47c80b3810 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 -- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c80b3810 msgr2=0x7f47c80b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c80b3810 0x7f47c80b3c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f47c00098e0 tx=0x7f47c00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 -- 172.21.15.170:0/1099445992 shutdown_connections 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c80b3810 0x7f47c80b3c90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/1099445992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c806c0f0 0x7f47c8159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 -- 172.21.15.170:0/1099445992 >> 172.21.15.170:0/1099445992 conn(0x7f47c80ce430 msgr2=0x7f47c80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 -- 172.21.15.170:0/1099445992 shutdown_connections 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.425+0000 7f47cdc9e700 1 -- 172.21.15.170:0/1099445992 wait complete. 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.426+0000 7f47cdc9e700 1 Processor -- start 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 -- start start 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c80b3810 0x7f47c806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c8143050 0x7f47c815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815cd90 con 0x7f47c806c0f0 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c815b120 con 0x7f47c8143050 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47cdc9e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c813a200 con 0x7f47c80b3810 2023-10-08T16:10:09.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47ccc9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47ccc9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49458/0 (socket says 172.21.15.170:49458) 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.427+0000 7f47ccc9c700 1 -- 172.21.15.170:0/822444529 learned_addr learned my addr 172.21.15.170:0/822444529 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47bffff700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c80b3810 0x7f47c806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47cd49d700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c8143050 0x7f47c815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 -- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c80b3810 msgr2=0x7f47c806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c80b3810 0x7f47c806d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 -- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c8143050 msgr2=0x7f47c815a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c8143050 0x7f47c815a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 -- 172.21.15.170:0/822444529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0009420 con 0x7f47c806c0f0 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47ccc9c700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47c400cc10 tx=0x7f47c4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400a8b0 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c400aa30 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c813a4e0 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c400adf0 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.428+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c8139420 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.429+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c813a060 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.430+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f47c4012610 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.430+0000 7f47bcff9700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 0x7f47b40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.430+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f47c4084c00 con 0x7f47c806c0f0 2023-10-08T16:10:09.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.430+0000 7f47bffff700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 0x7f47b40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.430+0000 7f47bffff700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 0x7f47b40696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47c816de50 tx=0x7f47b80091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.434+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47c813a060 con 0x7f47c806c0f0 2023-10-08T16:10:09.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.613+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47c813a060 con 0x7f47b4067200 2023-10-08T16:10:09.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.621+0000 7f47bcff9700 1 -- 172.21.15.170:0/822444529 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286362 (secure 0 0 0) 0x7f47c813a060 con 0x7f47b4067200 2023-10-08T16:10:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:09.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 msgr2=0x7f47b40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 0x7f47b40696b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f47c816de50 tx=0x7f47b80091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 msgr2=0x7f47c815c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f47c400cc10 tx=0x7f47c4001a20 comp rx=0 tx=0).stop 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 shutdown_connections 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f47c8143050 0x7f47c815a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47b4067200 0x7f47b40696b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47c806c0f0 0x7f47c815c850 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 --2- 172.21.15.170:0/822444529 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f47c80b3810 0x7f47c806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 >> 172.21.15.170:0/822444529 conn(0x7f47c80ce430 msgr2=0x7f47c80adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 shutdown_connections 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.624+0000 7f47cdc9e700 1 -- 172.21.15.170:0/822444529 wait complete. 2023-10-08T16:10:09.628 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:09.636 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2023-10-08T16:10:08.292693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2404240,"kb_used_data":9552,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752570480,"statfs":{"total":773094113280,"available":770632171520,"internally_reserved":0,"allocated":9781248,"data_stored":4858098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454632"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.846576+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855906+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845596+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847904+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849710+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849731+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849180+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854318+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847899+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851444+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854692+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847882+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851768+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:34:03.864214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848862+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852017+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848718+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848600+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850460+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:03:08.970273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854164+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851237+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:55.867665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854404+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851540+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849101+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854806+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850963+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848987+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848807+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851334+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:28:34.817905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848424+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845597+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852366+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:39.030916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859417+0000","last_change":"2023-10-08T16:10:03.859417+0000","last_active":"2023-10-08T16:10:03.859417+0000","last_peered":"2023-10-08T16:10:03.859417+0000","last_clean":"2023-10-08T16:10:03.859417+0000","last_became_active":"2023-10-08T16:10:03.859153+0000","last_became_peered":"2023-10-08T16:10:03.859153+0000","last_unstale":"2023-10-08T16:10:03.859417+0000","last_undegraded":"2023-10-08T16:10:03.859417+0000","last_fullsized":"2023-10-08T16:10:03.859417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848401+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859490+0000","last_change":"2023-10-08T16:10:03.859490+0000","last_active":"2023-10-08T16:10:03.859490+0000","last_peered":"2023-10-08T16:10:03.859490+0000","last_clean":"2023-10-08T16:10:03.859490+0000","last_became_active":"2023-10-08T16:10:03.859218+0000","last_became_peered":"2023-10-08T16:10:03.859218+0000","last_unstale":"2023-10-08T16:10:03.859490+0000","last_undegraded":"2023-10-08T16:10:03.859490+0000","last_fullsized":"2023-10-08T16:10:03.859490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845567+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862941+0000","last_change":"2023-10-08T16:10:03.862941+0000","last_active":"2023-10-08T16:10:03.862941+0000","last_peered":"2023-10-08T16:10:03.862941+0000","last_clean":"2023-10-08T16:10:03.862941+0000","last_became_active":"2023-10-08T16:10:03.862732+0000","last_became_peered":"2023-10-08T16:10:03.862732+0000","last_unstale":"2023-10-08T16:10:03.862941+0000","last_undegraded":"2023-10-08T16:10:03.862941+0000","last_fullsized":"2023-10-08T16:10:03.862941+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.852544+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862782+0000","last_change":"2023-10-08T16:10:03.862782+0000","last_active":"2023-10-08T16:10:03.862782+0000","last_peered":"2023-10-08T16:10:03.862782+0000","last_clean":"2023-10-08T16:10:03.862782+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.862782+0000","last_undegraded":"2023-10-08T16:10:03.862782+0000","last_fullsized":"2023-10-08T16:10:03.862782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:13.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849013+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857135+0000","last_change":"2023-10-08T16:10:03.857135+0000","last_active":"2023-10-08T16:10:03.857135+0000","last_peered":"2023-10-08T16:10:03.857135+0000","last_clean":"2023-10-08T16:10:03.857135+0000","last_became_active":"2023-10-08T16:10:03.856911+0000","last_became_peered":"2023-10-08T16:10:03.856911+0000","last_unstale":"2023-10-08T16:10:03.857135+0000","last_undegraded":"2023-10-08T16:10:03.857135+0000","last_fullsized":"2023-10-08T16:10:03.857135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850231+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859382+0000","last_change":"2023-10-08T16:10:03.859382+0000","last_active":"2023-10-08T16:10:03.859382+0000","last_peered":"2023-10-08T16:10:03.859382+0000","last_clean":"2023-10-08T16:10:03.859382+0000","last_became_active":"2023-10-08T16:10:03.859137+0000","last_became_peered":"2023-10-08T16:10:03.859137+0000","last_unstale":"2023-10-08T16:10:03.859382+0000","last_undegraded":"2023-10-08T16:10:03.859382+0000","last_fullsized":"2023-10-08T16:10:03.859382+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847828+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863315+0000","last_change":"2023-10-08T16:10:03.863315+0000","last_active":"2023-10-08T16:10:03.863315+0000","last_peered":"2023-10-08T16:10:03.863315+0000","last_clean":"2023-10-08T16:10:03.863315+0000","last_became_active":"2023-10-08T16:10:03.863021+0000","last_became_peered":"2023-10-08T16:10:03.863021+0000","last_unstale":"2023-10-08T16:10:03.863315+0000","last_undegraded":"2023-10-08T16:10:03.863315+0000","last_fullsized":"2023-10-08T16:10:03.863315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853269+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.850832+0000","last_change":"2023-10-08T16:10:03.850832+0000","last_active":"2023-10-08T16:10:03.850832+0000","last_peered":"2023-10-08T16:10:03.850832+0000","last_clean":"2023-10-08T16:10:03.850832+0000","last_became_active":"2023-10-08T16:10:03.850090+0000","last_became_peered":"2023-10-08T16:10:03.850090+0000","last_unstale":"2023-10-08T16:10:03.850832+0000","last_undegraded":"2023-10-08T16:10:03.850832+0000","last_fullsized":"2023-10-08T16:10:03.850832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:14.511584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.854417+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.853913+0000","last_change":"2023-10-08T16:10:03.853913+0000","last_active":"2023-10-08T16:10:03.853913+0000","last_peered":"2023-10-08T16:10:03.853913+0000","last_clean":"2023-10-08T16:10:03.853913+0000","last_became_active":"2023-10-08T16:10:03.853638+0000","last_became_peered":"2023-10-08T16:10:03.853638+0000","last_unstale":"2023-10-08T16:10:03.853913+0000","last_undegraded":"2023-10-08T16:10:03.853913+0000","last_fullsized":"2023-10-08T16:10:03.853913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.853636+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867602+0000","last_change":"2023-10-08T16:10:03.867602+0000","last_active":"2023-10-08T16:10:03.867602+0000","last_peered":"2023-10-08T16:10:03.867602+0000","last_clean":"2023-10-08T16:10:03.867602+0000","last_became_active":"2023-10-08T16:10:03.867359+0000","last_became_peered":"2023-10-08T16:10:03.867359+0000","last_unstale":"2023-10-08T16:10:03.867602+0000","last_undegraded":"2023-10-08T16:10:03.867602+0000","last_fullsized":"2023-10-08T16:10:03.867602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848490+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862435+0000","last_change":"2023-10-08T16:10:03.862435+0000","last_active":"2023-10-08T16:10:03.862435+0000","last_peered":"2023-10-08T16:10:03.862435+0000","last_clean":"2023-10-08T16:10:03.862435+0000","last_became_active":"2023-10-08T16:10:03.862150+0000","last_became_peered":"2023-10-08T16:10:03.862150+0000","last_unstale":"2023-10-08T16:10:03.862435+0000","last_undegraded":"2023-10-08T16:10:03.862435+0000","last_fullsized":"2023-10-08T16:10:03.862435+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850485+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859113+0000","last_change":"2023-10-08T16:10:03.859113+0000","last_active":"2023-10-08T16:10:03.859113+0000","last_peered":"2023-10-08T16:10:03.859113+0000","last_clean":"2023-10-08T16:10:03.859113+0000","last_became_active":"2023-10-08T16:10:03.858920+0000","last_became_peered":"2023-10-08T16:10:03.858920+0000","last_unstale":"2023-10-08T16:10:03.859113+0000","last_undegraded":"2023-10-08T16:10:03.859113+0000","last_fullsized":"2023-10-08T16:10:03.859113+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:23.618518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847341+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.860299+0000","last_change":"2023-10-08T16:10:03.860299+0000","last_active":"2023-10-08T16:10:03.860299+0000","last_peered":"2023-10-08T16:10:03.860299+0000","last_clean":"2023-10-08T16:10:03.860299+0000","last_became_active":"2023-10-08T16:10:03.860082+0000","last_became_peered":"2023-10-08T16:10:03.860082+0000","last_unstale":"2023-10-08T16:10:03.860299+0000","last_undegraded":"2023-10-08T16:10:03.860299+0000","last_fullsized":"2023-10-08T16:10:03.860299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:24.121795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844997+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857314+0000","last_change":"2023-10-08T16:10:03.857314+0000","last_active":"2023-10-08T16:10:03.857314+0000","last_peered":"2023-10-08T16:10:03.857314+0000","last_clean":"2023-10-08T16:10:03.857314+0000","last_became_active":"2023-10-08T16:10:03.857042+0000","last_became_peered":"2023-10-08T16:10:03.857042+0000","last_unstale":"2023-10-08T16:10:03.857314+0000","last_undegraded":"2023-10-08T16:10:03.857314+0000","last_fullsized":"2023-10-08T16:10:03.857314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:46.752089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851825+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862333+0000","last_change":"2023-10-08T16:10:03.862333+0000","last_active":"2023-10-08T16:10:03.862333+0000","last_peered":"2023-10-08T16:10:03.862333+0000","last_clean":"2023-10-08T16:10:03.862333+0000","last_became_active":"2023-10-08T16:10:03.862068+0000","last_became_peered":"2023-10-08T16:10:03.862068+0000","last_unstale":"2023-10-08T16:10:03.862333+0000","last_undegraded":"2023-10-08T16:10:03.862333+0000","last_fullsized":"2023-10-08T16:10:03.862333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:30.932977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851551+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:21.793105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855130+0000","last_change":"2023-10-08T16:10:03.855130+0000","last_active":"2023-10-08T16:10:03.855130+0000","last_peered":"2023-10-08T16:10:03.855130+0000","last_clean":"2023-10-08T16:10:03.855130+0000","last_became_active":"2023-10-08T16:10:03.854858+0000","last_became_peered":"2023-10-08T16:10:03.854858+0000","last_unstale":"2023-10-08T16:10:03.855130+0000","last_undegraded":"2023-10-08T16:10:03.855130+0000","last_fullsized":"2023-10-08T16:10:03.855130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848684+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859396+0000","last_change":"2023-10-08T16:10:03.859396+0000","last_active":"2023-10-08T16:10:03.859396+0000","last_peered":"2023-10-08T16:10:03.859396+0000","last_clean":"2023-10-08T16:10:03.859396+0000","last_became_active":"2023-10-08T16:10:03.859124+0000","last_became_peered":"2023-10-08T16:10:03.859124+0000","last_unstale":"2023-10-08T16:10:03.859396+0000","last_undegraded":"2023-10-08T16:10:03.859396+0000","last_fullsized":"2023-10-08T16:10:03.859396+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848654+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.854991+0000","last_change":"2023-10-08T16:10:03.854991+0000","last_active":"2023-10-08T16:10:03.854991+0000","last_peered":"2023-10-08T16:10:03.854991+0000","last_clean":"2023-10-08T16:10:03.854991+0000","last_became_active":"2023-10-08T16:10:03.854728+0000","last_became_peered":"2023-10-08T16:10:03.854728+0000","last_unstale":"2023-10-08T16:10:03.854991+0000","last_undegraded":"2023-10-08T16:10:03.854991+0000","last_fullsized":"2023-10-08T16:10:03.854991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850001+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867125+0000","last_change":"2023-10-08T16:10:03.867125+0000","last_active":"2023-10-08T16:10:03.867125+0000","last_peered":"2023-10-08T16:10:03.867125+0000","last_clean":"2023-10-08T16:10:03.867125+0000","last_became_active":"2023-10-08T16:10:03.866922+0000","last_became_peered":"2023-10-08T16:10:03.866922+0000","last_unstale":"2023-10-08T16:10:03.867125+0000","last_undegraded":"2023-10-08T16:10:03.867125+0000","last_fullsized":"2023-10-08T16:10:03.867125+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849640+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:57:50.717870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.862486+0000","last_change":"2023-10-08T16:10:03.862486+0000","last_active":"2023-10-08T16:10:03.862486+0000","last_peered":"2023-10-08T16:10:03.862486+0000","last_clean":"2023-10-08T16:10:03.862486+0000","last_became_active":"2023-10-08T16:10:03.862165+0000","last_became_peered":"2023-10-08T16:10:03.862165+0000","last_unstale":"2023-10-08T16:10:03.862486+0000","last_undegraded":"2023-10-08T16:10:03.862486+0000","last_fullsized":"2023-10-08T16:10:03.862486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850795+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:16.625454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855666+0000","last_change":"2023-10-08T16:10:03.855666+0000","last_active":"2023-10-08T16:10:03.855666+0000","last_peered":"2023-10-08T16:10:03.855666+0000","last_clean":"2023-10-08T16:10:03.855666+0000","last_became_active":"2023-10-08T16:10:03.855286+0000","last_became_peered":"2023-10-08T16:10:03.855286+0000","last_unstale":"2023-10-08T16:10:03.855666+0000","last_undegraded":"2023-10-08T16:10:03.855666+0000","last_fullsized":"2023-10-08T16:10:03.855666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850693+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:55:15.457966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857899+0000","last_change":"2023-10-08T16:10:03.857899+0000","last_active":"2023-10-08T16:10:03.857899+0000","last_peered":"2023-10-08T16:10:03.857899+0000","last_clean":"2023-10-08T16:10:03.857899+0000","last_became_active":"2023-10-08T16:10:03.857621+0000","last_became_peered":"2023-10-08T16:10:03.857621+0000","last_unstale":"2023-10-08T16:10:03.857899+0000","last_undegraded":"2023-10-08T16:10:03.857899+0000","last_fullsized":"2023-10-08T16:10:03.857899+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:49.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.847733+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.855883+0000","last_change":"2023-10-08T16:10:03.855883+0000","last_active":"2023-10-08T16:10:03.855883+0000","last_peered":"2023-10-08T16:10:03.855883+0000","last_clean":"2023-10-08T16:10:03.855883+0000","last_became_active":"2023-10-08T16:10:03.855617+0000","last_became_peered":"2023-10-08T16:10:03.855617+0000","last_unstale":"2023-10-08T16:10:03.855883+0000","last_undegraded":"2023-10-08T16:10:03.855883+0000","last_fullsized":"2023-10-08T16:10:03.855883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:14.237332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.855191+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859539+0000","last_change":"2023-10-08T16:10:03.859539+0000","last_active":"2023-10-08T16:10:03.859539+0000","last_peered":"2023-10-08T16:10:03.859539+0000","last_clean":"2023-10-08T16:10:03.859539+0000","last_became_active":"2023-10-08T16:10:03.859277+0000","last_became_peered":"2023-10-08T16:10:03.859277+0000","last_unstale":"2023-10-08T16:10:03.859539+0000","last_undegraded":"2023-10-08T16:10:03.859539+0000","last_fullsized":"2023-10-08T16:10:03.859539+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:58.278821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.844983+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:59.285201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.867454+0000","last_change":"2023-10-08T16:10:03.867454+0000","last_active":"2023-10-08T16:10:03.867454+0000","last_peered":"2023-10-08T16:10:03.867454+0000","last_clean":"2023-10-08T16:10:03.867454+0000","last_became_active":"2023-10-08T16:10:03.867150+0000","last_became_peered":"2023-10-08T16:10:03.867150+0000","last_unstale":"2023-10-08T16:10:03.867454+0000","last_undegraded":"2023-10-08T16:10:03.867454+0000","last_fullsized":"2023-10-08T16:10:03.867454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:49.978809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.845004+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863109+0000","last_change":"2023-10-08T16:10:03.863109+0000","last_active":"2023-10-08T16:10:03.863109+0000","last_peered":"2023-10-08T16:10:03.863109+0000","last_clean":"2023-10-08T16:10:03.863109+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:03.863109+0000","last_undegraded":"2023-10-08T16:10:03.863109+0000","last_fullsized":"2023-10-08T16:10:03.863109+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:46.037098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848039+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857051+0000","last_change":"2023-10-08T16:10:03.857051+0000","last_active":"2023-10-08T16:10:03.857051+0000","last_peered":"2023-10-08T16:10:03.857051+0000","last_clean":"2023-10-08T16:10:03.857051+0000","last_became_active":"2023-10-08T16:10:03.856770+0000","last_became_peered":"2023-10-08T16:10:03.856770+0000","last_unstale":"2023-10-08T16:10:03.857051+0000","last_undegraded":"2023-10-08T16:10:03.857051+0000","last_fullsized":"2023-10-08T16:10:03.857051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.849203+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:59.761610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.859409+0000","last_change":"2023-10-08T16:10:03.859409+0000","last_active":"2023-10-08T16:10:03.859409+0000","last_peered":"2023-10-08T16:10:03.859409+0000","last_clean":"2023-10-08T16:10:03.859409+0000","last_became_active":"2023-10-08T16:10:03.859104+0000","last_became_peered":"2023-10-08T16:10:03.859104+0000","last_unstale":"2023-10-08T16:10:03.859409+0000","last_undegraded":"2023-10-08T16:10:03.859409+0000","last_fullsized":"2023-10-08T16:10:03.859409+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.851897+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.857623+0000","last_change":"2023-10-08T16:10:03.857623+0000","last_active":"2023-10-08T16:10:03.857623+0000","last_peered":"2023-10-08T16:10:03.857623+0000","last_clean":"2023-10-08T16:10:03.857623+0000","last_became_active":"2023-10-08T16:10:03.857351+0000","last_became_peered":"2023-10-08T16:10:03.857351+0000","last_unstale":"2023-10-08T16:10:03.857623+0000","last_undegraded":"2023-10-08T16:10:03.857623+0000","last_fullsized":"2023-10-08T16:10:03.857623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850225+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:54.252446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856881+0000","last_change":"2023-10-08T16:10:03.856881+0000","last_active":"2023-10-08T16:10:03.856881+0000","last_peered":"2023-10-08T16:10:03.856881+0000","last_clean":"2023-10-08T16:10:03.856881+0000","last_became_active":"2023-10-08T16:10:03.856607+0000","last_became_peered":"2023-10-08T16:10:03.856607+0000","last_unstale":"2023-10-08T16:10:03.856881+0000","last_undegraded":"2023-10-08T16:10:03.856881+0000","last_fullsized":"2023-10-08T16:10:03.856881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.848298+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.863035+0000","last_change":"2023-10-08T16:10:03.863035+0000","last_active":"2023-10-08T16:10:03.863035+0000","last_peered":"2023-10-08T16:10:03.863035+0000","last_clean":"2023-10-08T16:10:03.863035+0000","last_became_active":"2023-10-08T16:10:03.862801+0000","last_became_peered":"2023-10-08T16:10:03.862801+0000","last_unstale":"2023-10-08T16:10:03.863035+0000","last_undegraded":"2023-10-08T16:10:03.863035+0000","last_fullsized":"2023-10-08T16:10:03.863035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:31.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:03.838041+0000","last_change":"2023-10-08T16:10:03.850391+0000","last_active":"2023-10-08T16:10:03.838041+0000","last_peered":"2023-10-08T16:10:03.838041+0000","last_clean":"2023-10-08T16:10:03.838041+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:03.838041+0000","last_undegraded":"2023-10-08T16:10:03.838041+0000","last_fullsized":"2023-10-08T16:10:03.838041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:03.296819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T16:10:03.856920+0000","last_change":"2023-10-08T16:10:03.856920+0000","last_active":"2023-10-08T16:10:03.856920+0000","last_peered":"2023-10-08T16:10:03.856920+0000","last_clean":"2023-10-08T16:10:03.856920+0000","last_became_active":"2023-10-08T16:10:03.856673+0000","last_became_peered":"2023-10-08T16:10:03.856673+0000","last_unstale":"2023-10-08T16:10:03.856920+0000","last_undegraded":"2023-10-08T16:10:03.856920+0000","last_fullsized":"2023-10-08T16:10:03.856920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":219,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-08T16:10:02.842902+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:02.842902+0000","last_peered":"2023-10-08T16:10:02.842902+0000","last_clean":"2023-10-08T16:10:02.842902+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:02.842902+0000","last_undegraded":"2023-10-08T16:10:02.842902+0000","last_fullsized":"2023-10-08T16:10:02.842902+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298920,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072920,"statfs":{"total":96636764160,"available":96330670080,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.43099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.48199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.505}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.58199999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.38600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574918,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.56999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.48199999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.219}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.28199999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.312},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.27000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303016,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068824,"statfs":{"total":96636764160,"available":96326475776,"internally_reserved":0,"allocated":1671168,"data_stored":1051604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.313},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.29899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.378},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.34999999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.505},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.58799999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.57299999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.623}]}]},{"osd":2,"up_from":13,"seq":55834574918,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.28499999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.314},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.34899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.628},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.81499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574918,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302436,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069404,"statfs":{"total":96636764160,"available":96327069696,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.505},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.375},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.34899999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.46700000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.495}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.432}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.623}]}]},{"osd":4,"up_from":13,"seq":55834574919,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298340,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073500,"statfs":{"total":96636764160,"available":96331264000,"internally_reserved":0,"allocated":1077248,"data_stored":461236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63400000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298328,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073512,"statfs":{"total":96636764160,"available":96331276288,"internally_reserved":0,"allocated":1064960,"data_stored":454973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.503},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.46200000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.41599999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.439},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.624}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.58499999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64200000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:09.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-10-08T16:10:09.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.846576+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.855906+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.845596+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847904+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849710+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-10-08T16:10:09.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849731+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-10-08T16:10:09.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849180+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-10-08T16:10:09.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848444+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854318+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847899+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851444+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854692+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-10-08T16:10:09.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847882+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851768+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:34:03.864214+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848862+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-10-08T16:10:09.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.852017+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-10-08T16:10:09.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848718+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-10-08T16:10:09.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848600+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850460+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:03:08.970273+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854164+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851237+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:45:55.867665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854404+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847417+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851540+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849101+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854806+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850963+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848987+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848807+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851334+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:28:34.817905+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848424+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-10-08T16:10:09.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.845597+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.852366+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:19:39.030916+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848401+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.845567+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.852544+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849013+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850231+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847828+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.853269+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-10-08T16:10:09.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.854417+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.853636+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848490+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850485+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847341+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-10-08T16:10:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.844997+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:04:50.947553+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851825+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851551+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:43:21.793105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848684+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-10-08T16:10:09.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848654+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850001+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849640+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:57:50.717870+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850795+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:45:16.625454+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850693+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:55:15.457966+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-10-08T16:10:09.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.847733+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.855191+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.844983+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:47:59.285201+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.845004+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:34:34.467877+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848039+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:57:34.478077+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.656 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.849203+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:05:59.761610+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.851897+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850225+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:07:54.252446+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.848298+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:44:09.653631+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-10-08T16:10:09.657 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:03.838041+0000', 'last_change': '2023-10-08T16:10:03.850391+0000', 'last_active': '2023-10-08T16:10:03.838041+0000', 'last_peered': '2023-10-08T16:10:03.838041+0000', 'last_clean': '2023-10-08T16:10:03.838041+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:03.838041+0000', 'last_undegraded': '2023-10-08T16:10:03.838041+0000', 'last_fullsized': '2023-10-08T16:10:03.838041+0000', 'mapping_epoch': 87, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 87, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:03.838041+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:37:03.296819+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:09.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T16:10:09.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.792+0000 7f255c8c4700 1 Processor -- start 2023-10-08T16:10:09.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.792+0000 7f255c8c4700 1 -- start start 2023-10-08T16:10:09.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255c8c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f25580cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255c8c4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f255815cd70 0x7f255815d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255c8c4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25580cfa60 con 0x7f25580ce900 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255c8c4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25580cee50 con 0x7f255815cd70 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255c8c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25580cf130 con 0x7f255815c9a0 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f254e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f25580ce900 0x7f25580cb750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45604/0 (socket says 172.21.15.170:45604) 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f254e59c700 1 -- 172.21.15.170:0/145039861 learned_addr learned my addr 172.21.15.170:0/145039861 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f2556d9d700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f25580cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203407606 0 0) 0x7f25580cfa60 con 0x7f25580ce900 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255759e700 1 --2- 172.21.15.170:0/145039861 >> v2:172.21.15.191:3300/0 conn(0x7f255815cd70 0x7f255815d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.793+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2544003570 con 0x7f25580ce900 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206497882 0 0) 0x7f2544003570 con 0x7f25580ce900 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 msgr2=0x7f25580cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f25580cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 >> v2:172.21.15.191:3300/0 conn(0x7f255815cd70 msgr2=0x7f255815d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 --2- 172.21.15.170:0/145039861 >> v2:172.21.15.191:3300/0 conn(0x7f255815cd70 0x7f255815d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25580cf410 con 0x7f25580ce900 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f2540001470 con 0x7f25580ce900 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 >> v1:172.21.15.170:6789/0 conn(0x7f25580ce900 legacy=0x7f25580cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f2544003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2544003fe0 0x7f2544004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2544004d50 0x7f25440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25580cee50 con 0x7f2544004d50 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25580cf130 con 0x7f2544003fe0 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254400ba90 con 0x7f255815c9a0 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2544004d50 0x7f25440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f2556d9d700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f2544003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f254e59c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2544003fe0 0x7f2544004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 -- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 msgr2=0x7f2544003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f255815c9a0 0x7f2544003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 -- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2544003fe0 msgr2=0x7f2544004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:09.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2544003fe0 0x7f2544004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 -- 172.21.15.170:0/145039861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544003570 con 0x7f2544004d50 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f254e59c700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2544003fe0 0x7f2544004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255759e700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2544004d50 0x7f25440051c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f255000e210 tx=0x7f255000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654555a340deda3d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000ea60 con 0x7f2544004d50 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255000f040 con 0x7f2544004d50 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.794+0000 7f255659c700 1 -- 172.21.15.170:0/145039861 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000ee10 con 0x7f2544004d50 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 -- 172.21.15.170:0/145039861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2544004d50 msgr2=0x7f25440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 --2- 172.21.15.170:0/145039861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2544004d50 0x7f25440051c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f255000e210 tx=0x7f255000b040 comp rx=0 tx=0).stop 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255759e700 1 -- 172.21.15.170:0/145039861 reap_dead start 2023-10-08T16:10:09.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 -- 172.21.15.170:0/145039861 shutdown_connections 2023-10-08T16:10:09.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 -- 172.21.15.170:0/145039861 >> 172.21.15.170:0/145039861 conn(0x7f25580d05c0 msgr2=0x7f25580d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:09.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 -- 172.21.15.170:0/145039861 shutdown_connections 2023-10-08T16:10:09.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.795+0000 7f255c8c4700 1 -- 172.21.15.170:0/145039861 wait complete. 2023-10-08T16:10:09.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.796+0000 7f255c8c4700 1 Processor -- start 2023-10-08T16:10:09.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.796+0000 7f255c8c4700 1 -- start start 2023-10-08T16:10:09.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2544004d50 0x7f2558141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 0x7f2558156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25580c8970 con 0x7f25580c94a0 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2558157230 con 0x7f2558156880 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255c8c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25580b4640 con 0x7f2544004d50 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49476/0 (socket says 172.21.15.170:49476) 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 -- 172.21.15.170:0/218141262 learned_addr learned my addr 172.21.15.170:0/218141262 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f2556d9d700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2544004d50 0x7f2558141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f254e59c700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 0x7f2558156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 -- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2544004d50 msgr2=0x7f2558141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2544004d50 0x7f2558141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 -- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 msgr2=0x7f2558156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 0x7f2558156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 -- 172.21.15.170:0/218141262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558141c60 con 0x7f25580c94a0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f254e59c700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 0x7f2558156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.797+0000 7f255759e700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2544004730 tx=0x7f255001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.798+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255001e640 con 0x7f25580c94a0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.798+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255001e7a0 con 0x7f25580c94a0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.798+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25580b3a30 con 0x7f25580c94a0 2023-10-08T16:10:09.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.798+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255001eb60 con 0x7f25580c94a0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.798+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25580b3ff0 con 0x7f25580c94a0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.799+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25580b47d0 con 0x7f25580c94a0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.799+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2550024070 con 0x7f25580c94a0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.799+0000 7f254f7fe700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 0x7f25380696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.799+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25500870f0 con 0x7f25580c94a0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.799+0000 7f2556d9d700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 0x7f25380696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.800+0000 7f2556d9d700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 0x7f25380696b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2548006f00 tx=0x7f25480091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:09.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:09.803+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25500540c0 con 0x7f25580c94a0 2023-10-08T16:10:10.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.062+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f25580cf6f0 con 0x7f25580c94a0 2023-10-08T16:10:10.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.063+0000 7f254f7fe700 1 -- 172.21.15.170:0/218141262 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1455 (secure 0 0 0) 0x7f25500576e0 con 0x7f25580c94a0 2023-10-08T16:10:10.064 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:10.064 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":351,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":89,"num_osds":8,"num_up_osds":8,"osd_up_since":1696781068,"num_in_osds":8,"osd_in_since":1696781061,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2461941760,"bytes_avail":770632171520,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":56,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":6393}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2023-10-08T16:09:54.290319+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T16:10:10.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 msgr2=0x7f25380696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 0x7f25380696b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2548006f00 tx=0x7f25480091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 msgr2=0x7f25580c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2544004730 tx=0x7f255001a000 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 shutdown_connections 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2538067200 0x7f25380696b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25580c94a0 0x7f25580c9910 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2544004d50 0x7f2558141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 --2- 172.21.15.170:0/218141262 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2558156880 0x7f2558156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 >> 172.21.15.170:0/218141262 conn(0x7f25580d05c0 msgr2=0x7f2558068840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 shutdown_connections 2023-10-08T16:10:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.065+0000 7f255c8c4700 1 -- 172.21.15.170:0/218141262 wait complete. 2023-10-08T16:10:10.077 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T16:10:10.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:10.155 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:10:10.155+0000 7f22f2fd6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:10:10.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.192+0000 7ffad2b88700 1 Processor -- start 2023-10-08T16:10:10.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.192+0000 7ffad2b88700 1 -- start start 2023-10-08T16:10:10.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2b88700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ffacc159420 0x7ffacc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2b88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 0x7ffacc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2b88700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffacc0b11d0 con 0x7ffacc15d230 2023-10-08T16:10:10.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2b88700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffacc0b14b0 con 0x7ffacc159420 2023-10-08T16:10:10.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2b88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc0b1790 con 0x7ffacc1597f0 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2387700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ffacc15d230 0x7ffacc0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45618/0 (socket says 172.21.15.170:45618) 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad2387700 1 -- 172.21.15.170:0/569923430 learned_addr learned my addr 172.21.15.170:0/569923430 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1b86700 1 --2- 172.21.15.170:0/569923430 >> v2:172.21.15.191:3300/0 conn(0x7ffacc159420 0x7ffacc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1385700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 0x7ffacc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734957974 0 0) 0x7ffacc0b11d0 con 0x7ffacc15d230 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab8003570 con 0x7ffacc15d230 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1b86700 1 -- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 msgr2=0x7ffacc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1b86700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 0x7ffacc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1b86700 1 -- 172.21.15.170:0/569923430 >> v1:172.21.15.170:6789/0 conn(0x7ffacc15d230 legacy=0x7ffacc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545953717 0 0) 0x7ffab8003570 con 0x7ffacc15d230 2023-10-08T16:10:10.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1b86700 1 -- 172.21.15.170:0/569923430 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffacc0b1a70 con 0x7ffacc159420 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.193+0000 7ffad1385700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 0x7ffacc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad1b86700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc0b5f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffac40098e0 tx=0x7ffac40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ee819cf1452167 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80017f0 con 0x7ffacc159420 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4011420 con 0x7ffacc159420 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad0b84700 1 -- 172.21.15.170:0/569923430 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac40117c0 con 0x7ffacc159420 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 -- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 msgr2=0x7ffacc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc0b5f40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffac40098e0 tx=0x7ffac40091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 -- 172.21.15.170:0/569923430 shutdown_connections 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc0b5f40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 --2- 172.21.15.170:0/569923430 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc1597f0 0x7ffacc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.194+0000 7ffad2b88700 1 -- 172.21.15.170:0/569923430 >> 172.21.15.170:0/569923430 conn(0x7ffacc0d0660 msgr2=0x7ffacc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:10.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.195+0000 7ffad2b88700 1 -- 172.21.15.170:0/569923430 shutdown_connections 2023-10-08T16:10:10.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.195+0000 7ffad2b88700 1 -- 172.21.15.170:0/569923430 wait complete. 2023-10-08T16:10:10.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 Processor -- start 2023-10-08T16:10:10.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 -- start start 2023-10-08T16:10:10.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15d230 0x7ffacc169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc15a910 con 0x7ffacc1597f0 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc159d00 con 0x7ffacc159420 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad2b88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffacc159fe0 con 0x7ffacc15d230 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad1385700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad1385700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49484/0 (socket says 172.21.15.170:49484) 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.196+0000 7ffad1385700 1 -- 172.21.15.170:0/3473702198 learned_addr learned my addr 172.21.15.170:0/3473702198 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad2387700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15d230 0x7ffacc169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1b86700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 -- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15d230 msgr2=0x7ffacc169ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15d230 0x7ffacc169ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 -- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 msgr2=0x7ffacc156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 -- 172.21.15.170:0/3473702198 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc009420 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad1385700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00ec80 tx=0x7ffabc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00c920 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffabc00ca80 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffabc00ce00 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac4009420 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.197+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffacc15a5a0 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.198+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffacc04c3c0 con 0x7ffacc1597f0 2023-10-08T16:10:10.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.198+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7ffabc013680 con 0x7ffacc1597f0 2023-10-08T16:10:10.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.199+0000 7ffac1ffb700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 0x7ffab00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:10.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.199+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ffabc084d60 con 0x7ffacc1597f0 2023-10-08T16:10:10.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.199+0000 7ffad1b86700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 0x7ffab00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:10.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.199+0000 7ffad1b86700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 0x7ffab00696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffac40093f0 tx=0x7ffac4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:10.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.203+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffabc051c80 con 0x7ffacc1597f0 2023-10-08T16:10:10.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.385+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffacc15a150 con 0x7ffab0067200 2023-10-08T16:10:10.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.393+0000 7ffac1ffb700 1 -- 172.21.15.170:0/3473702198 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289316 (secure 0 0 0) 0x7ffacc15a150 con 0x7ffab0067200 2023-10-08T16:10:10.396 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 msgr2=0x7ffab00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 0x7ffab00696b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffac40093f0 tx=0x7ffac4016000 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 msgr2=0x7ffacc0c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00ec80 tx=0x7ffabc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 shutdown_connections 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ffacc15d230 0x7ffacc169ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ffacc159420 0x7ffacc156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ffab0067200 0x7ffab00696b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 --2- 172.21.15.170:0/3473702198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffacc1597f0 0x7ffacc0c5dc0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:10.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.396+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 >> 172.21.15.170:0/3473702198 conn(0x7ffacc0d0660 msgr2=0x7ffacc0b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:10.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.397+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 shutdown_connections 2023-10-08T16:10:10.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:10.397+0000 7ffad2b88700 1 -- 172.21.15.170:0/3473702198 wait complete. 2023-10-08T16:10:10.400 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:10.409 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2023-10-08T16:10:10.293058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2404520,"kb_used_data":9832,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752570200,"statfs":{"total":773094113280,"available":770631884800,"internally_reserved":0,"allocated":10067968,"data_stored":4979644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.450525"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859739+0000","last_change":"2023-10-08T16:10:04.858512+0000","last_active":"2023-10-08T16:10:04.859739+0000","last_peered":"2023-10-08T16:10:04.859739+0000","last_clean":"2023-10-08T16:10:04.859739+0000","last_became_active":"2023-10-08T16:10:04.856420+0000","last_became_peered":"2023-10-08T16:10:04.856420+0000","last_unstale":"2023-10-08T16:10:04.859739+0000","last_undegraded":"2023-10-08T16:10:04.859739+0000","last_fullsized":"2023-10-08T16:10:04.859739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871412+0000","last_change":"2023-10-08T16:10:04.867644+0000","last_active":"2023-10-08T16:10:04.871412+0000","last_peered":"2023-10-08T16:10:04.871412+0000","last_clean":"2023-10-08T16:10:04.871412+0000","last_became_active":"2023-10-08T16:10:04.866553+0000","last_became_peered":"2023-10-08T16:10:04.866553+0000","last_unstale":"2023-10-08T16:10:04.871412+0000","last_undegraded":"2023-10-08T16:10:04.871412+0000","last_fullsized":"2023-10-08T16:10:04.871412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:43.947260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859777+0000","last_change":"2023-10-08T16:10:04.857925+0000","last_active":"2023-10-08T16:10:04.859777+0000","last_peered":"2023-10-08T16:10:04.859777+0000","last_clean":"2023-10-08T16:10:04.859777+0000","last_became_active":"2023-10-08T16:10:04.857660+0000","last_became_peered":"2023-10-08T16:10:04.857660+0000","last_unstale":"2023-10-08T16:10:04.859777+0000","last_undegraded":"2023-10-08T16:10:04.859777+0000","last_fullsized":"2023-10-08T16:10:04.859777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857603+0000","last_change":"2023-10-08T16:10:04.855606+0000","last_active":"2023-10-08T16:10:04.857603+0000","last_peered":"2023-10-08T16:10:04.857603+0000","last_clean":"2023-10-08T16:10:04.857603+0000","last_became_active":"2023-10-08T16:10:04.855157+0000","last_became_peered":"2023-10-08T16:10:04.855157+0000","last_unstale":"2023-10-08T16:10:04.857603+0000","last_undegraded":"2023-10-08T16:10:04.857603+0000","last_fullsized":"2023-10-08T16:10:04.857603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872824+0000","last_change":"2023-10-08T16:10:04.863817+0000","last_active":"2023-10-08T16:10:04.872824+0000","last_peered":"2023-10-08T16:10:04.872824+0000","last_clean":"2023-10-08T16:10:04.872824+0000","last_became_active":"2023-10-08T16:10:04.863540+0000","last_became_peered":"2023-10-08T16:10:04.863540+0000","last_unstale":"2023-10-08T16:10:04.872824+0000","last_undegraded":"2023-10-08T16:10:04.872824+0000","last_fullsized":"2023-10-08T16:10:04.872824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876390+0000","last_change":"2023-10-08T16:10:04.866534+0000","last_active":"2023-10-08T16:10:04.876390+0000","last_peered":"2023-10-08T16:10:04.876390+0000","last_clean":"2023-10-08T16:10:04.876390+0000","last_became_active":"2023-10-08T16:10:04.866237+0000","last_became_peered":"2023-10-08T16:10:04.866237+0000","last_unstale":"2023-10-08T16:10:04.876390+0000","last_undegraded":"2023-10-08T16:10:04.876390+0000","last_fullsized":"2023-10-08T16:10:04.876390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:34.800715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876461+0000","last_change":"2023-10-08T16:10:04.867705+0000","last_active":"2023-10-08T16:10:04.876461+0000","last_peered":"2023-10-08T16:10:04.876461+0000","last_clean":"2023-10-08T16:10:04.876461+0000","last_became_active":"2023-10-08T16:10:04.867436+0000","last_became_peered":"2023-10-08T16:10:04.867436+0000","last_unstale":"2023-10-08T16:10:04.876461+0000","last_undegraded":"2023-10-08T16:10:04.876461+0000","last_fullsized":"2023-10-08T16:10:04.876461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861635+0000","last_change":"2023-10-08T16:10:04.860107+0000","last_active":"2023-10-08T16:10:04.861635+0000","last_peered":"2023-10-08T16:10:04.861635+0000","last_clean":"2023-10-08T16:10:04.861635+0000","last_became_active":"2023-10-08T16:10:04.859658+0000","last_became_peered":"2023-10-08T16:10:04.859658+0000","last_unstale":"2023-10-08T16:10:04.861635+0000","last_undegraded":"2023-10-08T16:10:04.861635+0000","last_fullsized":"2023-10-08T16:10:04.861635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:30.911532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871687+0000","last_change":"2023-10-08T16:10:04.862038+0000","last_active":"2023-10-08T16:10:04.871687+0000","last_peered":"2023-10-08T16:10:04.871687+0000","last_clean":"2023-10-08T16:10:04.871687+0000","last_became_active":"2023-10-08T16:10:04.861746+0000","last_became_peered":"2023-10-08T16:10:04.861746+0000","last_unstale":"2023-10-08T16:10:04.871687+0000","last_undegraded":"2023-10-08T16:10:04.871687+0000","last_fullsized":"2023-10-08T16:10:04.871687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875189+0000","last_change":"2023-10-08T16:10:04.868752+0000","last_active":"2023-10-08T16:10:04.875189+0000","last_peered":"2023-10-08T16:10:04.875189+0000","last_clean":"2023-10-08T16:10:04.875189+0000","last_became_active":"2023-10-08T16:10:04.864408+0000","last_became_peered":"2023-10-08T16:10:04.864408+0000","last_unstale":"2023-10-08T16:10:04.875189+0000","last_undegraded":"2023-10-08T16:10:04.875189+0000","last_fullsized":"2023-10-08T16:10:04.875189+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:32.803419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876033+0000","last_change":"2023-10-08T16:10:04.865163+0000","last_active":"2023-10-08T16:10:04.876033+0000","last_peered":"2023-10-08T16:10:04.876033+0000","last_clean":"2023-10-08T16:10:04.876033+0000","last_became_active":"2023-10-08T16:10:04.864821+0000","last_became_peered":"2023-10-08T16:10:04.864821+0000","last_unstale":"2023-10-08T16:10:04.876033+0000","last_undegraded":"2023-10-08T16:10:04.876033+0000","last_fullsized":"2023-10-08T16:10:04.876033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:05.440731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876415+0000","last_change":"2023-10-08T16:10:04.872779+0000","last_active":"2023-10-08T16:10:04.876415+0000","last_peered":"2023-10-08T16:10:04.876415+0000","last_clean":"2023-10-08T16:10:04.876415+0000","last_became_active":"2023-10-08T16:10:04.872079+0000","last_became_peered":"2023-10-08T16:10:04.872079+0000","last_unstale":"2023-10-08T16:10:04.876415+0000","last_undegraded":"2023-10-08T16:10:04.876415+0000","last_fullsized":"2023-10-08T16:10:04.876415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.863134+0000","last_change":"2023-10-08T16:10:04.861826+0000","last_active":"2023-10-08T16:10:04.863134+0000","last_peered":"2023-10-08T16:10:04.863134+0000","last_clean":"2023-10-08T16:10:04.863134+0000","last_became_active":"2023-10-08T16:10:04.861579+0000","last_became_peered":"2023-10-08T16:10:04.861579+0000","last_unstale":"2023-10-08T16:10:04.863134+0000","last_undegraded":"2023-10-08T16:10:04.863134+0000","last_fullsized":"2023-10-08T16:10:04.863134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865558+0000","last_change":"2023-10-08T16:10:04.864671+0000","last_active":"2023-10-08T16:10:04.865558+0000","last_peered":"2023-10-08T16:10:04.865558+0000","last_clean":"2023-10-08T16:10:04.865558+0000","last_became_active":"2023-10-08T16:10:04.864414+0000","last_became_peered":"2023-10-08T16:10:04.864414+0000","last_unstale":"2023-10-08T16:10:04.865558+0000","last_undegraded":"2023-10-08T16:10:04.865558+0000","last_fullsized":"2023-10-08T16:10:04.865558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:34.143021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.856924+0000","last_change":"2023-10-08T16:10:04.854692+0000","last_active":"2023-10-08T16:10:04.856924+0000","last_peered":"2023-10-08T16:10:04.856924+0000","last_clean":"2023-10-08T16:10:04.856924+0000","last_became_active":"2023-10-08T16:10:04.854453+0000","last_became_peered":"2023-10-08T16:10:04.854453+0000","last_unstale":"2023-10-08T16:10:04.856924+0000","last_undegraded":"2023-10-08T16:10:04.856924+0000","last_fullsized":"2023-10-08T16:10:04.856924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.863818+0000","last_change":"2023-10-08T16:10:04.858600+0000","last_active":"2023-10-08T16:10:04.863818+0000","last_peered":"2023-10-08T16:10:04.863818+0000","last_clean":"2023-10-08T16:10:04.863818+0000","last_became_active":"2023-10-08T16:10:04.858320+0000","last_became_peered":"2023-10-08T16:10:04.858320+0000","last_unstale":"2023-10-08T16:10:04.863818+0000","last_undegraded":"2023-10-08T16:10:04.863818+0000","last_fullsized":"2023-10-08T16:10:04.863818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:30.911532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862428+0000","last_change":"2023-10-08T16:10:04.859672+0000","last_active":"2023-10-08T16:10:04.862428+0000","last_peered":"2023-10-08T16:10:04.862428+0000","last_clean":"2023-10-08T16:10:04.862428+0000","last_became_active":"2023-10-08T16:10:04.859413+0000","last_became_peered":"2023-10-08T16:10:04.859413+0000","last_unstale":"2023-10-08T16:10:04.862428+0000","last_undegraded":"2023-10-08T16:10:04.862428+0000","last_fullsized":"2023-10-08T16:10:04.862428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865091+0000","last_change":"2023-10-08T16:10:04.864133+0000","last_active":"2023-10-08T16:10:04.865091+0000","last_peered":"2023-10-08T16:10:04.865091+0000","last_clean":"2023-10-08T16:10:04.865091+0000","last_became_active":"2023-10-08T16:10:04.863618+0000","last_became_peered":"2023-10-08T16:10:04.863618+0000","last_unstale":"2023-10-08T16:10:04.865091+0000","last_undegraded":"2023-10-08T16:10:04.865091+0000","last_fullsized":"2023-10-08T16:10:04.865091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854508+0000","last_change":"2023-10-08T16:10:04.850440+0000","last_active":"2023-10-08T16:10:04.854508+0000","last_peered":"2023-10-08T16:10:04.854508+0000","last_clean":"2023-10-08T16:10:04.854508+0000","last_became_active":"2023-10-08T16:10:04.850132+0000","last_became_peered":"2023-10-08T16:10:04.850132+0000","last_unstale":"2023-10-08T16:10:04.854508+0000","last_undegraded":"2023-10-08T16:10:04.854508+0000","last_fullsized":"2023-10-08T16:10:04.854508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:18.809308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871864+0000","last_change":"2023-10-08T16:10:04.862675+0000","last_active":"2023-10-08T16:10:04.871864+0000","last_peered":"2023-10-08T16:10:04.871864+0000","last_clean":"2023-10-08T16:10:04.871864+0000","last_became_active":"2023-10-08T16:10:04.862402+0000","last_became_peered":"2023-10-08T16:10:04.862402+0000","last_unstale":"2023-10-08T16:10:04.871864+0000","last_undegraded":"2023-10-08T16:10:04.871864+0000","last_fullsized":"2023-10-08T16:10:04.871864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865931+0000","last_change":"2023-10-08T16:10:04.864596+0000","last_active":"2023-10-08T16:10:04.865931+0000","last_peered":"2023-10-08T16:10:04.865931+0000","last_clean":"2023-10-08T16:10:04.865931+0000","last_became_active":"2023-10-08T16:10:04.863904+0000","last_became_peered":"2023-10-08T16:10:04.863904+0000","last_unstale":"2023-10-08T16:10:04.865931+0000","last_undegraded":"2023-10-08T16:10:04.865931+0000","last_fullsized":"2023-10-08T16:10:04.865931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:31.607081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.868741+0000","last_change":"2023-10-08T16:10:04.866160+0000","last_active":"2023-10-08T16:10:04.868741+0000","last_peered":"2023-10-08T16:10:04.868741+0000","last_clean":"2023-10-08T16:10:04.868741+0000","last_became_active":"2023-10-08T16:10:04.865846+0000","last_became_peered":"2023-10-08T16:10:04.865846+0000","last_unstale":"2023-10-08T16:10:04.868741+0000","last_undegraded":"2023-10-08T16:10:04.868741+0000","last_fullsized":"2023-10-08T16:10:04.868741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:05.440731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.868980+0000","last_change":"2023-10-08T16:10:04.866215+0000","last_active":"2023-10-08T16:10:04.868980+0000","last_peered":"2023-10-08T16:10:04.868980+0000","last_clean":"2023-10-08T16:10:04.868980+0000","last_became_active":"2023-10-08T16:10:04.865951+0000","last_became_peered":"2023-10-08T16:10:04.865951+0000","last_unstale":"2023-10-08T16:10:04.868980+0000","last_undegraded":"2023-10-08T16:10:04.868980+0000","last_fullsized":"2023-10-08T16:10:04.868980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:29.068707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866022+0000","last_change":"2023-10-08T16:10:04.860722+0000","last_active":"2023-10-08T16:10:04.866022+0000","last_peered":"2023-10-08T16:10:04.866022+0000","last_clean":"2023-10-08T16:10:04.866022+0000","last_became_active":"2023-10-08T16:10:04.860474+0000","last_became_peered":"2023-10-08T16:10:04.860474+0000","last_unstale":"2023-10-08T16:10:04.866022+0000","last_undegraded":"2023-10-08T16:10:04.866022+0000","last_fullsized":"2023-10-08T16:10:04.866022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:29.068707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874717+0000","last_change":"2023-10-08T16:10:04.867406+0000","last_active":"2023-10-08T16:10:04.874717+0000","last_peered":"2023-10-08T16:10:04.874717+0000","last_clean":"2023-10-08T16:10:04.874717+0000","last_became_active":"2023-10-08T16:10:04.867081+0000","last_became_peered":"2023-10-08T16:10:04.867081+0000","last_unstale":"2023-10-08T16:10:04.874717+0000","last_undegraded":"2023-10-08T16:10:04.874717+0000","last_fullsized":"2023-10-08T16:10:04.874717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862587+0000","last_change":"2023-10-08T16:10:04.861206+0000","last_active":"2023-10-08T16:10:04.862587+0000","last_peered":"2023-10-08T16:10:04.862587+0000","last_clean":"2023-10-08T16:10:04.862587+0000","last_became_active":"2023-10-08T16:10:04.860961+0000","last_became_peered":"2023-10-08T16:10:04.860961+0000","last_unstale":"2023-10-08T16:10:04.862587+0000","last_undegraded":"2023-10-08T16:10:04.862587+0000","last_fullsized":"2023-10-08T16:10:04.862587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:34.800715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876686+0000","last_change":"2023-10-08T16:10:04.871483+0000","last_active":"2023-10-08T16:10:04.876686+0000","last_peered":"2023-10-08T16:10:04.876686+0000","last_clean":"2023-10-08T16:10:04.876686+0000","last_became_active":"2023-10-08T16:10:04.870548+0000","last_became_peered":"2023-10-08T16:10:04.870548+0000","last_unstale":"2023-10-08T16:10:04.876686+0000","last_undegraded":"2023-10-08T16:10:04.876686+0000","last_fullsized":"2023-10-08T16:10:04.876686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875323+0000","last_change":"2023-10-08T16:10:04.868243+0000","last_active":"2023-10-08T16:10:04.875323+0000","last_peered":"2023-10-08T16:10:04.875323+0000","last_clean":"2023-10-08T16:10:04.875323+0000","last_became_active":"2023-10-08T16:10:04.867970+0000","last_became_peered":"2023-10-08T16:10:04.867970+0000","last_unstale":"2023-10-08T16:10:04.875323+0000","last_undegraded":"2023-10-08T16:10:04.875323+0000","last_fullsized":"2023-10-08T16:10:04.875323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.864291+0000","last_change":"2023-10-08T16:10:04.863101+0000","last_active":"2023-10-08T16:10:04.864291+0000","last_peered":"2023-10-08T16:10:04.864291+0000","last_clean":"2023-10-08T16:10:04.864291+0000","last_became_active":"2023-10-08T16:10:04.862827+0000","last_became_peered":"2023-10-08T16:10:04.862827+0000","last_unstale":"2023-10-08T16:10:04.864291+0000","last_undegraded":"2023-10-08T16:10:04.864291+0000","last_fullsized":"2023-10-08T16:10:04.864291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:42.557884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854754+0000","last_change":"2023-10-08T16:10:04.850190+0000","last_active":"2023-10-08T16:10:04.854754+0000","last_peered":"2023-10-08T16:10:04.854754+0000","last_clean":"2023-10-08T16:10:04.854754+0000","last_became_active":"2023-10-08T16:10:04.849935+0000","last_became_peered":"2023-10-08T16:10:04.849935+0000","last_unstale":"2023-10-08T16:10:04.854754+0000","last_undegraded":"2023-10-08T16:10:04.854754+0000","last_fullsized":"2023-10-08T16:10:04.854754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:23.797032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871369+0000","last_change":"2023-10-08T16:10:04.870436+0000","last_active":"2023-10-08T16:10:04.871369+0000","last_peered":"2023-10-08T16:10:04.871369+0000","last_clean":"2023-10-08T16:10:04.871369+0000","last_became_active":"2023-10-08T16:10:04.870041+0000","last_became_peered":"2023-10-08T16:10:04.870041+0000","last_unstale":"2023-10-08T16:10:04.871369+0000","last_undegraded":"2023-10-08T16:10:04.871369+0000","last_fullsized":"2023-10-08T16:10:04.871369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859692+0000","last_change":"2023-10-08T16:10:04.858462+0000","last_active":"2023-10-08T16:10:04.859692+0000","last_peered":"2023-10-08T16:10:04.859692+0000","last_clean":"2023-10-08T16:10:04.859692+0000","last_became_active":"2023-10-08T16:10:04.858207+0000","last_became_peered":"2023-10-08T16:10:04.858207+0000","last_unstale":"2023-10-08T16:10:04.859692+0000","last_undegraded":"2023-10-08T16:10:04.859692+0000","last_fullsized":"2023-10-08T16:10:04.859692+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865239+0000","last_change":"2023-10-08T16:10:04.862333+0000","last_active":"2023-10-08T16:10:04.865239+0000","last_peered":"2023-10-08T16:10:04.865239+0000","last_clean":"2023-10-08T16:10:04.865239+0000","last_became_active":"2023-10-08T16:10:04.862034+0000","last_became_peered":"2023-10-08T16:10:04.862034+0000","last_unstale":"2023-10-08T16:10:04.865239+0000","last_undegraded":"2023-10-08T16:10:04.865239+0000","last_fullsized":"2023-10-08T16:10:04.865239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:18.078100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"88'4","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875881+0000","last_change":"2023-10-08T16:10:03.859417+0000","last_active":"2023-10-08T16:10:04.875881+0000","last_peered":"2023-10-08T16:10:04.875881+0000","last_clean":"2023-10-08T16:10:04.875881+0000","last_became_active":"2023-10-08T16:10:03.859153+0000","last_became_peered":"2023-10-08T16:10:03.859153+0000","last_unstale":"2023-10-08T16:10:04.875881+0000","last_undegraded":"2023-10-08T16:10:04.875881+0000","last_fullsized":"2023-10-08T16:10:04.875881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871374+0000","last_change":"2023-10-08T16:10:04.870602+0000","last_active":"2023-10-08T16:10:04.871374+0000","last_peered":"2023-10-08T16:10:04.871374+0000","last_clean":"2023-10-08T16:10:04.871374+0000","last_became_active":"2023-10-08T16:10:04.869867+0000","last_became_peered":"2023-10-08T16:10:04.869867+0000","last_unstale":"2023-10-08T16:10:04.871374+0000","last_undegraded":"2023-10-08T16:10:04.871374+0000","last_fullsized":"2023-10-08T16:10:04.871374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876470+0000","last_change":"2023-10-08T16:10:03.859490+0000","last_active":"2023-10-08T16:10:04.876470+0000","last_peered":"2023-10-08T16:10:04.876470+0000","last_clean":"2023-10-08T16:10:04.876470+0000","last_became_active":"2023-10-08T16:10:03.859218+0000","last_became_peered":"2023-10-08T16:10:03.859218+0000","last_unstale":"2023-10-08T16:10:04.876470+0000","last_undegraded":"2023-10-08T16:10:04.876470+0000","last_fullsized":"2023-10-08T16:10:04.876470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.864748+0000","last_change":"2023-10-08T16:10:04.863769+0000","last_active":"2023-10-08T16:10:04.864748+0000","last_peered":"2023-10-08T16:10:04.864748+0000","last_clean":"2023-10-08T16:10:04.864748+0000","last_became_active":"2023-10-08T16:10:04.863514+0000","last_became_peered":"2023-10-08T16:10:04.863514+0000","last_unstale":"2023-10-08T16:10:04.864748+0000","last_undegraded":"2023-10-08T16:10:04.864748+0000","last_fullsized":"2023-10-08T16:10:04.864748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871116+0000","last_change":"2023-10-08T16:10:03.862941+0000","last_active":"2023-10-08T16:10:04.871116+0000","last_peered":"2023-10-08T16:10:04.871116+0000","last_clean":"2023-10-08T16:10:04.871116+0000","last_became_active":"2023-10-08T16:10:03.862732+0000","last_became_peered":"2023-10-08T16:10:03.862732+0000","last_unstale":"2023-10-08T16:10:04.871116+0000","last_undegraded":"2023-10-08T16:10:04.871116+0000","last_fullsized":"2023-10-08T16:10:04.871116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875810+0000","last_change":"2023-10-08T16:10:04.868792+0000","last_active":"2023-10-08T16:10:04.875810+0000","last_peered":"2023-10-08T16:10:04.875810+0000","last_clean":"2023-10-08T16:10:04.875810+0000","last_became_active":"2023-10-08T16:10:04.868523+0000","last_became_peered":"2023-10-08T16:10:04.868523+0000","last_unstale":"2023-10-08T16:10:04.875810+0000","last_undegraded":"2023-10-08T16:10:04.875810+0000","last_fullsized":"2023-10-08T16:10:04.875810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876867+0000","last_change":"2023-10-08T16:10:03.862782+0000","last_active":"2023-10-08T16:10:04.876867+0000","last_peered":"2023-10-08T16:10:04.876867+0000","last_clean":"2023-10-08T16:10:04.876867+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:04.876867+0000","last_undegraded":"2023-10-08T16:10:04.876867+0000","last_fullsized":"2023-10-08T16:10:04.876867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:13.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866867+0000","last_change":"2023-10-08T16:10:04.862896+0000","last_active":"2023-10-08T16:10:04.866867+0000","last_peered":"2023-10-08T16:10:04.866867+0000","last_clean":"2023-10-08T16:10:04.866867+0000","last_became_active":"2023-10-08T16:10:04.862642+0000","last_became_peered":"2023-10-08T16:10:04.862642+0000","last_unstale":"2023-10-08T16:10:04.866867+0000","last_undegraded":"2023-10-08T16:10:04.866867+0000","last_fullsized":"2023-10-08T16:10:04.866867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876387+0000","last_change":"2023-10-08T16:10:03.857135+0000","last_active":"2023-10-08T16:10:04.876387+0000","last_peered":"2023-10-08T16:10:04.876387+0000","last_clean":"2023-10-08T16:10:04.876387+0000","last_became_active":"2023-10-08T16:10:03.856911+0000","last_became_peered":"2023-10-08T16:10:03.856911+0000","last_unstale":"2023-10-08T16:10:04.876387+0000","last_undegraded":"2023-10-08T16:10:04.876387+0000","last_fullsized":"2023-10-08T16:10:04.876387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876738+0000","last_change":"2023-10-08T16:10:04.872848+0000","last_active":"2023-10-08T16:10:04.876738+0000","last_peered":"2023-10-08T16:10:04.876738+0000","last_clean":"2023-10-08T16:10:04.876738+0000","last_became_active":"2023-10-08T16:10:04.872593+0000","last_became_peered":"2023-10-08T16:10:04.872593+0000","last_unstale":"2023-10-08T16:10:04.876738+0000","last_undegraded":"2023-10-08T16:10:04.876738+0000","last_fullsized":"2023-10-08T16:10:04.876738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:21.771026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872486+0000","last_change":"2023-10-08T16:10:03.859382+0000","last_active":"2023-10-08T16:10:04.872486+0000","last_peered":"2023-10-08T16:10:04.872486+0000","last_clean":"2023-10-08T16:10:04.872486+0000","last_became_active":"2023-10-08T16:10:03.859137+0000","last_became_peered":"2023-10-08T16:10:03.859137+0000","last_unstale":"2023-10-08T16:10:04.872486+0000","last_undegraded":"2023-10-08T16:10:04.872486+0000","last_fullsized":"2023-10-08T16:10:04.872486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861516+0000","last_change":"2023-10-08T16:10:04.860545+0000","last_active":"2023-10-08T16:10:04.861516+0000","last_peered":"2023-10-08T16:10:04.861516+0000","last_clean":"2023-10-08T16:10:04.861516+0000","last_became_active":"2023-10-08T16:10:04.860262+0000","last_became_peered":"2023-10-08T16:10:04.860262+0000","last_unstale":"2023-10-08T16:10:04.861516+0000","last_undegraded":"2023-10-08T16:10:04.861516+0000","last_fullsized":"2023-10-08T16:10:04.861516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.856728+0000","last_change":"2023-10-08T16:10:03.863315+0000","last_active":"2023-10-08T16:10:04.856728+0000","last_peered":"2023-10-08T16:10:04.856728+0000","last_clean":"2023-10-08T16:10:04.856728+0000","last_became_active":"2023-10-08T16:10:03.863021+0000","last_became_peered":"2023-10-08T16:10:03.863021+0000","last_unstale":"2023-10-08T16:10:04.856728+0000","last_undegraded":"2023-10-08T16:10:04.856728+0000","last_fullsized":"2023-10-08T16:10:04.856728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875168+0000","last_change":"2023-10-08T16:10:04.867758+0000","last_active":"2023-10-08T16:10:04.875168+0000","last_peered":"2023-10-08T16:10:04.875168+0000","last_clean":"2023-10-08T16:10:04.875168+0000","last_became_active":"2023-10-08T16:10:04.867480+0000","last_became_peered":"2023-10-08T16:10:04.867480+0000","last_unstale":"2023-10-08T16:10:04.875168+0000","last_undegraded":"2023-10-08T16:10:04.875168+0000","last_fullsized":"2023-10-08T16:10:04.875168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.845232+0000","last_change":"2023-10-08T16:10:03.850832+0000","last_active":"2023-10-08T16:10:04.845232+0000","last_peered":"2023-10-08T16:10:04.845232+0000","last_clean":"2023-10-08T16:10:04.845232+0000","last_became_active":"2023-10-08T16:10:03.850090+0000","last_became_peered":"2023-10-08T16:10:03.850090+0000","last_unstale":"2023-10-08T16:10:04.845232+0000","last_undegraded":"2023-10-08T16:10:04.845232+0000","last_fullsized":"2023-10-08T16:10:04.845232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:14.511584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862608+0000","last_change":"2023-10-08T16:10:04.861486+0000","last_active":"2023-10-08T16:10:04.862608+0000","last_peered":"2023-10-08T16:10:04.862608+0000","last_clean":"2023-10-08T16:10:04.862608+0000","last_became_active":"2023-10-08T16:10:04.861197+0000","last_became_peered":"2023-10-08T16:10:04.861197+0000","last_unstale":"2023-10-08T16:10:04.862608+0000","last_undegraded":"2023-10-08T16:10:04.862608+0000","last_fullsized":"2023-10-08T16:10:04.862608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:15.955435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.846162+0000","last_change":"2023-10-08T16:10:03.853913+0000","last_active":"2023-10-08T16:10:04.846162+0000","last_peered":"2023-10-08T16:10:04.846162+0000","last_clean":"2023-10-08T16:10:04.846162+0000","last_became_active":"2023-10-08T16:10:03.853638+0000","last_became_peered":"2023-10-08T16:10:03.853638+0000","last_unstale":"2023-10-08T16:10:04.846162+0000","last_undegraded":"2023-10-08T16:10:04.846162+0000","last_fullsized":"2023-10-08T16:10:04.846162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876324+0000","last_change":"2023-10-08T16:10:04.873255+0000","last_active":"2023-10-08T16:10:04.876324+0000","last_peered":"2023-10-08T16:10:04.876324+0000","last_clean":"2023-10-08T16:10:04.876324+0000","last_became_active":"2023-10-08T16:10:04.872996+0000","last_became_peered":"2023-10-08T16:10:04.872996+0000","last_unstale":"2023-10-08T16:10:04.876324+0000","last_undegraded":"2023-10-08T16:10:04.876324+0000","last_fullsized":"2023-10-08T16:10:04.876324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854031+0000","last_change":"2023-10-08T16:10:03.867602+0000","last_active":"2023-10-08T16:10:04.854031+0000","last_peered":"2023-10-08T16:10:04.854031+0000","last_clean":"2023-10-08T16:10:04.854031+0000","last_became_active":"2023-10-08T16:10:03.867359+0000","last_became_peered":"2023-10-08T16:10:03.867359+0000","last_unstale":"2023-10-08T16:10:04.854031+0000","last_undegraded":"2023-10-08T16:10:04.854031+0000","last_fullsized":"2023-10-08T16:10:04.854031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875344+0000","last_change":"2023-10-08T16:10:04.868282+0000","last_active":"2023-10-08T16:10:04.875344+0000","last_peered":"2023-10-08T16:10:04.875344+0000","last_clean":"2023-10-08T16:10:04.875344+0000","last_became_active":"2023-10-08T16:10:04.868078+0000","last_became_peered":"2023-10-08T16:10:04.868078+0000","last_unstale":"2023-10-08T16:10:04.875344+0000","last_undegraded":"2023-10-08T16:10:04.875344+0000","last_fullsized":"2023-10-08T16:10:04.875344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"88'3","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.877059+0000","last_change":"2023-10-08T16:10:03.862435+0000","last_active":"2023-10-08T16:10:04.877059+0000","last_peered":"2023-10-08T16:10:04.877059+0000","last_clean":"2023-10-08T16:10:04.877059+0000","last_became_active":"2023-10-08T16:10:03.862150+0000","last_became_peered":"2023-10-08T16:10:03.862150+0000","last_unstale":"2023-10-08T16:10:04.877059+0000","last_undegraded":"2023-10-08T16:10:04.877059+0000","last_fullsized":"2023-10-08T16:10:04.877059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866232+0000","last_change":"2023-10-08T16:10:04.859540+0000","last_active":"2023-10-08T16:10:04.866232+0000","last_peered":"2023-10-08T16:10:04.866232+0000","last_clean":"2023-10-08T16:10:04.866232+0000","last_became_active":"2023-10-08T16:10:04.859292+0000","last_became_peered":"2023-10-08T16:10:04.859292+0000","last_unstale":"2023-10-08T16:10:04.866232+0000","last_undegraded":"2023-10-08T16:10:04.866232+0000","last_fullsized":"2023-10-08T16:10:04.866232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:43.947260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"88'6","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.878368+0000","last_change":"2023-10-08T16:10:03.859113+0000","last_active":"2023-10-08T16:10:04.878368+0000","last_peered":"2023-10-08T16:10:04.878368+0000","last_clean":"2023-10-08T16:10:04.878368+0000","last_became_active":"2023-10-08T16:10:03.858920+0000","last_became_peered":"2023-10-08T16:10:03.858920+0000","last_unstale":"2023-10-08T16:10:04.878368+0000","last_undegraded":"2023-10-08T16:10:04.878368+0000","last_fullsized":"2023-10-08T16:10:04.878368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:23.618518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861378+0000","last_change":"2023-10-08T16:10:04.860406+0000","last_active":"2023-10-08T16:10:04.861378+0000","last_peered":"2023-10-08T16:10:04.861378+0000","last_clean":"2023-10-08T16:10:04.861378+0000","last_became_active":"2023-10-08T16:10:04.860162+0000","last_became_peered":"2023-10-08T16:10:04.860162+0000","last_unstale":"2023-10-08T16:10:04.861378+0000","last_undegraded":"2023-10-08T16:10:04.861378+0000","last_fullsized":"2023-10-08T16:10:04.861378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876245+0000","last_change":"2023-10-08T16:10:03.860299+0000","last_active":"2023-10-08T16:10:04.876245+0000","last_peered":"2023-10-08T16:10:04.876245+0000","last_clean":"2023-10-08T16:10:04.876245+0000","last_became_active":"2023-10-08T16:10:03.860082+0000","last_became_peered":"2023-10-08T16:10:03.860082+0000","last_unstale":"2023-10-08T16:10:04.876245+0000","last_undegraded":"2023-10-08T16:10:04.876245+0000","last_fullsized":"2023-10-08T16:10:04.876245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:24.121795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858636+0000","last_change":"2023-10-08T16:10:04.855343+0000","last_active":"2023-10-08T16:10:04.858636+0000","last_peered":"2023-10-08T16:10:04.858636+0000","last_clean":"2023-10-08T16:10:04.858636+0000","last_became_active":"2023-10-08T16:10:04.854749+0000","last_became_peered":"2023-10-08T16:10:04.854749+0000","last_unstale":"2023-10-08T16:10:04.858636+0000","last_undegraded":"2023-10-08T16:10:04.858636+0000","last_fullsized":"2023-10-08T16:10:04.858636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:32.803419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849044+0000","last_change":"2023-10-08T16:10:03.857314+0000","last_active":"2023-10-08T16:10:04.849044+0000","last_peered":"2023-10-08T16:10:04.849044+0000","last_clean":"2023-10-08T16:10:04.849044+0000","last_became_active":"2023-10-08T16:10:03.857042+0000","last_became_peered":"2023-10-08T16:10:03.857042+0000","last_unstale":"2023-10-08T16:10:04.849044+0000","last_undegraded":"2023-10-08T16:10:04.849044+0000","last_fullsized":"2023-10-08T16:10:04.849044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:46.752089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874886+0000","last_change":"2023-10-08T16:10:04.867050+0000","last_active":"2023-10-08T16:10:04.874886+0000","last_peered":"2023-10-08T16:10:04.874886+0000","last_clean":"2023-10-08T16:10:04.874886+0000","last_became_active":"2023-10-08T16:10:04.866800+0000","last_became_peered":"2023-10-08T16:10:04.866800+0000","last_unstale":"2023-10-08T16:10:04.874886+0000","last_undegraded":"2023-10-08T16:10:04.874886+0000","last_fullsized":"2023-10-08T16:10:04.874886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858128+0000","last_change":"2023-10-08T16:10:03.862333+0000","last_active":"2023-10-08T16:10:04.858128+0000","last_peered":"2023-10-08T16:10:04.858128+0000","last_clean":"2023-10-08T16:10:04.858128+0000","last_became_active":"2023-10-08T16:10:03.862068+0000","last_became_peered":"2023-10-08T16:10:03.862068+0000","last_unstale":"2023-10-08T16:10:04.858128+0000","last_undegraded":"2023-10-08T16:10:04.858128+0000","last_fullsized":"2023-10-08T16:10:04.858128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:30.932977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865217+0000","last_change":"2023-10-08T16:10:04.863043+0000","last_active":"2023-10-08T16:10:04.865217+0000","last_peered":"2023-10-08T16:10:04.865217+0000","last_clean":"2023-10-08T16:10:04.865217+0000","last_became_active":"2023-10-08T16:10:04.862804+0000","last_became_peered":"2023-10-08T16:10:04.862804+0000","last_unstale":"2023-10-08T16:10:04.865217+0000","last_undegraded":"2023-10-08T16:10:04.865217+0000","last_fullsized":"2023-10-08T16:10:04.865217+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:17.060804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"88'8","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.877819+0000","last_change":"2023-10-08T16:10:03.855130+0000","last_active":"2023-10-08T16:10:04.877819+0000","last_peered":"2023-10-08T16:10:04.877819+0000","last_clean":"2023-10-08T16:10:04.877819+0000","last_became_active":"2023-10-08T16:10:03.854858+0000","last_became_peered":"2023-10-08T16:10:03.854858+0000","last_unstale":"2023-10-08T16:10:04.877819+0000","last_undegraded":"2023-10-08T16:10:04.877819+0000","last_fullsized":"2023-10-08T16:10:04.877819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875457+0000","last_change":"2023-10-08T16:10:04.872815+0000","last_active":"2023-10-08T16:10:04.875457+0000","last_peered":"2023-10-08T16:10:04.875457+0000","last_clean":"2023-10-08T16:10:04.875457+0000","last_became_active":"2023-10-08T16:10:04.872571+0000","last_became_peered":"2023-10-08T16:10:04.872571+0000","last_unstale":"2023-10-08T16:10:04.875457+0000","last_undegraded":"2023-10-08T16:10:04.875457+0000","last_fullsized":"2023-10-08T16:10:04.875457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.870666+0000","last_change":"2023-10-08T16:10:03.859396+0000","last_active":"2023-10-08T16:10:04.870666+0000","last_peered":"2023-10-08T16:10:04.870666+0000","last_clean":"2023-10-08T16:10:04.870666+0000","last_became_active":"2023-10-08T16:10:03.859124+0000","last_became_peered":"2023-10-08T16:10:03.859124+0000","last_unstale":"2023-10-08T16:10:04.870666+0000","last_undegraded":"2023-10-08T16:10:04.870666+0000","last_fullsized":"2023-10-08T16:10:04.870666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.873775+0000","last_change":"2023-10-08T16:10:04.869805+0000","last_active":"2023-10-08T16:10:04.873775+0000","last_peered":"2023-10-08T16:10:04.873775+0000","last_clean":"2023-10-08T16:10:04.873775+0000","last_became_active":"2023-10-08T16:10:04.869557+0000","last_became_peered":"2023-10-08T16:10:04.869557+0000","last_unstale":"2023-10-08T16:10:04.873775+0000","last_undegraded":"2023-10-08T16:10:04.873775+0000","last_fullsized":"2023-10-08T16:10:04.873775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876369+0000","last_change":"2023-10-08T16:10:03.854991+0000","last_active":"2023-10-08T16:10:04.876369+0000","last_peered":"2023-10-08T16:10:04.876369+0000","last_clean":"2023-10-08T16:10:04.876369+0000","last_became_active":"2023-10-08T16:10:03.854728+0000","last_became_peered":"2023-10-08T16:10:03.854728+0000","last_unstale":"2023-10-08T16:10:04.876369+0000","last_undegraded":"2023-10-08T16:10:04.876369+0000","last_fullsized":"2023-10-08T16:10:04.876369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876657+0000","last_change":"2023-10-08T16:10:04.866715+0000","last_active":"2023-10-08T16:10:04.876657+0000","last_peered":"2023-10-08T16:10:04.876657+0000","last_clean":"2023-10-08T16:10:04.876657+0000","last_became_active":"2023-10-08T16:10:04.866413+0000","last_became_peered":"2023-10-08T16:10:04.866413+0000","last_unstale":"2023-10-08T16:10:04.876657+0000","last_undegraded":"2023-10-08T16:10:04.876657+0000","last_fullsized":"2023-10-08T16:10:04.876657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:15.955435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851463+0000","last_change":"2023-10-08T16:10:03.867125+0000","last_active":"2023-10-08T16:10:04.851463+0000","last_peered":"2023-10-08T16:10:04.851463+0000","last_clean":"2023-10-08T16:10:04.851463+0000","last_became_active":"2023-10-08T16:10:03.866922+0000","last_became_peered":"2023-10-08T16:10:03.866922+0000","last_unstale":"2023-10-08T16:10:04.851463+0000","last_undegraded":"2023-10-08T16:10:04.851463+0000","last_fullsized":"2023-10-08T16:10:04.851463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862528+0000","last_change":"2023-10-08T16:10:04.856753+0000","last_active":"2023-10-08T16:10:04.862528+0000","last_peered":"2023-10-08T16:10:04.862528+0000","last_clean":"2023-10-08T16:10:04.862528+0000","last_became_active":"2023-10-08T16:10:04.856499+0000","last_became_peered":"2023-10-08T16:10:04.856499+0000","last_unstale":"2023-10-08T16:10:04.862528+0000","last_undegraded":"2023-10-08T16:10:04.862528+0000","last_fullsized":"2023-10-08T16:10:04.862528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:53.737297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871003+0000","last_change":"2023-10-08T16:10:03.862486+0000","last_active":"2023-10-08T16:10:04.871003+0000","last_peered":"2023-10-08T16:10:04.871003+0000","last_clean":"2023-10-08T16:10:04.871003+0000","last_became_active":"2023-10-08T16:10:03.862165+0000","last_became_peered":"2023-10-08T16:10:03.862165+0000","last_unstale":"2023-10-08T16:10:04.871003+0000","last_undegraded":"2023-10-08T16:10:04.871003+0000","last_fullsized":"2023-10-08T16:10:04.871003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861826+0000","last_change":"2023-10-08T16:10:04.859488+0000","last_active":"2023-10-08T16:10:04.861826+0000","last_peered":"2023-10-08T16:10:04.861826+0000","last_clean":"2023-10-08T16:10:04.861826+0000","last_became_active":"2023-10-08T16:10:04.859243+0000","last_became_peered":"2023-10-08T16:10:04.859243+0000","last_unstale":"2023-10-08T16:10:04.861826+0000","last_undegraded":"2023-10-08T16:10:04.861826+0000","last_fullsized":"2023-10-08T16:10:04.861826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:26.662940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849775+0000","last_change":"2023-10-08T16:10:03.855666+0000","last_active":"2023-10-08T16:10:04.849775+0000","last_peered":"2023-10-08T16:10:04.849775+0000","last_clean":"2023-10-08T16:10:04.849775+0000","last_became_active":"2023-10-08T16:10:03.855286+0000","last_became_peered":"2023-10-08T16:10:03.855286+0000","last_unstale":"2023-10-08T16:10:04.849775+0000","last_undegraded":"2023-10-08T16:10:04.849775+0000","last_fullsized":"2023-10-08T16:10:04.849775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861766+0000","last_change":"2023-10-08T16:10:04.856146+0000","last_active":"2023-10-08T16:10:04.861766+0000","last_peered":"2023-10-08T16:10:04.861766+0000","last_clean":"2023-10-08T16:10:04.861766+0000","last_became_active":"2023-10-08T16:10:04.855873+0000","last_became_peered":"2023-10-08T16:10:04.855873+0000","last_unstale":"2023-10-08T16:10:04.861766+0000","last_undegraded":"2023-10-08T16:10:04.861766+0000","last_fullsized":"2023-10-08T16:10:04.861766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:58.668805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849736+0000","last_change":"2023-10-08T16:10:03.857899+0000","last_active":"2023-10-08T16:10:04.849736+0000","last_peered":"2023-10-08T16:10:04.849736+0000","last_clean":"2023-10-08T16:10:04.849736+0000","last_became_active":"2023-10-08T16:10:03.857621+0000","last_became_peered":"2023-10-08T16:10:03.857621+0000","last_unstale":"2023-10-08T16:10:04.849736+0000","last_undegraded":"2023-10-08T16:10:04.849736+0000","last_fullsized":"2023-10-08T16:10:04.849736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:49.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874864+0000","last_change":"2023-10-08T16:10:04.868005+0000","last_active":"2023-10-08T16:10:04.874864+0000","last_peered":"2023-10-08T16:10:04.874864+0000","last_clean":"2023-10-08T16:10:04.874864+0000","last_became_active":"2023-10-08T16:10:04.867742+0000","last_became_peered":"2023-10-08T16:10:04.867742+0000","last_unstale":"2023-10-08T16:10:04.874864+0000","last_undegraded":"2023-10-08T16:10:04.874864+0000","last_fullsized":"2023-10-08T16:10:04.874864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851678+0000","last_change":"2023-10-08T16:10:03.855883+0000","last_active":"2023-10-08T16:10:04.851678+0000","last_peered":"2023-10-08T16:10:04.851678+0000","last_clean":"2023-10-08T16:10:04.851678+0000","last_became_active":"2023-10-08T16:10:03.855617+0000","last_became_peered":"2023-10-08T16:10:03.855617+0000","last_unstale":"2023-10-08T16:10:04.851678+0000","last_undegraded":"2023-10-08T16:10:04.851678+0000","last_fullsized":"2023-10-08T16:10:04.851678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:14.237332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871394+0000","last_change":"2023-10-08T16:10:04.867487+0000","last_active":"2023-10-08T16:10:04.871394+0000","last_peered":"2023-10-08T16:10:04.871394+0000","last_clean":"2023-10-08T16:10:04.871394+0000","last_became_active":"2023-10-08T16:10:04.865857+0000","last_became_peered":"2023-10-08T16:10:04.865857+0000","last_unstale":"2023-10-08T16:10:04.871394+0000","last_undegraded":"2023-10-08T16:10:04.871394+0000","last_fullsized":"2023-10-08T16:10:04.871394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:21.771026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876256+0000","last_change":"2023-10-08T16:10:03.859539+0000","last_active":"2023-10-08T16:10:04.876256+0000","last_peered":"2023-10-08T16:10:04.876256+0000","last_clean":"2023-10-08T16:10:04.876256+0000","last_became_active":"2023-10-08T16:10:03.859277+0000","last_became_peered":"2023-10-08T16:10:03.859277+0000","last_unstale":"2023-10-08T16:10:04.876256+0000","last_undegraded":"2023-10-08T16:10:04.876256+0000","last_fullsized":"2023-10-08T16:10:04.876256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:58.278821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.850046+0000","last_change":"2023-10-08T16:10:04.849722+0000","last_active":"2023-10-08T16:10:04.850046+0000","last_peered":"2023-10-08T16:10:04.850046+0000","last_clean":"2023-10-08T16:10:04.850046+0000","last_became_active":"2023-10-08T16:10:04.849448+0000","last_became_peered":"2023-10-08T16:10:04.849448+0000","last_unstale":"2023-10-08T16:10:04.850046+0000","last_undegraded":"2023-10-08T16:10:04.850046+0000","last_fullsized":"2023-10-08T16:10:04.850046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857374+0000","last_change":"2023-10-08T16:10:03.867454+0000","last_active":"2023-10-08T16:10:04.857374+0000","last_peered":"2023-10-08T16:10:04.857374+0000","last_clean":"2023-10-08T16:10:04.857374+0000","last_became_active":"2023-10-08T16:10:03.867150+0000","last_became_peered":"2023-10-08T16:10:03.867150+0000","last_unstale":"2023-10-08T16:10:04.857374+0000","last_undegraded":"2023-10-08T16:10:04.857374+0000","last_fullsized":"2023-10-08T16:10:04.857374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:49.978809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872897+0000","last_change":"2023-10-08T16:10:04.869601+0000","last_active":"2023-10-08T16:10:04.872897+0000","last_peered":"2023-10-08T16:10:04.872897+0000","last_clean":"2023-10-08T16:10:04.872897+0000","last_became_active":"2023-10-08T16:10:04.869348+0000","last_became_peered":"2023-10-08T16:10:04.869348+0000","last_unstale":"2023-10-08T16:10:04.872897+0000","last_undegraded":"2023-10-08T16:10:04.872897+0000","last_fullsized":"2023-10-08T16:10:04.872897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851335+0000","last_change":"2023-10-08T16:10:03.863109+0000","last_active":"2023-10-08T16:10:04.851335+0000","last_peered":"2023-10-08T16:10:04.851335+0000","last_clean":"2023-10-08T16:10:04.851335+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:04.851335+0000","last_undegraded":"2023-10-08T16:10:04.851335+0000","last_fullsized":"2023-10-08T16:10:04.851335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:46.037098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874773+0000","last_change":"2023-10-08T16:10:04.867578+0000","last_active":"2023-10-08T16:10:04.874773+0000","last_peered":"2023-10-08T16:10:04.874773+0000","last_clean":"2023-10-08T16:10:04.874773+0000","last_became_active":"2023-10-08T16:10:04.867296+0000","last_became_peered":"2023-10-08T16:10:04.867296+0000","last_unstale":"2023-10-08T16:10:04.874773+0000","last_undegraded":"2023-10-08T16:10:04.874773+0000","last_fullsized":"2023-10-08T16:10:04.874773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854981+0000","last_change":"2023-10-08T16:10:03.857051+0000","last_active":"2023-10-08T16:10:04.854981+0000","last_peered":"2023-10-08T16:10:04.854981+0000","last_clean":"2023-10-08T16:10:04.854981+0000","last_became_active":"2023-10-08T16:10:03.856770+0000","last_became_peered":"2023-10-08T16:10:03.856770+0000","last_unstale":"2023-10-08T16:10:04.854981+0000","last_undegraded":"2023-10-08T16:10:04.854981+0000","last_fullsized":"2023-10-08T16:10:04.854981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865103+0000","last_change":"2023-10-08T16:10:04.862789+0000","last_active":"2023-10-08T16:10:04.865103+0000","last_peered":"2023-10-08T16:10:04.865103+0000","last_clean":"2023-10-08T16:10:04.865103+0000","last_became_active":"2023-10-08T16:10:04.862526+0000","last_became_peered":"2023-10-08T16:10:04.862526+0000","last_unstale":"2023-10-08T16:10:04.865103+0000","last_undegraded":"2023-10-08T16:10:04.865103+0000","last_fullsized":"2023-10-08T16:10:04.865103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:32:12.016581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858151+0000","last_change":"2023-10-08T16:10:03.859409+0000","last_active":"2023-10-08T16:10:04.858151+0000","last_peered":"2023-10-08T16:10:04.858151+0000","last_clean":"2023-10-08T16:10:04.858151+0000","last_became_active":"2023-10-08T16:10:03.859104+0000","last_became_peered":"2023-10-08T16:10:03.859104+0000","last_unstale":"2023-10-08T16:10:04.858151+0000","last_undegraded":"2023-10-08T16:10:04.858151+0000","last_fullsized":"2023-10-08T16:10:04.858151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875874+0000","last_change":"2023-10-08T16:10:04.870211+0000","last_active":"2023-10-08T16:10:04.875874+0000","last_peered":"2023-10-08T16:10:04.875874+0000","last_clean":"2023-10-08T16:10:04.875874+0000","last_became_active":"2023-10-08T16:10:04.869907+0000","last_became_peered":"2023-10-08T16:10:04.869907+0000","last_unstale":"2023-10-08T16:10:04.875874+0000","last_undegraded":"2023-10-08T16:10:04.875874+0000","last_fullsized":"2023-10-08T16:10:04.875874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857857+0000","last_change":"2023-10-08T16:10:03.857623+0000","last_active":"2023-10-08T16:10:04.857857+0000","last_peered":"2023-10-08T16:10:04.857857+0000","last_clean":"2023-10-08T16:10:04.857857+0000","last_became_active":"2023-10-08T16:10:03.857351+0000","last_became_peered":"2023-10-08T16:10:03.857351+0000","last_unstale":"2023-10-08T16:10:04.857857+0000","last_undegraded":"2023-10-08T16:10:04.857857+0000","last_fullsized":"2023-10-08T16:10:04.857857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865135+0000","last_change":"2023-10-08T16:10:04.862545+0000","last_active":"2023-10-08T16:10:04.865135+0000","last_peered":"2023-10-08T16:10:04.865135+0000","last_clean":"2023-10-08T16:10:04.865135+0000","last_became_active":"2023-10-08T16:10:04.862294+0000","last_became_peered":"2023-10-08T16:10:04.862294+0000","last_unstale":"2023-10-08T16:10:04.865135+0000","last_undegraded":"2023-10-08T16:10:04.865135+0000","last_fullsized":"2023-10-08T16:10:04.865135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:32.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851485+0000","last_change":"2023-10-08T16:10:03.856881+0000","last_active":"2023-10-08T16:10:04.851485+0000","last_peered":"2023-10-08T16:10:04.851485+0000","last_clean":"2023-10-08T16:10:04.851485+0000","last_became_active":"2023-10-08T16:10:03.856607+0000","last_became_peered":"2023-10-08T16:10:03.856607+0000","last_unstale":"2023-10-08T16:10:04.851485+0000","last_undegraded":"2023-10-08T16:10:04.851485+0000","last_fullsized":"2023-10-08T16:10:04.851485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875286+0000","last_change":"2023-10-08T16:10:04.869172+0000","last_active":"2023-10-08T16:10:04.875286+0000","last_peered":"2023-10-08T16:10:04.875286+0000","last_clean":"2023-10-08T16:10:04.875286+0000","last_became_active":"2023-10-08T16:10:04.864575+0000","last_became_peered":"2023-10-08T16:10:04.864575+0000","last_unstale":"2023-10-08T16:10:04.875286+0000","last_undegraded":"2023-10-08T16:10:04.875286+0000","last_fullsized":"2023-10-08T16:10:04.875286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.847916+0000","last_change":"2023-10-08T16:10:03.863035+0000","last_active":"2023-10-08T16:10:04.847916+0000","last_peered":"2023-10-08T16:10:04.847916+0000","last_clean":"2023-10-08T16:10:04.847916+0000","last_became_active":"2023-10-08T16:10:03.862801+0000","last_became_peered":"2023-10-08T16:10:03.862801+0000","last_unstale":"2023-10-08T16:10:04.847916+0000","last_undegraded":"2023-10-08T16:10:04.847916+0000","last_fullsized":"2023-10-08T16:10:04.847916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:31.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862224+0000","last_change":"2023-10-08T16:10:04.856941+0000","last_active":"2023-10-08T16:10:04.862224+0000","last_peered":"2023-10-08T16:10:04.862224+0000","last_clean":"2023-10-08T16:10:04.862224+0000","last_became_active":"2023-10-08T16:10:04.856694+0000","last_became_peered":"2023-10-08T16:10:04.856694+0000","last_unstale":"2023-10-08T16:10:04.862224+0000","last_undegraded":"2023-10-08T16:10:04.862224+0000","last_fullsized":"2023-10-08T16:10:04.862224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:39.023795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876559+0000","last_change":"2023-10-08T16:10:03.856920+0000","last_active":"2023-10-08T16:10:04.876559+0000","last_peered":"2023-10-08T16:10:04.876559+0000","last_clean":"2023-10-08T16:10:04.876559+0000","last_became_active":"2023-10-08T16:10:03.856673+0000","last_became_peered":"2023-10-08T16:10:03.856673+0000","last_unstale":"2023-10-08T16:10:04.876559+0000","last_undegraded":"2023-10-08T16:10:04.876559+0000","last_fullsized":"2023-10-08T16:10:04.876559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.853991+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:04.853991+0000","last_peered":"2023-10-08T16:10:04.853991+0000","last_clean":"2023-10-08T16:10:04.853991+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:04.853991+0000","last_undegraded":"2023-10-08T16:10:04.853991+0000","last_fullsized":"2023-10-08T16:10:04.853991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298952,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072888,"statfs":{"total":96636764160,"available":96330637312,"internally_reserved":0,"allocated":1703936,"data_stored":1064767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.75},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.52700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.69199999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.70399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":474911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70799999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.497},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1101824,"data_stored":474287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.308}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58999999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.625},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303044,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068796,"statfs":{"total":96636764160,"available":96326447104,"internally_reserved":0,"allocated":1699840,"data_stored":1065209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.372},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.42599999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.85699999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1105920,"data_stored":474321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38100000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.312},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.749}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":475845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.437},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.45700000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.70299999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":475947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71299999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.495},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.74399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":474357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.39900000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.5}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:13.416 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:13.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.543+0000 7f775019c700 1 Processor -- start 2023-10-08T16:10:13.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.543+0000 7f775019c700 1 -- start start 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f775019c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f774815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f775019c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f774815db10 0x7f7748069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f775019c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774815a160 con 0x7f7748159890 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f775019c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480d13a0 con 0x7f774815db10 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f775019c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774815e140 con 0x7f7748159c60 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774df38700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7748159890 0x7f774815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45626/0 (socket says 172.21.15.170:45626) 2023-10-08T16:10:13.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774df38700 1 -- 172.21.15.170:0/1051531409 learned_addr learned my addr 172.21.15.170:0/1051531409 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774d737700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f774815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623597297 0 0) 0x7f774815a160 con 0x7f7748159890 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774e739700 1 --2- 172.21.15.170:0/1051531409 >> v2:172.21.15.191:3300/0 conn(0x7f774815db10 0x7f7748069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7734003570 con 0x7f7748159890 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242646947 0 0) 0x7f7734003570 con 0x7f7748159890 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 msgr2=0x7f774815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f774815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 >> v2:172.21.15.191:3300/0 conn(0x7f774815db10 msgr2=0x7f7748069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 --2- 172.21.15.170:0/1051531409 >> v2:172.21.15.191:3300/0 conn(0x7f774815db10 0x7f7748069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774815a3d0 con 0x7f7748159890 2023-10-08T16:10:13.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7744001470 con 0x7f7748159890 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 >> v1:172.21.15.170:6789/0 conn(0x7f7748159890 legacy=0x7f774815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f7734003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7734003fe0 0x7f7734004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734004d50 0x7f77340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480d13a0 con 0x7f7734003fe0 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774815e140 con 0x7f7734004d50 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.544+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773400ba90 con 0x7f7748159c60 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7734003fe0 0x7f7734004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774df38700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f7734003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774e739700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734004d50 0x7f77340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 -- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 msgr2=0x7f7734003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:13.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7748159c60 0x7f7734003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 -- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734004d50 msgr2=0x7f77340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734004d50 0x7f77340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 -- 172.21.15.170:0/1051531409 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734003570 con 0x7f7734003fe0 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774d737700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7734003fe0 0x7f7734004810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f774815a090 tx=0x7f77380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b36c92346d174f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800e6b0 con 0x7f7734003fe0 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773800e810 con 0x7f7734003fe0 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.545+0000 7f774cf36700 1 -- 172.21.15.170:0/1051531409 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800ebd0 con 0x7f7734003fe0 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 -- 172.21.15.170:0/1051531409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7734003fe0 msgr2=0x7f7734004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 --2- 172.21.15.170:0/1051531409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7734003fe0 0x7f7734004810 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f774815a090 tx=0x7f77380091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f774e739700 1 -- 172.21.15.170:0/1051531409 reap_dead start 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 -- 172.21.15.170:0/1051531409 shutdown_connections 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 -- 172.21.15.170:0/1051531409 >> 172.21.15.170:0/1051531409 conn(0x7f7748068e20 msgr2=0x7f77480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 -- 172.21.15.170:0/1051531409 shutdown_connections 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.546+0000 7f775019c700 1 -- 172.21.15.170:0/1051531409 wait complete. 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.547+0000 7f775019c700 1 Processor -- start 2023-10-08T16:10:13.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.547+0000 7f775019c700 1 -- start start 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.547+0000 7f775019c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734003fe0 0x7f77481440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77480b0780 0x7f77480b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480c83a0 con 0x7f77480c79f0 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480ca6a0 con 0x7f7734003fe0 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480cab30 con 0x7f77480b0780 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49506/0 (socket says 172.21.15.170:49506) 2023-10-08T16:10:13.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 -- 172.21.15.170:0/633925081 learned_addr learned my addr 172.21.15.170:0/633925081 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 -- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77480b0780 msgr2=0x7f77480b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77480b0780 0x7f77480b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 -- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734003fe0 msgr2=0x7f77481440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734003fe0 0x7f77481440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 -- 172.21.15.170:0/633925081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77480c9d70 con 0x7f77480c79f0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f774d737700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7734004730 tx=0x7f773801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738022070 con 0x7f77480c79f0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773800e980 con 0x7f77480c79f0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773801c450 con 0x7f77480c79f0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480ca050 con 0x7f77480c79f0 2023-10-08T16:10:13.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.548+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77480c9a20 con 0x7f77480c79f0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.549+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77480c83a0 con 0x7f77480c79f0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.550+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f773801c8a0 con 0x7f77480c79f0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.550+0000 7f773dffb700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 0x7f772c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.550+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f77380860d0 con 0x7f77480c79f0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.550+0000 7f774df38700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 0x7f772c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.551+0000 7f774df38700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 0x7f772c069730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f77440011e0 tx=0x7f7744001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.555+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7738052fb0 con 0x7f77480c79f0 2023-10-08T16:10:13.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.732+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77480ca810 con 0x7f772c067280 2023-10-08T16:10:13.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.740+0000 7f773dffb700 1 -- 172.21.15.170:0/633925081 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289316 (secure 0 0 0) 0x7f77480ca810 con 0x7f772c067280 2023-10-08T16:10:13.744 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:13.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 msgr2=0x7f772c069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 0x7f772c069730 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f77440011e0 tx=0x7f7744001e90 comp rx=0 tx=0).stop 2023-10-08T16:10:13.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 msgr2=0x7f77480c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7734004730 tx=0x7f773801a000 comp rx=0 tx=0).stop 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 shutdown_connections 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f77480b0780 0x7f77480b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7734003fe0 0x7f77481440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f772c067280 0x7f772c069730 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 --2- 172.21.15.170:0/633925081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77480c79f0 0x7f77480c7e60 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 >> 172.21.15.170:0/633925081 conn(0x7f7748068e20 msgr2=0x7f77481587a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.743+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 shutdown_connections 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.744+0000 7f775019c700 1 -- 172.21.15.170:0/633925081 wait complete. 2023-10-08T16:10:13.747 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:13.756 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2023-10-08T16:10:12.293578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2404520,"kb_used_data":9832,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752570200,"statfs":{"total":773094113280,"available":770631884800,"internally_reserved":0,"allocated":10067968,"data_stored":4979644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001628"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859739+0000","last_change":"2023-10-08T16:10:04.858512+0000","last_active":"2023-10-08T16:10:04.859739+0000","last_peered":"2023-10-08T16:10:04.859739+0000","last_clean":"2023-10-08T16:10:04.859739+0000","last_became_active":"2023-10-08T16:10:04.856420+0000","last_became_peered":"2023-10-08T16:10:04.856420+0000","last_unstale":"2023-10-08T16:10:04.859739+0000","last_undegraded":"2023-10-08T16:10:04.859739+0000","last_fullsized":"2023-10-08T16:10:04.859739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:31.939357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871412+0000","last_change":"2023-10-08T16:10:04.867644+0000","last_active":"2023-10-08T16:10:04.871412+0000","last_peered":"2023-10-08T16:10:04.871412+0000","last_clean":"2023-10-08T16:10:04.871412+0000","last_became_active":"2023-10-08T16:10:04.866553+0000","last_became_peered":"2023-10-08T16:10:04.866553+0000","last_unstale":"2023-10-08T16:10:04.871412+0000","last_undegraded":"2023-10-08T16:10:04.871412+0000","last_fullsized":"2023-10-08T16:10:04.871412+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:43.947260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859777+0000","last_change":"2023-10-08T16:10:04.857925+0000","last_active":"2023-10-08T16:10:04.859777+0000","last_peered":"2023-10-08T16:10:04.859777+0000","last_clean":"2023-10-08T16:10:04.859777+0000","last_became_active":"2023-10-08T16:10:04.857660+0000","last_became_peered":"2023-10-08T16:10:04.857660+0000","last_unstale":"2023-10-08T16:10:04.859777+0000","last_undegraded":"2023-10-08T16:10:04.859777+0000","last_fullsized":"2023-10-08T16:10:04.859777+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:24.624899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857603+0000","last_change":"2023-10-08T16:10:04.855606+0000","last_active":"2023-10-08T16:10:04.857603+0000","last_peered":"2023-10-08T16:10:04.857603+0000","last_clean":"2023-10-08T16:10:04.857603+0000","last_became_active":"2023-10-08T16:10:04.855157+0000","last_became_peered":"2023-10-08T16:10:04.855157+0000","last_unstale":"2023-10-08T16:10:04.857603+0000","last_undegraded":"2023-10-08T16:10:04.857603+0000","last_fullsized":"2023-10-08T16:10:04.857603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872824+0000","last_change":"2023-10-08T16:10:04.863817+0000","last_active":"2023-10-08T16:10:04.872824+0000","last_peered":"2023-10-08T16:10:04.872824+0000","last_clean":"2023-10-08T16:10:04.872824+0000","last_became_active":"2023-10-08T16:10:04.863540+0000","last_became_peered":"2023-10-08T16:10:04.863540+0000","last_unstale":"2023-10-08T16:10:04.872824+0000","last_undegraded":"2023-10-08T16:10:04.872824+0000","last_fullsized":"2023-10-08T16:10:04.872824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:21.468575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876390+0000","last_change":"2023-10-08T16:10:04.866534+0000","last_active":"2023-10-08T16:10:04.876390+0000","last_peered":"2023-10-08T16:10:04.876390+0000","last_clean":"2023-10-08T16:10:04.876390+0000","last_became_active":"2023-10-08T16:10:04.866237+0000","last_became_peered":"2023-10-08T16:10:04.866237+0000","last_unstale":"2023-10-08T16:10:04.876390+0000","last_undegraded":"2023-10-08T16:10:04.876390+0000","last_fullsized":"2023-10-08T16:10:04.876390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:34.800715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876461+0000","last_change":"2023-10-08T16:10:04.867705+0000","last_active":"2023-10-08T16:10:04.876461+0000","last_peered":"2023-10-08T16:10:04.876461+0000","last_clean":"2023-10-08T16:10:04.876461+0000","last_became_active":"2023-10-08T16:10:04.867436+0000","last_became_peered":"2023-10-08T16:10:04.867436+0000","last_unstale":"2023-10-08T16:10:04.876461+0000","last_undegraded":"2023-10-08T16:10:04.876461+0000","last_fullsized":"2023-10-08T16:10:04.876461+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:43:41.959195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861635+0000","last_change":"2023-10-08T16:10:04.860107+0000","last_active":"2023-10-08T16:10:04.861635+0000","last_peered":"2023-10-08T16:10:04.861635+0000","last_clean":"2023-10-08T16:10:04.861635+0000","last_became_active":"2023-10-08T16:10:04.859658+0000","last_became_peered":"2023-10-08T16:10:04.859658+0000","last_unstale":"2023-10-08T16:10:04.861635+0000","last_undegraded":"2023-10-08T16:10:04.861635+0000","last_fullsized":"2023-10-08T16:10:04.861635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:30.911532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871687+0000","last_change":"2023-10-08T16:10:04.862038+0000","last_active":"2023-10-08T16:10:04.871687+0000","last_peered":"2023-10-08T16:10:04.871687+0000","last_clean":"2023-10-08T16:10:04.871687+0000","last_became_active":"2023-10-08T16:10:04.861746+0000","last_became_peered":"2023-10-08T16:10:04.861746+0000","last_unstale":"2023-10-08T16:10:04.871687+0000","last_undegraded":"2023-10-08T16:10:04.871687+0000","last_fullsized":"2023-10-08T16:10:04.871687+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875189+0000","last_change":"2023-10-08T16:10:04.868752+0000","last_active":"2023-10-08T16:10:04.875189+0000","last_peered":"2023-10-08T16:10:04.875189+0000","last_clean":"2023-10-08T16:10:04.875189+0000","last_became_active":"2023-10-08T16:10:04.864408+0000","last_became_peered":"2023-10-08T16:10:04.864408+0000","last_unstale":"2023-10-08T16:10:04.875189+0000","last_undegraded":"2023-10-08T16:10:04.875189+0000","last_fullsized":"2023-10-08T16:10:04.875189+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:32.803419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876033+0000","last_change":"2023-10-08T16:10:04.865163+0000","last_active":"2023-10-08T16:10:04.876033+0000","last_peered":"2023-10-08T16:10:04.876033+0000","last_clean":"2023-10-08T16:10:04.876033+0000","last_became_active":"2023-10-08T16:10:04.864821+0000","last_became_peered":"2023-10-08T16:10:04.864821+0000","last_unstale":"2023-10-08T16:10:04.876033+0000","last_undegraded":"2023-10-08T16:10:04.876033+0000","last_fullsized":"2023-10-08T16:10:04.876033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:05.440731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876415+0000","last_change":"2023-10-08T16:10:04.872779+0000","last_active":"2023-10-08T16:10:04.876415+0000","last_peered":"2023-10-08T16:10:04.876415+0000","last_clean":"2023-10-08T16:10:04.876415+0000","last_became_active":"2023-10-08T16:10:04.872079+0000","last_became_peered":"2023-10-08T16:10:04.872079+0000","last_unstale":"2023-10-08T16:10:04.876415+0000","last_undegraded":"2023-10-08T16:10:04.876415+0000","last_fullsized":"2023-10-08T16:10:04.876415+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:17:59.822102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.863134+0000","last_change":"2023-10-08T16:10:04.861826+0000","last_active":"2023-10-08T16:10:04.863134+0000","last_peered":"2023-10-08T16:10:04.863134+0000","last_clean":"2023-10-08T16:10:04.863134+0000","last_became_active":"2023-10-08T16:10:04.861579+0000","last_became_peered":"2023-10-08T16:10:04.861579+0000","last_unstale":"2023-10-08T16:10:04.863134+0000","last_undegraded":"2023-10-08T16:10:04.863134+0000","last_fullsized":"2023-10-08T16:10:04.863134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865558+0000","last_change":"2023-10-08T16:10:04.864671+0000","last_active":"2023-10-08T16:10:04.865558+0000","last_peered":"2023-10-08T16:10:04.865558+0000","last_clean":"2023-10-08T16:10:04.865558+0000","last_became_active":"2023-10-08T16:10:04.864414+0000","last_became_peered":"2023-10-08T16:10:04.864414+0000","last_unstale":"2023-10-08T16:10:04.865558+0000","last_undegraded":"2023-10-08T16:10:04.865558+0000","last_fullsized":"2023-10-08T16:10:04.865558+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:34.143021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.856924+0000","last_change":"2023-10-08T16:10:04.854692+0000","last_active":"2023-10-08T16:10:04.856924+0000","last_peered":"2023-10-08T16:10:04.856924+0000","last_clean":"2023-10-08T16:10:04.856924+0000","last_became_active":"2023-10-08T16:10:04.854453+0000","last_became_peered":"2023-10-08T16:10:04.854453+0000","last_unstale":"2023-10-08T16:10:04.856924+0000","last_undegraded":"2023-10-08T16:10:04.856924+0000","last_fullsized":"2023-10-08T16:10:04.856924+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.863818+0000","last_change":"2023-10-08T16:10:04.858600+0000","last_active":"2023-10-08T16:10:04.863818+0000","last_peered":"2023-10-08T16:10:04.863818+0000","last_clean":"2023-10-08T16:10:04.863818+0000","last_became_active":"2023-10-08T16:10:04.858320+0000","last_became_peered":"2023-10-08T16:10:04.858320+0000","last_unstale":"2023-10-08T16:10:04.863818+0000","last_undegraded":"2023-10-08T16:10:04.863818+0000","last_fullsized":"2023-10-08T16:10:04.863818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:30.911532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862428+0000","last_change":"2023-10-08T16:10:04.859672+0000","last_active":"2023-10-08T16:10:04.862428+0000","last_peered":"2023-10-08T16:10:04.862428+0000","last_clean":"2023-10-08T16:10:04.862428+0000","last_became_active":"2023-10-08T16:10:04.859413+0000","last_became_peered":"2023-10-08T16:10:04.859413+0000","last_unstale":"2023-10-08T16:10:04.862428+0000","last_undegraded":"2023-10-08T16:10:04.862428+0000","last_fullsized":"2023-10-08T16:10:04.862428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865091+0000","last_change":"2023-10-08T16:10:04.864133+0000","last_active":"2023-10-08T16:10:04.865091+0000","last_peered":"2023-10-08T16:10:04.865091+0000","last_clean":"2023-10-08T16:10:04.865091+0000","last_became_active":"2023-10-08T16:10:04.863618+0000","last_became_peered":"2023-10-08T16:10:04.863618+0000","last_unstale":"2023-10-08T16:10:04.865091+0000","last_undegraded":"2023-10-08T16:10:04.865091+0000","last_fullsized":"2023-10-08T16:10:04.865091+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:05:14.886258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854508+0000","last_change":"2023-10-08T16:10:04.850440+0000","last_active":"2023-10-08T16:10:04.854508+0000","last_peered":"2023-10-08T16:10:04.854508+0000","last_clean":"2023-10-08T16:10:04.854508+0000","last_became_active":"2023-10-08T16:10:04.850132+0000","last_became_peered":"2023-10-08T16:10:04.850132+0000","last_unstale":"2023-10-08T16:10:04.854508+0000","last_undegraded":"2023-10-08T16:10:04.854508+0000","last_fullsized":"2023-10-08T16:10:04.854508+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:18.809308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871864+0000","last_change":"2023-10-08T16:10:04.862675+0000","last_active":"2023-10-08T16:10:04.871864+0000","last_peered":"2023-10-08T16:10:04.871864+0000","last_clean":"2023-10-08T16:10:04.871864+0000","last_became_active":"2023-10-08T16:10:04.862402+0000","last_became_peered":"2023-10-08T16:10:04.862402+0000","last_unstale":"2023-10-08T16:10:04.871864+0000","last_undegraded":"2023-10-08T16:10:04.871864+0000","last_fullsized":"2023-10-08T16:10:04.871864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:32.650827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865931+0000","last_change":"2023-10-08T16:10:04.864596+0000","last_active":"2023-10-08T16:10:04.865931+0000","last_peered":"2023-10-08T16:10:04.865931+0000","last_clean":"2023-10-08T16:10:04.865931+0000","last_became_active":"2023-10-08T16:10:04.863904+0000","last_became_peered":"2023-10-08T16:10:04.863904+0000","last_unstale":"2023-10-08T16:10:04.865931+0000","last_undegraded":"2023-10-08T16:10:04.865931+0000","last_fullsized":"2023-10-08T16:10:04.865931+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:31.607081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.868741+0000","last_change":"2023-10-08T16:10:04.866160+0000","last_active":"2023-10-08T16:10:04.868741+0000","last_peered":"2023-10-08T16:10:04.868741+0000","last_clean":"2023-10-08T16:10:04.868741+0000","last_became_active":"2023-10-08T16:10:04.865846+0000","last_became_peered":"2023-10-08T16:10:04.865846+0000","last_unstale":"2023-10-08T16:10:04.868741+0000","last_undegraded":"2023-10-08T16:10:04.868741+0000","last_fullsized":"2023-10-08T16:10:04.868741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:05.440731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.868980+0000","last_change":"2023-10-08T16:10:04.866215+0000","last_active":"2023-10-08T16:10:04.868980+0000","last_peered":"2023-10-08T16:10:04.868980+0000","last_clean":"2023-10-08T16:10:04.868980+0000","last_became_active":"2023-10-08T16:10:04.865951+0000","last_became_peered":"2023-10-08T16:10:04.865951+0000","last_unstale":"2023-10-08T16:10:04.868980+0000","last_undegraded":"2023-10-08T16:10:04.868980+0000","last_fullsized":"2023-10-08T16:10:04.868980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:29.068707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866022+0000","last_change":"2023-10-08T16:10:04.860722+0000","last_active":"2023-10-08T16:10:04.866022+0000","last_peered":"2023-10-08T16:10:04.866022+0000","last_clean":"2023-10-08T16:10:04.866022+0000","last_became_active":"2023-10-08T16:10:04.860474+0000","last_became_peered":"2023-10-08T16:10:04.860474+0000","last_unstale":"2023-10-08T16:10:04.866022+0000","last_undegraded":"2023-10-08T16:10:04.866022+0000","last_fullsized":"2023-10-08T16:10:04.866022+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:29.068707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874717+0000","last_change":"2023-10-08T16:10:04.867406+0000","last_active":"2023-10-08T16:10:04.874717+0000","last_peered":"2023-10-08T16:10:04.874717+0000","last_clean":"2023-10-08T16:10:04.874717+0000","last_became_active":"2023-10-08T16:10:04.867081+0000","last_became_peered":"2023-10-08T16:10:04.867081+0000","last_unstale":"2023-10-08T16:10:04.874717+0000","last_undegraded":"2023-10-08T16:10:04.874717+0000","last_fullsized":"2023-10-08T16:10:04.874717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862587+0000","last_change":"2023-10-08T16:10:04.861206+0000","last_active":"2023-10-08T16:10:04.862587+0000","last_peered":"2023-10-08T16:10:04.862587+0000","last_clean":"2023-10-08T16:10:04.862587+0000","last_became_active":"2023-10-08T16:10:04.860961+0000","last_became_peered":"2023-10-08T16:10:04.860961+0000","last_unstale":"2023-10-08T16:10:04.862587+0000","last_undegraded":"2023-10-08T16:10:04.862587+0000","last_fullsized":"2023-10-08T16:10:04.862587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:02:34.800715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876686+0000","last_change":"2023-10-08T16:10:04.871483+0000","last_active":"2023-10-08T16:10:04.876686+0000","last_peered":"2023-10-08T16:10:04.876686+0000","last_clean":"2023-10-08T16:10:04.876686+0000","last_became_active":"2023-10-08T16:10:04.870548+0000","last_became_peered":"2023-10-08T16:10:04.870548+0000","last_unstale":"2023-10-08T16:10:04.876686+0000","last_undegraded":"2023-10-08T16:10:04.876686+0000","last_fullsized":"2023-10-08T16:10:04.876686+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875323+0000","last_change":"2023-10-08T16:10:04.868243+0000","last_active":"2023-10-08T16:10:04.875323+0000","last_peered":"2023-10-08T16:10:04.875323+0000","last_clean":"2023-10-08T16:10:04.875323+0000","last_became_active":"2023-10-08T16:10:04.867970+0000","last_became_peered":"2023-10-08T16:10:04.867970+0000","last_unstale":"2023-10-08T16:10:04.875323+0000","last_undegraded":"2023-10-08T16:10:04.875323+0000","last_fullsized":"2023-10-08T16:10:04.875323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:15.243712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.864291+0000","last_change":"2023-10-08T16:10:04.863101+0000","last_active":"2023-10-08T16:10:04.864291+0000","last_peered":"2023-10-08T16:10:04.864291+0000","last_clean":"2023-10-08T16:10:04.864291+0000","last_became_active":"2023-10-08T16:10:04.862827+0000","last_became_peered":"2023-10-08T16:10:04.862827+0000","last_unstale":"2023-10-08T16:10:04.864291+0000","last_undegraded":"2023-10-08T16:10:04.864291+0000","last_fullsized":"2023-10-08T16:10:04.864291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:42.557884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854754+0000","last_change":"2023-10-08T16:10:04.850190+0000","last_active":"2023-10-08T16:10:04.854754+0000","last_peered":"2023-10-08T16:10:04.854754+0000","last_clean":"2023-10-08T16:10:04.854754+0000","last_became_active":"2023-10-08T16:10:04.849935+0000","last_became_peered":"2023-10-08T16:10:04.849935+0000","last_unstale":"2023-10-08T16:10:04.854754+0000","last_undegraded":"2023-10-08T16:10:04.854754+0000","last_fullsized":"2023-10-08T16:10:04.854754+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:53:23.797032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871369+0000","last_change":"2023-10-08T16:10:04.870436+0000","last_active":"2023-10-08T16:10:04.871369+0000","last_peered":"2023-10-08T16:10:04.871369+0000","last_clean":"2023-10-08T16:10:04.871369+0000","last_became_active":"2023-10-08T16:10:04.870041+0000","last_became_peered":"2023-10-08T16:10:04.870041+0000","last_unstale":"2023-10-08T16:10:04.871369+0000","last_undegraded":"2023-10-08T16:10:04.871369+0000","last_fullsized":"2023-10-08T16:10:04.871369+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:39.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.859692+0000","last_change":"2023-10-08T16:10:04.858462+0000","last_active":"2023-10-08T16:10:04.859692+0000","last_peered":"2023-10-08T16:10:04.859692+0000","last_clean":"2023-10-08T16:10:04.859692+0000","last_became_active":"2023-10-08T16:10:04.858207+0000","last_became_peered":"2023-10-08T16:10:04.858207+0000","last_unstale":"2023-10-08T16:10:04.859692+0000","last_undegraded":"2023-10-08T16:10:04.859692+0000","last_fullsized":"2023-10-08T16:10:04.859692+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865239+0000","last_change":"2023-10-08T16:10:04.862333+0000","last_active":"2023-10-08T16:10:04.865239+0000","last_peered":"2023-10-08T16:10:04.865239+0000","last_clean":"2023-10-08T16:10:04.865239+0000","last_became_active":"2023-10-08T16:10:04.862034+0000","last_became_peered":"2023-10-08T16:10:04.862034+0000","last_unstale":"2023-10-08T16:10:04.865239+0000","last_undegraded":"2023-10-08T16:10:04.865239+0000","last_fullsized":"2023-10-08T16:10:04.865239+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:18.078100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"88'4","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875881+0000","last_change":"2023-10-08T16:10:03.859417+0000","last_active":"2023-10-08T16:10:04.875881+0000","last_peered":"2023-10-08T16:10:04.875881+0000","last_clean":"2023-10-08T16:10:04.875881+0000","last_became_active":"2023-10-08T16:10:03.859153+0000","last_became_peered":"2023-10-08T16:10:03.859153+0000","last_unstale":"2023-10-08T16:10:04.875881+0000","last_undegraded":"2023-10-08T16:10:04.875881+0000","last_fullsized":"2023-10-08T16:10:04.875881+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871374+0000","last_change":"2023-10-08T16:10:04.870602+0000","last_active":"2023-10-08T16:10:04.871374+0000","last_peered":"2023-10-08T16:10:04.871374+0000","last_clean":"2023-10-08T16:10:04.871374+0000","last_became_active":"2023-10-08T16:10:04.869867+0000","last_became_peered":"2023-10-08T16:10:04.869867+0000","last_unstale":"2023-10-08T16:10:04.871374+0000","last_undegraded":"2023-10-08T16:10:04.871374+0000","last_fullsized":"2023-10-08T16:10:04.871374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:50:45.325619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876470+0000","last_change":"2023-10-08T16:10:03.859490+0000","last_active":"2023-10-08T16:10:04.876470+0000","last_peered":"2023-10-08T16:10:04.876470+0000","last_clean":"2023-10-08T16:10:04.876470+0000","last_became_active":"2023-10-08T16:10:03.859218+0000","last_became_peered":"2023-10-08T16:10:03.859218+0000","last_unstale":"2023-10-08T16:10:04.876470+0000","last_undegraded":"2023-10-08T16:10:04.876470+0000","last_fullsized":"2023-10-08T16:10:04.876470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:33.681234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.864748+0000","last_change":"2023-10-08T16:10:04.863769+0000","last_active":"2023-10-08T16:10:04.864748+0000","last_peered":"2023-10-08T16:10:04.864748+0000","last_clean":"2023-10-08T16:10:04.864748+0000","last_became_active":"2023-10-08T16:10:04.863514+0000","last_became_peered":"2023-10-08T16:10:04.863514+0000","last_unstale":"2023-10-08T16:10:04.864748+0000","last_undegraded":"2023-10-08T16:10:04.864748+0000","last_fullsized":"2023-10-08T16:10:04.864748+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871116+0000","last_change":"2023-10-08T16:10:03.862941+0000","last_active":"2023-10-08T16:10:04.871116+0000","last_peered":"2023-10-08T16:10:04.871116+0000","last_clean":"2023-10-08T16:10:04.871116+0000","last_became_active":"2023-10-08T16:10:03.862732+0000","last_became_peered":"2023-10-08T16:10:03.862732+0000","last_unstale":"2023-10-08T16:10:04.871116+0000","last_undegraded":"2023-10-08T16:10:04.871116+0000","last_fullsized":"2023-10-08T16:10:04.871116+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875810+0000","last_change":"2023-10-08T16:10:04.868792+0000","last_active":"2023-10-08T16:10:04.875810+0000","last_peered":"2023-10-08T16:10:04.875810+0000","last_clean":"2023-10-08T16:10:04.875810+0000","last_became_active":"2023-10-08T16:10:04.868523+0000","last_became_peered":"2023-10-08T16:10:04.868523+0000","last_unstale":"2023-10-08T16:10:04.875810+0000","last_undegraded":"2023-10-08T16:10:04.875810+0000","last_fullsized":"2023-10-08T16:10:04.875810+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:16.388293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876867+0000","last_change":"2023-10-08T16:10:03.862782+0000","last_active":"2023-10-08T16:10:04.876867+0000","last_peered":"2023-10-08T16:10:04.876867+0000","last_clean":"2023-10-08T16:10:04.876867+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:04.876867+0000","last_undegraded":"2023-10-08T16:10:04.876867+0000","last_fullsized":"2023-10-08T16:10:04.876867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:13.716867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866867+0000","last_change":"2023-10-08T16:10:04.862896+0000","last_active":"2023-10-08T16:10:04.866867+0000","last_peered":"2023-10-08T16:10:04.866867+0000","last_clean":"2023-10-08T16:10:04.866867+0000","last_became_active":"2023-10-08T16:10:04.862642+0000","last_became_peered":"2023-10-08T16:10:04.862642+0000","last_unstale":"2023-10-08T16:10:04.866867+0000","last_undegraded":"2023-10-08T16:10:04.866867+0000","last_fullsized":"2023-10-08T16:10:04.866867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:18.716614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876387+0000","last_change":"2023-10-08T16:10:03.857135+0000","last_active":"2023-10-08T16:10:04.876387+0000","last_peered":"2023-10-08T16:10:04.876387+0000","last_clean":"2023-10-08T16:10:04.876387+0000","last_became_active":"2023-10-08T16:10:03.856911+0000","last_became_peered":"2023-10-08T16:10:03.856911+0000","last_unstale":"2023-10-08T16:10:04.876387+0000","last_undegraded":"2023-10-08T16:10:04.876387+0000","last_fullsized":"2023-10-08T16:10:04.876387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876738+0000","last_change":"2023-10-08T16:10:04.872848+0000","last_active":"2023-10-08T16:10:04.876738+0000","last_peered":"2023-10-08T16:10:04.876738+0000","last_clean":"2023-10-08T16:10:04.876738+0000","last_became_active":"2023-10-08T16:10:04.872593+0000","last_became_peered":"2023-10-08T16:10:04.872593+0000","last_unstale":"2023-10-08T16:10:04.876738+0000","last_undegraded":"2023-10-08T16:10:04.876738+0000","last_fullsized":"2023-10-08T16:10:04.876738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:21.771026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872486+0000","last_change":"2023-10-08T16:10:03.859382+0000","last_active":"2023-10-08T16:10:04.872486+0000","last_peered":"2023-10-08T16:10:04.872486+0000","last_clean":"2023-10-08T16:10:04.872486+0000","last_became_active":"2023-10-08T16:10:03.859137+0000","last_became_peered":"2023-10-08T16:10:03.859137+0000","last_unstale":"2023-10-08T16:10:04.872486+0000","last_undegraded":"2023-10-08T16:10:04.872486+0000","last_fullsized":"2023-10-08T16:10:04.872486+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861516+0000","last_change":"2023-10-08T16:10:04.860545+0000","last_active":"2023-10-08T16:10:04.861516+0000","last_peered":"2023-10-08T16:10:04.861516+0000","last_clean":"2023-10-08T16:10:04.861516+0000","last_became_active":"2023-10-08T16:10:04.860262+0000","last_became_peered":"2023-10-08T16:10:04.860262+0000","last_unstale":"2023-10-08T16:10:04.861516+0000","last_undegraded":"2023-10-08T16:10:04.861516+0000","last_fullsized":"2023-10-08T16:10:04.861516+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.856728+0000","last_change":"2023-10-08T16:10:03.863315+0000","last_active":"2023-10-08T16:10:04.856728+0000","last_peered":"2023-10-08T16:10:04.856728+0000","last_clean":"2023-10-08T16:10:04.856728+0000","last_became_active":"2023-10-08T16:10:03.863021+0000","last_became_peered":"2023-10-08T16:10:03.863021+0000","last_unstale":"2023-10-08T16:10:04.856728+0000","last_undegraded":"2023-10-08T16:10:04.856728+0000","last_fullsized":"2023-10-08T16:10:04.856728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875168+0000","last_change":"2023-10-08T16:10:04.867758+0000","last_active":"2023-10-08T16:10:04.875168+0000","last_peered":"2023-10-08T16:10:04.875168+0000","last_clean":"2023-10-08T16:10:04.875168+0000","last_became_active":"2023-10-08T16:10:04.867480+0000","last_became_peered":"2023-10-08T16:10:04.867480+0000","last_unstale":"2023-10-08T16:10:04.875168+0000","last_undegraded":"2023-10-08T16:10:04.875168+0000","last_fullsized":"2023-10-08T16:10:04.875168+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:43:52.232028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.845232+0000","last_change":"2023-10-08T16:10:03.850832+0000","last_active":"2023-10-08T16:10:04.845232+0000","last_peered":"2023-10-08T16:10:04.845232+0000","last_clean":"2023-10-08T16:10:04.845232+0000","last_became_active":"2023-10-08T16:10:03.850090+0000","last_became_peered":"2023-10-08T16:10:03.850090+0000","last_unstale":"2023-10-08T16:10:04.845232+0000","last_undegraded":"2023-10-08T16:10:04.845232+0000","last_fullsized":"2023-10-08T16:10:04.845232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:54:14.511584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862608+0000","last_change":"2023-10-08T16:10:04.861486+0000","last_active":"2023-10-08T16:10:04.862608+0000","last_peered":"2023-10-08T16:10:04.862608+0000","last_clean":"2023-10-08T16:10:04.862608+0000","last_became_active":"2023-10-08T16:10:04.861197+0000","last_became_peered":"2023-10-08T16:10:04.861197+0000","last_unstale":"2023-10-08T16:10:04.862608+0000","last_undegraded":"2023-10-08T16:10:04.862608+0000","last_fullsized":"2023-10-08T16:10:04.862608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:15.955435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.846162+0000","last_change":"2023-10-08T16:10:03.853913+0000","last_active":"2023-10-08T16:10:04.846162+0000","last_peered":"2023-10-08T16:10:04.846162+0000","last_clean":"2023-10-08T16:10:04.846162+0000","last_became_active":"2023-10-08T16:10:03.853638+0000","last_became_peered":"2023-10-08T16:10:03.853638+0000","last_unstale":"2023-10-08T16:10:04.846162+0000","last_undegraded":"2023-10-08T16:10:04.846162+0000","last_fullsized":"2023-10-08T16:10:04.846162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876324+0000","last_change":"2023-10-08T16:10:04.873255+0000","last_active":"2023-10-08T16:10:04.876324+0000","last_peered":"2023-10-08T16:10:04.876324+0000","last_clean":"2023-10-08T16:10:04.876324+0000","last_became_active":"2023-10-08T16:10:04.872996+0000","last_became_peered":"2023-10-08T16:10:04.872996+0000","last_unstale":"2023-10-08T16:10:04.876324+0000","last_undegraded":"2023-10-08T16:10:04.876324+0000","last_fullsized":"2023-10-08T16:10:04.876324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:25:09.875311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854031+0000","last_change":"2023-10-08T16:10:03.867602+0000","last_active":"2023-10-08T16:10:04.854031+0000","last_peered":"2023-10-08T16:10:04.854031+0000","last_clean":"2023-10-08T16:10:04.854031+0000","last_became_active":"2023-10-08T16:10:03.867359+0000","last_became_peered":"2023-10-08T16:10:03.867359+0000","last_unstale":"2023-10-08T16:10:04.854031+0000","last_undegraded":"2023-10-08T16:10:04.854031+0000","last_fullsized":"2023-10-08T16:10:04.854031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875344+0000","last_change":"2023-10-08T16:10:04.868282+0000","last_active":"2023-10-08T16:10:04.875344+0000","last_peered":"2023-10-08T16:10:04.875344+0000","last_clean":"2023-10-08T16:10:04.875344+0000","last_became_active":"2023-10-08T16:10:04.868078+0000","last_became_peered":"2023-10-08T16:10:04.868078+0000","last_unstale":"2023-10-08T16:10:04.875344+0000","last_undegraded":"2023-10-08T16:10:04.875344+0000","last_fullsized":"2023-10-08T16:10:04.875344+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:47:14.723247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"88'3","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.877059+0000","last_change":"2023-10-08T16:10:03.862435+0000","last_active":"2023-10-08T16:10:04.877059+0000","last_peered":"2023-10-08T16:10:04.877059+0000","last_clean":"2023-10-08T16:10:04.877059+0000","last_became_active":"2023-10-08T16:10:03.862150+0000","last_became_peered":"2023-10-08T16:10:03.862150+0000","last_unstale":"2023-10-08T16:10:04.877059+0000","last_undegraded":"2023-10-08T16:10:04.877059+0000","last_fullsized":"2023-10-08T16:10:04.877059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:22.362510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.866232+0000","last_change":"2023-10-08T16:10:04.859540+0000","last_active":"2023-10-08T16:10:04.866232+0000","last_peered":"2023-10-08T16:10:04.866232+0000","last_clean":"2023-10-08T16:10:04.866232+0000","last_became_active":"2023-10-08T16:10:04.859292+0000","last_became_peered":"2023-10-08T16:10:04.859292+0000","last_unstale":"2023-10-08T16:10:04.866232+0000","last_undegraded":"2023-10-08T16:10:04.866232+0000","last_fullsized":"2023-10-08T16:10:04.866232+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:08:43.947260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"88'6","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.878368+0000","last_change":"2023-10-08T16:10:03.859113+0000","last_active":"2023-10-08T16:10:04.878368+0000","last_peered":"2023-10-08T16:10:04.878368+0000","last_clean":"2023-10-08T16:10:04.878368+0000","last_became_active":"2023-10-08T16:10:03.858920+0000","last_became_peered":"2023-10-08T16:10:03.858920+0000","last_unstale":"2023-10-08T16:10:04.878368+0000","last_undegraded":"2023-10-08T16:10:04.878368+0000","last_fullsized":"2023-10-08T16:10:04.878368+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:12:23.618518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861378+0000","last_change":"2023-10-08T16:10:04.860406+0000","last_active":"2023-10-08T16:10:04.861378+0000","last_peered":"2023-10-08T16:10:04.861378+0000","last_clean":"2023-10-08T16:10:04.861378+0000","last_became_active":"2023-10-08T16:10:04.860162+0000","last_became_peered":"2023-10-08T16:10:04.860162+0000","last_unstale":"2023-10-08T16:10:04.861378+0000","last_undegraded":"2023-10-08T16:10:04.861378+0000","last_fullsized":"2023-10-08T16:10:04.861378+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:12:54.875836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876245+0000","last_change":"2023-10-08T16:10:03.860299+0000","last_active":"2023-10-08T16:10:04.876245+0000","last_peered":"2023-10-08T16:10:04.876245+0000","last_clean":"2023-10-08T16:10:04.876245+0000","last_became_active":"2023-10-08T16:10:03.860082+0000","last_became_peered":"2023-10-08T16:10:03.860082+0000","last_unstale":"2023-10-08T16:10:04.876245+0000","last_undegraded":"2023-10-08T16:10:04.876245+0000","last_fullsized":"2023-10-08T16:10:04.876245+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:24.121795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858636+0000","last_change":"2023-10-08T16:10:04.855343+0000","last_active":"2023-10-08T16:10:04.858636+0000","last_peered":"2023-10-08T16:10:04.858636+0000","last_clean":"2023-10-08T16:10:04.858636+0000","last_became_active":"2023-10-08T16:10:04.854749+0000","last_became_peered":"2023-10-08T16:10:04.854749+0000","last_unstale":"2023-10-08T16:10:04.858636+0000","last_undegraded":"2023-10-08T16:10:04.858636+0000","last_fullsized":"2023-10-08T16:10:04.858636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:34:32.803419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849044+0000","last_change":"2023-10-08T16:10:03.857314+0000","last_active":"2023-10-08T16:10:04.849044+0000","last_peered":"2023-10-08T16:10:04.849044+0000","last_clean":"2023-10-08T16:10:04.849044+0000","last_became_active":"2023-10-08T16:10:03.857042+0000","last_became_peered":"2023-10-08T16:10:03.857042+0000","last_unstale":"2023-10-08T16:10:04.849044+0000","last_undegraded":"2023-10-08T16:10:04.849044+0000","last_fullsized":"2023-10-08T16:10:04.849044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:26:46.752089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874886+0000","last_change":"2023-10-08T16:10:04.867050+0000","last_active":"2023-10-08T16:10:04.874886+0000","last_peered":"2023-10-08T16:10:04.874886+0000","last_clean":"2023-10-08T16:10:04.874886+0000","last_became_active":"2023-10-08T16:10:04.866800+0000","last_became_peered":"2023-10-08T16:10:04.866800+0000","last_unstale":"2023-10-08T16:10:04.874886+0000","last_undegraded":"2023-10-08T16:10:04.874886+0000","last_fullsized":"2023-10-08T16:10:04.874886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:50.985190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858128+0000","last_change":"2023-10-08T16:10:03.862333+0000","last_active":"2023-10-08T16:10:04.858128+0000","last_peered":"2023-10-08T16:10:04.858128+0000","last_clean":"2023-10-08T16:10:04.858128+0000","last_became_active":"2023-10-08T16:10:03.862068+0000","last_became_peered":"2023-10-08T16:10:03.862068+0000","last_unstale":"2023-10-08T16:10:04.858128+0000","last_undegraded":"2023-10-08T16:10:04.858128+0000","last_fullsized":"2023-10-08T16:10:04.858128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:30.932977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865217+0000","last_change":"2023-10-08T16:10:04.863043+0000","last_active":"2023-10-08T16:10:04.865217+0000","last_peered":"2023-10-08T16:10:04.865217+0000","last_clean":"2023-10-08T16:10:04.865217+0000","last_became_active":"2023-10-08T16:10:04.862804+0000","last_became_peered":"2023-10-08T16:10:04.862804+0000","last_unstale":"2023-10-08T16:10:04.865217+0000","last_undegraded":"2023-10-08T16:10:04.865217+0000","last_fullsized":"2023-10-08T16:10:04.865217+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:47:17.060804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"88'8","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.877819+0000","last_change":"2023-10-08T16:10:03.855130+0000","last_active":"2023-10-08T16:10:04.877819+0000","last_peered":"2023-10-08T16:10:04.877819+0000","last_clean":"2023-10-08T16:10:04.877819+0000","last_became_active":"2023-10-08T16:10:03.854858+0000","last_became_peered":"2023-10-08T16:10:03.854858+0000","last_unstale":"2023-10-08T16:10:04.877819+0000","last_undegraded":"2023-10-08T16:10:04.877819+0000","last_fullsized":"2023-10-08T16:10:04.877819+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:25.607176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875457+0000","last_change":"2023-10-08T16:10:04.872815+0000","last_active":"2023-10-08T16:10:04.875457+0000","last_peered":"2023-10-08T16:10:04.875457+0000","last_clean":"2023-10-08T16:10:04.875457+0000","last_became_active":"2023-10-08T16:10:04.872571+0000","last_became_peered":"2023-10-08T16:10:04.872571+0000","last_unstale":"2023-10-08T16:10:04.875457+0000","last_undegraded":"2023-10-08T16:10:04.875457+0000","last_fullsized":"2023-10-08T16:10:04.875457+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:56:23.368890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.870666+0000","last_change":"2023-10-08T16:10:03.859396+0000","last_active":"2023-10-08T16:10:04.870666+0000","last_peered":"2023-10-08T16:10:04.870666+0000","last_clean":"2023-10-08T16:10:04.870666+0000","last_became_active":"2023-10-08T16:10:03.859124+0000","last_became_peered":"2023-10-08T16:10:03.859124+0000","last_unstale":"2023-10-08T16:10:04.870666+0000","last_undegraded":"2023-10-08T16:10:04.870666+0000","last_fullsized":"2023-10-08T16:10:04.870666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:57.097347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.873775+0000","last_change":"2023-10-08T16:10:04.869805+0000","last_active":"2023-10-08T16:10:04.873775+0000","last_peered":"2023-10-08T16:10:04.873775+0000","last_clean":"2023-10-08T16:10:04.873775+0000","last_became_active":"2023-10-08T16:10:04.869557+0000","last_became_peered":"2023-10-08T16:10:04.869557+0000","last_unstale":"2023-10-08T16:10:04.873775+0000","last_undegraded":"2023-10-08T16:10:04.873775+0000","last_fullsized":"2023-10-08T16:10:04.873775+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:31:29.308842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876369+0000","last_change":"2023-10-08T16:10:03.854991+0000","last_active":"2023-10-08T16:10:04.876369+0000","last_peered":"2023-10-08T16:10:04.876369+0000","last_clean":"2023-10-08T16:10:04.876369+0000","last_became_active":"2023-10-08T16:10:03.854728+0000","last_became_peered":"2023-10-08T16:10:03.854728+0000","last_unstale":"2023-10-08T16:10:04.876369+0000","last_undegraded":"2023-10-08T16:10:04.876369+0000","last_fullsized":"2023-10-08T16:10:04.876369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:27.298977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876657+0000","last_change":"2023-10-08T16:10:04.866715+0000","last_active":"2023-10-08T16:10:04.876657+0000","last_peered":"2023-10-08T16:10:04.876657+0000","last_clean":"2023-10-08T16:10:04.876657+0000","last_became_active":"2023-10-08T16:10:04.866413+0000","last_became_peered":"2023-10-08T16:10:04.866413+0000","last_unstale":"2023-10-08T16:10:04.876657+0000","last_undegraded":"2023-10-08T16:10:04.876657+0000","last_fullsized":"2023-10-08T16:10:04.876657+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:15.955435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851463+0000","last_change":"2023-10-08T16:10:03.867125+0000","last_active":"2023-10-08T16:10:04.851463+0000","last_peered":"2023-10-08T16:10:04.851463+0000","last_clean":"2023-10-08T16:10:04.851463+0000","last_became_active":"2023-10-08T16:10:03.866922+0000","last_became_peered":"2023-10-08T16:10:03.866922+0000","last_unstale":"2023-10-08T16:10:04.851463+0000","last_undegraded":"2023-10-08T16:10:04.851463+0000","last_fullsized":"2023-10-08T16:10:04.851463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:27.402328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862528+0000","last_change":"2023-10-08T16:10:04.856753+0000","last_active":"2023-10-08T16:10:04.862528+0000","last_peered":"2023-10-08T16:10:04.862528+0000","last_clean":"2023-10-08T16:10:04.862528+0000","last_became_active":"2023-10-08T16:10:04.856499+0000","last_became_peered":"2023-10-08T16:10:04.856499+0000","last_unstale":"2023-10-08T16:10:04.862528+0000","last_undegraded":"2023-10-08T16:10:04.862528+0000","last_fullsized":"2023-10-08T16:10:04.862528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:53.737297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871003+0000","last_change":"2023-10-08T16:10:03.862486+0000","last_active":"2023-10-08T16:10:04.871003+0000","last_peered":"2023-10-08T16:10:04.871003+0000","last_clean":"2023-10-08T16:10:04.871003+0000","last_became_active":"2023-10-08T16:10:03.862165+0000","last_became_peered":"2023-10-08T16:10:03.862165+0000","last_unstale":"2023-10-08T16:10:04.871003+0000","last_undegraded":"2023-10-08T16:10:04.871003+0000","last_fullsized":"2023-10-08T16:10:04.871003+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861826+0000","last_change":"2023-10-08T16:10:04.859488+0000","last_active":"2023-10-08T16:10:04.861826+0000","last_peered":"2023-10-08T16:10:04.861826+0000","last_clean":"2023-10-08T16:10:04.861826+0000","last_became_active":"2023-10-08T16:10:04.859243+0000","last_became_peered":"2023-10-08T16:10:04.859243+0000","last_unstale":"2023-10-08T16:10:04.861826+0000","last_undegraded":"2023-10-08T16:10:04.861826+0000","last_fullsized":"2023-10-08T16:10:04.861826+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:56:26.662940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849775+0000","last_change":"2023-10-08T16:10:03.855666+0000","last_active":"2023-10-08T16:10:04.849775+0000","last_peered":"2023-10-08T16:10:04.849775+0000","last_clean":"2023-10-08T16:10:04.849775+0000","last_became_active":"2023-10-08T16:10:03.855286+0000","last_became_peered":"2023-10-08T16:10:03.855286+0000","last_unstale":"2023-10-08T16:10:04.849775+0000","last_undegraded":"2023-10-08T16:10:04.849775+0000","last_fullsized":"2023-10-08T16:10:04.849775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.861766+0000","last_change":"2023-10-08T16:10:04.856146+0000","last_active":"2023-10-08T16:10:04.861766+0000","last_peered":"2023-10-08T16:10:04.861766+0000","last_clean":"2023-10-08T16:10:04.861766+0000","last_became_active":"2023-10-08T16:10:04.855873+0000","last_became_peered":"2023-10-08T16:10:04.855873+0000","last_unstale":"2023-10-08T16:10:04.861766+0000","last_undegraded":"2023-10-08T16:10:04.861766+0000","last_fullsized":"2023-10-08T16:10:04.861766+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:58.668805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.849736+0000","last_change":"2023-10-08T16:10:03.857899+0000","last_active":"2023-10-08T16:10:04.849736+0000","last_peered":"2023-10-08T16:10:04.849736+0000","last_clean":"2023-10-08T16:10:04.849736+0000","last_became_active":"2023-10-08T16:10:03.857621+0000","last_became_peered":"2023-10-08T16:10:03.857621+0000","last_unstale":"2023-10-08T16:10:04.849736+0000","last_undegraded":"2023-10-08T16:10:04.849736+0000","last_fullsized":"2023-10-08T16:10:04.849736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:49.915019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874864+0000","last_change":"2023-10-08T16:10:04.868005+0000","last_active":"2023-10-08T16:10:04.874864+0000","last_peered":"2023-10-08T16:10:04.874864+0000","last_clean":"2023-10-08T16:10:04.874864+0000","last_became_active":"2023-10-08T16:10:04.867742+0000","last_became_peered":"2023-10-08T16:10:04.867742+0000","last_unstale":"2023-10-08T16:10:04.874864+0000","last_undegraded":"2023-10-08T16:10:04.874864+0000","last_fullsized":"2023-10-08T16:10:04.874864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:51.898348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851678+0000","last_change":"2023-10-08T16:10:03.855883+0000","last_active":"2023-10-08T16:10:04.851678+0000","last_peered":"2023-10-08T16:10:04.851678+0000","last_clean":"2023-10-08T16:10:04.851678+0000","last_became_active":"2023-10-08T16:10:03.855617+0000","last_became_peered":"2023-10-08T16:10:03.855617+0000","last_unstale":"2023-10-08T16:10:04.851678+0000","last_undegraded":"2023-10-08T16:10:04.851678+0000","last_fullsized":"2023-10-08T16:10:04.851678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:14.237332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.871394+0000","last_change":"2023-10-08T16:10:04.867487+0000","last_active":"2023-10-08T16:10:04.871394+0000","last_peered":"2023-10-08T16:10:04.871394+0000","last_clean":"2023-10-08T16:10:04.871394+0000","last_became_active":"2023-10-08T16:10:04.865857+0000","last_became_peered":"2023-10-08T16:10:04.865857+0000","last_unstale":"2023-10-08T16:10:04.871394+0000","last_undegraded":"2023-10-08T16:10:04.871394+0000","last_fullsized":"2023-10-08T16:10:04.871394+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:21.771026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"88'2","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876256+0000","last_change":"2023-10-08T16:10:03.859539+0000","last_active":"2023-10-08T16:10:04.876256+0000","last_peered":"2023-10-08T16:10:04.876256+0000","last_clean":"2023-10-08T16:10:04.876256+0000","last_became_active":"2023-10-08T16:10:03.859277+0000","last_became_peered":"2023-10-08T16:10:03.859277+0000","last_unstale":"2023-10-08T16:10:04.876256+0000","last_undegraded":"2023-10-08T16:10:04.876256+0000","last_fullsized":"2023-10-08T16:10:04.876256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:47:58.278821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.850046+0000","last_change":"2023-10-08T16:10:04.849722+0000","last_active":"2023-10-08T16:10:04.850046+0000","last_peered":"2023-10-08T16:10:04.850046+0000","last_clean":"2023-10-08T16:10:04.850046+0000","last_became_active":"2023-10-08T16:10:04.849448+0000","last_became_peered":"2023-10-08T16:10:04.849448+0000","last_unstale":"2023-10-08T16:10:04.850046+0000","last_undegraded":"2023-10-08T16:10:04.850046+0000","last_fullsized":"2023-10-08T16:10:04.850046+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:44:58.103727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857374+0000","last_change":"2023-10-08T16:10:03.867454+0000","last_active":"2023-10-08T16:10:04.857374+0000","last_peered":"2023-10-08T16:10:04.857374+0000","last_clean":"2023-10-08T16:10:04.857374+0000","last_became_active":"2023-10-08T16:10:03.867150+0000","last_became_peered":"2023-10-08T16:10:03.867150+0000","last_unstale":"2023-10-08T16:10:04.857374+0000","last_undegraded":"2023-10-08T16:10:04.857374+0000","last_fullsized":"2023-10-08T16:10:04.857374+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:03:49.978809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.872897+0000","last_change":"2023-10-08T16:10:04.869601+0000","last_active":"2023-10-08T16:10:04.872897+0000","last_peered":"2023-10-08T16:10:04.872897+0000","last_clean":"2023-10-08T16:10:04.872897+0000","last_became_active":"2023-10-08T16:10:04.869348+0000","last_became_peered":"2023-10-08T16:10:04.869348+0000","last_unstale":"2023-10-08T16:10:04.872897+0000","last_undegraded":"2023-10-08T16:10:04.872897+0000","last_fullsized":"2023-10-08T16:10:04.872897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:47.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851335+0000","last_change":"2023-10-08T16:10:03.863109+0000","last_active":"2023-10-08T16:10:04.851335+0000","last_peered":"2023-10-08T16:10:04.851335+0000","last_clean":"2023-10-08T16:10:04.851335+0000","last_became_active":"2023-10-08T16:10:03.862451+0000","last_became_peered":"2023-10-08T16:10:03.862451+0000","last_unstale":"2023-10-08T16:10:04.851335+0000","last_undegraded":"2023-10-08T16:10:04.851335+0000","last_fullsized":"2023-10-08T16:10:04.851335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:46.037098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.874773+0000","last_change":"2023-10-08T16:10:04.867578+0000","last_active":"2023-10-08T16:10:04.874773+0000","last_peered":"2023-10-08T16:10:04.874773+0000","last_clean":"2023-10-08T16:10:04.874773+0000","last_became_active":"2023-10-08T16:10:04.867296+0000","last_became_peered":"2023-10-08T16:10:04.867296+0000","last_unstale":"2023-10-08T16:10:04.874773+0000","last_undegraded":"2023-10-08T16:10:04.874773+0000","last_fullsized":"2023-10-08T16:10:04.874773+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:09:28.408708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.854981+0000","last_change":"2023-10-08T16:10:03.857051+0000","last_active":"2023-10-08T16:10:04.854981+0000","last_peered":"2023-10-08T16:10:04.854981+0000","last_clean":"2023-10-08T16:10:04.854981+0000","last_became_active":"2023-10-08T16:10:03.856770+0000","last_became_peered":"2023-10-08T16:10:03.856770+0000","last_unstale":"2023-10-08T16:10:04.854981+0000","last_undegraded":"2023-10-08T16:10:04.854981+0000","last_fullsized":"2023-10-08T16:10:04.854981+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:04:57.885395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865103+0000","last_change":"2023-10-08T16:10:04.862789+0000","last_active":"2023-10-08T16:10:04.865103+0000","last_peered":"2023-10-08T16:10:04.865103+0000","last_clean":"2023-10-08T16:10:04.865103+0000","last_became_active":"2023-10-08T16:10:04.862526+0000","last_became_peered":"2023-10-08T16:10:04.862526+0000","last_unstale":"2023-10-08T16:10:04.865103+0000","last_undegraded":"2023-10-08T16:10:04.865103+0000","last_fullsized":"2023-10-08T16:10:04.865103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:32:12.016581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.858151+0000","last_change":"2023-10-08T16:10:03.859409+0000","last_active":"2023-10-08T16:10:04.858151+0000","last_peered":"2023-10-08T16:10:04.858151+0000","last_clean":"2023-10-08T16:10:04.858151+0000","last_became_active":"2023-10-08T16:10:03.859104+0000","last_became_peered":"2023-10-08T16:10:03.859104+0000","last_unstale":"2023-10-08T16:10:04.858151+0000","last_undegraded":"2023-10-08T16:10:04.858151+0000","last_fullsized":"2023-10-08T16:10:04.858151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875874+0000","last_change":"2023-10-08T16:10:04.870211+0000","last_active":"2023-10-08T16:10:04.875874+0000","last_peered":"2023-10-08T16:10:04.875874+0000","last_clean":"2023-10-08T16:10:04.875874+0000","last_became_active":"2023-10-08T16:10:04.869907+0000","last_became_peered":"2023-10-08T16:10:04.869907+0000","last_unstale":"2023-10-08T16:10:04.875874+0000","last_undegraded":"2023-10-08T16:10:04.875874+0000","last_fullsized":"2023-10-08T16:10:04.875874+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:21:03.206335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.857857+0000","last_change":"2023-10-08T16:10:03.857623+0000","last_active":"2023-10-08T16:10:04.857857+0000","last_peered":"2023-10-08T16:10:04.857857+0000","last_clean":"2023-10-08T16:10:04.857857+0000","last_became_active":"2023-10-08T16:10:03.857351+0000","last_became_peered":"2023-10-08T16:10:03.857351+0000","last_unstale":"2023-10-08T16:10:04.857857+0000","last_undegraded":"2023-10-08T16:10:04.857857+0000","last_fullsized":"2023-10-08T16:10:04.857857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:15.506453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.865135+0000","last_change":"2023-10-08T16:10:04.862545+0000","last_active":"2023-10-08T16:10:04.865135+0000","last_peered":"2023-10-08T16:10:04.865135+0000","last_clean":"2023-10-08T16:10:04.865135+0000","last_became_active":"2023-10-08T16:10:04.862294+0000","last_became_peered":"2023-10-08T16:10:04.862294+0000","last_unstale":"2023-10-08T16:10:04.865135+0000","last_undegraded":"2023-10-08T16:10:04.865135+0000","last_fullsized":"2023-10-08T16:10:04.865135+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:32.632927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.851485+0000","last_change":"2023-10-08T16:10:03.856881+0000","last_active":"2023-10-08T16:10:04.851485+0000","last_peered":"2023-10-08T16:10:04.851485+0000","last_clean":"2023-10-08T16:10:04.851485+0000","last_became_active":"2023-10-08T16:10:03.856607+0000","last_became_peered":"2023-10-08T16:10:03.856607+0000","last_unstale":"2023-10-08T16:10:04.851485+0000","last_undegraded":"2023-10-08T16:10:04.851485+0000","last_fullsized":"2023-10-08T16:10:04.851485+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:50.891968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.875286+0000","last_change":"2023-10-08T16:10:04.869172+0000","last_active":"2023-10-08T16:10:04.875286+0000","last_peered":"2023-10-08T16:10:04.875286+0000","last_clean":"2023-10-08T16:10:04.875286+0000","last_became_active":"2023-10-08T16:10:04.864575+0000","last_became_peered":"2023-10-08T16:10:04.864575+0000","last_unstale":"2023-10-08T16:10:04.875286+0000","last_undegraded":"2023-10-08T16:10:04.875286+0000","last_fullsized":"2023-10-08T16:10:04.875286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:42:34.687614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.847916+0000","last_change":"2023-10-08T16:10:03.863035+0000","last_active":"2023-10-08T16:10:04.847916+0000","last_peered":"2023-10-08T16:10:04.847916+0000","last_clean":"2023-10-08T16:10:04.847916+0000","last_became_active":"2023-10-08T16:10:03.862801+0000","last_became_peered":"2023-10-08T16:10:03.862801+0000","last_unstale":"2023-10-08T16:10:04.847916+0000","last_undegraded":"2023-10-08T16:10:04.847916+0000","last_fullsized":"2023-10-08T16:10:04.847916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:56:31.485100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.862224+0000","last_change":"2023-10-08T16:10:04.856941+0000","last_active":"2023-10-08T16:10:04.862224+0000","last_peered":"2023-10-08T16:10:04.862224+0000","last_clean":"2023-10-08T16:10:04.862224+0000","last_became_active":"2023-10-08T16:10:04.856694+0000","last_became_peered":"2023-10-08T16:10:04.856694+0000","last_unstale":"2023-10-08T16:10:04.862224+0000","last_undegraded":"2023-10-08T16:10:04.862224+0000","last_fullsized":"2023-10-08T16:10:04.862224+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:03.838041+0000","last_clean_scrub_stamp":"2023-10-08T16:10:03.838041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:19:39.023795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"88'1","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.876559+0000","last_change":"2023-10-08T16:10:03.856920+0000","last_active":"2023-10-08T16:10:04.876559+0000","last_peered":"2023-10-08T16:10:04.876559+0000","last_clean":"2023-10-08T16:10:04.876559+0000","last_became_active":"2023-10-08T16:10:03.856673+0000","last_became_peered":"2023-10-08T16:10:03.856673+0000","last_unstale":"2023-10-08T16:10:04.876559+0000","last_undegraded":"2023-10-08T16:10:04.876559+0000","last_fullsized":"2023-10-08T16:10:04.876559+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:02.831661+0000","last_clean_scrub_stamp":"2023-10-08T16:10:02.831661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:50:20.462195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-08T16:10:04.853991+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:04.853991+0000","last_peered":"2023-10-08T16:10:04.853991+0000","last_clean":"2023-10-08T16:10:04.853991+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:04.853991+0000","last_undegraded":"2023-10-08T16:10:04.853991+0000","last_fullsized":"2023-10-08T16:10:04.853991+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298952,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072888,"statfs":{"total":96636764160,"available":96330637312,"internally_reserved":0,"allocated":1703936,"data_stored":1064767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.58599999999999997,"15min":0.80800000000000005},"min":{"1min":0.308,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67300000000000004,"5min":8.0939999999999994,"15min":8.0939999999999994},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52100000000000002,"15min":0.54000000000000004},"min":{"1min":0.35099999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63300000000000001,"5min":1.083,"15min":1.083},"last":0.75},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59999999999999998,"15min":0.84599999999999997},"min":{"1min":0.32900000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69099999999999995,"5min":8.2040000000000006,"15min":8.2040000000000006},"last":0.52700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.50700000000000001,"15min":0.54400000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":1.175,"15min":1.175},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.62,"15min":0.871},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78000000000000003,"5min":8.4339999999999993,"15min":8.4339999999999993},"last":0.72099999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52800000000000002,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73099999999999998,"5min":1.214,"15min":1.214},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.63400000000000001,"15min":0.875},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.68600000000000005,"5min":8.3409999999999993,"15min":8.3409999999999993},"last":0.69199999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.45400000000000001,"15min":0.48999999999999999},"min":{"1min":0.29899999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.55700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65200000000000002,"5min":0.752,"15min":0.752},"last":0.65100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.315,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78400000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57099999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69599999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56299999999999994,"15min":0.58599999999999997},"min":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.70399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298372,"kb_used_data":1084,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073468,"statfs":{"total":96636764160,"available":96331231232,"internally_reserved":0,"allocated":1110016,"data_stored":474911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70799999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.497},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.42699999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302460,"kb_used_data":1076,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069380,"statfs":{"total":96636764160,"available":96327045120,"internally_reserved":0,"allocated":1101824,"data_stored":474287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.308}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42899999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.58999999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60099999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.625},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.53100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303044,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068796,"statfs":{"total":96636764160,"available":96326447104,"internally_reserved":0,"allocated":1699840,"data_stored":1065209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.372},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.44400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.42599999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.85699999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302464,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069376,"statfs":{"total":96636764160,"available":96327041024,"internally_reserved":0,"allocated":1105920,"data_stored":474321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.38100000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.312},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.627},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.749}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.64700000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.57299999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574919,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":475845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.437},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.45700000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.54600000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.70299999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.67100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":475947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.68300000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.69799999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65300000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71299999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.495},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.74399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298368,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073472,"statfs":{"total":96636764160,"available":96331235328,"internally_reserved":0,"allocated":1105920,"data_stored":474357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.54700000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.39900000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.5}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.42799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:13.850 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:10:13.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:10:13.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.971+0000 7f53e4e56700 1 Processor -- start 2023-10-08T16:10:13.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.971+0000 7f53e4e56700 1 -- start start 2023-10-08T16:10:13.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53e4e56700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f53e0159950 0x7f53e00cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53e4e56700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 0x7f53e0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53e4e56700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e00ce380 con 0x7f53e0069aa0 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53e4e56700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e00cec20 con 0x7f53e0159950 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53e4e56700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e0069ee0 con 0x7f53e00af320 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f53e0159950 0x7f53e00cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53ded9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f53e0069aa0 0x7f53e015d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45638/0 (socket says 172.21.15.170:45638) 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53ded9d700 1 -- 172.21.15.170:0/552706197 learned_addr learned my addr 172.21.15.170:0/552706197 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 --2- 172.21.15.170:0/552706197 >> v2:172.21.15.170:3300/0 conn(0x7f53e0159950 0x7f53e00cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49522/0 (socket says 172.21.15.170:49522) 2023-10-08T16:10:13.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53d5d9b700 1 --2- 172.21.15.170:0/552706197 >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 0x7f53e0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 -- 172.21.15.170:0/552706197 >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 msgr2=0x7f53e0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:13.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 --2- 172.21.15.170:0/552706197 >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 0x7f53e0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 -- 172.21.15.170:0/552706197 >> v1:172.21.15.170:6789/0 conn(0x7f53e0069aa0 legacy=0x7f53e015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53ddd9b700 1 -- 172.21.15.170:0/552706197 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288731838 0 0) 0x7f53e00ce380 con 0x7f53e0069aa0 2023-10-08T16:10:13.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 -- 172.21.15.170:0/552706197 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00afa90 con 0x7f53e0159950 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53d5d9b700 1 --2- 172.21.15.170:0/552706197 >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 0x7f53e0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53de59c700 1 --2- 172.21.15.170:0/552706197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e00cbb90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f53e015d450 tx=0x7f53d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b1f9120de9e19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53ddd9b700 1 -- 172.21.15.170:0/552706197 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000b6b0 con 0x7f53e0159950 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.972+0000 7f53ddd9b700 1 -- 172.21.15.170:0/552706197 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000b810 con 0x7f53e0159950 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53ddd9b700 1 -- 172.21.15.170:0/552706197 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000bbb0 con 0x7f53e0159950 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 -- 172.21.15.170:0/552706197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 msgr2=0x7f53e00cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 --2- 172.21.15.170:0/552706197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e00cbb90 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f53e015d450 tx=0x7f53d00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 -- 172.21.15.170:0/552706197 shutdown_connections 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 --2- 172.21.15.170:0/552706197 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e00cbb90 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 --2- 172.21.15.170:0/552706197 >> v2:172.21.15.191:3300/0 conn(0x7f53e00af320 0x7f53e0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 -- 172.21.15.170:0/552706197 >> 172.21.15.170:0/552706197 conn(0x7f53e0068e00 msgr2=0x7f53e00d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:13.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 -- 172.21.15.170:0/552706197 shutdown_connections 2023-10-08T16:10:13.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.973+0000 7f53e4e56700 1 -- 172.21.15.170:0/552706197 wait complete. 2023-10-08T16:10:13.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.974+0000 7f53e4e56700 1 Processor -- start 2023-10-08T16:10:13.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.974+0000 7f53e4e56700 1 -- start start 2023-10-08T16:10:13.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53e0069aa0 0x7f53e00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53e00af320 0x7f53e015bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b5910 con 0x7f53e0159950 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e00b4a60 con 0x7f53e00af320 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53e4e56700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e0140d80 con 0x7f53e0069aa0 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49532/0 (socket says 172.21.15.170:49532) 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 -- 172.21.15.170:0/3236952117 learned_addr learned my addr 172.21.15.170:0/3236952117 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 -- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53e0069aa0 msgr2=0x7f53e00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53e0069aa0 0x7f53e00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 -- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53e00af320 msgr2=0x7f53e015bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53d5d9b700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53e00af320 0x7f53e015bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53e00af320 0x7f53e015bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 -- 172.21.15.170:0/3236952117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009420 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.975+0000 7f53ded9d700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f53d800cc90 tx=0x7f53d8001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.976+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d800cea0 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.976+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d800ab70 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.976+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e00b4d00 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.976+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d80070f0 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.976+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e00b52c0 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.977+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e00718b0 con 0x7f53e0159950 2023-10-08T16:10:13.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.977+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f53d80074a0 con 0x7f53e0159950 2023-10-08T16:10:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.977+0000 7f53d6ffd700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 0x7f53c00696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.978+0000 7f53de59c700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 0x7f53c00696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.978+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f53d8084e60 con 0x7f53e0159950 2023-10-08T16:10:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.978+0000 7f53de59c700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 0x7f53c00696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f53d00091e0 tx=0x7f53d0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:13.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:13.982+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53d8051e00 con 0x7f53e0159950 2023-10-08T16:10:14.165 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:10:14.168 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:10:14.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.243+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f53e00b5a50 con 0x7f53e0159950 2023-10-08T16:10:14.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.247+0000 7f53d6ffd700 1 -- 172.21.15.170:0/3236952117 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f53d8055420 con 0x7f53e0159950 2023-10-08T16:10:14.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 msgr2=0x7f53c00696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 0x7f53c00696b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f53d00091e0 tx=0x7f53d0017000 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 msgr2=0x7f53e015c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f53d800cc90 tx=0x7f53d8001a80 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 shutdown_connections 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f53c0067200 0x7f53c00696b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53e0159950 0x7f53e015c0b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f53e0069aa0 0x7f53e00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 --2- 172.21.15.170:0/3236952117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53e00af320 0x7f53e015bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 >> 172.21.15.170:0/3236952117 conn(0x7f53e0068e00 msgr2=0x7f53e00cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:14.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 shutdown_connections 2023-10-08T16:10:14.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.249+0000 7f53e4e56700 1 -- 172.21.15.170:0/3236952117 wait complete. 2023-10-08T16:10:14.251 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:10:14.261 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:14.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.416+0000 7f28527a2700 1 Processor -- start 2023-10-08T16:10:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.416+0000 7f28527a2700 1 -- start start 2023-10-08T16:10:14.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28527a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 0x7f284c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28527a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f284c15d230 0x7f284c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28527a2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c15dbd0 con 0x7f284c1597f0 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28527a2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c0b2980 con 0x7f284c15d230 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28527a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0b11d0 con 0x7f284c159420 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f28517a0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 0x7f284c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f2851fa1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f284c15d230 0x7f284c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f2850f9f700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f284c1597f0 0x7f284c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:39256/0 (socket says 172.21.15.170:39256) 2023-10-08T16:10:14.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.417+0000 7f2850f9f700 1 -- 172.21.15.170:0/1835960690 learned_addr learned my addr 172.21.15.170:0/1835960690 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2851fa1700 1 -- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 msgr2=0x7f284c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2851fa1700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 0x7f284c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2851fa1700 1 -- 172.21.15.170:0/1835960690 >> v1:172.21.15.191:6789/0 conn(0x7f284c1597f0 legacy=0x7f284c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2851fa1700 1 -- 172.21.15.170:0/1835960690 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0b14b0 con 0x7f284c15d230 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2843fff700 1 -- 172.21.15.170:0/1835960690 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640805276 0 0) 0x7f283c001110 con 0x7f284c1597f0 2023-10-08T16:10:14.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f28517a0700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 0x7f284c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2851fa1700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c15d230 0x7f284c15d690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28440098e0 tx=0x7f28440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94347bddb2bb1bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2843fff700 1 -- 172.21.15.170:0/1835960690 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400b640 con 0x7f284c15d230 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2843fff700 1 -- 172.21.15.170:0/1835960690 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400b7a0 con 0x7f284c15d230 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f2843fff700 1 -- 172.21.15.170:0/1835960690 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400bb40 con 0x7f284c15d230 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f28527a2700 1 -- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c15d230 msgr2=0x7f284c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.418+0000 7f28527a2700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c15d230 0x7f284c15d690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28440098e0 tx=0x7f28440091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 -- 172.21.15.170:0/1835960690 shutdown_connections 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c15d230 0x7f284c15d690 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 --2- 172.21.15.170:0/1835960690 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c159420 0x7f284c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 -- 172.21.15.170:0/1835960690 >> 172.21.15.170:0/1835960690 conn(0x7f284c0d0660 msgr2=0x7f284c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:14.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 -- 172.21.15.170:0/1835960690 shutdown_connections 2023-10-08T16:10:14.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.419+0000 7f28527a2700 1 -- 172.21.15.170:0/1835960690 wait complete. 2023-10-08T16:10:14.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.420+0000 7f28527a2700 1 Processor -- start 2023-10-08T16:10:14.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.420+0000 7f28527a2700 1 -- start start 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.420+0000 7f28527a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c159420 0x7f284c153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c1597f0 0x7f284c156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15c2c0 con 0x7f284c15d230 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c5dd0 con 0x7f284c159420 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0c54a0 con 0x7f284c1597f0 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49546/0 (socket says 172.21.15.170:49546) 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 -- 172.21.15.170:0/2943739850 learned_addr learned my addr 172.21.15.170:0/2943739850 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28517a0700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c159420 0x7f284c153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2850f9f700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c1597f0 0x7f284c156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 -- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c1597f0 msgr2=0x7f284c156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c1597f0 0x7f284c156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 -- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c159420 msgr2=0x7f284c153710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c159420 0x7f284c153710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 -- 172.21.15.170:0/2943739850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2848009a20 con 0x7f284c15d230 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f2851fa1700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f284c089340 tx=0x7f2844017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400b910 con 0x7f284c15d230 2023-10-08T16:10:14.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284400bab0 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2844025670 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2844009420 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.421+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c0c5a60 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.422+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c0c5f80 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.423+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f2844023070 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.423+0000 7f28417fa700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 0x7f28300696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.423+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2844085dd0 con 0x7f284c15d230 2023-10-08T16:10:14.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.423+0000 7f28517a0700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 0x7f28300696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:14.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.424+0000 7f28517a0700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 0x7f28300696b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2848009fd0 tx=0x7f284800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:14.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.427+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f284c0c5f80 con 0x7f284c15d230 2023-10-08T16:10:14.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.653+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f284c0c5f40 con 0x7f284c15d230 2023-10-08T16:10:14.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.654+0000 7f28417fa700 1 -- 172.21.15.170:0/2943739850 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f2844015090 con 0x7f284c15d230 2023-10-08T16:10:14.655 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:14.655 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":56,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:14.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 msgr2=0x7f28300696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 0x7f28300696b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2848009fd0 tx=0x7f284800b040 comp rx=0 tx=0).stop 2023-10-08T16:10:14.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 msgr2=0x7f284c156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:14.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f284c089340 tx=0x7f2844017000 comp rx=0 tx=0).stop 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 shutdown_connections 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2830067200 0x7f28300696b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f284c15d230 0x7f284c156f60 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f284c159420 0x7f284c153710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 --2- 172.21.15.170:0/2943739850 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f284c1597f0 0x7f284c156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 >> 172.21.15.170:0/2943739850 conn(0x7f284c0d0660 msgr2=0x7f284c0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 shutdown_connections 2023-10-08T16:10:14.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:14.656+0000 7f28527a2700 1 -- 172.21.15.170:0/2943739850 wait complete. 2023-10-08T16:10:14.659 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 56 2023-10-08T16:10:15.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:10:04.228318+0000', 'modified': '2023-10-08T16:10:07.860819+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6393}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6393': {'gid': 6393, 'name': 'd', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6834/3892735210', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 3892735210}, {'type': 'v1', 'addr': '172.21.15.191:6834', 'nonce': 3892735210}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:10:15.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:10:15.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T16:10:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.190+0000 7f4e9ec95700 1 Processor -- start 2023-10-08T16:10:15.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.190+0000 7f4e9ec95700 1 -- start start 2023-10-08T16:10:15.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ec95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e9815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ec95700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4e98069560 0x7f4e980699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ec95700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9815a160 con 0x7f4e98159c60 2023-10-08T16:10:15.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ec95700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e98069f20 con 0x7f4e98069560 2023-10-08T16:10:15.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ec95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980d13a0 con 0x7f4e98159890 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9d232700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4e98069560 0x7f4e980699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e8ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4e98159c60 0x7f4e9815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53716/0 (socket says 172.21.15.170:53716) 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e8ffff700 1 -- 172.21.15.170:0/3098232338 learned_addr learned my addr 172.21.15.170:0/3098232338 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.191+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e9815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9d232700 1 -- 172.21.15.170:0/3098232338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 msgr2=0x7f4e9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9d232700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e9815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9d232700 1 -- 172.21.15.170:0/3098232338 >> v1:172.21.15.170:6789/0 conn(0x7f4e98159c60 legacy=0x7f4e9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9d232700 1 -- 172.21.15.170:0/3098232338 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9815e140 con 0x7f4e98069560 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9d232700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e980699e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e900098e0 tx=0x7f4e900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31676062ae34f9f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e8f7fe700 1 -- 172.21.15.170:0/3098232338 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90009e80 con 0x7f4e98069560 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e8f7fe700 1 -- 172.21.15.170:0/3098232338 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9000b860 con 0x7f4e98069560 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e8f7fe700 1 -- 172.21.15.170:0/3098232338 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000bbe0 con 0x7f4e98069560 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 -- 172.21.15.170:0/3098232338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 msgr2=0x7f4e980699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e980699e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4e900098e0 tx=0x7f4e900091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 -- 172.21.15.170:0/3098232338 shutdown_connections 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e9815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/3098232338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e980699e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 -- 172.21.15.170:0/3098232338 >> 172.21.15.170:0/3098232338 conn(0x7f4e98068e20 msgr2=0x7f4e980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 -- 172.21.15.170:0/3098232338 shutdown_connections 2023-10-08T16:10:15.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.192+0000 7f4e9ec95700 1 -- 172.21.15.170:0/3098232338 wait complete. 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.193+0000 7f4e9ec95700 1 Processor -- start 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 -- start start 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e98154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 0x7f4e98155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e98151ae0 con 0x7f4e98069560 2023-10-08T16:10:15.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980b0940 con 0x7f4e98159c60 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ec95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e980ce9b0 con 0x7f4e98159890 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53842/0 (socket says 172.21.15.170:53842) 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.194+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1503933984 learned_addr learned my addr 172.21.15.170:0/1503933984 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e98154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9d232700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 0x7f4e98155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 msgr2=0x7f4e98154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e98154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 msgr2=0x7f4e98155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 0x7f4e98155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1503933984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90009420 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9d232700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 0x7f4e98155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e190 tx=0x7f4e94009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e8cff9700 1 -- 172.21.15.170:0/1503933984 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400ed80 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e8cff9700 1 -- 172.21.15.170:0/1503933984 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94014420 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e8cff9700 1 -- 172.21.15.170:0/1503933984 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940147e0 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e980cec90 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.195+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e980b05d0 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.197+0000 7f4e8cff9700 1 -- 172.21.15.170:0/1503933984 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f4e94014b90 con 0x7f4e98069560 2023-10-08T16:10:15.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.197+0000 7f4e8cff9700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 0x7f4e840696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.197+0000 7f4e8cff9700 1 -- 172.21.15.170:0/1503933984 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e94085100 con 0x7f4e98069560 2023-10-08T16:10:15.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.197+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 0x7f4e840696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.197+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 0x7f4e840696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e9816df30 tx=0x7f4e88007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.199+0000 7f4ea7f8db80 1 Processor -- start 2023-10-08T16:10:15.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.199+0000 7f4ea7f8db80 1 -- start start 2023-10-08T16:10:15.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbb04d00 0x55e5bbae3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbab6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bba11a70 con 0x55e5bbabaa50 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bba0f270 con 0x55e5bbae3f60 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5bba0b060 con 0x55e5bbb04d00 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53856/0 (socket says 172.21.15.170:53856) 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1692954446 learned_addr learned my addr 172.21.15.170:0/1692954446 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbb04d00 msgr2=0x55e5bbae3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbb04d00 0x55e5bbae3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 msgr2=0x55e5bbab6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbab6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1692954446 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5bb8a5270 con 0x55e5bbabaa50 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e94083990 tx=0x7f4e94052150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfc8f29f4469dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e7dffb700 1 -- 172.21.15.170:0/1692954446 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e940112c0 con 0x55e5bbabaa50 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e7dffb700 1 -- 172.21.15.170:0/1692954446 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e940563e0 con 0x55e5bbabaa50 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.200+0000 7f4e7dffb700 1 -- 172.21.15.170:0/1692954446 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94055ce0 con 0x55e5bbabaa50 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1692954446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 msgr2=0x55e5bbabae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4e94083990 tx=0x7f4e94052150 comp rx=0 tx=0).stop 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1692954446 shutdown_connections 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbab6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbabae20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1692954446 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbb04d00 0x55e5bbae3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1692954446 >> 172.21.15.170:0/1692954446 conn(0x55e5bbabc370 msgr2=0x55e5bbabc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1692954446 shutdown_connections 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1692954446 wait complete. 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.201+0000 7f4ea7f8db80 1 -- start start 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 Processor -- start 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbaf8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbb04d00 0x55e5bbafeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bb9b6890 con 0x55e5bbabaa50 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bb9d1ea0 con 0x55e5bbb04d00 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.202+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5bb88de30 con 0x55e5bbae3f60 2023-10-08T16:10:15.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4ea7f8db80 10 client.0 _ll_register_callbacks cb 0x55e5bba7e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53860/0 (socket says 172.21.15.170:53860) 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1139666908 learned_addr learned my addr 172.21.15.170:0/1139666908 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9d232700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbb04d00 0x55e5bbafeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbaf8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 msgr2=0x55e5bbaf8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbaf8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbb04d00 msgr2=0x55e5bbafeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbb04d00 0x55e5bbafeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 -- 172.21.15.170:0/1139666908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e980b05d0 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbaf8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e940559e0 tx=0x7f4e9406a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f97d0a591d4e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9405cc80 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e94014e40 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5bbaafb70 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e94055ce0 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4ea7f8db80 10 client.6552 fetch_fsmap learned FSMap version 56 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e5bb847fe0 con 0x55e5bbabaa50 2023-10-08T16:10:15.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5bbaafb70 con 0x55e5bbabaa50 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4e9405c7f0 con 0x55e5bbabaa50 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.203+0000 7f4ea7f8db80 10 client.6552 fetch_fsmap finished waiting for FSMap version 56 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4ea7f8db80 10 client.6552 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4ea7f8db80 20 client.6552 populate_metadata read hostname 'smithi170' 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4ea7f8db80 4 client.6552 mds_command: new command op to 6393 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 --> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5bba8caa0 con 0x55e5bbafaae0 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4e76ffd700 10 client.6552 ms_handle_connect on v2:172.21.15.191:6832/3892735210 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.204+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbafaae0 0x55e5bbafaeb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938ad9b5338d31da server_cookie=8cb289930804595b in_seq=0 out_seq=0 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4e76ffd700 1 -- 172.21.15.170:0/1139666908 <== mds.0 v2:172.21.15.191:6832/3892735210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55e5bba8caa0 con 0x55e5bbafaae0 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4e76ffd700 10 client.6552 handle_command_reply: tid=0 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 client.6552 shutdown 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 msgr2=0x55e5bbafd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e940559e0 tx=0x7f4e9406a000 comp rx=0 tx=0).stop 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 shutdown_connections 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbafaae0 0x55e5bbafaeb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbb04d00 0x55e5bbafeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4e76ffd700 0 client.6552 ms_handle_reset on v2:172.21.15.191:6832/3892735210 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbafd200 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/1139666908 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbaf8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 >> 172.21.15.170:0/1139666908 conn(0x55e5bbabc370 msgr2=0x55e5bbb00520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 shutdown_connections 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/1139666908 wait complete. 2023-10-08T16:10:15.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.205+0000 7f4ea7f8db80 20 client.6552 trim_cache size 0 max 16384 2023-10-08T16:10:15.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 Processor -- start 2023-10-08T16:10:15.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 -- start start 2023-10-08T16:10:15.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbafc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbab62a0 con 0x55e5bbabaa50 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbaf9850 con 0x55e5bbafaae0 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbaffe60 con 0x55e5bbae3f60 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53874/0 (socket says 172.21.15.170:53874) 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.222+0000 7f4e9ca31700 1 -- 172.21.15.170:0/611580167 learned_addr learned my addr 172.21.15.170:0/611580167 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbafc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9d232700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 -- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 msgr2=0x55e5bbafc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbafc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 -- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbafaae0 msgr2=0x55e5bbafaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 -- 172.21.15.170:0/611580167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5bb9d9780 con 0x55e5bbabaa50 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbafc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4e940850d0 tx=0x7f4e94070070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0779ceed819505 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e7e7fc700 1 -- 172.21.15.170:0/611580167 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94055ce0 con 0x55e5bbabaa50 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e7e7fc700 1 -- 172.21.15.170:0/611580167 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e940112c0 con 0x55e5bbabaa50 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4e7e7fc700 1 -- 172.21.15.170:0/611580167 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9405cc80 con 0x55e5bbabaa50 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/611580167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 msgr2=0x55e5bbab34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4e940850d0 tx=0x7f4e94070070 comp rx=0 tx=0).stop 2023-10-08T16:10:15.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.223+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/611580167 shutdown_connections 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbab34b0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/611580167 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbae3f60 0x55e5bbafc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/611580167 >> 172.21.15.170:0/611580167 conn(0x55e5bbabc370 msgr2=0x55e5bbabc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/611580167 shutdown_connections 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/611580167 wait complete. 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 -- start start 2023-10-08T16:10:15.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.224+0000 7f4ea7f8db80 1 Processor -- start 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbb637b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbb67040 con 0x55e5bbabaa50 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbab6740 con 0x55e5bbae3f60 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5bbb0f330 con 0x55e5bbafaae0 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4ea7f8db80 10 client.0 _ll_register_callbacks cb 0x55e5bba7e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53886/0 (socket says 172.21.15.170:53886) 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 -- 172.21.15.170:0/764720254 learned_addr learned my addr 172.21.15.170:0/764720254 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbb637b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9d232700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 -- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbafaae0 msgr2=0x55e5bbafaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 -- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 msgr2=0x55e5bbb637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbb637b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.225+0000 7f4e9ca31700 1 -- 172.21.15.170:0/764720254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5bb847fe0 con 0x55e5bbabaa50 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e9ca31700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e940521e0 tx=0x7f4e94069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb20ad63d41214 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9405cc80 con 0x55e5bbabaa50 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4e94075020 con 0x55e5bbabaa50 2023-10-08T16:10:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5bb9ade00 con 0x55e5bbabaa50 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e940112c0 con 0x55e5bbabaa50 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 10 client.6558 fetch_fsmap learned FSMap version 56 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55e5bb9c0470 con 0x55e5bbabaa50 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5bb9ade00 con 0x55e5bbabaa50 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 56) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4e94068040 con 0x55e5bbabaa50 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 10 client.6558 fetch_fsmap finished waiting for FSMap version 56 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 10 client.6558 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 20 client.6558 populate_metadata read hostname 'smithi170' 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbb04d00 0x55e5bbafb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 4 client.6558 mds_command: new command op to 6393 tid=0[{"prefix": "status"}] 2023-10-08T16:10:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 --> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5bb9adb20 con 0x55e5bbb04d00 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.226+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbb04d00 0x55e5bbafb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4e8ffff700 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbb04d00 0x55e5bbafb3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9211d2c05801ce85 server_cookie=acad0afdf6a7e5bc in_seq=0 out_seq=0 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4e76ffd700 10 client.6558 ms_handle_connect on v2:172.21.15.191:6832/3892735210 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4e76ffd700 1 -- 172.21.15.170:0/764720254 <== mds.0 v2:172.21.15.191:6832/3892735210 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e5bb9adb20 con 0x55e5bbb04d00 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4e76ffd700 10 client.6558 handle_command_reply: tid=0 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 client.6558 shutdown 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 msgr2=0x55e5bbb0fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4e940521e0 tx=0x7f4e94069000 comp rx=0 tx=0).stop 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 shutdown_connections 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:6832/3892735210,v1:172.21.15.191:6834/3892735210] conn(0x55e5bbb04d00 0x55e5bbafb3f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55e5bbafaae0 0x55e5bbafaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5bbabaa50 0x55e5bbb0fd80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4e76ffd700 0 client.6558 ms_handle_reset on v2:172.21.15.191:6832/3892735210 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 --2- 172.21.15.170:0/764720254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55e5bbae3f60 0x55e5bbb637b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 >> 172.21.15.170:0/764720254 conn(0x55e5bbabc370 msgr2=0x55e5bbab34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 shutdown_connections 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 1 -- 172.21.15.170:0/764720254 wait complete. 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.227+0000 7f4ea7f8db80 20 client.6558 trim_cache size 0 max 16384 2023-10-08T16:10:15.235 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6393, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.375675268, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 56, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 89, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 89, 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.472507026000001 2023-10-08T16:10:15.236 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 msgr2=0x7f4e840696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 0x7f4e840696b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e9816df30 tx=0x7f4e88007eb0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 msgr2=0x7f4e9814d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400e190 tx=0x7f4e94009920 comp rx=0 tx=0).stop 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 shutdown_connections 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4e98159c60 0x7f4e98155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4e84067200 0x7f4e840696b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e98069560 0x7f4e9814d1b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 --2- 172.21.15.170:0/1503933984 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4e98159890 0x7f4e98154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 >> 172.21.15.170:0/1503933984 conn(0x7f4e98068e20 msgr2=0x7f4e98158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 shutdown_connections 2023-10-08T16:10:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.228+0000 7f4e9ec95700 1 -- 172.21.15.170:0/1503933984 wait complete. 2023-10-08T16:10:15.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:10:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.354+0000 7f61174d7700 1 Processor -- start 2023-10-08T16:10:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.354+0000 7f61174d7700 1 -- start start 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 0x7f611015b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f61100b5ea0 0x7f6110068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 0x7f611015c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61100b2a40 con 0x7f61100b5ea0 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611015c480 con 0x7f6110159c10 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f61174d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611015b5d0 con 0x7f6110068bb0 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f61100b5ea0 0x7f6110068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f61100b5ea0 0x7f6110068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53898/0 (socket says 172.21.15.170:53898) 2023-10-08T16:10:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 -- 172.21.15.170:0/1992865429 learned_addr learned my addr 172.21.15.170:0/1992865429 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6115273700 1 --2- 172.21.15.170:0/1992865429 >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 0x7f611015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6115a74700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 0x7f611015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 -- 172.21.15.170:0/1992865429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 msgr2=0x7f611015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 0x7f611015c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 -- 172.21.15.170:0/1992865429 >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 msgr2=0x7f611015b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 --2- 172.21.15.170:0/1992865429 >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 0x7f611015b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 -- 172.21.15.170:0/1992865429 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611015b870 con 0x7f61100b5ea0 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6115273700 1 --2- 172.21.15.170:0/1992865429 >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 0x7f611015b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6115a74700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 0x7f611015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6114a72700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61100b5ea0 0x7f6110068670 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6110099f10 tx=0x7f6100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1b2f9047350f9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.355+0000 7f6107fff700 1 -- 172.21.15.170:0/1992865429 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610000b860 con 0x7f61100b5ea0 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f6107fff700 1 -- 172.21.15.170:0/1992865429 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610000b9c0 con 0x7f61100b5ea0 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f6107fff700 1 -- 172.21.15.170:0/1992865429 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610000bd60 con 0x7f61100b5ea0 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 -- 172.21.15.170:0/1992865429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61100b5ea0 msgr2=0x7f6110068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61100b5ea0 0x7f6110068670 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6110099f10 tx=0x7f6100009250 comp rx=0 tx=0).stop 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 -- 172.21.15.170:0/1992865429 shutdown_connections 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6110068bb0 0x7f611015c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 --2- 172.21.15.170:0/1992865429 >> v2:172.21.15.191:3300/0 conn(0x7f6110159c10 0x7f611015b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 --2- 172.21.15.170:0/1992865429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61100b5ea0 0x7f6110068670 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 -- 172.21.15.170:0/1992865429 >> 172.21.15.170:0/1992865429 conn(0x7f6110099a20 msgr2=0x7f61100af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 -- 172.21.15.170:0/1992865429 shutdown_connections 2023-10-08T16:10:15.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.356+0000 7f61174d7700 1 -- 172.21.15.170:0/1992865429 wait complete. 2023-10-08T16:10:15.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.357+0000 7f61174d7700 1 Processor -- start 2023-10-08T16:10:15.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.357+0000 7f61174d7700 1 -- start start 2023-10-08T16:10:15.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61100b5ea0 0x7f61100d0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 0x7f61100b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100afaa0 con 0x7f6110068bb0 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611016a800 con 0x7f6110159c10 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f61174d7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61100aee20 con 0x7f61100b5ea0 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53904/0 (socket says 172.21.15.170:53904) 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 -- 172.21.15.170:0/556691276 learned_addr learned my addr 172.21.15.170:0/556691276 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115a74700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 0x7f61100b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6114a72700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61100b5ea0 0x7f61100d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 -- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61100b5ea0 msgr2=0x7f61100d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61100b5ea0 0x7f61100d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 -- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 msgr2=0x7f61100b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 0x7f61100b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115273700 1 -- 172.21.15.170:0/556691276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61000094f0 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.358+0000 7f6115a74700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 0x7f61100b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f6115273700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d860 tx=0x7f610c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c009610 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c009770 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61100af100 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c009b10 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.359+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61100af6c0 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.360+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61100afd10 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.360+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f610c014430 con 0x7f6110068bb0 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.360+0000 7f61057fa700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 0x7f60f40696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:15.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.360+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f610c084db0 con 0x7f6110068bb0 2023-10-08T16:10:15.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.361+0000 7f6114a72700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 0x7f60f40696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:15.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.361+0000 7f6114a72700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 0x7f60f40696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6110099f10 tx=0x7f6100017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:15.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.365+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f610c051cd0 con 0x7f6110068bb0 2023-10-08T16:10:15.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.591+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f611004c3c0 con 0x7f6110068bb0 2023-10-08T16:10:15.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.592+0000 7f61057fa700 1 -- 172.21.15.170:0/556691276 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f610c0552f0 con 0x7f6110068bb0 2023-10-08T16:10:15.592 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:10:15.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 msgr2=0x7f60f40696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 0x7f60f40696b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6110099f10 tx=0x7f6100017000 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 msgr2=0x7f611016a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d860 tx=0x7f610c008040 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 shutdown_connections 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6110159c10 0x7f61100b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60f4067200 0x7f60f40696b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6110068bb0 0x7f611016a2c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 --2- 172.21.15.170:0/556691276 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f61100b5ea0 0x7f61100d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 >> 172.21.15.170:0/556691276 conn(0x7f6110099a20 msgr2=0x7f611009a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 shutdown_connections 2023-10-08T16:10:15.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:15.594+0000 7f61174d7700 1 -- 172.21.15.170:0/556691276 wait complete. 2023-10-08T16:10:15.606 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:10:15.606 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:10:15.606 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:10:15.606 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:10:15.606 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:15.607 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:10:15.607 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:10:15.607 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:10:15.607 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:15.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.607 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:10:15.622 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:15.623 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42135sec preferred_lft 42135sec 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:10:15.624 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:10:15.625 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:15.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.626 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:10:15.686 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:10:15.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.687 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:10:15.743 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:10:15.743 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:10:15.744 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:10:15.744 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:10:15.744 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:10:15.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.744 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:10:15.744 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:10:15.744 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:10:15.744 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:10:15.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:10:15.745 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:10:15.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.845 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:10:15.859 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:10:15.859 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:10:15.859 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:15.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:15.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:10:15.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:15.924 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:15.979 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:15.979 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:15.980 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:16.034 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:10:16.034 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:10:16.034 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:16.114 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:16.115 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:16.133 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:16.133 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:16.134 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:16.156 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:10:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:10:16.158 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:10:16.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:16.226 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:10:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.380+0000 7fb16ee92700 1 Processor -- start 2023-10-08T16:10:16.381 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.380+0000 7fb16ee92700 1 -- start start 2023-10-08T16:10:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16ee92700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1681578b0 0x7fb168155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16ee92700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb1680af2f0 0x7fb1680af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16ee92700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680ad930 con 0x7fb168155c10 2023-10-08T16:10:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16ee92700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680b0c60 con 0x7fb1680af2f0 2023-10-08T16:10:16.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16ee92700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680acd20 con 0x7fb1681578b0 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb1680af2f0 0x7fb1680af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16cc2e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb168155c10 0x7fb1680ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53724/0 (socket says 172.21.15.170:53724) 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb1680af2f0 0x7fb1680af710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53920/0 (socket says 172.21.15.170:53920) 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16cc2e700 1 -- 172.21.15.170:0/1998769038 learned_addr learned my addr 172.21.15.170:0/1998769038 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 -- 172.21.15.170:0/1998769038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1681578b0 msgr2=0x7fb168155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 --2- 172.21.15.170:0/1998769038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1681578b0 0x7fb168155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 -- 172.21.15.170:0/1998769038 >> v1:172.21.15.170:6789/0 conn(0x7fb168155c10 legacy=0x7fb1680ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb15f7fe700 1 -- 172.21.15.170:0/1998769038 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820394302 0 0) 0x7fb1680ad930 con 0x7fb168155c10 2023-10-08T16:10:16.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.381+0000 7fb16d42f700 1 -- 172.21.15.170:0/1998769038 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680ad000 con 0x7fb1680af2f0 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16d42f700 1 --2- 172.21.15.170:0/1998769038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1680af2f0 0x7fb1680af710 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb16000aa90 tx=0x7fb16000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bd0acf252fc408 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb15f7fe700 1 -- 172.21.15.170:0/1998769038 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000ca40 con 0x7fb1680af2f0 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb15f7fe700 1 -- 172.21.15.170:0/1998769038 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000cba0 con 0x7fb1680af2f0 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb15f7fe700 1 -- 172.21.15.170:0/1998769038 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160018400 con 0x7fb1680af2f0 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 -- 172.21.15.170:0/1998769038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1680af2f0 msgr2=0x7fb1680af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 --2- 172.21.15.170:0/1998769038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1680af2f0 0x7fb1680af710 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb16000aa90 tx=0x7fb16000a330 comp rx=0 tx=0).stop 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 -- 172.21.15.170:0/1998769038 shutdown_connections 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 --2- 172.21.15.170:0/1998769038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1680af2f0 0x7fb1680af710 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 --2- 172.21.15.170:0/1998769038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1681578b0 0x7fb168155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 -- 172.21.15.170:0/1998769038 >> 172.21.15.170:0/1998769038 conn(0x7fb16806a320 msgr2=0x7fb1680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 -- 172.21.15.170:0/1998769038 shutdown_connections 2023-10-08T16:10:16.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.382+0000 7fb16ee92700 1 -- 172.21.15.170:0/1998769038 wait complete. 2023-10-08T16:10:16.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.383+0000 7fb16ee92700 1 Processor -- start 2023-10-08T16:10:16.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 -- start start 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 0x7fb1680aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1681578b0 0x7fb1681549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680c8b20 con 0x7fb168155c10 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb168169950 con 0x7fb1681578b0 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb16ee92700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680ccd90 con 0x7fb1680af2f0 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb15ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb15ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53930/0 (socket says 172.21.15.170:53930) 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.384+0000 7fb15ffff700 1 -- 172.21.15.170:0/2948435146 learned_addr learned my addr 172.21.15.170:0/2948435146 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb16cc2e700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 0x7fb1680aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb16d42f700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1681578b0 0x7fb1681549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:16.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 -- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 msgr2=0x7fb1680aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 0x7fb1680aac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 -- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1681578b0 msgr2=0x7fb1681549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1681578b0 0x7fb1681549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 -- 172.21.15.170:0/2948435146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16000a5d0 con 0x7fb168155c10 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb16cc2e700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 0x7fb1680aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15ffff700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b840 tx=0x7fb1580091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800f510 con 0x7fb168155c10 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15800f690 con 0x7fb168155c10 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1680aa500 con 0x7fb168155c10 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800fa50 con 0x7fb168155c10 2023-10-08T16:10:16.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.385+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1680ab1b0 con 0x7fb168155c10 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.386+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1680aa360 con 0x7fb168155c10 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.386+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fb15801a400 con 0x7fb168155c10 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.387+0000 7fb15cff9700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 0x7fb1540696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.387+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb158084510 con 0x7fb168155c10 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.387+0000 7fb16cc2e700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 0x7fb1540696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.387+0000 7fb16cc2e700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 0x7fb1540696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb164001560 tx=0x7fb164009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:16.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.391+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1680aa360 con 0x7fb168155c10 2023-10-08T16:10:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.616+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb168169af0 con 0x7fb168155c10 2023-10-08T16:10:16.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.616+0000 7fb15cff9700 1 -- 172.21.15.170:0/2948435146 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb168169af0 con 0x7fb168155c10 2023-10-08T16:10:16.617 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:10:16.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 msgr2=0x7fb1540696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 0x7fb1540696b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb164001560 tx=0x7fb164009040 comp rx=0 tx=0).stop 2023-10-08T16:10:16.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 msgr2=0x7fb168154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:16.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb15800b840 tx=0x7fb1580091d0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 shutdown_connections 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb1681578b0 0x7fb1681549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1680af2f0 0x7fb1680aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb154067200 0x7fb1540696b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 --2- 172.21.15.170:0/2948435146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168155c10 0x7fb168154470 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 >> 172.21.15.170:0/2948435146 conn(0x7fb16806a320 msgr2=0x7fb1681533c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 shutdown_connections 2023-10-08T16:10:16.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:16.619+0000 7fb16ee92700 1 -- 172.21.15.170:0/2948435146 wait complete. 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:10:16.631 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:10:16.632 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:10:16.632 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:16.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.632 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:10:16.652 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:10:16.652 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:10:16.652 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:16.653 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42135sec preferred_lft 42135sec 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:10:16.654 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:16.655 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:10:16.656 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:10:16.656 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:16.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.656 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:10:16.719 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:10:16.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.721 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:10:16.779 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:10:16.779 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:10:16.779 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:10:16.779 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:10:16.779 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:10:16.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:10:16.780 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:10:16.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.884 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:10:16.899 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:10:16.899 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:10:16.899 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:16.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:16.899 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:10:16.963 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:16.963 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:17.017 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:17.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:17.018 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:17.072 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:10:17.073 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:10:17.073 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:17.152 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:17.152 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:17.171 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:17.172 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:17.172 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:17.194 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:17.195 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:17.195 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:17.195 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:17.195 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:10:17.196 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:10:17.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:10:17.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:17.198 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:17.262 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:10:17.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.385+0000 7fac47fff700 1 Processor -- start 2023-10-08T16:10:17.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.385+0000 7fac47fff700 1 -- start start 2023-10-08T16:10:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac47fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fac48159450 0x7fac4815d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac47fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480cfa80 con 0x7fac480ce850 2023-10-08T16:10:17.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac47fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480cee70 con 0x7fac48159820 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac47fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac480b3670 con 0x7fac48159450 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac477fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fac480ce850 0x7fac480b05c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53726/0 (socket says 172.21.15.170:53726) 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac477fe700 1 -- 172.21.15.170:0/3339256783 learned_addr learned my addr 172.21.15.170:0/3339256783 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac46ffd700 1 --2- 172.21.15.170:0/3339256783 >> v2:172.21.15.191:3300/0 conn(0x7fac48159450 0x7fac4815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525364544 0 0) 0x7fac480cfa80 con 0x7fac480ce850 2023-10-08T16:10:17.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac30003570 con 0x7fac480ce850 2023-10-08T16:10:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595008078 0 0) 0x7fac480cee70 con 0x7fac48159820 2023-10-08T16:10:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac480cfa80 con 0x7fac48159820 2023-10-08T16:10:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac46ffd700 1 -- 172.21.15.170:0/3339256783 >> v1:172.21.15.191:6789/0 conn(0x7fac48159820 legacy=0x7fac480ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac46ffd700 1 -- 172.21.15.170:0/3339256783 >> v1:172.21.15.170:6789/0 conn(0x7fac480ce850 legacy=0x7fac480b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:17.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.386+0000 7fac46ffd700 1 -- 172.21.15.170:0/3339256783 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480cf150 con 0x7fac48159450 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac46ffd700 1 --2- 172.21.15.170:0/3339256783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac48159450 0x7fac4815d240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e60 tx=0x7fac38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17eb64f7deb671e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3800e070 con 0x7fac48159450 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3800bab0 con 0x7fac48159450 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac45ffb700 1 -- 172.21.15.170:0/3339256783 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3800be70 con 0x7fac48159450 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 -- 172.21.15.170:0/3339256783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac48159450 msgr2=0x7fac4815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 --2- 172.21.15.170:0/3339256783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac48159450 0x7fac4815d240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fac38009e60 tx=0x7fac38009780 comp rx=0 tx=0).stop 2023-10-08T16:10:17.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 -- 172.21.15.170:0/3339256783 shutdown_connections 2023-10-08T16:10:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 --2- 172.21.15.170:0/3339256783 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac48159450 0x7fac4815d240 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 -- 172.21.15.170:0/3339256783 >> 172.21.15.170:0/3339256783 conn(0x7fac480d0570 msgr2=0x7fac480d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 -- 172.21.15.170:0/3339256783 shutdown_connections 2023-10-08T16:10:17.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.387+0000 7fac47fff700 1 -- 172.21.15.170:0/3339256783 wait complete. 2023-10-08T16:10:17.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.388+0000 7fac47fff700 1 Processor -- start 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 -- start start 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac48159820 0x7fac4814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48150520 con 0x7fac48159450 2023-10-08T16:10:17.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48150e80 con 0x7fac480ce850 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac47fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4815c610 con 0x7fac48159820 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac467fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac46ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac467fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53944/0 (socket says 172.21.15.170:53944) 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac467fc700 1 -- 172.21.15.170:0/2785820753 learned_addr learned my addr 172.21.15.170:0/2785820753 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac46ffd700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:44722/0 (socket says 172.21.15.170:44722) 2023-10-08T16:10:17.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.389+0000 7fac477fe700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac48159820 0x7fac4814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 -- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac48159820 msgr2=0x7fac4814df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac48159820 0x7fac4814df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 -- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 msgr2=0x7fac48150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 -- 172.21.15.170:0/2785820753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac38009a20 con 0x7fac48159450 2023-10-08T16:10:17.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac46ffd700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac467fc700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c7f0 tx=0x7fac3c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0015f0 con 0x7fac48159450 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c001770 con 0x7fac48159450 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c001b30 con 0x7fac48159450 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4816ae30 con 0x7fac48159450 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.390+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac48150140 con 0x7fac48159450 2023-10-08T16:10:17.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.391+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac48150660 con 0x7fac48159450 2023-10-08T16:10:17.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.391+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fac3c01b400 con 0x7fac48159450 2023-10-08T16:10:17.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.392+0000 7fac2effd700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 0x7fac340696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:17.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.392+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac3c084710 con 0x7fac48159450 2023-10-08T16:10:17.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.392+0000 7fac46ffd700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 0x7fac340696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:17.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.392+0000 7fac46ffd700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 0x7fac340696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac38009460 tx=0x7fac38019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:17.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.396+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac48150660 con 0x7fac48159450 2023-10-08T16:10:17.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:17.623+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fac48150660 con 0x7fac48159450 2023-10-08T16:10:18.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.495+0000 7fac2effd700 1 -- 172.21.15.170:0/2785820753 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 136+0+0 (secure 0 0 0) 0x7fac48150660 con 0x7fac48159450 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 msgr2=0x7fac340696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 0x7fac340696b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac38009460 tx=0x7fac38019000 comp rx=0 tx=0).stop 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 msgr2=0x7fac480b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c7f0 tx=0x7fac3c007dd0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 shutdown_connections 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac48159820 0x7fac4814df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac480ce850 0x7fac48150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac34067200 0x7fac340696b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 --2- 172.21.15.170:0/2785820753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac48159450 0x7fac480b0df0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 >> 172.21.15.170:0/2785820753 conn(0x7fac480d0570 msgr2=0x7fac480b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:18.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 shutdown_connections 2023-10-08T16:10:18.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.497+0000 7fac47fff700 1 -- 172.21.15.170:0/2785820753 wait complete. 2023-10-08T16:10:18.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:10:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.625+0000 7f7404319700 1 Processor -- start 2023-10-08T16:10:18.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.625+0000 7f7404319700 1 -- start start 2023-10-08T16:10:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f7404319700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73fc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f7404319700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc15d7e0 con 0x7f73fc06bcd0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f7404319700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73fc15dac0 con 0x7f73fc15df50 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f7404319700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc15aa30 con 0x7f73fc06c0a0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f74020b5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f73fc06bcd0 0x7f73fc15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53730/0 (socket says 172.21.15.170:53730) 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.626+0000 7f74020b5700 1 -- 172.21.15.170:0/2827766711 learned_addr learned my addr 172.21.15.170:0/2827766711 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190014610 0 0) 0x7f73fc15d7e0 con 0x7f73fc06bcd0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74018b4700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73fc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e4003570 con 0x7f73fc06bcd0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1527603647 0 0) 0x7f73e4003570 con 0x7f73fc06bcd0 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 msgr2=0x7f73fc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73fc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 >> v1:172.21.15.191:6789/0 conn(0x7f73fc15df50 legacy=0x7f73fc0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc15ad10 con 0x7f73fc06bcd0 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f73ec0017f0 con 0x7f73fc06bcd0 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 >> v1:172.21.15.170:6789/0 conn(0x7f73fc06bcd0 legacy=0x7f73fc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e4003a90 0x7f73e4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4004d50 0x7f73e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc15aa30 con 0x7f73e4003a90 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400bb70 con 0x7f73e4004d50 2023-10-08T16:10:18.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400bd30 con 0x7f73fc06c0a0 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e4003a90 0x7f73e4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74018b4700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74028b6700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4004d50 0x7f73e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 -- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 msgr2=0x7f73e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc06c0a0 0x7f73e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 -- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4004d50 msgr2=0x7f73e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4004d50 0x7f73e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 -- 172.21.15.170:0/2827766711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4003570 con 0x7f73e4003a90 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.627+0000 7f74020b5700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e4003a90 0x7f73e4003e60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00a940 tx=0x7f73fc0ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16938c22078713f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0070d0 con 0x7f73e4003a90 2023-10-08T16:10:18.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec007230 con 0x7f73e4003a90 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f74010b3700 1 -- 172.21.15.170:0/2827766711 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0075f0 con 0x7f73e4003a90 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 -- 172.21.15.170:0/2827766711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e4003a90 msgr2=0x7f73e4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 --2- 172.21.15.170:0/2827766711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73e4003a90 0x7f73e4003e60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00a940 tx=0x7f73fc0ce4d0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f74028b6700 1 -- 172.21.15.170:0/2827766711 reap_dead start 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 -- 172.21.15.170:0/2827766711 shutdown_connections 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 -- 172.21.15.170:0/2827766711 >> 172.21.15.170:0/2827766711 conn(0x7f73fc0ce600 msgr2=0x7f73fc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 -- 172.21.15.170:0/2827766711 shutdown_connections 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.628+0000 7f7404319700 1 -- 172.21.15.170:0/2827766711 wait complete. 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.629+0000 7f7404319700 1 Processor -- start 2023-10-08T16:10:18.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.629+0000 7f7404319700 1 -- start start 2023-10-08T16:10:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4003a90 0x7f73fc156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc0c49a0 0x7f73fc14a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc14a860 con 0x7f73fc0c4530 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc169de0 con 0x7f73e4003a90 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f7404319700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73fc16a0c0 con 0x7f73fc0c49a0 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53972/0 (socket says 172.21.15.170:53972) 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 -- 172.21.15.170:0/957532618 learned_addr learned my addr 172.21.15.170:0/957532618 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74028b6700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc0c49a0 0x7f73fc14a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74020b5700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4003a90 0x7f73fc156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 -- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc0c49a0 msgr2=0x7f73fc14a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc0c49a0 0x7f73fc14a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 -- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4003a90 msgr2=0x7f73fc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4003a90 0x7f73fc156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.630+0000 7f74018b4700 1 -- 172.21.15.170:0/957532618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc16a3a0 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f74018b4700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f73f0011d40 tx=0x7f73f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f000fa10 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f000fb70 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73fc16a680 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0014440 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.631+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73fc169af0 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.632+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73fc0c7780 con 0x7f73fc0c4530 2023-10-08T16:10:18.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.632+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f73f0014870 con 0x7f73fc0c4530 2023-10-08T16:10:18.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.632+0000 7f73fa7fc700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 0x7f73e80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.633+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f73f0018030 con 0x7f73fc0c4530 2023-10-08T16:10:18.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.633+0000 7f74020b5700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 0x7f73e80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.633+0000 7f74020b5700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 0x7f73e80696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00a430 tx=0x7f73ec015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.637+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73f00521f0 con 0x7f73fc0c4530 2023-10-08T16:10:18.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.846+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f73fc16a230 con 0x7f73e8067200 2023-10-08T16:10:18.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.847+0000 7f73fa7fc700 1 -- 172.21.15.170:0/957532618 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f73fc16a230 con 0x7f73e8067200 2023-10-08T16:10:18.848 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:10:18.848 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:18.848 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:10:18.848 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:18.848 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:10:18.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.849+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 msgr2=0x7f73e80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.849+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 0x7f73e80696b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00a430 tx=0x7f73ec015000 comp rx=0 tx=0).stop 2023-10-08T16:10:18.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.849+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 msgr2=0x7f73fc149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.849+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f73f0011d40 tx=0x7f73f00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 shutdown_connections 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73fc0c49a0 0x7f73fc14a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73e4003a90 0x7f73fc156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f73e8067200 0x7f73e80696b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 --2- 172.21.15.170:0/957532618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73fc0c4530 0x7f73fc149de0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 >> 172.21.15.170:0/957532618 conn(0x7f73fc0ce600 msgr2=0x7f73fc15bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 shutdown_connections 2023-10-08T16:10:18.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.850+0000 7f7404319700 1 -- 172.21.15.170:0/957532618 wait complete. 2023-10-08T16:10:18.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:10:18.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.979+0000 7fcba2db0700 1 Processor -- start 2023-10-08T16:10:18.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.979+0000 7fcba2db0700 1 -- start start 2023-10-08T16:10:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba2db0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb9c159c60 0x7fcb9c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba2db0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb9c15db10 0x7fcb9c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba2db0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c15a160 con 0x7fcb9c159890 2023-10-08T16:10:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba2db0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c15df50 con 0x7fcb9c15db10 2023-10-08T16:10:18.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba2db0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0d13a0 con 0x7fcb9c159c60 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba134d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcb9c15db10 0x7fcb9c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba0b4c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcb9c159890 0x7fcb9c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53734/0 (socket says 172.21.15.170:53734) 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba0b4c700 1 -- 172.21.15.170:0/1353446078 learned_addr learned my addr 172.21.15.170:0/1353446078 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcb9bfff700 1 --2- 172.21.15.170:0/1353446078 >> v2:172.21.15.191:3300/0 conn(0x7fcb9c159c60 0x7fcb9c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba134d700 1 -- 172.21.15.170:0/1353446078 >> v2:172.21.15.191:3300/0 conn(0x7fcb9c159c60 msgr2=0x7fcb9c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba134d700 1 --2- 172.21.15.170:0/1353446078 >> v2:172.21.15.191:3300/0 conn(0x7fcb9c159c60 0x7fcb9c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.980+0000 7fcba134d700 1 -- 172.21.15.170:0/1353446078 >> v1:172.21.15.170:6789/0 conn(0x7fcb9c159890 legacy=0x7fcb9c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba134d700 1 -- 172.21.15.170:0/1353446078 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c15e140 con 0x7fcb9c15db10 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcb9b7fe700 1 -- 172.21.15.170:0/1353446078 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942384746 0 0) 0x7fcb9c15a160 con 0x7fcb9c159890 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba134d700 1 --2- 172.21.15.170:0/1353446078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c15db10 0x7fcb9c069560 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b67f0 tx=0x7fcb940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc9c1a07a98c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcb9b7fe700 1 -- 172.21.15.170:0/1353446078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400b6b0 con 0x7fcb9c15db10 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcb9b7fe700 1 -- 172.21.15.170:0/1353446078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9400b810 con 0x7fcb9c15db10 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcb9b7fe700 1 -- 172.21.15.170:0/1353446078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400bbb0 con 0x7fcb9c15db10 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 -- 172.21.15.170:0/1353446078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c15db10 msgr2=0x7fcb9c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 --2- 172.21.15.170:0/1353446078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c15db10 0x7fcb9c069560 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0b67f0 tx=0x7fcb940091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 -- 172.21.15.170:0/1353446078 shutdown_connections 2023-10-08T16:10:18.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 --2- 172.21.15.170:0/1353446078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c15db10 0x7fcb9c069560 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 --2- 172.21.15.170:0/1353446078 >> v2:172.21.15.191:3300/0 conn(0x7fcb9c159c60 0x7fcb9c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 -- 172.21.15.170:0/1353446078 >> 172.21.15.170:0/1353446078 conn(0x7fcb9c068e20 msgr2=0x7fcb9c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:18.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 -- 172.21.15.170:0/1353446078 shutdown_connections 2023-10-08T16:10:18.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.981+0000 7fcba2db0700 1 -- 172.21.15.170:0/1353446078 wait complete. 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.982+0000 7fcba2db0700 1 Processor -- start 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 -- start start 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb9c159890 0x7fcb9c151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb9c15db10 0x7fcb9c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0b0750 con 0x7fcb9c159c60 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0cf6a0 con 0x7fcb9c15db10 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcba2db0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0cf980 con 0x7fcb9c159890 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcb9bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcb9bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53990/0 (socket says 172.21.15.170:53990) 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.983+0000 7fcb9bfff700 1 -- 172.21.15.170:0/3283793551 learned_addr learned my addr 172.21.15.170:0/3283793551 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcba0b4c700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb9c159890 0x7fcb9c151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcba134d700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb9c15db10 0x7fcb9c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 -- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb9c159890 msgr2=0x7fcb9c151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb9c159890 0x7fcb9c151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 -- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb9c15db10 msgr2=0x7fcb9c0b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb9c15db10 0x7fcb9c0b5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 -- 172.21.15.170:0/3283793551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb94009420 con 0x7fcb9c159c60 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb9bfff700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d890 tx=0x7fcb84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840125f0 con 0x7fcb9c159c60 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84012750 con 0x7fcb9c159c60 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c0cfc60 con 0x7fcb9c159c60 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84012af0 con 0x7fcb9c159c60 2023-10-08T16:10:18.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.984+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c15f930 con 0x7fcb9c159c60 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.985+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c0b0540 con 0x7fcb9c159c60 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.985+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fcb8401e460 con 0x7fcb9c159c60 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.986+0000 7fcb98ff9700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 0x7fcb7c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.986+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb84084ef0 con 0x7fcb9c159c60 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.986+0000 7fcba0b4c700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 0x7fcb7c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:18.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.986+0000 7fcba0b4c700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 0x7fcb7c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c002250 tx=0x7fcb8c001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:18.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:18.990+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb9c0b0540 con 0x7fcb9c159c60 2023-10-08T16:10:19.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.171+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcb9c0b0540 con 0x7fcb9c159c60 2023-10-08T16:10:19.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.172+0000 7fcb98ff9700 1 -- 172.21.15.170:0/3283793551 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcb9c0b0540 con 0x7fcb9c159c60 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 msgr2=0x7fcb7c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 0x7fcb7c0696b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c002250 tx=0x7fcb8c001a20 comp rx=0 tx=0).stop 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 msgr2=0x7fcb9c0cf000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d890 tx=0x7fcb84008040 comp rx=0 tx=0).stop 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 shutdown_connections 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb9c15db10 0x7fcb9c0b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb9c159890 0x7fcb9c151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb7c067200 0x7fcb7c0696b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 --2- 172.21.15.170:0/3283793551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb9c159c60 0x7fcb9c0cf000 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 >> 172.21.15.170:0/3283793551 conn(0x7fcb9c068e20 msgr2=0x7fcb9c15b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 shutdown_connections 2023-10-08T16:10:19.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.174+0000 7fcba2db0700 1 -- 172.21.15.170:0/3283793551 wait complete. 2023-10-08T16:10:19.187 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs --human_readable 2023-10-08T16:10:19.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.303+0000 7fca80a87700 1 Processor -- start 2023-10-08T16:10:19.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.303+0000 7fca80a87700 1 -- start start 2023-10-08T16:10:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca80a87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 0x7fca7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca80a87700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 0x7fca7c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca80a87700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c15a160 con 0x7fca7c159c60 2023-10-08T16:10:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca80a87700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c069f20 con 0x7fca7c069560 2023-10-08T16:10:19.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca80a87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0d13a0 con 0x7fca7c159890 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca79d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fca7c159c60 0x7fca7c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53744/0 (socket says 172.21.15.170:53744) 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.304+0000 7fca79d9b700 1 -- 172.21.15.170:0/2620004612 learned_addr learned my addr 172.21.15.170:0/2620004612 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 --2- 172.21.15.170:0/2620004612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 0x7fca7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7ad9d700 1 --2- 172.21.15.170:0/2620004612 >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 0x7fca7c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7959a700 1 -- 172.21.15.170:0/2620004612 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712675096 0 0) 0x7fca7c15a160 con 0x7fca7c159c60 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7959a700 1 -- 172.21.15.170:0/2620004612 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca64003570 con 0x7fca7c159c60 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 -- 172.21.15.170:0/2620004612 >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 msgr2=0x7fca7c0699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 --2- 172.21.15.170:0/2620004612 >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 0x7fca7c0699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 -- 172.21.15.170:0/2620004612 >> v1:172.21.15.170:6789/0 conn(0x7fca7c159c60 legacy=0x7fca7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 -- 172.21.15.170:0/2620004612 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c15e140 con 0x7fca7c159890 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7ad9d700 1 --2- 172.21.15.170:0/2620004612 >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 0x7fca7c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7a59c700 1 --2- 172.21.15.170:0/2620004612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 0x7fca7c15abd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009e60 tx=0x7fca6c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d94653cbcb543c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7959a700 1 -- 172.21.15.170:0/2620004612 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00e070 con 0x7fca7c159890 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7959a700 1 -- 172.21.15.170:0/2620004612 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c00bab0 con 0x7fca7c159890 2023-10-08T16:10:19.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.305+0000 7fca7959a700 1 -- 172.21.15.170:0/2620004612 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00be30 con 0x7fca7c159890 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 -- 172.21.15.170:0/2620004612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 msgr2=0x7fca7c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 --2- 172.21.15.170:0/2620004612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 0x7fca7c15abd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca6c009e60 tx=0x7fca6c009780 comp rx=0 tx=0).stop 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 -- 172.21.15.170:0/2620004612 shutdown_connections 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 --2- 172.21.15.170:0/2620004612 >> v2:172.21.15.191:3300/0 conn(0x7fca7c069560 0x7fca7c0699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 --2- 172.21.15.170:0/2620004612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c159890 0x7fca7c15abd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 -- 172.21.15.170:0/2620004612 >> 172.21.15.170:0/2620004612 conn(0x7fca7c068e20 msgr2=0x7fca7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:19.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 -- 172.21.15.170:0/2620004612 shutdown_connections 2023-10-08T16:10:19.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.306+0000 7fca80a87700 1 -- 172.21.15.170:0/2620004612 wait complete. 2023-10-08T16:10:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.307+0000 7fca80a87700 1 Processor -- start 2023-10-08T16:10:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.307+0000 7fca80a87700 1 -- start start 2023-10-08T16:10:19.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c069560 0x7fca7c149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca7c159890 0x7fca7c0b5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b3f40 con 0x7fca7c159c60 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c149850 con 0x7fca7c159890 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0b4d30 con 0x7fca7c069560 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54006/0 (socket says 172.21.15.170:54006) 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 -- 172.21.15.170:0/911088214 learned_addr learned my addr 172.21.15.170:0/911088214 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7a59c700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c069560 0x7fca7c149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca79d9b700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca7c159890 0x7fca7c0b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 -- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c069560 msgr2=0x7fca7c149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c069560 0x7fca7c149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 -- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca7c159890 msgr2=0x7fca7c0b5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca7c159890 0x7fca7c0b5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 -- 172.21.15.170:0/911088214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca74009420 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca7ad9d700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fca7400ea50 tx=0x7fca740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400c920 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7400ca80 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c009a20 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7400ce40 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.308+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c0b4480 con 0x7fca7c159c60 2023-10-08T16:10:19.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.309+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c04c3c0 con 0x7fca7c159c60 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.310+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fca74013680 con 0x7fca7c159c60 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.310+0000 7fca6a7fc700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 0x7fca5c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.310+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca74084b00 con 0x7fca7c159c60 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.310+0000 7fca7a59c700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 0x7fca5c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.310+0000 7fca7a59c700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 0x7fca5c0696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0097c0 tx=0x7fca6c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.314+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca74051a20 con 0x7fca7c159c60 2023-10-08T16:10:19.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.523+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "human_readable": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fca7c06a620 con 0x7fca5c067200 2023-10-08T16:10:19.525 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.525+0000 7f60a202f700 -1 client.6240 remove_session_caps still has dirty|flushing caps on 0x10000000001.head(faked_ino=0 nref=4 ll_ref=0 cap_refs={} open={} mode=40755 size=0/0 nlink=1 btime=2023-10-08T16:09:48.709887+0000 mtime=2023-10-08T16:09:48.709887+0000 ctime=2023-10-08T16:09:48.714594+0000 change_attr=2 caps=- flushing_caps=Ax COMPLETE parents=0x10000000000.head["group_0000000000460347"] has_dir_layout 0x560b3e7ade00) 2023-10-08T16:10:19.528 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.528+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:19.528 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.528+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:19.528 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.528+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:19.528 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.528+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:19.528 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:19.528+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:19.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.533+0000 7fca6a7fc700 1 -- 172.21.15.170:0/911088214 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+471 (secure 0 0 0) 0x7fca7c06a620 con 0x7fca5c067200 2023-10-08T16:10:19.533 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:19.533 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2023-10-08T16:10:19.533 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "pools": { 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "data": [ 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": " 340G", 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_data", 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: "used": " 0 " 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:19.534 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": [ 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": " 340G", 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_metadata", 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: "used": "64.0k" 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:19.535 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:19.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 msgr2=0x7fca5c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 0x7fca5c0696b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0097c0 tx=0x7fca6c019000 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 msgr2=0x7fca7c1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fca7400ea50 tx=0x7fca740091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 shutdown_connections 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca5c067200 0x7fca5c0696b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca7c159c60 0x7fca7c1577f0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca7c069560 0x7fca7c149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 --2- 172.21.15.170:0/911088214 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca7c159890 0x7fca7c0b5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 >> 172.21.15.170:0/911088214 conn(0x7fca7c068e20 msgr2=0x7fca7c155320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 shutdown_connections 2023-10-08T16:10:19.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.535+0000 7fca80a87700 1 -- 172.21.15.170:0/911088214 wait complete. 2023-10-08T16:10:19.548 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:10:19.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:19.590 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:19.635 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:10:19.635 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:10:19.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:10:19.635 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:19.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:10:19.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:19.679 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:10:19.734 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:10:19.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:10:19.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:19.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:10:19.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:19.780 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:10:19.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.912+0000 7fd5664f0700 1 Processor -- start 2023-10-08T16:10:19.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.912+0000 7fd5664f0700 1 -- start start 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd560159890 0x7fd56015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd560159c60 0x7fd56015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56015db10 0x7fd560069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56015df50 con 0x7fd560159890 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56015a160 con 0x7fd560159c60 2023-10-08T16:10:19.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd5664f0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600d13a0 con 0x7fd56015db10 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd55ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd560159890 0x7fd56015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd55ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd560159890 0x7fd56015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54042/0 (socket says 172.21.15.170:54042) 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd55ffff700 1 -- 172.21.15.170:0/318896159 learned_addr learned my addr 172.21.15.170:0/318896159 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd564a8d700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56015db10 0x7fd560069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.913+0000 7fd55f7fe700 1 --2- 172.21.15.170:0/318896159 >> v2:172.21.15.191:3300/0 conn(0x7fd560159c60 0x7fd56015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 -- 172.21.15.170:0/318896159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56015db10 msgr2=0x7fd560069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56015db10 0x7fd560069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 -- 172.21.15.170:0/318896159 >> v2:172.21.15.191:3300/0 conn(0x7fd560159c60 msgr2=0x7fd56015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 --2- 172.21.15.170:0/318896159 >> v2:172.21.15.191:3300/0 conn(0x7fd560159c60 0x7fd56015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 -- 172.21.15.170:0/318896159 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56015e140 con 0x7fd560159890 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55ffff700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd560159890 0x7fd56015abd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b67f0 tx=0x7fd550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a391293c9017755 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55effd700 1 -- 172.21.15.170:0/318896159 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000b640 con 0x7fd560159890 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55effd700 1 -- 172.21.15.170:0/318896159 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000b7a0 con 0x7fd560159890 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd55effd700 1 -- 172.21.15.170:0/318896159 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000bb60 con 0x7fd560159890 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 -- 172.21.15.170:0/318896159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd560159890 msgr2=0x7fd56015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd560159890 0x7fd56015abd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b67f0 tx=0x7fd550009250 comp rx=0 tx=0).stop 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 -- 172.21.15.170:0/318896159 shutdown_connections 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd56015db10 0x7fd560069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 --2- 172.21.15.170:0/318896159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd560159890 0x7fd56015abd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 --2- 172.21.15.170:0/318896159 >> v2:172.21.15.191:3300/0 conn(0x7fd560159c60 0x7fd56015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 -- 172.21.15.170:0/318896159 >> 172.21.15.170:0/318896159 conn(0x7fd560068e20 msgr2=0x7fd5600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 -- 172.21.15.170:0/318896159 shutdown_connections 2023-10-08T16:10:19.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.914+0000 7fd5664f0700 1 -- 172.21.15.170:0/318896159 wait complete. 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.915+0000 7fd5664f0700 1 Processor -- start 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 -- start start 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd560159890 0x7fd5600b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd560159c60 0x7fd5600c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560153650 con 0x7fd56015db10 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600c9dd0 con 0x7fd560159890 2023-10-08T16:10:19.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd5664f0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5600b3630 con 0x7fd560159c60 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.916+0000 7fd564a8d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54048/0 (socket says 172.21.15.170:54048) 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 -- 172.21.15.170:0/2459049932 learned_addr learned my addr 172.21.15.170:0/2459049932 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd55f7fe700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd560159c60 0x7fd5600c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd55ffff700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd560159890 0x7fd5600b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 -- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd560159c60 msgr2=0x7fd5600c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd560159c60 0x7fd5600c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 -- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd560159890 msgr2=0x7fd5600b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd560159890 0x7fd5600b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 -- 172.21.15.170:0/2459049932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500094f0 con 0x7fd56015db10 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd564a8d700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e90 tx=0x7fd55800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55800cb70 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55800f040 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5600b3910 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558014480 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.917+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5600b3ed0 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.918+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56004c3c0 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.918+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7fd558014830 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.919+0000 7fd547fff700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 0x7fd54c0696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.919+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd558085460 con 0x7fd56015db10 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.919+0000 7fd55ffff700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 0x7fd54c0696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:19.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.919+0000 7fd55ffff700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 0x7fd54c0696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b67f0 tx=0x7fd550017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:19.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:19.923+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd558052270 con 0x7fd56015db10 2023-10-08T16:10:20.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.150+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd56006a5b0 con 0x7fd56015db10 2023-10-08T16:10:20.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.151+0000 7fd547fff700 1 -- 172.21.15.170:0/2459049932 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fd558055890 con 0x7fd56015db10 2023-10-08T16:10:20.151 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:20.151 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 msgr2=0x7fd54c0696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 0x7fd54c0696b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd5600b67f0 tx=0x7fd550017000 comp rx=0 tx=0).stop 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 msgr2=0x7fd5600c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e90 tx=0x7fd55800b040 comp rx=0 tx=0).stop 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 shutdown_connections 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd54c067200 0x7fd54c0696b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd56015db10 0x7fd5600c9750 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd560159890 0x7fd5600b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 --2- 172.21.15.170:0/2459049932 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd560159c60 0x7fd5600c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 >> 172.21.15.170:0/2459049932 conn(0x7fd560068e20 msgr2=0x7fd5600c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 shutdown_connections 2023-10-08T16:10:20.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.153+0000 7fd5664f0700 1 -- 172.21.15.170:0/2459049932 wait complete. 2023-10-08T16:10:20.155 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 57 2023-10-08T16:10:20.166 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:20.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.280+0000 7f5facc31700 1 Processor -- start 2023-10-08T16:10:20.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.280+0000 7f5facc31700 1 -- start start 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5facc31700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5fa81597f0 0x7f5fa80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5facc31700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80b11d0 con 0x7f5fa8159420 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5facc31700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80b14b0 con 0x7f5fa815d230 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5facc31700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa80b1790 con 0x7f5fa81597f0 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa6ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5fa81597f0 0x7f5fa80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5fa8159420 0x7f5fa80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53746/0 (socket says 172.21.15.170:53746) 2023-10-08T16:10:20.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa77fe700 1 -- 172.21.15.170:0/3594758169 learned_addr learned my addr 172.21.15.170:0/3594758169 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa6ffd700 1 -- 172.21.15.170:0/3594758169 >> v1:172.21.15.191:6789/0 conn(0x7f5fa815d230 legacy=0x7f5fa815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa6ffd700 1 -- 172.21.15.170:0/3594758169 >> v1:172.21.15.170:6789/0 conn(0x7f5fa8159420 legacy=0x7f5fa80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa6ffd700 1 -- 172.21.15.170:0/3594758169 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80b1a70 con 0x7f5fa81597f0 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa67fc700 1 -- 172.21.15.170:0/3594758169 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468343226 0 0) 0x7f5fa80b11d0 con 0x7f5fa8159420 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa6ffd700 1 --2- 172.21.15.170:0/3594758169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa81597f0 0x7f5fa80b2270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0098e0 tx=0x7f5f9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add399d006e2a59e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.281+0000 7f5fa67fc700 1 -- 172.21.15.170:0/3594758169 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c009e80 con 0x7f5fa81597f0 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5fa67fc700 1 -- 172.21.15.170:0/3594758169 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00b820 con 0x7f5fa81597f0 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5fa67fc700 1 -- 172.21.15.170:0/3594758169 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00bba0 con 0x7f5fa81597f0 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 -- 172.21.15.170:0/3594758169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa81597f0 msgr2=0x7f5fa80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 --2- 172.21.15.170:0/3594758169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa81597f0 0x7f5fa80b2270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0098e0 tx=0x7f5f9c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 -- 172.21.15.170:0/3594758169 shutdown_connections 2023-10-08T16:10:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 --2- 172.21.15.170:0/3594758169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa81597f0 0x7f5fa80b2270 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 -- 172.21.15.170:0/3594758169 >> 172.21.15.170:0/3594758169 conn(0x7f5fa80d0660 msgr2=0x7f5fa80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 -- 172.21.15.170:0/3594758169 shutdown_connections 2023-10-08T16:10:20.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.282+0000 7f5facc31700 1 -- 172.21.15.170:0/3594758169 wait complete. 2023-10-08T16:10:20.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.283+0000 7f5facc31700 1 Processor -- start 2023-10-08T16:10:20.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.283+0000 7f5facc31700 1 -- start start 2023-10-08T16:10:20.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fa81597f0 0x7f5fa8154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fa815d230 0x7f5fa8155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80ca2e0 con 0x7f5fa8159420 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa815bbc0 con 0x7f5fa81597f0 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5facc31700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa80c5400 con 0x7f5fa815d230 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54062/0 (socket says 172.21.15.170:54062) 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 -- 172.21.15.170:0/501774033 learned_addr learned my addr 172.21.15.170:0/501774033 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 -- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fa815d230 msgr2=0x7f5fa8155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fa815d230 0x7f5fa8155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 -- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fa81597f0 msgr2=0x7f5fa8154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fa81597f0 0x7f5fa8154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 -- 172.21.15.170:0/501774033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c009420 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.284+0000 7f5fa77fe700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800cd80 tx=0x7f5f98001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.285+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9800ab10 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.285+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9800ac70 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.285+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f980070f0 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.285+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa80c96d0 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.285+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa80c9c90 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.286+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa804c3c0 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.286+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 10) v1 ==== 84995+0+0 (secure 0 0 0) 0x7f5f980074e0 con 0x7f5fa8159420 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.286+0000 7f5f8f7fe700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 0x7f5f940696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.286+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f98084eb0 con 0x7f5fa8159420 2023-10-08T16:10:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.286+0000 7f5fa6ffd700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 0x7f5f940696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.287+0000 7f5fa6ffd700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 0x7f5f940696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8159c70 tx=0x7f5f9c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.291+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f98051e50 con 0x7f5fa8159420 2023-10-08T16:10:20.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.476+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fa80b1d50 con 0x7f5fa8159420 2023-10-08T16:10:20.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.477+0000 7f5f8f7fe700 1 -- 172.21.15.170:0/501774033 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f5f98055470 con 0x7f5fa8159420 2023-10-08T16:10:20.477 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:20.477 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":89,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:05.843576+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 msgr2=0x7f5f940696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 0x7f5f940696b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8159c70 tx=0x7f5f9c016000 comp rx=0 tx=0).stop 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 msgr2=0x7f5fa815b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800cd80 tx=0x7f5f98001ae0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 shutdown_connections 2023-10-08T16:10:20.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5fa815d230 0x7f5fa8155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f94067200 0x7f5f940696b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5fa8159420 0x7f5fa815b680 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 --2- 172.21.15.170:0/501774033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5fa81597f0 0x7f5fa8154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 >> 172.21.15.170:0/501774033 conn(0x7f5fa80d0660 msgr2=0x7f5fa8069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 shutdown_connections 2023-10-08T16:10:20.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.479+0000 7f5facc31700 1 -- 172.21.15.170:0/501774033 wait complete. 2023-10-08T16:10:20.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:10:02.565366+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:10:03.191465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:10:20.492 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:10:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.610+0000 7f5f236de700 1 Processor -- start 2023-10-08T16:10:20.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.610+0000 7f5f236de700 1 -- start start 2023-10-08T16:10:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f236de700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5f1c0cd1e0 0x7f5f1c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f236de700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c15c2a0 con 0x7f5f1c06bcd0 2023-10-08T16:10:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f236de700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c15b690 con 0x7f5f1c141500 2023-10-08T16:10:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f236de700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c15b970 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f21c7b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5f1c0cd1e0 0x7f5f1c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f2147a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5f1c06bcd0 0x7f5f1c140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53760/0 (socket says 172.21.15.170:53760) 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.611+0000 7f5f2147a700 1 -- 172.21.15.170:0/1393600794 learned_addr learned my addr 172.21.15.170:0/1393600794 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f21c7b700 1 -- 172.21.15.170:0/1393600794 >> v1:172.21.15.191:6789/0 conn(0x7f5f1c141500 legacy=0x7f5f1c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f21c7b700 1 -- 172.21.15.170:0/1393600794 >> v1:172.21.15.170:6789/0 conn(0x7f5f1c06bcd0 legacy=0x7f5f1c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f21c7b700 1 -- 172.21.15.170:0/1393600794 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c15bc50 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f13fff700 1 -- 172.21.15.170:0/1393600794 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709202602 0 0) 0x7f5f1c15c2a0 con 0x7f5f1c06bcd0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f21c7b700 1 --2- 172.21.15.170:0/1393600794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c0c9510 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098e0 tx=0x7f5f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e45b0c12998df9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f13fff700 1 -- 172.21.15.170:0/1393600794 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14009e80 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f13fff700 1 -- 172.21.15.170:0/1393600794 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1400b820 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f13fff700 1 -- 172.21.15.170:0/1393600794 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400bba0 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 -- 172.21.15.170:0/1393600794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0cd1e0 msgr2=0x7f5f1c0c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 --2- 172.21.15.170:0/1393600794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c0c9510 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5f140098e0 tx=0x7f5f140091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 -- 172.21.15.170:0/1393600794 shutdown_connections 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 --2- 172.21.15.170:0/1393600794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c0c9510 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 -- 172.21.15.170:0/1393600794 >> 172.21.15.170:0/1393600794 conn(0x7f5f1c0ce600 msgr2=0x7f5f1c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 -- 172.21.15.170:0/1393600794 shutdown_connections 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.612+0000 7f5f236de700 1 -- 172.21.15.170:0/1393600794 wait complete. 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.613+0000 7f5f236de700 1 Processor -- start 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.613+0000 7f5f236de700 1 -- start start 2023-10-08T16:10:20.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c06bcd0 0x7f5f1c16b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c13be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c154810 con 0x7f5f1c141500 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c153c00 con 0x7f5f1c0cd1e0 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f236de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c153ee0 con 0x7f5f1c06bcd0 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.614+0000 7f5f21c7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54082/0 (socket says 172.21.15.170:54082) 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 -- 172.21.15.170:0/822807544 learned_addr learned my addr 172.21.15.170:0/822807544 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f2147a700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c06bcd0 0x7f5f1c16b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f20c79700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c13be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 -- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c06bcd0 msgr2=0x7f5f1c16b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c06bcd0 0x7f5f1c16b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 -- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c0cd1e0 msgr2=0x7f5f1c13be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c13be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 -- 172.21.15.170:0/822807544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f14009420 con 0x7f5f1c141500 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f21c7b700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5f140093f0 tx=0x7f5f14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400bb60 con 0x7f5f1c141500 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f14020070 con 0x7f5f1c141500 2023-10-08T16:10:20.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c1541c0 con 0x7f5f1c141500 2023-10-08T16:10:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1401b6a0 con 0x7f5f1c141500 2023-10-08T16:10:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.615+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c16be10 con 0x7f5f1c141500 2023-10-08T16:10:20.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.616+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c16c000 con 0x7f5f1c141500 2023-10-08T16:10:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.616+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5f14022030 con 0x7f5f1c141500 2023-10-08T16:10:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.617+0000 7f5f117fa700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 0x7f5f00069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.617+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f14086b70 con 0x7f5f1c141500 2023-10-08T16:10:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.617+0000 7f5f2147a700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 0x7f5f00069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.617+0000 7f5f2147a700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 0x7f5f00069740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f18002250 tx=0x7f5f18001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.621+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f1c16c000 con 0x7f5f1c141500 2023-10-08T16:10:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.801+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f1c16c000 con 0x7f5f1c141500 2023-10-08T16:10:20.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.802+0000 7f5f117fa700 1 -- 172.21.15.170:0/822807544 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f1c16c000 con 0x7f5f1c141500 2023-10-08T16:10:20.803 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 msgr2=0x7f5f00069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 0x7f5f00069740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f18002250 tx=0x7f5f18001a20 comp rx=0 tx=0).stop 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 msgr2=0x7f5f1c13c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5f140093f0 tx=0x7f5f14016000 comp rx=0 tx=0).stop 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 shutdown_connections 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5f00067290 0x7f5f00069740 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5f1c141500 0x7f5f1c13c340 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5f1c06bcd0 0x7f5f1c16b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 --2- 172.21.15.170:0/822807544 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5f1c0cd1e0 0x7f5f1c13be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 >> 172.21.15.170:0/822807544 conn(0x7f5f1c0ce600 msgr2=0x7f5f1c0c3920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 shutdown_connections 2023-10-08T16:10:20.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.804+0000 7f5f236de700 1 -- 172.21.15.170:0/822807544 wait complete. 2023-10-08T16:10:20.817 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:10:20.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.936+0000 7f00908bf700 1 Processor -- start 2023-10-08T16:10:20.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.936+0000 7f00908bf700 1 -- start start 2023-10-08T16:10:20.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 0x7f008c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 0x7f008c15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f008c15df50 0x7f008c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c15d7e0 con 0x7f008c15df50 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f008c15dac0 con 0x7f008c06bcd0 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f00908bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c15aa30 con 0x7f008c06c0a0 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f008ad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f008c15df50 0x7f008c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f008ad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f008c15df50 0x7f008c15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54084/0 (socket says 172.21.15.170:54084) 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.937+0000 7f008ad9d700 1 -- 172.21.15.170:0/694040981 learned_addr learned my addr 172.21.15.170:0/694040981 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f0089d9b700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 0x7f008c15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008a59c700 1 --2- 172.21.15.170:0/694040981 >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 0x7f008c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 -- 172.21.15.170:0/694040981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 msgr2=0x7f008c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 0x7f008c15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 -- 172.21.15.170:0/694040981 >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 msgr2=0x7f008c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:20.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 --2- 172.21.15.170:0/694040981 >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 0x7f008c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 -- 172.21.15.170:0/694040981 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f008c15ad10 con 0x7f008c15df50 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f0089d9b700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 0x7f008c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008a59c700 1 --2- 172.21.15.170:0/694040981 >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 0x7f008c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008ad9d700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c15e3c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f008c0691a0 tx=0x7f0084009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac714619cfa04f97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008959a700 1 -- 172.21.15.170:0/694040981 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400b860 con 0x7f008c15df50 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008959a700 1 -- 172.21.15.170:0/694040981 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008400b9c0 con 0x7f008c15df50 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.938+0000 7f008959a700 1 -- 172.21.15.170:0/694040981 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400bd80 con 0x7f008c15df50 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 -- 172.21.15.170:0/694040981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 msgr2=0x7f008c15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c15e3c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f008c0691a0 tx=0x7f0084009250 comp rx=0 tx=0).stop 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 -- 172.21.15.170:0/694040981 shutdown_connections 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c15e3c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 --2- 172.21.15.170:0/694040981 >> v2:172.21.15.191:3300/0 conn(0x7f008c06bcd0 0x7f008c0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 --2- 172.21.15.170:0/694040981 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06c0a0 0x7f008c15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 -- 172.21.15.170:0/694040981 >> 172.21.15.170:0/694040981 conn(0x7f008c0ce600 msgr2=0x7f008c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 -- 172.21.15.170:0/694040981 shutdown_connections 2023-10-08T16:10:20.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.939+0000 7f00908bf700 1 -- 172.21.15.170:0/694040981 wait complete. 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.940+0000 7f00908bf700 1 Processor -- start 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.940+0000 7f00908bf700 1 -- start start 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06bcd0 0x7f008c169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 0x7f008c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0b2e60 con 0x7f008c15df50 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008c156610 con 0x7f008c06c0a0 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008c0b2530 con 0x7f008c06bcd0 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54090/0 (socket says 172.21.15.170:54090) 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 -- 172.21.15.170:0/3240571536 learned_addr learned my addr 172.21.15.170:0/3240571536 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008a59c700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06bcd0 0x7f008c169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f0089d9b700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 0x7f008c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 -- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06bcd0 msgr2=0x7f008c169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06bcd0 0x7f008c169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 -- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 msgr2=0x7f008c14b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 0x7f008c14b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 -- 172.21.15.170:0/3240571536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840094f0 con 0x7f008c15df50 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f0089d9b700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 0x7f008c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f008ad9d700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f008c069220 tx=0x7f0084017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008400b860 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084006a90 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c0b2810 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008401a880 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.941+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008c14b710 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.942+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008c0b2fd0 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.942+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f008401abf0 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.943+0000 7f007a7fc700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 0x7f006c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.943+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f00840866b0 con 0x7f008c15df50 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.943+0000 7f008a59c700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 0x7f006c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:20.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.943+0000 7f008a59c700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 0x7f006c069740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f007c0076a0 tx=0x7f007c009930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:20.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:20.947+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0084053590 con 0x7f008c15df50 2023-10-08T16:10:21.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.128+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f008c04c3c0 con 0x7f008c15df50 2023-10-08T16:10:21.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.128+0000 7f007a7fc700 1 -- 172.21.15.170:0/3240571536 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00840150a0 con 0x7f008c15df50 2023-10-08T16:10:21.129 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 msgr2=0x7f006c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 0x7f006c069740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f007c0076a0 tx=0x7f007c009930 comp rx=0 tx=0).stop 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 msgr2=0x7f008c158db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f008c069220 tx=0x7f0084017000 comp rx=0 tx=0).stop 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 shutdown_connections 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f006c067290 0x7f006c069740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f008c15df50 0x7f008c158db0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f008c06bcd0 0x7f008c169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 --2- 172.21.15.170:0/3240571536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f008c06c0a0 0x7f008c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 >> 172.21.15.170:0/3240571536 conn(0x7f008c0ce600 msgr2=0x7f008c157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 shutdown_connections 2023-10-08T16:10:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.130+0000 7f00908bf700 1 -- 172.21.15.170:0/3240571536 wait complete. 2023-10-08T16:10:21.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:10:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.260+0000 7f59176d5700 1 Processor -- start 2023-10-08T16:10:21.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.260+0000 7f59176d5700 1 -- start start 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f59176d5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5910159890 0x7f591015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f59176d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5910069560 0x7f59100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f59176d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5910069ee0 con 0x7f5910159c60 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f59176d5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591015a160 con 0x7f5910159890 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f59176d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59100d13a0 con 0x7f5910069560 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5910159890 0x7f591015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5910159890 0x7f591015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54102/0 (socket says 172.21.15.170:54102) 2023-10-08T16:10:21.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 -- 172.21.15.170:0/4223104970 learned_addr learned my addr 172.21.15.170:0/4223104970 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915c72700 1 --2- 172.21.15.170:0/4223104970 >> v2:172.21.15.191:3300/0 conn(0x7f5910069560 0x7f59100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 -- 172.21.15.170:0/4223104970 >> v2:172.21.15.191:3300/0 conn(0x7f5910069560 msgr2=0x7f59100699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 --2- 172.21.15.170:0/4223104970 >> v2:172.21.15.191:3300/0 conn(0x7f5910069560 0x7f59100699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 -- 172.21.15.170:0/4223104970 >> v1:172.21.15.191:6789/0 conn(0x7f5910159c60 legacy=0x7f591015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.261+0000 7f5915471700 1 -- 172.21.15.170:0/4223104970 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591015e140 con 0x7f5910159890 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f5915471700 1 --2- 172.21.15.170:0/4223104970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f591015abd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f591006a890 tx=0x7f5900009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff3337d0fa93eed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f590ffff700 1 -- 172.21.15.170:0/4223104970 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590000e070 con 0x7f5910159890 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f590ffff700 1 -- 172.21.15.170:0/4223104970 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590000ba50 con 0x7f5910159890 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f590ffff700 1 -- 172.21.15.170:0/4223104970 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000092e0 con 0x7f5910159890 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 -- 172.21.15.170:0/4223104970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 msgr2=0x7f591015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 --2- 172.21.15.170:0/4223104970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f591015abd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f591006a890 tx=0x7f5900009840 comp rx=0 tx=0).stop 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 -- 172.21.15.170:0/4223104970 shutdown_connections 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 --2- 172.21.15.170:0/4223104970 >> v2:172.21.15.191:3300/0 conn(0x7f5910069560 0x7f59100699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 --2- 172.21.15.170:0/4223104970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f591015abd0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 -- 172.21.15.170:0/4223104970 >> 172.21.15.170:0/4223104970 conn(0x7f5910068e20 msgr2=0x7f59100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 -- 172.21.15.170:0/4223104970 shutdown_connections 2023-10-08T16:10:21.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.262+0000 7f59176d5700 1 -- 172.21.15.170:0/4223104970 wait complete. 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.263+0000 7f59176d5700 1 Processor -- start 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.263+0000 7f59176d5700 1 -- start start 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5910069560 0x7f59100b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910159c60 0x7f5910152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59100b36c0 con 0x7f5910159890 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59100c87a0 con 0x7f5910159c60 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f59176d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59100c8a70 con 0x7f5910069560 2023-10-08T16:10:21.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f5914c70700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f5914c70700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54106/0 (socket says 172.21.15.170:54106) 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f5914c70700 1 -- 172.21.15.170:0/2643678811 learned_addr learned my addr 172.21.15.170:0/2643678811 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.264+0000 7f5915c72700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910159c60 0x7f5910152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5915471700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5910069560 0x7f59100b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 -- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5910069560 msgr2=0x7f59100b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5910069560 0x7f59100b60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 -- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910159c60 msgr2=0x7f5910152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910159c60 0x7f5910152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 -- 172.21.15.170:0/2643678811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900009ae0 con 0x7f5910159890 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f5914c70700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f590400c4e0 tx=0x7f590400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904011590 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904011710 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59100c8d50 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904011ad0 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.265+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59100c9310 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.266+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59100b3870 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.266+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f590401b3b0 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.267+0000 7f590d7fa700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 0x7f58fc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.267+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5904083e00 con 0x7f5910159890 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.267+0000 7f5915471700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 0x7f58fc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.267+0000 7f5915471700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 0x7f58fc069740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5900009880 tx=0x7f5900018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.271+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59100b3870 con 0x7f5910159890 2023-10-08T16:10:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.451+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f591004c3c0 con 0x7f5910159890 2023-10-08T16:10:21.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.452+0000 7f590d7fa700 1 -- 172.21.15.170:0/2643678811 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5904017070 con 0x7f5910159890 2023-10-08T16:10:21.452 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 msgr2=0x7f58fc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 0x7f58fc069740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5900009880 tx=0x7f5900018000 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 msgr2=0x7f59100c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f590400c4e0 tx=0x7f590400c7f0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 shutdown_connections 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5910159c60 0x7f5910152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5910069560 0x7f59100b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f58fc067290 0x7f58fc069740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 --2- 172.21.15.170:0/2643678811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5910159890 0x7f59100c8260 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 >> 172.21.15.170:0/2643678811 conn(0x7f5910068e20 msgr2=0x7f5910153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 shutdown_connections 2023-10-08T16:10:21.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.454+0000 7f59176d5700 1 -- 172.21.15.170:0/2643678811 wait complete. 2023-10-08T16:10:21.466 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:21.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.580+0000 7f817b4ec700 1 Processor -- start 2023-10-08T16:10:21.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.580+0000 7f817b4ec700 1 -- start start 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f817b4ec700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81740b1420 0x7f81740ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f817b4ec700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740cc300 con 0x7f81740ccf90 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f817b4ec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740ad370 con 0x7f81740cbf30 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f817b4ec700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8174156bf0 con 0x7f81740b1420 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f8178a87700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81740b1420 0x7f81740ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f8179a89700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f81740ccf90 0x7f81740c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53762/0 (socket says 172.21.15.170:53762) 2023-10-08T16:10:21.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.581+0000 7f8178a87700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f81740b1420 0x7f81740ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54116/0 (socket says 172.21.15.170:54116) 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f8179a89700 1 -- 172.21.15.170:0/1930471321 learned_addr learned my addr 172.21.15.170:0/1930471321 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f8178a87700 1 -- 172.21.15.170:0/1930471321 >> v1:172.21.15.191:6789/0 conn(0x7f81740cbf30 legacy=0x7f81740b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f8178a87700 1 -- 172.21.15.170:0/1930471321 >> v1:172.21.15.170:6789/0 conn(0x7f81740ccf90 legacy=0x7f81740c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f8178a87700 1 -- 172.21.15.170:0/1930471321 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81740ad650 con 0x7f81740b1420 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f816bfff700 1 -- 172.21.15.170:0/1930471321 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679655264 0 0) 0x7f81740cc300 con 0x7f81740ccf90 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f8178a87700 1 --2- 172.21.15.170:0/1930471321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740b1420 0x7f81740ae410 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f81640098e0 tx=0x7f81640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e589142efef7a02b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f816bfff700 1 -- 172.21.15.170:0/1930471321 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164009e80 con 0x7f81740b1420 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f816bfff700 1 -- 172.21.15.170:0/1930471321 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816400b860 con 0x7f81740b1420 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.582+0000 7f816bfff700 1 -- 172.21.15.170:0/1930471321 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816400bbe0 con 0x7f81740b1420 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 -- 172.21.15.170:0/1930471321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740b1420 msgr2=0x7f81740ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 --2- 172.21.15.170:0/1930471321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740b1420 0x7f81740ae410 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f81640098e0 tx=0x7f81640091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 -- 172.21.15.170:0/1930471321 shutdown_connections 2023-10-08T16:10:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 --2- 172.21.15.170:0/1930471321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740b1420 0x7f81740ae410 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 -- 172.21.15.170:0/1930471321 >> 172.21.15.170:0/1930471321 conn(0x7f817406a320 msgr2=0x7f81740cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 -- 172.21.15.170:0/1930471321 shutdown_connections 2023-10-08T16:10:21.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.583+0000 7f817b4ec700 1 -- 172.21.15.170:0/1930471321 wait complete. 2023-10-08T16:10:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.584+0000 7f817b4ec700 1 Processor -- start 2023-10-08T16:10:21.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.584+0000 7f817b4ec700 1 -- start start 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 0x7f817416b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81740ccf90 0x7f8174167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817416c720 con 0x7f81740cbf30 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740c2c80 con 0x7f81740ccf90 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817416bb10 con 0x7f81740b1420 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54126/0 (socket says 172.21.15.170:54126) 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 -- 172.21.15.170:0/2010953498 learned_addr learned my addr 172.21.15.170:0/2010953498 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 -- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 msgr2=0x7f817416b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8179288700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 0x7f817416b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8179a89700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81740ccf90 0x7f8174167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 0x7f817416b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 -- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81740ccf90 msgr2=0x7f8174167fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81740ccf90 0x7f8174167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 -- 172.21.15.170:0/2010953498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164009420 con 0x7f81740cbf30 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8179288700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 0x7f817416b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f8178a87700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8164013000 tx=0x7f8164017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164011420 con 0x7f81740cbf30 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8164011580 con 0x7f81740cbf30 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817416bdf0 con 0x7f81740cbf30 2023-10-08T16:10:21.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164011920 con 0x7f81740cbf30 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.585+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817416c3b0 con 0x7f81740cbf30 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.586+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817404c3c0 con 0x7f81740cbf30 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.587+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8164011c90 con 0x7f81740cbf30 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.587+0000 7f81697fa700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 0x7f8158069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.587+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8164086620 con 0x7f81740cbf30 2023-10-08T16:10:21.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.587+0000 7f8179288700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 0x7f8158069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.588+0000 7f8179288700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 0x7f8158069740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8170001840 tx=0x7f8170009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.591+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8164053500 con 0x7f81740cbf30 2023-10-08T16:10:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.814+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81740b0400 con 0x7f81740cbf30 2023-10-08T16:10:21.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.815+0000 7f81697fa700 1 -- 172.21.15.170:0/2010953498 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f8164056b20 con 0x7f81740cbf30 2023-10-08T16:10:21.815 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:21.815 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 msgr2=0x7f8158069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 0x7f8158069740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8170001840 tx=0x7f8170009040 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 msgr2=0x7f81740c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8164013000 tx=0x7f8164017000 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 shutdown_connections 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81740ccf90 0x7f8174167fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81740b1420 0x7f817416b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8158067290 0x7f8158069740 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 --2- 172.21.15.170:0/2010953498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81740cbf30 0x7f81740c2740 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 >> 172.21.15.170:0/2010953498 conn(0x7f817406a320 msgr2=0x7f81740c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 shutdown_connections 2023-10-08T16:10:21.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.817+0000 7f817b4ec700 1 -- 172.21.15.170:0/2010953498 wait complete. 2023-10-08T16:10:21.819 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 57 2023-10-08T16:10:21.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.943+0000 7f5c35375700 1 Processor -- start 2023-10-08T16:10:21.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.943+0000 7f5c35375700 1 -- start start 2023-10-08T16:10:21.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c35375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c3015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c35375700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3015a160 con 0x7f5c30069560 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c35375700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c300d13a0 con 0x7f5c30159c60 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c35375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3015e140 con 0x7f5c30159890 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5c30069560 0x7f5c300ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53774/0 (socket says 172.21.15.170:53774) 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/2774517329 learned_addr learned my addr 172.21.15.170:0/2774517329 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2effd700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c3015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477561411 0 0) 0x7f5c3015a160 con 0x7f5c30069560 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c18003570 con 0x7f5c30069560 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.944+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358681533 0 0) 0x7f5c18003570 con 0x7f5c30069560 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 msgr2=0x7f5c3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c3015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 >> v1:172.21.15.191:6789/0 conn(0x7f5c30159c60 legacy=0x7f5c3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3015a3d0 con 0x7f5c30069560 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f5c280017f0 con 0x7f5c30069560 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 >> v1:172.21.15.170:6789/0 conn(0x7f5c30069560 legacy=0x7f5c300ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c18003a90 0x7f5c18003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18004810 0x7f5c18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3015e140 con 0x7f5c18003a90 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1800bb10 con 0x7f5c18004810 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1800bcd0 con 0x7f5c30159890 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c18003a90 0x7f5c18003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2f7fe700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2e7fc700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18004810 0x7f5c18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 -- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 msgr2=0x7f5c180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c30159890 0x7f5c180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 -- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18004810 msgr2=0x7f5c18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18004810 0x7f5c18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 -- 172.21.15.170:0/2774517329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18003570 con 0x7f5c18003a90 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2effd700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c18003a90 0x7f5c18003e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009c00 tx=0x7f5c200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f82527eabf4142 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000e710 con 0x7f5c18003a90 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2000e870 con 0x7f5c18003a90 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.945+0000 7f5c2dffb700 1 -- 172.21.15.170:0/2774517329 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000ec30 con 0x7f5c18003a90 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 -- 172.21.15.170:0/2774517329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c18003a90 msgr2=0x7f5c18003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 --2- 172.21.15.170:0/2774517329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c18003a90 0x7f5c18003e60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009c00 tx=0x7f5c200091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/2774517329 reap_dead start 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 -- 172.21.15.170:0/2774517329 shutdown_connections 2023-10-08T16:10:21.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 -- 172.21.15.170:0/2774517329 >> 172.21.15.170:0/2774517329 conn(0x7f5c30068e20 msgr2=0x7f5c300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:21.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 -- 172.21.15.170:0/2774517329 shutdown_connections 2023-10-08T16:10:21.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.946+0000 7f5c35375700 1 -- 172.21.15.170:0/2774517329 wait complete. 2023-10-08T16:10:21.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.947+0000 7f5c35375700 1 Processor -- start 2023-10-08T16:10:21.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.947+0000 7f5c35375700 1 -- start start 2023-10-08T16:10:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18003a90 0x7f5c30144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 0x7f5c3015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30151e50 con 0x7f5c300c7d10 2023-10-08T16:10:21.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3015ce80 con 0x7f5c18003a90 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c35375700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3015bf70 con 0x7f5c300c78a0 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54140/0 (socket says 172.21.15.170:54140) 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/1629284645 learned_addr learned my addr 172.21.15.170:0/1629284645 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2e7fc700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 0x7f5c3015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2effd700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18003a90 0x7f5c30144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 msgr2=0x7f5c3015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 0x7f5c3015c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18003a90 msgr2=0x7f5c30144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18003a90 0x7f5c30144350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:21.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 -- 172.21.15.170:0/1629284645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30144890 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2e7fc700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 0x7f5c3015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.948+0000 7f5c2f7fe700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5c3006a890 tx=0x7f5c280021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.949+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800ae00 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.949+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c280133f0 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.949+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280137b0 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.949+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c300cd4b0 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.949+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c300ca240 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.950+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3004c3c0 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.950+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f5c28013be0 con 0x7f5c300c7d10 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.950+0000 7f5c16ffd700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 0x7f5c1c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:21.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.950+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c280854c0 con 0x7f5c300c7d10 2023-10-08T16:10:21.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.950+0000 7f5c2effd700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 0x7f5c1c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:21.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.951+0000 7f5c2effd700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 0x7f5c1c069740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c200091e0 tx=0x7f5c2001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:21.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:21.955+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c28052320 con 0x7f5c300c7d10 2023-10-08T16:10:22.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.140+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c3015c0b0 con 0x7f5c300c7d10 2023-10-08T16:10:22.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.141+0000 7f5c16ffd700 1 -- 172.21.15.170:0/1629284645 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f5c28055940 con 0x7f5c300c7d10 2023-10-08T16:10:22.141 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:22.142 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":89,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:05.843576+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 msgr2=0x7f5c1c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 0x7f5c1c069740 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c200091e0 tx=0x7f5c2001a000 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 msgr2=0x7f5c3015c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5c3006a890 tx=0x7f5c280021f0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 shutdown_connections 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c1c067290 0x7f5c1c069740 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5c300c7d10 0x7f5c3015c940 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5c18003a90 0x7f5c30144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 --2- 172.21.15.170:0/1629284645 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5c300c78a0 0x7f5c3015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 >> 172.21.15.170:0/1629284645 conn(0x7f5c30068e20 msgr2=0x7f5c301522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:22.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 shutdown_connections 2023-10-08T16:10:22.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.143+0000 7f5c35375700 1 -- 172.21.15.170:0/1629284645 wait complete. 2023-10-08T16:10:22.156 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:10:22.156 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.269+0000 7f8ccf64a700 1 Processor -- start 2023-10-08T16:10:22.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.269+0000 7f8ccf64a700 1 -- start start 2023-10-08T16:10:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccf64a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc8159890 0x7f8cc815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccf64a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 0x7f8cc80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccf64a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc8069ee0 con 0x7f8cc8159c60 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccf64a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc815a160 con 0x7f8cc8159890 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8cccbe5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8cc8159c60 0x7f8cc815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53776/0 (socket says 172.21.15.170:53776) 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccf64a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80d13a0 con 0x7f8cc8069560 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8ccd3e6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc8159890 0x7f8cc815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.270+0000 7f8cccbe5700 1 -- 172.21.15.170:0/629842348 learned_addr learned my addr 172.21.15.170:0/629842348 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccdbe7700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 0x7f8cc80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccd3e6700 1 -- 172.21.15.170:0/629842348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 msgr2=0x7f8cc80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:22.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccd3e6700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 0x7f8cc80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccd3e6700 1 -- 172.21.15.170:0/629842348 >> v1:172.21.15.170:6789/0 conn(0x7f8cc8159c60 legacy=0x7f8cc815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8cbffff700 1 -- 172.21.15.170:0/629842348 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859396502 0 0) 0x7f8cc8069ee0 con 0x7f8cc8159c60 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccd3e6700 1 -- 172.21.15.170:0/629842348 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc815e140 con 0x7f8cc8159890 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccdbe7700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 0x7f8cc80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8ccd3e6700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159890 0x7f8cc815abd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098e0 tx=0x7f8cc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40c51975952e22b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8cbffff700 1 -- 172.21.15.170:0/629842348 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4009e80 con 0x7f8cc8159890 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8cbffff700 1 -- 172.21.15.170:0/629842348 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400b860 con 0x7f8cc8159890 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.271+0000 7f8cbffff700 1 -- 172.21.15.170:0/629842348 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400bbe0 con 0x7f8cc8159890 2023-10-08T16:10:22.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 -- 172.21.15.170:0/629842348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159890 msgr2=0x7f8cc815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159890 0x7f8cc815abd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40098e0 tx=0x7f8cc40091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 -- 172.21.15.170:0/629842348 shutdown_connections 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8069560 0x7f8cc80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/629842348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159890 0x7f8cc815abd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 -- 172.21.15.170:0/629842348 >> 172.21.15.170:0/629842348 conn(0x7f8cc8068e20 msgr2=0x7f8cc80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 -- 172.21.15.170:0/629842348 shutdown_connections 2023-10-08T16:10:22.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.272+0000 7f8ccf64a700 1 -- 172.21.15.170:0/629842348 wait complete. 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.273+0000 7f8ccf64a700 1 Processor -- start 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.273+0000 7f8ccf64a700 1 -- start start 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8069560 0x7f8cc8154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8159890 0x7f8cc80c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc814d020 con 0x7f8cc8159c60 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80c9000 con 0x7f8cc8069560 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80cd9a0 con 0x7f8cc8159890 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54156/0 (socket says 172.21.15.170:54156) 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 -- 172.21.15.170:0/1581757042 learned_addr learned my addr 172.21.15.170:0/1581757042 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 -- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8159890 msgr2=0x7f8cc80c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8cccbe5700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8159890 0x7f8cc80c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8159890 0x7f8cc80c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccd3e6700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8069560 0x7f8cc8154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 -- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8069560 msgr2=0x7f8cc8154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8069560 0x7f8cc8154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 -- 172.21.15.170:0/1581757042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009420 con 0x7f8cc8159c60 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccdbe7700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000d100 tx=0x7f8cc00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000dda0 con 0x7f8cc8159c60 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000f040 con 0x7f8cc8159c60 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80c9210 con 0x7f8cc8159c60 2023-10-08T16:10:22.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.274+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0015690 con 0x7f8cc8159c60 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.275+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc80c97d0 con 0x7f8cc8159c60 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.275+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc804c3c0 con 0x7f8cc8159c60 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.276+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8cc001e3b0 con 0x7f8cc8159c60 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.276+0000 7f8cbd7fa700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 0x7f8cac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.276+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8cc00851e0 con 0x7f8cc8159c60 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.276+0000 7f8ccd3e6700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 0x7f8cac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.277+0000 7f8ccd3e6700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 0x7f8cac069740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4003080 tx=0x7f8cc4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.282 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.280+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cc0051ef0 con 0x7f8cc8159c60 2023-10-08T16:10:22.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.507+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc806a5b0 con 0x7f8cc8159c60 2023-10-08T16:10:22.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.508+0000 7f8cbd7fa700 1 -- 172.21.15.170:0/1581757042 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f8cc0055510 con 0x7f8cc8159c60 2023-10-08T16:10:22.578 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:22.597 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 msgr2=0x7f8cac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 0x7f8cac069740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4003080 tx=0x7f8cc4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 msgr2=0x7f8cc80cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000d100 tx=0x7f8cc00099b0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 shutdown_connections 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cac067290 0x7f8cac069740 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc8159c60 0x7f8cc80cf890 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc8069560 0x7f8cc8154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 --2- 172.21.15.170:0/1581757042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc8159890 0x7f8cc80c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 >> 172.21.15.170:0/1581757042 conn(0x7f8cc8068e20 msgr2=0x7f8cc81587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 shutdown_connections 2023-10-08T16:10:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.510+0000 7f8ccf64a700 1 -- 172.21.15.170:0/1581757042 wait complete. 2023-10-08T16:10:22.656 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 57 2023-10-08T16:10:22.656 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:22.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.770+0000 7fc4c7abc700 1 Processor -- start 2023-10-08T16:10:22.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.770+0000 7fc4c7abc700 1 -- start start 2023-10-08T16:10:22.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c7abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006c0a0 0x7fc4c00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c7abc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc4c015df50 0x7fc4c015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c7abc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c015d7e0 con 0x7fc4c006bcd0 2023-10-08T16:10:22.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c7abc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c015dac0 con 0x7fc4c015df50 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c7abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015aa30 con 0x7fc4c006c0a0 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c6059700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc4c015df50 0x7fc4c015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c5858700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc4c006bcd0 0x7fc4c015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53788/0 (socket says 172.21.15.170:53788) 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.771+0000 7fc4c5858700 1 -- 172.21.15.170:0/485489794 learned_addr learned my addr 172.21.15.170:0/485489794 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c5057700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006c0a0 0x7fc4c00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c6059700 1 -- 172.21.15.170:0/485489794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006c0a0 msgr2=0x7fc4c00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c6059700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006c0a0 0x7fc4c00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c6059700 1 -- 172.21.15.170:0/485489794 >> v1:172.21.15.170:6789/0 conn(0x7fc4c006bcd0 legacy=0x7fc4c015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c6059700 1 -- 172.21.15.170:0/485489794 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c015ad10 con 0x7fc4c015df50 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c4856700 1 -- 172.21.15.170:0/485489794 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725165802 0 0) 0x7fc4c015d7e0 con 0x7fc4c006bcd0 2023-10-08T16:10:22.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c6059700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c015df50 0x7fc4c015e3c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098e0 tx=0x7fc4b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24cad0e06bb9840 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c4856700 1 -- 172.21.15.170:0/485489794 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8009e80 con 0x7fc4c015df50 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c4856700 1 -- 172.21.15.170:0/485489794 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800b860 con 0x7fc4c015df50 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c4856700 1 -- 172.21.15.170:0/485489794 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800bbe0 con 0x7fc4c015df50 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c7abc700 1 -- 172.21.15.170:0/485489794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c015df50 msgr2=0x7fc4c015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c015df50 0x7fc4c015e3c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098e0 tx=0x7fc4b80091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c7abc700 1 -- 172.21.15.170:0/485489794 shutdown_connections 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c015df50 0x7fc4c015e3c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.772+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/485489794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006c0a0 0x7fc4c00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.773+0000 7fc4c7abc700 1 -- 172.21.15.170:0/485489794 >> 172.21.15.170:0/485489794 conn(0x7fc4c00ce600 msgr2=0x7fc4c00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.773+0000 7fc4c7abc700 1 -- 172.21.15.170:0/485489794 shutdown_connections 2023-10-08T16:10:22.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.773+0000 7fc4c7abc700 1 -- 172.21.15.170:0/485489794 wait complete. 2023-10-08T16:10:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 Processor -- start 2023-10-08T16:10:22.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 -- start start 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006bcd0 0x7fc4c00c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c015df50 0x7fc4c00b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00b3540 con 0x7fc4c006c0a0 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00b2930 con 0x7fc4c015df50 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.774+0000 7fc4c7abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c00b2c10 con 0x7fc4c006bcd0 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54172/0 (socket says 172.21.15.170:54172) 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 -- 172.21.15.170:0/1469175986 learned_addr learned my addr 172.21.15.170:0/1469175986 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5858700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006bcd0 0x7fc4c00c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c6059700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c015df50 0x7fc4c00b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 -- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006bcd0 msgr2=0x7fc4c00c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006bcd0 0x7fc4c00c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 -- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c015df50 msgr2=0x7fc4c00b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c015df50 0x7fc4c00b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 -- 172.21.15.170:0/1469175986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8009420 con 0x7fc4c006c0a0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c5057700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d860 tx=0x7fc4b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40125f0 con 0x7fc4c006c0a0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b4012750 con 0x7fc4c006c0a0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c00b2ef0 con 0x7fc4c006c0a0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b4012b10 con 0x7fc4c006c0a0 2023-10-08T16:10:22.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.775+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00b25e0 con 0x7fc4c006c0a0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.776+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00b36b0 con 0x7fc4c006c0a0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.777+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc4b401e460 con 0x7fc4c006c0a0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.777+0000 7fc4bdffb700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 0x7fc4ac069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.777+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4b4084fe0 con 0x7fc4c006c0a0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.777+0000 7fc4c5858700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 0x7fc4ac069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.778+0000 7fc4c5858700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 0x7fc4ac069740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007240 tx=0x7fc4b0001970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:22.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:22.781+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4b4051e40 con 0x7fc4c006c0a0 2023-10-08T16:10:23.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.009+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4c004c3c0 con 0x7fc4c006c0a0 2023-10-08T16:10:23.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.009+0000 7fc4bdffb700 1 -- 172.21.15.170:0/1469175986 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4611 (secure 0 0 0) 0x7fc4b4055460 con 0x7fc4c006c0a0 2023-10-08T16:10:23.010 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:23.010 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:04.228318+0000","modified":"2023-10-08T16:10:07.860819+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6393},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6393":{"gid":6393,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/3892735210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3892735210},{"type":"v1","addr":"172.21.15.191:6834","nonce":3892735210}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 msgr2=0x7fc4ac069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 0x7fc4ac069740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0007240 tx=0x7fc4b0001970 comp rx=0 tx=0).stop 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 msgr2=0x7fc4c014b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400d860 tx=0x7fc4b4008040 comp rx=0 tx=0).stop 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 shutdown_connections 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4c015df50 0x7fc4c00b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc4c006bcd0 0x7fc4c00c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc4ac067290 0x7fc4ac069740 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 --2- 172.21.15.170:0/1469175986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4c006c0a0 0x7fc4c014b1e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 >> 172.21.15.170:0/1469175986 conn(0x7fc4c00ce600 msgr2=0x7fc4c0157370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 shutdown_connections 2023-10-08T16:10:23.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.011+0000 7fc4c7abc700 1 -- 172.21.15.170:0/1469175986 wait complete. 2023-10-08T16:10:23.013 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 57 2023-10-08T16:10:23.024 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:23.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.137+0000 7f6dd5233700 1 Processor -- start 2023-10-08T16:10:23.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.137+0000 7f6dd5233700 1 -- start start 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dd5233700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6dd0159b10 0x7f6dd00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dd5233700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6dd015d730 0x7f6dd015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dd5233700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00c74b0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dd5233700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00c8700 con 0x7f6dd015d730 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dd5233700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00c9070 con 0x7f6dd0159b10 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6dd015d730 0x7f6dd015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6dd015d730 0x7f6dd015dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54182/0 (socket says 172.21.15.170:54182) 2023-10-08T16:10:23.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 -- 172.21.15.170:0/2294019076 learned_addr learned my addr 172.21.15.170:0/2294019076 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dced9d700 1 --2- 172.21.15.170:0/2294019076 >> v2:172.21.15.191:3300/0 conn(0x7f6dd0159b10 0x7f6dd00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 -- 172.21.15.170:0/2294019076 >> v2:172.21.15.191:3300/0 conn(0x7f6dd0159b10 msgr2=0x7f6dd00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 --2- 172.21.15.170:0/2294019076 >> v2:172.21.15.191:3300/0 conn(0x7f6dd0159b10 0x7f6dd00cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 -- 172.21.15.170:0/2294019076 >> v1:172.21.15.191:6789/0 conn(0x7f6dd0159ee0 legacy=0x7f6dd015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.138+0000 7f6dc659c700 1 -- 172.21.15.170:0/2294019076 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd015e0e0 con 0x7f6dd015d730 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dc659c700 1 --2- 172.21.15.170:0/2294019076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd015d730 0x7f6dd015dba0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0063e90 tx=0x7f6db8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ce0c0eabbe3df3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dce59c700 1 -- 172.21.15.170:0/2294019076 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800b8f0 con 0x7f6dd015d730 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dce59c700 1 -- 172.21.15.170:0/2294019076 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db800ba50 con 0x7f6dd015d730 2023-10-08T16:10:23.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dce59c700 1 -- 172.21.15.170:0/2294019076 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800bdf0 con 0x7f6dd015d730 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 -- 172.21.15.170:0/2294019076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd015d730 msgr2=0x7f6dd015dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 --2- 172.21.15.170:0/2294019076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd015d730 0x7f6dd015dba0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0063e90 tx=0x7f6db8009250 comp rx=0 tx=0).stop 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 -- 172.21.15.170:0/2294019076 shutdown_connections 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 --2- 172.21.15.170:0/2294019076 >> v2:172.21.15.191:3300/0 conn(0x7f6dd0159b10 0x7f6dd00cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 --2- 172.21.15.170:0/2294019076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd015d730 0x7f6dd015dba0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 -- 172.21.15.170:0/2294019076 >> 172.21.15.170:0/2294019076 conn(0x7f6dd00c9870 msgr2=0x7f6dd00c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 -- 172.21.15.170:0/2294019076 shutdown_connections 2023-10-08T16:10:23.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.139+0000 7f6dd5233700 1 -- 172.21.15.170:0/2294019076 wait complete. 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.140+0000 7f6dd5233700 1 Processor -- start 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 -- start start 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 0x7f6dd015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 0x7f6dd015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00b0eb0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00b02a0 con 0x7f6dd0159b10 2023-10-08T16:10:23.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dd5233700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00b0580 con 0x7f6dd015d730 2023-10-08T16:10:23.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dc659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dc659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54198/0 (socket says 172.21.15.170:54198) 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dc659c700 1 -- 172.21.15.170:0/3144040419 learned_addr learned my addr 172.21.15.170:0/3144040419 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dced9d700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 0x7f6dd015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.141+0000 7f6dcf59e700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 0x7f6dd015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 -- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 msgr2=0x7f6dd015a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 0x7f6dd015a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 -- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 msgr2=0x7f6dd015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 0x7f6dd015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 -- 172.21.15.170:0/3144040419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80094f0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dced9d700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 0x7f6dd015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dcf59e700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 0x7f6dd015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc659c700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009290 tx=0x7f6db8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8006bc0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db8011e20 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd00b0860 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db801a880 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.142+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd00b1340 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.143+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd00b1560 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.143+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6db801acb0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.144+0000 7f6dc77fe700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 0x7f6db0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.144+0000 7f6dced9d700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 0x7f6db0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.144+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6db80865c0 con 0x7f6dd0159ee0 2023-10-08T16:10:23.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.144+0000 7f6dced9d700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 0x7f6db0069740 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006fd0 tx=0x7f6dc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.148+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6db8053420 con 0x7f6dd0159ee0 2023-10-08T16:10:23.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.333+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6dd0067e50 con 0x7f6dd0159ee0 2023-10-08T16:10:23.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.334+0000 7f6dc77fe700 1 -- 172.21.15.170:0/3144040419 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+15401 (secure 0 0 0) 0x7f6dd0067e50 con 0x7f6dd0159ee0 2023-10-08T16:10:23.335 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:23.335 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":89,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:05.843576+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:02.565366+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:03.191465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 msgr2=0x7f6db0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 0x7f6db0069740 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0006fd0 tx=0x7f6dc0008040 comp rx=0 tx=0).stop 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 msgr2=0x7f6dd00afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009290 tx=0x7f6db8017000 comp rx=0 tx=0).stop 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 shutdown_connections 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6dd015d730 0x7f6dd015a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6dd0159b10 0x7f6dd015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6db0067290 0x7f6db0069740 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 --2- 172.21.15.170:0/3144040419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dd0159ee0 0x7f6dd00afba0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 >> 172.21.15.170:0/3144040419 conn(0x7f6dd00c9870 msgr2=0x7f6dd0156870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 shutdown_connections 2023-10-08T16:10:23.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.336+0000 7f6dd5233700 1 -- 172.21.15.170:0/3144040419 wait complete. 2023-10-08T16:10:23.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:10:23.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.464+0000 7fbd7380e700 1 Processor -- start 2023-10-08T16:10:23.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.464+0000 7fbd7380e700 1 -- start start 2023-10-08T16:10:23.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd7380e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd7380e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd6c069560 0x7fbd6c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd7380e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6c069ee0 con 0x7fbd6c159c60 2023-10-08T16:10:23.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd7380e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c15a160 con 0x7fbd6c069560 2023-10-08T16:10:23.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd7380e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0d13a0 con 0x7fbd6c159890 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd71dab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd6c069560 0x7fbd6c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd71dab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd6c069560 0x7fbd6c0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54204/0 (socket says 172.21.15.170:54204) 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd71dab700 1 -- 172.21.15.170:0/996394591 learned_addr learned my addr 172.21.15.170:0/996394591 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.465+0000 7fbd715aa700 1 --2- 172.21.15.170:0/996394591 >> v2:172.21.15.191:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd71dab700 1 -- 172.21.15.170:0/996394591 >> v2:172.21.15.191:3300/0 conn(0x7fbd6c159890 msgr2=0x7fbd6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd71dab700 1 --2- 172.21.15.170:0/996394591 >> v2:172.21.15.191:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd71dab700 1 -- 172.21.15.170:0/996394591 >> v1:172.21.15.191:6789/0 conn(0x7fbd6c159c60 legacy=0x7fbd6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd63fff700 1 -- 172.21.15.170:0/996394591 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656932767 0 0) 0x7fbd6c069ee0 con 0x7fbd6c159c60 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd71dab700 1 -- 172.21.15.170:0/996394591 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c15e140 con 0x7fbd6c069560 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd71dab700 1 --2- 172.21.15.170:0/996394591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0699a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0d1090 tx=0x7fbd64009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7d59282adce5b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd63fff700 1 -- 172.21.15.170:0/996394591 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd64009e70 con 0x7fbd6c069560 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd63fff700 1 -- 172.21.15.170:0/996394591 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6400b860 con 0x7fbd6c069560 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd63fff700 1 -- 172.21.15.170:0/996394591 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6400bc00 con 0x7fbd6c069560 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 -- 172.21.15.170:0/996394591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 msgr2=0x7fbd6c0699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 --2- 172.21.15.170:0/996394591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0699a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0d1090 tx=0x7fbd64009250 comp rx=0 tx=0).stop 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 -- 172.21.15.170:0/996394591 shutdown_connections 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 --2- 172.21.15.170:0/996394591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0699a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 --2- 172.21.15.170:0/996394591 >> v2:172.21.15.191:3300/0 conn(0x7fbd6c159890 0x7fbd6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.466+0000 7fbd7380e700 1 -- 172.21.15.170:0/996394591 >> 172.21.15.170:0/996394591 conn(0x7fbd6c068e20 msgr2=0x7fbd6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.467+0000 7fbd7380e700 1 -- 172.21.15.170:0/996394591 shutdown_connections 2023-10-08T16:10:23.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.467+0000 7fbd7380e700 1 -- 172.21.15.170:0/996394591 wait complete. 2023-10-08T16:10:23.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 Processor -- start 2023-10-08T16:10:23.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 -- start start 2023-10-08T16:10:23.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd6c159890 0x7fbd6c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 0x7fbd6c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c16dfd0 con 0x7fbd6c069560 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c16d6a0 con 0x7fbd6c159890 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.468+0000 7fbd7380e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c16d980 con 0x7fbd6c159c60 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54214/0 (socket says 172.21.15.170:54214) 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 -- 172.21.15.170:0/3027716012 learned_addr learned my addr 172.21.15.170:0/3027716012 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd71dab700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 0x7fbd6c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd70da9700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd6c159890 0x7fbd6c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 -- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 msgr2=0x7fbd6c0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 0x7fbd6c0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 -- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd6c159890 msgr2=0x7fbd6c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd6c159890 0x7fbd6c0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 -- 172.21.15.170:0/3027716012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640094f0 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd71dab700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 0x7fbd6c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd715aa700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ef0 tx=0x7fbd6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800cc50 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6800f040 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c16dc60 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68014480 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.469+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c0cfc10 con 0x7fbd6c069560 2023-10-08T16:10:23.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.470+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c16e2b0 con 0x7fbd6c069560 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.470+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fbd680147f0 con 0x7fbd6c069560 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.471+0000 7fbd617fa700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 0x7fbd50069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.471+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbd68085630 con 0x7fbd6c069560 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.471+0000 7fbd70da9700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 0x7fbd50069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.471+0000 7fbd70da9700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 0x7fbd50069740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0cd800 tx=0x7fbd5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.475+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd68052380 con 0x7fbd6c069560 2023-10-08T16:10:23.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.701+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd6c0cc600 con 0x7fbd6c069560 2023-10-08T16:10:23.702 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:10:23.702+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:10:23.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.706+0000 7fbd617fa700 1 -- 172.21.15.170:0/3027716012 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd680559a0 con 0x7fbd6c069560 2023-10-08T16:10:23.707 INFO:tasks.ceph.mds.d.smithi191.stderr: -902> 2023-10-08T16:10:10.155+0000 7f22f2fd6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 msgr2=0x7fbd50069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 0x7fbd50069740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0cd800 tx=0x7fbd5c009040 comp rx=0 tx=0).stop 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 msgr2=0x7fbd6c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009ef0 tx=0x7fbd6800b040 comp rx=0 tx=0).stop 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 shutdown_connections 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6c159c60 0x7fbd6c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd50067290 0x7fbd50069740 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd6c069560 0x7fbd6c0b5ad0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 --2- 172.21.15.170:0/3027716012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd6c159890 0x7fbd6c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 >> 172.21.15.170:0/3027716012 conn(0x7fbd6c068e20 msgr2=0x7fbd6c157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 shutdown_connections 2023-10-08T16:10:23.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.708+0000 7fbd7380e700 1 -- 172.21.15.170:0/3027716012 wait complete. 2023-10-08T16:10:23.710 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:10:23.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:10:23.730 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:10:23.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.836+0000 7f8f72f53700 1 Processor -- start 2023-10-08T16:10:23.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.836+0000 7f8f72f53700 1 -- start start 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f72f53700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8f6c159890 0x7f8f6c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f72f53700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f72f53700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c15a160 con 0x7f8f6c15db10 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f72f53700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0d13a0 con 0x7f8f6c159890 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f72f53700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15e140 con 0x7f8f6c159c60 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f714f0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f6c15db10 0x7f8f6c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:53796/0 (socket says 172.21.15.170:53796) 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f714f0700 1 -- 172.21.15.170:0/2376184536 learned_addr learned my addr 172.21.15.170:0/2376184536 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:23.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f70cef700 1 --2- 172.21.15.170:0/2376184536 >> v2:172.21.15.191:3300/0 conn(0x7f8f6c159890 0x7f8f6c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.837+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609919540 0 0) 0x7f8f6c15a160 con 0x7f8f6c15db10 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f54003570 con 0x7f8f6c15db10 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2428583178 0 0) 0x7f8f54003570 con 0x7f8f6c15db10 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 msgr2=0x7f8f6c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 >> v2:172.21.15.191:3300/0 conn(0x7f8f6c159890 msgr2=0x7f8f6c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 --2- 172.21.15.170:0/2376184536 >> v2:172.21.15.191:3300/0 conn(0x7f8f6c159890 0x7f8f6c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c15a3d0 con 0x7f8f6c15db10 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8f64001470 con 0x7f8f6c15db10 2023-10-08T16:10:23.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 >> v1:172.21.15.170:6789/0 conn(0x7f8f6c15db10 legacy=0x7f8f6c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f54003a60 0x7f8f54003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f54004d50 0x7f8f540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0d13a0 con 0x7f8f54003a60 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c15e140 con 0x7f8f54004d50 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5400ba60 con 0x7f8f6c159c60 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f70cef700 1 --2- 172.21.15.170:0/2376184536 >> v2:172.21.15.191:3300/0 conn(0x7f8f6c159890 0x7f8f6c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f714f0700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f54004d50 0x7f8f540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f70cef700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f54003a60 0x7f8f54003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 -- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f54004d50 msgr2=0x7f8f540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f54004d50 0x7f8f540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 -- 172.21.15.170:0/2376184536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f54003a60 msgr2=0x7f8f54003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f54003a60 0x7f8f54003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 -- 172.21.15.170:0/2376184536 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54003570 con 0x7f8f6c159c60 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.838+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f54004810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c15a090 tx=0x7f8f600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e7aa10df00965c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000e6b0 con 0x7f8f6c159c60 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6000e810 con 0x7f8f6c159c60 2023-10-08T16:10:23.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f6b7fe700 1 -- 172.21.15.170:0/2376184536 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6000ebd0 con 0x7f8f6c159c60 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 -- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 msgr2=0x7f8f54004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 --2- 172.21.15.170:0/2376184536 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f54004810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c15a090 tx=0x7f8f600091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f714f0700 1 -- 172.21.15.170:0/2376184536 reap_dead start 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 -- 172.21.15.170:0/2376184536 shutdown_connections 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 -- 172.21.15.170:0/2376184536 >> 172.21.15.170:0/2376184536 conn(0x7f8f6c068e20 msgr2=0x7f8f6c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 -- 172.21.15.170:0/2376184536 shutdown_connections 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.839+0000 7f8f72f53700 1 -- 172.21.15.170:0/2376184536 wait complete. 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.840+0000 7f8f72f53700 1 Processor -- start 2023-10-08T16:10:23.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.840+0000 7f8f72f53700 1 -- start start 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c16a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f6c0cde60 0x7f8f6c148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c16ad20 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0c73b0 con 0x7f8f6c0cde60 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f72f53700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c16d790 con 0x7f8f6c159c60 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54232/0 (socket says 172.21.15.170:54232) 2023-10-08T16:10:23.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 -- 172.21.15.170:0/1925679173 learned_addr learned my addr 172.21.15.170:0/1925679173 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f70cef700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c16a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f6bfff700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f6c0cde60 0x7f8f6c148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 -- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 msgr2=0x7f8f6c16a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c16a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 -- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f6c0cde60 msgr2=0x7f8f6c148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f6c0cde60 0x7f8f6c148e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.841+0000 7f8f714f0700 1 -- 172.21.15.170:0/1925679173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c16da70 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f714f0700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c06a890 tx=0x7f8f64001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64018030 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400f970 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c16dd50 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400fd10 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.842+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c149380 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.843+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c04c3c0 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.843+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8f6401b070 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.844+0000 7f8f68ff9700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 0x7f8f58069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.844+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8f64019070 con 0x7f8f6c0c6a10 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.844+0000 7f8f70cef700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 0x7f8f58069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:23.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.844+0000 7f8f70cef700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 0x7f8f58069740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0094d0 tx=0x7f8f5c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:23.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:23.848+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f64051710 con 0x7f8f6c0c6a10 2023-10-08T16:10:24.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.075+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8f6c06a5b0 con 0x7f8f6c0c6a10 2023-10-08T16:10:24.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.708+0000 7f8f68ff9700 1 -- 172.21.15.170:0/1925679173 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8f64054d30 con 0x7f8f6c0c6a10 2023-10-08T16:10:24.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 msgr2=0x7f8f58069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 0x7f8f58069740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0094d0 tx=0x7f8f5c00b040 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 msgr2=0x7f8f6c0c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c06a890 tx=0x7f8f64001e90 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 shutdown_connections 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f58067290 0x7f8f58069740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f6c0c6a10 0x7f8f6c0c6e70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f6c159c60 0x7f8f6c16a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 --2- 172.21.15.170:0/1925679173 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f6c0cde60 0x7f8f6c148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 >> 172.21.15.170:0/1925679173 conn(0x7f8f6c068e20 msgr2=0x7f8f6c149d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 shutdown_connections 2023-10-08T16:10:24.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.710+0000 7f8f72f53700 1 -- 172.21.15.170:0/1925679173 wait complete. 2023-10-08T16:10:24.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:10:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 Processor -- start 2023-10-08T16:10:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 -- start start 2023-10-08T16:10:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 0x7fec8c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fec8c0b55a0 0x7fec8c0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c0b5b00 con 0x7fec8c159ec0 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c0b5da0 con 0x7fec8c0b55a0 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.842+0000 7fec92abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0b6080 con 0x7fec8c0cd910 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fec8c0b55a0 0x7fec8c0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 0x7fec8c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec90858700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fec8c159ec0 0x7fec8c0b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:39326/0 (socket says 172.21.15.170:39326) 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec90858700 1 -- 172.21.15.170:0/2031940118 learned_addr learned my addr 172.21.15.170:0/2031940118 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fec8c0b55a0 0x7fec8c0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:44950/0 (socket says 172.21.15.170:44950) 2023-10-08T16:10:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 -- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 msgr2=0x7fec8c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 0x7fec8c0ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 -- 172.21.15.170:0/2031940118 >> v1:172.21.15.191:6789/0 conn(0x7fec8c159ec0 legacy=0x7fec8c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec8b7fe700 1 -- 172.21.15.170:0/2031940118 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611327691 0 0) 0x7fec8c0b5b00 con 0x7fec8c159ec0 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 -- 172.21.15.170:0/2031940118 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0cc100 con 0x7fec8c0b55a0 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec8bfff700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 0x7fec8c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.843+0000 7fec91059700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c0b55a0 0x7fec8c0ce880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fec840098e0 tx=0x7fec840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f551dc6dcd79cf92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec8b7fe700 1 -- 172.21.15.170:0/2031940118 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400b640 con 0x7fec8c0b55a0 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec8b7fe700 1 -- 172.21.15.170:0/2031940118 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8400b7a0 con 0x7fec8c0b55a0 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec8b7fe700 1 -- 172.21.15.170:0/2031940118 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400bb40 con 0x7fec8c0b55a0 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 -- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c0b55a0 msgr2=0x7fec8c0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c0b55a0 0x7fec8c0ce880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fec840098e0 tx=0x7fec840091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 -- 172.21.15.170:0/2031940118 shutdown_connections 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c0b55a0 0x7fec8c0ce880 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 --2- 172.21.15.170:0/2031940118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0cd910 0x7fec8c0ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 -- 172.21.15.170:0/2031940118 >> 172.21.15.170:0/2031940118 conn(0x7fec8c0999f0 msgr2=0x7fec8c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 -- 172.21.15.170:0/2031940118 shutdown_connections 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.844+0000 7fec92abc700 1 -- 172.21.15.170:0/2031940118 wait complete. 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.845+0000 7fec92abc700 1 Processor -- start 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 -- start start 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0b55a0 0x7fec8c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c1591a0 0x7fec8c161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c068390 con 0x7fec8c159ec0 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c068650 con 0x7fec8c1591a0 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec92abc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c068930 con 0x7fec8c0b55a0 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54242/0 (socket says 172.21.15.170:54242) 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec8bfff700 1 -- 172.21.15.170:0/1707412326 learned_addr learned my addr 172.21.15.170:0/1707412326 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.846+0000 7fec90858700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0b55a0 0x7fec8c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec91059700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c1591a0 0x7fec8c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 -- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0b55a0 msgr2=0x7fec8c158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0b55a0 0x7fec8c158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 -- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c1591a0 msgr2=0x7fec8c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c1591a0 0x7fec8c161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 -- 172.21.15.170:0/1707412326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74009420 con 0x7fec8c159ec0 2023-10-08T16:10:24.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec8bfff700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fec7400ec80 tx=0x7fec740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400c920 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400ca80 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec84009420 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400ce00 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.847+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c068ef0 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.848+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c063c90 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.848+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fec74013680 con 0x7fec8c159ec0 2023-10-08T16:10:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.849+0000 7fec88ff9700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 0x7fec6c069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:24.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.849+0000 7fec90858700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 0x7fec6c069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:24.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.849+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fec74084e70 con 0x7fec8c159ec0 2023-10-08T16:10:24.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.849+0000 7fec90858700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 0x7fec6c069770 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec7c007260 tx=0x7fec7c001960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:24.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:24.853+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec74051c80 con 0x7fec8c159ec0 2023-10-08T16:10:25.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.038+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec8c0690e0 con 0x7fec8c159ec0 2023-10-08T16:10:25.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.709+0000 7fec88ff9700 1 -- 172.21.15.170:0/1707412326 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7fec740552a0 con 0x7fec8c159ec0 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.712+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 msgr2=0x7fec6c069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.712+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 0x7fec6c069770 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fec7c007260 tx=0x7fec7c001960 comp rx=0 tx=0).stop 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.712+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 msgr2=0x7fec8c160de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.712+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fec7400ec80 tx=0x7fec740091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 shutdown_connections 2023-10-08T16:10:25.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fec8c1591a0 0x7fec8c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fec8c0b55a0 0x7fec8c158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fec6c0672c0 0x7fec6c069770 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 --2- 172.21.15.170:0/1707412326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec8c159ec0 0x7fec8c160de0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 >> 172.21.15.170:0/1707412326 conn(0x7fec8c0999f0 msgr2=0x7fec8c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 shutdown_connections 2023-10-08T16:10:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.713+0000 7fec92abc700 1 -- 172.21.15.170:0/1707412326 wait complete. 2023-10-08T16:10:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:10:25.728 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:10:25.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.847+0000 7fa4ea391700 1 Processor -- start 2023-10-08T16:10:25.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.847+0000 7fa4ea391700 1 -- start start 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4ea391700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa4e4153850 0x7fa4e41596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4ea391700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4e4157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4ea391700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e415a480 con 0x7fa4e4153480 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4ea391700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e415a070 con 0x7fa4e4153850 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4ea391700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40afb40 con 0x7fa4e4159bf0 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4e938f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa4e4153480 0x7fa4e40ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33420/0 (socket says 172.21.15.170:33420) 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4e938f700 1 -- 172.21.15.170:0/1552230200 learned_addr learned my addr 172.21.15.170:0/1552230200 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4e4157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3267095421 0 0) 0x7fa4e415a480 con 0x7fa4e4153480 2023-10-08T16:10:25.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4d0003570 con 0x7fa4e4153480 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.848+0000 7fa4e8b8e700 1 --2- 172.21.15.170:0/1552230200 >> v2:172.21.15.191:3300/0 conn(0x7fa4e4153850 0x7fa4e41596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2433301148 0 0) 0x7fa4d0003570 con 0x7fa4e4153480 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 msgr2=0x7fa4e4157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4e4157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 >> v2:172.21.15.191:3300/0 conn(0x7fa4e4153850 msgr2=0x7fa4e41596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 --2- 172.21.15.170:0/1552230200 >> v2:172.21.15.191:3300/0 conn(0x7fa4e4153850 0x7fa4e41596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40afe20 con 0x7fa4e4153480 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fa4e0001470 con 0x7fa4e4153480 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 >> v1:172.21.15.170:6789/0 conn(0x7fa4e4153480 legacy=0x7fa4e40ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4d0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4d0003fe0 0x7fa4d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4d0004d50 0x7fa4d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e415a070 con 0x7fa4d0004d50 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40afb40 con 0x7fa4d0003fe0 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d000dca0 con 0x7fa4e4159bf0 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4d0004d50 0x7fa4d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e938f700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4d0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e8b8e700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4d0003fe0 0x7fa4d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 msgr2=0x7fa4d0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e4159bf0 0x7fa4d0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4d0003fe0 msgr2=0x7fa4d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4d0003fe0 0x7fa4d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1552230200 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0003570 con 0x7fa4d0004d50 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e8b8e700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4d0003fe0 0x7fa4d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4d0004d50 0x7fa4d00051c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e210 tx=0x7fa4dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7945e7fdd3edce10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00ea60 con 0x7fa4d0004d50 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc00f040 con 0x7fa4d0004d50 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.849+0000 7fa4dbfff700 1 -- 172.21.15.170:0/1552230200 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00ee10 con 0x7fa4d0004d50 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 -- 172.21.15.170:0/1552230200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4d0004d50 msgr2=0x7fa4d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1552230200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4d0004d50 0x7fa4d00051c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc00e210 tx=0x7fa4dc00b040 comp rx=0 tx=0).stop 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1552230200 reap_dead start 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 -- 172.21.15.170:0/1552230200 shutdown_connections 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 -- 172.21.15.170:0/1552230200 >> 172.21.15.170:0/1552230200 conn(0x7fa4e4068e20 msgr2=0x7fa4e40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:25.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 -- 172.21.15.170:0/1552230200 shutdown_connections 2023-10-08T16:10:25.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.850+0000 7fa4ea391700 1 -- 172.21.15.170:0/1552230200 wait complete. 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.851+0000 7fa4ea391700 1 Processor -- start 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.851+0000 7fa4ea391700 1 -- start start 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 0x7fa4e40c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e40cb6d0 0x7fa4e40c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40ab210 con 0x7fa4e40a98d0 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40aa600 con 0x7fa4e40cb6d0 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4ea391700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40aa8e0 con 0x7fa4e40cb300 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38822/0 (socket says 172.21.15.170:38822) 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1794883255 learned_addr learned my addr 172.21.15.170:0/1794883255 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e938f700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 0x7fa4e40c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e8b8e700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e40cb6d0 0x7fa4e40c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 msgr2=0x7fa4e40c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 0x7fa4e40c19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e40cb6d0 msgr2=0x7fa4e40c1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e40cb6d0 0x7fa4e40c1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e9b90700 1 -- 172.21.15.170:0/1794883255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40aabc0 con 0x7fa4e40a98d0 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.852+0000 7fa4e938f700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 0x7fa4e40c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4e9b90700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc016000 tx=0x7fa4dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc00ee60 con 0x7fa4e40a98d0 2023-10-08T16:10:25.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc022070 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e40aaea0 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc027680 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.853+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e416cff0 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.854+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e40aed20 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.854+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fa4dc024030 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.854+0000 7fa4d97fa700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 0x7fa4c8069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.855+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5353+0+0 (secure 0 0 0) 0x7fa4dc0869a0 con 0x7fa4e40a98d0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.855+0000 7fa4e938f700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 0x7fa4c8069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:25.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.855+0000 7fa4e938f700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 0x7fa4c8069740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000e10 tx=0x7fa4e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:25.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:25.859+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4dc053960 con 0x7fa4e40a98d0 2023-10-08T16:10:26.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.040+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4e404c3c0 con 0x7fa4e40a98d0 2023-10-08T16:10:26.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.716+0000 7fa4d97fa700 1 -- 172.21.15.170:0/1794883255 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa4dc056f80 con 0x7fa4e40a98d0 2023-10-08T16:10:26.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.720+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 msgr2=0x7fa4c8069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.720+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 0x7fa4c8069740 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0000e10 tx=0x7fa4e0009040 comp rx=0 tx=0).stop 2023-10-08T16:10:26.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.720+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 msgr2=0x7fa4e40a9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.720+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc016000 tx=0x7fa4dc01a000 comp rx=0 tx=0).stop 2023-10-08T16:10:26.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 shutdown_connections 2023-10-08T16:10:26.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4c8067290 0x7fa4c8069740 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4e40a98d0 0x7fa4e40a9d40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4e40cb300 0x7fa4e40c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 --2- 172.21.15.170:0/1794883255 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4e40cb6d0 0x7fa4e40c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 >> 172.21.15.170:0/1794883255 conn(0x7fa4e4068e20 msgr2=0x7fa4e40694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 shutdown_connections 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.721+0000 7fa4ea391700 1 -- 172.21.15.170:0/1794883255 wait complete. 2023-10-08T16:10:26.723 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:10:26.735 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:10:26.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.851+0000 7f4638e7f700 1 Processor -- start 2023-10-08T16:10:26.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.851+0000 7f4638e7f700 1 -- start start 2023-10-08T16:10:26.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4638e7f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46341575a0 0x7f46340cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4638e7f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4634063580 0x7f463406e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4638e7f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463406ee10 con 0x7f463406e9b0 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4638e7f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340b1100 con 0x7f46341575a0 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4638e7f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c3ae0 con 0x7f4634063580 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46341575a0 0x7f46340cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4633fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f463406e9b0 0x7f46340ac9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33432/0 (socket says 172.21.15.170:33432) 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46341575a0 0x7f46340cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38832/0 (socket says 172.21.15.170:38832) 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4633fff700 1 -- 172.21.15.170:0/2065628101 learned_addr learned my addr 172.21.15.170:0/2065628101 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f462affd700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4634063580 0x7f463406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 -- 172.21.15.170:0/2065628101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4634063580 msgr2=0x7f463406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4634063580 0x7f463406e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 -- 172.21.15.170:0/2065628101 >> v1:172.21.15.170:6789/0 conn(0x7f463406e9b0 legacy=0x7f46340ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f46337fe700 1 -- 172.21.15.170:0/2065628101 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46340cbe60 con 0x7f46341575a0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.852+0000 7f4632ffd700 1 -- 172.21.15.170:0/2065628101 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533190936 0 0) 0x7f462c001490 con 0x7f463406e9b0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f46337fe700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f46340cb610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f46240098e0 tx=0x7f46240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b043879981f2f0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4632ffd700 1 -- 172.21.15.170:0/2065628101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624009e80 con 0x7f46341575a0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4632ffd700 1 -- 172.21.15.170:0/2065628101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462400b860 con 0x7f46341575a0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4632ffd700 1 -- 172.21.15.170:0/2065628101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400bbe0 con 0x7f46341575a0 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 -- 172.21.15.170:0/2065628101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 msgr2=0x7f46340cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f46340cb610 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f46240098e0 tx=0x7f46240091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 -- 172.21.15.170:0/2065628101 shutdown_connections 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f46340cb610 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2065628101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4634063580 0x7f463406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 -- 172.21.15.170:0/2065628101 >> 172.21.15.170:0/2065628101 conn(0x7f46340c9f70 msgr2=0x7f463406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:26.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 -- 172.21.15.170:0/2065628101 shutdown_connections 2023-10-08T16:10:26.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.853+0000 7f4638e7f700 1 -- 172.21.15.170:0/2065628101 wait complete. 2023-10-08T16:10:26.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.854+0000 7f4638e7f700 1 Processor -- start 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 -- start start 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4634063580 0x7f4634157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463406e9b0 0x7f463414cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463414c8a0 con 0x7f46341575a0 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463414bcd0 con 0x7f4634063580 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4638e7f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463414bf70 con 0x7f463406e9b0 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4633fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4633fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38834/0 (socket says 172.21.15.170:38834) 2023-10-08T16:10:26.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.855+0000 7f4633fff700 1 -- 172.21.15.170:0/2700649030 learned_addr learned my addr 172.21.15.170:0/2700649030 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f46337fe700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4634063580 0x7f4634157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f462affd700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463406e9b0 0x7f463414cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 -- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463406e9b0 msgr2=0x7f463414cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463406e9b0 0x7f463414cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 -- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4634063580 msgr2=0x7f4634157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4634063580 0x7f4634157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 -- 172.21.15.170:0/2700649030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624009420 con 0x7f46341575a0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4633fff700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cc90 tx=0x7f462c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00cea0 con 0x7f46341575a0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c004bb0 con 0x7f46341575a0 2023-10-08T16:10:26.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463414c250 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00ab70 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.856+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463414a7f0 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.857+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463406cfa0 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.857+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f462c0070f0 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.858+0000 7f462bfff700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 0x7f4614069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.858+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f462c084350 con 0x7f46341575a0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.858+0000 7f46337fe700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 0x7f4614069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:26.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.858+0000 7f46337fe700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 0x7f4614069760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46240091e0 tx=0x7f4624017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:26.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:26.862+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f462c051610 con 0x7f46341575a0 2023-10-08T16:10:27.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.044+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f463406ee10 con 0x7f46341575a0 2023-10-08T16:10:27.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.048+0000 7f462bfff700 1 -- 172.21.15.170:0/2700649030 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f463406ee10 con 0x7f46341575a0 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 msgr2=0x7f4614069760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 0x7f4614069760 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46240091e0 tx=0x7f4624017000 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 msgr2=0x7f463414d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f462c00cc90 tx=0x7f462c001a50 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 shutdown_connections 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46140672b0 0x7f4614069760 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46341575a0 0x7f463414d270 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4634063580 0x7f4634157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 --2- 172.21.15.170:0/2700649030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f463406e9b0 0x7f463414cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 >> 172.21.15.170:0/2700649030 conn(0x7f46340c9f70 msgr2=0x7f463406dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:27.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 shutdown_connections 2023-10-08T16:10:27.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.050+0000 7f4638e7f700 1 -- 172.21.15.170:0/2700649030 wait complete. 2023-10-08T16:10:27.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup' 2023-10-08T16:10:27.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.177+0000 7fe40e523700 1 Processor -- start 2023-10-08T16:10:27.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.177+0000 7fe40e523700 1 -- start start 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40e523700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 0x7fe40815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40e523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 0x7fe4080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40e523700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe408069ee0 con 0x7fe408159c60 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40e523700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40815a160 con 0x7fe408159890 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40e523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080d13a0 con 0x7fe408069560 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe407fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 0x7fe40815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40cac0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 0x7fe4080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe4077fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fe408159c60 0x7fe40815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36786/0 (socket says 172.21.15.170:36786) 2023-10-08T16:10:27.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe4077fe700 1 -- 172.21.15.170:0/1109301447 learned_addr learned my addr 172.21.15.170:0/1109301447 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe406ffd700 1 -- 172.21.15.170:0/1109301447 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876247115 0 0) 0x7fe408069ee0 con 0x7fe408159c60 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe406ffd700 1 -- 172.21.15.170:0/1109301447 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f0003570 con 0x7fe408159c60 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40cac0700 1 -- 172.21.15.170:0/1109301447 >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 msgr2=0x7fe40815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40cac0700 1 --2- 172.21.15.170:0/1109301447 >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 0x7fe40815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40cac0700 1 -- 172.21.15.170:0/1109301447 >> v1:172.21.15.191:6789/0 conn(0x7fe408159c60 legacy=0x7fe40815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe40cac0700 1 -- 172.21.15.170:0/1109301447 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40815e140 con 0x7fe408069560 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.178+0000 7fe407fff700 1 --2- 172.21.15.170:0/1109301447 >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 0x7fe40815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40cac0700 1 --2- 172.21.15.170:0/1109301447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 0x7fe4080699a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe400009e60 tx=0x7fe400009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eccc66ed8bc50e2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe406ffd700 1 -- 172.21.15.170:0/1109301447 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40000e070 con 0x7fe408069560 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe406ffd700 1 -- 172.21.15.170:0/1109301447 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40000bab0 con 0x7fe408069560 2023-10-08T16:10:27.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe406ffd700 1 -- 172.21.15.170:0/1109301447 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40000be30 con 0x7fe408069560 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 -- 172.21.15.170:0/1109301447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 msgr2=0x7fe4080699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 --2- 172.21.15.170:0/1109301447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 0x7fe4080699a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe400009e60 tx=0x7fe400009780 comp rx=0 tx=0).stop 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 -- 172.21.15.170:0/1109301447 shutdown_connections 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 --2- 172.21.15.170:0/1109301447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408069560 0x7fe4080699a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 --2- 172.21.15.170:0/1109301447 >> v2:172.21.15.191:3300/0 conn(0x7fe408159890 0x7fe40815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 -- 172.21.15.170:0/1109301447 >> 172.21.15.170:0/1109301447 conn(0x7fe408068e20 msgr2=0x7fe4080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 -- 172.21.15.170:0/1109301447 shutdown_connections 2023-10-08T16:10:27.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.179+0000 7fe40e523700 1 -- 172.21.15.170:0/1109301447 wait complete. 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.180+0000 7fe40e523700 1 Processor -- start 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 -- start start 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe408159890 0x7fe4080b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408159c60 0x7fe4080b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe408155920 con 0x7fe408069560 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40815be60 con 0x7fe408159890 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe40e523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4080b6430 con 0x7fe408159c60 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.181+0000 7fe407fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38846/0 (socket says 172.21.15.170:38846) 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 -- 172.21.15.170:0/3780947623 learned_addr learned my addr 172.21.15.170:0/3780947623 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe40cac0700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408159c60 0x7fe4080b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe4077fe700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe408159890 0x7fe4080b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 -- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408159c60 msgr2=0x7fe4080b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408159c60 0x7fe4080b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 -- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe408159890 msgr2=0x7fe4080b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe408159890 0x7fe4080b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 -- 172.21.15.170:0/3780947623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f8009420 con 0x7fe408069560 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe407fff700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800ec80 tx=0x7fe3f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800c920 con 0x7fe408069560 2023-10-08T16:10:27.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f800ca80 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe400009a20 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800ce00 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.182+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40815b810 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.183+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40804c3c0 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.184+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fe3f8013680 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.184+0000 7fe3effff700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 0x7fe3f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.184+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe3f8084b50 con 0x7fe408069560 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.184+0000 7fe4077fe700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 0x7fe3f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.184+0000 7fe4077fe700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 0x7fe3f4069740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe40815d7d0 tx=0x7fe3fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.188+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3f8051cb0 con 0x7fe408069560 2023-10-08T16:10:27.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.446+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]} v 0) v1 -- 0x7fe40815bbb0 con 0x7fe408069560 2023-10-08T16:10:27.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.715+0000 7fe3effff700 1 -- 172.21.15.170:0/3780947623 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_with_human_readable_flag_without_subvolumegroup"]}]=0 v279) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe3f80552d0 con 0x7fe408069560 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 msgr2=0x7fe3f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 0x7fe3f4069740 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe40815d7d0 tx=0x7fe3fc009040 comp rx=0 tx=0).stop 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 msgr2=0x7fe4081553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe3f800ec80 tx=0x7fe3f80091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 shutdown_connections 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe408159c60 0x7fe4080b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe3f4067290 0x7fe3f4069740 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe408069560 0x7fe4081553e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 --2- 172.21.15.170:0/3780947623 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe408159890 0x7fe4080b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 >> 172.21.15.170:0/3780947623 conn(0x7fe408068e20 msgr2=0x7fe4080b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 shutdown_connections 2023-10-08T16:10:27.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.717+0000 7fe40e523700 1 -- 172.21.15.170:0/3780947623 wait complete. 2023-10-08T16:10:27.731 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info --human_readable' command without subvolume group ... ok 2023-10-08T16:10:27.731 INFO:tasks.cephfs_test_runner:Starting test: test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) Tests the 'fs volume info' command without subvolume group 2023-10-08T16:10:27.731 INFO:tasks.cephfs_test_runner:test_volume_info_without_subvolumegroup (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:10:27.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-08T16:10:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 Processor -- start 2023-10-08T16:10:27.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 -- start start 2023-10-08T16:10:27.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f3330140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f333006c0a0 0x7f3330141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f333015c2a0 con 0x7f33300c8e00 2023-10-08T16:10:27.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f333015b690 con 0x7f333006c0a0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.845+0000 7f3336fbe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333015b970 con 0x7f333006bcd0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f333006c0a0 0x7f3330141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f333006c0a0 0x7f3330141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38856/0 (socket says 172.21.15.170:38856) 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 -- 172.21.15.170:0/3136709720 learned_addr learned my addr 172.21.15.170:0/3136709720 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3334d5a700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f3330140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 -- 172.21.15.170:0/3136709720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 msgr2=0x7f3330140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f3330140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 -- 172.21.15.170:0/3136709720 >> v1:172.21.15.191:6789/0 conn(0x7f33300c8e00 legacy=0x7f33300c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 -- 172.21.15.170:0/3136709720 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333015bc50 con 0x7f333006c0a0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f3327fff700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f3330141330 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f33200099b0 tx=0x7f3320009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b78bce4b99cb7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f33277fe700 1 -- 172.21.15.170:0/3136709720 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000b820 con 0x7f333006c0a0 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f33277fe700 1 -- 172.21.15.170:0/3136709720 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332000b980 con 0x7f333006c0a0 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.846+0000 7f33277fe700 1 -- 172.21.15.170:0/3136709720 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332000bd20 con 0x7f333006c0a0 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 -- 172.21.15.170:0/3136709720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 msgr2=0x7f3330141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f3330141330 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f33200099b0 tx=0x7f3320009250 comp rx=0 tx=0).stop 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 -- 172.21.15.170:0/3136709720 shutdown_connections 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f3330140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 --2- 172.21.15.170:0/3136709720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f3330141330 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 -- 172.21.15.170:0/3136709720 >> 172.21.15.170:0/3136709720 conn(0x7f33300ce600 msgr2=0x7f33300cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 -- 172.21.15.170:0/3136709720 shutdown_connections 2023-10-08T16:10:27.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.847+0000 7f3336fbe700 1 -- 172.21.15.170:0/3136709720 wait complete. 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.848+0000 7f3336fbe700 1 Processor -- start 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.848+0000 7f3336fbe700 1 -- start start 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f33300c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33300c8e00 0x7f333015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300b03b0 con 0x7f333006c0a0 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c72e0 con 0x7f33300c8e00 2023-10-08T16:10:27.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3336fbe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33300c75c0 con 0x7f333006bcd0 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38868/0 (socket says 172.21.15.170:38868) 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 -- 172.21.15.170:0/122061334 learned_addr learned my addr 172.21.15.170:0/122061334 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 -- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 msgr2=0x7f33300c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f33300c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 -- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33300c8e00 msgr2=0x7f333015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33300c8e00 0x7f333015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 -- 172.21.15.170:0/122061334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200094f0 con 0x7f333006c0a0 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.849+0000 7f3327fff700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3320009290 tx=0x7f3320016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011d70 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320020070 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33300c78a0 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332001b6a0 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33300c7e60 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.850+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33300c8050 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.851+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f3320022030 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.851+0000 7f3324ff9700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 0x7f331c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.851+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3320086770 con 0x7f333006c0a0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.851+0000 7f3334d5a700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 0x7f331c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:27.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.852+0000 7f3334d5a700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 0x7f331c069740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f332c007eb0 tx=0x7f332c0073a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:27.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:27.856+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33200538d0 con 0x7f333006c0a0 2023-10-08T16:10:28.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.115+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f33300c8050 con 0x7f333006c0a0 2023-10-08T16:10:28.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.717+0000 7f3324ff9700 1 -- 172.21.15.170:0/122061334 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v280) v1 ==== 159+0+0 (secure 0 0 0) 0x7f3320056ef0 con 0x7f333006c0a0 2023-10-08T16:10:28.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 msgr2=0x7f331c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 0x7f331c069740 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f332c007eb0 tx=0x7f332c0073a0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 msgr2=0x7f33300afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f3320009290 tx=0x7f3320016000 comp rx=0 tx=0).stop 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 shutdown_connections 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33300c8e00 0x7f333015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f333006bcd0 0x7f33300c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f331c067290 0x7f331c069740 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 --2- 172.21.15.170:0/122061334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333006c0a0 0x7f33300afe70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 >> 172.21.15.170:0/122061334 conn(0x7f33300ce600 msgr2=0x7f333013fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 shutdown_connections 2023-10-08T16:10:28.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.720+0000 7f3336fbe700 1 -- 172.21.15.170:0/122061334 wait complete. 2023-10-08T16:10:28.734 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:10:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.851+0000 7f1ce520b700 1 Processor -- start 2023-10-08T16:10:28.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.851+0000 7f1ce520b700 1 -- start start 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1ce520b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 0x7f1ce015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1ce520b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 0x7f1ce0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1ce520b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce015a160 con 0x7f1ce0159890 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1ce520b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce015df50 con 0x7f1ce0159c60 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1ce520b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00d13a0 con 0x7f1ce015db10 2023-10-08T16:10:28.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1cd659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 0x7f1ce015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1cdf59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 0x7f1ce0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1cded9d700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f1ce0159890 0x7f1ce015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36812/0 (socket says 172.21.15.170:36812) 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.852+0000 7f1cded9d700 1 -- 172.21.15.170:0/2160770427 learned_addr learned my addr 172.21.15.170:0/2160770427 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2160770427 >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 msgr2=0x7f1ce015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cdf59e700 1 --2- 172.21.15.170:0/2160770427 >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 0x7f1ce015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2160770427 >> v1:172.21.15.191:6789/0 conn(0x7f1ce0159890 legacy=0x7f1ce015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2160770427 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce015e140 con 0x7f1ce015db10 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cd659c700 1 --2- 172.21.15.170:0/2160770427 >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 0x7f1ce015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cdf59e700 1 --2- 172.21.15.170:0/2160770427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 0x7f1ce0069560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009e60 tx=0x7f1cd8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1822aeb9396fc888 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cde59c700 1 -- 172.21.15.170:0/2160770427 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd800e070 con 0x7f1ce015db10 2023-10-08T16:10:28.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cde59c700 1 -- 172.21.15.170:0/2160770427 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd800bb60 con 0x7f1ce015db10 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.853+0000 7f1cde59c700 1 -- 172.21.15.170:0/2160770427 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8012430 con 0x7f1ce015db10 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 -- 172.21.15.170:0/2160770427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 msgr2=0x7f1ce0069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2160770427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 0x7f1ce0069560 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1cd8009e60 tx=0x7f1cd8009780 comp rx=0 tx=0).stop 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 -- 172.21.15.170:0/2160770427 shutdown_connections 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2160770427 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce015db10 0x7f1ce0069560 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2160770427 >> v2:172.21.15.191:3300/0 conn(0x7f1ce0159c60 0x7f1ce015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 -- 172.21.15.170:0/2160770427 >> 172.21.15.170:0/2160770427 conn(0x7f1ce0068e20 msgr2=0x7f1ce00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 -- 172.21.15.170:0/2160770427 shutdown_connections 2023-10-08T16:10:28.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.854+0000 7f1ce520b700 1 -- 172.21.15.170:0/2160770427 wait complete. 2023-10-08T16:10:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.855+0000 7f1ce520b700 1 Processor -- start 2023-10-08T16:10:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.855+0000 7f1ce520b700 1 -- start start 2023-10-08T16:10:28.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 0x7f1ce00cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 0x7f1ce014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00ce2b0 con 0x7f1ce015db10 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00ce590 con 0x7f1ce0159890 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce00ce870 con 0x7f1ce0159c60 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38882/0 (socket says 172.21.15.170:38882) 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2516043131 learned_addr learned my addr 172.21.15.170:0/2516043131 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cded9d700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 0x7f1ce00cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cd659c700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 0x7f1ce014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 msgr2=0x7f1ce014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 0x7f1ce014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 msgr2=0x7f1ce00cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 0x7f1ce00cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 -- 172.21.15.170:0/2516043131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc8009420 con 0x7f1ce015db10 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cded9d700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 0x7f1ce00cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cd659c700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 0x7f1ce014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cdf59e700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0070e20 tx=0x7f1cd8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8027470 con 0x7f1ce015db10 2023-10-08T16:10:28.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd8027610 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd8009a20 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80279d0 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.856+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce00cdfc0 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.857+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce004c3c0 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.858+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f1cd8024070 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.858+0000 7f1cd77fe700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 0x7f1cc0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.858+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1cd8087270 con 0x7f1ce015db10 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.858+0000 7f1cded9d700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 0x7f1cc0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:28.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.859+0000 7f1cded9d700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 0x7f1cc0069740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007fd0 tx=0x7f1cd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:28.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:28.863+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cd80543d0 con 0x7f1ce015db10 2023-10-08T16:10:29.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.045+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ce00caba0 con 0x7f1ce015db10 2023-10-08T16:10:29.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.049+0000 7f1cd77fe700 1 -- 172.21.15.170:0/2516043131 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1cd80579f0 con 0x7f1ce015db10 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 msgr2=0x7f1cc0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 0x7f1cc0069740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007fd0 tx=0x7f1cd0009040 comp rx=0 tx=0).stop 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 msgr2=0x7f1ce00b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0070e20 tx=0x7f1cd8019000 comp rx=0 tx=0).stop 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 shutdown_connections 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1cc0067290 0x7f1cc0069740 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ce015db10 0x7f1ce00b3c50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ce0159890 0x7f1ce00cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 --2- 172.21.15.170:0/2516043131 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ce0159c60 0x7f1ce014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 >> 172.21.15.170:0/2516043131 conn(0x7f1ce0068e20 msgr2=0x7f1ce0152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 shutdown_connections 2023-10-08T16:10:29.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.051+0000 7f1ce520b700 1 -- 172.21.15.170:0/2516043131 wait complete. 2023-10-08T16:10:29.065 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:10:29.065 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:29.103 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:29.148 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:10:29.148 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:10:29.148 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:29.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.260+0000 7f25b7fad700 1 Processor -- start 2023-10-08T16:10:29.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.260+0000 7f25b7fad700 1 -- start start 2023-10-08T16:10:29.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b7fad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b7fad700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25b00ccf90 0x7f25b00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b7fad700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00cc300 con 0x7f25b00cbf30 2023-10-08T16:10:29.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b7fad700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b0156bf0 con 0x7f25b00ccf90 2023-10-08T16:10:29.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b7fad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ae950 con 0x7f25b00b1420 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b654a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25b00ccf90 0x7f25b00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b654a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25b00ccf90 0x7f25b00c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38896/0 (socket says 172.21.15.170:38896) 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.261+0000 7f25b654a700 1 -- 172.21.15.170:0/1109979745 learned_addr learned my addr 172.21.15.170:0/1109979745 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b5548700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b654a700 1 -- 172.21.15.170:0/1109979745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 msgr2=0x7f25b00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b654a700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b654a700 1 -- 172.21.15.170:0/1109979745 >> v1:172.21.15.191:6789/0 conn(0x7f25b00cbf30 legacy=0x7f25b00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b654a700 1 -- 172.21.15.170:0/1109979745 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00ad370 con 0x7f25b00ccf90 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b654a700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b00c9740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d1d0 tx=0x7f25a800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e88bbb30b93dfd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b4d47700 1 -- 172.21.15.170:0/1109979745 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8010070 con 0x7f25b00ccf90 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b4d47700 1 -- 172.21.15.170:0/1109979745 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a800b0d0 con 0x7f25b00ccf90 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b4d47700 1 -- 172.21.15.170:0/1109979745 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800b490 con 0x7f25b00ccf90 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b7fad700 1 -- 172.21.15.170:0/1109979745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 msgr2=0x7f25b00c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.262+0000 7f25b7fad700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b00c9740 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f25a800d1d0 tx=0x7f25a800ba50 comp rx=0 tx=0).stop 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 -- 172.21.15.170:0/1109979745 shutdown_connections 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b00c9740 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 --2- 172.21.15.170:0/1109979745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 -- 172.21.15.170:0/1109979745 >> 172.21.15.170:0/1109979745 conn(0x7f25b006a320 msgr2=0x7f25b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 -- 172.21.15.170:0/1109979745 shutdown_connections 2023-10-08T16:10:29.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.263+0000 7f25b7fad700 1 -- 172.21.15.170:0/1109979745 wait complete. 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 Processor -- start 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 -- start start 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25b00cbf30 0x7f25b00c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00c75f0 con 0x7f25b00ccf90 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00c6d00 con 0x7f25b00cbf30 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.264+0000 7f25b7fad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00c6fa0 con 0x7f25b00b1420 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38900/0 (socket says 172.21.15.170:38900) 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 -- 172.21.15.170:0/584569342 learned_addr learned my addr 172.21.15.170:0/584569342 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b5d49700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b5548700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25b00cbf30 0x7f25b00c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 -- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 msgr2=0x7f25b00c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 -- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25b00cbf30 msgr2=0x7f25b00c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25b00cbf30 0x7f25b00c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 -- 172.21.15.170:0/584569342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a800bcf0 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b654a700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f25b0064290 tx=0x7f25a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a800deb0 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8007170 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b00c7280 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8027690 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.265+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00c55c0 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.266+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b004c3c0 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.267+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f25a8003040 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.267+0000 7f25a5ffb700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 0x7f2594069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.267+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f25a80886b0 con 0x7f25b00ccf90 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.267+0000 7f25b5d49700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 0x7f2594069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.271 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.267+0000 7f25b5d49700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 0x7f2594069740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f25ac003a80 tx=0x7f25ac007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.271+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25a8055700 con 0x7f25b00ccf90 2023-10-08T16:10:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.495+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25b00c7730 con 0x7f25b00ccf90 2023-10-08T16:10:29.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.496+0000 7f25a5ffb700 1 -- 172.21.15.170:0/584569342 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f25a8022df0 con 0x7f25b00ccf90 2023-10-08T16:10:29.497 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:29.497 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5723,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/841612162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":841612162},{"type":"v1","addr":"172.21.15.191:6834","nonce":841612162}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":5965,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4199637827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4199637827},{"type":"v1","addr":"172.21.15.191:6835","nonce":4199637827}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/4207268432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4207268432},{"type":"v1","addr":"172.21.15.170:6836","nonce":4207268432}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":6381,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1102809576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1102809576},{"type":"v1","addr":"172.21.15.170:6837","nonce":1102809576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 msgr2=0x7f2594069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 0x7f2594069740 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f25ac003a80 tx=0x7f25ac007ec0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 msgr2=0x7f25b0166370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f25b0064290 tx=0x7f25a801a000 comp rx=0 tx=0).stop 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 shutdown_connections 2023-10-08T16:10:29.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2594067290 0x7f2594069740 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25b00ccf90 0x7f25b0166370 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25b00b1420 0x7f25b00c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 --2- 172.21.15.170:0/584569342 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25b00cbf30 0x7f25b00c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 >> 172.21.15.170:0/584569342 conn(0x7f25b006a320 msgr2=0x7f25b00cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 shutdown_connections 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.498+0000 7f25b7fad700 1 -- 172.21.15.170:0/584569342 wait complete. 2023-10-08T16:10:29.501 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 59 2023-10-08T16:10:29.512 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:10:29.512 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:10:29.512 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:10:29.512 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:29.513 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:10:29.513 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:10:29.513 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:10:29.513 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:29.513 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:10:29.513 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:10:29.514 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:10:29.514 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:29.514 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:10:29.514 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:10:29.514 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:10:29.514 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:29.520 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:10:29.521 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:10:29.521 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:10:29.522 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:10:29.525 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:10:29.525 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:10:29.526 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:10:29.526 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:10:29.563 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:10:29.564 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:29.564 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:10:29.564 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:29.567 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:10:29.567 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:29.568 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:10:29.568 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:29.568 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:10:29.637 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:10:29.637 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:10:29.638 INFO:tasks.ceph.mds.a.smithi170.stderr:2023-10-08T16:10:29.636+0000 7f6234af0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:10:29.649 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:10:29.649 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:10:29.649 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:10:29.648+0000 7f33f0238700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T16:10:29.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.688+0000 7f7fc96d0700 1 Processor -- start 2023-10-08T16:10:29.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.688+0000 7f7fc96d0700 1 -- start start 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc96d0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7fc4069560 0x7f7fc40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc96d0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc4069ee0 con 0x7f7fc4159890 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc96d0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc415a160 con 0x7f7fc4159c60 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc96d0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc40d13a0 con 0x7f7fc4069560 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7fc4159890 0x7f7fc415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33454/0 (socket says 172.21.15.170:33454) 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc2ffd700 1 -- 172.21.15.170:0/1439935272 learned_addr learned my addr 172.21.15.170:0/1439935272 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:29.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc37fe700 1 --2- 172.21.15.170:0/1439935272 >> v2:172.21.15.191:3300/0 conn(0x7f7fc4069560 0x7f7fc40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59592279 0 0) 0x7f7fc4069ee0 con 0x7f7fc4159890 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fac003570 con 0x7f7fc4159890 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4284007421 0 0) 0x7f7fac003570 con 0x7f7fc4159890 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc37fe700 1 -- 172.21.15.170:0/1439935272 >> v1:172.21.15.191:6789/0 conn(0x7f7fc4159c60 legacy=0x7f7fc415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc37fe700 1 -- 172.21.15.170:0/1439935272 >> v1:172.21.15.170:6789/0 conn(0x7f7fc4159890 legacy=0x7f7fc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc37fe700 1 -- 172.21.15.170:0/1439935272 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc415e140 con 0x7f7fc4069560 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.689+0000 7f7fc37fe700 1 --2- 172.21.15.170:0/1439935272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4069560 0x7f7fc40699a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009e60 tx=0x7f7fbc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39db536cf517a327 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00e070 con 0x7f7fc4069560 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00b850 con 0x7f7fc4069560 2023-10-08T16:10:29.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc1ffb700 1 -- 172.21.15.170:0/1439935272 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4001470 con 0x7f7fc4069560 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1439935272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4069560 msgr2=0x7f7fc40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1439935272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4069560 0x7f7fc40699a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009e60 tx=0x7f7fbc009780 comp rx=0 tx=0).stop 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1439935272 shutdown_connections 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1439935272 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4069560 0x7f7fc40699a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1439935272 >> 172.21.15.170:0/1439935272 conn(0x7f7fc4068e20 msgr2=0x7f7fc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1439935272 shutdown_connections 2023-10-08T16:10:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.690+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1439935272 wait complete. 2023-10-08T16:10:29.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.691+0000 7f7fc96d0700 1 Processor -- start 2023-10-08T16:10:29.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.691+0000 7f7fc96d0700 1 -- start start 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4159890 0x7f7fc40cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fc4159c60 0x7f7fc416d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40b1f20 con 0x7f7fc4069560 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc41487b0 con 0x7f7fc4159890 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc96d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40b1680 con 0x7f7fc4159c60 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38928/0 (socket says 172.21.15.170:38928) 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 -- 172.21.15.170:0/1984858800 learned_addr learned my addr 172.21.15.170:0/1984858800 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:29.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc37fe700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fc4159c60 0x7f7fc416d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc27fc700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4159890 0x7f7fc40cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 -- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fc4159c60 msgr2=0x7f7fc416d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fc4159c60 0x7f7fc416d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 -- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4159890 msgr2=0x7f7fc40cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.692+0000 7f7fc2ffd700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4159890 0x7f7fc40cf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7fc2ffd700 1 -- 172.21.15.170:0/1984858800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc009a20 con 0x7f7fc4069560 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7fc2ffd700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400cb90 tx=0x7f7fb4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4007e90 con 0x7f7fc4069560 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb4007360 con 0x7f7fc4069560 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc40b1960 con 0x7f7fc4069560 2023-10-08T16:10:29.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4004830 con 0x7f7fc4069560 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.693+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc40c8790 con 0x7f7fc4069560 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.694+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc40c85f0 con 0x7f7fc4069560 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.694+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7fb4004be0 con 0x7f7fc4069560 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.695+0000 7f7faaffd700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 0x7f7fb0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.695+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7fb4083c10 con 0x7f7fc4069560 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.695+0000 7f7fc27fc700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 0x7f7fb0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.695+0000 7f7fc27fc700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 0x7f7fb0069740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4153ce0 tx=0x7f7fb8007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:29.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.699+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fc40c85f0 con 0x7f7fc4069560 2023-10-08T16:10:29.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:29.903+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7fc40c85f0 con 0x7f7fc4069560 2023-10-08T16:10:30.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.737+0000 7f7faaffd700 1 -- 172.21.15.170:0/1984858800 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7fc40c85f0 con 0x7f7fc4069560 2023-10-08T16:10:30.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 msgr2=0x7f7fb0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 0x7f7fb0069740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4153ce0 tx=0x7f7fb8007ec0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 msgr2=0x7f7fc4148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400cb90 tx=0x7f7fb4001b50 comp rx=0 tx=0).stop 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 shutdown_connections 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7fc4159c60 0x7f7fc416d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fb0067290 0x7f7fb0069740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4069560 0x7f7fc4148270 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 --2- 172.21.15.170:0/1984858800 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7fc4159890 0x7f7fc40cf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.739+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 >> 172.21.15.170:0/1984858800 conn(0x7f7fc4068e20 msgr2=0x7f7fc4155570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.740+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 shutdown_connections 2023-10-08T16:10:30.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.740+0000 7f7fc96d0700 1 -- 172.21.15.170:0/1984858800 wait complete. 2023-10-08T16:10:30.743 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:10:30.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:10:30.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.870+0000 7fc3bd7c0700 1 Processor -- start 2023-10-08T16:10:30.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.870+0000 7fc3bd7c0700 1 -- start start 2023-10-08T16:10:30.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3bd7c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3b80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3bd7c0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b815d7e0 con 0x7fc3b806c0a0 2023-10-08T16:10:30.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3bd7c0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b815dac0 con 0x7fc3b815df50 2023-10-08T16:10:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3bd7c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b815aa30 con 0x7fc3b806bcd0 2023-10-08T16:10:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3b67fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc3b806c0a0 0x7fc3b815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33460/0 (socket says 172.21.15.170:33460) 2023-10-08T16:10:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.871+0000 7fc3b67fc700 1 -- 172.21.15.170:0/995663931 learned_addr learned my addr 172.21.15.170:0/995663931 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3b80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56683859 0 0) 0x7fc3b815d7e0 con 0x7fc3b806c0a0 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a4003570 con 0x7fc3b806c0a0 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877924347 0 0) 0x7fc3b815dac0 con 0x7fc3b815df50 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b815d7e0 con 0x7fc3b815df50 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2804161070 0 0) 0x7fc3a4003570 con 0x7fc3b806c0a0 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 msgr2=0x7fc3b80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:30.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3b80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 >> v1:172.21.15.191:6789/0 conn(0x7fc3b815df50 legacy=0x7fc3b80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b815ad10 con 0x7fc3b806c0a0 2023-10-08T16:10:30.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fc3a0001470 con 0x7fc3b806c0a0 2023-10-08T16:10:30.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3b80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:30.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 >> v1:172.21.15.170:6789/0 conn(0x7fc3b806c0a0 legacy=0x7fc3b815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3a4003c20 0x7fc3a4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4004d50 0x7fc3a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b815aa30 con 0x7fc3a4003c20 2023-10-08T16:10:30.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a400bbf0 con 0x7fc3a4004d50 2023-10-08T16:10:30.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a400bd60 con 0x7fc3b806bcd0 2023-10-08T16:10:30.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3a4003c20 0x7fc3a4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b67fc700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b77fe700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4004d50 0x7fc3a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 -- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 msgr2=0x7fc3a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:30.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 -- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4004d50 msgr2=0x7fc3a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:30.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4004d50 0x7fc3a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b6ffd700 1 -- 172.21.15.170:0/995663931 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a8009a20 con 0x7fc3a4003c20 2023-10-08T16:10:30.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b77fe700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4004d50 0x7fc3a40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:30.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.872+0000 7fc3b67fc700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b806bcd0 0x7fc3a4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:30.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3a4003c20 0x7fc3a4004010 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e3c0 tx=0x7fc3a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8ab3ddc6230576 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:30.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8009c50 con 0x7fc3a4003c20 2023-10-08T16:10:30.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a800f040 con 0x7fc3a4003c20 2023-10-08T16:10:30.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3b5ffb700 1 -- 172.21.15.170:0/995663931 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800eea0 con 0x7fc3a4003c20 2023-10-08T16:10:30.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/995663931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3a4003c20 msgr2=0x7fc3a4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/995663931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3a4003c20 0x7fc3a4004010 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e3c0 tx=0x7fc3a800b040 comp rx=0 tx=0).stop 2023-10-08T16:10:30.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3b77fe700 1 -- 172.21.15.170:0/995663931 reap_dead start 2023-10-08T16:10:30.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/995663931 shutdown_connections 2023-10-08T16:10:30.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/995663931 >> 172.21.15.170:0/995663931 conn(0x7fc3b80ce600 msgr2=0x7fc3b80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:30.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/995663931 shutdown_connections 2023-10-08T16:10:30.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.873+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/995663931 wait complete. 2023-10-08T16:10:30.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.874+0000 7fc3bd7c0700 1 Processor -- start 2023-10-08T16:10:30.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 -- start start 2023-10-08T16:10:30.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4003c20 0x7fc3b80b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b8155420 0x7fc3b80c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80c4d20 con 0x7fc3b80c48b0 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b806aea0 con 0x7fc3a4003c20 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3bd7c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80b3ad0 con 0x7fc3b8155420 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38946/0 (socket says 172.21.15.170:38946) 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3b77fe700 1 -- 172.21.15.170:0/1772050068 learned_addr learned my addr 172.21.15.170:0/1772050068 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3b67fc700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b8155420 0x7fc3b80c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.875+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4003c20 0x7fc3b80b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 -- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b8155420 msgr2=0x7fc3b80c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b8155420 0x7fc3b80c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 -- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4003c20 msgr2=0x7fc3b80b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4003c20 0x7fc3b80b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 -- 172.21.15.170:0/1772050068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b816ab90 con 0x7fc3b80c48b0 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3b77fe700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00010e0 tx=0x7fc3b0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000ee40 con 0x7fc3b80c48b0 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b0013420 con 0x7fc3b80c48b0 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b816a8b0 con 0x7fc3b80c48b0 2023-10-08T16:10:30.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b00137e0 con 0x7fc3b80c48b0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.876+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b80b3760 con 0x7fc3b80c48b0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.877+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b80b32e0 con 0x7fc3b80c48b0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.877+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fc3b0013b50 con 0x7fc3b80c48b0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.878+0000 7fc3aeffd700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 0x7fc398069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.878+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc3b00843d0 con 0x7fc3b80c48b0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.878+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 0x7fc398069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.878+0000 7fc3b6ffd700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 0x7fc398069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80099f0 tx=0x7fc3a801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:30.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:30.882+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3b80b32e0 con 0x7fc3b80c48b0 2023-10-08T16:10:31.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.137+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc3b80b32e0 con 0x7fc3b80c48b0 2023-10-08T16:10:31.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.138+0000 7fc3aeffd700 1 -- 172.21.15.170:0/1772050068 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc3b80b32e0 con 0x7fc3b80c48b0 2023-10-08T16:10:31.139 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:10:31.140 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.141 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.142 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.143 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:31.144 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:10:31.145 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:10:31.146 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:10:31.147 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.148 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.149 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.150 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.152 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:10:31.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:31.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:10:31.156 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.157 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:10:31.158 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:31.159 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.160 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:10:31.161 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:10:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:10:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:31.162 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:10:31.162 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 msgr2=0x7fc398069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 0x7fc398069740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80099f0 tx=0x7fc3a801a000 comp rx=0 tx=0).stop 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 msgr2=0x7fc3b8169c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00010e0 tx=0x7fc3b0009040 comp rx=0 tx=0).stop 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 shutdown_connections 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc398067290 0x7fc398069740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3b80c48b0 0x7fc3b8169c30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3a4003c20 0x7fc3b80b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 --2- 172.21.15.170:0/1772050068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3b8155420 0x7fc3b80c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 >> 172.21.15.170:0/1772050068 conn(0x7fc3b80ce600 msgr2=0x7fc3b80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 shutdown_connections 2023-10-08T16:10:31.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.140+0000 7fc3bd7c0700 1 -- 172.21.15.170:0/1772050068 wait complete. 2023-10-08T16:10:31.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:31.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.280+0000 7f40b1ac8700 1 Processor -- start 2023-10-08T16:10:31.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.280+0000 7f40b1ac8700 1 -- start start 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40b1ac8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f40ac159890 0x7f40ac15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40b1ac8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac15a160 con 0x7f40ac069560 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40b1ac8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac0d13a0 con 0x7f40ac159c60 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40b1ac8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac15e140 con 0x7f40ac159890 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f40ac159890 0x7f40ac15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40abfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f40ac069560 0x7f40ac0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33462/0 (socket says 172.21.15.170:33462) 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40abfff700 1 -- 172.21.15.170:0/1654911691 learned_addr learned my addr 172.21.15.170:0/1654911691 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 --2- 172.21.15.170:0/1654911691 >> v2:172.21.15.170:3300/0 conn(0x7f40ac159890 0x7f40ac15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38960/0 (socket says 172.21.15.170:38960) 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 -- 172.21.15.170:0/1654911691 >> v1:172.21.15.191:6789/0 conn(0x7f40ac159c60 legacy=0x7f40ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:31.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 -- 172.21.15.170:0/1654911691 >> v1:172.21.15.170:6789/0 conn(0x7f40ac069560 legacy=0x7f40ac0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 -- 172.21.15.170:0/1654911691 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac15a3d0 con 0x7f40ac159890 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40aa7fc700 1 -- 172.21.15.170:0/1654911691 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371366950 0 0) 0x7f40ac15a160 con 0x7f40ac069560 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40ab7fe700 1 --2- 172.21.15.170:0/1654911691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac159890 0x7f40ac15abd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f409c009e60 tx=0x7f409c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e17ef987286ac1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40aa7fc700 1 -- 172.21.15.170:0/1654911691 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00e070 con 0x7f40ac159890 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40aa7fc700 1 -- 172.21.15.170:0/1654911691 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c00bbf0 con 0x7f40ac159890 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.281+0000 7f40aa7fc700 1 -- 172.21.15.170:0/1654911691 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c012430 con 0x7f40ac159890 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 -- 172.21.15.170:0/1654911691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac159890 msgr2=0x7f40ac15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/1654911691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac159890 0x7f40ac15abd0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f409c009e60 tx=0x7f409c009780 comp rx=0 tx=0).stop 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 -- 172.21.15.170:0/1654911691 shutdown_connections 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/1654911691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac159890 0x7f40ac15abd0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 -- 172.21.15.170:0/1654911691 >> 172.21.15.170:0/1654911691 conn(0x7f40ac068e20 msgr2=0x7f40ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 -- 172.21.15.170:0/1654911691 shutdown_connections 2023-10-08T16:10:31.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.282+0000 7f40b1ac8700 1 -- 172.21.15.170:0/1654911691 wait complete. 2023-10-08T16:10:31.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.283+0000 7f40b1ac8700 1 Processor -- start 2023-10-08T16:10:31.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.283+0000 7f40b1ac8700 1 -- start start 2023-10-08T16:10:31.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40ac159890 0x7f40ac151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40ac159c60 0x7f40ac1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac15cac0 con 0x7f40ac069560 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac151e00 con 0x7f40ac159890 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40b1ac8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac152000 con 0x7f40ac159c60 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38966/0 (socket says 172.21.15.170:38966) 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 -- 172.21.15.170:0/4184909492 learned_addr learned my addr 172.21.15.170:0/4184909492 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40aaffd700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40ac159890 0x7f40ac151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40abfff700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40ac159c60 0x7f40ac1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 -- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40ac159c60 msgr2=0x7f40ac1518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40ac159c60 0x7f40ac1518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 -- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40ac159890 msgr2=0x7f40ac151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40ac159890 0x7f40ac151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.284+0000 7f40ab7fe700 1 -- 172.21.15.170:0/4184909492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c009a20 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40ab7fe700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f40ac070e20 tx=0x7f409c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c012e80 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c01e640 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac1522e0 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c01e9e0 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.285+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac1528a0 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.286+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac04c3c0 con 0x7f40ac069560 2023-10-08T16:10:31.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.286+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f409c024070 con 0x7f40ac069560 2023-10-08T16:10:31.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.286+0000 7f40937fe700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 0x7f4098069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.287+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f409c087140 con 0x7f40ac069560 2023-10-08T16:10:31.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.287+0000 7f40aaffd700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 0x7f4098069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.287+0000 7f40aaffd700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 0x7f4098069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.291+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f409c0543a0 con 0x7f40ac069560 2023-10-08T16:10:31.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.474+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40ac0b1440 con 0x7f40ac069560 2023-10-08T16:10:31.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.475+0000 7f40937fe700 1 -- 172.21.15.170:0/4184909492 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+12183 (secure 0 0 0) 0x7f409c0579c0 con 0x7f40ac069560 2023-10-08T16:10:31.476 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:31.476 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":97,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:30.736320+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:31.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 msgr2=0x7f4098069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 0x7f4098069740 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2023-10-08T16:10:31.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 msgr2=0x7f40ac14d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f40ac070e20 tx=0x7f409c018000 comp rx=0 tx=0).stop 2023-10-08T16:10:31.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 shutdown_connections 2023-10-08T16:10:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40ac159c60 0x7f40ac1518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4098067290 0x7f4098069740 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40ac069560 0x7f40ac14d3e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 --2- 172.21.15.170:0/4184909492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40ac159890 0x7f40ac151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 >> 172.21.15.170:0/4184909492 conn(0x7f40ac068e20 msgr2=0x7f40ac0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 shutdown_connections 2023-10-08T16:10:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.477+0000 7f40b1ac8700 1 -- 172.21.15.170:0/4184909492 wait complete. 2023-10-08T16:10:31.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:10:31.491 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:10:31.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.606+0000 7f9f537ad700 1 Processor -- start 2023-10-08T16:10:31.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.606+0000 7f9f537ad700 1 -- start start 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f537ad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c0cb710 0x7f9f4c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f537ad700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f537ad700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4c0689e0 con 0x7f9f4c159930 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f537ad700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4c0cdb80 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f537ad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c0cbcc0 con 0x7f9f4c0cb710 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f51d4a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f51d4a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38982/0 (socket says 172.21.15.170:38982) 2023-10-08T16:10:31.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f51d4a700 1 -- 172.21.15.170:0/1452660702 learned_addr learned my addr 172.21.15.170:0/1452660702 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.607+0000 7f9f50d48700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c0cb710 0x7f9f4c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f51d4a700 1 -- 172.21.15.170:0/1452660702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c0cb710 msgr2=0x7f9f4c159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f51d4a700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c0cb710 0x7f9f4c159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f51d4a700 1 -- 172.21.15.170:0/1452660702 >> v1:172.21.15.191:6789/0 conn(0x7f9f4c159930 legacy=0x7f9f4c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f51d4a700 1 -- 172.21.15.170:0/1452660702 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c0cbfa0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f51d4a700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f20 tx=0x7f9f44009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b80b980b224bb48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f43fff700 1 -- 172.21.15.170:0/1452660702 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400e070 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f43fff700 1 -- 172.21.15.170:0/1452660702 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4400bb10 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f43fff700 1 -- 172.21.15.170:0/1452660702 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f440092e0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f537ad700 1 -- 172.21.15.170:0/1452660702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 msgr2=0x7f9f4c0cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.608+0000 7f9f537ad700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009f20 tx=0x7f9f44009840 comp rx=0 tx=0).stop 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 -- 172.21.15.170:0/1452660702 shutdown_connections 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c0cd640 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 --2- 172.21.15.170:0/1452660702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c0cb710 0x7f9f4c159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 -- 172.21.15.170:0/1452660702 >> 172.21.15.170:0/1452660702 conn(0x7f9f4c0999d0 msgr2=0x7f9f4c0b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 -- 172.21.15.170:0/1452660702 shutdown_connections 2023-10-08T16:10:31.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.609+0000 7f9f537ad700 1 -- 172.21.15.170:0/1452660702 wait complete. 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.610+0000 7f9f537ad700 1 Processor -- start 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.610+0000 7f9f537ad700 1 -- start start 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4c0cb710 0x7f9f4c09a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 0x7f9f4c16a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c09b9d0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c157290 con 0x7f9f4c0cb710 2023-10-08T16:10:31.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c09adc0 con 0x7f9f4c159930 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38986/0 (socket says 172.21.15.170:38986) 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 -- 172.21.15.170:0/2569031768 learned_addr learned my addr 172.21.15.170:0/2569031768 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f51549700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4c0cb710 0x7f9f4c09a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f51d4a700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 0x7f9f4c16a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 -- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 msgr2=0x7f9f4c16a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 0x7f9f4c16a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 -- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4c0cb710 msgr2=0x7f9f4c09a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4c0cb710 0x7f9f4c09a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 -- 172.21.15.170:0/2569031768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009ae0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f51d4a700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 0x7f9f4c16a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f50d48700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00d8c0 tx=0x7f9f3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00de60 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3c00b780 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c09b0a0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c00bb20 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.611+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c09b660 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.612+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c09bb80 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.613+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9f3c01e460 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.613+0000 7f9f417fa700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 0x7f9f30069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.613+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9f3c084d60 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.613+0000 7f9f51549700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 0x7f9f30069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.613+0000 7f9f51549700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 0x7f9f30069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f48001710 tx=0x7f9f48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.617+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f4c09bb80 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.799+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f4c0b59e0 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.799+0000 7f9f417fa700 1 -- 172.21.15.170:0/2569031768 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f3c051f10 con 0x7f9f4c0cd1c0 2023-10-08T16:10:31.801 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 msgr2=0x7f9f30069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 0x7f9f30069740 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f48001710 tx=0x7f9f48009040 comp rx=0 tx=0).stop 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 msgr2=0x7f9f4c169ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00d8c0 tx=0x7f9f3c008040 comp rx=0 tx=0).stop 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 shutdown_connections 2023-10-08T16:10:31.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9f4c159930 0x7f9f4c16a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9f4c0cb710 0x7f9f4c09a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9f30067290 0x7f9f30069740 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 --2- 172.21.15.170:0/2569031768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f4c0cd1c0 0x7f9f4c169ec0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 >> 172.21.15.170:0/2569031768 conn(0x7f9f4c0999d0 msgr2=0x7f9f4c158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 shutdown_connections 2023-10-08T16:10:31.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.801+0000 7f9f537ad700 1 -- 172.21.15.170:0/2569031768 wait complete. 2023-10-08T16:10:31.815 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:10:31.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 Processor -- start 2023-10-08T16:10:31.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 -- start start 2023-10-08T16:10:31.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b40c77b0 con 0x7f16b40688d0 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b40c8720 con 0x7f16b406b660 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.928+0000 7f16ba99d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b406ac60 con 0x7f16b4155310 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:34862/0 (socket says 172.21.15.170:34862) 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b37fe700 1 -- 172.21.15.170:0/3990128533 learned_addr learned my addr 172.21.15.170:0/3990128533 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:31.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b8f3a700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f16b40688d0 0x7f16b415b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36878/0 (socket says 172.21.15.170:36878) 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 -- 172.21.15.170:0/3990128533 >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 msgr2=0x7f16b4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 --2- 172.21.15.170:0/3990128533 >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 -- 172.21.15.170:0/3990128533 >> v1:172.21.15.191:6789/0 conn(0x7f16b40688d0 legacy=0x7f16b415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 -- 172.21.15.170:0/3990128533 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b406b1d0 con 0x7f16b4155310 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b37fe700 1 --2- 172.21.15.170:0/3990128533 >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.929+0000 7f16b3fff700 1 --2- 172.21.15.170:0/3990128533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40af4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098e0 tx=0x7f16a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0af5806ccd1ca1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16b2ffd700 1 -- 172.21.15.170:0/3990128533 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400b6d0 con 0x7f16b4155310 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16b2ffd700 1 -- 172.21.15.170:0/3990128533 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a400b830 con 0x7f16b4155310 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16b2ffd700 1 -- 172.21.15.170:0/3990128533 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a400bbd0 con 0x7f16b4155310 2023-10-08T16:10:31.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 -- 172.21.15.170:0/3990128533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 msgr2=0x7f16b40af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 --2- 172.21.15.170:0/3990128533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40af4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f16a40098e0 tx=0x7f16a40091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 -- 172.21.15.170:0/3990128533 shutdown_connections 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 --2- 172.21.15.170:0/3990128533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40af4c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 --2- 172.21.15.170:0/3990128533 >> v2:172.21.15.191:3300/0 conn(0x7f16b406b660 0x7f16b4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 -- 172.21.15.170:0/3990128533 >> 172.21.15.170:0/3990128533 conn(0x7f16b40c9860 msgr2=0x7f16b40c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 -- 172.21.15.170:0/3990128533 shutdown_connections 2023-10-08T16:10:31.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.930+0000 7f16ba99d700 1 -- 172.21.15.170:0/3990128533 wait complete. 2023-10-08T16:10:31.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.931+0000 7f16ba99d700 1 Processor -- start 2023-10-08T16:10:31.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.931+0000 7f16ba99d700 1 -- start start 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 0x7f16b4142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40ad790 con 0x7f16b406b660 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40acb80 con 0x7f16b40688d0 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16ba99d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b40ace60 con 0x7f16b4155310 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38990/0 (socket says 172.21.15.170:38990) 2023-10-08T16:10:31.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16b37fe700 1 -- 172.21.15.170:0/1476126831 learned_addr learned my addr 172.21.15.170:0/1476126831 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16b3fff700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 0x7f16b4142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.932+0000 7f16b8f3a700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 -- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 msgr2=0x7f16b40adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 -- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 msgr2=0x7f16b4142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 0x7f16b4142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 -- 172.21.15.170:0/1476126831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80094f0 con 0x7f16b406b660 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b3fff700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 0x7f16b4142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16b37fe700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f16a800ec50 tx=0x7f16a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800c850 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a800c9b0 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a4009420 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a800cd50 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.933+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b40ad420 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.934+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b40ad900 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.934+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f16a8013680 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.935+0000 7f169bfff700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 0x7f16a0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.935+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f16a80847a0 con 0x7f16b406b660 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.935+0000 7f16b3fff700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 0x7f16a0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:31.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.935+0000 7f16b3fff700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 0x7f16a0069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16a40093f0 tx=0x7f16a4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:31.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:31.939+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16a8051950 con 0x7f16b406b660 2023-10-08T16:10:32.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.123+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f16b40ac8d0 con 0x7f16b406b660 2023-10-08T16:10:32.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.124+0000 7f169bfff700 1 -- 172.21.15.170:0/1476126831 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) v1 ==== 84+0+65 (secure 0 0 0) 0x7f16a8054f70 con 0x7f16b406b660 2023-10-08T16:10:32.125 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:32.126 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:10:32.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 msgr2=0x7f16a0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 0x7f16a0069740 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f16a40093f0 tx=0x7f16a4017000 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 msgr2=0x7f16b40b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f16a800ec50 tx=0x7f16a8009250 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 shutdown_connections 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16b4155310 0x7f16b40adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16b40688d0 0x7f16b4142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f16a0067290 0x7f16a0069740 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 --2- 172.21.15.170:0/1476126831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16b406b660 0x7f16b40b2640 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 >> 172.21.15.170:0/1476126831 conn(0x7f16b40c9860 msgr2=0x7f16b4153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 shutdown_connections 2023-10-08T16:10:32.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.126+0000 7f16ba99d700 1 -- 172.21.15.170:0/1476126831 wait complete. 2023-10-08T16:10:32.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:10:32.139 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:10:32.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.256+0000 7f6e489e7700 1 Processor -- start 2023-10-08T16:10:32.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.256+0000 7f6e489e7700 1 -- start start 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e489e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e4406ab40 0x7f6e440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e489e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6e440af8c0 0x7f6e44159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e489e7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e44068900 con 0x7f6e44068390 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e489e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44068be0 con 0x7f6e4406ab40 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e489e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44068ec0 con 0x7f6e440af8c0 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e4406ab40 0x7f6e440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6e4406ab40 0x7f6e440699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39000/0 (socket says 172.21.15.170:39000) 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 -- 172.21.15.170:0/2498905172 learned_addr learned my addr 172.21.15.170:0/2498905172 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e42d9d700 1 --2- 172.21.15.170:0/2498905172 >> v2:172.21.15.191:3300/0 conn(0x7f6e440af8c0 0x7f6e44159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 -- 172.21.15.170:0/2498905172 >> v2:172.21.15.191:3300/0 conn(0x7f6e440af8c0 msgr2=0x7f6e44159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 --2- 172.21.15.170:0/2498905172 >> v2:172.21.15.191:3300/0 conn(0x7f6e440af8c0 0x7f6e44159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 -- 172.21.15.170:0/2498905172 >> v1:172.21.15.170:6789/0 conn(0x7f6e44068390 legacy=0x7f6e440c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 -- 172.21.15.170:0/2498905172 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440691a0 con 0x7f6e4406ab40 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e41d9b700 1 -- 172.21.15.170:0/2498905172 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971919480 0 0) 0x7f6e44068900 con 0x7f6e44068390 2023-10-08T16:10:32.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e4259c700 1 --2- 172.21.15.170:0/2498905172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e4406ab40 0x7f6e440699a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415d680 tx=0x7f6e34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f349ec4128d410 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.257+0000 7f6e41d9b700 1 -- 172.21.15.170:0/2498905172 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34009e70 con 0x7f6e4406ab40 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e41d9b700 1 -- 172.21.15.170:0/2498905172 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3400b8f0 con 0x7f6e4406ab40 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e41d9b700 1 -- 172.21.15.170:0/2498905172 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3400bc90 con 0x7f6e4406ab40 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 -- 172.21.15.170:0/2498905172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e4406ab40 msgr2=0x7f6e440699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2498905172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e4406ab40 0x7f6e440699a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415d680 tx=0x7f6e34009250 comp rx=0 tx=0).stop 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 -- 172.21.15.170:0/2498905172 shutdown_connections 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2498905172 >> v2:172.21.15.191:3300/0 conn(0x7f6e440af8c0 0x7f6e44159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2498905172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e4406ab40 0x7f6e440699a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 -- 172.21.15.170:0/2498905172 >> 172.21.15.170:0/2498905172 conn(0x7f6e440cd640 msgr2=0x7f6e440cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 -- 172.21.15.170:0/2498905172 shutdown_connections 2023-10-08T16:10:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.258+0000 7f6e489e7700 1 -- 172.21.15.170:0/2498905172 wait complete. 2023-10-08T16:10:32.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.259+0000 7f6e489e7700 1 Processor -- start 2023-10-08T16:10:32.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.259+0000 7f6e489e7700 1 -- start start 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e44068390 0x7f6e44158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e4406ab40 0x7f6e440c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44169fe0 con 0x7f6e440af8c0 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e441591f0 con 0x7f6e44068390 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e489e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44169c90 con 0x7f6e4406ab40 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39014/0 (socket says 172.21.15.170:39014) 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 -- 172.21.15.170:0/2818006181 learned_addr learned my addr 172.21.15.170:0/2818006181 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e4259c700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e44068390 0x7f6e44158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e39d9b700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e4406ab40 0x7f6e440c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 -- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e4406ab40 msgr2=0x7f6e440c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e4406ab40 0x7f6e440c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 -- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e44068390 msgr2=0x7f6e44158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e44068390 0x7f6e44158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.260+0000 7f6e42d9d700 1 -- 172.21.15.170:0/2818006181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340094f0 con 0x7f6e440af8c0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e42d9d700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e20 tx=0x7f6e3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00cb50 con 0x7f6e440af8c0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00f040 con 0x7f6e440af8c0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440c6610 con 0x7f6e440af8c0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c014480 con 0x7f6e440af8c0 2023-10-08T16:10:32.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.261+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440c6bd0 con 0x7f6e440af8c0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.262+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4416a190 con 0x7f6e440af8c0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.262+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6e3c014830 con 0x7f6e440af8c0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.262+0000 7f6e3affd700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 0x7f6e24069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.262+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6e3c085360 con 0x7f6e440af8c0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.262+0000 7f6e4259c700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 0x7f6e24069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.263+0000 7f6e4259c700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 0x7f6e24069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415d680 tx=0x7f6e34017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.267+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e4416a190 con 0x7f6e440af8c0 2023-10-08T16:10:32.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.452+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6e440718b0 con 0x7f6e440af8c0 2023-10-08T16:10:32.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.743+0000 7f6e3affd700 1 -- 172.21.15.170:0/2818006181 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6e3c052400 con 0x7f6e440af8c0 2023-10-08T16:10:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 msgr2=0x7f6e24069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 0x7f6e24069740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6e4415d680 tx=0x7f6e34017000 comp rx=0 tx=0).stop 2023-10-08T16:10:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 msgr2=0x7f6e4415bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e20 tx=0x7f6e3c00b040 comp rx=0 tx=0).stop 2023-10-08T16:10:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 shutdown_connections 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e24067290 0x7f6e24069740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e440af8c0 0x7f6e4415bd40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e44068390 0x7f6e44158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 --2- 172.21.15.170:0/2818006181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e4406ab40 0x7f6e440c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 >> 172.21.15.170:0/2818006181 conn(0x7f6e440cd640 msgr2=0x7f6e440cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 shutdown_connections 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.745+0000 7f6e489e7700 1 -- 172.21.15.170:0/2818006181 wait complete. 2023-10-08T16:10:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:10:32.762 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:10:32.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.883+0000 7fbbcbed8700 1 Processor -- start 2023-10-08T16:10:32.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.883+0000 7fbbcbed8700 1 -- start start 2023-10-08T16:10:32.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbcbed8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbbc4159950 0x7fbbc40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbcbed8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbcbed8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc40ce380 con 0x7fbbc4069aa0 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbcbed8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc4069f20 con 0x7fbbc4159950 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbcbed8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40cec20 con 0x7fbbc40af320 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbc9473700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbc9c74700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbbc4159950 0x7fbbc40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbca475700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fbbc4069aa0 0x7fbbc415d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:36884/0 (socket says 172.21.15.170:36884) 2023-10-08T16:10:32.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.884+0000 7fbbca475700 1 -- 172.21.15.170:0/3945281394 learned_addr learned my addr 172.21.15.170:0/3945281394 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:32.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc8c72700 1 -- 172.21.15.170:0/3945281394 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014859208 0 0) 0x7fbbc40ce380 con 0x7fbbc4069aa0 2023-10-08T16:10:32.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc8c72700 1 -- 172.21.15.170:0/3945281394 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac003570 con 0x7fbbc4069aa0 2023-10-08T16:10:32.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 msgr2=0x7fbbc4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:32.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9c74700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3945281394 >> v1:172.21.15.191:6789/0 conn(0x7fbbc4069aa0 legacy=0x7fbbc415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3945281394 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc40afa90 con 0x7fbbc4159950 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9473700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc9c74700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40cbb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009e60 tx=0x7fbbb4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8046d05753f74a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc8c72700 1 -- 172.21.15.170:0/3945281394 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400e070 con 0x7fbbc4159950 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc8c72700 1 -- 172.21.15.170:0/3945281394 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb400bab0 con 0x7fbbc4159950 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.885+0000 7fbbc8c72700 1 -- 172.21.15.170:0/3945281394 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb400be70 con 0x7fbbc4159950 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 msgr2=0x7fbbc40cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40cbb90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4009e60 tx=0x7fbbb4009780 comp rx=0 tx=0).stop 2023-10-08T16:10:32.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3945281394 shutdown_connections 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40cbb90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3945281394 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3945281394 >> 172.21.15.170:0/3945281394 conn(0x7fbbc4068e00 msgr2=0x7fbbc40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3945281394 shutdown_connections 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.886+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3945281394 wait complete. 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.887+0000 7fbbcbed8700 1 Processor -- start 2023-10-08T16:10:32.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.887+0000 7fbbcbed8700 1 -- start start 2023-10-08T16:10:32.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc416ab90 con 0x7fbbc4069aa0 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc416d520 con 0x7fbbc4159950 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc416d800 con 0x7fbbc40af320 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39022/0 (socket says 172.21.15.170:39022) 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3953557227 learned_addr learned my addr 172.21.15.170:0/3953557227 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:32.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9473700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbca475700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 msgr2=0x7fbbc4150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 msgr2=0x7fbbc40b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 -- 172.21.15.170:0/3953557227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009420 con 0x7fbbc4069aa0 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbca475700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc9c74700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4070e20 tx=0x7fbbb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4012d00 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4012e60 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb4009a20 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.888+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40276c0 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.889+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc416d1b0 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.890+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc416d9c0 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.890+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fbbb4024070 con 0x7fbbc4069aa0 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.890+0000 7fbbc1ffb700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 0x7fbbb0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:32.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.891+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fbbb4086dc0 con 0x7fbbc4069aa0 2023-10-08T16:10:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.891+0000 7fbbc9473700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 0x7fbbb0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.891+0000 7fbbc9473700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 0x7fbbb0069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80093f0 tx=0x7fbbb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:32.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:32.895+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbb4053e00 con 0x7fbbc4069aa0 2023-10-08T16:10:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.078+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbbc40ce5d0 con 0x7fbbc4069aa0 2023-10-08T16:10:33.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.748+0000 7fbbc1ffb700 1 -- 172.21.15.170:0/3953557227 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbbb4057420 con 0x7fbbc4069aa0 2023-10-08T16:10:33.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.753+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 msgr2=0x7fbbb0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.753+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 0x7fbbb0069740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80093f0 tx=0x7fbbb8008040 comp rx=0 tx=0).stop 2023-10-08T16:10:33.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.753+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 msgr2=0x7fbbc416a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.753+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbbc4070e20 tx=0x7fbbb4019000 comp rx=0 tx=0).stop 2023-10-08T16:10:33.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 shutdown_connections 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbbc4159950 0x7fbbc40b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbbb0067290 0x7fbbb0069740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbbc4069aa0 0x7fbbc416a650 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 --2- 172.21.15.170:0/3953557227 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbbc40af320 0x7fbbc4150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.755+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 >> 172.21.15.170:0/3953557227 conn(0x7fbbc4068e00 msgr2=0x7fbbc4151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:33.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.756+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 shutdown_connections 2023-10-08T16:10:33.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.757+0000 7fbbcbed8700 1 -- 172.21.15.170:0/3953557227 wait complete. 2023-10-08T16:10:33.761 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:10:33.779 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:10:33.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.897+0000 7fcac51d8700 1 Processor -- start 2023-10-08T16:10:33.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.897+0000 7fcac51d8700 1 -- start start 2023-10-08T16:10:33.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcac51d8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcac00b4e70 0x7fcac00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:33.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcac51d8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00cfb00 con 0x7fcac00b1b40 2023-10-08T16:10:33.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcac51d8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac015e060 con 0x7fcac00b16d0 2023-10-08T16:10:33.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcac51d8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac00b5c40 con 0x7fcac00b4e70 2023-10-08T16:10:33.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcabffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcac00b4e70 0x7fcac00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcabffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcac00b4e70 0x7fcac00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39026/0 (socket says 172.21.15.170:39026) 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcabffff700 1 -- 172.21.15.170:0/211746644 learned_addr learned my addr 172.21.15.170:0/211746644 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.898+0000 7fcac49d7700 1 --1- 172.21.15.170:0/211746644 >> v1:172.21.15.170:6789/0 conn(0x7fcac00b1b40 0x7fcac015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33478/0 (socket says 172.21.15.170:33478) 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabffff700 1 -- 172.21.15.170:0/211746644 >> v1:172.21.15.191:6789/0 conn(0x7fcac00b16d0 legacy=0x7fcac015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabffff700 1 -- 172.21.15.170:0/211746644 >> v1:172.21.15.170:6789/0 conn(0x7fcac00b1b40 legacy=0x7fcac015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabffff700 1 -- 172.21.15.170:0/211746644 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00b5f20 con 0x7fcac00b4e70 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabf7fe700 1 -- 172.21.15.170:0/211746644 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034551514 0 0) 0x7fcac00cfb00 con 0x7fcac00b1b40 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabffff700 1 --2- 172.21.15.170:0/211746644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b4e70 0x7fcac00ccff0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fcab00099b0 tx=0x7fcab0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57d08d971afa951 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabf7fe700 1 -- 172.21.15.170:0/211746644 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000b860 con 0x7fcac00b4e70 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabf7fe700 1 -- 172.21.15.170:0/211746644 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab000b9c0 con 0x7fcac00b4e70 2023-10-08T16:10:33.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcabf7fe700 1 -- 172.21.15.170:0/211746644 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000bd60 con 0x7fcac00b4e70 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcac51d8700 1 -- 172.21.15.170:0/211746644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b4e70 msgr2=0x7fcac00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcac51d8700 1 --2- 172.21.15.170:0/211746644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b4e70 0x7fcac00ccff0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fcab00099b0 tx=0x7fcab0009250 comp rx=0 tx=0).stop 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcac51d8700 1 -- 172.21.15.170:0/211746644 shutdown_connections 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcac51d8700 1 --2- 172.21.15.170:0/211746644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b4e70 0x7fcac00ccff0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.899+0000 7fcac51d8700 1 -- 172.21.15.170:0/211746644 >> 172.21.15.170:0/211746644 conn(0x7fcac00d10c0 msgr2=0x7fcac00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:33.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.900+0000 7fcac51d8700 1 -- 172.21.15.170:0/211746644 shutdown_connections 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.900+0000 7fcac51d8700 1 -- 172.21.15.170:0/211746644 wait complete. 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 Processor -- start 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 -- start start 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcac00b1b40 0x7fcac015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac00b4e70 0x7fcac00c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac015b8c0 con 0x7fcac00b16d0 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0154830 con 0x7fcac00b4e70 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcac51d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac015acb0 con 0x7fcac00b1b40 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcabffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:33.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcabffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39030/0 (socket says 172.21.15.170:39030) 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.901+0000 7fcabffff700 1 -- 172.21.15.170:0/1304131990 learned_addr learned my addr 172.21.15.170:0/1304131990 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 -- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcac00b1b40 msgr2=0x7fcac015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcac00b1b40 0x7fcac015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 -- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac00b4e70 msgr2=0x7fcac00c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac00b4e70 0x7fcac00c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 -- 172.21.15.170:0/1304131990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094f0 con 0x7fcac00b16d0 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabffff700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006f50 tx=0x7fcab0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab001f070 con 0x7fcac00b16d0 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0011d10 con 0x7fcac00b16d0 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac015af90 con 0x7fcac00b16d0 2023-10-08T16:10:33.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0025650 con 0x7fcac00b16d0 2023-10-08T16:10:33.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.902+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac015b550 con 0x7fcac00b16d0 2023-10-08T16:10:33.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.903+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0088990 con 0x7fcac00b16d0 2023-10-08T16:10:33.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.904+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcab0023070 con 0x7fcac00b16d0 2023-10-08T16:10:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.904+0000 7fcabcff9700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 0x7fcaa0069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.904+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fcab0085fd0 con 0x7fcac00b16d0 2023-10-08T16:10:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.904+0000 7fcab7fff700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 0x7fcaa0069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:33.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.904+0000 7fcab7fff700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 0x7fcaa0069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcac0069140 tx=0x7fcaa8007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:33.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:33.908+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcab0052e90 con 0x7fcac00b16d0 2023-10-08T16:10:34.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.134+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcac00b5370 con 0x7fcac00b16d0 2023-10-08T16:10:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.760+0000 7fcabcff9700 1 -- 172.21.15.170:0/1304131990 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcab00150c0 con 0x7fcac00b16d0 2023-10-08T16:10:34.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.772+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 msgr2=0x7fcaa0069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.772+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 0x7fcaa0069740 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcac0069140 tx=0x7fcaa8007b40 comp rx=0 tx=0).stop 2023-10-08T16:10:34.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.772+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 msgr2=0x7fcac01542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.772+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcab0006f50 tx=0x7fcab0017000 comp rx=0 tx=0).stop 2023-10-08T16:10:34.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 shutdown_connections 2023-10-08T16:10:34.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcac00b4e70 0x7fcac00c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcaa0067290 0x7fcaa0069740 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcac00b16d0 0x7fcac01542f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 --2- 172.21.15.170:0/1304131990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcac00b1b40 0x7fcac015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.773+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 >> 172.21.15.170:0/1304131990 conn(0x7fcac00d10c0 msgr2=0x7fcac00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:34.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.775+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 shutdown_connections 2023-10-08T16:10:34.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.775+0000 7fcac51d8700 1 -- 172.21.15.170:0/1304131990 wait complete. 2023-10-08T16:10:34.778 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 16 and data pool 17 2023-10-08T16:10:34.790 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:34.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.903+0000 7fcc56fe8700 1 Processor -- start 2023-10-08T16:10:34.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.903+0000 7fcc56fe8700 1 -- start start 2023-10-08T16:10:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc56fe8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc50159420 0x7fcc500b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc56fe8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc5015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc56fe8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc500b2980 con 0x7fcc501597f0 2023-10-08T16:10:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc56fe8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5015dbb0 con 0x7fcc50159420 2023-10-08T16:10:34.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc56fe8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500b11d0 con 0x7fcc5015d230 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc50159420 0x7fcc500b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcc50159420 0x7fcc500b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39044/0 (socket says 172.21.15.170:39044) 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 -- 172.21.15.170:0/148515211 learned_addr learned my addr 172.21.15.170:0/148515211 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc557e5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcc501597f0 0x7fcc500b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33484/0 (socket says 172.21.15.170:33484) 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc567e7700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc5015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 -- 172.21.15.170:0/148515211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 msgr2=0x7fcc5015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc5015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 -- 172.21.15.170:0/148515211 >> v1:172.21.15.170:6789/0 conn(0x7fcc501597f0 legacy=0x7fcc500b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc55fe6700 1 -- 172.21.15.170:0/148515211 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc500b14b0 con 0x7fcc50159420 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc54fe4700 1 -- 172.21.15.170:0/148515211 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2631386656 0 0) 0x7fcc500b2980 con 0x7fcc501597f0 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.904+0000 7fcc567e7700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc5015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:34.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc55fe6700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500b5f40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009f20 tx=0x7fcc40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29b2770e60aac00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:34.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc54fe4700 1 -- 172.21.15.170:0/148515211 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4000e070 con 0x7fcc50159420 2023-10-08T16:10:34.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc54fe4700 1 -- 172.21.15.170:0/148515211 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4000ba80 con 0x7fcc50159420 2023-10-08T16:10:34.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc54fe4700 1 -- 172.21.15.170:0/148515211 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc400092e0 con 0x7fcc50159420 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 -- 172.21.15.170:0/148515211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 msgr2=0x7fcc500b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500b5f40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009f20 tx=0x7fcc40009840 comp rx=0 tx=0).stop 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 -- 172.21.15.170:0/148515211 shutdown_connections 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc5015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/148515211 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500b5f40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 -- 172.21.15.170:0/148515211 >> 172.21.15.170:0/148515211 conn(0x7fcc500d0660 msgr2=0x7fcc500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 -- 172.21.15.170:0/148515211 shutdown_connections 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.905+0000 7fcc56fe8700 1 -- 172.21.15.170:0/148515211 wait complete. 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 Processor -- start 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 -- start start 2023-10-08T16:10:34.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 0x7fcc5014a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc50068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc50156920 con 0x7fcc50159420 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500c6ea0 con 0x7fcc501597f0 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc56fe8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc500c6290 con 0x7fcc5015d230 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc55fe6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc55fe6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39056/0 (socket says 172.21.15.170:39056) 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.907+0000 7fcc55fe6700 1 -- 172.21.15.170:0/3203891893 learned_addr learned my addr 172.21.15.170:0/3203891893 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc567e7700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc50068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc557e5700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 0x7fcc5014a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 -- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 msgr2=0x7fcc50068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc50068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 -- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 msgr2=0x7fcc5014a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 0x7fcc5014a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 -- 172.21.15.170:0/3203891893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40009ae0 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc557e5700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 0x7fcc5014a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc55fe6700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcc50089340 tx=0x7fcc40018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40008c60 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40008de0 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc500c6570 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc400266b0 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.908+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc500c6b30 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.909+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5015dd50 con 0x7fcc50159420 2023-10-08T16:10:34.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.910+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcc40003040 con 0x7fcc50159420 2023-10-08T16:10:34.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.910+0000 7fcc4dffb700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 0x7fcc3c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:34.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.910+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcc40086ae0 con 0x7fcc50159420 2023-10-08T16:10:34.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.910+0000 7fcc557e5700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 0x7fcc3c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:34.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.910+0000 7fcc557e5700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 0x7fcc3c069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcc500b0fb0 tx=0x7fcc44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:34.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:34.914+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc400538e0 con 0x7fcc50159420 2023-10-08T16:10:35.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.099+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc5004c3c0 con 0x7fcc50159420 2023-10-08T16:10:35.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.100+0000 7fcc4dffb700 1 -- 172.21.15.170:0/3203891893 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15417 (secure 0 0 0) 0x7fcc40056f00 con 0x7fcc50159420 2023-10-08T16:10:35.102 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:35.102 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:34.136179+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 msgr2=0x7fcc3c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 0x7fcc3c069740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcc500b0fb0 tx=0x7fcc44009040 comp rx=0 tx=0).stop 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 msgr2=0x7fcc500c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcc50089340 tx=0x7fcc40018000 comp rx=0 tx=0).stop 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 shutdown_connections 2023-10-08T16:10:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcc5015d230 0x7fcc50068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc3c067290 0x7fcc3c069740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc50159420 0x7fcc500c5c00 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 --2- 172.21.15.170:0/3203891893 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcc501597f0 0x7fcc5014a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 >> 172.21.15.170:0/3203891893 conn(0x7fcc500d0660 msgr2=0x7fcc50158280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 shutdown_connections 2023-10-08T16:10:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.102+0000 7fcc56fe8700 1 -- 172.21.15.170:0/3203891893 wait complete. 2023-10-08T16:10:35.116 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:35.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.228+0000 7f8b26477700 1 Processor -- start 2023-10-08T16:10:35.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.228+0000 7f8b26477700 1 -- start start 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b26477700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b200b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b26477700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200b11d0 con 0x7f8b20159420 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b26477700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b200b14b0 con 0x7f8b2015d230 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b26477700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b1790 con 0x7f8b201597f0 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b25475700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8b20159420 0x7f8b200b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33622/0 (socket says 172.21.15.170:33622) 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b25475700 1 -- 172.21.15.170:0/1712167265 learned_addr learned my addr 172.21.15.170:0/1712167265 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:35.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b24c74700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b200b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173183208 0 0) 0x7f8b200b11d0 con 0x7f8b20159420 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.229+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b0c003570 con 0x7f8b20159420 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483536788 0 0) 0x7f8b200b14b0 con 0x7f8b2015d230 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b200b11d0 con 0x7f8b2015d230 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336847307 0 0) 0x7f8b0c003570 con 0x7f8b20159420 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 msgr2=0x7f8b200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b200b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 >> v1:172.21.15.191:6789/0 conn(0x7f8b2015d230 legacy=0x7f8b200b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b200b1a70 con 0x7f8b20159420 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8b1c001470 con 0x7f8b20159420 2023-10-08T16:10:35.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 >> v1:172.21.15.170:6789/0 conn(0x7f8b20159420 legacy=0x7f8b200b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c003c20 0x7f8b0c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c004810 0x7f8b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200b1790 con 0x7f8b0c004810 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00bb40 con 0x7f8b0c003c20 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c00bd00 con 0x7f8b201597f0 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c004810 0x7f8b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b25475700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c003c20 0x7f8b0c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b25c76700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 -- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 msgr2=0x7f8b0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:35.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 -- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c003c20 msgr2=0x7f8b0c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c003c20 0x7f8b0c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 -- 172.21.15.170:0/1712167265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c003570 con 0x7f8b0c004810 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b25c76700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b201597f0 0x7f8b0c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b24c74700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c004810 0x7f8b0c004c80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009c00 tx=0x7f8b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f042759df569a241 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000e710 con 0x7f8b0c004810 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1000e870 con 0x7f8b0c004810 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.230+0000 7f8b17fff700 1 -- 172.21.15.170:0/1712167265 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000ec30 con 0x7f8b0c004810 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 -- 172.21.15.170:0/1712167265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c004810 msgr2=0x7f8b0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 --2- 172.21.15.170:0/1712167265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c004810 0x7f8b0c004c80 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009c00 tx=0x7f8b100091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b25c76700 1 -- 172.21.15.170:0/1712167265 reap_dead start 2023-10-08T16:10:35.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 -- 172.21.15.170:0/1712167265 shutdown_connections 2023-10-08T16:10:35.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 -- 172.21.15.170:0/1712167265 >> 172.21.15.170:0/1712167265 conn(0x7f8b200d0660 msgr2=0x7f8b200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:35.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 -- 172.21.15.170:0/1712167265 shutdown_connections 2023-10-08T16:10:35.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.231+0000 7f8b26477700 1 -- 172.21.15.170:0/1712167265 wait complete. 2023-10-08T16:10:35.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.232+0000 7f8b26477700 1 Processor -- start 2023-10-08T16:10:35.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.232+0000 7f8b26477700 1 -- start start 2023-10-08T16:10:35.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c004810 0x7f8b20169680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20169bc0 0x7f8b20155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b201566d0 con 0x7f8b20156230 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2016ac40 con 0x7f8b0c004810 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b26477700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2016a070 con 0x7f8b20169bc0 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37876/0 (socket says 172.21.15.170:37876) 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 -- 172.21.15.170:0/3506575765 learned_addr learned my addr 172.21.15.170:0/3506575765 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25475700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c004810 0x7f8b20169680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b24c74700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20169bc0 0x7f8b20155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 -- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20169bc0 msgr2=0x7f8b20155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20169bc0 0x7f8b20155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 -- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c004810 msgr2=0x7f8b20169680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c004810 0x7f8b20169680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.233+0000 7f8b25c76700 1 -- 172.21.15.170:0/3506575765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2016a310 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b25c76700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f8b2015d600 tx=0x7f8b18009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18017070 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b180032e0 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18006ba0 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2016a5f0 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.234+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b200c8030 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.235+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2016add0 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.235+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8b18001680 con 0x7f8b20156230 2023-10-08T16:10:35.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.235+0000 7f8b157fa700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 0x7f8b04069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.235+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b18084490 con 0x7f8b20156230 2023-10-08T16:10:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.235+0000 7f8b25475700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 0x7f8b04069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.236+0000 7f8b25475700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 0x7f8b04069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c001200 tx=0x7f8b1c001e10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.240+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b18054d80 con 0x7f8b20156230 2023-10-08T16:10:35.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.421+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b2004c3c0 con 0x7f8b20156230 2023-10-08T16:10:35.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.422+0000 7f8b157fa700 1 -- 172.21.15.170:0/3506575765 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+15417 (secure 0 0 0) 0x7f8b18054910 con 0x7f8b20156230 2023-10-08T16:10:35.424 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:35.424 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:34.136179+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 msgr2=0x7f8b04069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 0x7f8b04069740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c001200 tx=0x7f8b1c001e10 comp rx=0 tx=0).stop 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 msgr2=0x7f8b200c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f8b2015d600 tx=0x7f8b18009040 comp rx=0 tx=0).stop 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 shutdown_connections 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b04067290 0x7f8b04069740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b20156230 0x7f8b200c7af0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b0c004810 0x7f8b20169680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 --2- 172.21.15.170:0/3506575765 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b20169bc0 0x7f8b20155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 >> 172.21.15.170:0/3506575765 conn(0x7f8b200d0660 msgr2=0x7f8b20158360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:35.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 shutdown_connections 2023-10-08T16:10:35.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.424+0000 7f8b26477700 1 -- 172.21.15.170:0/3506575765 wait complete. 2023-10-08T16:10:35.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:10:35.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.553+0000 7fd418c32700 1 Processor -- start 2023-10-08T16:10:35.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.553+0000 7fd418c32700 1 -- start start 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd418c32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd418c32700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd414069560 0x7fd4140699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd418c32700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41415a160 con 0x7fd414159c60 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd418c32700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd414069f20 con 0x7fd414069560 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd418c32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140d13a0 con 0x7fd414159890 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd414069560 0x7fd4140699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd41259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd414069560 0x7fd4140699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37880/0 (socket says 172.21.15.170:37880) 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd41259c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:40438/0 (socket says 172.21.15.170:40438) 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 -- 172.21.15.170:0/3964637880 learned_addr learned my addr 172.21.15.170:0/3964637880 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd411d9b700 1 --1- 172.21.15.170:0/3964637880 >> v1:172.21.15.191:6789/0 conn(0x7fd414159c60 0x7fd41415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:46934/0 (socket says 172.21.15.170:46934) 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 -- 172.21.15.170:0/3964637880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 msgr2=0x7fd41415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 -- 172.21.15.170:0/3964637880 >> v1:172.21.15.191:6789/0 conn(0x7fd414159c60 legacy=0x7fd41415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 -- 172.21.15.170:0/3964637880 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41415e140 con 0x7fd414069560 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd41159a700 1 -- 172.21.15.170:0/3964637880 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327766429 0 0) 0x7fd41415a160 con 0x7fd414159c60 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd41259c700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:35.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd412d9d700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140699e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009f20 tx=0x7fd40c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1833aea57177880 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.554+0000 7fd41159a700 1 -- 172.21.15.170:0/3964637880 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00e070 con 0x7fd414069560 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd41159a700 1 -- 172.21.15.170:0/3964637880 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c00b9d0 con 0x7fd414069560 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd41159a700 1 -- 172.21.15.170:0/3964637880 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0092e0 con 0x7fd414069560 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 -- 172.21.15.170:0/3964637880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 msgr2=0x7fd4140699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140699e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009f20 tx=0x7fd40c009840 comp rx=0 tx=0).stop 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 -- 172.21.15.170:0/3964637880 shutdown_connections 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140699e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 --2- 172.21.15.170:0/3964637880 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159890 0x7fd41415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 -- 172.21.15.170:0/3964637880 >> 172.21.15.170:0/3964637880 conn(0x7fd414068e20 msgr2=0x7fd4140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 -- 172.21.15.170:0/3964637880 shutdown_connections 2023-10-08T16:10:35.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.555+0000 7fd418c32700 1 -- 172.21.15.170:0/3964637880 wait complete. 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.556+0000 7fd418c32700 1 Processor -- start 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.556+0000 7fd418c32700 1 -- start start 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd414159890 0x7fd41414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159c60 0x7fd4140c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140ce5e0 con 0x7fd414069560 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41415d6e0 con 0x7fd414159890 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd418c32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140ce010 con 0x7fd414159c60 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37882/0 (socket says 172.21.15.170:37882) 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 -- 172.21.15.170:0/1463751046 learned_addr learned my addr 172.21.15.170:0/1463751046 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd411d9b700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd414159890 0x7fd41414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd412d9d700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159c60 0x7fd4140c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 -- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159c60 msgr2=0x7fd4140c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159c60 0x7fd4140c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 -- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd414159890 msgr2=0x7fd41414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd414159890 0x7fd41414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 -- 172.21.15.170:0/1463751046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009420 con 0x7fd414069560 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.557+0000 7fd41259c700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec80 tx=0x7fd4040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400c920 con 0x7fd414069560 2023-10-08T16:10:35.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400ca80 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400ce00 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40c009ae0 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41414d8d0 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.558+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41415d9c0 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.559+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fd404013680 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.559+0000 7fd4027fc700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 0x7fd3f4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.560+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd404084e00 con 0x7fd414069560 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.560+0000 7fd411d9b700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 0x7fd3f4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:35.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.560+0000 7fd411d9b700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 0x7fd3f4069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd4140b6630 tx=0x7fd408007ef0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:35.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.564+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd404051c60 con 0x7fd414069560 2023-10-08T16:10:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:35.797+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd41404c3c0 con 0x7fd414069560 2023-10-08T16:10:36.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.784+0000 7fd4027fc700 1 -- 172.21.15.170:0/1463751046 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd404055280 con 0x7fd414069560 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 msgr2=0x7fd3f4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 0x7fd3f4069740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd4140b6630 tx=0x7fd408007ef0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 msgr2=0x7fd4140cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fd40400ec80 tx=0x7fd4040091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 shutdown_connections 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd414159c60 0x7fd4140c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd3f4067290 0x7fd3f4069740 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd414069560 0x7fd4140cdad0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 --2- 172.21.15.170:0/1463751046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd414159890 0x7fd41414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 >> 172.21.15.170:0/1463751046 conn(0x7fd414068e20 msgr2=0x7fd4140d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 shutdown_connections 2023-10-08T16:10:36.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.786+0000 7fd418c32700 1 -- 172.21.15.170:0/1463751046 wait complete. 2023-10-08T16:10:36.801 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:10:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.920+0000 7fdf954e8700 1 Processor -- start 2023-10-08T16:10:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.920+0000 7fdf954e8700 1 -- start start 2023-10-08T16:10:36.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf954e8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf900caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf954e8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdf9015df50 0x7fdf9015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf954e8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9015d7e0 con 0x7fdf9006bcd0 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf954e8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9015dac0 con 0x7fdf9015df50 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf954e8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015aa30 con 0x7fdf9006c0a0 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdf9006bcd0 0x7fdf9015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33638/0 (socket says 172.21.15.170:33638) 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8effd700 1 -- 172.21.15.170:0/1596570150 learned_addr learned my addr 172.21.15.170:0/1596570150 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698552849 0 0) 0x7fdf9015d7e0 con 0x7fdf9006bcd0 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf900caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8f7fe700 1 --2- 172.21.15.170:0/1596570150 >> v2:172.21.15.191:3300/0 conn(0x7fdf9015df50 0x7fdf9015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf78003570 con 0x7fdf9006bcd0 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1410681586 0 0) 0x7fdf78003570 con 0x7fdf9006bcd0 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 msgr2=0x7fdf900caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf900caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 >> v2:172.21.15.191:3300/0 conn(0x7fdf9015df50 msgr2=0x7fdf9015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 --2- 172.21.15.170:0/1596570150 >> v2:172.21.15.191:3300/0 conn(0x7fdf9015df50 0x7fdf9015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9015ad10 con 0x7fdf9006bcd0 2023-10-08T16:10:36.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fdf800017f0 con 0x7fdf9006bcd0 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 >> v1:172.21.15.170:6789/0 conn(0x7fdf9006bcd0 legacy=0x7fdf9015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf78003a60 0x7fdf78003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf78004810 0x7fdf78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015dac0 con 0x7fdf78004810 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015aa30 con 0x7fdf78003a60 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7800bae0 con 0x7fdf9006c0a0 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf78004810 0x7fdf78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8f7fe700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8effd700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf78003a60 0x7fdf78003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 -- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 msgr2=0x7fdf780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 -- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf78003a60 msgr2=0x7fdf78003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf78003a60 0x7fdf78003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 -- 172.21.15.170:0/1596570150 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78003570 con 0x7fdf78004810 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8effd700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf78003a60 0x7fdf78003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8f7fe700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf9006c0a0 0x7fdf780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf78004810 0x7fdf78004c80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009c00 tx=0x7fdf840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd1ff5e28618f1c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400e710 con 0x7fdf78004810 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8400e870 con 0x7fdf78004810 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.922+0000 7fdf8dffb700 1 -- 172.21.15.170:0/1596570150 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8400ec30 con 0x7fdf78004810 2023-10-08T16:10:36.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 -- 172.21.15.170:0/1596570150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf78004810 msgr2=0x7fdf78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 --2- 172.21.15.170:0/1596570150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf78004810 0x7fdf78004c80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009c00 tx=0x7fdf840091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf8f7fe700 1 -- 172.21.15.170:0/1596570150 reap_dead start 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 -- 172.21.15.170:0/1596570150 shutdown_connections 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 -- 172.21.15.170:0/1596570150 >> 172.21.15.170:0/1596570150 conn(0x7fdf900ce600 msgr2=0x7fdf900cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 -- 172.21.15.170:0/1596570150 shutdown_connections 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.923+0000 7fdf954e8700 1 -- 172.21.15.170:0/1596570150 wait complete. 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.924+0000 7fdf954e8700 1 Processor -- start 2023-10-08T16:10:36.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.924+0000 7fdf954e8700 1 -- start start 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf90153be0 0x7fdf90154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 0x7fdf900c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90152c40 con 0x7fdf900b1c80 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900c42f0 con 0x7fdf900c4780 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf954e8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900c36e0 con 0x7fdf90153be0 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37914/0 (socket says 172.21.15.170:37914) 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 -- 172.21.15.170:0/2285117160 learned_addr learned my addr 172.21.15.170:0/2285117160 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:36.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8f7fe700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 0x7fdf900c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf90153be0 0x7fdf90154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 -- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf90153be0 msgr2=0x7fdf90154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf90153be0 0x7fdf90154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 -- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 msgr2=0x7fdf900c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 0x7fdf900c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 -- 172.21.15.170:0/2285117160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf900c39c0 con 0x7fdf900b1c80 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8f7fe700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 0x7fdf900c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.925+0000 7fdf8effd700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdf80001350 tx=0x7fdf80001840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:36.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.926+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80012590 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.926+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf800126f0 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.926+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf900c3ca0 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.926+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80012a70 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.926+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9016a360 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.927+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9004c3c0 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.927+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fdf8001d460 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.927+0000 7fdf76ffd700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 0x7fdf7c069730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.928+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 0x7fdf7c069730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.928+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdf800855f0 con 0x7fdf900b1c80 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.928+0000 7fdf8e7fc700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 0x7fdf7c069730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf9004d1b0 tx=0x7fdf8401a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:36.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:36.932+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf80052530 con 0x7fdf900b1c80 2023-10-08T16:10:37.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.161+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdf9015aff0 con 0x7fdf900b1c80 2023-10-08T16:10:37.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.785+0000 7fdf76ffd700 1 -- 172.21.15.170:0/2285117160 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf80055b50 con 0x7fdf900b1c80 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 msgr2=0x7fdf7c069730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 0x7fdf7c069730 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf9004d1b0 tx=0x7fdf8401a000 comp rx=0 tx=0).stop 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 msgr2=0x7fdf900b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdf80001350 tx=0x7fdf80001840 comp rx=0 tx=0).stop 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 shutdown_connections 2023-10-08T16:10:37.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf900c4780 0x7fdf900c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf7c067280 0x7fdf7c069730 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf900b1c80 0x7fdf900b2050 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 --2- 172.21.15.170:0/2285117160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdf90153be0 0x7fdf90154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.787+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 >> 172.21.15.170:0/2285117160 conn(0x7fdf900ce600 msgr2=0x7fdf9015c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.788+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 shutdown_connections 2023-10-08T16:10:37.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.788+0000 7fdf954e8700 1 -- 172.21.15.170:0/2285117160 wait complete. 2023-10-08T16:10:37.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.916+0000 7f83fe9f2700 1 Processor -- start 2023-10-08T16:10:37.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.916+0000 7f83fe9f2700 1 -- start start 2023-10-08T16:10:37.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83fe9f2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f83f80b1c20 0x7f83f81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83fe9f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83f80abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83fe9f2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f8157c80 con 0x7f83f80b1850 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83fe9f2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f806fc30 con 0x7f83f80b1c20 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83fe9f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80cc260 con 0x7f83f8157820 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83f7fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f83f80b1850 0x7f83f80ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33648/0 (socket says 172.21.15.170:33648) 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.917+0000 7f83f7fff700 1 -- 172.21.15.170:0/794435320 learned_addr learned my addr 172.21.15.170:0/794435320 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83fcf8f700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83f80abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 --2- 172.21.15.170:0/794435320 >> v2:172.21.15.191:3300/0 conn(0x7f83f80b1c20 0x7f83f81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217390242 0 0) 0x7f83f8157c80 con 0x7f83f80b1850 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e0003570 con 0x7f83f80b1850 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2920065426 0 0) 0x7f83e0003570 con 0x7f83f80b1850 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 msgr2=0x7f83f80abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83f80abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 >> v2:172.21.15.191:3300/0 conn(0x7f83f80b1c20 msgr2=0x7f83f81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 --2- 172.21.15.170:0/794435320 >> v2:172.21.15.191:3300/0 conn(0x7f83f80b1c20 0x7f83f81572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80cb300 con 0x7f83f80b1850 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f83e8001470 con 0x7f83f80b1850 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 >> v1:172.21.15.170:6789/0 conn(0x7f83f80b1850 legacy=0x7f83f80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83e0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e0003fe0 0x7f83e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83e0004d50 0x7f83e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806fc30 con 0x7f83e0003fe0 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80cc260 con 0x7f83e0004d50 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e000ba90 con 0x7f83f8157820 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e0003fe0 0x7f83e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f7fff700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83e0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83fcf8f700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83e0004d50 0x7f83e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 -- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 msgr2=0x7f83e0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83f8157820 0x7f83e0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 -- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83e0004d50 msgr2=0x7f83e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83e0004d50 0x7f83e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.918+0000 7f83f77fe700 1 -- 172.21.15.170:0/794435320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0003570 con 0x7f83e0003fe0 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83f77fe700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e0003fe0 0x7f83e0004810 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009c00 tx=0x7f83ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f772d2f7494bcef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00e6b0 con 0x7f83e0003fe0 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00e810 con 0x7f83e0003fe0 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83f6ffd700 1 -- 172.21.15.170:0/794435320 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00ebd0 con 0x7f83e0003fe0 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 -- 172.21.15.170:0/794435320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e0003fe0 msgr2=0x7f83e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/794435320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83e0003fe0 0x7f83e0004810 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009c00 tx=0x7f83ec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fcf8f700 1 -- 172.21.15.170:0/794435320 reap_dead start 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 -- 172.21.15.170:0/794435320 shutdown_connections 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 -- 172.21.15.170:0/794435320 >> 172.21.15.170:0/794435320 conn(0x7f83f806a320 msgr2=0x7f83f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 -- 172.21.15.170:0/794435320 shutdown_connections 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.919+0000 7f83fe9f2700 1 -- 172.21.15.170:0/794435320 wait complete. 2023-10-08T16:10:37.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.920+0000 7f83fe9f2700 1 Processor -- start 2023-10-08T16:10:37.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 -- start start 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83e0003fe0 0x7f83f816b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83f80a62b0 0x7f83f815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f81598f0 con 0x7f83f80a5e40 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8158a70 con 0x7f83f80a62b0 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fe9f2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f815d200 con 0x7f83e0003fe0 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83f77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83f77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37944/0 (socket says 172.21.15.170:37944) 2023-10-08T16:10:37.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83f77fe700 1 -- 172.21.15.170:0/3527192649 learned_addr learned my addr 172.21.15.170:0/3527192649 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83f7fff700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83e0003fe0 0x7f83f816b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.921+0000 7f83fcf8f700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83f80a62b0 0x7f83f815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 -- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83e0003fe0 msgr2=0x7f83f816b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83e0003fe0 0x7f83f816b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 -- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83f80a62b0 msgr2=0x7f83f815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83f80a62b0 0x7f83f815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 -- 172.21.15.170:0/3527192649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8158c70 con 0x7f83f80a5e40 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83f77fe700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0091e0 tx=0x7f83ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec022070 con 0x7f83f80a5e40 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00e980 con 0x7f83f80a5e40 2023-10-08T16:10:37.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f8158f50 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00e6b0 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.922+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f8159510 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.923+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f804c3c0 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.923+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f83ec01c450 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.924+0000 7f83dffff700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 0x7f83e4069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.924+0000 7f83f7fff700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 0x7f83e4069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.924+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83ec086310 con 0x7f83f80a5e40 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.924+0000 7f83f7fff700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 0x7f83e4069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000e50 tx=0x7f83e8001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:37.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:37.929+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83ec053220 con 0x7f83f80a5e40 2023-10-08T16:10:38.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.151+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83f80cb5e0 con 0x7f83f80a5e40 2023-10-08T16:10:38.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.152+0000 7f83dffff700 1 -- 172.21.15.170:0/3527192649 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f83ec056840 con 0x7f83f80a5e40 2023-10-08T16:10:38.153 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:38.154 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 msgr2=0x7f83e4069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 0x7f83e4069740 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000e50 tx=0x7f83e8001e90 comp rx=0 tx=0).stop 2023-10-08T16:10:38.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 msgr2=0x7f83f815c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0091e0 tx=0x7f83ec01a000 comp rx=0 tx=0).stop 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 shutdown_connections 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83f80a62b0 0x7f83f815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83e0003fe0 0x7f83f816b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83e4067290 0x7f83e4069740 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 --2- 172.21.15.170:0/3527192649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83f80a5e40 0x7f83f815c780 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 >> 172.21.15.170:0/3527192649 conn(0x7f83f806a320 msgr2=0x7f83f80acfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 shutdown_connections 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.154+0000 7f83fe9f2700 1 -- 172.21.15.170:0/3527192649 wait complete. 2023-10-08T16:10:38.159 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 65 2023-10-08T16:10:38.168 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:38.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.282+0000 7fb498e32700 1 Processor -- start 2023-10-08T16:10:38.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.282+0000 7fb498e32700 1 -- start start 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb494159420 0x7fb4940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb4940b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb49415d230 0x7fb49415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49415dbd0 con 0x7fb494159420 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4940b11d0 con 0x7fb49415d230 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb498e32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940b14b0 con 0x7fb4941597f0 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb494159420 0x7fb4940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb494159420 0x7fb4940b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37960/0 (socket says 172.21.15.170:37960) 2023-10-08T16:10:38.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 -- 172.21.15.170:0/290431676 learned_addr learned my addr 172.21.15.170:0/290431676 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb493fff700 1 --2- 172.21.15.170:0/290431676 >> v2:172.21.15.191:3300/0 conn(0x7fb49415d230 0x7fb49415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb492ffd700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb4940b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 -- 172.21.15.170:0/290431676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 msgr2=0x7fb4940b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb4940b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 -- 172.21.15.170:0/290431676 >> v2:172.21.15.191:3300/0 conn(0x7fb49415d230 msgr2=0x7fb49415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 --2- 172.21.15.170:0/290431676 >> v2:172.21.15.191:3300/0 conn(0x7fb49415d230 0x7fb49415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 -- 172.21.15.170:0/290431676 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940b1790 con 0x7fb494159420 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4937fe700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb494159420 0x7fb4940b5f40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d1720 tx=0x7fb484009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9642d796a3a6baad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4927fc700 1 -- 172.21.15.170:0/290431676 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400e070 con 0x7fb494159420 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.283+0000 7fb4927fc700 1 -- 172.21.15.170:0/290431676 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400ba50 con 0x7fb494159420 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb4927fc700 1 -- 172.21.15.170:0/290431676 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840092e0 con 0x7fb494159420 2023-10-08T16:10:38.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 -- 172.21.15.170:0/290431676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb494159420 msgr2=0x7fb4940b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb494159420 0x7fb4940b5f40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fb4940d1720 tx=0x7fb484009840 comp rx=0 tx=0).stop 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 -- 172.21.15.170:0/290431676 shutdown_connections 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 --2- 172.21.15.170:0/290431676 >> v2:172.21.15.191:3300/0 conn(0x7fb49415d230 0x7fb49415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb494159420 0x7fb4940b5f40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 --2- 172.21.15.170:0/290431676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb4940b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 -- 172.21.15.170:0/290431676 >> 172.21.15.170:0/290431676 conn(0x7fb4940d0660 msgr2=0x7fb4940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 -- 172.21.15.170:0/290431676 shutdown_connections 2023-10-08T16:10:38.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.284+0000 7fb498e32700 1 -- 172.21.15.170:0/290431676 wait complete. 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.285+0000 7fb498e32700 1 Processor -- start 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.285+0000 7fb498e32700 1 -- start start 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb494159420 0x7fb49414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb494156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49414a880 con 0x7fb49415d230 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c9c10 con 0x7fb494159420 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb498e32700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940c6930 con 0x7fb4941597f0 2023-10-08T16:10:38.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37964/0 (socket says 172.21.15.170:37964) 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 -- 172.21.15.170:0/3357741608 learned_addr learned my addr 172.21.15.170:0/3357741608 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb492ffd700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb494156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb4937fe700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb494159420 0x7fb49414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 -- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 msgr2=0x7fb494156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb494156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 -- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb494159420 msgr2=0x7fb49414a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb494159420 0x7fb49414a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.286+0000 7fb493fff700 1 -- 172.21.15.170:0/3357741608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009ae0 con 0x7fb49415d230 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb493fff700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d860 tx=0x7fb48c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c0125f0 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48c012790 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4940c6c10 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48c012b50 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.287+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4940c71d0 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.288+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49404c3c0 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.288+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fb48c013480 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.288+0000 7fb47b7fe700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 0x7fb480069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.289+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb48c084d40 con 0x7fb49415d230 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.289+0000 7fb4937fe700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 0x7fb480069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.289+0000 7fb4937fe700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 0x7fb480069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb494158060 tx=0x7fb484018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.294+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb48c051ba0 con 0x7fb49415d230 2023-10-08T16:10:38.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.477+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4940b1a70 con 0x7fb49415d230 2023-10-08T16:10:38.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.478+0000 7fb47b7fe700 1 -- 172.21.15.170:0/3357741608 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7fb48c0551c0 con 0x7fb49415d230 2023-10-08T16:10:38.479 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:38.479 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:35.749539+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 msgr2=0x7fb480069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 0x7fb480069740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb494158060 tx=0x7fb484018000 comp rx=0 tx=0).stop 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 msgr2=0x7fb494157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00d860 tx=0x7fb48c008040 comp rx=0 tx=0).stop 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 shutdown_connections 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb480067290 0x7fb480069740 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49415d230 0x7fb494157110 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb494159420 0x7fb49414a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 --2- 172.21.15.170:0/3357741608 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4941597f0 0x7fb494156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 >> 172.21.15.170:0/3357741608 conn(0x7fb4940d0660 msgr2=0x7fb4940b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 shutdown_connections 2023-10-08T16:10:38.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.480+0000 7fb498e32700 1 -- 172.21.15.170:0/3357741608 wait complete. 2023-10-08T16:10:38.493 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:10:38.494 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:38.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.607+0000 7f8005cf5700 1 Processor -- start 2023-10-08T16:10:38.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.607+0000 7f8005cf5700 1 -- start start 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8005cf5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8000159420 0x7f80000b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8005cf5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f800015d230 0x7f800015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8005cf5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80000b2980 con 0x7f80001597f0 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8005cf5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800015db70 con 0x7f800015d230 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8005cf5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000b11d0 con 0x7f8000159420 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f80054f4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f800015d230 0x7f800015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f80054f4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f800015d230 0x7f800015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37972/0 (socket says 172.21.15.170:37972) 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f80054f4700 1 -- 172.21.15.170:0/3331461269 learned_addr learned my addr 172.21.15.170:0/3331461269 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f8004cf3700 1 --2- 172.21.15.170:0/3331461269 >> v2:172.21.15.191:3300/0 conn(0x7f8000159420 0x7f80000b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.608+0000 7f80054f4700 1 -- 172.21.15.170:0/3331461269 >> v2:172.21.15.191:3300/0 conn(0x7f8000159420 msgr2=0x7f80000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f80054f4700 1 --2- 172.21.15.170:0/3331461269 >> v2:172.21.15.191:3300/0 conn(0x7f8000159420 0x7f80000b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f80054f4700 1 -- 172.21.15.170:0/3331461269 >> v1:172.21.15.191:6789/0 conn(0x7f80001597f0 legacy=0x7f80000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f80054f4700 1 -- 172.21.15.170:0/3331461269 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000b14b0 con 0x7f800015d230 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f80054f4700 1 --2- 172.21.15.170:0/3331461269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800015d230 0x7f800015d630 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f80000b2f20 tx=0x7f7ff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac74ce419f5329d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f7fff7fe700 1 -- 172.21.15.170:0/3331461269 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800b860 con 0x7f800015d230 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f7fff7fe700 1 -- 172.21.15.170:0/3331461269 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff800b9c0 con 0x7f800015d230 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f7fff7fe700 1 -- 172.21.15.170:0/3331461269 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800bd60 con 0x7f800015d230 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 -- 172.21.15.170:0/3331461269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800015d230 msgr2=0x7f800015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 --2- 172.21.15.170:0/3331461269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800015d230 0x7f800015d630 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f80000b2f20 tx=0x7f7ff8009250 comp rx=0 tx=0).stop 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 -- 172.21.15.170:0/3331461269 shutdown_connections 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 --2- 172.21.15.170:0/3331461269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f800015d230 0x7f800015d630 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 --2- 172.21.15.170:0/3331461269 >> v2:172.21.15.191:3300/0 conn(0x7f8000159420 0x7f80000b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 -- 172.21.15.170:0/3331461269 >> 172.21.15.170:0/3331461269 conn(0x7f80000d0660 msgr2=0x7f80000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 -- 172.21.15.170:0/3331461269 shutdown_connections 2023-10-08T16:10:38.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.609+0000 7f8005cf5700 1 -- 172.21.15.170:0/3331461269 wait complete. 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.610+0000 7f8005cf5700 1 Processor -- start 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 -- start start 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8000159420 0x7f8000155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f800015d230 0x7f80000c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80000c9740 con 0x7f80001597f0 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80000c5400 con 0x7f800015d230 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.611+0000 7f8005cf5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80000c5650 con 0x7f8000159420 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37984/0 (socket says 172.21.15.170:37984) 2023-10-08T16:10:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f8004cf3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8000159420 0x7f8000155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 -- 172.21.15.170:0/217752463 learned_addr learned my addr 172.21.15.170:0/217752463 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f80054f4700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f800015d230 0x7f80000c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 -- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8000159420 msgr2=0x7f8000155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8000159420 0x7f8000155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 -- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f800015d230 msgr2=0x7f80000c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f800015d230 0x7f80000c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 -- 172.21.15.170:0/217752463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80094f0 con 0x7f80001597f0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffffff700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c7c0 tx=0x7f7ff4007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011590 con 0x7f80001597f0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff40116f0 con 0x7f80001597f0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80000c5930 con 0x7f80001597f0 2023-10-08T16:10:38.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011ab0 con 0x7f80001597f0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.612+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80000c5ef0 con 0x7f80001597f0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.613+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800004c3c0 con 0x7f80001597f0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.614+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7ff401b3b0 con 0x7f80001597f0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.614+0000 7f7ffcff9700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 0x7f7fec069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.614+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ff4083e40 con 0x7f80001597f0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.614+0000 7f8004cf3700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 0x7f7fec069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.614+0000 7f8004cf3700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 0x7f7fec069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009fd0 tx=0x7f7ff0009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.618+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ff4050ca0 con 0x7f80001597f0 2023-10-08T16:10:38.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.796+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80000b1790 con 0x7f7fec067290 2023-10-08T16:10:38.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.804+0000 7f7ffcff9700 1 -- 172.21.15.170:0/217752463 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286548 (secure 0 0 0) 0x7f80000b1790 con 0x7f7fec067290 2023-10-08T16:10:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 msgr2=0x7f7fec069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 0x7f7fec069740 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009fd0 tx=0x7f7ff0009880 comp rx=0 tx=0).stop 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 msgr2=0x7f80000c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c7c0 tx=0x7f7ff4007f50 comp rx=0 tx=0).stop 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 shutdown_connections 2023-10-08T16:10:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f800015d230 0x7f80000c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8000159420 0x7f8000155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7fec067290 0x7f7fec069740 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 --2- 172.21.15.170:0/217752463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f80001597f0 0x7f80000c66f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.807+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 >> 172.21.15.170:0/217752463 conn(0x7f80000d0660 msgr2=0x7f80001586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.808+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 shutdown_connections 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.808+0000 7f8005cf5700 1 -- 172.21.15.170:0/217752463 wait complete. 2023-10-08T16:10:38.811 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:38.820 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-10-08T16:10:38.298588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405480,"kb_used_data":10792,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752569240,"statfs":{"total":773094113280,"available":770630901760,"internally_reserved":0,"allocated":11051008,"data_stored":5417592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555417"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.764209+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:04.752369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:10.607412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757132+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758755+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:19.486276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755308+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:00.816633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751803+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758103+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756169+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761263+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:45:38.964981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761247+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:07.766197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751846+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759204+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:05.459867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756405+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755366+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751806+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757139+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752896+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761185+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756681+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752689+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:45.260005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754178+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753975+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751934+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754842+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752234+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:35.450350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760754+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:42.324138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752524+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753403+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753733+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757821+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755472+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752055+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751912+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766666+0000","last_change":"2023-10-08T16:10:33.766666+0000","last_active":"2023-10-08T16:10:33.766666+0000","last_peered":"2023-10-08T16:10:33.766666+0000","last_clean":"2023-10-08T16:10:33.766666+0000","last_became_active":"2023-10-08T16:10:33.764988+0000","last_became_peered":"2023-10-08T16:10:33.764988+0000","last_unstale":"2023-10-08T16:10:33.766666+0000","last_undegraded":"2023-10-08T16:10:33.766666+0000","last_fullsized":"2023-10-08T16:10:33.766666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:49.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759961+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765370+0000","last_change":"2023-10-08T16:10:33.765370+0000","last_active":"2023-10-08T16:10:33.765370+0000","last_peered":"2023-10-08T16:10:33.765370+0000","last_clean":"2023-10-08T16:10:33.765370+0000","last_became_active":"2023-10-08T16:10:33.764896+0000","last_became_peered":"2023-10-08T16:10:33.764896+0000","last_unstale":"2023-10-08T16:10:33.765370+0000","last_undegraded":"2023-10-08T16:10:33.765370+0000","last_fullsized":"2023-10-08T16:10:33.765370+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751725+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.763286+0000","last_change":"2023-10-08T16:10:33.763286+0000","last_active":"2023-10-08T16:10:33.763286+0000","last_peered":"2023-10-08T16:10:33.763286+0000","last_clean":"2023-10-08T16:10:33.763286+0000","last_became_active":"2023-10-08T16:10:33.763028+0000","last_became_peered":"2023-10-08T16:10:33.763028+0000","last_unstale":"2023-10-08T16:10:33.763286+0000","last_undegraded":"2023-10-08T16:10:33.763286+0000","last_fullsized":"2023-10-08T16:10:33.763286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:03.701535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756126+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767700+0000","last_change":"2023-10-08T16:10:33.767700+0000","last_active":"2023-10-08T16:10:33.767700+0000","last_peered":"2023-10-08T16:10:33.767700+0000","last_clean":"2023-10-08T16:10:33.767700+0000","last_became_active":"2023-10-08T16:10:33.766926+0000","last_became_peered":"2023-10-08T16:10:33.766926+0000","last_unstale":"2023-10-08T16:10:33.767700+0000","last_undegraded":"2023-10-08T16:10:33.767700+0000","last_fullsized":"2023-10-08T16:10:33.767700+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:01.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757686+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760988+0000","last_change":"2023-10-08T16:10:33.760988+0000","last_active":"2023-10-08T16:10:33.760988+0000","last_peered":"2023-10-08T16:10:33.760988+0000","last_clean":"2023-10-08T16:10:33.760988+0000","last_became_active":"2023-10-08T16:10:33.760705+0000","last_became_peered":"2023-10-08T16:10:33.760705+0000","last_unstale":"2023-10-08T16:10:33.760988+0000","last_undegraded":"2023-10-08T16:10:33.760988+0000","last_fullsized":"2023-10-08T16:10:33.760988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753230+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765098+0000","last_change":"2023-10-08T16:10:33.765098+0000","last_active":"2023-10-08T16:10:33.765098+0000","last_peered":"2023-10-08T16:10:33.765098+0000","last_clean":"2023-10-08T16:10:33.765098+0000","last_became_active":"2023-10-08T16:10:33.764637+0000","last_became_peered":"2023-10-08T16:10:33.764637+0000","last_unstale":"2023-10-08T16:10:33.765098+0000","last_undegraded":"2023-10-08T16:10:33.765098+0000","last_fullsized":"2023-10-08T16:10:33.765098+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756417+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761688+0000","last_change":"2023-10-08T16:10:33.761688+0000","last_active":"2023-10-08T16:10:33.761688+0000","last_peered":"2023-10-08T16:10:33.761688+0000","last_clean":"2023-10-08T16:10:33.761688+0000","last_became_active":"2023-10-08T16:10:33.761196+0000","last_became_peered":"2023-10-08T16:10:33.761196+0000","last_unstale":"2023-10-08T16:10:33.761688+0000","last_undegraded":"2023-10-08T16:10:33.761688+0000","last_fullsized":"2023-10-08T16:10:33.761688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:27.197085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757514+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:53.047014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770625+0000","last_change":"2023-10-08T16:10:33.770625+0000","last_active":"2023-10-08T16:10:33.770625+0000","last_peered":"2023-10-08T16:10:33.770625+0000","last_clean":"2023-10-08T16:10:33.770625+0000","last_became_active":"2023-10-08T16:10:33.770366+0000","last_became_peered":"2023-10-08T16:10:33.770366+0000","last_unstale":"2023-10-08T16:10:33.770625+0000","last_undegraded":"2023-10-08T16:10:33.770625+0000","last_fullsized":"2023-10-08T16:10:33.770625+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:29.471251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757179+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:59.123802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767567+0000","last_change":"2023-10-08T16:10:33.767567+0000","last_active":"2023-10-08T16:10:33.767567+0000","last_peered":"2023-10-08T16:10:33.767567+0000","last_clean":"2023-10-08T16:10:33.767567+0000","last_became_active":"2023-10-08T16:10:33.767321+0000","last_became_peered":"2023-10-08T16:10:33.767321+0000","last_unstale":"2023-10-08T16:10:33.767567+0000","last_undegraded":"2023-10-08T16:10:33.767567+0000","last_fullsized":"2023-10-08T16:10:33.767567+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:13:23.776656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757333+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764988+0000","last_change":"2023-10-08T16:10:33.764988+0000","last_active":"2023-10-08T16:10:33.764988+0000","last_peered":"2023-10-08T16:10:33.764988+0000","last_clean":"2023-10-08T16:10:33.764988+0000","last_became_active":"2023-10-08T16:10:33.764739+0000","last_became_peered":"2023-10-08T16:10:33.764739+0000","last_unstale":"2023-10-08T16:10:33.764988+0000","last_undegraded":"2023-10-08T16:10:33.764988+0000","last_fullsized":"2023-10-08T16:10:33.764988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751897+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768462+0000","last_change":"2023-10-08T16:10:33.768462+0000","last_active":"2023-10-08T16:10:33.768462+0000","last_peered":"2023-10-08T16:10:33.768462+0000","last_clean":"2023-10-08T16:10:33.768462+0000","last_became_active":"2023-10-08T16:10:33.768242+0000","last_became_peered":"2023-10-08T16:10:33.768242+0000","last_unstale":"2023-10-08T16:10:33.768462+0000","last_undegraded":"2023-10-08T16:10:33.768462+0000","last_fullsized":"2023-10-08T16:10:33.768462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:59.803650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758164+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773222+0000","last_change":"2023-10-08T16:10:33.773222+0000","last_active":"2023-10-08T16:10:33.773222+0000","last_peered":"2023-10-08T16:10:33.773222+0000","last_clean":"2023-10-08T16:10:33.773222+0000","last_became_active":"2023-10-08T16:10:33.772922+0000","last_became_peered":"2023-10-08T16:10:33.772922+0000","last_unstale":"2023-10-08T16:10:33.773222+0000","last_undegraded":"2023-10-08T16:10:33.773222+0000","last_fullsized":"2023-10-08T16:10:33.773222+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:07.790273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753477+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762563+0000","last_change":"2023-10-08T16:10:33.762563+0000","last_active":"2023-10-08T16:10:33.762563+0000","last_peered":"2023-10-08T16:10:33.762563+0000","last_clean":"2023-10-08T16:10:33.762563+0000","last_became_active":"2023-10-08T16:10:33.762298+0000","last_became_peered":"2023-10-08T16:10:33.762298+0000","last_unstale":"2023-10-08T16:10:33.762563+0000","last_undegraded":"2023-10-08T16:10:33.762563+0000","last_fullsized":"2023-10-08T16:10:33.762563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:13:51.956683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766497+0000","last_change":"2023-10-08T16:10:33.766497+0000","last_active":"2023-10-08T16:10:33.766497+0000","last_peered":"2023-10-08T16:10:33.766497+0000","last_clean":"2023-10-08T16:10:33.766497+0000","last_became_active":"2023-10-08T16:10:33.766186+0000","last_became_peered":"2023-10-08T16:10:33.766186+0000","last_unstale":"2023-10-08T16:10:33.766497+0000","last_undegraded":"2023-10-08T16:10:33.766497+0000","last_fullsized":"2023-10-08T16:10:33.766497+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:02.257047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752088+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770711+0000","last_change":"2023-10-08T16:10:33.770711+0000","last_active":"2023-10-08T16:10:33.770711+0000","last_peered":"2023-10-08T16:10:33.770711+0000","last_clean":"2023-10-08T16:10:33.770711+0000","last_became_active":"2023-10-08T16:10:33.770495+0000","last_became_peered":"2023-10-08T16:10:33.770495+0000","last_unstale":"2023-10-08T16:10:33.770711+0000","last_undegraded":"2023-10-08T16:10:33.770711+0000","last_fullsized":"2023-10-08T16:10:33.770711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:50.671846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755500+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:12.462984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773350+0000","last_change":"2023-10-08T16:10:33.773350+0000","last_active":"2023-10-08T16:10:33.773350+0000","last_peered":"2023-10-08T16:10:33.773350+0000","last_clean":"2023-10-08T16:10:33.773350+0000","last_became_active":"2023-10-08T16:10:33.772989+0000","last_became_peered":"2023-10-08T16:10:33.772989+0000","last_unstale":"2023-10-08T16:10:33.773350+0000","last_undegraded":"2023-10-08T16:10:33.773350+0000","last_fullsized":"2023-10-08T16:10:33.773350+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:18.004513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751707+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760706+0000","last_change":"2023-10-08T16:10:33.760706+0000","last_active":"2023-10-08T16:10:33.760706+0000","last_peered":"2023-10-08T16:10:33.760706+0000","last_clean":"2023-10-08T16:10:33.760706+0000","last_became_active":"2023-10-08T16:10:33.760439+0000","last_became_peered":"2023-10-08T16:10:33.760439+0000","last_unstale":"2023-10-08T16:10:33.760706+0000","last_undegraded":"2023-10-08T16:10:33.760706+0000","last_fullsized":"2023-10-08T16:10:33.760706+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:34.341551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754402+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764394+0000","last_change":"2023-10-08T16:10:33.764394+0000","last_active":"2023-10-08T16:10:33.764394+0000","last_peered":"2023-10-08T16:10:33.764394+0000","last_clean":"2023-10-08T16:10:33.764394+0000","last_became_active":"2023-10-08T16:10:33.764077+0000","last_became_peered":"2023-10-08T16:10:33.764077+0000","last_unstale":"2023-10-08T16:10:33.764394+0000","last_undegraded":"2023-10-08T16:10:33.764394+0000","last_fullsized":"2023-10-08T16:10:33.764394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:27.558248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754255+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773846+0000","last_change":"2023-10-08T16:10:33.773846+0000","last_active":"2023-10-08T16:10:33.773846+0000","last_peered":"2023-10-08T16:10:33.773846+0000","last_clean":"2023-10-08T16:10:33.773846+0000","last_became_active":"2023-10-08T16:10:33.773563+0000","last_became_peered":"2023-10-08T16:10:33.773563+0000","last_unstale":"2023-10-08T16:10:33.773846+0000","last_undegraded":"2023-10-08T16:10:33.773846+0000","last_fullsized":"2023-10-08T16:10:33.773846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:58:45.919412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754208+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.759531+0000","last_change":"2023-10-08T16:10:33.759531+0000","last_active":"2023-10-08T16:10:33.759531+0000","last_peered":"2023-10-08T16:10:33.759531+0000","last_clean":"2023-10-08T16:10:33.759531+0000","last_became_active":"2023-10-08T16:10:33.759312+0000","last_became_peered":"2023-10-08T16:10:33.759312+0000","last_unstale":"2023-10-08T16:10:33.759531+0000","last_undegraded":"2023-10-08T16:10:33.759531+0000","last_fullsized":"2023-10-08T16:10:33.759531+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758276+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:23.087283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768853+0000","last_change":"2023-10-08T16:10:33.768853+0000","last_active":"2023-10-08T16:10:33.768853+0000","last_peered":"2023-10-08T16:10:33.768853+0000","last_clean":"2023-10-08T16:10:33.768853+0000","last_became_active":"2023-10-08T16:10:33.768563+0000","last_became_peered":"2023-10-08T16:10:33.768563+0000","last_unstale":"2023-10-08T16:10:33.768853+0000","last_undegraded":"2023-10-08T16:10:33.768853+0000","last_fullsized":"2023-10-08T16:10:33.768853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:57.969527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755709+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761930+0000","last_change":"2023-10-08T16:10:33.761930+0000","last_active":"2023-10-08T16:10:33.761930+0000","last_peered":"2023-10-08T16:10:33.761930+0000","last_clean":"2023-10-08T16:10:33.761930+0000","last_became_active":"2023-10-08T16:10:33.761680+0000","last_became_peered":"2023-10-08T16:10:33.761680+0000","last_unstale":"2023-10-08T16:10:33.761930+0000","last_undegraded":"2023-10-08T16:10:33.761930+0000","last_fullsized":"2023-10-08T16:10:33.761930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751469+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766371+0000","last_change":"2023-10-08T16:10:33.766371+0000","last_active":"2023-10-08T16:10:33.766371+0000","last_peered":"2023-10-08T16:10:33.766371+0000","last_clean":"2023-10-08T16:10:33.766371+0000","last_became_active":"2023-10-08T16:10:33.766082+0000","last_became_peered":"2023-10-08T16:10:33.766082+0000","last_unstale":"2023-10-08T16:10:33.766371+0000","last_undegraded":"2023-10-08T16:10:33.766371+0000","last_fullsized":"2023-10-08T16:10:33.766371+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:58.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760703+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770919+0000","last_change":"2023-10-08T16:10:33.770919+0000","last_active":"2023-10-08T16:10:33.770919+0000","last_peered":"2023-10-08T16:10:33.770919+0000","last_clean":"2023-10-08T16:10:33.770919+0000","last_became_active":"2023-10-08T16:10:33.770608+0000","last_became_peered":"2023-10-08T16:10:33.770608+0000","last_unstale":"2023-10-08T16:10:33.770919+0000","last_undegraded":"2023-10-08T16:10:33.770919+0000","last_fullsized":"2023-10-08T16:10:33.770919+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:37.099695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.763791+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:15.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766192+0000","last_change":"2023-10-08T16:10:33.766192+0000","last_active":"2023-10-08T16:10:33.766192+0000","last_peered":"2023-10-08T16:10:33.766192+0000","last_clean":"2023-10-08T16:10:33.766192+0000","last_became_active":"2023-10-08T16:10:33.765245+0000","last_became_peered":"2023-10-08T16:10:33.765245+0000","last_unstale":"2023-10-08T16:10:33.766192+0000","last_undegraded":"2023-10-08T16:10:33.766192+0000","last_fullsized":"2023-10-08T16:10:33.766192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:49.157336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759697+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:19:27.448117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766948+0000","last_change":"2023-10-08T16:10:33.766948+0000","last_active":"2023-10-08T16:10:33.766948+0000","last_peered":"2023-10-08T16:10:33.766948+0000","last_clean":"2023-10-08T16:10:33.766948+0000","last_became_active":"2023-10-08T16:10:33.766644+0000","last_became_peered":"2023-10-08T16:10:33.766644+0000","last_unstale":"2023-10-08T16:10:33.766948+0000","last_undegraded":"2023-10-08T16:10:33.766948+0000","last_fullsized":"2023-10-08T16:10:33.766948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:54.343260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756582+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762562+0000","last_change":"2023-10-08T16:10:33.762562+0000","last_active":"2023-10-08T16:10:33.762562+0000","last_peered":"2023-10-08T16:10:33.762562+0000","last_clean":"2023-10-08T16:10:33.762562+0000","last_became_active":"2023-10-08T16:10:33.762261+0000","last_became_peered":"2023-10-08T16:10:33.762261+0000","last_unstale":"2023-10-08T16:10:33.762562+0000","last_undegraded":"2023-10-08T16:10:33.762562+0000","last_fullsized":"2023-10-08T16:10:33.762562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:18.712472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752101+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.103975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765165+0000","last_change":"2023-10-08T16:10:33.765165+0000","last_active":"2023-10-08T16:10:33.765165+0000","last_peered":"2023-10-08T16:10:33.765165+0000","last_clean":"2023-10-08T16:10:33.765165+0000","last_became_active":"2023-10-08T16:10:33.764738+0000","last_became_peered":"2023-10-08T16:10:33.764738+0000","last_unstale":"2023-10-08T16:10:33.765165+0000","last_undegraded":"2023-10-08T16:10:33.765165+0000","last_fullsized":"2023-10-08T16:10:33.765165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751869+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.755373+0000","last_change":"2023-10-08T16:10:33.755373+0000","last_active":"2023-10-08T16:10:33.755373+0000","last_peered":"2023-10-08T16:10:33.755373+0000","last_clean":"2023-10-08T16:10:33.755373+0000","last_became_active":"2023-10-08T16:10:33.755019+0000","last_became_peered":"2023-10-08T16:10:33.755019+0000","last_unstale":"2023-10-08T16:10:33.755373+0000","last_undegraded":"2023-10-08T16:10:33.755373+0000","last_fullsized":"2023-10-08T16:10:33.755373+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:32.618356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755200+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767441+0000","last_change":"2023-10-08T16:10:33.767441+0000","last_active":"2023-10-08T16:10:33.767441+0000","last_peered":"2023-10-08T16:10:33.767441+0000","last_clean":"2023-10-08T16:10:33.767441+0000","last_became_active":"2023-10-08T16:10:33.767184+0000","last_became_peered":"2023-10-08T16:10:33.767184+0000","last_unstale":"2023-10-08T16:10:33.767441+0000","last_undegraded":"2023-10-08T16:10:33.767441+0000","last_fullsized":"2023-10-08T16:10:33.767441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:44.856255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754992+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766449+0000","last_change":"2023-10-08T16:10:33.766449+0000","last_active":"2023-10-08T16:10:33.766449+0000","last_peered":"2023-10-08T16:10:33.766449+0000","last_clean":"2023-10-08T16:10:33.766449+0000","last_became_active":"2023-10-08T16:10:33.766244+0000","last_became_peered":"2023-10-08T16:10:33.766244+0000","last_unstale":"2023-10-08T16:10:33.766449+0000","last_undegraded":"2023-10-08T16:10:33.766449+0000","last_fullsized":"2023-10-08T16:10:33.766449+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:02.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752142+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773844+0000","last_change":"2023-10-08T16:10:33.773844+0000","last_active":"2023-10-08T16:10:33.773844+0000","last_peered":"2023-10-08T16:10:33.773844+0000","last_clean":"2023-10-08T16:10:33.773844+0000","last_became_active":"2023-10-08T16:10:33.773481+0000","last_became_peered":"2023-10-08T16:10:33.773481+0000","last_unstale":"2023-10-08T16:10:33.773844+0000","last_undegraded":"2023-10-08T16:10:33.773844+0000","last_fullsized":"2023-10-08T16:10:33.773844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:41.626242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-08T16:10:32.749980+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:32.749980+0000","last_peered":"2023-10-08T16:10:32.749980+0000","last_clean":"2023-10-08T16:10:32.749980+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:32.749980+0000","last_undegraded":"2023-10-08T16:10:32.749980+0000","last_fullsized":"2023-10-08T16:10:32.749980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.57299999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.53600000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67200000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.748}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.60699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.626},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303168,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068672,"statfs":{"total":96636764160,"available":96326320128,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.314},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.252},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.44}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.46800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.624}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.432},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:38.826 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:38.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.974+0000 7fac1dd37700 1 Processor -- start 2023-10-08T16:10:38.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.974+0000 7fac1dd37700 1 -- start start 2023-10-08T16:10:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac1dd37700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fac18159890 0x7fac1815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac1dd37700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac1dd37700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac18069ee0 con 0x7fac18159c60 2023-10-08T16:10:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac1dd37700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1815a160 con 0x7fac18159890 2023-10-08T16:10:38.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac1dd37700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac180d13a0 con 0x7fac18069560 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fac18159890 0x7fac1815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fac18159890 0x7fac1815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:37996/0 (socket says 172.21.15.170:37996) 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac16ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fac18159c60 0x7fac1815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33664/0 (socket says 172.21.15.170:33664) 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 -- 172.21.15.170:0/840126449 learned_addr learned my addr 172.21.15.170:0/840126449 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac17fff700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 -- 172.21.15.170:0/840126449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 msgr2=0x7fac180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac177fe700 1 -- 172.21.15.170:0/840126449 >> v1:172.21.15.170:6789/0 conn(0x7fac18159c60 legacy=0x7fac1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac177fe700 1 -- 172.21.15.170:0/840126449 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1815e140 con 0x7fac18159890 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.975+0000 7fac167fc700 1 -- 172.21.15.170:0/840126449 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140595343 0 0) 0x7fac18069ee0 con 0x7fac18159c60 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac177fe700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159890 0x7fac1815abd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fac08009f20 tx=0x7fac08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86015ce36eef9943 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac167fc700 1 -- 172.21.15.170:0/840126449 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800e070 con 0x7fac18159890 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac167fc700 1 -- 172.21.15.170:0/840126449 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800ba80 con 0x7fac18159890 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac167fc700 1 -- 172.21.15.170:0/840126449 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac080092e0 con 0x7fac18159890 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 -- 172.21.15.170:0/840126449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159890 msgr2=0x7fac1815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159890 0x7fac1815abd0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fac08009f20 tx=0x7fac08009840 comp rx=0 tx=0).stop 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 -- 172.21.15.170:0/840126449 shutdown_connections 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac180699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 --2- 172.21.15.170:0/840126449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159890 0x7fac1815abd0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 -- 172.21.15.170:0/840126449 >> 172.21.15.170:0/840126449 conn(0x7fac18068e20 msgr2=0x7fac180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 -- 172.21.15.170:0/840126449 shutdown_connections 2023-10-08T16:10:38.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.976+0000 7fac1dd37700 1 -- 172.21.15.170:0/840126449 wait complete. 2023-10-08T16:10:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 Processor -- start 2023-10-08T16:10:38.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 -- start start 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac1814ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 0x7fac1816d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1816dee0 con 0x7fac18159c60 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac180b3730 con 0x7fac18159890 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac1dd37700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac180b3a10 con 0x7fac18069560 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac17fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac17fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38004/0 (socket says 172.21.15.170:38004) 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.978+0000 7fac17fff700 1 -- 172.21.15.170:0/2110876145 learned_addr learned my addr 172.21.15.170:0/2110876145 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac177fe700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac1814ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac16ffd700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 0x7fac1816d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 -- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 msgr2=0x7fac1814ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac1814ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 -- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 msgr2=0x7fac1816d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 0x7fac1816d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 -- 172.21.15.170:0/2110876145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08009ae0 con 0x7fac18159c60 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac16ffd700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 0x7fac1816d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac17fff700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d860 tx=0x7fac10008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100125f0 con 0x7fac18159c60 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10012750 con 0x7fac18159c60 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac180cd1e0 con 0x7fac18159c60 2023-10-08T16:10:38.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10012b10 con 0x7fac18159c60 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.979+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac180cfb90 con 0x7fac18159c60 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.980+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1804c3c0 con 0x7fac18159c60 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.980+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fac1001e460 con 0x7fac18159c60 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.981+0000 7fabff7fe700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 0x7fac04069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.981+0000 7fac177fe700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 0x7fac04069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.981+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fac100850d0 con 0x7fac18159c60 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.981+0000 7fac177fe700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 0x7fac04069740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac08009880 tx=0x7fac08018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:38.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:38.985+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac100559c0 con 0x7fac18159c60 2023-10-08T16:10:39.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.164+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac180cf970 con 0x7fac04067290 2023-10-08T16:10:39.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.172+0000 7fabff7fe700 1 -- 172.21.15.170:0/2110876145 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286548 (secure 0 0 0) 0x7fac180cf970 con 0x7fac04067290 2023-10-08T16:10:39.176 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 msgr2=0x7fac04069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 0x7fac04069740 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac08009880 tx=0x7fac08018000 comp rx=0 tx=0).stop 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 msgr2=0x7fac180cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d860 tx=0x7fac10008040 comp rx=0 tx=0).stop 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 shutdown_connections 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fac04067290 0x7fac04069740 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac18159c60 0x7fac180cedd0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fac18069560 0x7fac1814ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 --2- 172.21.15.170:0/2110876145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fac18159890 0x7fac1816d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 >> 172.21.15.170:0/2110876145 conn(0x7fac18068e20 msgr2=0x7fac180b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.175+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 shutdown_connections 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.176+0000 7fac1dd37700 1 -- 172.21.15.170:0/2110876145 wait complete. 2023-10-08T16:10:39.179 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:39.188 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-10-08T16:10:38.298588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405480,"kb_used_data":10792,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752569240,"statfs":{"total":773094113280,"available":770630901760,"internally_reserved":0,"allocated":11051008,"data_stored":5417592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555417"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.764209+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:04.752369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:10.607412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757132+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758755+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:19.486276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755308+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:00.816633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751803+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758103+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756169+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761263+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:45:38.964981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761247+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:07.766197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751846+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759204+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:05.459867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756405+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755366+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751806+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757139+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752896+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761185+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756681+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752689+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:45.260005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754178+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753975+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751934+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754842+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752234+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:35.450350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760754+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:42.324138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752524+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753403+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753733+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757821+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755472+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752055+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751912+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766666+0000","last_change":"2023-10-08T16:10:33.766666+0000","last_active":"2023-10-08T16:10:33.766666+0000","last_peered":"2023-10-08T16:10:33.766666+0000","last_clean":"2023-10-08T16:10:33.766666+0000","last_became_active":"2023-10-08T16:10:33.764988+0000","last_became_peered":"2023-10-08T16:10:33.764988+0000","last_unstale":"2023-10-08T16:10:33.766666+0000","last_undegraded":"2023-10-08T16:10:33.766666+0000","last_fullsized":"2023-10-08T16:10:33.766666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:49.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759961+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765370+0000","last_change":"2023-10-08T16:10:33.765370+0000","last_active":"2023-10-08T16:10:33.765370+0000","last_peered":"2023-10-08T16:10:33.765370+0000","last_clean":"2023-10-08T16:10:33.765370+0000","last_became_active":"2023-10-08T16:10:33.764896+0000","last_became_peered":"2023-10-08T16:10:33.764896+0000","last_unstale":"2023-10-08T16:10:33.765370+0000","last_undegraded":"2023-10-08T16:10:33.765370+0000","last_fullsized":"2023-10-08T16:10:33.765370+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751725+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.763286+0000","last_change":"2023-10-08T16:10:33.763286+0000","last_active":"2023-10-08T16:10:33.763286+0000","last_peered":"2023-10-08T16:10:33.763286+0000","last_clean":"2023-10-08T16:10:33.763286+0000","last_became_active":"2023-10-08T16:10:33.763028+0000","last_became_peered":"2023-10-08T16:10:33.763028+0000","last_unstale":"2023-10-08T16:10:33.763286+0000","last_undegraded":"2023-10-08T16:10:33.763286+0000","last_fullsized":"2023-10-08T16:10:33.763286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:03.701535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756126+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767700+0000","last_change":"2023-10-08T16:10:33.767700+0000","last_active":"2023-10-08T16:10:33.767700+0000","last_peered":"2023-10-08T16:10:33.767700+0000","last_clean":"2023-10-08T16:10:33.767700+0000","last_became_active":"2023-10-08T16:10:33.766926+0000","last_became_peered":"2023-10-08T16:10:33.766926+0000","last_unstale":"2023-10-08T16:10:33.767700+0000","last_undegraded":"2023-10-08T16:10:33.767700+0000","last_fullsized":"2023-10-08T16:10:33.767700+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:01.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757686+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760988+0000","last_change":"2023-10-08T16:10:33.760988+0000","last_active":"2023-10-08T16:10:33.760988+0000","last_peered":"2023-10-08T16:10:33.760988+0000","last_clean":"2023-10-08T16:10:33.760988+0000","last_became_active":"2023-10-08T16:10:33.760705+0000","last_became_peered":"2023-10-08T16:10:33.760705+0000","last_unstale":"2023-10-08T16:10:33.760988+0000","last_undegraded":"2023-10-08T16:10:33.760988+0000","last_fullsized":"2023-10-08T16:10:33.760988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753230+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765098+0000","last_change":"2023-10-08T16:10:33.765098+0000","last_active":"2023-10-08T16:10:33.765098+0000","last_peered":"2023-10-08T16:10:33.765098+0000","last_clean":"2023-10-08T16:10:33.765098+0000","last_became_active":"2023-10-08T16:10:33.764637+0000","last_became_peered":"2023-10-08T16:10:33.764637+0000","last_unstale":"2023-10-08T16:10:33.765098+0000","last_undegraded":"2023-10-08T16:10:33.765098+0000","last_fullsized":"2023-10-08T16:10:33.765098+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756417+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761688+0000","last_change":"2023-10-08T16:10:33.761688+0000","last_active":"2023-10-08T16:10:33.761688+0000","last_peered":"2023-10-08T16:10:33.761688+0000","last_clean":"2023-10-08T16:10:33.761688+0000","last_became_active":"2023-10-08T16:10:33.761196+0000","last_became_peered":"2023-10-08T16:10:33.761196+0000","last_unstale":"2023-10-08T16:10:33.761688+0000","last_undegraded":"2023-10-08T16:10:33.761688+0000","last_fullsized":"2023-10-08T16:10:33.761688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:27.197085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757514+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:53.047014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770625+0000","last_change":"2023-10-08T16:10:33.770625+0000","last_active":"2023-10-08T16:10:33.770625+0000","last_peered":"2023-10-08T16:10:33.770625+0000","last_clean":"2023-10-08T16:10:33.770625+0000","last_became_active":"2023-10-08T16:10:33.770366+0000","last_became_peered":"2023-10-08T16:10:33.770366+0000","last_unstale":"2023-10-08T16:10:33.770625+0000","last_undegraded":"2023-10-08T16:10:33.770625+0000","last_fullsized":"2023-10-08T16:10:33.770625+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:29.471251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757179+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:59.123802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767567+0000","last_change":"2023-10-08T16:10:33.767567+0000","last_active":"2023-10-08T16:10:33.767567+0000","last_peered":"2023-10-08T16:10:33.767567+0000","last_clean":"2023-10-08T16:10:33.767567+0000","last_became_active":"2023-10-08T16:10:33.767321+0000","last_became_peered":"2023-10-08T16:10:33.767321+0000","last_unstale":"2023-10-08T16:10:33.767567+0000","last_undegraded":"2023-10-08T16:10:33.767567+0000","last_fullsized":"2023-10-08T16:10:33.767567+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:13:23.776656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757333+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764988+0000","last_change":"2023-10-08T16:10:33.764988+0000","last_active":"2023-10-08T16:10:33.764988+0000","last_peered":"2023-10-08T16:10:33.764988+0000","last_clean":"2023-10-08T16:10:33.764988+0000","last_became_active":"2023-10-08T16:10:33.764739+0000","last_became_peered":"2023-10-08T16:10:33.764739+0000","last_unstale":"2023-10-08T16:10:33.764988+0000","last_undegraded":"2023-10-08T16:10:33.764988+0000","last_fullsized":"2023-10-08T16:10:33.764988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751897+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768462+0000","last_change":"2023-10-08T16:10:33.768462+0000","last_active":"2023-10-08T16:10:33.768462+0000","last_peered":"2023-10-08T16:10:33.768462+0000","last_clean":"2023-10-08T16:10:33.768462+0000","last_became_active":"2023-10-08T16:10:33.768242+0000","last_became_peered":"2023-10-08T16:10:33.768242+0000","last_unstale":"2023-10-08T16:10:33.768462+0000","last_undegraded":"2023-10-08T16:10:33.768462+0000","last_fullsized":"2023-10-08T16:10:33.768462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:59.803650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758164+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773222+0000","last_change":"2023-10-08T16:10:33.773222+0000","last_active":"2023-10-08T16:10:33.773222+0000","last_peered":"2023-10-08T16:10:33.773222+0000","last_clean":"2023-10-08T16:10:33.773222+0000","last_became_active":"2023-10-08T16:10:33.772922+0000","last_became_peered":"2023-10-08T16:10:33.772922+0000","last_unstale":"2023-10-08T16:10:33.773222+0000","last_undegraded":"2023-10-08T16:10:33.773222+0000","last_fullsized":"2023-10-08T16:10:33.773222+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:07.790273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753477+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762563+0000","last_change":"2023-10-08T16:10:33.762563+0000","last_active":"2023-10-08T16:10:33.762563+0000","last_peered":"2023-10-08T16:10:33.762563+0000","last_clean":"2023-10-08T16:10:33.762563+0000","last_became_active":"2023-10-08T16:10:33.762298+0000","last_became_peered":"2023-10-08T16:10:33.762298+0000","last_unstale":"2023-10-08T16:10:33.762563+0000","last_undegraded":"2023-10-08T16:10:33.762563+0000","last_fullsized":"2023-10-08T16:10:33.762563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:13:51.956683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766497+0000","last_change":"2023-10-08T16:10:33.766497+0000","last_active":"2023-10-08T16:10:33.766497+0000","last_peered":"2023-10-08T16:10:33.766497+0000","last_clean":"2023-10-08T16:10:33.766497+0000","last_became_active":"2023-10-08T16:10:33.766186+0000","last_became_peered":"2023-10-08T16:10:33.766186+0000","last_unstale":"2023-10-08T16:10:33.766497+0000","last_undegraded":"2023-10-08T16:10:33.766497+0000","last_fullsized":"2023-10-08T16:10:33.766497+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:02.257047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752088+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770711+0000","last_change":"2023-10-08T16:10:33.770711+0000","last_active":"2023-10-08T16:10:33.770711+0000","last_peered":"2023-10-08T16:10:33.770711+0000","last_clean":"2023-10-08T16:10:33.770711+0000","last_became_active":"2023-10-08T16:10:33.770495+0000","last_became_peered":"2023-10-08T16:10:33.770495+0000","last_unstale":"2023-10-08T16:10:33.770711+0000","last_undegraded":"2023-10-08T16:10:33.770711+0000","last_fullsized":"2023-10-08T16:10:33.770711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:50.671846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755500+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:12.462984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773350+0000","last_change":"2023-10-08T16:10:33.773350+0000","last_active":"2023-10-08T16:10:33.773350+0000","last_peered":"2023-10-08T16:10:33.773350+0000","last_clean":"2023-10-08T16:10:33.773350+0000","last_became_active":"2023-10-08T16:10:33.772989+0000","last_became_peered":"2023-10-08T16:10:33.772989+0000","last_unstale":"2023-10-08T16:10:33.773350+0000","last_undegraded":"2023-10-08T16:10:33.773350+0000","last_fullsized":"2023-10-08T16:10:33.773350+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:18.004513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751707+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760706+0000","last_change":"2023-10-08T16:10:33.760706+0000","last_active":"2023-10-08T16:10:33.760706+0000","last_peered":"2023-10-08T16:10:33.760706+0000","last_clean":"2023-10-08T16:10:33.760706+0000","last_became_active":"2023-10-08T16:10:33.760439+0000","last_became_peered":"2023-10-08T16:10:33.760439+0000","last_unstale":"2023-10-08T16:10:33.760706+0000","last_undegraded":"2023-10-08T16:10:33.760706+0000","last_fullsized":"2023-10-08T16:10:33.760706+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:34.341551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754402+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764394+0000","last_change":"2023-10-08T16:10:33.764394+0000","last_active":"2023-10-08T16:10:33.764394+0000","last_peered":"2023-10-08T16:10:33.764394+0000","last_clean":"2023-10-08T16:10:33.764394+0000","last_became_active":"2023-10-08T16:10:33.764077+0000","last_became_peered":"2023-10-08T16:10:33.764077+0000","last_unstale":"2023-10-08T16:10:33.764394+0000","last_undegraded":"2023-10-08T16:10:33.764394+0000","last_fullsized":"2023-10-08T16:10:33.764394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:27.558248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754255+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773846+0000","last_change":"2023-10-08T16:10:33.773846+0000","last_active":"2023-10-08T16:10:33.773846+0000","last_peered":"2023-10-08T16:10:33.773846+0000","last_clean":"2023-10-08T16:10:33.773846+0000","last_became_active":"2023-10-08T16:10:33.773563+0000","last_became_peered":"2023-10-08T16:10:33.773563+0000","last_unstale":"2023-10-08T16:10:33.773846+0000","last_undegraded":"2023-10-08T16:10:33.773846+0000","last_fullsized":"2023-10-08T16:10:33.773846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:58:45.919412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754208+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.759531+0000","last_change":"2023-10-08T16:10:33.759531+0000","last_active":"2023-10-08T16:10:33.759531+0000","last_peered":"2023-10-08T16:10:33.759531+0000","last_clean":"2023-10-08T16:10:33.759531+0000","last_became_active":"2023-10-08T16:10:33.759312+0000","last_became_peered":"2023-10-08T16:10:33.759312+0000","last_unstale":"2023-10-08T16:10:33.759531+0000","last_undegraded":"2023-10-08T16:10:33.759531+0000","last_fullsized":"2023-10-08T16:10:33.759531+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758276+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:23.087283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768853+0000","last_change":"2023-10-08T16:10:33.768853+0000","last_active":"2023-10-08T16:10:33.768853+0000","last_peered":"2023-10-08T16:10:33.768853+0000","last_clean":"2023-10-08T16:10:33.768853+0000","last_became_active":"2023-10-08T16:10:33.768563+0000","last_became_peered":"2023-10-08T16:10:33.768563+0000","last_unstale":"2023-10-08T16:10:33.768853+0000","last_undegraded":"2023-10-08T16:10:33.768853+0000","last_fullsized":"2023-10-08T16:10:33.768853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:57.969527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755709+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761930+0000","last_change":"2023-10-08T16:10:33.761930+0000","last_active":"2023-10-08T16:10:33.761930+0000","last_peered":"2023-10-08T16:10:33.761930+0000","last_clean":"2023-10-08T16:10:33.761930+0000","last_became_active":"2023-10-08T16:10:33.761680+0000","last_became_peered":"2023-10-08T16:10:33.761680+0000","last_unstale":"2023-10-08T16:10:33.761930+0000","last_undegraded":"2023-10-08T16:10:33.761930+0000","last_fullsized":"2023-10-08T16:10:33.761930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751469+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766371+0000","last_change":"2023-10-08T16:10:33.766371+0000","last_active":"2023-10-08T16:10:33.766371+0000","last_peered":"2023-10-08T16:10:33.766371+0000","last_clean":"2023-10-08T16:10:33.766371+0000","last_became_active":"2023-10-08T16:10:33.766082+0000","last_became_peered":"2023-10-08T16:10:33.766082+0000","last_unstale":"2023-10-08T16:10:33.766371+0000","last_undegraded":"2023-10-08T16:10:33.766371+0000","last_fullsized":"2023-10-08T16:10:33.766371+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:58.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760703+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770919+0000","last_change":"2023-10-08T16:10:33.770919+0000","last_active":"2023-10-08T16:10:33.770919+0000","last_peered":"2023-10-08T16:10:33.770919+0000","last_clean":"2023-10-08T16:10:33.770919+0000","last_became_active":"2023-10-08T16:10:33.770608+0000","last_became_peered":"2023-10-08T16:10:33.770608+0000","last_unstale":"2023-10-08T16:10:33.770919+0000","last_undegraded":"2023-10-08T16:10:33.770919+0000","last_fullsized":"2023-10-08T16:10:33.770919+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:37.099695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.763791+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:15.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766192+0000","last_change":"2023-10-08T16:10:33.766192+0000","last_active":"2023-10-08T16:10:33.766192+0000","last_peered":"2023-10-08T16:10:33.766192+0000","last_clean":"2023-10-08T16:10:33.766192+0000","last_became_active":"2023-10-08T16:10:33.765245+0000","last_became_peered":"2023-10-08T16:10:33.765245+0000","last_unstale":"2023-10-08T16:10:33.766192+0000","last_undegraded":"2023-10-08T16:10:33.766192+0000","last_fullsized":"2023-10-08T16:10:33.766192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:49.157336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759697+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:19:27.448117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766948+0000","last_change":"2023-10-08T16:10:33.766948+0000","last_active":"2023-10-08T16:10:33.766948+0000","last_peered":"2023-10-08T16:10:33.766948+0000","last_clean":"2023-10-08T16:10:33.766948+0000","last_became_active":"2023-10-08T16:10:33.766644+0000","last_became_peered":"2023-10-08T16:10:33.766644+0000","last_unstale":"2023-10-08T16:10:33.766948+0000","last_undegraded":"2023-10-08T16:10:33.766948+0000","last_fullsized":"2023-10-08T16:10:33.766948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:54.343260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756582+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762562+0000","last_change":"2023-10-08T16:10:33.762562+0000","last_active":"2023-10-08T16:10:33.762562+0000","last_peered":"2023-10-08T16:10:33.762562+0000","last_clean":"2023-10-08T16:10:33.762562+0000","last_became_active":"2023-10-08T16:10:33.762261+0000","last_became_peered":"2023-10-08T16:10:33.762261+0000","last_unstale":"2023-10-08T16:10:33.762562+0000","last_undegraded":"2023-10-08T16:10:33.762562+0000","last_fullsized":"2023-10-08T16:10:33.762562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:18.712472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752101+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.103975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765165+0000","last_change":"2023-10-08T16:10:33.765165+0000","last_active":"2023-10-08T16:10:33.765165+0000","last_peered":"2023-10-08T16:10:33.765165+0000","last_clean":"2023-10-08T16:10:33.765165+0000","last_became_active":"2023-10-08T16:10:33.764738+0000","last_became_peered":"2023-10-08T16:10:33.764738+0000","last_unstale":"2023-10-08T16:10:33.765165+0000","last_undegraded":"2023-10-08T16:10:33.765165+0000","last_fullsized":"2023-10-08T16:10:33.765165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751869+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.755373+0000","last_change":"2023-10-08T16:10:33.755373+0000","last_active":"2023-10-08T16:10:33.755373+0000","last_peered":"2023-10-08T16:10:33.755373+0000","last_clean":"2023-10-08T16:10:33.755373+0000","last_became_active":"2023-10-08T16:10:33.755019+0000","last_became_peered":"2023-10-08T16:10:33.755019+0000","last_unstale":"2023-10-08T16:10:33.755373+0000","last_undegraded":"2023-10-08T16:10:33.755373+0000","last_fullsized":"2023-10-08T16:10:33.755373+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:32.618356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755200+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767441+0000","last_change":"2023-10-08T16:10:33.767441+0000","last_active":"2023-10-08T16:10:33.767441+0000","last_peered":"2023-10-08T16:10:33.767441+0000","last_clean":"2023-10-08T16:10:33.767441+0000","last_became_active":"2023-10-08T16:10:33.767184+0000","last_became_peered":"2023-10-08T16:10:33.767184+0000","last_unstale":"2023-10-08T16:10:33.767441+0000","last_undegraded":"2023-10-08T16:10:33.767441+0000","last_fullsized":"2023-10-08T16:10:33.767441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:44.856255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754992+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766449+0000","last_change":"2023-10-08T16:10:33.766449+0000","last_active":"2023-10-08T16:10:33.766449+0000","last_peered":"2023-10-08T16:10:33.766449+0000","last_clean":"2023-10-08T16:10:33.766449+0000","last_became_active":"2023-10-08T16:10:33.766244+0000","last_became_peered":"2023-10-08T16:10:33.766244+0000","last_unstale":"2023-10-08T16:10:33.766449+0000","last_undegraded":"2023-10-08T16:10:33.766449+0000","last_fullsized":"2023-10-08T16:10:33.766449+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:02.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752142+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773844+0000","last_change":"2023-10-08T16:10:33.773844+0000","last_active":"2023-10-08T16:10:33.773844+0000","last_peered":"2023-10-08T16:10:33.773844+0000","last_clean":"2023-10-08T16:10:33.773844+0000","last_became_active":"2023-10-08T16:10:33.773481+0000","last_became_peered":"2023-10-08T16:10:33.773481+0000","last_unstale":"2023-10-08T16:10:33.773844+0000","last_undegraded":"2023-10-08T16:10:33.773844+0000","last_fullsized":"2023-10-08T16:10:33.773844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:41.626242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-08T16:10:32.749980+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:32.749980+0000","last_peered":"2023-10-08T16:10:32.749980+0000","last_clean":"2023-10-08T16:10:32.749980+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:32.749980+0000","last_undegraded":"2023-10-08T16:10:32.749980+0000","last_fullsized":"2023-10-08T16:10:32.749980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.57299999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.53600000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67200000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.748}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.60699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.626},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303168,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068672,"statfs":{"total":96636764160,"available":96326320128,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.314},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.252},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.44}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.46800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.624}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.432},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:39.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:39.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.409+0000 7f6e379db700 1 Processor -- start 2023-10-08T16:10:39.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.409+0000 7f6e379db700 1 -- start start 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e379db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e3015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e379db700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3015a160 con 0x7f6e30069560 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e379db700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e300d13a0 con 0x7f6e30159c60 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e379db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3015e140 con 0x7f6e30159890 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e35f78700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6e30069560 0x7f6e300ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33668/0 (socket says 172.21.15.170:33668) 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e35f78700 1 -- 172.21.15.170:0/4153912633 learned_addr learned my addr 172.21.15.170:0/4153912633 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:39.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e35777700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e3015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458325965 0 0) 0x7f6e3015a160 con 0x7f6e30069560 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.410+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003570 con 0x7f6e30069560 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1148154451 0 0) 0x7f6e18003570 con 0x7f6e30069560 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 msgr2=0x7f6e3015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e3015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 >> v1:172.21.15.191:6789/0 conn(0x7f6e30159c60 legacy=0x7f6e3015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3015a3d0 con 0x7f6e30069560 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6e28001470 con 0x7f6e30069560 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 >> v1:172.21.15.170:6789/0 conn(0x7f6e30069560 legacy=0x7f6e300ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e18003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.414 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e18004010 0x7f6e18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3015e140 con 0x7f6e18004d50 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800bbf0 con 0x7f6e18004010 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800bdb0 con 0x7f6e30159890 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35777700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e18003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e34f76700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e18004010 0x7f6e18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 -- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 msgr2=0x7f6e18003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e30159890 0x7f6e18003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 -- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e18004010 msgr2=0x7f6e18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:39.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e18004010 0x7f6e18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 -- 172.21.15.170:0/4153912633 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18003570 con 0x7f6e18004d50 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e34f76700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e18004010 0x7f6e18004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e35f78700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e180051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c390 tx=0x7f6e28001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc4f6079b1113f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800cc60 con 0x7f6e18004d50 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800cdc0 con 0x7f6e18004d50 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.411+0000 7f6e2ffff700 1 -- 172.21.15.170:0/4153912633 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28005b70 con 0x7f6e18004d50 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 -- 172.21.15.170:0/4153912633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 msgr2=0x7f6e180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 --2- 172.21.15.170:0/4153912633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e180051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800c390 tx=0x7f6e28001e50 comp rx=0 tx=0).stop 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e35f78700 1 -- 172.21.15.170:0/4153912633 reap_dead start 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 -- 172.21.15.170:0/4153912633 shutdown_connections 2023-10-08T16:10:39.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 -- 172.21.15.170:0/4153912633 >> 172.21.15.170:0/4153912633 conn(0x7f6e30068e20 msgr2=0x7f6e300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:39.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 -- 172.21.15.170:0/4153912633 shutdown_connections 2023-10-08T16:10:39.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.412+0000 7f6e379db700 1 -- 172.21.15.170:0/4153912633 wait complete. 2023-10-08T16:10:39.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.413+0000 7f6e379db700 1 Processor -- start 2023-10-08T16:10:39.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.413+0000 7f6e379db700 1 -- start start 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 0x7f6e300b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 0x7f6e3015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300ceaa0 con 0x7f6e18004d50 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300cde90 con 0x7f6e300cef30 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e379db700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300ce170 con 0x7f6e300b6250 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38024/0 (socket says 172.21.15.170:38024) 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 -- 172.21.15.170:0/249806620 learned_addr learned my addr 172.21.15.170:0/249806620 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:39.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35f78700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 0x7f6e3015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 -- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 msgr2=0x7f6e3015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 0x7f6e3015cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 -- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 msgr2=0x7f6e300b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e34f76700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 0x7f6e300b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 0x7f6e300b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35777700 1 -- 172.21.15.170:0/249806620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300ce450 con 0x7f6e18004d50 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e35f78700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 0x7f6e3015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.414+0000 7f6e34f76700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 0x7f6e300b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e35777700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6e20011f00 tx=0x7f6e2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2000e040 con 0x7f6e18004d50 2023-10-08T16:10:39.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2000f040 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e300ce730 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20015430 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.415+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e300cda80 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.416+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3004c3c0 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.416+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f6e20015860 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.416+0000 7f6e2d7fa700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 0x7f6e1c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.416+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e200859b0 con 0x7f6e18004d50 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.417+0000 7f6e34f76700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 0x7f6e1c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:39.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.417+0000 7f6e34f76700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 0x7f6e1c069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e24007bc0 tx=0x7f6e2400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:39.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.421+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e20052600 con 0x7f6e18004d50 2023-10-08T16:10:39.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.598+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e300cebe0 con 0x7f6e1c067290 2023-10-08T16:10:39.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.606+0000 7f6e2d7fa700 1 -- 172.21.15.170:0/249806620 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286548 (secure 0 0 0) 0x7f6e24010040 con 0x7f6e1c067290 2023-10-08T16:10:39.610 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 msgr2=0x7f6e1c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 0x7f6e1c069740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6e24007bc0 tx=0x7f6e2400c040 comp rx=0 tx=0).stop 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 msgr2=0x7f6e301493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6e20011f00 tx=0x7f6e2000b040 comp rx=0 tx=0).stop 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 shutdown_connections 2023-10-08T16:10:39.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e300b6250 0x7f6e3015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e1c067290 0x7f6e1c069740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e18004d50 0x7f6e301493a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 --2- 172.21.15.170:0/249806620 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e300cef30 0x7f6e300b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 >> 172.21.15.170:0/249806620 conn(0x7f6e30068e20 msgr2=0x7f6e30149d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 shutdown_connections 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:39.610+0000 7f6e379db700 1 -- 172.21.15.170:0/249806620 wait complete. 2023-10-08T16:10:39.613 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:39.622 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2023-10-08T16:10:38.298588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405480,"kb_used_data":10792,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752569240,"statfs":{"total":773094113280,"available":770630901760,"internally_reserved":0,"allocated":11051008,"data_stored":5417592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.555417"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.764209+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:04.752369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:10.607412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757132+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758755+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:19.486276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755308+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:00.816633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751803+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758103+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756169+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761263+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:45:38.964981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761247+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:07.766197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751846+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759204+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:05.459867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756405+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755366+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751806+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757139+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752896+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.761185+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756681+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752689+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:45.260005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754178+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753975+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751934+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754842+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752234+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:35.450350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760754+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:42.324138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752524+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753403+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753733+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757821+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755472+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752055+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751912+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766666+0000","last_change":"2023-10-08T16:10:33.766666+0000","last_active":"2023-10-08T16:10:33.766666+0000","last_peered":"2023-10-08T16:10:33.766666+0000","last_clean":"2023-10-08T16:10:33.766666+0000","last_became_active":"2023-10-08T16:10:33.764988+0000","last_became_peered":"2023-10-08T16:10:33.764988+0000","last_unstale":"2023-10-08T16:10:33.766666+0000","last_undegraded":"2023-10-08T16:10:33.766666+0000","last_fullsized":"2023-10-08T16:10:33.766666+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:49.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759961+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765370+0000","last_change":"2023-10-08T16:10:33.765370+0000","last_active":"2023-10-08T16:10:33.765370+0000","last_peered":"2023-10-08T16:10:33.765370+0000","last_clean":"2023-10-08T16:10:33.765370+0000","last_became_active":"2023-10-08T16:10:33.764896+0000","last_became_peered":"2023-10-08T16:10:33.764896+0000","last_unstale":"2023-10-08T16:10:33.765370+0000","last_undegraded":"2023-10-08T16:10:33.765370+0000","last_fullsized":"2023-10-08T16:10:33.765370+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751725+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.763286+0000","last_change":"2023-10-08T16:10:33.763286+0000","last_active":"2023-10-08T16:10:33.763286+0000","last_peered":"2023-10-08T16:10:33.763286+0000","last_clean":"2023-10-08T16:10:33.763286+0000","last_became_active":"2023-10-08T16:10:33.763028+0000","last_became_peered":"2023-10-08T16:10:33.763028+0000","last_unstale":"2023-10-08T16:10:33.763286+0000","last_undegraded":"2023-10-08T16:10:33.763286+0000","last_fullsized":"2023-10-08T16:10:33.763286+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:03.701535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756126+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767700+0000","last_change":"2023-10-08T16:10:33.767700+0000","last_active":"2023-10-08T16:10:33.767700+0000","last_peered":"2023-10-08T16:10:33.767700+0000","last_clean":"2023-10-08T16:10:33.767700+0000","last_became_active":"2023-10-08T16:10:33.766926+0000","last_became_peered":"2023-10-08T16:10:33.766926+0000","last_unstale":"2023-10-08T16:10:33.767700+0000","last_undegraded":"2023-10-08T16:10:33.767700+0000","last_fullsized":"2023-10-08T16:10:33.767700+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:01.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757686+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760988+0000","last_change":"2023-10-08T16:10:33.760988+0000","last_active":"2023-10-08T16:10:33.760988+0000","last_peered":"2023-10-08T16:10:33.760988+0000","last_clean":"2023-10-08T16:10:33.760988+0000","last_became_active":"2023-10-08T16:10:33.760705+0000","last_became_peered":"2023-10-08T16:10:33.760705+0000","last_unstale":"2023-10-08T16:10:33.760988+0000","last_undegraded":"2023-10-08T16:10:33.760988+0000","last_fullsized":"2023-10-08T16:10:33.760988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753230+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765098+0000","last_change":"2023-10-08T16:10:33.765098+0000","last_active":"2023-10-08T16:10:33.765098+0000","last_peered":"2023-10-08T16:10:33.765098+0000","last_clean":"2023-10-08T16:10:33.765098+0000","last_became_active":"2023-10-08T16:10:33.764637+0000","last_became_peered":"2023-10-08T16:10:33.764637+0000","last_unstale":"2023-10-08T16:10:33.765098+0000","last_undegraded":"2023-10-08T16:10:33.765098+0000","last_fullsized":"2023-10-08T16:10:33.765098+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756417+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761688+0000","last_change":"2023-10-08T16:10:33.761688+0000","last_active":"2023-10-08T16:10:33.761688+0000","last_peered":"2023-10-08T16:10:33.761688+0000","last_clean":"2023-10-08T16:10:33.761688+0000","last_became_active":"2023-10-08T16:10:33.761196+0000","last_became_peered":"2023-10-08T16:10:33.761196+0000","last_unstale":"2023-10-08T16:10:33.761688+0000","last_undegraded":"2023-10-08T16:10:33.761688+0000","last_fullsized":"2023-10-08T16:10:33.761688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:27.197085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757514+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:53.047014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770625+0000","last_change":"2023-10-08T16:10:33.770625+0000","last_active":"2023-10-08T16:10:33.770625+0000","last_peered":"2023-10-08T16:10:33.770625+0000","last_clean":"2023-10-08T16:10:33.770625+0000","last_became_active":"2023-10-08T16:10:33.770366+0000","last_became_peered":"2023-10-08T16:10:33.770366+0000","last_unstale":"2023-10-08T16:10:33.770625+0000","last_undegraded":"2023-10-08T16:10:33.770625+0000","last_fullsized":"2023-10-08T16:10:33.770625+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:29.471251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757179+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:55:59.123802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767567+0000","last_change":"2023-10-08T16:10:33.767567+0000","last_active":"2023-10-08T16:10:33.767567+0000","last_peered":"2023-10-08T16:10:33.767567+0000","last_clean":"2023-10-08T16:10:33.767567+0000","last_became_active":"2023-10-08T16:10:33.767321+0000","last_became_peered":"2023-10-08T16:10:33.767321+0000","last_unstale":"2023-10-08T16:10:33.767567+0000","last_undegraded":"2023-10-08T16:10:33.767567+0000","last_fullsized":"2023-10-08T16:10:33.767567+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:13:23.776656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.757333+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764988+0000","last_change":"2023-10-08T16:10:33.764988+0000","last_active":"2023-10-08T16:10:33.764988+0000","last_peered":"2023-10-08T16:10:33.764988+0000","last_clean":"2023-10-08T16:10:33.764988+0000","last_became_active":"2023-10-08T16:10:33.764739+0000","last_became_peered":"2023-10-08T16:10:33.764739+0000","last_unstale":"2023-10-08T16:10:33.764988+0000","last_undegraded":"2023-10-08T16:10:33.764988+0000","last_fullsized":"2023-10-08T16:10:33.764988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751897+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768462+0000","last_change":"2023-10-08T16:10:33.768462+0000","last_active":"2023-10-08T16:10:33.768462+0000","last_peered":"2023-10-08T16:10:33.768462+0000","last_clean":"2023-10-08T16:10:33.768462+0000","last_became_active":"2023-10-08T16:10:33.768242+0000","last_became_peered":"2023-10-08T16:10:33.768242+0000","last_unstale":"2023-10-08T16:10:33.768462+0000","last_undegraded":"2023-10-08T16:10:33.768462+0000","last_fullsized":"2023-10-08T16:10:33.768462+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:59.803650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758164+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773222+0000","last_change":"2023-10-08T16:10:33.773222+0000","last_active":"2023-10-08T16:10:33.773222+0000","last_peered":"2023-10-08T16:10:33.773222+0000","last_clean":"2023-10-08T16:10:33.773222+0000","last_became_active":"2023-10-08T16:10:33.772922+0000","last_became_peered":"2023-10-08T16:10:33.772922+0000","last_unstale":"2023-10-08T16:10:33.773222+0000","last_undegraded":"2023-10-08T16:10:33.773222+0000","last_fullsized":"2023-10-08T16:10:33.773222+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:07.790273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.753477+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762563+0000","last_change":"2023-10-08T16:10:33.762563+0000","last_active":"2023-10-08T16:10:33.762563+0000","last_peered":"2023-10-08T16:10:33.762563+0000","last_clean":"2023-10-08T16:10:33.762563+0000","last_became_active":"2023-10-08T16:10:33.762298+0000","last_became_peered":"2023-10-08T16:10:33.762298+0000","last_unstale":"2023-10-08T16:10:33.762563+0000","last_undegraded":"2023-10-08T16:10:33.762563+0000","last_fullsized":"2023-10-08T16:10:33.762563+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:13:51.956683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751414+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766497+0000","last_change":"2023-10-08T16:10:33.766497+0000","last_active":"2023-10-08T16:10:33.766497+0000","last_peered":"2023-10-08T16:10:33.766497+0000","last_clean":"2023-10-08T16:10:33.766497+0000","last_became_active":"2023-10-08T16:10:33.766186+0000","last_became_peered":"2023-10-08T16:10:33.766186+0000","last_unstale":"2023-10-08T16:10:33.766497+0000","last_undegraded":"2023-10-08T16:10:33.766497+0000","last_fullsized":"2023-10-08T16:10:33.766497+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:02.257047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752088+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770711+0000","last_change":"2023-10-08T16:10:33.770711+0000","last_active":"2023-10-08T16:10:33.770711+0000","last_peered":"2023-10-08T16:10:33.770711+0000","last_clean":"2023-10-08T16:10:33.770711+0000","last_became_active":"2023-10-08T16:10:33.770495+0000","last_became_peered":"2023-10-08T16:10:33.770495+0000","last_unstale":"2023-10-08T16:10:33.770711+0000","last_undegraded":"2023-10-08T16:10:33.770711+0000","last_fullsized":"2023-10-08T16:10:33.770711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:50.671846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755500+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:52:12.462984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773350+0000","last_change":"2023-10-08T16:10:33.773350+0000","last_active":"2023-10-08T16:10:33.773350+0000","last_peered":"2023-10-08T16:10:33.773350+0000","last_clean":"2023-10-08T16:10:33.773350+0000","last_became_active":"2023-10-08T16:10:33.772989+0000","last_became_peered":"2023-10-08T16:10:33.772989+0000","last_unstale":"2023-10-08T16:10:33.773350+0000","last_undegraded":"2023-10-08T16:10:33.773350+0000","last_fullsized":"2023-10-08T16:10:33.773350+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:18.004513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751707+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.760706+0000","last_change":"2023-10-08T16:10:33.760706+0000","last_active":"2023-10-08T16:10:33.760706+0000","last_peered":"2023-10-08T16:10:33.760706+0000","last_clean":"2023-10-08T16:10:33.760706+0000","last_became_active":"2023-10-08T16:10:33.760439+0000","last_became_peered":"2023-10-08T16:10:33.760439+0000","last_unstale":"2023-10-08T16:10:33.760706+0000","last_undegraded":"2023-10-08T16:10:33.760706+0000","last_fullsized":"2023-10-08T16:10:33.760706+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:34.341551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754402+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.764394+0000","last_change":"2023-10-08T16:10:33.764394+0000","last_active":"2023-10-08T16:10:33.764394+0000","last_peered":"2023-10-08T16:10:33.764394+0000","last_clean":"2023-10-08T16:10:33.764394+0000","last_became_active":"2023-10-08T16:10:33.764077+0000","last_became_peered":"2023-10-08T16:10:33.764077+0000","last_unstale":"2023-10-08T16:10:33.764394+0000","last_undegraded":"2023-10-08T16:10:33.764394+0000","last_fullsized":"2023-10-08T16:10:33.764394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:27.558248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754255+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773846+0000","last_change":"2023-10-08T16:10:33.773846+0000","last_active":"2023-10-08T16:10:33.773846+0000","last_peered":"2023-10-08T16:10:33.773846+0000","last_clean":"2023-10-08T16:10:33.773846+0000","last_became_active":"2023-10-08T16:10:33.773563+0000","last_became_peered":"2023-10-08T16:10:33.773563+0000","last_unstale":"2023-10-08T16:10:33.773846+0000","last_undegraded":"2023-10-08T16:10:33.773846+0000","last_fullsized":"2023-10-08T16:10:33.773846+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:58:45.919412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754208+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.759531+0000","last_change":"2023-10-08T16:10:33.759531+0000","last_active":"2023-10-08T16:10:33.759531+0000","last_peered":"2023-10-08T16:10:33.759531+0000","last_clean":"2023-10-08T16:10:33.759531+0000","last_became_active":"2023-10-08T16:10:33.759312+0000","last_became_peered":"2023-10-08T16:10:33.759312+0000","last_unstale":"2023-10-08T16:10:33.759531+0000","last_undegraded":"2023-10-08T16:10:33.759531+0000","last_fullsized":"2023-10-08T16:10:33.759531+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.758276+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:23.087283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.768853+0000","last_change":"2023-10-08T16:10:33.768853+0000","last_active":"2023-10-08T16:10:33.768853+0000","last_peered":"2023-10-08T16:10:33.768853+0000","last_clean":"2023-10-08T16:10:33.768853+0000","last_became_active":"2023-10-08T16:10:33.768563+0000","last_became_peered":"2023-10-08T16:10:33.768563+0000","last_unstale":"2023-10-08T16:10:33.768853+0000","last_undegraded":"2023-10-08T16:10:33.768853+0000","last_fullsized":"2023-10-08T16:10:33.768853+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:57.969527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755709+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.761930+0000","last_change":"2023-10-08T16:10:33.761930+0000","last_active":"2023-10-08T16:10:33.761930+0000","last_peered":"2023-10-08T16:10:33.761930+0000","last_clean":"2023-10-08T16:10:33.761930+0000","last_became_active":"2023-10-08T16:10:33.761680+0000","last_became_peered":"2023-10-08T16:10:33.761680+0000","last_unstale":"2023-10-08T16:10:33.761930+0000","last_undegraded":"2023-10-08T16:10:33.761930+0000","last_fullsized":"2023-10-08T16:10:33.761930+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751469+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766371+0000","last_change":"2023-10-08T16:10:33.766371+0000","last_active":"2023-10-08T16:10:33.766371+0000","last_peered":"2023-10-08T16:10:33.766371+0000","last_clean":"2023-10-08T16:10:33.766371+0000","last_became_active":"2023-10-08T16:10:33.766082+0000","last_became_peered":"2023-10-08T16:10:33.766082+0000","last_unstale":"2023-10-08T16:10:33.766371+0000","last_undegraded":"2023-10-08T16:10:33.766371+0000","last_fullsized":"2023-10-08T16:10:33.766371+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:58.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.760703+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.770919+0000","last_change":"2023-10-08T16:10:33.770919+0000","last_active":"2023-10-08T16:10:33.770919+0000","last_peered":"2023-10-08T16:10:33.770919+0000","last_clean":"2023-10-08T16:10:33.770919+0000","last_became_active":"2023-10-08T16:10:33.770608+0000","last_became_peered":"2023-10-08T16:10:33.770608+0000","last_unstale":"2023-10-08T16:10:33.770919+0000","last_undegraded":"2023-10-08T16:10:33.770919+0000","last_fullsized":"2023-10-08T16:10:33.770919+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:37.099695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.763791+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:15.268409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766192+0000","last_change":"2023-10-08T16:10:33.766192+0000","last_active":"2023-10-08T16:10:33.766192+0000","last_peered":"2023-10-08T16:10:33.766192+0000","last_clean":"2023-10-08T16:10:33.766192+0000","last_became_active":"2023-10-08T16:10:33.765245+0000","last_became_peered":"2023-10-08T16:10:33.765245+0000","last_unstale":"2023-10-08T16:10:33.766192+0000","last_undegraded":"2023-10-08T16:10:33.766192+0000","last_fullsized":"2023-10-08T16:10:33.766192+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:49.157336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.759697+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:19:27.448117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766948+0000","last_change":"2023-10-08T16:10:33.766948+0000","last_active":"2023-10-08T16:10:33.766948+0000","last_peered":"2023-10-08T16:10:33.766948+0000","last_clean":"2023-10-08T16:10:33.766948+0000","last_became_active":"2023-10-08T16:10:33.766644+0000","last_became_peered":"2023-10-08T16:10:33.766644+0000","last_unstale":"2023-10-08T16:10:33.766948+0000","last_undegraded":"2023-10-08T16:10:33.766948+0000","last_fullsized":"2023-10-08T16:10:33.766948+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:54.343260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.756582+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.762562+0000","last_change":"2023-10-08T16:10:33.762562+0000","last_active":"2023-10-08T16:10:33.762562+0000","last_peered":"2023-10-08T16:10:33.762562+0000","last_clean":"2023-10-08T16:10:33.762562+0000","last_became_active":"2023-10-08T16:10:33.762261+0000","last_became_peered":"2023-10-08T16:10:33.762261+0000","last_unstale":"2023-10-08T16:10:33.762562+0000","last_undegraded":"2023-10-08T16:10:33.762562+0000","last_fullsized":"2023-10-08T16:10:33.762562+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:18.712472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752101+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:38.103975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.765165+0000","last_change":"2023-10-08T16:10:33.765165+0000","last_active":"2023-10-08T16:10:33.765165+0000","last_peered":"2023-10-08T16:10:33.765165+0000","last_clean":"2023-10-08T16:10:33.765165+0000","last_became_active":"2023-10-08T16:10:33.764738+0000","last_became_peered":"2023-10-08T16:10:33.764738+0000","last_unstale":"2023-10-08T16:10:33.765165+0000","last_undegraded":"2023-10-08T16:10:33.765165+0000","last_fullsized":"2023-10-08T16:10:33.765165+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.751869+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.755373+0000","last_change":"2023-10-08T16:10:33.755373+0000","last_active":"2023-10-08T16:10:33.755373+0000","last_peered":"2023-10-08T16:10:33.755373+0000","last_clean":"2023-10-08T16:10:33.755373+0000","last_became_active":"2023-10-08T16:10:33.755019+0000","last_became_peered":"2023-10-08T16:10:33.755019+0000","last_unstale":"2023-10-08T16:10:33.755373+0000","last_undegraded":"2023-10-08T16:10:33.755373+0000","last_fullsized":"2023-10-08T16:10:33.755373+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:32.618356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.755200+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.767441+0000","last_change":"2023-10-08T16:10:33.767441+0000","last_active":"2023-10-08T16:10:33.767441+0000","last_peered":"2023-10-08T16:10:33.767441+0000","last_clean":"2023-10-08T16:10:33.767441+0000","last_became_active":"2023-10-08T16:10:33.767184+0000","last_became_peered":"2023-10-08T16:10:33.767184+0000","last_unstale":"2023-10-08T16:10:33.767441+0000","last_undegraded":"2023-10-08T16:10:33.767441+0000","last_fullsized":"2023-10-08T16:10:33.767441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:44.856255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.754992+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.766449+0000","last_change":"2023-10-08T16:10:33.766449+0000","last_active":"2023-10-08T16:10:33.766449+0000","last_peered":"2023-10-08T16:10:33.766449+0000","last_clean":"2023-10-08T16:10:33.766449+0000","last_became_active":"2023-10-08T16:10:33.766244+0000","last_became_peered":"2023-10-08T16:10:33.766244+0000","last_unstale":"2023-10-08T16:10:33.766449+0000","last_undegraded":"2023-10-08T16:10:33.766449+0000","last_fullsized":"2023-10-08T16:10:33.766449+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:02.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:10:33.743141+0000","last_change":"2023-10-08T16:10:33.752142+0000","last_active":"2023-10-08T16:10:33.743141+0000","last_peered":"2023-10-08T16:10:33.743141+0000","last_clean":"2023-10-08T16:10:33.743141+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:10:33.743141+0000","last_undegraded":"2023-10-08T16:10:33.743141+0000","last_fullsized":"2023-10-08T16:10:33.743141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":8,"reported_epoch":99,"state":"active+clean","last_fresh":"2023-10-08T16:10:33.773844+0000","last_change":"2023-10-08T16:10:33.773844+0000","last_active":"2023-10-08T16:10:33.773844+0000","last_peered":"2023-10-08T16:10:33.773844+0000","last_clean":"2023-10-08T16:10:33.773844+0000","last_became_active":"2023-10-08T16:10:33.773481+0000","last_became_peered":"2023-10-08T16:10:33.773481+0000","last_unstale":"2023-10-08T16:10:33.773844+0000","last_undegraded":"2023-10-08T16:10:33.773844+0000","last_fullsized":"2023-10-08T16:10:33.773844+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:41.626242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":98,"state":"active+clean","last_fresh":"2023-10-08T16:10:32.749980+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:32.749980+0000","last_peered":"2023-10-08T16:10:32.749980+0000","last_clean":"2023-10-08T16:10:32.749980+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:32.749980+0000","last_undegraded":"2023-10-08T16:10:32.749980+0000","last_fullsized":"2023-10-08T16:10:32.749980+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299072,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072768,"statfs":{"total":96636764160,"available":96330514432,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.57299999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.504},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.47299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.52000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.53600000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.48699999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.38200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67200000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.45000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.748}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.79500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.78100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34599999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.33300000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.60699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.626},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.53300000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.59799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303168,"kb_used_data":1784,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068672,"statfs":{"total":96636764160,"available":96326320128,"internally_reserved":0,"allocated":1826816,"data_stored":1119975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.314},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.313,"15min":0.308},"min":{"1min":0.20999999999999999,"5min":0.188,"15min":0.188},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33700000000000002,"15min":0.35299999999999998},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47999999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.26400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.35099999999999998},"min":{"1min":0.252,"5min":0.222,"15min":0.222},"max":{"1min":0.55300000000000005,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.36099999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.53100000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34200000000000003,"15min":0.34999999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57099999999999995,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.252},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.38800000000000001},"min":{"1min":0.30199999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34100000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.75600000000000001,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.33400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.82799999999999996,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.38200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.69399999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.59299999999999997,"15min":0.68000000000000005},"min":{"1min":0.35799999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77900000000000003,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.42899999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79000000000000004,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.58199999999999996},"min":{"1min":0.40000000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81899999999999995,"5min":0.871,"15min":0.871},"last":0.44}]},{"osd":7,"last update":"Sun Oct 8 16:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60999999999999999,"15min":0.623},"min":{"1min":0.434,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.46800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302588,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069252,"statfs":{"total":96636764160,"available":96326914048,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.34200000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.29499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.39500000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.55500000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.63500000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.69399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.505,"15min":0.53400000000000003},"min":{"1min":0.33600000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68100000000000005,"5min":1.579,"15min":1.579},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.442,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.45800000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.5,"15min":0.48599999999999999},"min":{"1min":0.33700000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.53900000000000003},"min":{"1min":0.442,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70199999999999996,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.58899999999999997},"min":{"1min":0.432,"5min":0.254,"15min":0.254},"max":{"1min":0.75,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.752},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73499999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.747,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56499999999999995},"min":{"1min":0.43099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.505,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.373,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78700000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.80000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54300000000000004,"15min":0.56699999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70099999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.624}]},{"osd":7,"last update":"Sun Oct 8 16:09:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58299999999999996,"15min":0.60899999999999999},"min":{"1min":0.42899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76800000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.41399999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.72599999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.77600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298492,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073348,"statfs":{"total":96636764160,"available":96331108352,"internally_reserved":0,"allocated":1232896,"data_stored":529607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.56599999999999995,"15min":0.65900000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.74199999999999999,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.55800000000000005,"15min":0.65600000000000003},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66500000000000004,"5min":4.0119999999999996,"15min":4.0119999999999996},"last":0.47599999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.66300000000000003,"15min":1.0609999999999999},"min":{"1min":0.32400000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.71799999999999997,"5min":12.227,"15min":12.227},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.56200000000000006,"15min":0.57299999999999995},"min":{"1min":0.38400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.70599999999999996,"5min":1.252,"15min":1.252},"last":0.46200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.67900000000000005},"min":{"1min":0.38,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72599999999999998,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.64700000000000002,"5min":1.444,"15min":1.444},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.66800000000000004,"15min":1.0069999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70799999999999996,"5min":9.3170000000000002,"15min":9.3170000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.60499999999999998,"15min":0.71499999999999997},"min":{"1min":0.373,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.71999999999999997,"5min":3.8719999999999999,"15min":3.8719999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.33300000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.61699999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.496},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.69399999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.68899999999999995,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.432},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.77700000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52500000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.72599999999999998,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:39.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.764209+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:55:04.752369+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752414+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:20:10.607412+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757132+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.758755+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:05:19.486276+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755308+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:12:00.816633+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751803+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.758103+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756169+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-10-08T16:10:39.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.761263+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:45:38.964981+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.761247+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:15:07.766197+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751846+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.759204+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:11:05.459867+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756405+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2023-10-08T16:10:39.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755366+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751806+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757139+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752896+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-10-08T16:10:39.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.761185+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756681+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752689+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:36:45.260005+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754178+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2023-10-08T16:10:39.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.753975+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751934+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754842+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752234+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:47:35.450350+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.760754+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:20:42.324138+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752524+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.753403+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.753733+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757821+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-10-08T16:10:39.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755472+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752055+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751912+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.759961+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751725+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:58:50.161617+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756126+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757686+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.753230+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756417+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757514+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:46:53.047014+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757179+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:55:59.123802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.757333+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751897+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-08T16:10:39.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.758164+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:01:28.562528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:10:39.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-10-08T16:10:39.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.753477+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-10-08T16:10:39.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751414+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:40:32.765867+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752088+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755500+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:52:12.462984+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-08T16:10:39.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751707+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:03:21.773749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754402+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:19:49.612194+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754255+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:51:10.046569+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-10-08T16:10:39.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754208+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.758276+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:27:23.087283+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755709+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751469+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:56:32.347664+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.760703+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.763791+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:26:15.268409+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.759697+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:19:27.448117+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-08T16:10:39.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.756582+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752101+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:02:38.103975+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.751869+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.755200+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-08T16:10:39.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.754992+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:10:39.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-10-08T16:10:39.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:10:33.743141+0000', 'last_change': '2023-10-08T16:10:33.752142+0000', 'last_active': '2023-10-08T16:10:33.743141+0000', 'last_peered': '2023-10-08T16:10:33.743141+0000', 'last_clean': '2023-10-08T16:10:33.743141+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:10:33.743141+0000', 'last_undegraded': '2023-10-08T16:10:33.743141+0000', 'last_fullsized': '2023-10-08T16:10:33.743141+0000', 'mapping_epoch': 99, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 99, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'last_clean_scrub_stamp': '2023-10-08T16:10:33.743141+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:10:39.902 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T16:10:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 Processor -- start 2023-10-08T16:10:40.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 -- start start 2023-10-08T16:10:40.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 0x7f7f6815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7f680afaa0 0x7f7f681594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680c8b70 con 0x7f7f6806af10 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6815de80 con 0x7f7f680afaa0 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f6e9d2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6815e160 con 0x7f7f6806ab40 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.019+0000 7f7f677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7f680afaa0 0x7f7f681594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7f680afaa0 0x7f7f681594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38030/0 (socket says 172.21.15.170:38030) 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f6cf6f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7f6806af10 0x7f7f6806e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33674/0 (socket says 172.21.15.170:33674) 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 -- 172.21.15.170:0/2349056212 learned_addr learned my addr 172.21.15.170:0/2349056212 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f67fff700 1 --2- 172.21.15.170:0/2349056212 >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 0x7f7f6815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 -- 172.21.15.170:0/2349056212 >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 msgr2=0x7f7f6815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 --2- 172.21.15.170:0/2349056212 >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 0x7f7f6815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 -- 172.21.15.170:0/2349056212 >> v1:172.21.15.170:6789/0 conn(0x7f7f6806af10 legacy=0x7f7f6806e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f66ffd700 1 -- 172.21.15.170:0/2349056212 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537587463 0 0) 0x7f7f60001490 con 0x7f7f6806af10 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 -- 172.21.15.170:0/2349056212 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6815e440 con 0x7f7f680afaa0 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f67fff700 1 --2- 172.21.15.170:0/2349056212 >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 0x7f7f6815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f677fe700 1 --2- 172.21.15.170:0/2349056212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f680afaa0 0x7f7f681594f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7f6806e6b0 tx=0x7f7f5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1cafbfba3b7088 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f66ffd700 1 -- 172.21.15.170:0/2349056212 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00b860 con 0x7f7f680afaa0 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f66ffd700 1 -- 172.21.15.170:0/2349056212 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00b9c0 con 0x7f7f680afaa0 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.020+0000 7f7f66ffd700 1 -- 172.21.15.170:0/2349056212 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00bd60 con 0x7f7f680afaa0 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/2349056212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f680afaa0 msgr2=0x7f7f681594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/2349056212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f680afaa0 0x7f7f681594f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f7f6806e6b0 tx=0x7f7f5c009250 comp rx=0 tx=0).stop 2023-10-08T16:10:40.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/2349056212 shutdown_connections 2023-10-08T16:10:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/2349056212 >> v2:172.21.15.191:3300/0 conn(0x7f7f6806ab40 0x7f7f6815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/2349056212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f680afaa0 0x7f7f681594f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/2349056212 >> 172.21.15.170:0/2349056212 conn(0x7f7f680cd640 msgr2=0x7f7f680cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/2349056212 shutdown_connections 2023-10-08T16:10:40.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.021+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/2349056212 wait complete. 2023-10-08T16:10:40.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.022+0000 7f7f6e9d2700 1 Processor -- start 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.022+0000 7f7f6e9d2700 1 -- start start 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6806ab40 0x7f7f680b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 0x7f7f680a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b2d50 con 0x7f7f6806af10 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b2f60 con 0x7f7f680afaa0 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680c23d0 con 0x7f7f6806ab40 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38038/0 (socket says 172.21.15.170:38038) 2023-10-08T16:10:40.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 -- 172.21.15.170:0/3459264883 learned_addr learned my addr 172.21.15.170:0/3459264883 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f67fff700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6806ab40 0x7f7f680b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6cf6f700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 0x7f7f680a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 -- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6806ab40 msgr2=0x7f7f680b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6806ab40 0x7f7f680b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 -- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 msgr2=0x7f7f680a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 0x7f7f680a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 -- 172.21.15.170:0/3459264883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0094f0 con 0x7f7f6806af10 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6cf6f700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 0x7f7f680a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f677fe700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006f50 tx=0x7f7f5c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c01a450 con 0x7f7f6806af10 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c01a5d0 con 0x7f7f6806af10 2023-10-08T16:10:40.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f680c26b0 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c01a990 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.023+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f680c2080 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.024+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6804c3c0 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.025+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7f5c01ad00 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.025+0000 7f7f4ffff700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 0x7f7f54069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.025+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f5c086600 con 0x7f7f6806af10 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.025+0000 7f7f67fff700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 0x7f7f54069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.025+0000 7f7f67fff700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 0x7f7f54069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800ad40 tx=0x7f7f58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.029+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f5c0534e0 con 0x7f7f6806af10 2023-10-08T16:10:40.056 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:10:40.055+0000 7f33ea22c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:10:40.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.287+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7f68068390 con 0x7f7f6806af10 2023-10-08T16:10:40.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.287+0000 7f7f4ffff700 1 -- 172.21.15.170:0/3459264883 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1456 (secure 0 0 0) 0x7f7f5c0155c0 con 0x7f7f6806af10 2023-10-08T16:10:40.289 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:40.289 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":101,"num_osds":8,"num_up_osds":8,"osd_up_since":1696781068,"num_in_osds":8,"osd_in_since":1696781061,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2463211520,"bytes_avail":770630901760,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":65,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6771}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-10-08T16:10:24.295991+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T16:10:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 msgr2=0x7f7f54069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 0x7f7f54069740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800ad40 tx=0x7f7f58008040 comp rx=0 tx=0).stop 2023-10-08T16:10:40.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 msgr2=0x7f7f6815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c006f50 tx=0x7f7f5c017000 comp rx=0 tx=0).stop 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 shutdown_connections 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f680afaa0 0x7f7f680a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f6806ab40 0x7f7f680b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f54067290 0x7f7f54069740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 --2- 172.21.15.170:0/3459264883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f6806af10 0x7f7f6815bb30 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 >> 172.21.15.170:0/3459264883 conn(0x7f7f680cd640 msgr2=0x7f7f680cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 shutdown_connections 2023-10-08T16:10:40.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.290+0000 7f7f6e9d2700 1 -- 172.21.15.170:0/3459264883 wait complete. 2023-10-08T16:10:40.302 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T16:10:40.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.416+0000 7f4d201a3700 1 Processor -- start 2023-10-08T16:10:40.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.416+0000 7f4d201a3700 1 -- start start 2023-10-08T16:10:40.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d201a3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4d18069560 0x7f4d180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d201a3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d18069ee0 con 0x7f4d18159c60 2023-10-08T16:10:40.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d201a3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1815a160 con 0x7f4d18159890 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d201a3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d180d13a0 con 0x7f4d18069560 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d1d73e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4d18159c60 0x7f4d1815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33690/0 (socket says 172.21.15.170:33690) 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.417+0000 7f4d1d73e700 1 -- 172.21.15.170:0/554160364 learned_addr learned my addr 172.21.15.170:0/554160364 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1e740700 1 --2- 172.21.15.170:0/554160364 >> v2:172.21.15.191:3300/0 conn(0x7f4d18069560 0x7f4d180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899948426 0 0) 0x7f4d18069ee0 con 0x7f4d18159c60 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d00003570 con 0x7f4d18159c60 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3766555325 0 0) 0x7f4d00003570 con 0x7f4d18159c60 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 >> v2:172.21.15.191:3300/0 conn(0x7f4d18069560 msgr2=0x7f4d180699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:40.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 --2- 172.21.15.170:0/554160364 >> v2:172.21.15.191:3300/0 conn(0x7f4d18069560 0x7f4d180699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 >> v1:172.21.15.191:6789/0 conn(0x7f4d18159890 legacy=0x7f4d1815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1815e140 con 0x7f4d18159c60 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4d0c001470 con 0x7f4d18159c60 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 >> v1:172.21.15.170:6789/0 conn(0x7f4d18159c60 legacy=0x7f4d1815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d00003a60 0x7f4d00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 0x7f4d00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d000051c0 0x7f4d0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180d13a0 con 0x7f4d00004810 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0000c080 con 0x7f4d00003a60 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0000c240 con 0x7f4d000051c0 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1df3f700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d00003a60 0x7f4d00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 0x7f4d00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1e740700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d000051c0 0x7f4d0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1d73e700 1 -- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d000051c0 msgr2=0x7f4d0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d000051c0 0x7f4d0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1d73e700 1 -- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d00003a60 msgr2=0x7f4d00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:40.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.418+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d00003a60 0x7f4d00003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1d73e700 1 -- 172.21.15.170:0/554160364 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1815e140 con 0x7f4d00004810 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1e740700 1 -- 172.21.15.170:0/554160364 reap_dead start 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 0x7f4d00004c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c3e0 tx=0x7f4d0c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cf5f90c1dc4e1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00cd10 con 0x7f4d00004810 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00ce70 con 0x7f4d00004810 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d1cf3d700 1 -- 172.21.15.170:0/554160364 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c005b50 con 0x7f4d00004810 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 -- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 msgr2=0x7f4d00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 0x7f4d00004c80 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00c3e0 tx=0x7f4d0c007940 comp rx=0 tx=0).stop 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 -- 172.21.15.170:0/554160364 shutdown_connections 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 --2- 172.21.15.170:0/554160364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d00004810 0x7f4d00004c80 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 -- 172.21.15.170:0/554160364 >> 172.21.15.170:0/554160364 conn(0x7f4d18068e20 msgr2=0x7f4d180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:40.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 -- 172.21.15.170:0/554160364 shutdown_connections 2023-10-08T16:10:40.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.419+0000 7f4d201a3700 1 -- 172.21.15.170:0/554160364 wait complete. 2023-10-08T16:10:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.420+0000 7f4d201a3700 1 Processor -- start 2023-10-08T16:10:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 -- start start 2023-10-08T16:10:40.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d00004810 0x7f4d1816e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 0x7f4d180c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180cf120 con 0x7f4d181525d0 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1816e670 con 0x7f4d180ca260 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d201a3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180b5380 con 0x7f4d00004810 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d1e740700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d1e740700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38066/0 (socket says 172.21.15.170:38066) 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d1e740700 1 -- 172.21.15.170:0/1357921716 learned_addr learned my addr 172.21.15.170:0/1357921716 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d1df3f700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d00004810 0x7f4d1816e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.421+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 0x7f4d180c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 -- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d00004810 msgr2=0x7f4d1816e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d00004810 0x7f4d1816e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 -- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 msgr2=0x7f4d180c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 0x7f4d180c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 -- 172.21.15.170:0/1357921716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180ca760 con 0x7f4d181525d0 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1d73e700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 0x7f4d180c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d1e740700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4d10006f00 tx=0x7f4d100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000f7d0 con 0x7f4d181525d0 2023-10-08T16:10:40.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1000f930 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d180b4a50 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000fcd0 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.422+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d180b5010 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.423+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1804c3c0 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.423+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f4d10014650 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.424+0000 7f4d15ffb700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 0x7f4d04069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.424+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4d10018030 con 0x7f4d181525d0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.424+0000 7f4d1df3f700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 0x7f4d04069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:40.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.424+0000 7f4d1df3f700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 0x7f4d04069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4d08000f80 tx=0x7f4d08001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:40.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.428+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d10052090 con 0x7f4d181525d0 2023-10-08T16:10:40.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.607+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4d180b54c0 con 0x7f4d04067290 2023-10-08T16:10:40.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.615+0000 7f4d15ffb700 1 -- 172.21.15.170:0/1357921716 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289621 (secure 0 0 0) 0x7f4d180b54c0 con 0x7f4d04067290 2023-10-08T16:10:40.620 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 msgr2=0x7f4d04069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 0x7f4d04069740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4d08000f80 tx=0x7f4d08001640 comp rx=0 tx=0).stop 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 msgr2=0x7f4d180cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4d10006f00 tx=0x7f4d100091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 shutdown_connections 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d04067290 0x7f4d04069740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d181525d0 0x7f4d180cf3e0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d00004810 0x7f4d1816e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 --2- 172.21.15.170:0/1357921716 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d180ca260 0x7f4d180c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:40.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 >> 172.21.15.170:0/1357921716 conn(0x7f4d18068e20 msgr2=0x7f4d18152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:40.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 shutdown_connections 2023-10-08T16:10:40.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:40.618+0000 7f4d201a3700 1 -- 172.21.15.170:0/1357921716 wait complete. 2023-10-08T16:10:40.624 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:40.630 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-10-08T16:10:40.298859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405736,"kb_used_data":11048,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752568984,"statfs":{"total":773094113280,"available":770630639616,"internally_reserved":0,"allocated":11313152,"data_stored":5526612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.550132"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.776945+0000","last_change":"2023-10-08T16:10:34.776241+0000","last_active":"2023-10-08T16:10:34.776945+0000","last_peered":"2023-10-08T16:10:34.776945+0000","last_clean":"2023-10-08T16:10:34.776945+0000","last_became_active":"2023-10-08T16:10:34.773348+0000","last_became_peered":"2023-10-08T16:10:34.773348+0000","last_unstale":"2023-10-08T16:10:34.776945+0000","last_undegraded":"2023-10-08T16:10:34.776945+0000","last_fullsized":"2023-10-08T16:10:34.776945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:09.090277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777902+0000","last_change":"2023-10-08T16:10:34.762496+0000","last_active":"2023-10-08T16:10:34.777902+0000","last_peered":"2023-10-08T16:10:34.777902+0000","last_clean":"2023-10-08T16:10:34.777902+0000","last_became_active":"2023-10-08T16:10:34.762209+0000","last_became_peered":"2023-10-08T16:10:34.762209+0000","last_unstale":"2023-10-08T16:10:34.777902+0000","last_undegraded":"2023-10-08T16:10:34.777902+0000","last_fullsized":"2023-10-08T16:10:34.777902+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775031+0000","last_change":"2023-10-08T16:10:34.770582+0000","last_active":"2023-10-08T16:10:34.775031+0000","last_peered":"2023-10-08T16:10:34.775031+0000","last_clean":"2023-10-08T16:10:34.775031+0000","last_became_active":"2023-10-08T16:10:34.770264+0000","last_became_peered":"2023-10-08T16:10:34.770264+0000","last_unstale":"2023-10-08T16:10:34.775031+0000","last_undegraded":"2023-10-08T16:10:34.775031+0000","last_fullsized":"2023-10-08T16:10:34.775031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772448+0000","last_change":"2023-10-08T16:10:34.770983+0000","last_active":"2023-10-08T16:10:34.772448+0000","last_peered":"2023-10-08T16:10:34.772448+0000","last_clean":"2023-10-08T16:10:34.772448+0000","last_became_active":"2023-10-08T16:10:34.770747+0000","last_became_peered":"2023-10-08T16:10:34.770747+0000","last_unstale":"2023-10-08T16:10:34.772448+0000","last_undegraded":"2023-10-08T16:10:34.772448+0000","last_fullsized":"2023-10-08T16:10:34.772448+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:40.928641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758225+0000","last_change":"2023-10-08T16:10:34.755991+0000","last_active":"2023-10-08T16:10:34.758225+0000","last_peered":"2023-10-08T16:10:34.758225+0000","last_clean":"2023-10-08T16:10:34.758225+0000","last_became_active":"2023-10-08T16:10:34.755732+0000","last_became_peered":"2023-10-08T16:10:34.755732+0000","last_unstale":"2023-10-08T16:10:34.758225+0000","last_undegraded":"2023-10-08T16:10:34.758225+0000","last_fullsized":"2023-10-08T16:10:34.758225+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:41.953657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761446+0000","last_change":"2023-10-08T16:10:34.756033+0000","last_active":"2023-10-08T16:10:34.761446+0000","last_peered":"2023-10-08T16:10:34.761446+0000","last_clean":"2023-10-08T16:10:34.761446+0000","last_became_active":"2023-10-08T16:10:34.755594+0000","last_became_peered":"2023-10-08T16:10:34.755594+0000","last_unstale":"2023-10-08T16:10:34.761446+0000","last_undegraded":"2023-10-08T16:10:34.761446+0000","last_fullsized":"2023-10-08T16:10:34.761446+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777736+0000","last_change":"2023-10-08T16:10:34.770939+0000","last_active":"2023-10-08T16:10:34.777736+0000","last_peered":"2023-10-08T16:10:34.777736+0000","last_clean":"2023-10-08T16:10:34.777736+0000","last_became_active":"2023-10-08T16:10:34.768022+0000","last_became_peered":"2023-10-08T16:10:34.768022+0000","last_unstale":"2023-10-08T16:10:34.777736+0000","last_undegraded":"2023-10-08T16:10:34.777736+0000","last_fullsized":"2023-10-08T16:10:34.777736+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775692+0000","last_change":"2023-10-08T16:10:34.774404+0000","last_active":"2023-10-08T16:10:34.775692+0000","last_peered":"2023-10-08T16:10:34.775692+0000","last_clean":"2023-10-08T16:10:34.775692+0000","last_became_active":"2023-10-08T16:10:34.773063+0000","last_became_peered":"2023-10-08T16:10:34.773063+0000","last_unstale":"2023-10-08T16:10:34.775692+0000","last_undegraded":"2023-10-08T16:10:34.775692+0000","last_fullsized":"2023-10-08T16:10:34.775692+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772341+0000","last_change":"2023-10-08T16:10:34.770037+0000","last_active":"2023-10-08T16:10:34.772341+0000","last_peered":"2023-10-08T16:10:34.772341+0000","last_clean":"2023-10-08T16:10:34.772341+0000","last_became_active":"2023-10-08T16:10:34.769789+0000","last_became_peered":"2023-10-08T16:10:34.769789+0000","last_unstale":"2023-10-08T16:10:34.772341+0000","last_undegraded":"2023-10-08T16:10:34.772341+0000","last_fullsized":"2023-10-08T16:10:34.772341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:13.454151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777697+0000","last_change":"2023-10-08T16:10:34.776920+0000","last_active":"2023-10-08T16:10:34.777697+0000","last_peered":"2023-10-08T16:10:34.777697+0000","last_clean":"2023-10-08T16:10:34.777697+0000","last_became_active":"2023-10-08T16:10:34.776681+0000","last_became_peered":"2023-10-08T16:10:34.776681+0000","last_unstale":"2023-10-08T16:10:34.777697+0000","last_undegraded":"2023-10-08T16:10:34.777697+0000","last_fullsized":"2023-10-08T16:10:34.777697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:10.186542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777919+0000","last_change":"2023-10-08T16:10:34.767141+0000","last_active":"2023-10-08T16:10:34.777919+0000","last_peered":"2023-10-08T16:10:34.777919+0000","last_clean":"2023-10-08T16:10:34.777919+0000","last_became_active":"2023-10-08T16:10:34.766891+0000","last_became_peered":"2023-10-08T16:10:34.766891+0000","last_unstale":"2023-10-08T16:10:34.777919+0000","last_undegraded":"2023-10-08T16:10:34.777919+0000","last_fullsized":"2023-10-08T16:10:34.777919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772430+0000","last_change":"2023-10-08T16:10:34.770488+0000","last_active":"2023-10-08T16:10:34.772430+0000","last_peered":"2023-10-08T16:10:34.772430+0000","last_clean":"2023-10-08T16:10:34.772430+0000","last_became_active":"2023-10-08T16:10:34.769995+0000","last_became_peered":"2023-10-08T16:10:34.769995+0000","last_unstale":"2023-10-08T16:10:34.772430+0000","last_undegraded":"2023-10-08T16:10:34.772430+0000","last_fullsized":"2023-10-08T16:10:34.772430+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:36.437119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774957+0000","last_change":"2023-10-08T16:10:34.773518+0000","last_active":"2023-10-08T16:10:34.774957+0000","last_peered":"2023-10-08T16:10:34.774957+0000","last_clean":"2023-10-08T16:10:34.774957+0000","last_became_active":"2023-10-08T16:10:34.773131+0000","last_became_peered":"2023-10-08T16:10:34.773131+0000","last_unstale":"2023-10-08T16:10:34.774957+0000","last_undegraded":"2023-10-08T16:10:34.774957+0000","last_fullsized":"2023-10-08T16:10:34.774957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:55.347540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774012+0000","last_change":"2023-10-08T16:10:34.767502+0000","last_active":"2023-10-08T16:10:34.774012+0000","last_peered":"2023-10-08T16:10:34.774012+0000","last_clean":"2023-10-08T16:10:34.774012+0000","last_became_active":"2023-10-08T16:10:34.767202+0000","last_became_peered":"2023-10-08T16:10:34.767202+0000","last_unstale":"2023-10-08T16:10:34.774012+0000","last_undegraded":"2023-10-08T16:10:34.774012+0000","last_fullsized":"2023-10-08T16:10:34.774012+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.762670+0000","last_change":"2023-10-08T16:10:34.756490+0000","last_active":"2023-10-08T16:10:34.762670+0000","last_peered":"2023-10-08T16:10:34.762670+0000","last_clean":"2023-10-08T16:10:34.762670+0000","last_became_active":"2023-10-08T16:10:34.756220+0000","last_became_peered":"2023-10-08T16:10:34.756220+0000","last_unstale":"2023-10-08T16:10:34.762670+0000","last_undegraded":"2023-10-08T16:10:34.762670+0000","last_fullsized":"2023-10-08T16:10:34.762670+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778267+0000","last_change":"2023-10-08T16:10:34.777472+0000","last_active":"2023-10-08T16:10:34.778267+0000","last_peered":"2023-10-08T16:10:34.778267+0000","last_clean":"2023-10-08T16:10:34.778267+0000","last_became_active":"2023-10-08T16:10:34.777232+0000","last_became_peered":"2023-10-08T16:10:34.777232+0000","last_unstale":"2023-10-08T16:10:34.778267+0000","last_undegraded":"2023-10-08T16:10:34.778267+0000","last_fullsized":"2023-10-08T16:10:34.778267+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:23.635270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780757+0000","last_change":"2023-10-08T16:10:34.769074+0000","last_active":"2023-10-08T16:10:34.780757+0000","last_peered":"2023-10-08T16:10:34.780757+0000","last_clean":"2023-10-08T16:10:34.780757+0000","last_became_active":"2023-10-08T16:10:34.768606+0000","last_became_peered":"2023-10-08T16:10:34.768606+0000","last_unstale":"2023-10-08T16:10:34.780757+0000","last_undegraded":"2023-10-08T16:10:34.780757+0000","last_fullsized":"2023-10-08T16:10:34.780757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768168+0000","last_change":"2023-10-08T16:10:34.762792+0000","last_active":"2023-10-08T16:10:34.768168+0000","last_peered":"2023-10-08T16:10:34.768168+0000","last_clean":"2023-10-08T16:10:34.768168+0000","last_became_active":"2023-10-08T16:10:34.762518+0000","last_became_peered":"2023-10-08T16:10:34.762518+0000","last_unstale":"2023-10-08T16:10:34.768168+0000","last_undegraded":"2023-10-08T16:10:34.768168+0000","last_fullsized":"2023-10-08T16:10:34.768168+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:28.201365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.771552+0000","last_change":"2023-10-08T16:10:34.762905+0000","last_active":"2023-10-08T16:10:34.771552+0000","last_peered":"2023-10-08T16:10:34.771552+0000","last_clean":"2023-10-08T16:10:34.771552+0000","last_became_active":"2023-10-08T16:10:34.762592+0000","last_became_peered":"2023-10-08T16:10:34.762592+0000","last_unstale":"2023-10-08T16:10:34.771552+0000","last_undegraded":"2023-10-08T16:10:34.771552+0000","last_fullsized":"2023-10-08T16:10:34.771552+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:47:41.629319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781037+0000","last_change":"2023-10-08T16:10:34.773587+0000","last_active":"2023-10-08T16:10:34.781037+0000","last_peered":"2023-10-08T16:10:34.781037+0000","last_clean":"2023-10-08T16:10:34.781037+0000","last_became_active":"2023-10-08T16:10:34.773323+0000","last_became_peered":"2023-10-08T16:10:34.773323+0000","last_unstale":"2023-10-08T16:10:34.781037+0000","last_undegraded":"2023-10-08T16:10:34.781037+0000","last_fullsized":"2023-10-08T16:10:34.781037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780964+0000","last_change":"2023-10-08T16:10:34.770454+0000","last_active":"2023-10-08T16:10:34.780964+0000","last_peered":"2023-10-08T16:10:34.780964+0000","last_clean":"2023-10-08T16:10:34.780964+0000","last_became_active":"2023-10-08T16:10:34.769660+0000","last_became_peered":"2023-10-08T16:10:34.769660+0000","last_unstale":"2023-10-08T16:10:34.780964+0000","last_undegraded":"2023-10-08T16:10:34.780964+0000","last_fullsized":"2023-10-08T16:10:34.780964+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767096+0000","last_change":"2023-10-08T16:10:34.761459+0000","last_active":"2023-10-08T16:10:34.767096+0000","last_peered":"2023-10-08T16:10:34.767096+0000","last_clean":"2023-10-08T16:10:34.767096+0000","last_became_active":"2023-10-08T16:10:34.761215+0000","last_became_peered":"2023-10-08T16:10:34.761215+0000","last_unstale":"2023-10-08T16:10:34.767096+0000","last_undegraded":"2023-10-08T16:10:34.767096+0000","last_fullsized":"2023-10-08T16:10:34.767096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780968+0000","last_change":"2023-10-08T16:10:34.772442+0000","last_active":"2023-10-08T16:10:34.780968+0000","last_peered":"2023-10-08T16:10:34.780968+0000","last_clean":"2023-10-08T16:10:34.780968+0000","last_became_active":"2023-10-08T16:10:34.772168+0000","last_became_peered":"2023-10-08T16:10:34.772168+0000","last_unstale":"2023-10-08T16:10:34.780968+0000","last_undegraded":"2023-10-08T16:10:34.780968+0000","last_fullsized":"2023-10-08T16:10:34.780968+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777150+0000","last_change":"2023-10-08T16:10:34.774563+0000","last_active":"2023-10-08T16:10:34.777150+0000","last_peered":"2023-10-08T16:10:34.777150+0000","last_clean":"2023-10-08T16:10:34.777150+0000","last_became_active":"2023-10-08T16:10:34.774110+0000","last_became_peered":"2023-10-08T16:10:34.774110+0000","last_unstale":"2023-10-08T16:10:34.777150+0000","last_undegraded":"2023-10-08T16:10:34.777150+0000","last_fullsized":"2023-10-08T16:10:34.777150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:29.395963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780794+0000","last_change":"2023-10-08T16:10:34.769228+0000","last_active":"2023-10-08T16:10:34.780794+0000","last_peered":"2023-10-08T16:10:34.780794+0000","last_clean":"2023-10-08T16:10:34.780794+0000","last_became_active":"2023-10-08T16:10:34.768852+0000","last_became_peered":"2023-10-08T16:10:34.768852+0000","last_unstale":"2023-10-08T16:10:34.780794+0000","last_undegraded":"2023-10-08T16:10:34.780794+0000","last_fullsized":"2023-10-08T16:10:34.780794+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772265+0000","last_change":"2023-10-08T16:10:34.768805+0000","last_active":"2023-10-08T16:10:34.772265+0000","last_peered":"2023-10-08T16:10:34.772265+0000","last_clean":"2023-10-08T16:10:34.772265+0000","last_became_active":"2023-10-08T16:10:34.768420+0000","last_became_peered":"2023-10-08T16:10:34.768420+0000","last_unstale":"2023-10-08T16:10:34.772265+0000","last_undegraded":"2023-10-08T16:10:34.772265+0000","last_fullsized":"2023-10-08T16:10:34.772265+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:08.900903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.763786+0000","last_change":"2023-10-08T16:10:34.762065+0000","last_active":"2023-10-08T16:10:34.763786+0000","last_peered":"2023-10-08T16:10:34.763786+0000","last_clean":"2023-10-08T16:10:34.763786+0000","last_became_active":"2023-10-08T16:10:34.761777+0000","last_became_peered":"2023-10-08T16:10:34.761777+0000","last_unstale":"2023-10-08T16:10:34.763786+0000","last_undegraded":"2023-10-08T16:10:34.763786+0000","last_fullsized":"2023-10-08T16:10:34.763786+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:12.031551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768846+0000","last_change":"2023-10-08T16:10:34.767771+0000","last_active":"2023-10-08T16:10:34.768846+0000","last_peered":"2023-10-08T16:10:34.768846+0000","last_clean":"2023-10-08T16:10:34.768846+0000","last_became_active":"2023-10-08T16:10:34.766840+0000","last_became_peered":"2023-10-08T16:10:34.766840+0000","last_unstale":"2023-10-08T16:10:34.768846+0000","last_undegraded":"2023-10-08T16:10:34.768846+0000","last_fullsized":"2023-10-08T16:10:34.768846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:47:41.629319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781074+0000","last_change":"2023-10-08T16:10:34.776477+0000","last_active":"2023-10-08T16:10:34.781074+0000","last_peered":"2023-10-08T16:10:34.781074+0000","last_clean":"2023-10-08T16:10:34.781074+0000","last_became_active":"2023-10-08T16:10:34.776228+0000","last_became_peered":"2023-10-08T16:10:34.776228+0000","last_unstale":"2023-10-08T16:10:34.781074+0000","last_undegraded":"2023-10-08T16:10:34.781074+0000","last_fullsized":"2023-10-08T16:10:34.781074+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777824+0000","last_change":"2023-10-08T16:10:34.772998+0000","last_active":"2023-10-08T16:10:34.777824+0000","last_peered":"2023-10-08T16:10:34.777824+0000","last_clean":"2023-10-08T16:10:34.777824+0000","last_became_active":"2023-10-08T16:10:34.772746+0000","last_became_peered":"2023-10-08T16:10:34.772746+0000","last_unstale":"2023-10-08T16:10:34.777824+0000","last_undegraded":"2023-10-08T16:10:34.777824+0000","last_fullsized":"2023-10-08T16:10:34.777824+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772196+0000","last_change":"2023-10-08T16:10:34.767838+0000","last_active":"2023-10-08T16:10:34.772196+0000","last_peered":"2023-10-08T16:10:34.772196+0000","last_clean":"2023-10-08T16:10:34.772196+0000","last_became_active":"2023-10-08T16:10:34.767545+0000","last_became_peered":"2023-10-08T16:10:34.767545+0000","last_unstale":"2023-10-08T16:10:34.772196+0000","last_undegraded":"2023-10-08T16:10:34.772196+0000","last_fullsized":"2023-10-08T16:10:34.772196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:19.547237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761998+0000","last_change":"2023-10-08T16:10:34.755887+0000","last_active":"2023-10-08T16:10:34.761998+0000","last_peered":"2023-10-08T16:10:34.761998+0000","last_clean":"2023-10-08T16:10:34.761998+0000","last_became_active":"2023-10-08T16:10:34.755622+0000","last_became_peered":"2023-10-08T16:10:34.755622+0000","last_unstale":"2023-10-08T16:10:34.761998+0000","last_undegraded":"2023-10-08T16:10:34.761998+0000","last_fullsized":"2023-10-08T16:10:34.761998+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761679+0000","last_change":"2023-10-08T16:10:34.756936+0000","last_active":"2023-10-08T16:10:34.761679+0000","last_peered":"2023-10-08T16:10:34.761679+0000","last_clean":"2023-10-08T16:10:34.761679+0000","last_became_active":"2023-10-08T16:10:34.756710+0000","last_became_peered":"2023-10-08T16:10:34.756710+0000","last_unstale":"2023-10-08T16:10:34.761679+0000","last_undegraded":"2023-10-08T16:10:34.761679+0000","last_fullsized":"2023-10-08T16:10:34.761679+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:28.201365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775581+0000","last_change":"2023-10-08T16:10:33.766666+0000","last_active":"2023-10-08T16:10:34.775581+0000","last_peered":"2023-10-08T16:10:34.775581+0000","last_clean":"2023-10-08T16:10:34.775581+0000","last_became_active":"2023-10-08T16:10:33.764988+0000","last_became_peered":"2023-10-08T16:10:33.764988+0000","last_unstale":"2023-10-08T16:10:34.775581+0000","last_undegraded":"2023-10-08T16:10:34.775581+0000","last_fullsized":"2023-10-08T16:10:34.775581+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:49.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780579+0000","last_change":"2023-10-08T16:10:34.773742+0000","last_active":"2023-10-08T16:10:34.780579+0000","last_peered":"2023-10-08T16:10:34.780579+0000","last_clean":"2023-10-08T16:10:34.780579+0000","last_became_active":"2023-10-08T16:10:34.773425+0000","last_became_peered":"2023-10-08T16:10:34.773425+0000","last_unstale":"2023-10-08T16:10:34.780579+0000","last_undegraded":"2023-10-08T16:10:34.780579+0000","last_fullsized":"2023-10-08T16:10:34.780579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:12.031551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778396+0000","last_change":"2023-10-08T16:10:33.765370+0000","last_active":"2023-10-08T16:10:34.778396+0000","last_peered":"2023-10-08T16:10:34.778396+0000","last_clean":"2023-10-08T16:10:34.778396+0000","last_became_active":"2023-10-08T16:10:33.764896+0000","last_became_peered":"2023-10-08T16:10:33.764896+0000","last_unstale":"2023-10-08T16:10:34.778396+0000","last_undegraded":"2023-10-08T16:10:34.778396+0000","last_fullsized":"2023-10-08T16:10:34.778396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761341+0000","last_change":"2023-10-08T16:10:34.755901+0000","last_active":"2023-10-08T16:10:34.761341+0000","last_peered":"2023-10-08T16:10:34.761341+0000","last_clean":"2023-10-08T16:10:34.761341+0000","last_became_active":"2023-10-08T16:10:34.755587+0000","last_became_peered":"2023-10-08T16:10:34.755587+0000","last_unstale":"2023-10-08T16:10:34.761341+0000","last_undegraded":"2023-10-08T16:10:34.761341+0000","last_fullsized":"2023-10-08T16:10:34.761341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775050+0000","last_change":"2023-10-08T16:10:33.763286+0000","last_active":"2023-10-08T16:10:34.775050+0000","last_peered":"2023-10-08T16:10:34.775050+0000","last_clean":"2023-10-08T16:10:34.775050+0000","last_became_active":"2023-10-08T16:10:33.763028+0000","last_became_peered":"2023-10-08T16:10:33.763028+0000","last_unstale":"2023-10-08T16:10:34.775050+0000","last_undegraded":"2023-10-08T16:10:34.775050+0000","last_fullsized":"2023-10-08T16:10:34.775050+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:03.701535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.771589+0000","last_change":"2023-10-08T16:10:34.764169+0000","last_active":"2023-10-08T16:10:34.771589+0000","last_peered":"2023-10-08T16:10:34.771589+0000","last_clean":"2023-10-08T16:10:34.771589+0000","last_became_active":"2023-10-08T16:10:34.763157+0000","last_became_peered":"2023-10-08T16:10:34.763157+0000","last_unstale":"2023-10-08T16:10:34.771589+0000","last_undegraded":"2023-10-08T16:10:34.771589+0000","last_fullsized":"2023-10-08T16:10:34.771589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775000+0000","last_change":"2023-10-08T16:10:33.767700+0000","last_active":"2023-10-08T16:10:34.775000+0000","last_peered":"2023-10-08T16:10:34.775000+0000","last_clean":"2023-10-08T16:10:34.775000+0000","last_became_active":"2023-10-08T16:10:33.766926+0000","last_became_peered":"2023-10-08T16:10:33.766926+0000","last_unstale":"2023-10-08T16:10:34.775000+0000","last_undegraded":"2023-10-08T16:10:34.775000+0000","last_fullsized":"2023-10-08T16:10:34.775000+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:01.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772171+0000","last_change":"2023-10-08T16:10:34.766554+0000","last_active":"2023-10-08T16:10:34.772171+0000","last_peered":"2023-10-08T16:10:34.772171+0000","last_clean":"2023-10-08T16:10:34.772171+0000","last_became_active":"2023-10-08T16:10:34.766310+0000","last_became_peered":"2023-10-08T16:10:34.766310+0000","last_unstale":"2023-10-08T16:10:34.772171+0000","last_undegraded":"2023-10-08T16:10:34.772171+0000","last_fullsized":"2023-10-08T16:10:34.772171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:08.743491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767796+0000","last_change":"2023-10-08T16:10:33.760988+0000","last_active":"2023-10-08T16:10:34.767796+0000","last_peered":"2023-10-08T16:10:34.767796+0000","last_clean":"2023-10-08T16:10:34.767796+0000","last_became_active":"2023-10-08T16:10:33.760705+0000","last_became_peered":"2023-10-08T16:10:33.760705+0000","last_unstale":"2023-10-08T16:10:34.767796+0000","last_undegraded":"2023-10-08T16:10:34.767796+0000","last_fullsized":"2023-10-08T16:10:34.767796+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778063+0000","last_change":"2023-10-08T16:10:34.763970+0000","last_active":"2023-10-08T16:10:34.778063+0000","last_peered":"2023-10-08T16:10:34.778063+0000","last_clean":"2023-10-08T16:10:34.778063+0000","last_became_active":"2023-10-08T16:10:34.763645+0000","last_became_peered":"2023-10-08T16:10:34.763645+0000","last_unstale":"2023-10-08T16:10:34.778063+0000","last_undegraded":"2023-10-08T16:10:34.778063+0000","last_fullsized":"2023-10-08T16:10:34.778063+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772004+0000","last_change":"2023-10-08T16:10:33.765098+0000","last_active":"2023-10-08T16:10:34.772004+0000","last_peered":"2023-10-08T16:10:34.772004+0000","last_clean":"2023-10-08T16:10:34.772004+0000","last_became_active":"2023-10-08T16:10:33.764637+0000","last_became_peered":"2023-10-08T16:10:33.764637+0000","last_unstale":"2023-10-08T16:10:34.772004+0000","last_undegraded":"2023-10-08T16:10:34.772004+0000","last_fullsized":"2023-10-08T16:10:34.772004+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772209+0000","last_change":"2023-10-08T16:10:34.764753+0000","last_active":"2023-10-08T16:10:34.772209+0000","last_peered":"2023-10-08T16:10:34.772209+0000","last_clean":"2023-10-08T16:10:34.772209+0000","last_became_active":"2023-10-08T16:10:34.764527+0000","last_became_peered":"2023-10-08T16:10:34.764527+0000","last_unstale":"2023-10-08T16:10:34.772209+0000","last_undegraded":"2023-10-08T16:10:34.772209+0000","last_fullsized":"2023-10-08T16:10:34.772209+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:08:07.036302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757832+0000","last_change":"2023-10-08T16:10:33.761688+0000","last_active":"2023-10-08T16:10:34.757832+0000","last_peered":"2023-10-08T16:10:34.757832+0000","last_clean":"2023-10-08T16:10:34.757832+0000","last_became_active":"2023-10-08T16:10:33.761196+0000","last_became_peered":"2023-10-08T16:10:33.761196+0000","last_unstale":"2023-10-08T16:10:34.757832+0000","last_undegraded":"2023-10-08T16:10:34.757832+0000","last_fullsized":"2023-10-08T16:10:34.757832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:27.197085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777711+0000","last_change":"2023-10-08T16:10:34.768417+0000","last_active":"2023-10-08T16:10:34.777711+0000","last_peered":"2023-10-08T16:10:34.777711+0000","last_clean":"2023-10-08T16:10:34.777711+0000","last_became_active":"2023-10-08T16:10:34.767713+0000","last_became_peered":"2023-10-08T16:10:34.767713+0000","last_unstale":"2023-10-08T16:10:34.777711+0000","last_undegraded":"2023-10-08T16:10:34.777711+0000","last_fullsized":"2023-10-08T16:10:34.777711+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757922+0000","last_change":"2023-10-08T16:10:33.770625+0000","last_active":"2023-10-08T16:10:34.757922+0000","last_peered":"2023-10-08T16:10:34.757922+0000","last_clean":"2023-10-08T16:10:34.757922+0000","last_became_active":"2023-10-08T16:10:33.770366+0000","last_became_peered":"2023-10-08T16:10:33.770366+0000","last_unstale":"2023-10-08T16:10:34.757922+0000","last_undegraded":"2023-10-08T16:10:34.757922+0000","last_fullsized":"2023-10-08T16:10:34.757922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:29.471251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767917+0000","last_change":"2023-10-08T16:10:34.763954+0000","last_active":"2023-10-08T16:10:34.767917+0000","last_peered":"2023-10-08T16:10:34.767917+0000","last_clean":"2023-10-08T16:10:34.767917+0000","last_became_active":"2023-10-08T16:10:34.763611+0000","last_became_peered":"2023-10-08T16:10:34.763611+0000","last_unstale":"2023-10-08T16:10:34.767917+0000","last_undegraded":"2023-10-08T16:10:34.767917+0000","last_fullsized":"2023-10-08T16:10:34.767917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753181+0000","last_change":"2023-10-08T16:10:33.767567+0000","last_active":"2023-10-08T16:10:34.753181+0000","last_peered":"2023-10-08T16:10:34.753181+0000","last_clean":"2023-10-08T16:10:34.753181+0000","last_became_active":"2023-10-08T16:10:33.767321+0000","last_became_peered":"2023-10-08T16:10:33.767321+0000","last_unstale":"2023-10-08T16:10:34.753181+0000","last_undegraded":"2023-10-08T16:10:34.753181+0000","last_fullsized":"2023-10-08T16:10:34.753181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:13:23.776656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775279+0000","last_change":"2023-10-08T16:10:34.773581+0000","last_active":"2023-10-08T16:10:34.775279+0000","last_peered":"2023-10-08T16:10:34.775279+0000","last_clean":"2023-10-08T16:10:34.775279+0000","last_became_active":"2023-10-08T16:10:34.773300+0000","last_became_peered":"2023-10-08T16:10:34.773300+0000","last_unstale":"2023-10-08T16:10:34.775279+0000","last_undegraded":"2023-10-08T16:10:34.775279+0000","last_fullsized":"2023-10-08T16:10:34.775279+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753910+0000","last_change":"2023-10-08T16:10:33.764988+0000","last_active":"2023-10-08T16:10:34.753910+0000","last_peered":"2023-10-08T16:10:34.753910+0000","last_clean":"2023-10-08T16:10:34.753910+0000","last_became_active":"2023-10-08T16:10:33.764739+0000","last_became_peered":"2023-10-08T16:10:33.764739+0000","last_unstale":"2023-10-08T16:10:34.753910+0000","last_undegraded":"2023-10-08T16:10:34.753910+0000","last_fullsized":"2023-10-08T16:10:34.753910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768075+0000","last_change":"2023-10-08T16:10:34.763271+0000","last_active":"2023-10-08T16:10:34.768075+0000","last_peered":"2023-10-08T16:10:34.768075+0000","last_clean":"2023-10-08T16:10:34.768075+0000","last_became_active":"2023-10-08T16:10:34.763025+0000","last_became_peered":"2023-10-08T16:10:34.763025+0000","last_unstale":"2023-10-08T16:10:34.768075+0000","last_undegraded":"2023-10-08T16:10:34.768075+0000","last_fullsized":"2023-10-08T16:10:34.768075+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777653+0000","last_change":"2023-10-08T16:10:33.768462+0000","last_active":"2023-10-08T16:10:34.777653+0000","last_peered":"2023-10-08T16:10:34.777653+0000","last_clean":"2023-10-08T16:10:34.777653+0000","last_became_active":"2023-10-08T16:10:33.768242+0000","last_became_peered":"2023-10-08T16:10:33.768242+0000","last_unstale":"2023-10-08T16:10:34.777653+0000","last_undegraded":"2023-10-08T16:10:34.777653+0000","last_fullsized":"2023-10-08T16:10:34.777653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:59.803650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780607+0000","last_change":"2023-10-08T16:10:34.776374+0000","last_active":"2023-10-08T16:10:34.780607+0000","last_peered":"2023-10-08T16:10:34.780607+0000","last_clean":"2023-10-08T16:10:34.780607+0000","last_became_active":"2023-10-08T16:10:34.776089+0000","last_became_peered":"2023-10-08T16:10:34.776089+0000","last_unstale":"2023-10-08T16:10:34.780607+0000","last_undegraded":"2023-10-08T16:10:34.780607+0000","last_fullsized":"2023-10-08T16:10:34.780607+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781742+0000","last_change":"2023-10-08T16:10:33.773222+0000","last_active":"2023-10-08T16:10:34.781742+0000","last_peered":"2023-10-08T16:10:34.781742+0000","last_clean":"2023-10-08T16:10:34.781742+0000","last_became_active":"2023-10-08T16:10:33.772922+0000","last_became_peered":"2023-10-08T16:10:33.772922+0000","last_unstale":"2023-10-08T16:10:34.781742+0000","last_undegraded":"2023-10-08T16:10:34.781742+0000","last_fullsized":"2023-10-08T16:10:34.781742+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:07.790273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768050+0000","last_change":"2023-10-08T16:10:34.762452+0000","last_active":"2023-10-08T16:10:34.768050+0000","last_peered":"2023-10-08T16:10:34.768050+0000","last_clean":"2023-10-08T16:10:34.768050+0000","last_became_active":"2023-10-08T16:10:34.762210+0000","last_became_peered":"2023-10-08T16:10:34.762210+0000","last_unstale":"2023-10-08T16:10:34.768050+0000","last_undegraded":"2023-10-08T16:10:34.768050+0000","last_fullsized":"2023-10-08T16:10:34.768050+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:55.347540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775926+0000","last_change":"2023-10-08T16:10:33.762563+0000","last_active":"2023-10-08T16:10:34.775926+0000","last_peered":"2023-10-08T16:10:34.775926+0000","last_clean":"2023-10-08T16:10:34.775926+0000","last_became_active":"2023-10-08T16:10:33.762298+0000","last_became_peered":"2023-10-08T16:10:33.762298+0000","last_unstale":"2023-10-08T16:10:34.775926+0000","last_undegraded":"2023-10-08T16:10:34.775926+0000","last_fullsized":"2023-10-08T16:10:34.775926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:13:51.956683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777053+0000","last_change":"2023-10-08T16:10:34.775705+0000","last_active":"2023-10-08T16:10:34.777053+0000","last_peered":"2023-10-08T16:10:34.777053+0000","last_clean":"2023-10-08T16:10:34.777053+0000","last_became_active":"2023-10-08T16:10:34.775461+0000","last_became_peered":"2023-10-08T16:10:34.775461+0000","last_unstale":"2023-10-08T16:10:34.777053+0000","last_undegraded":"2023-10-08T16:10:34.777053+0000","last_fullsized":"2023-10-08T16:10:34.777053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752385+0000","last_change":"2023-10-08T16:10:33.766497+0000","last_active":"2023-10-08T16:10:34.752385+0000","last_peered":"2023-10-08T16:10:34.752385+0000","last_clean":"2023-10-08T16:10:34.752385+0000","last_became_active":"2023-10-08T16:10:33.766186+0000","last_became_peered":"2023-10-08T16:10:33.766186+0000","last_unstale":"2023-10-08T16:10:34.752385+0000","last_undegraded":"2023-10-08T16:10:34.752385+0000","last_fullsized":"2023-10-08T16:10:34.752385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:02.257047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.773664+0000","last_change":"2023-10-08T16:10:34.771712+0000","last_active":"2023-10-08T16:10:34.773664+0000","last_peered":"2023-10-08T16:10:34.773664+0000","last_clean":"2023-10-08T16:10:34.773664+0000","last_became_active":"2023-10-08T16:10:34.771444+0000","last_became_peered":"2023-10-08T16:10:34.771444+0000","last_unstale":"2023-10-08T16:10:34.773664+0000","last_undegraded":"2023-10-08T16:10:34.773664+0000","last_fullsized":"2023-10-08T16:10:34.773664+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778688+0000","last_change":"2023-10-08T16:10:33.770711+0000","last_active":"2023-10-08T16:10:34.778688+0000","last_peered":"2023-10-08T16:10:34.778688+0000","last_clean":"2023-10-08T16:10:34.778688+0000","last_became_active":"2023-10-08T16:10:33.770495+0000","last_became_peered":"2023-10-08T16:10:33.770495+0000","last_unstale":"2023-10-08T16:10:34.778688+0000","last_undegraded":"2023-10-08T16:10:34.778688+0000","last_fullsized":"2023-10-08T16:10:34.778688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:50.671846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774772+0000","last_change":"2023-10-08T16:10:34.768435+0000","last_active":"2023-10-08T16:10:34.774772+0000","last_peered":"2023-10-08T16:10:34.774772+0000","last_clean":"2023-10-08T16:10:34.774772+0000","last_became_active":"2023-10-08T16:10:34.768187+0000","last_became_peered":"2023-10-08T16:10:34.768187+0000","last_unstale":"2023-10-08T16:10:34.774772+0000","last_undegraded":"2023-10-08T16:10:34.774772+0000","last_fullsized":"2023-10-08T16:10:34.774772+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:55.082299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780677+0000","last_change":"2023-10-08T16:10:33.773350+0000","last_active":"2023-10-08T16:10:34.780677+0000","last_peered":"2023-10-08T16:10:34.780677+0000","last_clean":"2023-10-08T16:10:34.780677+0000","last_became_active":"2023-10-08T16:10:33.772989+0000","last_became_peered":"2023-10-08T16:10:33.772989+0000","last_unstale":"2023-10-08T16:10:34.780677+0000","last_undegraded":"2023-10-08T16:10:34.780677+0000","last_fullsized":"2023-10-08T16:10:34.780677+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:18.004513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774952+0000","last_change":"2023-10-08T16:10:34.772942+0000","last_active":"2023-10-08T16:10:34.774952+0000","last_peered":"2023-10-08T16:10:34.774952+0000","last_clean":"2023-10-08T16:10:34.774952+0000","last_became_active":"2023-10-08T16:10:34.772686+0000","last_became_peered":"2023-10-08T16:10:34.772686+0000","last_unstale":"2023-10-08T16:10:34.774952+0000","last_undegraded":"2023-10-08T16:10:34.774952+0000","last_fullsized":"2023-10-08T16:10:34.774952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780988+0000","last_change":"2023-10-08T16:10:33.760706+0000","last_active":"2023-10-08T16:10:34.780988+0000","last_peered":"2023-10-08T16:10:34.780988+0000","last_clean":"2023-10-08T16:10:34.780988+0000","last_became_active":"2023-10-08T16:10:33.760439+0000","last_became_peered":"2023-10-08T16:10:33.760439+0000","last_unstale":"2023-10-08T16:10:34.780988+0000","last_undegraded":"2023-10-08T16:10:34.780988+0000","last_fullsized":"2023-10-08T16:10:34.780988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:34.341551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774667+0000","last_change":"2023-10-08T16:10:34.771623+0000","last_active":"2023-10-08T16:10:34.774667+0000","last_peered":"2023-10-08T16:10:34.774667+0000","last_clean":"2023-10-08T16:10:34.774667+0000","last_became_active":"2023-10-08T16:10:34.770995+0000","last_became_peered":"2023-10-08T16:10:34.770995+0000","last_unstale":"2023-10-08T16:10:34.774667+0000","last_undegraded":"2023-10-08T16:10:34.774667+0000","last_fullsized":"2023-10-08T16:10:34.774667+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772729+0000","last_change":"2023-10-08T16:10:33.764394+0000","last_active":"2023-10-08T16:10:34.772729+0000","last_peered":"2023-10-08T16:10:34.772729+0000","last_clean":"2023-10-08T16:10:34.772729+0000","last_became_active":"2023-10-08T16:10:33.764077+0000","last_became_peered":"2023-10-08T16:10:33.764077+0000","last_unstale":"2023-10-08T16:10:34.772729+0000","last_undegraded":"2023-10-08T16:10:34.772729+0000","last_fullsized":"2023-10-08T16:10:34.772729+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:27.558248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775010+0000","last_change":"2023-10-08T16:10:34.769083+0000","last_active":"2023-10-08T16:10:34.775010+0000","last_peered":"2023-10-08T16:10:34.775010+0000","last_clean":"2023-10-08T16:10:34.775010+0000","last_became_active":"2023-10-08T16:10:34.768807+0000","last_became_peered":"2023-10-08T16:10:34.768807+0000","last_unstale":"2023-10-08T16:10:34.775010+0000","last_undegraded":"2023-10-08T16:10:34.775010+0000","last_fullsized":"2023-10-08T16:10:34.775010+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:29.395963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.756380+0000","last_change":"2023-10-08T16:10:33.773846+0000","last_active":"2023-10-08T16:10:34.756380+0000","last_peered":"2023-10-08T16:10:34.756380+0000","last_clean":"2023-10-08T16:10:34.756380+0000","last_became_active":"2023-10-08T16:10:33.773563+0000","last_became_peered":"2023-10-08T16:10:33.773563+0000","last_unstale":"2023-10-08T16:10:34.756380+0000","last_undegraded":"2023-10-08T16:10:34.756380+0000","last_fullsized":"2023-10-08T16:10:34.756380+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:58:45.919412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774626+0000","last_change":"2023-10-08T16:10:34.770945+0000","last_active":"2023-10-08T16:10:34.774626+0000","last_peered":"2023-10-08T16:10:34.774626+0000","last_clean":"2023-10-08T16:10:34.774626+0000","last_became_active":"2023-10-08T16:10:34.770681+0000","last_became_peered":"2023-10-08T16:10:34.770681+0000","last_unstale":"2023-10-08T16:10:34.774626+0000","last_undegraded":"2023-10-08T16:10:34.774626+0000","last_fullsized":"2023-10-08T16:10:34.774626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775711+0000","last_change":"2023-10-08T16:10:33.759531+0000","last_active":"2023-10-08T16:10:34.775711+0000","last_peered":"2023-10-08T16:10:34.775711+0000","last_clean":"2023-10-08T16:10:34.775711+0000","last_became_active":"2023-10-08T16:10:33.759312+0000","last_became_peered":"2023-10-08T16:10:33.759312+0000","last_unstale":"2023-10-08T16:10:34.775711+0000","last_undegraded":"2023-10-08T16:10:34.775711+0000","last_fullsized":"2023-10-08T16:10:34.775711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777838+0000","last_change":"2023-10-08T16:10:34.776952+0000","last_active":"2023-10-08T16:10:34.777838+0000","last_peered":"2023-10-08T16:10:34.777838+0000","last_clean":"2023-10-08T16:10:34.777838+0000","last_became_active":"2023-10-08T16:10:34.776702+0000","last_became_peered":"2023-10-08T16:10:34.776702+0000","last_unstale":"2023-10-08T16:10:34.777838+0000","last_undegraded":"2023-10-08T16:10:34.777838+0000","last_fullsized":"2023-10-08T16:10:34.777838+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:48.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752982+0000","last_change":"2023-10-08T16:10:33.768853+0000","last_active":"2023-10-08T16:10:34.752982+0000","last_peered":"2023-10-08T16:10:34.752982+0000","last_clean":"2023-10-08T16:10:34.752982+0000","last_became_active":"2023-10-08T16:10:33.768563+0000","last_became_peered":"2023-10-08T16:10:33.768563+0000","last_unstale":"2023-10-08T16:10:34.752982+0000","last_undegraded":"2023-10-08T16:10:34.752982+0000","last_fullsized":"2023-10-08T16:10:34.752982+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:57.969527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777046+0000","last_change":"2023-10-08T16:10:34.774673+0000","last_active":"2023-10-08T16:10:34.777046+0000","last_peered":"2023-10-08T16:10:34.777046+0000","last_clean":"2023-10-08T16:10:34.777046+0000","last_became_active":"2023-10-08T16:10:34.773949+0000","last_became_peered":"2023-10-08T16:10:34.773949+0000","last_unstale":"2023-10-08T16:10:34.777046+0000","last_undegraded":"2023-10-08T16:10:34.777046+0000","last_fullsized":"2023-10-08T16:10:34.777046+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752225+0000","last_change":"2023-10-08T16:10:33.761930+0000","last_active":"2023-10-08T16:10:34.752225+0000","last_peered":"2023-10-08T16:10:34.752225+0000","last_clean":"2023-10-08T16:10:34.752225+0000","last_became_active":"2023-10-08T16:10:33.761680+0000","last_became_peered":"2023-10-08T16:10:33.761680+0000","last_unstale":"2023-10-08T16:10:34.752225+0000","last_undegraded":"2023-10-08T16:10:34.752225+0000","last_fullsized":"2023-10-08T16:10:34.752225+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768423+0000","last_change":"2023-10-08T16:10:34.763880+0000","last_active":"2023-10-08T16:10:34.768423+0000","last_peered":"2023-10-08T16:10:34.768423+0000","last_clean":"2023-10-08T16:10:34.768423+0000","last_became_active":"2023-10-08T16:10:34.763182+0000","last_became_peered":"2023-10-08T16:10:34.763182+0000","last_unstale":"2023-10-08T16:10:34.768423+0000","last_undegraded":"2023-10-08T16:10:34.768423+0000","last_fullsized":"2023-10-08T16:10:34.768423+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758069+0000","last_change":"2023-10-08T16:10:33.766371+0000","last_active":"2023-10-08T16:10:34.758069+0000","last_peered":"2023-10-08T16:10:34.758069+0000","last_clean":"2023-10-08T16:10:34.758069+0000","last_became_active":"2023-10-08T16:10:33.766082+0000","last_became_peered":"2023-10-08T16:10:33.766082+0000","last_unstale":"2023-10-08T16:10:34.758069+0000","last_undegraded":"2023-10-08T16:10:34.758069+0000","last_fullsized":"2023-10-08T16:10:34.758069+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:58.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775617+0000","last_change":"2023-10-08T16:10:34.773031+0000","last_active":"2023-10-08T16:10:34.775617+0000","last_peered":"2023-10-08T16:10:34.775617+0000","last_clean":"2023-10-08T16:10:34.775617+0000","last_became_active":"2023-10-08T16:10:34.772788+0000","last_became_peered":"2023-10-08T16:10:34.772788+0000","last_unstale":"2023-10-08T16:10:34.775617+0000","last_undegraded":"2023-10-08T16:10:34.775617+0000","last_fullsized":"2023-10-08T16:10:34.775617+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772738+0000","last_change":"2023-10-08T16:10:33.770919+0000","last_active":"2023-10-08T16:10:34.772738+0000","last_peered":"2023-10-08T16:10:34.772738+0000","last_clean":"2023-10-08T16:10:34.772738+0000","last_became_active":"2023-10-08T16:10:33.770608+0000","last_became_peered":"2023-10-08T16:10:33.770608+0000","last_unstale":"2023-10-08T16:10:34.772738+0000","last_undegraded":"2023-10-08T16:10:34.772738+0000","last_fullsized":"2023-10-08T16:10:34.772738+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:37.099695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.779768+0000","last_change":"2023-10-08T16:10:34.779313+0000","last_active":"2023-10-08T16:10:34.779768+0000","last_peered":"2023-10-08T16:10:34.779768+0000","last_clean":"2023-10-08T16:10:34.779768+0000","last_became_active":"2023-10-08T16:10:34.779078+0000","last_became_peered":"2023-10-08T16:10:34.779078+0000","last_unstale":"2023-10-08T16:10:34.779768+0000","last_undegraded":"2023-10-08T16:10:34.779768+0000","last_fullsized":"2023-10-08T16:10:34.779768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:04.168085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753265+0000","last_change":"2023-10-08T16:10:33.766192+0000","last_active":"2023-10-08T16:10:34.753265+0000","last_peered":"2023-10-08T16:10:34.753265+0000","last_clean":"2023-10-08T16:10:34.753265+0000","last_became_active":"2023-10-08T16:10:33.765245+0000","last_became_peered":"2023-10-08T16:10:33.765245+0000","last_unstale":"2023-10-08T16:10:34.753265+0000","last_undegraded":"2023-10-08T16:10:34.753265+0000","last_fullsized":"2023-10-08T16:10:34.753265+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:49.157336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772212+0000","last_change":"2023-10-08T16:10:34.765909+0000","last_active":"2023-10-08T16:10:34.772212+0000","last_peered":"2023-10-08T16:10:34.772212+0000","last_clean":"2023-10-08T16:10:34.772212+0000","last_became_active":"2023-10-08T16:10:34.765658+0000","last_became_peered":"2023-10-08T16:10:34.765658+0000","last_unstale":"2023-10-08T16:10:34.772212+0000","last_undegraded":"2023-10-08T16:10:34.772212+0000","last_fullsized":"2023-10-08T16:10:34.772212+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:46.612006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757102+0000","last_change":"2023-10-08T16:10:33.766948+0000","last_active":"2023-10-08T16:10:34.757102+0000","last_peered":"2023-10-08T16:10:34.757102+0000","last_clean":"2023-10-08T16:10:34.757102+0000","last_became_active":"2023-10-08T16:10:33.766644+0000","last_became_peered":"2023-10-08T16:10:33.766644+0000","last_unstale":"2023-10-08T16:10:34.757102+0000","last_undegraded":"2023-10-08T16:10:34.757102+0000","last_fullsized":"2023-10-08T16:10:34.757102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:54.343260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768058+0000","last_change":"2023-10-08T16:10:34.763318+0000","last_active":"2023-10-08T16:10:34.768058+0000","last_peered":"2023-10-08T16:10:34.768058+0000","last_clean":"2023-10-08T16:10:34.768058+0000","last_became_active":"2023-10-08T16:10:34.762522+0000","last_became_peered":"2023-10-08T16:10:34.762522+0000","last_unstale":"2023-10-08T16:10:34.768058+0000","last_undegraded":"2023-10-08T16:10:34.768058+0000","last_fullsized":"2023-10-08T16:10:34.768058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.754375+0000","last_change":"2023-10-08T16:10:33.762562+0000","last_active":"2023-10-08T16:10:34.754375+0000","last_peered":"2023-10-08T16:10:34.754375+0000","last_clean":"2023-10-08T16:10:34.754375+0000","last_became_active":"2023-10-08T16:10:33.762261+0000","last_became_peered":"2023-10-08T16:10:33.762261+0000","last_unstale":"2023-10-08T16:10:34.754375+0000","last_undegraded":"2023-10-08T16:10:34.754375+0000","last_fullsized":"2023-10-08T16:10:34.754375+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:18.712472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780921+0000","last_change":"2023-10-08T16:10:34.770209+0000","last_active":"2023-10-08T16:10:34.780921+0000","last_peered":"2023-10-08T16:10:34.780921+0000","last_clean":"2023-10-08T16:10:34.780921+0000","last_became_active":"2023-10-08T16:10:34.769625+0000","last_became_peered":"2023-10-08T16:10:34.769625+0000","last_unstale":"2023-10-08T16:10:34.780921+0000","last_undegraded":"2023-10-08T16:10:34.780921+0000","last_fullsized":"2023-10-08T16:10:34.780921+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.755134+0000","last_change":"2023-10-08T16:10:33.765165+0000","last_active":"2023-10-08T16:10:34.755134+0000","last_peered":"2023-10-08T16:10:34.755134+0000","last_clean":"2023-10-08T16:10:34.755134+0000","last_became_active":"2023-10-08T16:10:33.764738+0000","last_became_peered":"2023-10-08T16:10:33.764738+0000","last_unstale":"2023-10-08T16:10:34.755134+0000","last_undegraded":"2023-10-08T16:10:34.755134+0000","last_fullsized":"2023-10-08T16:10:34.755134+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780945+0000","last_change":"2023-10-08T16:10:34.769972+0000","last_active":"2023-10-08T16:10:34.780945+0000","last_peered":"2023-10-08T16:10:34.780945+0000","last_clean":"2023-10-08T16:10:34.780945+0000","last_became_active":"2023-10-08T16:10:34.769590+0000","last_became_peered":"2023-10-08T16:10:34.769590+0000","last_unstale":"2023-10-08T16:10:34.780945+0000","last_undegraded":"2023-10-08T16:10:34.780945+0000","last_fullsized":"2023-10-08T16:10:34.780945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757255+0000","last_change":"2023-10-08T16:10:33.755373+0000","last_active":"2023-10-08T16:10:34.757255+0000","last_peered":"2023-10-08T16:10:34.757255+0000","last_clean":"2023-10-08T16:10:34.757255+0000","last_became_active":"2023-10-08T16:10:33.755019+0000","last_became_peered":"2023-10-08T16:10:33.755019+0000","last_unstale":"2023-10-08T16:10:34.757255+0000","last_undegraded":"2023-10-08T16:10:34.757255+0000","last_fullsized":"2023-10-08T16:10:34.757255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:32.618356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777732+0000","last_change":"2023-10-08T16:10:34.775514+0000","last_active":"2023-10-08T16:10:34.777732+0000","last_peered":"2023-10-08T16:10:34.777732+0000","last_clean":"2023-10-08T16:10:34.777732+0000","last_became_active":"2023-10-08T16:10:34.775065+0000","last_became_peered":"2023-10-08T16:10:34.775065+0000","last_unstale":"2023-10-08T16:10:34.777732+0000","last_undegraded":"2023-10-08T16:10:34.777732+0000","last_fullsized":"2023-10-08T16:10:34.777732+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752876+0000","last_change":"2023-10-08T16:10:33.767441+0000","last_active":"2023-10-08T16:10:34.752876+0000","last_peered":"2023-10-08T16:10:34.752876+0000","last_clean":"2023-10-08T16:10:34.752876+0000","last_became_active":"2023-10-08T16:10:33.767184+0000","last_became_peered":"2023-10-08T16:10:33.767184+0000","last_unstale":"2023-10-08T16:10:34.752876+0000","last_undegraded":"2023-10-08T16:10:34.752876+0000","last_fullsized":"2023-10-08T16:10:34.752876+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:44.856255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772022+0000","last_change":"2023-10-08T16:10:34.764706+0000","last_active":"2023-10-08T16:10:34.772022+0000","last_peered":"2023-10-08T16:10:34.772022+0000","last_clean":"2023-10-08T16:10:34.772022+0000","last_became_active":"2023-10-08T16:10:34.764458+0000","last_became_peered":"2023-10-08T16:10:34.764458+0000","last_unstale":"2023-10-08T16:10:34.772022+0000","last_undegraded":"2023-10-08T16:10:34.772022+0000","last_fullsized":"2023-10-08T16:10:34.772022+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:53:39.219498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758012+0000","last_change":"2023-10-08T16:10:33.766449+0000","last_active":"2023-10-08T16:10:34.758012+0000","last_peered":"2023-10-08T16:10:34.758012+0000","last_clean":"2023-10-08T16:10:34.758012+0000","last_became_active":"2023-10-08T16:10:33.766244+0000","last_became_peered":"2023-10-08T16:10:33.766244+0000","last_unstale":"2023-10-08T16:10:34.758012+0000","last_undegraded":"2023-10-08T16:10:34.758012+0000","last_fullsized":"2023-10-08T16:10:34.758012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:02.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777166+0000","last_change":"2023-10-08T16:10:34.774688+0000","last_active":"2023-10-08T16:10:34.777166+0000","last_peered":"2023-10-08T16:10:34.777166+0000","last_clean":"2023-10-08T16:10:34.777166+0000","last_became_active":"2023-10-08T16:10:34.774416+0000","last_became_peered":"2023-10-08T16:10:34.774416+0000","last_unstale":"2023-10-08T16:10:34.777166+0000","last_undegraded":"2023-10-08T16:10:34.777166+0000","last_fullsized":"2023-10-08T16:10:34.777166+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.756007+0000","last_change":"2023-10-08T16:10:33.773844+0000","last_active":"2023-10-08T16:10:34.756007+0000","last_peered":"2023-10-08T16:10:34.756007+0000","last_clean":"2023-10-08T16:10:34.756007+0000","last_became_active":"2023-10-08T16:10:33.773481+0000","last_became_peered":"2023-10-08T16:10:33.773481+0000","last_unstale":"2023-10-08T16:10:34.756007+0000","last_undegraded":"2023-10-08T16:10:34.756007+0000","last_fullsized":"2023-10-08T16:10:34.756007+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:41.626242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.759908+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:34.759908+0000","last_peered":"2023-10-08T16:10:34.759908+0000","last_clean":"2023-10-08T16:10:34.759908+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:34.759908+0000","last_undegraded":"2023-10-08T16:10:34.759908+0000","last_fullsized":"2023-10-08T16:10:34.759908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299104,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072736,"statfs":{"total":96636764160,"available":96330481664,"internally_reserved":0,"allocated":1859584,"data_stored":1133614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.51400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":1.677},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.55100000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.629},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":542728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.621}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302620,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069220,"statfs":{"total":96636764160,"available":96326881280,"internally_reserved":0,"allocated":1265664,"data_stored":542782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.371},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303212,"kb_used_data":1828,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068628,"statfs":{"total":96636764160,"available":96326275072,"internally_reserved":0,"allocated":1871872,"data_stored":1134674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.314},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302612,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069228,"statfs":{"total":96636764160,"available":96326889472,"internally_reserved":0,"allocated":1257472,"data_stored":542658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.314},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.69599999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302628,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069212,"statfs":{"total":96636764160,"available":96326873088,"internally_reserved":0,"allocated":1273856,"data_stored":544264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.56699999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298516,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073324,"statfs":{"total":96636764160,"available":96331083776,"internally_reserved":0,"allocated":1257472,"data_stored":542658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.32000000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.497}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":543234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.76500000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.747}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.626}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.69799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:43.638 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:10:43.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.765+0000 7fcf2f59e700 1 Processor -- start 2023-10-08T16:10:43.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.765+0000 7fcf2f59e700 1 -- start start 2023-10-08T16:10:43.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2f59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcf301597f0 0x7fcf300b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2f59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3015dbb0 con 0x7fcf30159420 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2f59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf300b11d0 con 0x7fcf301597f0 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300b14b0 con 0x7fcf3015d230 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf25d9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcf301597f0 0x7fcf300b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2ed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:40598/0 (socket says 172.21.15.170:40598) 2023-10-08T16:10:43.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.766+0000 7fcf2ed9d700 1 -- 172.21.15.170:0/3962815942 learned_addr learned my addr 172.21.15.170:0/3962815942 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:43.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf25d9b700 1 -- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 msgr2=0x7fcf3015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:43.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf25d9b700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf25d9b700 1 -- 172.21.15.170:0/3962815942 >> v1:172.21.15.191:6789/0 conn(0x7fcf30159420 legacy=0x7fcf300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf25d9b700 1 -- 172.21.15.170:0/3962815942 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300b1790 con 0x7fcf301597f0 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf2ed9d700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf25d9b700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300b2270 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf180098e0 tx=0x7fcf180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9871472a882bb9b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf2dd9b700 1 -- 172.21.15.170:0/3962815942 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800b640 con 0x7fcf301597f0 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf2dd9b700 1 -- 172.21.15.170:0/3962815942 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1800b7a0 con 0x7fcf301597f0 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.767+0000 7fcf2dd9b700 1 -- 172.21.15.170:0/3962815942 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1800bb40 con 0x7fcf301597f0 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 msgr2=0x7fcf300b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300b2270 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcf180098e0 tx=0x7fcf180091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3962815942 shutdown_connections 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf3015d230 0x7fcf3015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3962815942 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300b2270 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3962815942 >> 172.21.15.170:0/3962815942 conn(0x7fcf300d0660 msgr2=0x7fcf300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:43.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3962815942 shutdown_connections 2023-10-08T16:10:43.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.768+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3962815942 wait complete. 2023-10-08T16:10:43.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.769+0000 7fcf2f59e700 1 Processor -- start 2023-10-08T16:10:43.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.769+0000 7fcf2f59e700 1 -- start start 2023-10-08T16:10:43.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf30159420 0x7fcf300ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c7b90 con 0x7fcf3015d230 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3016a360 con 0x7fcf301597f0 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3016a640 con 0x7fcf30159420 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38082/0 (socket says 172.21.15.170:38082) 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 -- 172.21.15.170:0/3441442488 learned_addr learned my addr 172.21.15.170:0/3441442488 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf25d9b700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2e59c700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf30159420 0x7fcf300ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 -- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf30159420 msgr2=0x7fcf300ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf30159420 0x7fcf300ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 -- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 msgr2=0x7fcf300c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 -- 172.21.15.170:0/3441442488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009ae0 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf2ed9d700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009f30 tx=0x7fcf2800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.770+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28018070 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.771+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2800f040 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.771+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf18009420 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.771+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800cb90 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.771+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3016a020 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.772+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf300d0430 con 0x7fcf3015d230 2023-10-08T16:10:43.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.772+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fcf28003020 con 0x7fcf3015d230 2023-10-08T16:10:43.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.772+0000 7fcf26ffd700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 0x7fcf10069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:43.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.772+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcf280848a0 con 0x7fcf3015d230 2023-10-08T16:10:43.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.773+0000 7fcf2e59c700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 0x7fcf10069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:43.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.773+0000 7fcf2e59c700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 0x7fcf10069770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf30157a50 tx=0x7fcf20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:43.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.777+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf2804d5a0 con 0x7fcf3015d230 2023-10-08T16:10:43.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.954+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf3016aa80 con 0x7fcf100672c0 2023-10-08T16:10:43.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.962+0000 7fcf26ffd700 1 -- 172.21.15.170:0/3441442488 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289621 (secure 0 0 0) 0x7fcf200015e0 con 0x7fcf100672c0 2023-10-08T16:10:43.966 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:43.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 msgr2=0x7fcf10069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 0x7fcf10069770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf30157a50 tx=0x7fcf20009040 comp rx=0 tx=0).stop 2023-10-08T16:10:43.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 msgr2=0x7fcf30169ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:43.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009f30 tx=0x7fcf2800b040 comp rx=0 tx=0).stop 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 shutdown_connections 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcf100672c0 0x7fcf10069770 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf3015d230 0x7fcf30169ae0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcf30159420 0x7fcf300ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 --2- 172.21.15.170:0/3441442488 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcf301597f0 0x7fcf300c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 >> 172.21.15.170:0/3441442488 conn(0x7fcf300d0660 msgr2=0x7fcf300b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 shutdown_connections 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:43.966+0000 7fcf2f59e700 1 -- 172.21.15.170:0/3441442488 wait complete. 2023-10-08T16:10:43.969 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:10:43.978 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2023-10-08T16:10:42.299444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2405736,"kb_used_data":11048,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752568984,"statfs":{"total":773094113280,"available":770630639616,"internally_reserved":0,"allocated":11313152,"data_stored":5526612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001717"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.776945+0000","last_change":"2023-10-08T16:10:34.776241+0000","last_active":"2023-10-08T16:10:34.776945+0000","last_peered":"2023-10-08T16:10:34.776945+0000","last_clean":"2023-10-08T16:10:34.776945+0000","last_became_active":"2023-10-08T16:10:34.773348+0000","last_became_peered":"2023-10-08T16:10:34.773348+0000","last_unstale":"2023-10-08T16:10:34.776945+0000","last_undegraded":"2023-10-08T16:10:34.776945+0000","last_fullsized":"2023-10-08T16:10:34.776945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:09.090277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777902+0000","last_change":"2023-10-08T16:10:34.762496+0000","last_active":"2023-10-08T16:10:34.777902+0000","last_peered":"2023-10-08T16:10:34.777902+0000","last_clean":"2023-10-08T16:10:34.777902+0000","last_became_active":"2023-10-08T16:10:34.762209+0000","last_became_peered":"2023-10-08T16:10:34.762209+0000","last_unstale":"2023-10-08T16:10:34.777902+0000","last_undegraded":"2023-10-08T16:10:34.777902+0000","last_fullsized":"2023-10-08T16:10:34.777902+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:26:41.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775031+0000","last_change":"2023-10-08T16:10:34.770582+0000","last_active":"2023-10-08T16:10:34.775031+0000","last_peered":"2023-10-08T16:10:34.775031+0000","last_clean":"2023-10-08T16:10:34.775031+0000","last_became_active":"2023-10-08T16:10:34.770264+0000","last_became_peered":"2023-10-08T16:10:34.770264+0000","last_unstale":"2023-10-08T16:10:34.775031+0000","last_undegraded":"2023-10-08T16:10:34.775031+0000","last_fullsized":"2023-10-08T16:10:34.775031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:41.197866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772448+0000","last_change":"2023-10-08T16:10:34.770983+0000","last_active":"2023-10-08T16:10:34.772448+0000","last_peered":"2023-10-08T16:10:34.772448+0000","last_clean":"2023-10-08T16:10:34.772448+0000","last_became_active":"2023-10-08T16:10:34.770747+0000","last_became_peered":"2023-10-08T16:10:34.770747+0000","last_unstale":"2023-10-08T16:10:34.772448+0000","last_undegraded":"2023-10-08T16:10:34.772448+0000","last_fullsized":"2023-10-08T16:10:34.772448+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:40.928641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758225+0000","last_change":"2023-10-08T16:10:34.755991+0000","last_active":"2023-10-08T16:10:34.758225+0000","last_peered":"2023-10-08T16:10:34.758225+0000","last_clean":"2023-10-08T16:10:34.758225+0000","last_became_active":"2023-10-08T16:10:34.755732+0000","last_became_peered":"2023-10-08T16:10:34.755732+0000","last_unstale":"2023-10-08T16:10:34.758225+0000","last_undegraded":"2023-10-08T16:10:34.758225+0000","last_fullsized":"2023-10-08T16:10:34.758225+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:41.953657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761446+0000","last_change":"2023-10-08T16:10:34.756033+0000","last_active":"2023-10-08T16:10:34.761446+0000","last_peered":"2023-10-08T16:10:34.761446+0000","last_clean":"2023-10-08T16:10:34.761446+0000","last_became_active":"2023-10-08T16:10:34.755594+0000","last_became_peered":"2023-10-08T16:10:34.755594+0000","last_unstale":"2023-10-08T16:10:34.761446+0000","last_undegraded":"2023-10-08T16:10:34.761446+0000","last_fullsized":"2023-10-08T16:10:34.761446+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:34:01.127874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777736+0000","last_change":"2023-10-08T16:10:34.770939+0000","last_active":"2023-10-08T16:10:34.777736+0000","last_peered":"2023-10-08T16:10:34.777736+0000","last_clean":"2023-10-08T16:10:34.777736+0000","last_became_active":"2023-10-08T16:10:34.768022+0000","last_became_peered":"2023-10-08T16:10:34.768022+0000","last_unstale":"2023-10-08T16:10:34.777736+0000","last_undegraded":"2023-10-08T16:10:34.777736+0000","last_fullsized":"2023-10-08T16:10:34.777736+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:28.282751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775692+0000","last_change":"2023-10-08T16:10:34.774404+0000","last_active":"2023-10-08T16:10:34.775692+0000","last_peered":"2023-10-08T16:10:34.775692+0000","last_clean":"2023-10-08T16:10:34.775692+0000","last_became_active":"2023-10-08T16:10:34.773063+0000","last_became_peered":"2023-10-08T16:10:34.773063+0000","last_unstale":"2023-10-08T16:10:34.775692+0000","last_undegraded":"2023-10-08T16:10:34.775692+0000","last_fullsized":"2023-10-08T16:10:34.775692+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772341+0000","last_change":"2023-10-08T16:10:34.770037+0000","last_active":"2023-10-08T16:10:34.772341+0000","last_peered":"2023-10-08T16:10:34.772341+0000","last_clean":"2023-10-08T16:10:34.772341+0000","last_became_active":"2023-10-08T16:10:34.769789+0000","last_became_peered":"2023-10-08T16:10:34.769789+0000","last_unstale":"2023-10-08T16:10:34.772341+0000","last_undegraded":"2023-10-08T16:10:34.772341+0000","last_fullsized":"2023-10-08T16:10:34.772341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:13.454151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777697+0000","last_change":"2023-10-08T16:10:34.776920+0000","last_active":"2023-10-08T16:10:34.777697+0000","last_peered":"2023-10-08T16:10:34.777697+0000","last_clean":"2023-10-08T16:10:34.777697+0000","last_became_active":"2023-10-08T16:10:34.776681+0000","last_became_peered":"2023-10-08T16:10:34.776681+0000","last_unstale":"2023-10-08T16:10:34.777697+0000","last_undegraded":"2023-10-08T16:10:34.777697+0000","last_fullsized":"2023-10-08T16:10:34.777697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:10.186542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777919+0000","last_change":"2023-10-08T16:10:34.767141+0000","last_active":"2023-10-08T16:10:34.777919+0000","last_peered":"2023-10-08T16:10:34.777919+0000","last_clean":"2023-10-08T16:10:34.777919+0000","last_became_active":"2023-10-08T16:10:34.766891+0000","last_became_peered":"2023-10-08T16:10:34.766891+0000","last_unstale":"2023-10-08T16:10:34.777919+0000","last_undegraded":"2023-10-08T16:10:34.777919+0000","last_fullsized":"2023-10-08T16:10:34.777919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:49.490701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772430+0000","last_change":"2023-10-08T16:10:34.770488+0000","last_active":"2023-10-08T16:10:34.772430+0000","last_peered":"2023-10-08T16:10:34.772430+0000","last_clean":"2023-10-08T16:10:34.772430+0000","last_became_active":"2023-10-08T16:10:34.769995+0000","last_became_peered":"2023-10-08T16:10:34.769995+0000","last_unstale":"2023-10-08T16:10:34.772430+0000","last_undegraded":"2023-10-08T16:10:34.772430+0000","last_fullsized":"2023-10-08T16:10:34.772430+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:50:36.437119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774957+0000","last_change":"2023-10-08T16:10:34.773518+0000","last_active":"2023-10-08T16:10:34.774957+0000","last_peered":"2023-10-08T16:10:34.774957+0000","last_clean":"2023-10-08T16:10:34.774957+0000","last_became_active":"2023-10-08T16:10:34.773131+0000","last_became_peered":"2023-10-08T16:10:34.773131+0000","last_unstale":"2023-10-08T16:10:34.774957+0000","last_undegraded":"2023-10-08T16:10:34.774957+0000","last_fullsized":"2023-10-08T16:10:34.774957+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:55.347540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774012+0000","last_change":"2023-10-08T16:10:34.767502+0000","last_active":"2023-10-08T16:10:34.774012+0000","last_peered":"2023-10-08T16:10:34.774012+0000","last_clean":"2023-10-08T16:10:34.774012+0000","last_became_active":"2023-10-08T16:10:34.767202+0000","last_became_peered":"2023-10-08T16:10:34.767202+0000","last_unstale":"2023-10-08T16:10:34.774012+0000","last_undegraded":"2023-10-08T16:10:34.774012+0000","last_fullsized":"2023-10-08T16:10:34.774012+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:56.780695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.762670+0000","last_change":"2023-10-08T16:10:34.756490+0000","last_active":"2023-10-08T16:10:34.762670+0000","last_peered":"2023-10-08T16:10:34.762670+0000","last_clean":"2023-10-08T16:10:34.762670+0000","last_became_active":"2023-10-08T16:10:34.756220+0000","last_became_peered":"2023-10-08T16:10:34.756220+0000","last_unstale":"2023-10-08T16:10:34.762670+0000","last_undegraded":"2023-10-08T16:10:34.762670+0000","last_fullsized":"2023-10-08T16:10:34.762670+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778267+0000","last_change":"2023-10-08T16:10:34.777472+0000","last_active":"2023-10-08T16:10:34.778267+0000","last_peered":"2023-10-08T16:10:34.778267+0000","last_clean":"2023-10-08T16:10:34.778267+0000","last_became_active":"2023-10-08T16:10:34.777232+0000","last_became_peered":"2023-10-08T16:10:34.777232+0000","last_unstale":"2023-10-08T16:10:34.778267+0000","last_undegraded":"2023-10-08T16:10:34.778267+0000","last_fullsized":"2023-10-08T16:10:34.778267+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:23.635270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780757+0000","last_change":"2023-10-08T16:10:34.769074+0000","last_active":"2023-10-08T16:10:34.780757+0000","last_peered":"2023-10-08T16:10:34.780757+0000","last_clean":"2023-10-08T16:10:34.780757+0000","last_became_active":"2023-10-08T16:10:34.768606+0000","last_became_peered":"2023-10-08T16:10:34.768606+0000","last_unstale":"2023-10-08T16:10:34.780757+0000","last_undegraded":"2023-10-08T16:10:34.780757+0000","last_fullsized":"2023-10-08T16:10:34.780757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768168+0000","last_change":"2023-10-08T16:10:34.762792+0000","last_active":"2023-10-08T16:10:34.768168+0000","last_peered":"2023-10-08T16:10:34.768168+0000","last_clean":"2023-10-08T16:10:34.768168+0000","last_became_active":"2023-10-08T16:10:34.762518+0000","last_became_peered":"2023-10-08T16:10:34.762518+0000","last_unstale":"2023-10-08T16:10:34.768168+0000","last_undegraded":"2023-10-08T16:10:34.768168+0000","last_fullsized":"2023-10-08T16:10:34.768168+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:28.201365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.771552+0000","last_change":"2023-10-08T16:10:34.762905+0000","last_active":"2023-10-08T16:10:34.771552+0000","last_peered":"2023-10-08T16:10:34.771552+0000","last_clean":"2023-10-08T16:10:34.771552+0000","last_became_active":"2023-10-08T16:10:34.762592+0000","last_became_peered":"2023-10-08T16:10:34.762592+0000","last_unstale":"2023-10-08T16:10:34.771552+0000","last_undegraded":"2023-10-08T16:10:34.771552+0000","last_fullsized":"2023-10-08T16:10:34.771552+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:47:41.629319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781037+0000","last_change":"2023-10-08T16:10:34.773587+0000","last_active":"2023-10-08T16:10:34.781037+0000","last_peered":"2023-10-08T16:10:34.781037+0000","last_clean":"2023-10-08T16:10:34.781037+0000","last_became_active":"2023-10-08T16:10:34.773323+0000","last_became_peered":"2023-10-08T16:10:34.773323+0000","last_unstale":"2023-10-08T16:10:34.781037+0000","last_undegraded":"2023-10-08T16:10:34.781037+0000","last_fullsized":"2023-10-08T16:10:34.781037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780964+0000","last_change":"2023-10-08T16:10:34.770454+0000","last_active":"2023-10-08T16:10:34.780964+0000","last_peered":"2023-10-08T16:10:34.780964+0000","last_clean":"2023-10-08T16:10:34.780964+0000","last_became_active":"2023-10-08T16:10:34.769660+0000","last_became_peered":"2023-10-08T16:10:34.769660+0000","last_unstale":"2023-10-08T16:10:34.780964+0000","last_undegraded":"2023-10-08T16:10:34.780964+0000","last_fullsized":"2023-10-08T16:10:34.780964+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:42:42.444943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767096+0000","last_change":"2023-10-08T16:10:34.761459+0000","last_active":"2023-10-08T16:10:34.767096+0000","last_peered":"2023-10-08T16:10:34.767096+0000","last_clean":"2023-10-08T16:10:34.767096+0000","last_became_active":"2023-10-08T16:10:34.761215+0000","last_became_peered":"2023-10-08T16:10:34.761215+0000","last_unstale":"2023-10-08T16:10:34.767096+0000","last_undegraded":"2023-10-08T16:10:34.767096+0000","last_fullsized":"2023-10-08T16:10:34.767096+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780968+0000","last_change":"2023-10-08T16:10:34.772442+0000","last_active":"2023-10-08T16:10:34.780968+0000","last_peered":"2023-10-08T16:10:34.780968+0000","last_clean":"2023-10-08T16:10:34.780968+0000","last_became_active":"2023-10-08T16:10:34.772168+0000","last_became_peered":"2023-10-08T16:10:34.772168+0000","last_unstale":"2023-10-08T16:10:34.780968+0000","last_undegraded":"2023-10-08T16:10:34.780968+0000","last_fullsized":"2023-10-08T16:10:34.780968+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777150+0000","last_change":"2023-10-08T16:10:34.774563+0000","last_active":"2023-10-08T16:10:34.777150+0000","last_peered":"2023-10-08T16:10:34.777150+0000","last_clean":"2023-10-08T16:10:34.777150+0000","last_became_active":"2023-10-08T16:10:34.774110+0000","last_became_peered":"2023-10-08T16:10:34.774110+0000","last_unstale":"2023-10-08T16:10:34.777150+0000","last_undegraded":"2023-10-08T16:10:34.777150+0000","last_fullsized":"2023-10-08T16:10:34.777150+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:29.395963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780794+0000","last_change":"2023-10-08T16:10:34.769228+0000","last_active":"2023-10-08T16:10:34.780794+0000","last_peered":"2023-10-08T16:10:34.780794+0000","last_clean":"2023-10-08T16:10:34.780794+0000","last_became_active":"2023-10-08T16:10:34.768852+0000","last_became_peered":"2023-10-08T16:10:34.768852+0000","last_unstale":"2023-10-08T16:10:34.780794+0000","last_undegraded":"2023-10-08T16:10:34.780794+0000","last_fullsized":"2023-10-08T16:10:34.780794+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772265+0000","last_change":"2023-10-08T16:10:34.768805+0000","last_active":"2023-10-08T16:10:34.772265+0000","last_peered":"2023-10-08T16:10:34.772265+0000","last_clean":"2023-10-08T16:10:34.772265+0000","last_became_active":"2023-10-08T16:10:34.768420+0000","last_became_peered":"2023-10-08T16:10:34.768420+0000","last_unstale":"2023-10-08T16:10:34.772265+0000","last_undegraded":"2023-10-08T16:10:34.772265+0000","last_fullsized":"2023-10-08T16:10:34.772265+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:08.900903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.763786+0000","last_change":"2023-10-08T16:10:34.762065+0000","last_active":"2023-10-08T16:10:34.763786+0000","last_peered":"2023-10-08T16:10:34.763786+0000","last_clean":"2023-10-08T16:10:34.763786+0000","last_became_active":"2023-10-08T16:10:34.761777+0000","last_became_peered":"2023-10-08T16:10:34.761777+0000","last_unstale":"2023-10-08T16:10:34.763786+0000","last_undegraded":"2023-10-08T16:10:34.763786+0000","last_fullsized":"2023-10-08T16:10:34.763786+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:12.031551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768846+0000","last_change":"2023-10-08T16:10:34.767771+0000","last_active":"2023-10-08T16:10:34.768846+0000","last_peered":"2023-10-08T16:10:34.768846+0000","last_clean":"2023-10-08T16:10:34.768846+0000","last_became_active":"2023-10-08T16:10:34.766840+0000","last_became_peered":"2023-10-08T16:10:34.766840+0000","last_unstale":"2023-10-08T16:10:34.768846+0000","last_undegraded":"2023-10-08T16:10:34.768846+0000","last_fullsized":"2023-10-08T16:10:34.768846+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:47:41.629319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781074+0000","last_change":"2023-10-08T16:10:34.776477+0000","last_active":"2023-10-08T16:10:34.781074+0000","last_peered":"2023-10-08T16:10:34.781074+0000","last_clean":"2023-10-08T16:10:34.781074+0000","last_became_active":"2023-10-08T16:10:34.776228+0000","last_became_peered":"2023-10-08T16:10:34.776228+0000","last_unstale":"2023-10-08T16:10:34.781074+0000","last_undegraded":"2023-10-08T16:10:34.781074+0000","last_fullsized":"2023-10-08T16:10:34.781074+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777824+0000","last_change":"2023-10-08T16:10:34.772998+0000","last_active":"2023-10-08T16:10:34.777824+0000","last_peered":"2023-10-08T16:10:34.777824+0000","last_clean":"2023-10-08T16:10:34.777824+0000","last_became_active":"2023-10-08T16:10:34.772746+0000","last_became_peered":"2023-10-08T16:10:34.772746+0000","last_unstale":"2023-10-08T16:10:34.777824+0000","last_undegraded":"2023-10-08T16:10:34.777824+0000","last_fullsized":"2023-10-08T16:10:34.777824+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:53:59.525183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772196+0000","last_change":"2023-10-08T16:10:34.767838+0000","last_active":"2023-10-08T16:10:34.772196+0000","last_peered":"2023-10-08T16:10:34.772196+0000","last_clean":"2023-10-08T16:10:34.772196+0000","last_became_active":"2023-10-08T16:10:34.767545+0000","last_became_peered":"2023-10-08T16:10:34.767545+0000","last_unstale":"2023-10-08T16:10:34.772196+0000","last_undegraded":"2023-10-08T16:10:34.772196+0000","last_fullsized":"2023-10-08T16:10:34.772196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:19.547237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761998+0000","last_change":"2023-10-08T16:10:34.755887+0000","last_active":"2023-10-08T16:10:34.761998+0000","last_peered":"2023-10-08T16:10:34.761998+0000","last_clean":"2023-10-08T16:10:34.761998+0000","last_became_active":"2023-10-08T16:10:34.755622+0000","last_became_peered":"2023-10-08T16:10:34.755622+0000","last_unstale":"2023-10-08T16:10:34.761998+0000","last_undegraded":"2023-10-08T16:10:34.761998+0000","last_fullsized":"2023-10-08T16:10:34.761998+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:33.622637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761679+0000","last_change":"2023-10-08T16:10:34.756936+0000","last_active":"2023-10-08T16:10:34.761679+0000","last_peered":"2023-10-08T16:10:34.761679+0000","last_clean":"2023-10-08T16:10:34.761679+0000","last_became_active":"2023-10-08T16:10:34.756710+0000","last_became_peered":"2023-10-08T16:10:34.756710+0000","last_unstale":"2023-10-08T16:10:34.761679+0000","last_undegraded":"2023-10-08T16:10:34.761679+0000","last_fullsized":"2023-10-08T16:10:34.761679+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:28.201365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":14,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775581+0000","last_change":"2023-10-08T16:10:33.766666+0000","last_active":"2023-10-08T16:10:34.775581+0000","last_peered":"2023-10-08T16:10:34.775581+0000","last_clean":"2023-10-08T16:10:34.775581+0000","last_became_active":"2023-10-08T16:10:33.764988+0000","last_became_peered":"2023-10-08T16:10:33.764988+0000","last_unstale":"2023-10-08T16:10:34.775581+0000","last_undegraded":"2023-10-08T16:10:34.775581+0000","last_fullsized":"2023-10-08T16:10:34.775581+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:49:49.585207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780579+0000","last_change":"2023-10-08T16:10:34.773742+0000","last_active":"2023-10-08T16:10:34.780579+0000","last_peered":"2023-10-08T16:10:34.780579+0000","last_clean":"2023-10-08T16:10:34.780579+0000","last_became_active":"2023-10-08T16:10:34.773425+0000","last_became_peered":"2023-10-08T16:10:34.773425+0000","last_unstale":"2023-10-08T16:10:34.780579+0000","last_undegraded":"2023-10-08T16:10:34.780579+0000","last_fullsized":"2023-10-08T16:10:34.780579+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:12.031551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778396+0000","last_change":"2023-10-08T16:10:33.765370+0000","last_active":"2023-10-08T16:10:34.778396+0000","last_peered":"2023-10-08T16:10:34.778396+0000","last_clean":"2023-10-08T16:10:34.778396+0000","last_became_active":"2023-10-08T16:10:33.764896+0000","last_became_peered":"2023-10-08T16:10:33.764896+0000","last_unstale":"2023-10-08T16:10:34.778396+0000","last_undegraded":"2023-10-08T16:10:34.778396+0000","last_fullsized":"2023-10-08T16:10:34.778396+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.761341+0000","last_change":"2023-10-08T16:10:34.755901+0000","last_active":"2023-10-08T16:10:34.761341+0000","last_peered":"2023-10-08T16:10:34.761341+0000","last_clean":"2023-10-08T16:10:34.761341+0000","last_became_active":"2023-10-08T16:10:34.755587+0000","last_became_peered":"2023-10-08T16:10:34.755587+0000","last_unstale":"2023-10-08T16:10:34.761341+0000","last_undegraded":"2023-10-08T16:10:34.761341+0000","last_fullsized":"2023-10-08T16:10:34.761341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:28:33.311708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775050+0000","last_change":"2023-10-08T16:10:33.763286+0000","last_active":"2023-10-08T16:10:34.775050+0000","last_peered":"2023-10-08T16:10:34.775050+0000","last_clean":"2023-10-08T16:10:34.775050+0000","last_became_active":"2023-10-08T16:10:33.763028+0000","last_became_peered":"2023-10-08T16:10:33.763028+0000","last_unstale":"2023-10-08T16:10:34.775050+0000","last_undegraded":"2023-10-08T16:10:34.775050+0000","last_fullsized":"2023-10-08T16:10:34.775050+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:03.701535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.771589+0000","last_change":"2023-10-08T16:10:34.764169+0000","last_active":"2023-10-08T16:10:34.771589+0000","last_peered":"2023-10-08T16:10:34.771589+0000","last_clean":"2023-10-08T16:10:34.771589+0000","last_became_active":"2023-10-08T16:10:34.763157+0000","last_became_peered":"2023-10-08T16:10:34.763157+0000","last_unstale":"2023-10-08T16:10:34.771589+0000","last_undegraded":"2023-10-08T16:10:34.771589+0000","last_fullsized":"2023-10-08T16:10:34.771589+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775000+0000","last_change":"2023-10-08T16:10:33.767700+0000","last_active":"2023-10-08T16:10:34.775000+0000","last_peered":"2023-10-08T16:10:34.775000+0000","last_clean":"2023-10-08T16:10:34.775000+0000","last_became_active":"2023-10-08T16:10:33.766926+0000","last_became_peered":"2023-10-08T16:10:33.766926+0000","last_unstale":"2023-10-08T16:10:34.775000+0000","last_undegraded":"2023-10-08T16:10:34.775000+0000","last_fullsized":"2023-10-08T16:10:34.775000+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:01.551646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772171+0000","last_change":"2023-10-08T16:10:34.766554+0000","last_active":"2023-10-08T16:10:34.772171+0000","last_peered":"2023-10-08T16:10:34.772171+0000","last_clean":"2023-10-08T16:10:34.772171+0000","last_became_active":"2023-10-08T16:10:34.766310+0000","last_became_peered":"2023-10-08T16:10:34.766310+0000","last_unstale":"2023-10-08T16:10:34.772171+0000","last_undegraded":"2023-10-08T16:10:34.772171+0000","last_fullsized":"2023-10-08T16:10:34.772171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:08.743491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767796+0000","last_change":"2023-10-08T16:10:33.760988+0000","last_active":"2023-10-08T16:10:34.767796+0000","last_peered":"2023-10-08T16:10:34.767796+0000","last_clean":"2023-10-08T16:10:34.767796+0000","last_became_active":"2023-10-08T16:10:33.760705+0000","last_became_peered":"2023-10-08T16:10:33.760705+0000","last_unstale":"2023-10-08T16:10:34.767796+0000","last_undegraded":"2023-10-08T16:10:34.767796+0000","last_fullsized":"2023-10-08T16:10:34.767796+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778063+0000","last_change":"2023-10-08T16:10:34.763970+0000","last_active":"2023-10-08T16:10:34.778063+0000","last_peered":"2023-10-08T16:10:34.778063+0000","last_clean":"2023-10-08T16:10:34.778063+0000","last_became_active":"2023-10-08T16:10:34.763645+0000","last_became_peered":"2023-10-08T16:10:34.763645+0000","last_unstale":"2023-10-08T16:10:34.778063+0000","last_undegraded":"2023-10-08T16:10:34.778063+0000","last_fullsized":"2023-10-08T16:10:34.778063+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:23.227121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772004+0000","last_change":"2023-10-08T16:10:33.765098+0000","last_active":"2023-10-08T16:10:34.772004+0000","last_peered":"2023-10-08T16:10:34.772004+0000","last_clean":"2023-10-08T16:10:34.772004+0000","last_became_active":"2023-10-08T16:10:33.764637+0000","last_became_peered":"2023-10-08T16:10:33.764637+0000","last_unstale":"2023-10-08T16:10:34.772004+0000","last_undegraded":"2023-10-08T16:10:34.772004+0000","last_fullsized":"2023-10-08T16:10:34.772004+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.446069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772209+0000","last_change":"2023-10-08T16:10:34.764753+0000","last_active":"2023-10-08T16:10:34.772209+0000","last_peered":"2023-10-08T16:10:34.772209+0000","last_clean":"2023-10-08T16:10:34.772209+0000","last_became_active":"2023-10-08T16:10:34.764527+0000","last_became_peered":"2023-10-08T16:10:34.764527+0000","last_unstale":"2023-10-08T16:10:34.772209+0000","last_undegraded":"2023-10-08T16:10:34.772209+0000","last_fullsized":"2023-10-08T16:10:34.772209+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:08:07.036302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757832+0000","last_change":"2023-10-08T16:10:33.761688+0000","last_active":"2023-10-08T16:10:34.757832+0000","last_peered":"2023-10-08T16:10:34.757832+0000","last_clean":"2023-10-08T16:10:34.757832+0000","last_became_active":"2023-10-08T16:10:33.761196+0000","last_became_peered":"2023-10-08T16:10:33.761196+0000","last_unstale":"2023-10-08T16:10:34.757832+0000","last_undegraded":"2023-10-08T16:10:34.757832+0000","last_fullsized":"2023-10-08T16:10:34.757832+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:35:27.197085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777711+0000","last_change":"2023-10-08T16:10:34.768417+0000","last_active":"2023-10-08T16:10:34.777711+0000","last_peered":"2023-10-08T16:10:34.777711+0000","last_clean":"2023-10-08T16:10:34.777711+0000","last_became_active":"2023-10-08T16:10:34.767713+0000","last_became_peered":"2023-10-08T16:10:34.767713+0000","last_unstale":"2023-10-08T16:10:34.777711+0000","last_undegraded":"2023-10-08T16:10:34.777711+0000","last_fullsized":"2023-10-08T16:10:34.777711+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:03.261328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757922+0000","last_change":"2023-10-08T16:10:33.770625+0000","last_active":"2023-10-08T16:10:34.757922+0000","last_peered":"2023-10-08T16:10:34.757922+0000","last_clean":"2023-10-08T16:10:34.757922+0000","last_became_active":"2023-10-08T16:10:33.770366+0000","last_became_peered":"2023-10-08T16:10:33.770366+0000","last_unstale":"2023-10-08T16:10:34.757922+0000","last_undegraded":"2023-10-08T16:10:34.757922+0000","last_fullsized":"2023-10-08T16:10:34.757922+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:29.471251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.767917+0000","last_change":"2023-10-08T16:10:34.763954+0000","last_active":"2023-10-08T16:10:34.767917+0000","last_peered":"2023-10-08T16:10:34.767917+0000","last_clean":"2023-10-08T16:10:34.767917+0000","last_became_active":"2023-10-08T16:10:34.763611+0000","last_became_peered":"2023-10-08T16:10:34.763611+0000","last_unstale":"2023-10-08T16:10:34.767917+0000","last_undegraded":"2023-10-08T16:10:34.767917+0000","last_fullsized":"2023-10-08T16:10:34.767917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:49:15.378722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753181+0000","last_change":"2023-10-08T16:10:33.767567+0000","last_active":"2023-10-08T16:10:34.753181+0000","last_peered":"2023-10-08T16:10:34.753181+0000","last_clean":"2023-10-08T16:10:34.753181+0000","last_became_active":"2023-10-08T16:10:33.767321+0000","last_became_peered":"2023-10-08T16:10:33.767321+0000","last_unstale":"2023-10-08T16:10:34.753181+0000","last_undegraded":"2023-10-08T16:10:34.753181+0000","last_fullsized":"2023-10-08T16:10:34.753181+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:13:23.776656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775279+0000","last_change":"2023-10-08T16:10:34.773581+0000","last_active":"2023-10-08T16:10:34.775279+0000","last_peered":"2023-10-08T16:10:34.775279+0000","last_clean":"2023-10-08T16:10:34.775279+0000","last_became_active":"2023-10-08T16:10:34.773300+0000","last_became_peered":"2023-10-08T16:10:34.773300+0000","last_unstale":"2023-10-08T16:10:34.775279+0000","last_undegraded":"2023-10-08T16:10:34.775279+0000","last_fullsized":"2023-10-08T16:10:34.775279+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:19.716752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753910+0000","last_change":"2023-10-08T16:10:33.764988+0000","last_active":"2023-10-08T16:10:34.753910+0000","last_peered":"2023-10-08T16:10:34.753910+0000","last_clean":"2023-10-08T16:10:34.753910+0000","last_became_active":"2023-10-08T16:10:33.764739+0000","last_became_peered":"2023-10-08T16:10:33.764739+0000","last_unstale":"2023-10-08T16:10:34.753910+0000","last_undegraded":"2023-10-08T16:10:34.753910+0000","last_fullsized":"2023-10-08T16:10:34.753910+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:20:09.603131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768075+0000","last_change":"2023-10-08T16:10:34.763271+0000","last_active":"2023-10-08T16:10:34.768075+0000","last_peered":"2023-10-08T16:10:34.768075+0000","last_clean":"2023-10-08T16:10:34.768075+0000","last_became_active":"2023-10-08T16:10:34.763025+0000","last_became_peered":"2023-10-08T16:10:34.763025+0000","last_unstale":"2023-10-08T16:10:34.768075+0000","last_undegraded":"2023-10-08T16:10:34.768075+0000","last_fullsized":"2023-10-08T16:10:34.768075+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:03.613941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777653+0000","last_change":"2023-10-08T16:10:33.768462+0000","last_active":"2023-10-08T16:10:34.777653+0000","last_peered":"2023-10-08T16:10:34.777653+0000","last_clean":"2023-10-08T16:10:34.777653+0000","last_became_active":"2023-10-08T16:10:33.768242+0000","last_became_peered":"2023-10-08T16:10:33.768242+0000","last_unstale":"2023-10-08T16:10:34.777653+0000","last_undegraded":"2023-10-08T16:10:34.777653+0000","last_fullsized":"2023-10-08T16:10:34.777653+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:59.803650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780607+0000","last_change":"2023-10-08T16:10:34.776374+0000","last_active":"2023-10-08T16:10:34.780607+0000","last_peered":"2023-10-08T16:10:34.780607+0000","last_clean":"2023-10-08T16:10:34.780607+0000","last_became_active":"2023-10-08T16:10:34.776089+0000","last_became_peered":"2023-10-08T16:10:34.776089+0000","last_unstale":"2023-10-08T16:10:34.780607+0000","last_undegraded":"2023-10-08T16:10:34.780607+0000","last_fullsized":"2023-10-08T16:10:34.780607+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:00:53.513417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":16,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.781742+0000","last_change":"2023-10-08T16:10:33.773222+0000","last_active":"2023-10-08T16:10:34.781742+0000","last_peered":"2023-10-08T16:10:34.781742+0000","last_clean":"2023-10-08T16:10:34.781742+0000","last_became_active":"2023-10-08T16:10:33.772922+0000","last_became_peered":"2023-10-08T16:10:33.772922+0000","last_unstale":"2023-10-08T16:10:34.781742+0000","last_undegraded":"2023-10-08T16:10:34.781742+0000","last_fullsized":"2023-10-08T16:10:34.781742+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:07.790273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768050+0000","last_change":"2023-10-08T16:10:34.762452+0000","last_active":"2023-10-08T16:10:34.768050+0000","last_peered":"2023-10-08T16:10:34.768050+0000","last_clean":"2023-10-08T16:10:34.768050+0000","last_became_active":"2023-10-08T16:10:34.762210+0000","last_became_peered":"2023-10-08T16:10:34.762210+0000","last_unstale":"2023-10-08T16:10:34.768050+0000","last_undegraded":"2023-10-08T16:10:34.768050+0000","last_fullsized":"2023-10-08T16:10:34.768050+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:55.347540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775926+0000","last_change":"2023-10-08T16:10:33.762563+0000","last_active":"2023-10-08T16:10:34.775926+0000","last_peered":"2023-10-08T16:10:34.775926+0000","last_clean":"2023-10-08T16:10:34.775926+0000","last_became_active":"2023-10-08T16:10:33.762298+0000","last_became_peered":"2023-10-08T16:10:33.762298+0000","last_unstale":"2023-10-08T16:10:34.775926+0000","last_undegraded":"2023-10-08T16:10:34.775926+0000","last_fullsized":"2023-10-08T16:10:34.775926+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:13:51.956683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777053+0000","last_change":"2023-10-08T16:10:34.775705+0000","last_active":"2023-10-08T16:10:34.777053+0000","last_peered":"2023-10-08T16:10:34.777053+0000","last_clean":"2023-10-08T16:10:34.777053+0000","last_became_active":"2023-10-08T16:10:34.775461+0000","last_became_peered":"2023-10-08T16:10:34.775461+0000","last_unstale":"2023-10-08T16:10:34.777053+0000","last_undegraded":"2023-10-08T16:10:34.777053+0000","last_fullsized":"2023-10-08T16:10:34.777053+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752385+0000","last_change":"2023-10-08T16:10:33.766497+0000","last_active":"2023-10-08T16:10:34.752385+0000","last_peered":"2023-10-08T16:10:34.752385+0000","last_clean":"2023-10-08T16:10:34.752385+0000","last_became_active":"2023-10-08T16:10:33.766186+0000","last_became_peered":"2023-10-08T16:10:33.766186+0000","last_unstale":"2023-10-08T16:10:34.752385+0000","last_undegraded":"2023-10-08T16:10:34.752385+0000","last_fullsized":"2023-10-08T16:10:34.752385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:02.257047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.773664+0000","last_change":"2023-10-08T16:10:34.771712+0000","last_active":"2023-10-08T16:10:34.773664+0000","last_peered":"2023-10-08T16:10:34.773664+0000","last_clean":"2023-10-08T16:10:34.773664+0000","last_became_active":"2023-10-08T16:10:34.771444+0000","last_became_peered":"2023-10-08T16:10:34.771444+0000","last_unstale":"2023-10-08T16:10:34.773664+0000","last_undegraded":"2023-10-08T16:10:34.773664+0000","last_fullsized":"2023-10-08T16:10:34.773664+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.475532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":18,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.778688+0000","last_change":"2023-10-08T16:10:33.770711+0000","last_active":"2023-10-08T16:10:34.778688+0000","last_peered":"2023-10-08T16:10:34.778688+0000","last_clean":"2023-10-08T16:10:34.778688+0000","last_became_active":"2023-10-08T16:10:33.770495+0000","last_became_peered":"2023-10-08T16:10:33.770495+0000","last_unstale":"2023-10-08T16:10:34.778688+0000","last_undegraded":"2023-10-08T16:10:34.778688+0000","last_fullsized":"2023-10-08T16:10:34.778688+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:49:50.671846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774772+0000","last_change":"2023-10-08T16:10:34.768435+0000","last_active":"2023-10-08T16:10:34.774772+0000","last_peered":"2023-10-08T16:10:34.774772+0000","last_clean":"2023-10-08T16:10:34.774772+0000","last_became_active":"2023-10-08T16:10:34.768187+0000","last_became_peered":"2023-10-08T16:10:34.768187+0000","last_unstale":"2023-10-08T16:10:34.774772+0000","last_undegraded":"2023-10-08T16:10:34.774772+0000","last_fullsized":"2023-10-08T16:10:34.774772+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:46:55.082299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780677+0000","last_change":"2023-10-08T16:10:33.773350+0000","last_active":"2023-10-08T16:10:34.780677+0000","last_peered":"2023-10-08T16:10:34.780677+0000","last_clean":"2023-10-08T16:10:34.780677+0000","last_became_active":"2023-10-08T16:10:33.772989+0000","last_became_peered":"2023-10-08T16:10:33.772989+0000","last_unstale":"2023-10-08T16:10:34.780677+0000","last_undegraded":"2023-10-08T16:10:34.780677+0000","last_fullsized":"2023-10-08T16:10:34.780677+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:18.004513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774952+0000","last_change":"2023-10-08T16:10:34.772942+0000","last_active":"2023-10-08T16:10:34.774952+0000","last_peered":"2023-10-08T16:10:34.774952+0000","last_clean":"2023-10-08T16:10:34.774952+0000","last_became_active":"2023-10-08T16:10:34.772686+0000","last_became_peered":"2023-10-08T16:10:34.772686+0000","last_unstale":"2023-10-08T16:10:34.774952+0000","last_undegraded":"2023-10-08T16:10:34.774952+0000","last_fullsized":"2023-10-08T16:10:34.774952+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780988+0000","last_change":"2023-10-08T16:10:33.760706+0000","last_active":"2023-10-08T16:10:34.780988+0000","last_peered":"2023-10-08T16:10:34.780988+0000","last_clean":"2023-10-08T16:10:34.780988+0000","last_became_active":"2023-10-08T16:10:33.760439+0000","last_became_peered":"2023-10-08T16:10:33.760439+0000","last_unstale":"2023-10-08T16:10:34.780988+0000","last_undegraded":"2023-10-08T16:10:34.780988+0000","last_fullsized":"2023-10-08T16:10:34.780988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:34.341551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774667+0000","last_change":"2023-10-08T16:10:34.771623+0000","last_active":"2023-10-08T16:10:34.774667+0000","last_peered":"2023-10-08T16:10:34.774667+0000","last_clean":"2023-10-08T16:10:34.774667+0000","last_became_active":"2023-10-08T16:10:34.770995+0000","last_became_peered":"2023-10-08T16:10:34.770995+0000","last_unstale":"2023-10-08T16:10:34.774667+0000","last_undegraded":"2023-10-08T16:10:34.774667+0000","last_fullsized":"2023-10-08T16:10:34.774667+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:06:51.238920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":11,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772729+0000","last_change":"2023-10-08T16:10:33.764394+0000","last_active":"2023-10-08T16:10:34.772729+0000","last_peered":"2023-10-08T16:10:34.772729+0000","last_clean":"2023-10-08T16:10:34.772729+0000","last_became_active":"2023-10-08T16:10:33.764077+0000","last_became_peered":"2023-10-08T16:10:33.764077+0000","last_unstale":"2023-10-08T16:10:34.772729+0000","last_undegraded":"2023-10-08T16:10:34.772729+0000","last_fullsized":"2023-10-08T16:10:34.772729+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:01:27.558248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775010+0000","last_change":"2023-10-08T16:10:34.769083+0000","last_active":"2023-10-08T16:10:34.775010+0000","last_peered":"2023-10-08T16:10:34.775010+0000","last_clean":"2023-10-08T16:10:34.775010+0000","last_became_active":"2023-10-08T16:10:34.768807+0000","last_became_peered":"2023-10-08T16:10:34.768807+0000","last_unstale":"2023-10-08T16:10:34.775010+0000","last_undegraded":"2023-10-08T16:10:34.775010+0000","last_fullsized":"2023-10-08T16:10:34.775010+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:29.395963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.756380+0000","last_change":"2023-10-08T16:10:33.773846+0000","last_active":"2023-10-08T16:10:34.756380+0000","last_peered":"2023-10-08T16:10:34.756380+0000","last_clean":"2023-10-08T16:10:34.756380+0000","last_became_active":"2023-10-08T16:10:33.773563+0000","last_became_peered":"2023-10-08T16:10:33.773563+0000","last_unstale":"2023-10-08T16:10:34.756380+0000","last_undegraded":"2023-10-08T16:10:34.756380+0000","last_fullsized":"2023-10-08T16:10:34.756380+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:58:45.919412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.774626+0000","last_change":"2023-10-08T16:10:34.770945+0000","last_active":"2023-10-08T16:10:34.774626+0000","last_peered":"2023-10-08T16:10:34.774626+0000","last_clean":"2023-10-08T16:10:34.774626+0000","last_became_active":"2023-10-08T16:10:34.770681+0000","last_became_peered":"2023-10-08T16:10:34.770681+0000","last_unstale":"2023-10-08T16:10:34.774626+0000","last_undegraded":"2023-10-08T16:10:34.774626+0000","last_fullsized":"2023-10-08T16:10:34.774626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:14.997272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775711+0000","last_change":"2023-10-08T16:10:33.759531+0000","last_active":"2023-10-08T16:10:34.775711+0000","last_peered":"2023-10-08T16:10:34.775711+0000","last_clean":"2023-10-08T16:10:34.775711+0000","last_became_active":"2023-10-08T16:10:33.759312+0000","last_became_peered":"2023-10-08T16:10:33.759312+0000","last_unstale":"2023-10-08T16:10:34.775711+0000","last_undegraded":"2023-10-08T16:10:34.775711+0000","last_fullsized":"2023-10-08T16:10:34.775711+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:48:02.135460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777838+0000","last_change":"2023-10-08T16:10:34.776952+0000","last_active":"2023-10-08T16:10:34.777838+0000","last_peered":"2023-10-08T16:10:34.777838+0000","last_clean":"2023-10-08T16:10:34.777838+0000","last_became_active":"2023-10-08T16:10:34.776702+0000","last_became_peered":"2023-10-08T16:10:34.776702+0000","last_unstale":"2023-10-08T16:10:34.777838+0000","last_undegraded":"2023-10-08T16:10:34.777838+0000","last_fullsized":"2023-10-08T16:10:34.777838+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:22:48.420525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752982+0000","last_change":"2023-10-08T16:10:33.768853+0000","last_active":"2023-10-08T16:10:34.752982+0000","last_peered":"2023-10-08T16:10:34.752982+0000","last_clean":"2023-10-08T16:10:34.752982+0000","last_became_active":"2023-10-08T16:10:33.768563+0000","last_became_peered":"2023-10-08T16:10:33.768563+0000","last_unstale":"2023-10-08T16:10:34.752982+0000","last_undegraded":"2023-10-08T16:10:34.752982+0000","last_fullsized":"2023-10-08T16:10:34.752982+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:38:57.969527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777046+0000","last_change":"2023-10-08T16:10:34.774673+0000","last_active":"2023-10-08T16:10:34.777046+0000","last_peered":"2023-10-08T16:10:34.777046+0000","last_clean":"2023-10-08T16:10:34.777046+0000","last_became_active":"2023-10-08T16:10:34.773949+0000","last_became_peered":"2023-10-08T16:10:34.773949+0000","last_unstale":"2023-10-08T16:10:34.777046+0000","last_undegraded":"2023-10-08T16:10:34.777046+0000","last_fullsized":"2023-10-08T16:10:34.777046+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:59.993718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752225+0000","last_change":"2023-10-08T16:10:33.761930+0000","last_active":"2023-10-08T16:10:34.752225+0000","last_peered":"2023-10-08T16:10:34.752225+0000","last_clean":"2023-10-08T16:10:34.752225+0000","last_became_active":"2023-10-08T16:10:33.761680+0000","last_became_peered":"2023-10-08T16:10:33.761680+0000","last_unstale":"2023-10-08T16:10:34.752225+0000","last_undegraded":"2023-10-08T16:10:34.752225+0000","last_fullsized":"2023-10-08T16:10:34.752225+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768423+0000","last_change":"2023-10-08T16:10:34.763880+0000","last_active":"2023-10-08T16:10:34.768423+0000","last_peered":"2023-10-08T16:10:34.768423+0000","last_clean":"2023-10-08T16:10:34.768423+0000","last_became_active":"2023-10-08T16:10:34.763182+0000","last_became_peered":"2023-10-08T16:10:34.763182+0000","last_unstale":"2023-10-08T16:10:34.768423+0000","last_undegraded":"2023-10-08T16:10:34.768423+0000","last_fullsized":"2023-10-08T16:10:34.768423+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:42:44.389474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758069+0000","last_change":"2023-10-08T16:10:33.766371+0000","last_active":"2023-10-08T16:10:34.758069+0000","last_peered":"2023-10-08T16:10:34.758069+0000","last_clean":"2023-10-08T16:10:34.758069+0000","last_became_active":"2023-10-08T16:10:33.766082+0000","last_became_peered":"2023-10-08T16:10:33.766082+0000","last_unstale":"2023-10-08T16:10:34.758069+0000","last_undegraded":"2023-10-08T16:10:34.758069+0000","last_fullsized":"2023-10-08T16:10:34.758069+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:58.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.775617+0000","last_change":"2023-10-08T16:10:34.773031+0000","last_active":"2023-10-08T16:10:34.775617+0000","last_peered":"2023-10-08T16:10:34.775617+0000","last_clean":"2023-10-08T16:10:34.775617+0000","last_became_active":"2023-10-08T16:10:34.772788+0000","last_became_peered":"2023-10-08T16:10:34.772788+0000","last_unstale":"2023-10-08T16:10:34.775617+0000","last_undegraded":"2023-10-08T16:10:34.775617+0000","last_fullsized":"2023-10-08T16:10:34.775617+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:11:49.540524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":12,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772738+0000","last_change":"2023-10-08T16:10:33.770919+0000","last_active":"2023-10-08T16:10:34.772738+0000","last_peered":"2023-10-08T16:10:34.772738+0000","last_clean":"2023-10-08T16:10:34.772738+0000","last_became_active":"2023-10-08T16:10:33.770608+0000","last_became_peered":"2023-10-08T16:10:33.770608+0000","last_unstale":"2023-10-08T16:10:34.772738+0000","last_undegraded":"2023-10-08T16:10:34.772738+0000","last_fullsized":"2023-10-08T16:10:34.772738+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:02:37.099695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.779768+0000","last_change":"2023-10-08T16:10:34.779313+0000","last_active":"2023-10-08T16:10:34.779768+0000","last_peered":"2023-10-08T16:10:34.779768+0000","last_clean":"2023-10-08T16:10:34.779768+0000","last_became_active":"2023-10-08T16:10:34.779078+0000","last_became_peered":"2023-10-08T16:10:34.779078+0000","last_unstale":"2023-10-08T16:10:34.779768+0000","last_undegraded":"2023-10-08T16:10:34.779768+0000","last_fullsized":"2023-10-08T16:10:34.779768+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:04.168085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.753265+0000","last_change":"2023-10-08T16:10:33.766192+0000","last_active":"2023-10-08T16:10:34.753265+0000","last_peered":"2023-10-08T16:10:34.753265+0000","last_clean":"2023-10-08T16:10:34.753265+0000","last_became_active":"2023-10-08T16:10:33.765245+0000","last_became_peered":"2023-10-08T16:10:33.765245+0000","last_unstale":"2023-10-08T16:10:34.753265+0000","last_undegraded":"2023-10-08T16:10:34.753265+0000","last_fullsized":"2023-10-08T16:10:34.753265+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:49.157336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772212+0000","last_change":"2023-10-08T16:10:34.765909+0000","last_active":"2023-10-08T16:10:34.772212+0000","last_peered":"2023-10-08T16:10:34.772212+0000","last_clean":"2023-10-08T16:10:34.772212+0000","last_became_active":"2023-10-08T16:10:34.765658+0000","last_became_peered":"2023-10-08T16:10:34.765658+0000","last_unstale":"2023-10-08T16:10:34.772212+0000","last_undegraded":"2023-10-08T16:10:34.772212+0000","last_fullsized":"2023-10-08T16:10:34.772212+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:03:46.612006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757102+0000","last_change":"2023-10-08T16:10:33.766948+0000","last_active":"2023-10-08T16:10:34.757102+0000","last_peered":"2023-10-08T16:10:34.757102+0000","last_clean":"2023-10-08T16:10:34.757102+0000","last_became_active":"2023-10-08T16:10:33.766644+0000","last_became_peered":"2023-10-08T16:10:33.766644+0000","last_unstale":"2023-10-08T16:10:34.757102+0000","last_undegraded":"2023-10-08T16:10:34.757102+0000","last_fullsized":"2023-10-08T16:10:34.757102+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:54.343260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.768058+0000","last_change":"2023-10-08T16:10:34.763318+0000","last_active":"2023-10-08T16:10:34.768058+0000","last_peered":"2023-10-08T16:10:34.768058+0000","last_clean":"2023-10-08T16:10:34.768058+0000","last_became_active":"2023-10-08T16:10:34.762522+0000","last_became_peered":"2023-10-08T16:10:34.762522+0000","last_unstale":"2023-10-08T16:10:34.768058+0000","last_undegraded":"2023-10-08T16:10:34.768058+0000","last_fullsized":"2023-10-08T16:10:34.768058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:05:41.833152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.754375+0000","last_change":"2023-10-08T16:10:33.762562+0000","last_active":"2023-10-08T16:10:34.754375+0000","last_peered":"2023-10-08T16:10:34.754375+0000","last_clean":"2023-10-08T16:10:34.754375+0000","last_became_active":"2023-10-08T16:10:33.762261+0000","last_became_peered":"2023-10-08T16:10:33.762261+0000","last_unstale":"2023-10-08T16:10:34.754375+0000","last_undegraded":"2023-10-08T16:10:34.754375+0000","last_fullsized":"2023-10-08T16:10:34.754375+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:16:18.712472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780921+0000","last_change":"2023-10-08T16:10:34.770209+0000","last_active":"2023-10-08T16:10:34.780921+0000","last_peered":"2023-10-08T16:10:34.780921+0000","last_clean":"2023-10-08T16:10:34.780921+0000","last_became_active":"2023-10-08T16:10:34.769625+0000","last_became_peered":"2023-10-08T16:10:34.769625+0000","last_unstale":"2023-10-08T16:10:34.780921+0000","last_undegraded":"2023-10-08T16:10:34.780921+0000","last_fullsized":"2023-10-08T16:10:34.780921+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:08.794554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.755134+0000","last_change":"2023-10-08T16:10:33.765165+0000","last_active":"2023-10-08T16:10:34.755134+0000","last_peered":"2023-10-08T16:10:34.755134+0000","last_clean":"2023-10-08T16:10:34.755134+0000","last_became_active":"2023-10-08T16:10:33.764738+0000","last_became_peered":"2023-10-08T16:10:33.764738+0000","last_unstale":"2023-10-08T16:10:34.755134+0000","last_undegraded":"2023-10-08T16:10:34.755134+0000","last_fullsized":"2023-10-08T16:10:34.755134+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:40.994536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.780945+0000","last_change":"2023-10-08T16:10:34.769972+0000","last_active":"2023-10-08T16:10:34.780945+0000","last_peered":"2023-10-08T16:10:34.780945+0000","last_clean":"2023-10-08T16:10:34.780945+0000","last_became_active":"2023-10-08T16:10:34.769590+0000","last_became_peered":"2023-10-08T16:10:34.769590+0000","last_unstale":"2023-10-08T16:10:34.780945+0000","last_undegraded":"2023-10-08T16:10:34.780945+0000","last_fullsized":"2023-10-08T16:10:34.780945+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:57:19.008794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.757255+0000","last_change":"2023-10-08T16:10:33.755373+0000","last_active":"2023-10-08T16:10:34.757255+0000","last_peered":"2023-10-08T16:10:34.757255+0000","last_clean":"2023-10-08T16:10:34.757255+0000","last_became_active":"2023-10-08T16:10:33.755019+0000","last_became_peered":"2023-10-08T16:10:33.755019+0000","last_unstale":"2023-10-08T16:10:34.757255+0000","last_undegraded":"2023-10-08T16:10:34.757255+0000","last_fullsized":"2023-10-08T16:10:34.757255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:59:32.618356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777732+0000","last_change":"2023-10-08T16:10:34.775514+0000","last_active":"2023-10-08T16:10:34.777732+0000","last_peered":"2023-10-08T16:10:34.777732+0000","last_clean":"2023-10-08T16:10:34.777732+0000","last_became_active":"2023-10-08T16:10:34.775065+0000","last_became_peered":"2023-10-08T16:10:34.775065+0000","last_unstale":"2023-10-08T16:10:34.777732+0000","last_undegraded":"2023-10-08T16:10:34.777732+0000","last_fullsized":"2023-10-08T16:10:34.777732+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:15.117796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.752876+0000","last_change":"2023-10-08T16:10:33.767441+0000","last_active":"2023-10-08T16:10:34.752876+0000","last_peered":"2023-10-08T16:10:34.752876+0000","last_clean":"2023-10-08T16:10:34.752876+0000","last_became_active":"2023-10-08T16:10:33.767184+0000","last_became_peered":"2023-10-08T16:10:33.767184+0000","last_unstale":"2023-10-08T16:10:34.752876+0000","last_undegraded":"2023-10-08T16:10:34.752876+0000","last_fullsized":"2023-10-08T16:10:34.752876+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:15:44.856255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.772022+0000","last_change":"2023-10-08T16:10:34.764706+0000","last_active":"2023-10-08T16:10:34.772022+0000","last_peered":"2023-10-08T16:10:34.772022+0000","last_clean":"2023-10-08T16:10:34.772022+0000","last_became_active":"2023-10-08T16:10:34.764458+0000","last_became_peered":"2023-10-08T16:10:34.764458+0000","last_unstale":"2023-10-08T16:10:34.772022+0000","last_undegraded":"2023-10-08T16:10:34.772022+0000","last_fullsized":"2023-10-08T16:10:34.772022+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:53:39.219498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.758012+0000","last_change":"2023-10-08T16:10:33.766449+0000","last_active":"2023-10-08T16:10:34.758012+0000","last_peered":"2023-10-08T16:10:34.758012+0000","last_clean":"2023-10-08T16:10:34.758012+0000","last_became_active":"2023-10-08T16:10:33.766244+0000","last_became_peered":"2023-10-08T16:10:33.766244+0000","last_unstale":"2023-10-08T16:10:34.758012+0000","last_undegraded":"2023-10-08T16:10:34.758012+0000","last_fullsized":"2023-10-08T16:10:34.758012+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:02.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.777166+0000","last_change":"2023-10-08T16:10:34.774688+0000","last_active":"2023-10-08T16:10:34.777166+0000","last_peered":"2023-10-08T16:10:34.777166+0000","last_clean":"2023-10-08T16:10:34.777166+0000","last_became_active":"2023-10-08T16:10:34.774416+0000","last_became_peered":"2023-10-08T16:10:34.774416+0000","last_unstale":"2023-10-08T16:10:34.777166+0000","last_undegraded":"2023-10-08T16:10:34.777166+0000","last_fullsized":"2023-10-08T16:10:34.777166+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:33.743141+0000","last_clean_scrub_stamp":"2023-10-08T16:10:33.743141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:00.807930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.756007+0000","last_change":"2023-10-08T16:10:33.773844+0000","last_active":"2023-10-08T16:10:34.756007+0000","last_peered":"2023-10-08T16:10:34.756007+0000","last_clean":"2023-10-08T16:10:34.756007+0000","last_became_active":"2023-10-08T16:10:33.773481+0000","last_became_peered":"2023-10-08T16:10:33.773481+0000","last_unstale":"2023-10-08T16:10:34.756007+0000","last_undegraded":"2023-10-08T16:10:34.756007+0000","last_fullsized":"2023-10-08T16:10:34.756007+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:10:32.738860+0000","last_clean_scrub_stamp":"2023-10-08T16:10:32.738860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:41.626242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":100,"state":"active+clean","last_fresh":"2023-10-08T16:10:34.759908+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:10:34.759908+0000","last_peered":"2023-10-08T16:10:34.759908+0000","last_clean":"2023-10-08T16:10:34.759908+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:10:34.759908+0000","last_undegraded":"2023-10-08T16:10:34.759908+0000","last_fullsized":"2023-10-08T16:10:34.759908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299104,"kb_used_data":1816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072736,"statfs":{"total":96636764160,"available":96330481664,"internally_reserved":0,"allocated":1859584,"data_stored":1133614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.44900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.51400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":1.677},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.55100000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.629},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.58899999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298520,"kb_used_data":1232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073320,"statfs":{"total":96636764160,"available":96331079680,"internally_reserved":0,"allocated":1261568,"data_stored":542728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.48799999999999999},"min":{"1min":0.41499999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67400000000000004,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.53400000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.49299999999999999},"min":{"1min":0.39100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.38100000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66900000000000004,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.621}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53300000000000003,"15min":0.53900000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53200000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71899999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.73899999999999999,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.49099999999999999,"15min":0.51700000000000002},"min":{"1min":0.33400000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.41299999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69399999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45100000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.498,"15min":0.52600000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68000000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.47699999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.70499999999999996,"5min":0.874,"15min":0.874},"last":0.59299999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80500000000000005,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.45900000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71399999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302620,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069220,"statfs":{"total":96636764160,"available":96326881280,"internally_reserved":0,"allocated":1265664,"data_stored":542782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.30199999999999999,"15min":0.308},"min":{"1min":0.19500000000000001,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.434,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.32600000000000001,"15min":0.33600000000000002},"min":{"1min":0.249,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58199999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.32700000000000001,"15min":0.35699999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40699999999999997,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.371},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.372},"min":{"1min":0.245,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.46999999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35299999999999998,"15min":0.38700000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.316,"5min":0.33100000000000002,"15min":0.35899999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56399999999999995,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.40600000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.51100000000000001,"15min":0.52500000000000002},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.70399999999999996,"5min":1.077,"15min":1.077},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.33700000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.56699999999999995},"min":{"1min":0.36199999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54600000000000004,"15min":0.56799999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80600000000000005,"5min":1.099,"15min":1.099},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55800000000000005,"15min":0.58199999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55700000000000005,"15min":0.59399999999999997},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76700000000000002,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.77000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303212,"kb_used_data":1828,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068628,"statfs":{"total":96636764160,"available":96326275072,"internally_reserved":0,"allocated":1871872,"data_stored":1134674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.31,"5min":0.38100000000000001,"15min":0.55500000000000005},"min":{"1min":0.23100000000000001,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.40600000000000003,"5min":5.5599999999999996,"15min":5.5599999999999996},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.317,"5min":0.36599999999999999,"15min":0.48199999999999998},"min":{"1min":0.26000000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.47799999999999998,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.26100000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.40200000000000002,"15min":0.58899999999999997},"min":{"1min":0.26500000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.44,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.314},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35199999999999998,"15min":0.38100000000000001},"min":{"1min":0.253,"5min":0.219,"15min":0.219},"max":{"1min":0.40799999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29699999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38200000000000001,"15min":0.49199999999999999},"min":{"1min":0.215,"5min":0.188,"15min":0.188},"max":{"1min":1.6519999999999999,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.22700000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.44800000000000001,"15min":0.626},"min":{"1min":0.28799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.498,"5min":5.5970000000000004,"15min":5.5970000000000004},"last":0.33200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.53100000000000003,"15min":0.54600000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.71999999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58499999999999996,"15min":0.67300000000000004},"min":{"1min":0.39700000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.7649999999999999,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.67000000000000004},"min":{"1min":0.35699999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.847,"5min":3.2890000000000001,"15min":3.2890000000000001},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.61799999999999999,"15min":0.79700000000000004},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.70499999999999996,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.65400000000000003,"15min":0.81999999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.751,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.76800000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.747},"min":{"1min":0.42799999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.8979999999999999,"5min":3.5049999999999999,"15min":3.5049999999999999},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302612,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069228,"statfs":{"total":96636764160,"available":96326889472,"internally_reserved":0,"allocated":1257472,"data_stored":542658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.314},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.27900000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.30599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.57399999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.58399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.69599999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302628,"kb_used_data":1244,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069212,"statfs":{"total":96636764160,"available":96326873088,"internally_reserved":0,"allocated":1273856,"data_stored":544264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.32600000000000001,"15min":0.35599999999999998},"min":{"1min":0.22600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.46400000000000002,"5min":1.238,"15min":1.238},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.39400000000000002},"min":{"1min":0.221,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56599999999999995,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.35999999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.47999999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.371,"15min":0.41299999999999998},"min":{"1min":0.26500000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.59299999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.375,"15min":0.39900000000000002},"min":{"1min":0.29999999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.51900000000000002,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.42399999999999999},"min":{"1min":0.308,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.53300000000000003,"5min":1.327,"15min":1.327},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.60699999999999998},"min":{"1min":0.33300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.71599999999999997,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.375,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.58799999999999997},"min":{"1min":0.42999999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69499999999999995,"5min":1.27,"15min":1.27},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.621},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.754,"5min":1.879,"15min":1.879},"last":0.56699999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57999999999999996,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60899999999999999,"15min":0.627},"min":{"1min":0.52100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77700000000000002,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.70599999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:09:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61199999999999999,"15min":0.64800000000000002},"min":{"1min":0.435,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.97199999999999998,"5min":1.825,"15min":1.825},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0009999999999999,"5min":1.115,"15min":1.115},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298516,"kb_used_data":1228,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073324,"statfs":{"total":96636764160,"available":96331083776,"internally_reserved":0,"allocated":1257472,"data_stored":542658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.44500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.42199999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56499999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.32000000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.497}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298524,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073316,"statfs":{"total":96636764160,"available":96331075584,"internally_reserved":0,"allocated":1265664,"data_stored":543234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.76500000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.747}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.71399999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.52600000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.626}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.69799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:10:43.984 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:10:43.985 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:10:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.139+0000 7f7f8f145700 1 Processor -- start 2023-10-08T16:10:44.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.139+0000 7f7f8f145700 1 -- start start 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8f145700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7f880cf480 0x7f7f8815d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8f145700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f880b10f0 con 0x7f7f8815d760 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8f145700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8815db80 con 0x7f7f881595d0 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8f145700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8815e000 con 0x7f7f880cf480 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8e944700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7f8815d760 0x7f7f880b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:33692/0 (socket says 172.21.15.170:33692) 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8e944700 1 -- 172.21.15.170:0/333832119 learned_addr learned my addr 172.21.15.170:0/333832119 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d942700 1 --2- 172.21.15.170:0/333832119 >> v2:172.21.15.191:3300/0 conn(0x7f7f880cf480 0x7f7f8815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819110756 0 0) 0x7f7f880b10f0 con 0x7f7f8815d760 2023-10-08T16:10:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f74003570 con 0x7f7f8815d760 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188123518 0 0) 0x7f7f8815db80 con 0x7f7f881595d0 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f880b10f0 con 0x7f7f881595d0 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 609872055 0 0) 0x7f7f74003570 con 0x7f7f8815d760 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d942700 1 -- 172.21.15.170:0/333832119 >> v1:172.21.15.191:6789/0 conn(0x7f7f881595d0 legacy=0x7f7f880ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d942700 1 -- 172.21.15.170:0/333832119 >> v1:172.21.15.170:6789/0 conn(0x7f7f8815d760 legacy=0x7f7f880b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.140+0000 7f7f8d942700 1 -- 172.21.15.170:0/333832119 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880cf990 con 0x7f7f880cf480 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8d942700 1 --2- 172.21.15.170:0/333832119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f880cf480 0x7f7f8815d220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f780098e0 tx=0x7f7f780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73babafa01a9b938 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f800017f0 con 0x7f7f880cf480 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f78011420 con 0x7f7f880cf480 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8d141700 1 -- 172.21.15.170:0/333832119 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780117c0 con 0x7f7f880cf480 2023-10-08T16:10:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 -- 172.21.15.170:0/333832119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f880cf480 msgr2=0x7f7f8815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 --2- 172.21.15.170:0/333832119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f880cf480 0x7f7f8815d220 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f780098e0 tx=0x7f7f780091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 -- 172.21.15.170:0/333832119 shutdown_connections 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 --2- 172.21.15.170:0/333832119 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f880cf480 0x7f7f8815d220 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 -- 172.21.15.170:0/333832119 >> 172.21.15.170:0/333832119 conn(0x7f7f880d05f0 msgr2=0x7f7f880d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 -- 172.21.15.170:0/333832119 shutdown_connections 2023-10-08T16:10:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.141+0000 7f7f8f145700 1 -- 172.21.15.170:0/333832119 wait complete. 2023-10-08T16:10:44.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.142+0000 7f7f8f145700 1 Processor -- start 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 -- start start 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f880cf480 0x7f7f8816a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8815d760 0x7f7f8815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f880c5710 con 0x7f7f881595d0 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f88157230 con 0x7f7f8815d760 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8f145700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8816aac0 con 0x7f7f880cf480 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8d942700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8d942700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38098/0 (socket says 172.21.15.170:38098) 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8d942700 1 -- 172.21.15.170:0/2700613513 learned_addr learned my addr 172.21.15.170:0/2700613513 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8e143700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f880cf480 0x7f7f8816a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.143+0000 7f7f8e944700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8815d760 0x7f7f8815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 -- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f880cf480 msgr2=0x7f7f8816a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f880cf480 0x7f7f8816a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 -- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8815d760 msgr2=0x7f7f8815c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8815d760 0x7f7f8815c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 -- 172.21.15.170:0/2700613513 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78009420 con 0x7f7f881595d0 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8d942700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7f78002000 tx=0x7f7f78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7801cb30 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7801cc90 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7801a6a0 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f880ccbf0 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.144+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f88156fd0 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.145+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8804c3c0 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.145+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7f7801aa50 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.146+0000 7f7f7e7fc700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 0x7f7f6c069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.146+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7f78085f00 con 0x7f7f881595d0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.146+0000 7f7f8e143700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 0x7f7f6c069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.146+0000 7f7f8e143700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 0x7f7f6c069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f84003ae0 tx=0x7f7f84007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.150+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f78052d60 con 0x7f7f881595d0 2023-10-08T16:10:44.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:10:44.232 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:10:44.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.417+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7f880b3860 con 0x7f7f881595d0 2023-10-08T16:10:44.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.421+0000 7f7f7e7fc700 1 -- 172.21.15.170:0/2700613513 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7f780290e0 con 0x7f7f881595d0 2023-10-08T16:10:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 msgr2=0x7f7f6c069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 0x7f7f6c069740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f84003ae0 tx=0x7f7f84007d40 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 msgr2=0x7f7f88152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7f78002000 tx=0x7f7f78016000 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 shutdown_connections 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7f8815d760 0x7f7f8815c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7f880cf480 0x7f7f8816a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7f6c067290 0x7f7f6c069740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 --2- 172.21.15.170:0/2700613513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f881595d0 0x7f7f88152560 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 >> 172.21.15.170:0/2700613513 conn(0x7f7f880d05f0 msgr2=0x7f7f88152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 shutdown_connections 2023-10-08T16:10:44.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.423+0000 7f7f8f145700 1 -- 172.21.15.170:0/2700613513 wait complete. 2023-10-08T16:10:44.425 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:10:44.435 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:44.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.550+0000 7f766ced6700 1 Processor -- start 2023-10-08T16:10:44.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.550+0000 7f766ced6700 1 -- start start 2023-10-08T16:10:44.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f766ced6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f7668069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f766ced6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f766806a2b0 0x7f766806a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f766ced6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76680c96a0 con 0x7f76681599e0 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f766ced6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76680cb9e0 con 0x7f766806a2b0 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f766ced6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76680cab60 con 0x7f7668159db0 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f7666d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f766806a2b0 0x7f766806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f7666d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f766806a2b0 0x7f766806a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38114/0 (socket says 172.21.15.170:38114) 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.551+0000 7f7666d9d700 1 -- 172.21.15.170:0/3351471216 learned_addr learned my addr 172.21.15.170:0/3351471216 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:44.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f765dd9b700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f7668069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7666d9d700 1 -- 172.21.15.170:0/3351471216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 msgr2=0x7f7668069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7666d9d700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f7668069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7666d9d700 1 -- 172.21.15.170:0/3351471216 >> v1:172.21.15.191:6789/0 conn(0x7f76681599e0 legacy=0x7f7668069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7666d9d700 1 -- 172.21.15.170:0/3351471216 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76680cb420 con 0x7f766806a2b0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f765dd9b700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f7668069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7666d9d700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766806a2b0 0x7f766806a710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7660009f20 tx=0x7f7660009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897827acb90b15ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7665d9b700 1 -- 172.21.15.170:0/3351471216 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766000e070 con 0x7f766806a2b0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7665d9b700 1 -- 172.21.15.170:0/3351471216 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766000ba40 con 0x7f766806a2b0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f7665d9b700 1 -- 172.21.15.170:0/3351471216 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600092e0 con 0x7f766806a2b0 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 -- 172.21.15.170:0/3351471216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766806a2b0 msgr2=0x7f766806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766806a2b0 0x7f766806a710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7660009f20 tx=0x7f7660009840 comp rx=0 tx=0).stop 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 -- 172.21.15.170:0/3351471216 shutdown_connections 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f766806a2b0 0x7f766806a710 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 --2- 172.21.15.170:0/3351471216 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f7668069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 -- 172.21.15.170:0/3351471216 >> 172.21.15.170:0/3351471216 conn(0x7f76680c9860 msgr2=0x7f76680c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 -- 172.21.15.170:0/3351471216 shutdown_connections 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.552+0000 7f766ced6700 1 -- 172.21.15.170:0/3351471216 wait complete. 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 Processor -- start 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 -- start start 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766806a2b0 0x7f7668150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f766815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766815a9a0 con 0x7f76681599e0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668150d40 con 0x7f766806a2b0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.554+0000 7f766ced6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7668155220 con 0x7f7668159db0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38118/0 (socket says 172.21.15.170:38118) 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 -- 172.21.15.170:0/3658931053 learned_addr learned my addr 172.21.15.170:0/3658931053 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f7666d9d700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f766815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f766659c700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766806a2b0 0x7f7668150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 -- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 msgr2=0x7f766815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f766815be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 -- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766806a2b0 msgr2=0x7f7668150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766806a2b0 0x7f7668150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 -- 172.21.15.170:0/3658931053 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7660009ae0 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765dd9b700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f765000c830 tx=0x7f7650008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650011600 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650011760 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668155d90 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650011b20 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.555+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f766815a770 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.556+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f766804c3c0 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.556+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f7650018360 con 0x7f76681599e0 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.557+0000 7f765effd700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 0x7f7648069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:44.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.557+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7650084b10 con 0x7f76681599e0 2023-10-08T16:10:44.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.557+0000 7f766659c700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 0x7f7648069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:44.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.557+0000 7f766659c700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 0x7f7648069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7658003a30 tx=0x7f7658007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:44.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.561+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7650051860 con 0x7f76681599e0 2023-10-08T16:10:44.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.787+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7668063490 con 0x7f76681599e0 2023-10-08T16:10:44.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.787+0000 7f765effd700 1 -- 172.21.15.170:0/3658931053 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f7650015090 con 0x7f76681599e0 2023-10-08T16:10:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:44.788 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 msgr2=0x7f7648069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 0x7f7648069740 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7658003a30 tx=0x7f7658007ea0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 msgr2=0x7f766815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f765000c830 tx=0x7f7650008040 comp rx=0 tx=0).stop 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 shutdown_connections 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7668159db0 0x7f766815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f766806a2b0 0x7f7668150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7648067290 0x7f7648069740 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 --2- 172.21.15.170:0/3658931053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76681599e0 0x7f766815a1e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 >> 172.21.15.170:0/3658931053 conn(0x7f76680c9860 msgr2=0x7f76680c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 shutdown_connections 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:44.789+0000 7f766ced6700 1 -- 172.21.15.170:0/3658931053 wait complete. 2023-10-08T16:10:44.791 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 65 2023-10-08T16:10:45.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:10:34.136500+0000', 'modified': '2023-10-08T16:10:37.784482+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6771': {'gid': 6771, 'name': 'd', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/1404814921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6833', 'nonce': 1404814921}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1404814921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:10:45.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:10:45.206 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T16:10:45.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.321+0000 7f147678a700 1 Processor -- start 2023-10-08T16:10:45.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.321+0000 7f147678a700 1 -- start start 2023-10-08T16:10:45.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f147678a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1470159420 0x7f14700b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f147678a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 0x7f14700b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f147678a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14700b11d0 con 0x7f147015d230 2023-10-08T16:10:45.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f147678a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14700b14b0 con 0x7f1470159420 2023-10-08T16:10:45.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f147678a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14700b1790 con 0x7f14701597f0 2023-10-08T16:10:45.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f1475788700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1470159420 0x7f14700b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f1475788700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1470159420 0x7f14700b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56148/0 (socket says 172.21.15.170:56148) 2023-10-08T16:10:45.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.322+0000 7f1475788700 1 -- 172.21.15.170:0/3602998225 learned_addr learned my addr 172.21.15.170:0/3602998225 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1474f87700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 0x7f14700b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1475788700 1 -- 172.21.15.170:0/3602998225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 msgr2=0x7f14700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1475788700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 0x7f14700b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1475788700 1 -- 172.21.15.170:0/3602998225 >> v1:172.21.15.191:6789/0 conn(0x7f147015d230 legacy=0x7f14700b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1475788700 1 -- 172.21.15.170:0/3602998225 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700b1a70 con 0x7f1470159420 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1474f87700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 0x7f14700b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1475788700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1470159420 0x7f14700b5f40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f146c0099b0 tx=0x7f146c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a4a894bbe00f8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1467fff700 1 -- 172.21.15.170:0/3602998225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00b820 con 0x7f1470159420 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1467fff700 1 -- 172.21.15.170:0/3602998225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c00b980 con 0x7f1470159420 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f1467fff700 1 -- 172.21.15.170:0/3602998225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00bd20 con 0x7f1470159420 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f147678a700 1 -- 172.21.15.170:0/3602998225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1470159420 msgr2=0x7f14700b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.323+0000 7f147678a700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1470159420 0x7f14700b5f40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f146c0099b0 tx=0x7f146c009250 comp rx=0 tx=0).stop 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 -- 172.21.15.170:0/3602998225 shutdown_connections 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1470159420 0x7f14700b5f40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 --2- 172.21.15.170:0/3602998225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14701597f0 0x7f14700b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 -- 172.21.15.170:0/3602998225 >> 172.21.15.170:0/3602998225 conn(0x7f14700d0660 msgr2=0x7f14700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 -- 172.21.15.170:0/3602998225 shutdown_connections 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.324+0000 7f147678a700 1 -- 172.21.15.170:0/3602998225 wait complete. 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.325+0000 7f147678a700 1 Processor -- start 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.325+0000 7f147678a700 1 -- start start 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470159420 0x7f147014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 0x7f14701698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147014a6f0 con 0x7f14701597f0 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14700ce980 con 0x7f1470159420 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14700cef40 con 0x7f147015d230 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56160/0 (socket says 172.21.15.170:56160) 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 -- 172.21.15.170:0/194955680 learned_addr learned my addr 172.21.15.170:0/194955680 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1475788700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470159420 0x7f147014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1475f89700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 0x7f14701698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 -- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 msgr2=0x7f14701698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 0x7f14701698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 -- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470159420 msgr2=0x7f147014a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470159420 0x7f147014a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 -- 172.21.15.170:0/194955680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0094f0 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1475f89700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 0x7f14701698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f1474f87700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f146000c860 tx=0x7f1460008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f14657fa700 1 -- 172.21.15.170:0/194955680 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011600 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f14657fa700 1 -- 172.21.15.170:0/194955680 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460011760 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14700cf910 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f14657fa700 1 -- 172.21.15.170:0/194955680 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011b00 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.326+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147016a6e0 con 0x7f14701597f0 2023-10-08T16:10:45.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.328+0000 7f14657fa700 1 -- 172.21.15.170:0/194955680 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f146000a3e0 con 0x7f14701597f0 2023-10-08T16:10:45.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.328+0000 7f14657fa700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 0x7f1454069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.328+0000 7f14657fa700 1 -- 172.21.15.170:0/194955680 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1460084c10 con 0x7f14701597f0 2023-10-08T16:10:45.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.328+0000 7f1475788700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 0x7f1454069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.328+0000 7f1475788700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 0x7f1454069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f146c0099b0 tx=0x7f146c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.330+0000 7f1480ce4b80 1 Processor -- start 2023-10-08T16:10:45.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.330+0000 7f1480ce4b80 1 -- start start 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5700e6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a57009bb80 0x55a57009bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56fff2a70 con 0x55a5700e6480 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56fff0270 con 0x55a57009bb80 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a56ffec060 con 0x55a5700995f0 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56162/0 (socket says 172.21.15.170:56162) 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 -- 172.21.15.170:0/3893980419 learned_addr learned my addr 172.21.15.170:0/3893980419 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1474f87700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5700e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475f89700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a57009bb80 0x55a57009bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 -- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 msgr2=0x55a5700e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5700e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 -- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a57009bb80 msgr2=0x55a57009bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a57009bb80 0x55a57009bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 -- 172.21.15.170:0/3893980419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56fe86270 con 0x55a5700e6480 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1474f87700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5700e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f1475788700 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f146c00a010 tx=0x7f146c01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91114ec3ca517101 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f145a7fc700 1 -- 172.21.15.170:0/3893980419 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c006a90 con 0x55a5700e6480 2023-10-08T16:10:45.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f145a7fc700 1 -- 172.21.15.170:0/3893980419 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c011a90 con 0x55a5700e6480 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.331+0000 7f145a7fc700 1 -- 172.21.15.170:0/3893980419 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c011e10 con 0x55a5700e6480 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3893980419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 msgr2=0x55a5700c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f146c00a010 tx=0x7f146c01f040 comp rx=0 tx=0).stop 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3893980419 shutdown_connections 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a57009bb80 0x55a57009bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700c57d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3893980419 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5700e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3893980419 >> 172.21.15.170:0/3893980419 conn(0x55a57009d0c0 msgr2=0x55a57009d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3893980419 shutdown_connections 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3893980419 wait complete. 2023-10-08T16:10:45.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.332+0000 7f1480ce4b80 1 -- start start 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 Processor -- start 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 0x55a5700e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56ff97890 con 0x55a5700e6480 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a56ffb2ea0 con 0x55a5700995f0 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a56fe6ee30 con 0x55a57009bb80 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.333+0000 7f1480ce4b80 10 client.0 _ll_register_callbacks cb 0x55a57005f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1474f87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 0x55a5700e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475788700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56168/0 (socket says 172.21.15.170:56168) 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475788700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700e5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:35982/0 (socket says 172.21.15.170:35982) 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 -- 172.21.15.170:0/3978921096 learned_addr learned my addr 172.21.15.170:0/3978921096 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1474f87700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 0x55a5700e5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:54018/0 (socket says 172.21.15.170:54018) 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 -- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 msgr2=0x55a5700e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 0x55a5700e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 -- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 msgr2=0x55a5700e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 -- 172.21.15.170:0/3978921096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c023070 con 0x55a5700e6480 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1475f89700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f146800ef30 tx=0x7f1468007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac89e84471cb176 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f144f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468007e20 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f146800c9d0 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a570090cd0 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1468007bf0 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1480ce4b80 10 client.6924 fetch_fsmap learned FSMap version 65 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.334+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a56fe28fe0 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a570090cd0 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 65) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f146800f420 con 0x55a5700e6480 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 10 client.6924 fetch_fsmap finished waiting for FSMap version 65 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 10 client.6924 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 20 client.6924 populate_metadata read hostname 'smithi170' 2023-10-08T16:10:45.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a5700e2980 0x55a5700c7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 4 client.6924 mds_command: new command op to 6771 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 --> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a57006daa0 con 0x55a5700e2980 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1475788700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a5700e2980 0x55a5700c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f144f7fe700 10 client.6924 ms_handle_connect on v2:172.21.15.191:6833/1404814921 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.335+0000 7f1475788700 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a5700e2980 0x55a5700c7500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd0bfcd507db87cc server_cookie=cbb4e31af283d83f in_seq=0 out_seq=0 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f144f7fe700 1 -- 172.21.15.170:0/3978921096 <== mds.0 v2:172.21.15.191:6833/1404814921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a57006daa0 con 0x55a5700e2980 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f144f7fe700 10 client.6924 handle_command_reply: tid=0 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 client.6924 shutdown 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 msgr2=0x55a5700f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f146800ef30 tx=0x7f1468007a90 comp rx=0 tx=0).stop 2023-10-08T16:10:45.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 shutdown_connections 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a5700e2980 0x55a5700c7500 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5700e6480 0x55a5700f2500 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f144f7fe700 0 client.6924 ms_handle_reset on v2:172.21.15.191:6833/1404814921 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3978921096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a57009bb80 0x55a5700e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 >> 172.21.15.170:0/3978921096 conn(0x55a57009d0c0 msgr2=0x55a57009f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 shutdown_connections 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3978921096 wait complete. 2023-10-08T16:10:45.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.336+0000 7f1480ce4b80 20 client.6924 trim_cache size 0 max 16384 2023-10-08T16:10:45.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.353+0000 7f1480ce4b80 1 Processor -- start 2023-10-08T16:10:45.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.353+0000 7f1480ce4b80 1 -- start start 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 0x55a5700e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5700c7260 con 0x55a57009bb80 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5700949e0 con 0x55a5700995f0 2023-10-08T16:10:45.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5700e2360 con 0x55a5700e6480 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56170/0 (socket says 172.21.15.170:56170) 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 -- 172.21.15.170:0/3145803205 learned_addr learned my addr 172.21.15.170:0/3145803205 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 -- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 msgr2=0x55a5700e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1475788700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1475f89700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 0x55a5700e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 0x55a5700e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 -- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 msgr2=0x55a5700c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 -- 172.21.15.170:0/3145803205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a56ffba780 con 0x55a57009bb80 2023-10-08T16:10:45.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1475788700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1475f89700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 0x55a5700e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f1474f87700 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1460055320 tx=0x7f14600553f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c3ca40b11ba7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f145affd700 1 -- 172.21.15.170:0/3145803205 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460058870 con 0x55a57009bb80 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f145affd700 1 -- 172.21.15.170:0/3145803205 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146006b070 con 0x55a57009bb80 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.354+0000 7f145affd700 1 -- 172.21.15.170:0/3145803205 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146005c5c0 con 0x55a57009bb80 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3145803205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 msgr2=0x55a5700e5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1460055320 tx=0x7f14600553f0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3145803205 shutdown_connections 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700e6480 0x55a5700e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700995f0 0x55a5700c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/3145803205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e5220 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3145803205 >> 172.21.15.170:0/3145803205 conn(0x55a57009d0c0 msgr2=0x55a5700f1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3145803205 shutdown_connections 2023-10-08T16:10:45.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- 172.21.15.170:0/3145803205 wait complete. 2023-10-08T16:10:45.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.355+0000 7f1480ce4b80 1 -- start start 2023-10-08T16:10:45.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 Processor -- start 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5701503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700e6480 0x55a5700e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5700e0790 con 0x55a57009bb80 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5700e0900 con 0x55a5700e6480 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5700e19f0 con 0x55a5700995f0 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.356+0000 7f1480ce4b80 10 client.0 _ll_register_callbacks cb 0x55a57005f2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1475788700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5701503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56172/0 (socket says 172.21.15.170:56172) 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 -- 172.21.15.170:0/517172437 learned_addr learned my addr 172.21.15.170:0/517172437 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1475f89700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700e6480 0x55a5700e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 -- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 msgr2=0x55a5701503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5701503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 -- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700e6480 msgr2=0x55a5700e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700e6480 0x55a5700e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 -- 172.21.15.170:0/517172437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147016a6e0 con 0x55a57009bb80 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1474f87700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f14600550f0 tx=0x7f146001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358527429b951aeb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:10:45.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460071290 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f146005c5c0 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f146c011a90 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1460055160 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1480ce4b80 10 client.6930 fetch_fsmap learned FSMap version 65 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a56ffa1470 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c011a90 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 65) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1460051620 con 0x55a57009bb80 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.357+0000 7f1480ce4b80 10 client.6930 fetch_fsmap finished waiting for FSMap version 65 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 10 client.6930 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 20 client.6930 populate_metadata read hostname 'smithi170' 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a57009d8d0 0x55a57009dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 4 client.6930 mds_command: new command op to 6771 tid=0[{"prefix": "status"}] 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 --> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a56ff8ee00 con 0x55a57009d8d0 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1475788700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a57009d8d0 0x55a57009dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f144f7fe700 10 client.6930 ms_handle_connect on v2:172.21.15.191:6833/1404814921 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1475788700 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a57009d8d0 0x55a57009dca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c31a3ff0d4649e0 server_cookie=65de6e3144b09de6 in_seq=0 out_seq=0 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f144f7fe700 1 -- 172.21.15.170:0/517172437 <== mds.0 v2:172.21.15.191:6833/1404814921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a56ff8ee00 con 0x55a57009d8d0 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f144f7fe700 10 client.6930 handle_command_reply: tid=0 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 client.6930 shutdown 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 msgr2=0x55a5700e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f14600550f0 tx=0x7f146001c000 comp rx=0 tx=0).stop 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.358+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 shutdown_connections 2023-10-08T16:10:45.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:6833/1404814921,v1:172.21.15.191:6835/1404814921] conn(0x55a57009d8d0 0x55a57009dca0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55a5700e6480 0x55a5700e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f144f7fe700 0 client.6930 ms_handle_reset on v2:172.21.15.191:6833/1404814921 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55a5700995f0 0x55a5701503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 --2- 172.21.15.170:0/517172437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a57009bb80 0x55a5700e3780 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 >> 172.21.15.170:0/517172437 conn(0x55a57009d0c0 msgr2=0x55a5700c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 shutdown_connections 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 1 -- 172.21.15.170:0/517172437 wait complete. 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f1480ce4b80 20 client.6930 trim_cache size 0 max 16384 2023-10-08T16:10:45.367 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 6771, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.599999986, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 65, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 101, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 101, 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.708118260000001 2023-10-08T16:10:45.368 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:45.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 msgr2=0x7f1454069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 0x7f1454069740 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f146c0099b0 tx=0x7f146c016000 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 msgr2=0x7f14700cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f146000c860 tx=0x7f1460008040 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 shutdown_connections 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f147015d230 0x7f14701698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1470159420 0x7f147014a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1454067290 0x7f1454069740 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 --2- 172.21.15.170:0/194955680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14701597f0 0x7f14700cf3d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 >> 172.21.15.170:0/194955680 conn(0x7f14700d0660 msgr2=0x7f1470069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 shutdown_connections 2023-10-08T16:10:45.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.359+0000 7f147678a700 1 -- 172.21.15.170:0/194955680 wait complete. 2023-10-08T16:10:45.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:10:45.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.485+0000 7f8b6759e700 1 Processor -- start 2023-10-08T16:10:45.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.485+0000 7f8b6759e700 1 -- start start 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b68159420 0x7f8b680b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b6815d230 0x7f8b6815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6759e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b680b2980 con 0x7f8b681597f0 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6815dbb0 con 0x7f8b68159420 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b680b11d0 con 0x7f8b6815d230 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b68159420 0x7f8b680b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b68159420 0x7f8b680b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56176/0 (socket says 172.21.15.170:56176) 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 -- 172.21.15.170:0/297641386 learned_addr learned my addr 172.21.15.170:0/297641386 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b5dd9b700 1 --1- 172.21.15.170:0/297641386 >> v1:172.21.15.170:6789/0 conn(0x7f8b681597f0 0x7f8b680b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49500/0 (socket says 172.21.15.170:49500) 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b66d9d700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b6815d230 0x7f8b6815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 -- 172.21.15.170:0/297641386 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b6815d230 msgr2=0x7f8b6815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b6815d230 0x7f8b6815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 -- 172.21.15.170:0/297641386 >> v1:172.21.15.170:6789/0 conn(0x7f8b681597f0 legacy=0x7f8b680b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 -- 172.21.15.170:0/297641386 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680b14b0 con 0x7f8b68159420 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b65d9b700 1 -- 172.21.15.170:0/297641386 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599943955 0 0) 0x7f8b680b2980 con 0x7f8b681597f0 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.486+0000 7f8b6659c700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b680b5f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b580099b0 tx=0x7f8b58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcccf1c3f2f7fd82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b65d9b700 1 -- 172.21.15.170:0/297641386 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58009ea0 con 0x7f8b68159420 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b65d9b700 1 -- 172.21.15.170:0/297641386 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5800b860 con 0x7f8b68159420 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b65d9b700 1 -- 172.21.15.170:0/297641386 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800bc00 con 0x7f8b68159420 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 -- 172.21.15.170:0/297641386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 msgr2=0x7f8b680b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b680b5f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8b580099b0 tx=0x7f8b58009250 comp rx=0 tx=0).stop 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 -- 172.21.15.170:0/297641386 shutdown_connections 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b6815d230 0x7f8b6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 --2- 172.21.15.170:0/297641386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b680b5f40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 -- 172.21.15.170:0/297641386 >> 172.21.15.170:0/297641386 conn(0x7f8b680d0660 msgr2=0x7f8b680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 -- 172.21.15.170:0/297641386 shutdown_connections 2023-10-08T16:10:45.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.487+0000 7f8b6759e700 1 -- 172.21.15.170:0/297641386 wait complete. 2023-10-08T16:10:45.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.488+0000 7f8b6759e700 1 Processor -- start 2023-10-08T16:10:45.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.488+0000 7f8b6759e700 1 -- start start 2023-10-08T16:10:45.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 0x7f8b680c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 0x7f8b680c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815b3f0 con 0x7f8b68159420 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815a7e0 con 0x7f8b6815d230 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815aac0 con 0x7f8b681597f0 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56192/0 (socket says 172.21.15.170:56192) 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 -- 172.21.15.170:0/1789872484 learned_addr learned my addr 172.21.15.170:0/1789872484 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b66d9d700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 0x7f8b680c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b5dd9b700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 0x7f8b680c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 -- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 msgr2=0x7f8b680c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 0x7f8b680c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 -- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 msgr2=0x7f8b680c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 0x7f8b680c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b6659c700 1 -- 172.21.15.170:0/1789872484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580094f0 con 0x7f8b68159420 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b66d9d700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 0x7f8b680c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.489+0000 7f8b5dd9b700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 0x7f8b680c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b6659c700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b68089340 tx=0x7f8b58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800bbe0 con 0x7f8b68159420 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b58011420 con 0x7f8b68159420 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6815ada0 con 0x7f8b68159420 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580117c0 con 0x7f8b68159420 2023-10-08T16:10:45.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.490+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b681572e0 con 0x7f8b68159420 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.491+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6804c3c0 con 0x7f8b68159420 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.491+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8b58011b70 con 0x7f8b68159420 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.492+0000 7f8b5effd700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 0x7f8b48069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.492+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8b58086450 con 0x7f8b68159420 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.492+0000 7f8b5dd9b700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 0x7f8b48069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:45.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.492+0000 7f8b5dd9b700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 0x7f8b48069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ce380 tx=0x7f8b50007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:45.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.497+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b580532b0 con 0x7f8b68159420 2023-10-08T16:10:45.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.722+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b680b1790 con 0x7f8b68159420 2023-10-08T16:10:45.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.723+0000 7f8b5effd700 1 -- 172.21.15.170:0/1789872484 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b580568d0 con 0x7f8b68159420 2023-10-08T16:10:45.723 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:10:45.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 msgr2=0x7f8b48069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 0x7f8b48069740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ce380 tx=0x7f8b50007c90 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 msgr2=0x7f8b68156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8b68089340 tx=0x7f8b58017000 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 shutdown_connections 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b6815d230 0x7f8b680c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b48067290 0x7f8b48069740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b68159420 0x7f8b68156da0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 --2- 172.21.15.170:0/1789872484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b681597f0 0x7f8b680c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:45.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 >> 172.21.15.170:0/1789872484 conn(0x7f8b680d0660 msgr2=0x7f8b68069970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:45.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 shutdown_connections 2023-10-08T16:10:45.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:45.725+0000 7f8b6759e700 1 -- 172.21.15.170:0/1789872484 wait complete. 2023-10-08T16:10:45.738 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:10:45.738 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:10:45.738 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:10:45.738 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:10:45.738 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:45.739 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:10:45.739 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:10:45.739 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:10:45.739 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:45.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.739 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:10:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:10:45.755 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42105sec preferred_lft 42105sec 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:10:45.756 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.757 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:10:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:10:45.757 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:45.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.757 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:10:45.817 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:10:45.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.819 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:10:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:10:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:10:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:10:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:10:45.875 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:10:45.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:10:45.876 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:10:45.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.975 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:10:45.989 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:10:45.989 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:10:45.990 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:45.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:45.990 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:10:46.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:46.053 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:46.107 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:10:46.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:46.108 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:46.162 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:10:46.163 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:10:46.163 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:46.242 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:46.242 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:46.261 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:46.261 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:46.261 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:46.261 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:46.262 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:46.262 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:46.262 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:46.262 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:46.284 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:46.285 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:10:46.285 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:10:46.285 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:10:46.286 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:10:46.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:46.286 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:46.351 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:10:46.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.507+0000 7f8eba06e700 1 Processor -- start 2023-10-08T16:10:46.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.507+0000 7f8eba06e700 1 -- start start 2023-10-08T16:10:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eba06e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8eb40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eba06e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8eb41597f0 0x7f8eb40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eba06e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb40b11d0 con 0x7f8eb415d230 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eba06e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb40b14b0 con 0x7f8eb41597f0 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eba06e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40b1790 con 0x7f8eb4159420 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eb986d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8eb415d230 0x7f8eb40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49510/0 (socket says 172.21.15.170:49510) 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eb986d700 1 -- 172.21.15.170:0/3840144137 learned_addr learned my addr 172.21.15.170:0/3840144137 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eb906c700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8eb40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eb886b700 1 --2- 172.21.15.170:0/3840144137 >> v2:172.21.15.191:3300/0 conn(0x7f8eb41597f0 0x7f8eb40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874281876 0 0) 0x7f8eb40b11d0 con 0x7f8eb415d230 2023-10-08T16:10:46.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea0003570 con 0x7f8eb415d230 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020697402 0 0) 0x7f8ea0003570 con 0x7f8eb415d230 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 msgr2=0x7f8eb40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8eb40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 >> v2:172.21.15.191:3300/0 conn(0x7f8eb41597f0 msgr2=0x7f8eb40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 --2- 172.21.15.170:0/3840144137 >> v2:172.21.15.191:3300/0 conn(0x7f8eb41597f0 0x7f8eb40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb40b1a70 con 0x7f8eb415d230 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8eac0017f0 con 0x7f8eb415d230 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eb886b700 1 --2- 172.21.15.170:0/3840144137 >> v2:172.21.15.191:3300/0 conn(0x7f8eb41597f0 0x7f8eb40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.508+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 >> v1:172.21.15.170:6789/0 conn(0x7f8eb415d230 legacy=0x7f8eb40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ea0003a60 0x7f8ea0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0004810 0x7f8ea0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8ea00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40b14b0 con 0x7f8ea0003a60 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40b1790 con 0x7f8ea0004810 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea000ba60 con 0x7f8eb4159420 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ea0003a60 0x7f8ea0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb886b700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0004810 0x7f8ea0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb986d700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8ea00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 -- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 msgr2=0x7f8ea00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159420 0x7f8ea00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 -- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0004810 msgr2=0x7f8ea0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:46.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0004810 0x7f8ea0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 -- 172.21.15.170:0/3840144137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0003570 con 0x7f8ea0003a60 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eb906c700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ea0003a60 0x7f8ea0003e70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009c30 tx=0x7f8eb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69133f1038d71ea6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000e710 con 0x7f8ea0003a60 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb000e870 con 0x7f8ea0003a60 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.509+0000 7f8eabfff700 1 -- 172.21.15.170:0/3840144137 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb000ec30 con 0x7f8ea0003a60 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 -- 172.21.15.170:0/3840144137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ea0003a60 msgr2=0x7f8ea0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 --2- 172.21.15.170:0/3840144137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ea0003a60 0x7f8ea0003e70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009c30 tx=0x7f8eb00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eb986d700 1 -- 172.21.15.170:0/3840144137 reap_dead start 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 -- 172.21.15.170:0/3840144137 shutdown_connections 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 -- 172.21.15.170:0/3840144137 >> 172.21.15.170:0/3840144137 conn(0x7f8eb40d0660 msgr2=0x7f8eb40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:46.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 -- 172.21.15.170:0/3840144137 shutdown_connections 2023-10-08T16:10:46.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.510+0000 7f8eba06e700 1 -- 172.21.15.170:0/3840144137 wait complete. 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.511+0000 7f8eba06e700 1 Processor -- start 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.511+0000 7f8eba06e700 1 -- start start 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0003a60 0x7f8eb415c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 0x7f8eb4141e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4150fa0 con 0x7f8eb41423a0 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb415c660 con 0x7f8ea0003a60 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4150c70 con 0x7f8eb4159bc0 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56214/0 (socket says 172.21.15.170:56214) 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 -- 172.21.15.170:0/4012380449 learned_addr learned my addr 172.21.15.170:0/4012380449 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb906c700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0003a60 0x7f8eb415c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb886b700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 0x7f8eb4141e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 -- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 msgr2=0x7f8eb4141e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 0x7f8eb4141e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 -- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0003a60 msgr2=0x7f8eb415c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0003a60 0x7f8eb415c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 -- 172.21.15.170:0/4012380449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb415c0a0 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb886b700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 0x7f8eb4141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eb986d700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40b3180 tx=0x7f8eac001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00ebc0 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac00ed60 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb415c380 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac013670 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.512+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb4150770 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.513+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb4151150 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.514+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f8eac013a60 con 0x7f8eb41423a0 2023-10-08T16:10:46.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.514+0000 7f8ea97fa700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 0x7f8e98069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:46.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.514+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8eac085000 con 0x7f8eb41423a0 2023-10-08T16:10:46.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.514+0000 7f8eb906c700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 0x7f8e98069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:46.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.515+0000 7f8eb906c700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 0x7f8e98069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00091e0 tx=0x7f8eb001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:46.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.518+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8eb4151150 con 0x7f8eb41423a0 2023-10-08T16:10:46.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.746+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8eb404c3c0 con 0x7f8eb41423a0 2023-10-08T16:10:46.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.746+0000 7f8ea97fa700 1 -- 172.21.15.170:0/4012380449 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8eac051e60 con 0x7f8eb41423a0 2023-10-08T16:10:46.747 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:10:46.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.748+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 msgr2=0x7f8e98069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.748+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 0x7f8e98069740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00091e0 tx=0x7f8eb001a000 comp rx=0 tx=0).stop 2023-10-08T16:10:46.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 msgr2=0x7f8eb4151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:46.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40b3180 tx=0x7f8eac001860 comp rx=0 tx=0).stop 2023-10-08T16:10:46.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 shutdown_connections 2023-10-08T16:10:46.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e98067290 0x7f8e98069740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8eb41423a0 0x7f8eb4151430 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ea0003a60 0x7f8eb415c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 --2- 172.21.15.170:0/4012380449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8eb4159bc0 0x7f8eb4141e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:46.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 >> 172.21.15.170:0/4012380449 conn(0x7f8eb40d0660 msgr2=0x7f8eb40b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:46.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 shutdown_connections 2023-10-08T16:10:46.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:46.749+0000 7f8eba06e700 1 -- 172.21.15.170:0/4012380449 wait complete. 2023-10-08T16:10:46.761 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:10:46.761 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:10:46.761 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:10:46.761 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:10:46.761 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:46.762 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:10:46.762 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:10:46.762 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:10:46.762 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:46.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:46.762 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:10:46.779 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:10:46.779 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:10:46.779 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:10:46.780 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42105sec preferred_lft 42105sec 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:10:46.781 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:10:46.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:46.783 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:10:46.846 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:10:46.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:46.847 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:10:46.906 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:10:46.906 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:10:46.906 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:10:46.906 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:10:46.906 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:10:46.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:10:46.907 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:10:47.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:47.011 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:10:47.026 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:10:47.026 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:47.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:47.026 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:10:47.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:47.091 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:47.147 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:10:47.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:10:47.147 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:47.201 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:10:47.201 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:10:47.202 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:47.283 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:47.302 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:10:47.324 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:10:47.325 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:10:47.326 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:10:47.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:47.326 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:47.390 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:10:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.511+0000 7f8117a50700 1 Processor -- start 2023-10-08T16:10:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.511+0000 7f8117a50700 1 -- start start 2023-10-08T16:10:47.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8117a50700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8110155360 0x7f81100b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8117a50700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f81100ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8117a50700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100b4560 con 0x7f81100ceb60 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8117a50700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811015c320 con 0x7f8110155360 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8117a50700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100cef90 con 0x7f81100b40f0 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f8114feb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f81100ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.512+0000 7f81157ec700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8110155360 0x7f81100b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8115fed700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f81100ceb60 0x7f811006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:52256/0 (socket says 172.21.15.170:52256) 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8110155360 0x7f81100b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:36066/0 (socket says 172.21.15.170:36066) 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8115fed700 1 -- 172.21.15.170:0/940644379 learned_addr learned my addr 172.21.15.170:0/940644379 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 -- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 msgr2=0x7f81100ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f81100ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 -- 172.21.15.170:0/940644379 >> v1:172.21.15.191:6789/0 conn(0x7f81100ceb60 legacy=0x7f811006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8107fff700 1 -- 172.21.15.170:0/940644379 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371260473 0 0) 0x7f81100b4560 con 0x7f81100ceb60 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 -- 172.21.15.170:0/940644379 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811015b710 con 0x7f8110155360 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8114feb700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f81100ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f81157ec700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8110155360 0x7f81100b3bb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f810c009e60 tx=0x7f810c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e8f6d543a6838ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8107fff700 1 -- 172.21.15.170:0/940644379 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00e070 con 0x7f8110155360 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8107fff700 1 -- 172.21.15.170:0/940644379 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c00bb60 con 0x7f8110155360 2023-10-08T16:10:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.513+0000 7f8107fff700 1 -- 172.21.15.170:0/940644379 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c012430 con 0x7f8110155360 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 -- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8110155360 msgr2=0x7f81100b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8110155360 0x7f81100b3bb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f810c009e60 tx=0x7f810c009780 comp rx=0 tx=0).stop 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 -- 172.21.15.170:0/940644379 shutdown_connections 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8110155360 0x7f81100b3bb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 --2- 172.21.15.170:0/940644379 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f81100ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 -- 172.21.15.170:0/940644379 >> 172.21.15.170:0/940644379 conn(0x7f811006baf0 msgr2=0x7f811006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:47.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 -- 172.21.15.170:0/940644379 shutdown_connections 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.514+0000 7f8117a50700 1 -- 172.21.15.170:0/940644379 wait complete. 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.515+0000 7f8117a50700 1 Processor -- start 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.515+0000 7f8117a50700 1 -- start start 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f811015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81100ceb60 0x7f811014bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811015ab20 con 0x7f8110155360 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811016b500 con 0x7f81100ceb60 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8117a50700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811016b7e0 con 0x7f81100b40f0 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56228/0 (socket says 172.21.15.170:56228) 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 -- 172.21.15.170:0/520491549 learned_addr learned my addr 172.21.15.170:0/520491549 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f81157ec700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f811015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8114feb700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81100ceb60 0x7f811014bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 -- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 msgr2=0x7f811015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f811015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 -- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81100ceb60 msgr2=0x7f811014bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81100ceb60 0x7f811014bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.516+0000 7f8115fed700 1 -- 172.21.15.170:0/520491549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100009420 con 0x7f8110155360 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f8115fed700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f810800c7e0 tx=0x7f8108007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108005b30 con 0x7f8110155360 2023-10-08T16:10:47.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108005c90 con 0x7f8110155360 2023-10-08T16:10:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c009a20 con 0x7f8110155360 2023-10-08T16:10:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108007320 con 0x7f8110155360 2023-10-08T16:10:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.517+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811016bda0 con 0x7f8110155360 2023-10-08T16:10:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.518+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100718b0 con 0x7f8110155360 2023-10-08T16:10:47.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.518+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f810801b3b0 con 0x7f8110155360 2023-10-08T16:10:47.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.518+0000 7f81057fa700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 0x7f80f4069770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:47.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.519+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8108083e40 con 0x7f8110155360 2023-10-08T16:10:47.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.519+0000 7f81157ec700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 0x7f80f4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:47.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.519+0000 7f81157ec700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 0x7f80f4069770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f810c0097c0 tx=0x7f810c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:47.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.523+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8108050ca0 con 0x7f8110155360 2023-10-08T16:10:47.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:47.750+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f811015b9f0 con 0x7f8110155360 2023-10-08T16:10:48.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.501+0000 7f81057fa700 1 -- 172.21.15.170:0/520491549 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 136+0+0 (secure 0 0 0) 0x7f81080170c0 con 0x7f8110155360 2023-10-08T16:10:48.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 msgr2=0x7f80f4069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 0x7f80f4069770 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f810c0097c0 tx=0x7f810c019000 comp rx=0 tx=0).stop 2023-10-08T16:10:48.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 msgr2=0x7f811006d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f810800c7e0 tx=0x7f8108007d90 comp rx=0 tx=0).stop 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 shutdown_connections 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f40672c0 0x7f80f4069770 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8110155360 0x7f811006d130 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81100b40f0 0x7f811015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 --2- 172.21.15.170:0/520491549 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81100ceb60 0x7f811014bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 >> 172.21.15.170:0/520491549 conn(0x7f811006baf0 msgr2=0x7f81100ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 shutdown_connections 2023-10-08T16:10:48.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.504+0000 7f8117a50700 1 -- 172.21.15.170:0/520491549 wait complete. 2023-10-08T16:10:48.517 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:10:48.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.632+0000 7f0b1e444700 1 Processor -- start 2023-10-08T16:10:48.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.632+0000 7f0b1e444700 1 -- start start 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1e444700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b180ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1e444700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180cc300 con 0x7f0b180ccf90 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1e444700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b180ad370 con 0x7f0b180cbf30 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1e444700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18156bf0 con 0x7f0b180b1420 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1c9e1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0b180ccf90 0x7f0b180c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49522/0 (socket says 172.21.15.170:49522) 2023-10-08T16:10:48.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/2647230594 learned_addr learned my addr 172.21.15.170:0/2647230594 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449973119 0 0) 0x7f0b180cc300 con 0x7f0b180ccf90 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b177fe700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b180ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.633+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00003570 con 0x7f0b180ccf90 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 495423489 0 0) 0x7f0b00003570 con 0x7f0b180ccf90 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 msgr2=0x7f0b180ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b180ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 >> v1:172.21.15.191:6789/0 conn(0x7f0b180cbf30 legacy=0x7f0b180b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b180ad650 con 0x7f0b180ccf90 2023-10-08T16:10:48.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f0b100017f0 con 0x7f0b180ccf90 2023-10-08T16:10:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 >> v1:172.21.15.170:6789/0 conn(0x7f0b180ccf90 legacy=0x7f0b180c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00003a90 0x7f0b00003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b00004810 0x7f0b00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b000051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b18156bf0 con 0x7f0b00004810 2023-10-08T16:10:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000bb60 con 0x7f0b00003a90 2023-10-08T16:10:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000bd20 con 0x7f0b180b1420 2023-10-08T16:10:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b00004810 0x7f0b00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b17fff700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00003a90 0x7f0b00003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b1c9e1700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b000051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 -- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 msgr2=0x7f0b000051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180b1420 0x7f0b000051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 -- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00003a90 msgr2=0x7f0b00003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00003a90 0x7f0b00003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 -- 172.21.15.170:0/2647230594 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00003570 con 0x7f0b00004810 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b177fe700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b00004810 0x7f0b00004c80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009c00 tx=0x7f0b0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683eed8155bb5d76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00e710 con 0x7f0b00004810 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00e870 con 0x7f0b00004810 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.634+0000 7f0b16ffd700 1 -- 172.21.15.170:0/2647230594 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00ec30 con 0x7f0b00004810 2023-10-08T16:10:48.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 -- 172.21.15.170:0/2647230594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b00004810 msgr2=0x7f0b00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 --2- 172.21.15.170:0/2647230594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b00004810 0x7f0b00004c80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009c00 tx=0x7f0b0c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/2647230594 reap_dead start 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 -- 172.21.15.170:0/2647230594 shutdown_connections 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 -- 172.21.15.170:0/2647230594 >> 172.21.15.170:0/2647230594 conn(0x7f0b1806a320 msgr2=0x7f0b180cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 -- 172.21.15.170:0/2647230594 shutdown_connections 2023-10-08T16:10:48.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.635+0000 7f0b1e444700 1 -- 172.21.15.170:0/2647230594 wait complete. 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.636+0000 7f0b1e444700 1 Processor -- start 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.636+0000 7f0b1e444700 1 -- start start 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00004810 0x7f0b180a9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180a82f0 0x7f0b18154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180a9e70 con 0x7f0b18167900 2023-10-08T16:10:48.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180ab8a0 con 0x7f0b00004810 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1e444700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180a9bd0 con 0x7f0b180a82f0 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56232/0 (socket says 172.21.15.170:56232) 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/747036492 learned_addr learned my addr 172.21.15.170:0/747036492 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b17fff700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00004810 0x7f0b180a9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b177fe700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180a82f0 0x7f0b18154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180a82f0 msgr2=0x7f0b18154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180a82f0 0x7f0b18154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00004810 msgr2=0x7f0b180a9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.637+0000 7f0b1c9e1700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00004810 0x7f0b180a9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0b1c9e1700 1 -- 172.21.15.170:0/747036492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1806d9c0 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0b1c9e1700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0b180cd360 tx=0x7f0b100021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1000ac90 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1000ae10 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10013640 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1806dca0 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.638+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1806e260 con 0x7f0b18167900 2023-10-08T16:10:48.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.639+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1806e450 con 0x7f0b18167900 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.639+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f0b10013a70 con 0x7f0b18167900 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.639+0000 7f0afffff700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 0x7f0b04069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.640+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0b100855a0 con 0x7f0b18167900 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.640+0000 7f0b17fff700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 0x7f0b04069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.640+0000 7f0b17fff700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 0x7f0b04069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0b08003bb0 tx=0x7f0b08007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:48.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.644+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b100523c0 con 0x7f0b18167900 2023-10-08T16:10:48.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.855+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b1804c3c0 con 0x7f0b04067290 2023-10-08T16:10:48.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.856+0000 7f0afffff700 1 -- 172.21.15.170:0/747036492 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0b1804c3c0 con 0x7f0b04067290 2023-10-08T16:10:48.856 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:10:48.856 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:48.857 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:10:48.857 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:48.857 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:10:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 msgr2=0x7f0b04069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 0x7f0b04069740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0b08003bb0 tx=0x7f0b08007d30 comp rx=0 tx=0).stop 2023-10-08T16:10:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 msgr2=0x7f0b180aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0b180cd360 tx=0x7f0b100021d0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 shutdown_connections 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0b04067290 0x7f0b04069740 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b18167900 0x7f0b180aaa40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0b00004810 0x7f0b180a9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 --2- 172.21.15.170:0/747036492 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0b180a82f0 0x7f0b18154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 >> 172.21.15.170:0/747036492 conn(0x7f0b1806a320 msgr2=0x7f0b18155f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 shutdown_connections 2023-10-08T16:10:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.858+0000 7f0b1e444700 1 -- 172.21.15.170:0/747036492 wait complete. 2023-10-08T16:10:48.871 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:10:48.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.987+0000 7fdd9a167700 1 Processor -- start 2023-10-08T16:10:48.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.987+0000 7fdd9a167700 1 -- start start 2023-10-08T16:10:48.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd9a167700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd94159890 0x7fdd9415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:48.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd9a167700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9415a160 con 0x7fdd94159c60 2023-10-08T16:10:48.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd9a167700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940d13a0 con 0x7fdd94069560 2023-10-08T16:10:48.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd9a167700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9415e140 con 0x7fdd94159890 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd937fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd94159890 0x7fdd9415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd937fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd94159890 0x7fdd9415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56248/0 (socket says 172.21.15.170:56248) 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd937fe700 1 -- 172.21.15.170:0/1871828245 learned_addr learned my addr 172.21.15.170:0/1871828245 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.988+0000 7fdd92ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdd94159c60 0x7fdd9415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49534/0 (socket says 172.21.15.170:49534) 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd937fe700 1 -- 172.21.15.170:0/1871828245 >> v1:172.21.15.191:6789/0 conn(0x7fdd94069560 legacy=0x7fdd940ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd937fe700 1 -- 172.21.15.170:0/1871828245 >> v1:172.21.15.170:6789/0 conn(0x7fdd94159c60 legacy=0x7fdd9415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd937fe700 1 -- 172.21.15.170:0/1871828245 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9415a3d0 con 0x7fdd94159890 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd927fc700 1 -- 172.21.15.170:0/1871828245 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415900943 0 0) 0x7fdd9415a160 con 0x7fdd94159c60 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd937fe700 1 --2- 172.21.15.170:0/1871828245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94159890 0x7fdd9415abd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009f20 tx=0x7fdd84009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed34102ab556313b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd927fc700 1 -- 172.21.15.170:0/1871828245 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e070 con 0x7fdd94159890 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd927fc700 1 -- 172.21.15.170:0/1871828245 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400ba80 con 0x7fdd94159890 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd927fc700 1 -- 172.21.15.170:0/1871828245 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840092e0 con 0x7fdd94159890 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 -- 172.21.15.170:0/1871828245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94159890 msgr2=0x7fdd9415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 --2- 172.21.15.170:0/1871828245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94159890 0x7fdd9415abd0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009f20 tx=0x7fdd84009840 comp rx=0 tx=0).stop 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 -- 172.21.15.170:0/1871828245 shutdown_connections 2023-10-08T16:10:48.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 --2- 172.21.15.170:0/1871828245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94159890 0x7fdd9415abd0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:48.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 -- 172.21.15.170:0/1871828245 >> 172.21.15.170:0/1871828245 conn(0x7fdd94068e20 msgr2=0x7fdd940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:48.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.989+0000 7fdd9a167700 1 -- 172.21.15.170:0/1871828245 shutdown_connections 2023-10-08T16:10:48.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.990+0000 7fdd9a167700 1 -- 172.21.15.170:0/1871828245 wait complete. 2023-10-08T16:10:48.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 Processor -- start 2023-10-08T16:10:48.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 -- start start 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd94159890 0x7fdd94152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd94159c60 0x7fdd9415d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415de90 con 0x7fdd94069560 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94152730 con 0x7fdd94159c60 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd9a167700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd94151b20 con 0x7fdd94159890 2023-10-08T16:10:49.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.991+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56258/0 (socket says 172.21.15.170:56258) 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 -- 172.21.15.170:0/4059114813 learned_addr learned my addr 172.21.15.170:0/4059114813 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd92ffd700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd94159890 0x7fdd94152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd93fff700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd94159c60 0x7fdd9415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 -- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd94159890 msgr2=0x7fdd94152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd94159890 0x7fdd94152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 -- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd94159c60 msgr2=0x7fdd9415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd94159c60 0x7fdd9415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 -- 172.21.15.170:0/4059114813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009ae0 con 0x7fdd94069560 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd937fe700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009ab0 tx=0x7fdd84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007b60 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84007ce0 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd94151e00 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840266b0 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.992+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd941523c0 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.993+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94152a10 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.994+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7fdd8401d420 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.994+0000 7fdd7b7fe700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 0x7fdd80069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.994+0000 7fdd92ffd700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 0x7fdd80069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.994+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdd84086c10 con 0x7fdd94069560 2023-10-08T16:10:49.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.994+0000 7fdd92ffd700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 0x7fdd80069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd940cc720 tx=0x7fdd88007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:48.998+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd84053960 con 0x7fdd94069560 2023-10-08T16:10:49.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.180+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdd9404c3c0 con 0x7fdd94069560 2023-10-08T16:10:49.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.180+0000 7fdd7b7fe700 1 -- 172.21.15.170:0/4059114813 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdd84056f80 con 0x7fdd94069560 2023-10-08T16:10:49.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 msgr2=0x7fdd80069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 0x7fdd80069740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fdd940cc720 tx=0x7fdd88007d20 comp rx=0 tx=0).stop 2023-10-08T16:10:49.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 msgr2=0x7fdd94151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009ab0 tx=0x7fdd84018000 comp rx=0 tx=0).stop 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 shutdown_connections 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.182+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd94159c60 0x7fdd9415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd80067290 0x7fdd80069740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94069560 0x7fdd94151540 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 --2- 172.21.15.170:0/4059114813 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd94159890 0x7fdd94152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 >> 172.21.15.170:0/4059114813 conn(0x7fdd94068e20 msgr2=0x7fdd940c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 shutdown_connections 2023-10-08T16:10:49.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.183+0000 7fdd9a167700 1 -- 172.21.15.170:0/4059114813 wait complete. 2023-10-08T16:10:49.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume info cephfs 2023-10-08T16:10:49.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.325+0000 7f991757a700 1 Processor -- start 2023-10-08T16:10:49.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.325+0000 7f991757a700 1 -- start start 2023-10-08T16:10:49.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f991757a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f991006c0a0 0x7f99100caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f991757a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991015d7e0 con 0x7f991015df50 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f991757a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991015dac0 con 0x7f991006bcd0 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f991757a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991015aa30 con 0x7f991006c0a0 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f9915b17700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f991015df50 0x7f99100c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49536/0 (socket says 172.21.15.170:49536) 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f9915b17700 1 -- 172.21.15.170:0/4219503024 learned_addr learned my addr 172.21.15.170:0/4219503024 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807197039 0 0) 0x7f991015d7e0 con 0x7f991015df50 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f9914b15700 1 --2- 172.21.15.170:0/4219503024 >> v2:172.21.15.191:3300/0 conn(0x7f991006c0a0 0x7f99100caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f8003570 con 0x7f991015df50 2023-10-08T16:10:49.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330677445 0 0) 0x7f98f8003570 con 0x7f991015df50 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 >> v2:172.21.15.191:3300/0 conn(0x7f991006c0a0 msgr2=0x7f99100caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 --2- 172.21.15.170:0/4219503024 >> v2:172.21.15.191:3300/0 conn(0x7f991006c0a0 0x7f99100caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 >> v1:172.21.15.191:6789/0 conn(0x7f991006bcd0 legacy=0x7f991015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991015ad10 con 0x7f991015df50 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.326+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f9900001470 con 0x7f991015df50 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 >> v1:172.21.15.170:6789/0 conn(0x7f991015df50 legacy=0x7f99100c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 0x7f98f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8004810 0x7f98f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98f80052e0 0x7f98f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991015aa30 con 0x7f98f8003a60 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f800c2a0 con 0x7f98f8004810 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f800c460 con 0x7f98f80052e0 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 0x7f98f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9914b15700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8004810 0x7f98f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915b17700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98f80052e0 0x7f98f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 -- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98f80052e0 msgr2=0x7f98f800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98f80052e0 0x7f98f800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 -- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8004810 msgr2=0x7f98f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8004810 0x7f98f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 -- 172.21.15.170:0/4219503024 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f8003570 con 0x7f98f8003a60 2023-10-08T16:10:49.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915b17700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f98f80052e0 0x7f98f800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915b17700 1 -- 172.21.15.170:0/4219503024 reap_dead start 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f9915316700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 0x7f98f8003e70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9908001fc0 tx=0x7f9908001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f425e93bf96dd90 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908001950 con 0x7f98f8003a60 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800c8b0 con 0x7f98f8003a60 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.327+0000 7f990ffff700 1 -- 172.21.15.170:0/4219503024 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800cc50 con 0x7f98f8003a60 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 -- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 msgr2=0x7f98f8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 0x7f98f8003e70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9908001fc0 tx=0x7f9908001640 comp rx=0 tx=0).stop 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 -- 172.21.15.170:0/4219503024 shutdown_connections 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 --2- 172.21.15.170:0/4219503024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f8003a60 0x7f98f8003e70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 -- 172.21.15.170:0/4219503024 >> 172.21.15.170:0/4219503024 conn(0x7f99100ce600 msgr2=0x7f99100cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:49.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 -- 172.21.15.170:0/4219503024 shutdown_connections 2023-10-08T16:10:49.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.328+0000 7f991757a700 1 -- 172.21.15.170:0/4219503024 wait complete. 2023-10-08T16:10:49.335 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.329+0000 7f991757a700 1 Processor -- start 2023-10-08T16:10:49.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.329+0000 7f991757a700 1 -- start start 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8003a60 0x7f9910142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 0x7f99100c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991016db70 con 0x7f99100c9260 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100c8dd0 con 0x7f98f8003a60 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100c81c0 con 0x7f991016d6d0 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56280/0 (socket says 172.21.15.170:56280) 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 -- 172.21.15.170:0/4096527885 learned_addr learned my addr 172.21.15.170:0/4096527885 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:49.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9915316700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8003a60 0x7f9910142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9915b17700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 0x7f99100c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 -- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 msgr2=0x7f99100c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 0x7f99100c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 -- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8003a60 msgr2=0x7f9910142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8003a60 0x7f9910142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 -- 172.21.15.170:0/4096527885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100c84a0 con 0x7f99100c9260 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9915b17700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 0x7f99100c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f9914b15700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9904011d70 tx=0x7f99040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904009e20 con 0x7f99100c9260 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400fcd0 con 0x7f99100c9260 2023-10-08T16:10:49.365 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99100c8780 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904014650 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.330+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99100c96d0 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.331+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99100c8f60 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.332+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f9904014a80 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.332+0000 7f990d7fa700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 0x7f98fc069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.332+0000 7f9915316700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 0x7f98fc069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.332+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9904018030 con 0x7f99100c9260 2023-10-08T16:10:49.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.333+0000 7f9915316700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 0x7f98fc069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99100cc050 tx=0x7f9908015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.337+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9904052340 con 0x7f99100c9260 2023-10-08T16:10:49.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.546+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume info", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f991004c3c0 con 0x7f98fc067290 2023-10-08T16:10:49.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:49.550+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:49.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:49.550+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:49.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:49.550+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:49.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:49.550+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:49.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:10:49.550+0000 7f60a202f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.556+0000 7f990d7fa700 1 -- 172.21.15.170:0/4096527885 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+473 (secure 0 0 0) 0x7f991004c3c0 con 0x7f98fc067290 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout: "mon_addrs": [ 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.170:6789", 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6789", 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout: "172.21.15.191:6790" 2023-10-08T16:10:49.556 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "pools": { 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "data": [ 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365977731072, 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_data", 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 0 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: ], 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "metadata": [ 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:10:49.557 INFO:teuthology.orchestra.run.smithi170.stdout: "avail": 365977731072, 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs_metadata", 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout: "used": 65536 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:10:49.558 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:10:49.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 msgr2=0x7f98fc069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 0x7f98fc069740 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f99100cc050 tx=0x7f9908015000 comp rx=0 tx=0).stop 2023-10-08T16:10:49.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 msgr2=0x7f991016d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9904011d70 tx=0x7f99040091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 shutdown_connections 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f991016d6d0 0x7f99100c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f98f8003a60 0x7f9910142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f98fc067290 0x7f98fc069740 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 --2- 172.21.15.170:0/4096527885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99100c9260 0x7f991016d190 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 >> 172.21.15.170:0/4096527885 conn(0x7f99100ce600 msgr2=0x7f99101571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 shutdown_connections 2023-10-08T16:10:49.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.558+0000 7f991757a700 1 -- 172.21.15.170:0/4096527885 wait complete. 2023-10-08T16:10:49.570 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:10:49.570 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:49.608 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:49.710 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:10:49.711 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:10:49.711 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:10:49.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:10:49.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:10:49.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:49.761 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:10:49.776 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:10:49.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:10:49.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:10:49.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:10:49.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:10:49.818 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:10:49.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:49.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.988+0000 7f2c58bf9700 1 Processor -- start 2023-10-08T16:10:49.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.988+0000 7f2c58bf9700 1 -- start start 2023-10-08T16:10:49.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.989+0000 7f2c58bf9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2c540aca80 0x7f2c5406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.989+0000 7f2c58bf9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2c540c83c0 0x7f2c540cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.989+0000 7f2c58bf9700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5406bdb0 con 0x7f2c54069d60 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.989+0000 7f2c58bf9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c540cb370 con 0x7f2c540c83c0 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.989+0000 7f2c58bf9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c54159f60 con 0x7f2c540aca80 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2c540c83c0 0x7f2c540cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2c540c83c0 0x7f2c540cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56314/0 (socket says 172.21.15.170:56314) 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 -- 172.21.15.170:0/564936158 learned_addr learned my addr 172.21.15.170:0/564936158 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c5259c700 1 --2- 172.21.15.170:0/564936158 >> v2:172.21.15.191:3300/0 conn(0x7f2c540aca80 0x7f2c5406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 -- 172.21.15.170:0/564936158 >> v2:172.21.15.191:3300/0 conn(0x7f2c540aca80 msgr2=0x7f2c5406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 --2- 172.21.15.170:0/564936158 >> v2:172.21.15.191:3300/0 conn(0x7f2c540aca80 0x7f2c5406aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 -- 172.21.15.170:0/564936158 >> v1:172.21.15.191:6789/0 conn(0x7f2c54069d60 legacy=0x7f2c540c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 -- 172.21.15.170:0/564936158 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5406b510 con 0x7f2c540c83c0 2023-10-08T16:10:49.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c52d9d700 1 --2- 172.21.15.170:0/564936158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c540c83c0 0x7f2c540cb800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2c5406e570 tx=0x7f2c4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f548deb93118e7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c5159a700 1 -- 172.21.15.170:0/564936158 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c009e70 con 0x7f2c540c83c0 2023-10-08T16:10:49.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c5159a700 1 -- 172.21.15.170:0/564936158 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c00b860 con 0x7f2c540c83c0 2023-10-08T16:10:49.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.990+0000 7f2c5159a700 1 -- 172.21.15.170:0/564936158 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c00bc00 con 0x7f2c540c83c0 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 -- 172.21.15.170:0/564936158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c540c83c0 msgr2=0x7f2c540cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/564936158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c540c83c0 0x7f2c540cb800 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2c5406e570 tx=0x7f2c4c009250 comp rx=0 tx=0).stop 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 -- 172.21.15.170:0/564936158 shutdown_connections 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/564936158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c540c83c0 0x7f2c540cb800 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/564936158 >> v2:172.21.15.191:3300/0 conn(0x7f2c540aca80 0x7f2c5406aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 -- 172.21.15.170:0/564936158 >> 172.21.15.170:0/564936158 conn(0x7f2c540cd670 msgr2=0x7f2c540cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 -- 172.21.15.170:0/564936158 shutdown_connections 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.991+0000 7f2c58bf9700 1 -- 172.21.15.170:0/564936158 wait complete. 2023-10-08T16:10:49.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.992+0000 7f2c58bf9700 1 Processor -- start 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.992+0000 7f2c58bf9700 1 -- start start 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 0x7f2c540b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 0x7f2c5415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540aee50 con 0x7f2c54069d60 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540af820 con 0x7f2c540c83c0 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540ae240 con 0x7f2c540aca80 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56322/0 (socket says 172.21.15.170:56322) 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 -- 172.21.15.170:0/4047504818 learned_addr learned my addr 172.21.15.170:0/4047504818 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c52d9d700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 0x7f2c5415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c51d9b700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 0x7f2c540b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 -- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 msgr2=0x7f2c540b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 0x7f2c540b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 -- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 msgr2=0x7f2c5415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 0x7f2c5415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 -- 172.21.15.170:0/4047504818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0094f0 con 0x7f2c54069d60 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c52d9d700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 0x7f2c5415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c51d9b700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 0x7f2c540b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c5259c700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009f40 tx=0x7f2c4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400ccd0 con 0x7f2c54069d60 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4400f040 con 0x7f2c54069d60 2023-10-08T16:10:49.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c540ae520 con 0x7f2c54069d60 2023-10-08T16:10:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440146b0 con 0x7f2c54069d60 2023-10-08T16:10:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.993+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c540aeae0 con 0x7f2c54069d60 2023-10-08T16:10:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.994+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c540af130 con 0x7f2c54069d60 2023-10-08T16:10:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.995+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 11) v1 ==== 85046+0+0 (secure 0 0 0) 0x7f2c44014a20 con 0x7f2c54069d60 2023-10-08T16:10:49.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.995+0000 7f2c427fc700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 0x7f2c34069740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.995+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c440853c0 con 0x7f2c54069d60 2023-10-08T16:10:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.995+0000 7f2c51d9b700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 0x7f2c34069740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.996+0000 7f2c51d9b700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 0x7f2c34069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c5415f340 tx=0x7f2c48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:49.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:49.999+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c44052110 con 0x7f2c54069d60 2023-10-08T16:10:50.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.223+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c540aeca0 con 0x7f2c54069d60 2023-10-08T16:10:50.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.224+0000 7f2c427fc700 1 -- 172.21.15.170:0/4047504818 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f2c44055730 con 0x7f2c54069d60 2023-10-08T16:10:50.434 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:50.434 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:50.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 msgr2=0x7f2c34069740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 0x7f2c34069740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2c5415f340 tx=0x7f2c48009040 comp rx=0 tx=0).stop 2023-10-08T16:10:50.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 msgr2=0x7f2c540af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009f40 tx=0x7f2c4400b040 comp rx=0 tx=0).stop 2023-10-08T16:10:50.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 shutdown_connections 2023-10-08T16:10:50.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c540c83c0 0x7f2c5415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c34067290 0x7f2c34069740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c54069d60 0x7f2c540af2e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 --2- 172.21.15.170:0/4047504818 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c540aca80 0x7f2c540b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 >> 172.21.15.170:0/4047504818 conn(0x7f2c540cd670 msgr2=0x7f2c54158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:50.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 shutdown_connections 2023-10-08T16:10:50.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.226+0000 7f2c58bf9700 1 -- 172.21.15.170:0/4047504818 wait complete. 2023-10-08T16:10:50.439 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 66 2023-10-08T16:10:50.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:50.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.557+0000 7efcc85c4700 1 Processor -- start 2023-10-08T16:10:50.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.557+0000 7efcc85c4700 1 -- start start 2023-10-08T16:10:50.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc85c4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efcc01599c0 0x7efcc0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc85c4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efcc0069aa0 0x7efcc00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc85c4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc006a220 con 0x7efcc015ab90 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc85c4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc0069f40 con 0x7efcc0069aa0 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc85c4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc015a0b0 con 0x7efcc01599c0 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc5b5f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efcc0069aa0 0x7efcc00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc5b5f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efcc0069aa0 0x7efcc00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56324/0 (socket says 172.21.15.170:56324) 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.558+0000 7efcc5b5f700 1 -- 172.21.15.170:0/4075820242 learned_addr learned my addr 172.21.15.170:0/4075820242 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:50.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc6360700 1 --2- 172.21.15.170:0/4075820242 >> v2:172.21.15.191:3300/0 conn(0x7efcc01599c0 0x7efcc0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc5b5f700 1 -- 172.21.15.170:0/4075820242 >> v2:172.21.15.191:3300/0 conn(0x7efcc01599c0 msgr2=0x7efcc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc5b5f700 1 --2- 172.21.15.170:0/4075820242 >> v2:172.21.15.191:3300/0 conn(0x7efcc01599c0 0x7efcc0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc5b5f700 1 -- 172.21.15.170:0/4075820242 >> v1:172.21.15.191:6789/0 conn(0x7efcc015ab90 legacy=0x7efcc00ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc5b5f700 1 -- 172.21.15.170:0/4075820242 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc015a390 con 0x7efcc0069aa0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc5b5f700 1 --2- 172.21.15.170:0/4075820242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00cb5f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7efcb00099b0 tx=0x7efcb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ee85162344ec9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc535e700 1 -- 172.21.15.170:0/4075820242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb000b820 con 0x7efcc0069aa0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc535e700 1 -- 172.21.15.170:0/4075820242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb000b980 con 0x7efcc0069aa0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc535e700 1 -- 172.21.15.170:0/4075820242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb000bd20 con 0x7efcc0069aa0 2023-10-08T16:10:50.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc85c4700 1 -- 172.21.15.170:0/4075820242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 msgr2=0x7efcc00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc85c4700 1 --2- 172.21.15.170:0/4075820242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00cb5f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7efcb00099b0 tx=0x7efcb0009250 comp rx=0 tx=0).stop 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc85c4700 1 -- 172.21.15.170:0/4075820242 shutdown_connections 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc85c4700 1 --2- 172.21.15.170:0/4075820242 >> v2:172.21.15.191:3300/0 conn(0x7efcc01599c0 0x7efcc0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.559+0000 7efcc85c4700 1 --2- 172.21.15.170:0/4075820242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00cb5f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.560+0000 7efcc85c4700 1 -- 172.21.15.170:0/4075820242 >> 172.21.15.170:0/4075820242 conn(0x7efcc0068e10 msgr2=0x7efcc00d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.560+0000 7efcc85c4700 1 -- 172.21.15.170:0/4075820242 shutdown_connections 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.560+0000 7efcc85c4700 1 -- 172.21.15.170:0/4075820242 wait complete. 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 Processor -- start 2023-10-08T16:10:50.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 -- start start 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc01599c0 0x7efcc0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcc015ab90 0x7efcc01527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0157ac0 con 0x7efcc0069aa0 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00c8e80 con 0x7efcc01599c0 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc85c4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0152cf0 con 0x7efcc015ab90 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc6360700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc6360700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56330/0 (socket says 172.21.15.170:56330) 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc6360700 1 -- 172.21.15.170:0/210849470 learned_addr learned my addr 172.21.15.170:0/210849470 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc5b5f700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc01599c0 0x7efcc0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.561+0000 7efcc6b61700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcc015ab90 0x7efcc01527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 -- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcc015ab90 msgr2=0x7efcc01527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcc015ab90 0x7efcc01527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 -- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc01599c0 msgr2=0x7efcc0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc01599c0 0x7efcc0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 -- 172.21.15.170:0/210849470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb00094f0 con 0x7efcc0069aa0 2023-10-08T16:10:50.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc6360700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e190 tx=0x7efcbc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc00ecd0 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcbc00ee30 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc00b56f0 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcbc014670 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.562+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc00b5410 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.563+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc00cc820 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.563+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efcbc014a20 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.564+0000 7efcb67fc700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 0x7efca40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.564+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efcbc085380 con 0x7efcc0069aa0 2023-10-08T16:10:50.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.564+0000 7efcc5b5f700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 0x7efca40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.564+0000 7efcc5b5f700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 0x7efca40697d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efcc00c7770 tx=0x7efcb0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.568+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcbc052010 con 0x7efcc0069aa0 2023-10-08T16:10:50.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.751+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc0064550 con 0x7efcc0069aa0 2023-10-08T16:10:50.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.752+0000 7efcb67fc700 1 -- 172.21.15.170:0/210849470 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7efcbc009a60 con 0x7efcc0069aa0 2023-10-08T16:10:50.752 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:50.753 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:35.749539+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:50.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 msgr2=0x7efca40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 0x7efca40697d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efcc00c7770 tx=0x7efcb0017000 comp rx=0 tx=0).stop 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 msgr2=0x7efcc00b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00e190 tx=0x7efcbc009920 comp rx=0 tx=0).stop 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 shutdown_connections 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efcc015ab90 0x7efcc01527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efca4067320 0x7efca40697d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efcc0069aa0 0x7efcc00b4d60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 --2- 172.21.15.170:0/210849470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efcc01599c0 0x7efcc0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 >> 172.21.15.170:0/210849470 conn(0x7efcc0068e10 msgr2=0x7efcc00c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:50.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 shutdown_connections 2023-10-08T16:10:50.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.754+0000 7efcc85c4700 1 -- 172.21.15.170:0/210849470 wait complete. 2023-10-08T16:10:50.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:10:32.472119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:10:33.097320+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:10:50.767 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:10:50.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.884+0000 7fb37a99e700 1 Processor -- start 2023-10-08T16:10:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.884+0000 7fb37a99e700 1 -- start start 2023-10-08T16:10:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb37a99e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb374159890 0x7fb37415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb37a99e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 0x7fb3740699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb37a99e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb374069ee0 con 0x7fb374159c60 2023-10-08T16:10:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb37a99e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37415a160 con 0x7fb374159890 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb37a99e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3740d13a0 con 0x7fb374069560 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb373fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb374159890 0x7fb37415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb373fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb374159890 0x7fb37415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56344/0 (socket says 172.21.15.170:56344) 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb373fff700 1 -- 172.21.15.170:0/831294556 learned_addr learned my addr 172.21.15.170:0/831294556 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb378f3b700 1 --2- 172.21.15.170:0/831294556 >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 0x7fb3740699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.885+0000 7fb373fff700 1 -- 172.21.15.170:0/831294556 >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 msgr2=0x7fb3740699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb373fff700 1 --2- 172.21.15.170:0/831294556 >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 0x7fb3740699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb373fff700 1 -- 172.21.15.170:0/831294556 >> v1:172.21.15.191:6789/0 conn(0x7fb374159c60 legacy=0x7fb37415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb373fff700 1 -- 172.21.15.170:0/831294556 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37415e140 con 0x7fb374159890 2023-10-08T16:10:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb378f3b700 1 --2- 172.21.15.170:0/831294556 >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 0x7fb3740699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb373fff700 1 --2- 172.21.15.170:0/831294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb37415abd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb37406a890 tx=0x7fb364009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14336a5b2649a5ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb372ffd700 1 -- 172.21.15.170:0/831294556 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400b820 con 0x7fb374159890 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb372ffd700 1 -- 172.21.15.170:0/831294556 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36400b980 con 0x7fb374159890 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb372ffd700 1 -- 172.21.15.170:0/831294556 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36400bd20 con 0x7fb374159890 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 -- 172.21.15.170:0/831294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 msgr2=0x7fb37415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 --2- 172.21.15.170:0/831294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb37415abd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fb37406a890 tx=0x7fb364009250 comp rx=0 tx=0).stop 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 -- 172.21.15.170:0/831294556 shutdown_connections 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 --2- 172.21.15.170:0/831294556 >> v2:172.21.15.191:3300/0 conn(0x7fb374069560 0x7fb3740699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 --2- 172.21.15.170:0/831294556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb37415abd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 -- 172.21.15.170:0/831294556 >> 172.21.15.170:0/831294556 conn(0x7fb374068e20 msgr2=0x7fb3740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 -- 172.21.15.170:0/831294556 shutdown_connections 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.886+0000 7fb37a99e700 1 -- 172.21.15.170:0/831294556 wait complete. 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.887+0000 7fb37a99e700 1 Processor -- start 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 -- start start 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb374069560 0x7fb374152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb374159c60 0x7fb37414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37414d1f0 con 0x7fb374159890 2023-10-08T16:10:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3741531f0 con 0x7fb374069560 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb37a99e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37414d680 con 0x7fb374159c60 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb3737fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb3737fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56358/0 (socket says 172.21.15.170:56358) 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.888+0000 7fb3737fe700 1 -- 172.21.15.170:0/321954444 learned_addr learned my addr 172.21.15.170:0/321954444 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb373fff700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb374069560 0x7fb374152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb378f3b700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb374159c60 0x7fb37414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 -- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb374159c60 msgr2=0x7fb37414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb374159c60 0x7fb37414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 -- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb374069560 msgr2=0x7fb374152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb374069560 0x7fb374152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 -- 172.21.15.170:0/321954444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640094f0 con 0x7fb374159890 2023-10-08T16:10:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb3737fe700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb36800b610 tx=0x7fb368009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800f510 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36800f670 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800fa30 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37414c5e0 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.889+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37414cba0 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.890+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37404c3c0 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.890+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb36801a400 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.891+0000 7fb35bfff700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 0x7fb360069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.891+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb368084650 con 0x7fb374159890 2023-10-08T16:10:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.891+0000 7fb373fff700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 0x7fb360069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:50.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.891+0000 7fb373fff700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 0x7fb360069800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb37406a890 tx=0x7fb364016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:50.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:50.895+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb368051460 con 0x7fb374159890 2023-10-08T16:10:51.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.076+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb37414cf40 con 0x7fb374159890 2023-10-08T16:10:51.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.076+0000 7fb35bfff700 1 -- 172.21.15.170:0/321954444 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb368054a80 con 0x7fb374159890 2023-10-08T16:10:51.077 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.078+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 msgr2=0x7fb360069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.078+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 0x7fb360069800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb37406a890 tx=0x7fb364016000 comp rx=0 tx=0).stop 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.078+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 msgr2=0x7fb3741480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.078+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fb36800b610 tx=0x7fb368009120 comp rx=0 tx=0).stop 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 shutdown_connections 2023-10-08T16:10:51.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb374159c60 0x7fb37414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb374069560 0x7fb374152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb360067350 0x7fb360069800 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 --2- 172.21.15.170:0/321954444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb374159890 0x7fb3741480d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 >> 172.21.15.170:0/321954444 conn(0x7fb374068e20 msgr2=0x7fb374150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 shutdown_connections 2023-10-08T16:10:51.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.079+0000 7fb37a99e700 1 -- 172.21.15.170:0/321954444 wait complete. 2023-10-08T16:10:51.091 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:10:51.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.207+0000 7fb052565700 1 Processor -- start 2023-10-08T16:10:51.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.207+0000 7fb052565700 1 -- start start 2023-10-08T16:10:51.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb052565700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb052565700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb04c0cb230 0x7fb04c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb052565700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c0c9cb0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb052565700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c0cbbd0 con 0x7fb04c0cb230 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb052565700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c06dfc0 con 0x7fb04c159a60 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb051563700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb051563700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:54190/0 (socket says 172.21.15.170:54190) 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb051563700 1 -- 172.21.15.170:0/1731460154 learned_addr learned my addr 172.21.15.170:0/1731460154 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb050d62700 1 --1- 172.21.15.170:0/1731460154 >> v1:172.21.15.191:6789/0 conn(0x7fb04c0ccc00 0x7fb04c069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:52318/0 (socket says 172.21.15.170:52318) 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.208+0000 7fb051d64700 1 --2- 172.21.15.170:0/1731460154 >> v2:172.21.15.191:3300/0 conn(0x7fb04c0cb230 0x7fb04c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb051563700 1 -- 172.21.15.170:0/1731460154 >> v2:172.21.15.191:3300/0 conn(0x7fb04c0cb230 msgr2=0x7fb04c0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb051563700 1 --2- 172.21.15.170:0/1731460154 >> v2:172.21.15.191:3300/0 conn(0x7fb04c0cb230 0x7fb04c0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb051563700 1 -- 172.21.15.170:0/1731460154 >> v1:172.21.15.191:6789/0 conn(0x7fb04c0ccc00 legacy=0x7fb04c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb051563700 1 -- 172.21.15.170:0/1731460154 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0c9f30 con 0x7fb04c159a60 2023-10-08T16:10:51.210 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb051563700 1 --2- 172.21.15.170:0/1731460154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb048009f20 tx=0x7fb048009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=869cb1e69856b5c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb043fff700 1 -- 172.21.15.170:0/1731460154 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800e070 con 0x7fb04c159a60 2023-10-08T16:10:51.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb043fff700 1 -- 172.21.15.170:0/1731460154 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04800ba60 con 0x7fb04c159a60 2023-10-08T16:10:51.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.209+0000 7fb043fff700 1 -- 172.21.15.170:0/1731460154 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480092e0 con 0x7fb04c159a60 2023-10-08T16:10:51.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 -- 172.21.15.170:0/1731460154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 msgr2=0x7fb04c0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 --2- 172.21.15.170:0/1731460154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb048009f20 tx=0x7fb048009840 comp rx=0 tx=0).stop 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 -- 172.21.15.170:0/1731460154 shutdown_connections 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 --2- 172.21.15.170:0/1731460154 >> v2:172.21.15.191:3300/0 conn(0x7fb04c0cb230 0x7fb04c0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 --2- 172.21.15.170:0/1731460154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c0c9770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 -- 172.21.15.170:0/1731460154 >> 172.21.15.170:0/1731460154 conn(0x7fb04c06baa0 msgr2=0x7fb04c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 -- 172.21.15.170:0/1731460154 shutdown_connections 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.210+0000 7fb052565700 1 -- 172.21.15.170:0/1731460154 wait complete. 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.211+0000 7fb052565700 1 Processor -- start 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.211+0000 7fb052565700 1 -- start start 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb04c0cb230 0x7fb04c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c169fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c7160 con 0x7fb04c0ccc00 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c65e0 con 0x7fb04c0cb230 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb052565700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c68c0 con 0x7fb04c159a60 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56372/0 (socket says 172.21.15.170:56372) 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 -- 172.21.15.170:0/4289383665 learned_addr learned my addr 172.21.15.170:0/4289383665 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb051d64700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c169fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb051563700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb04c0cb230 0x7fb04c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 -- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 msgr2=0x7fb04c169fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c169fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 -- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb04c0cb230 msgr2=0x7fb04c0c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb04c0cb230 0x7fb04c0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 -- 172.21.15.170:0/4289383665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb048009ae0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.212+0000 7fb050d62700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c7c0 tx=0x7fb03c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.213+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c005a70 con 0x7fb04c0ccc00 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.213+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c005bd0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.213+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0c6ba0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.213+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0070f0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.213+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0c62e0 con 0x7fb04c0ccc00 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.214+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c0a9960 con 0x7fb04c0ccc00 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.214+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb03c007460 con 0x7fb04c0ccc00 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.214+0000 7fb0417fa700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 0x7fb0300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.215+0000 7fb051563700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 0x7fb0300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.215+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb03c084650 con 0x7fb04c0ccc00 2023-10-08T16:10:51.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.215+0000 7fb051563700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 0x7fb0300697d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb048009880 tx=0x7fb048018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.219+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb04c0a9960 con 0x7fb04c0ccc00 2023-10-08T16:10:51.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.399+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb04c0a9960 con 0x7fb04c0ccc00 2023-10-08T16:10:51.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.399+0000 7fb0417fa700 1 -- 172.21.15.170:0/4289383665 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb04c0a9960 con 0x7fb04c0ccc00 2023-10-08T16:10:51.400 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 msgr2=0x7fb0300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 0x7fb0300697d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb048009880 tx=0x7fb048018000 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 msgr2=0x7fb04c0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00c7c0 tx=0x7fb03c007f70 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 shutdown_connections 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb04c159a60 0x7fb04c169fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb04c0cb230 0x7fb04c0c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb030067320 0x7fb0300697d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 --2- 172.21.15.170:0/4289383665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb04c0ccc00 0x7fb04c0c5da0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 >> 172.21.15.170:0/4289383665 conn(0x7fb04c06baa0 msgr2=0x7fb04c0ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 shutdown_connections 2023-10-08T16:10:51.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.401+0000 7fb052565700 1 -- 172.21.15.170:0/4289383665 wait complete. 2023-10-08T16:10:51.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:10:51.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.531+0000 7f292a640700 1 Processor -- start 2023-10-08T16:10:51.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.531+0000 7f292a640700 1 -- start start 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f292a640700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2924159890 0x7f292415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f292a640700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f292415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f292a640700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292415a160 con 0x7f292415db10 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f292a640700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29240d13a0 con 0x7f2924159890 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f292a640700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292415e140 con 0x7f2924159c60 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2928bdd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f292415db10 0x7f2924069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49540/0 (socket says 172.21.15.170:49540) 2023-10-08T16:10:51.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2928bdd700 1 -- 172.21.15.170:0/3789188254 learned_addr learned my addr 172.21.15.170:0/3789188254 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f29237fe700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f292415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751949127 0 0) 0x7f292415a160 con 0x7f292415db10 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2923fff700 1 --2- 172.21.15.170:0/3789188254 >> v2:172.21.15.191:3300/0 conn(0x7f2924159890 0x7f292415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f290c003570 con 0x7f292415db10 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3901856434 0 0) 0x7f290c003570 con 0x7f292415db10 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 msgr2=0x7f292415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f292415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 >> v2:172.21.15.191:3300/0 conn(0x7f2924159890 msgr2=0x7f292415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 --2- 172.21.15.170:0/3789188254 >> v2:172.21.15.191:3300/0 conn(0x7f2924159890 0x7f292415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292415a3d0 con 0x7f292415db10 2023-10-08T16:10:51.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f291c0017f0 con 0x7f292415db10 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.532+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 >> v1:172.21.15.170:6789/0 conn(0x7f292415db10 legacy=0x7f2924069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c003a60 0x7f290c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f290c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c004d50 0x7f290c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240d13a0 con 0x7f290c004d50 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292415e140 con 0x7f290c003a60 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c014270 con 0x7f2924159c60 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c004d50 0x7f290c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2923fff700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c003a60 0x7f290c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f29237fe700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f290c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 -- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 msgr2=0x7f290c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2924159c60 0x7f290c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 -- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c003a60 msgr2=0x7f290c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f290c003a60 0x7f290c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 -- 172.21.15.170:0/3789188254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c003570 con 0x7f290c004d50 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2928bdd700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c004d50 0x7f290c0051c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f291c00a9a0 tx=0x7f291c002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1304c804f1bcb59b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c012440 con 0x7f290c004d50 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c0125c0 con 0x7f290c004d50 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.533+0000 7f2922ffd700 1 -- 172.21.15.170:0/3789188254 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c012980 con 0x7f290c004d50 2023-10-08T16:10:51.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 -- 172.21.15.170:0/3789188254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c004d50 msgr2=0x7f290c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 --2- 172.21.15.170:0/3789188254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f290c004d50 0x7f290c0051c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f291c00a9a0 tx=0x7f291c002210 comp rx=0 tx=0).stop 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f2928bdd700 1 -- 172.21.15.170:0/3789188254 reap_dead start 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 -- 172.21.15.170:0/3789188254 shutdown_connections 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 -- 172.21.15.170:0/3789188254 >> 172.21.15.170:0/3789188254 conn(0x7f2924068e20 msgr2=0x7f29240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 -- 172.21.15.170:0/3789188254 shutdown_connections 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.534+0000 7f292a640700 1 -- 172.21.15.170:0/3789188254 wait complete. 2023-10-08T16:10:51.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.535+0000 7f292a640700 1 Processor -- start 2023-10-08T16:10:51.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.535+0000 7f292a640700 1 -- start start 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29241442c0 0x7f29241520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f292415ba10 0x7f29240b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292415beb0 con 0x7f29240ca680 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29240b4940 con 0x7f29241442c0 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2924152a30 con 0x7f292415ba10 2023-10-08T16:10:51.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56394/0 (socket says 172.21.15.170:56394) 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 -- 172.21.15.170:0/2116753820 learned_addr learned my addr 172.21.15.170:0/2116753820 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f2928bdd700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f292415ba10 0x7f29240b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f2923fff700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29241442c0 0x7f29241520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 -- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f292415ba10 msgr2=0x7f29240b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f292415ba10 0x7f29240b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 -- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29241442c0 msgr2=0x7f29241520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29241442c0 0x7f29241520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 -- 172.21.15.170:0/2116753820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924152d10 con 0x7f29240ca680 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f29237fe700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2918011b40 tx=0x7f29180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800f940 con 0x7f29240ca680 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291800faa0 con 0x7f29240ca680 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800fe20 con 0x7f29240ca680 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2924152ff0 con 0x7f29240ca680 2023-10-08T16:10:51.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.536+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2924144690 con 0x7f29240ca680 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.537+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292404c3c0 con 0x7f29240ca680 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.538+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2918014650 con 0x7f29240ca680 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.538+0000 7f290bfff700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 0x7f29100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.538+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2918018030 con 0x7f29240ca680 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.538+0000 7f2923fff700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 0x7f29100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.539+0000 7f2923fff700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 0x7f29100697d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2914006f00 tx=0x7f29140091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.542+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2918052660 con 0x7f29240ca680 2023-10-08T16:10:51.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.730+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f29240b4a80 con 0x7f29240ca680 2023-10-08T16:10:51.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.730+0000 7f290bfff700 1 -- 172.21.15.170:0/2116753820 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29180220a0 con 0x7f29240ca680 2023-10-08T16:10:51.731 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:10:51.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 msgr2=0x7f29100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 0x7f29100697d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2914006f00 tx=0x7f29140091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 msgr2=0x7f292415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f2918011b40 tx=0x7f29180091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 shutdown_connections 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f292415ba10 0x7f29240b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29241442c0 0x7f29241520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2910067320 0x7f29100697d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 --2- 172.21.15.170:0/2116753820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29240ca680 0x7f292415b4d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 >> 172.21.15.170:0/2116753820 conn(0x7f2924068e20 msgr2=0x7f292415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 shutdown_connections 2023-10-08T16:10:51.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.733+0000 7f292a640700 1 -- 172.21.15.170:0/2116753820 wait complete. 2023-10-08T16:10:51.745 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:51.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.859+0000 7f6b200ca700 1 Processor -- start 2023-10-08T16:10:51.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.859+0000 7f6b200ca700 1 -- start start 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b200ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 0x7f6b18140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b200ca700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1815c2a0 con 0x7f6b1806c0a0 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b200ca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1815b690 con 0x7f6b180c9510 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b200ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1815b970 con 0x7f6b1806bcd0 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b1d665700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6b1806c0a0 0x7f6b180c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49548/0 (socket says 172.21.15.170:49548) 2023-10-08T16:10:51.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b1d665700 1 -- 172.21.15.170:0/4086614399 learned_addr learned my addr 172.21.15.170:0/4086614399 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.860+0000 7f6b1de66700 1 --2- 172.21.15.170:0/4086614399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 0x7f6b18140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1ce64700 1 -- 172.21.15.170:0/4086614399 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554244207 0 0) 0x7f6b1815c2a0 con 0x7f6b1806c0a0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1ce64700 1 -- 172.21.15.170:0/4086614399 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b04003570 con 0x7f6b1806c0a0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1de66700 1 -- 172.21.15.170:0/4086614399 >> v1:172.21.15.191:6789/0 conn(0x7f6b180c9510 legacy=0x7f6b180cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1de66700 1 -- 172.21.15.170:0/4086614399 >> v1:172.21.15.170:6789/0 conn(0x7f6b1806c0a0 legacy=0x7f6b180c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1de66700 1 -- 172.21.15.170:0/4086614399 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1815bc50 con 0x7f6b1806bcd0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1de66700 1 --2- 172.21.15.170:0/4086614399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 0x7f6b18140df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009910 tx=0x7f6b140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa90f7ee9cf497f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1ce64700 1 -- 172.21.15.170:0/4086614399 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400b680 con 0x7f6b1806bcd0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1ce64700 1 -- 172.21.15.170:0/4086614399 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400b7e0 con 0x7f6b1806bcd0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.861+0000 7f6b1ce64700 1 -- 172.21.15.170:0/4086614399 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400bba0 con 0x7f6b1806bcd0 2023-10-08T16:10:51.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 -- 172.21.15.170:0/4086614399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 msgr2=0x7f6b18140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 --2- 172.21.15.170:0/4086614399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 0x7f6b18140df0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009910 tx=0x7f6b140091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 -- 172.21.15.170:0/4086614399 shutdown_connections 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 --2- 172.21.15.170:0/4086614399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1806bcd0 0x7f6b18140df0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 -- 172.21.15.170:0/4086614399 >> 172.21.15.170:0/4086614399 conn(0x7f6b180ce600 msgr2=0x7f6b180cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 -- 172.21.15.170:0/4086614399 shutdown_connections 2023-10-08T16:10:51.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.862+0000 7f6b200ca700 1 -- 172.21.15.170:0/4086614399 wait complete. 2023-10-08T16:10:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.863+0000 7f6b200ca700 1 Processor -- start 2023-10-08T16:10:51.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.863+0000 7f6b200ca700 1 -- start start 2023-10-08T16:10:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 0x7f6b1813f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180c9510 0x7f6b180b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1813fd00 con 0x7f6b1806bcd0 2023-10-08T16:10:51.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1816c330 con 0x7f6b1806c0a0 2023-10-08T16:10:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b18168030 con 0x7f6b180c9510 2023-10-08T16:10:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56398/0 (socket says 172.21.15.170:56398) 2023-10-08T16:10:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 -- 172.21.15.170:0/2417466415 learned_addr learned my addr 172.21.15.170:0/2417466415 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:51.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1e667700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180c9510 0x7f6b180b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1d665700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 0x7f6b1813f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 -- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180c9510 msgr2=0x7f6b180b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180c9510 0x7f6b180b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 -- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 msgr2=0x7f6b1813f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 0x7f6b1813f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 -- 172.21.15.170:0/2417466415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b14009420 con 0x7f6b1806bcd0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1d665700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 0x7f6b1813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b1de66700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009fb0 tx=0x7f6b14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400bb60 con 0x7f6b1806bcd0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b14000c90 con 0x7f6b1806bcd0 2023-10-08T16:10:51.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1816ba90 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.864+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1401a660 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.865+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1816c050 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.865+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b18167e00 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.866+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6b1401a9d0 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.866+0000 7f6b0dffb700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 0x7f6afc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.866+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6b140869c0 con 0x7f6b1806bcd0 2023-10-08T16:10:51.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.866+0000 7f6b1d665700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 0x7f6afc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:51.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.867+0000 7f6b1d665700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 0x7f6afc0697d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6b180cbb60 tx=0x7f6b08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:51.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:51.871+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b18167e00 con 0x7f6b1806bcd0 2023-10-08T16:10:52.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.099+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b18167e00 con 0x7f6b1806bcd0 2023-10-08T16:10:52.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.099+0000 7f6b0dffb700 1 -- 172.21.15.170:0/2417466415 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f6b18167e00 con 0x7f6b1806bcd0 2023-10-08T16:10:52.100 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:52.100 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:52.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 msgr2=0x7f6afc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 0x7f6afc0697d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6b180cbb60 tx=0x7f6b08009040 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 msgr2=0x7f6b1816b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009fb0 tx=0x7f6b14017000 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 shutdown_connections 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b180c9510 0x7f6b180b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6afc067320 0x7f6afc0697d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b1806bcd0 0x7f6b1816b400 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 --2- 172.21.15.170:0/2417466415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b1806c0a0 0x7f6b1813f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 >> 172.21.15.170:0/2417466415 conn(0x7f6b180ce600 msgr2=0x7f6b18158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 shutdown_connections 2023-10-08T16:10:52.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.102+0000 7f6b200ca700 1 -- 172.21.15.170:0/2417466415 wait complete. 2023-10-08T16:10:52.104 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 66 2023-10-08T16:10:52.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:52.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.228+0000 7f7357fff700 1 Processor -- start 2023-10-08T16:10:52.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.228+0000 7f7357fff700 1 -- start start 2023-10-08T16:10:52.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7357fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f73580ad870 0x7f735806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7357fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 0x7f735806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7357fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73580b40f0 con 0x7f7358142ea0 2023-10-08T16:10:52.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7357fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735806c3a0 con 0x7f73580ad870 2023-10-08T16:10:52.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7357fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7358069640 con 0x7f735806bf20 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f734e7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f73580ad870 0x7f735806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7356ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7358142ea0 0x7f73580b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49556/0 (socket says 172.21.15.170:49556) 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f7356ffd700 1 -- 172.21.15.170:0/1656933046 learned_addr learned my addr 172.21.15.170:0/1656933046 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f73577fe700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 0x7f735806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f734e7fc700 1 -- 172.21.15.170:0/1656933046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 msgr2=0x7f735806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f734e7fc700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 0x7f735806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f734e7fc700 1 -- 172.21.15.170:0/1656933046 >> v1:172.21.15.170:6789/0 conn(0x7f7358142ea0 legacy=0x7f73580b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f734e7fc700 1 -- 172.21.15.170:0/1656933046 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7358069920 con 0x7f73580ad870 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.229+0000 7f73567fc700 1 -- 172.21.15.170:0/1656933046 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586383467 0 0) 0x7f73580b40f0 con 0x7f7358142ea0 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f73577fe700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 0x7f735806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:52.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f734e7fc700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73580ad870 0x7f735806b9e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f73400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b9006db4b5e2d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f73567fc700 1 -- 172.21.15.170:0/1656933046 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340009e80 con 0x7f73580ad870 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f73567fc700 1 -- 172.21.15.170:0/1656933046 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000b860 con 0x7f73580ad870 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f73567fc700 1 -- 172.21.15.170:0/1656933046 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000bbe0 con 0x7f73580ad870 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 -- 172.21.15.170:0/1656933046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73580ad870 msgr2=0x7f735806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73580ad870 0x7f735806b9e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f73400091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 -- 172.21.15.170:0/1656933046 shutdown_connections 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f735806bf20 0x7f735806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 --2- 172.21.15.170:0/1656933046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73580ad870 0x7f735806b9e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 -- 172.21.15.170:0/1656933046 >> 172.21.15.170:0/1656933046 conn(0x7f73580ce430 msgr2=0x7f73580ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 -- 172.21.15.170:0/1656933046 shutdown_connections 2023-10-08T16:10:52.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.230+0000 7f7357fff700 1 -- 172.21.15.170:0/1656933046 wait complete. 2023-10-08T16:10:52.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 Processor -- start 2023-10-08T16:10:52.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 -- start start 2023-10-08T16:10:52.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7358142ea0 0x7f735816d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7358159620 0x7f735815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735815e860 con 0x7f735806bf20 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735816afd0 con 0x7f7358159620 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7357fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735815d370 con 0x7f7358142ea0 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7356ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56416/0 (socket says 172.21.15.170:56416) 2023-10-08T16:10:52.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.232+0000 7f7356ffd700 1 -- 172.21.15.170:0/3068359490 learned_addr learned my addr 172.21.15.170:0/3068359490 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:52.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f734e7fc700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7358142ea0 0x7f735816d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f73577fe700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7358159620 0x7f735815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 -- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7358142ea0 msgr2=0x7f735816d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7358142ea0 0x7f735816d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 -- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7358159620 msgr2=0x7f735815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7358159620 0x7f735815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 -- 172.21.15.170:0/3068359490 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340009420 con 0x7f735806bf20 2023-10-08T16:10:52.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7356ffd700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f734800c5b0 tx=0x7f7348001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348005b30 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7348005c90 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735815d650 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348007300 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.233+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735815d930 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.234+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735804c3c0 con 0x7f735806bf20 2023-10-08T16:10:52.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.234+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f734801b3b0 con 0x7f735806bf20 2023-10-08T16:10:52.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.235+0000 7f734f7fe700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 0x7f73380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.235+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7348084160 con 0x7f735806bf20 2023-10-08T16:10:52.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.235+0000 7f734e7fc700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 0x7f73380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.235+0000 7f734e7fc700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 0x7f73380697d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f7340017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.239+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7348050db0 con 0x7f735806bf20 2023-10-08T16:10:52.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.421+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7358069c00 con 0x7f735806bf20 2023-10-08T16:10:52.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.422+0000 7f734f7fe700 1 -- 172.21.15.170:0/3068359490 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7f73480543d0 con 0x7f735806bf20 2023-10-08T16:10:52.423 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:52.423 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:35.749539+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 msgr2=0x7f73380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 0x7f73380697d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f7340017000 comp rx=0 tx=0).stop 2023-10-08T16:10:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 msgr2=0x7f735816d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f734800c5b0 tx=0x7f7348001070 comp rx=0 tx=0).stop 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 shutdown_connections 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7358159620 0x7f735815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7338067320 0x7f73380697d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f735806bf20 0x7f735816d360 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 --2- 172.21.15.170:0/3068359490 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7358142ea0 0x7f735816d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 >> 172.21.15.170:0/3068359490 conn(0x7f73580ce430 msgr2=0x7f73580ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 shutdown_connections 2023-10-08T16:10:52.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.424+0000 7f7357fff700 1 -- 172.21.15.170:0/3068359490 wait complete. 2023-10-08T16:10:52.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:10:52.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.554+0000 7efeda6b0700 1 Processor -- start 2023-10-08T16:10:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.554+0000 7efeda6b0700 1 -- start start 2023-10-08T16:10:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.555+0000 7efeda6b0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 0x7efed40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.555+0000 7efeda6b0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efed415d230 0x7efed415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.555+0000 7efeda6b0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed415dbd0 con 0x7efed4159420 2023-10-08T16:10:52.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.555+0000 7efeda6b0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed40b11d0 con 0x7efed415d230 2023-10-08T16:10:52.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.555+0000 7efeda6b0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed40b14b0 con 0x7efed41597f0 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed8ead700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 0x7efed40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7efed415d230 0x7efed415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed96ae700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7efed4159420 0x7efed40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:52342/0 (socket says 172.21.15.170:52342) 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed96ae700 1 -- 172.21.15.170:0/1417035275 learned_addr learned my addr 172.21.15.170:0/1417035275 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 -- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 msgr2=0x7efed40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:52.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 0x7efed40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 -- 172.21.15.170:0/1417035275 >> v1:172.21.15.191:6789/0 conn(0x7efed4159420 legacy=0x7efed40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 -- 172.21.15.170:0/1417035275 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40b1790 con 0x7efed415d230 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed8ead700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 0x7efed40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efecbfff700 1 -- 172.21.15.170:0/1417035275 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067741651 0 0) 0x7efed0001110 con 0x7efed4159420 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efed9eaf700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed415d230 0x7efed415d690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efecc009e60 tx=0x7efecc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae90f1d121a699aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efecbfff700 1 -- 172.21.15.170:0/1417035275 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc00e070 con 0x7efed415d230 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efecbfff700 1 -- 172.21.15.170:0/1417035275 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc00bb60 con 0x7efed415d230 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.556+0000 7efecbfff700 1 -- 172.21.15.170:0/1417035275 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc012430 con 0x7efed415d230 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 -- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed415d230 msgr2=0x7efed415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed415d230 0x7efed415d690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efecc009e60 tx=0x7efecc009780 comp rx=0 tx=0).stop 2023-10-08T16:10:52.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 -- 172.21.15.170:0/1417035275 shutdown_connections 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed415d230 0x7efed415d690 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 --2- 172.21.15.170:0/1417035275 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed41597f0 0x7efed40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 -- 172.21.15.170:0/1417035275 >> 172.21.15.170:0/1417035275 conn(0x7efed40d0660 msgr2=0x7efed40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 -- 172.21.15.170:0/1417035275 shutdown_connections 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.557+0000 7efeda6b0700 1 -- 172.21.15.170:0/1417035275 wait complete. 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.558+0000 7efeda6b0700 1 Processor -- start 2023-10-08T16:10:52.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.558+0000 7efeda6b0700 1 -- start start 2023-10-08T16:10:52.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 0x7efed40ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 0x7efed4169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed416a4f0 con 0x7efed415d230 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed416b070 con 0x7efed41597f0 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efeda6b0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed416a2f0 con 0x7efed4159420 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56422/0 (socket says 172.21.15.170:56422) 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 -- 172.21.15.170:0/2564144181 learned_addr learned my addr 172.21.15.170:0/2564144181 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed96ae700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 0x7efed40ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed8ead700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 0x7efed4169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 -- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 msgr2=0x7efed40ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 0x7efed40ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 -- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 msgr2=0x7efed4169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 0x7efed4169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed9eaf700 1 -- 172.21.15.170:0/2564144181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009420 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed8ead700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 0x7efed4169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.559+0000 7efed96ae700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 0x7efed40ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efed9eaf700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efecc012ad0 tx=0x7efecc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc012e10 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc01e630 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc009a20 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc01e9f0 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.560+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed416aa20 con 0x7efed415d230 2023-10-08T16:10:52.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.561+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed404c3c0 con 0x7efed415d230 2023-10-08T16:10:52.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.561+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efecc024070 con 0x7efed415d230 2023-10-08T16:10:52.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.561+0000 7efec97fa700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 0x7efeb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.562+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efecc087400 con 0x7efed415d230 2023-10-08T16:10:52.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.562+0000 7efed96ae700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 0x7efeb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.562+0000 7efed96ae700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 0x7efeb80697d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efed0001940 tx=0x7efed0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.566+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efecc054090 con 0x7efed415d230 2023-10-08T16:10:52.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.794+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed416b1b0 con 0x7efed415d230 2023-10-08T16:10:52.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.794+0000 7efec97fa700 1 -- 172.21.15.170:0/2564144181 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7efecc0576b0 con 0x7efed415d230 2023-10-08T16:10:52.795 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:52.795 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 msgr2=0x7efeb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 0x7efeb80697d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efed0001940 tx=0x7efed0009040 comp rx=0 tx=0).stop 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 msgr2=0x7efed4155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efecc012ad0 tx=0x7efecc019000 comp rx=0 tx=0).stop 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 shutdown_connections 2023-10-08T16:10:52.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efeb8067320 0x7efeb80697d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efed415d230 0x7efed4155140 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efed4159420 0x7efed40ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 --2- 172.21.15.170:0/2564144181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efed41597f0 0x7efed4169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 >> 172.21.15.170:0/2564144181 conn(0x7efed40d0660 msgr2=0x7efed4157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 shutdown_connections 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.796+0000 7efeda6b0700 1 -- 172.21.15.170:0/2564144181 wait complete. 2023-10-08T16:10:52.798 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 66 2023-10-08T16:10:52.809 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:52.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.922+0000 7f7de0d35700 1 Processor -- start 2023-10-08T16:10:52.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.922+0000 7f7de0d35700 1 -- start start 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7de0d35700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 0x7f7ddc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7de0d35700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ddc1597f0 0x7f7ddc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7de0d35700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0b11d0 con 0x7f7ddc15d230 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7de0d35700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b14b0 con 0x7f7ddc1597f0 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7de0d35700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b1790 con 0x7f7ddc159420 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ddc1597f0 0x7f7ddc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddbfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7ddc15d230 0x7f7ddc15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49560/0 (socket says 172.21.15.170:49560) 2023-10-08T16:10:52.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddbfff700 1 -- 172.21.15.170:0/1180146432 learned_addr learned my addr 172.21.15.170:0/1180146432 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 --2- 172.21.15.170:0/1180146432 >> v2:172.21.15.170:3300/0 conn(0x7f7ddc1597f0 0x7f7ddc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56428/0 (socket says 172.21.15.170:56428) 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 -- 172.21.15.170:0/1180146432 >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 msgr2=0x7f7ddc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddb7fe700 1 --2- 172.21.15.170:0/1180146432 >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 0x7f7ddc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7dda7fc700 1 -- 172.21.15.170:0/1180146432 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293570660 0 0) 0x7f7ddc0b11d0 con 0x7f7ddc15d230 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 --2- 172.21.15.170:0/1180146432 >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 0x7f7ddc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 -- 172.21.15.170:0/1180146432 >> v1:172.21.15.170:6789/0 conn(0x7f7ddc15d230 legacy=0x7f7ddc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddaffd700 1 -- 172.21.15.170:0/1180146432 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0b1a70 con 0x7f7ddc1597f0 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.923+0000 7f7ddb7fe700 1 --2- 172.21.15.170:0/1180146432 >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 0x7f7ddc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7ddaffd700 1 --2- 172.21.15.170:0/1180146432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc0b2270 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0b2f20 tx=0x7f7dd00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0cda26fb5c6404 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7dda7fc700 1 -- 172.21.15.170:0/1180146432 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000b6b0 con 0x7f7ddc1597f0 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7dda7fc700 1 -- 172.21.15.170:0/1180146432 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000b810 con 0x7f7ddc1597f0 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7dda7fc700 1 -- 172.21.15.170:0/1180146432 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000bbb0 con 0x7f7ddc1597f0 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 -- 172.21.15.170:0/1180146432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc1597f0 msgr2=0x7f7ddc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 --2- 172.21.15.170:0/1180146432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc0b2270 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0b2f20 tx=0x7f7dd00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 -- 172.21.15.170:0/1180146432 shutdown_connections 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 --2- 172.21.15.170:0/1180146432 >> v2:172.21.15.191:3300/0 conn(0x7f7ddc159420 0x7f7ddc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 --2- 172.21.15.170:0/1180146432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc0b2270 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 -- 172.21.15.170:0/1180146432 >> 172.21.15.170:0/1180146432 conn(0x7f7ddc0d0660 msgr2=0x7f7ddc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 -- 172.21.15.170:0/1180146432 shutdown_connections 2023-10-08T16:10:52.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.924+0000 7f7de0d35700 1 -- 172.21.15.170:0/1180146432 wait complete. 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 Processor -- start 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 -- start start 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ddc159420 0x7f7ddc0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc15b790 con 0x7f7ddc15d230 2023-10-08T16:10:52.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b56e0 con 0x7f7ddc1597f0 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7de0d35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b59c0 con 0x7f7ddc159420 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7ddbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7ddbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56440/0 (socket says 172.21.15.170:56440) 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.926+0000 7f7ddbfff700 1 -- 172.21.15.170:0/3124689520 learned_addr learned my addr 172.21.15.170:0/3124689520 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddaffd700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddb7fe700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ddc159420 0x7f7ddc0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 -- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ddc159420 msgr2=0x7f7ddc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ddc159420 0x7f7ddc0cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 -- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 msgr2=0x7f7ddc15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:52.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 -- 172.21.15.170:0/3124689520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009420 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddaffd700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7ddbfff700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c500 tx=0x7f7dd4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4005b30 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4005c90 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0b5ca0 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4007320 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.927+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc0b6260 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.928+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc169810 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.929+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7dd401b3b0 con 0x7f7ddc15d230 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.929+0000 7f7dc37fe700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 0x7f7dc80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:52.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.929+0000 7f7ddb7fe700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 0x7f7dc80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:52.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.929+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7dd4084140 con 0x7f7ddc15d230 2023-10-08T16:10:52.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.929+0000 7f7ddb7fe700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 0x7f7dc80697d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e650 tx=0x7f7dcc007710 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:52.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:52.933+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ddc169810 con 0x7f7ddc15d230 2023-10-08T16:10:53.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.156+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ddc169810 con 0x7f7ddc15d230 2023-10-08T16:10:53.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.156+0000 7f7dc37fe700 1 -- 172.21.15.170:0/3124689520 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4611 (secure 0 0 0) 0x7f7ddc169810 con 0x7f7ddc15d230 2023-10-08T16:10:53.157 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:53.157 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":66,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:10:34.136500+0000","modified":"2023-10-08T16:10:37.784482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6771":{"gid":6771,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1404814921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1404814921},{"type":"v1","addr":"172.21.15.191:6835","nonce":1404814921}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T16:10:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 msgr2=0x7f7dc80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 0x7f7dc80697d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00e650 tx=0x7f7dcc007710 comp rx=0 tx=0).stop 2023-10-08T16:10:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 msgr2=0x7f7ddc15b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c500 tx=0x7f7dd4007f60 comp rx=0 tx=0).stop 2023-10-08T16:10:53.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 shutdown_connections 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.158+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7dc8067320 0x7f7dc80697d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ddc15d230 0x7f7ddc15b250 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ddc159420 0x7f7ddc0cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 --2- 172.21.15.170:0/3124689520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ddc1597f0 0x7f7ddc15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 >> 172.21.15.170:0/3124689520 conn(0x7f7ddc0d0660 msgr2=0x7f7ddc0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 shutdown_connections 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.159+0000 7f7de0d35700 1 -- 172.21.15.170:0/3124689520 wait complete. 2023-10-08T16:10:53.160 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 66 2023-10-08T16:10:53.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:10:53.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.285+0000 7f15f359e700 1 Processor -- start 2023-10-08T16:10:53.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.285+0000 7f15f359e700 1 -- start start 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f15f4159420 0x7f15f40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f359e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f15f41597f0 0x7f15f40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f359e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f40b11d0 con 0x7f15f415d230 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f359e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40b14b0 con 0x7f15f41597f0 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40b1790 con 0x7f15f4159420 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.286+0000 7f15f1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f15f41597f0 0x7f15f40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f15f41597f0 0x7f15f40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56452/0 (socket says 172.21.15.170:56452) 2023-10-08T16:10:53.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 -- 172.21.15.170:0/1400418975 learned_addr learned my addr 172.21.15.170:0/1400418975 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f259c700 1 --2- 172.21.15.170:0/1400418975 >> v2:172.21.15.191:3300/0 conn(0x7f15f4159420 0x7f15f40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 -- 172.21.15.170:0/1400418975 >> v2:172.21.15.191:3300/0 conn(0x7f15f4159420 msgr2=0x7f15f40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 --2- 172.21.15.170:0/1400418975 >> v2:172.21.15.191:3300/0 conn(0x7f15f4159420 0x7f15f40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 -- 172.21.15.170:0/1400418975 >> v1:172.21.15.191:6789/0 conn(0x7f15f415d230 legacy=0x7f15f415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 -- 172.21.15.170:0/1400418975 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f40b1a70 con 0x7f15f41597f0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f1d9b700 1 --2- 172.21.15.170:0/1400418975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f40b2270 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15f40b2f20 tx=0x7f15dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bae5c15798c453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f159a700 1 -- 172.21.15.170:0/1400418975 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc009e70 con 0x7f15f41597f0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f159a700 1 -- 172.21.15.170:0/1400418975 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc00b860 con 0x7f15f41597f0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.287+0000 7f15f159a700 1 -- 172.21.15.170:0/1400418975 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00bc00 con 0x7f15f41597f0 2023-10-08T16:10:53.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 -- 172.21.15.170:0/1400418975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 msgr2=0x7f15f40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 --2- 172.21.15.170:0/1400418975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f40b2270 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15f40b2f20 tx=0x7f15dc009250 comp rx=0 tx=0).stop 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 -- 172.21.15.170:0/1400418975 shutdown_connections 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 --2- 172.21.15.170:0/1400418975 >> v2:172.21.15.191:3300/0 conn(0x7f15f4159420 0x7f15f40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 --2- 172.21.15.170:0/1400418975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f40b2270 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 -- 172.21.15.170:0/1400418975 >> 172.21.15.170:0/1400418975 conn(0x7f15f40d0660 msgr2=0x7f15f40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 -- 172.21.15.170:0/1400418975 shutdown_connections 2023-10-08T16:10:53.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.288+0000 7f15f359e700 1 -- 172.21.15.170:0/1400418975 wait complete. 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.289+0000 7f15f359e700 1 Processor -- start 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.289+0000 7f15f359e700 1 -- start start 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f4159420 0x7f15f414e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f415d230 0x7f15f415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40b4a60 con 0x7f15f41597f0 2023-10-08T16:10:53.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f414e9d0 con 0x7f15f415d230 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415ce00 con 0x7f15f4159420 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56454/0 (socket says 172.21.15.170:56454) 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 -- 172.21.15.170:0/2582691183 learned_addr learned my addr 172.21.15.170:0/2582691183 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f259c700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f4159420 0x7f15f414e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f2d9d700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f415d230 0x7f15f415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 -- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f4159420 msgr2=0x7f15f414e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f4159420 0x7f15f414e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 -- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f415d230 msgr2=0x7f15f415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f415d230 0x7f15f415c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 -- 172.21.15.170:0/2582691183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0094f0 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f1d9b700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f15f4089340 tx=0x7f15dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00bbe0 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc011420 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f40b4cb0 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0117c0 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.290+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f40b5270 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.291+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f404c3c0 con 0x7f15f41597f0 2023-10-08T16:10:53.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.292+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f15dc011bf0 con 0x7f15f41597f0 2023-10-08T16:10:53.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.292+0000 7f15ea7fc700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 0x7f15d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.292+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15dc086140 con 0x7f15f41597f0 2023-10-08T16:10:53.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.292+0000 7f15f259c700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 0x7f15d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.292+0000 7f15f259c700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 0x7f15d40697d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f15e400f570 tx=0x7f15e400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.296+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15dc052dd0 con 0x7f15f41597f0 2023-10-08T16:10:53.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.479+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15f40af710 con 0x7f15f41597f0 2023-10-08T16:10:53.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.480+0000 7f15ea7fc700 1 -- 172.21.15.170:0/2582691183 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+15404 (secure 0 0 0) 0x7f15dc0563f0 con 0x7f15f41597f0 2023-10-08T16:10:53.481 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:53.481 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:10:35.749539+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:10:32.472119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T16:10:33.097320+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 msgr2=0x7f15d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 0x7f15d40697d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f15e400f570 tx=0x7f15e400b040 comp rx=0 tx=0).stop 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 msgr2=0x7f15f4068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f15f4089340 tx=0x7f15dc017000 comp rx=0 tx=0).stop 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 shutdown_connections 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f15f415d230 0x7f15f415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f15f4159420 0x7f15f414e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f15d4067320 0x7f15d40697d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 --2- 172.21.15.170:0/2582691183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15f41597f0 0x7f15f4068390 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 >> 172.21.15.170:0/2582691183 conn(0x7f15f40d0660 msgr2=0x7f15f4156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 shutdown_connections 2023-10-08T16:10:53.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.482+0000 7f15f359e700 1 -- 172.21.15.170:0/2582691183 wait complete. 2023-10-08T16:10:53.495 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:10:53.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.611+0000 7f8392366700 1 Processor -- start 2023-10-08T16:10:53.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.611+0000 7f8392366700 1 -- start start 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.612+0000 7f8392366700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.612+0000 7f8392366700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f838c0ccf90 0x7f838c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.612+0000 7f8392366700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0cc300 con 0x7f838c0cbf30 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.612+0000 7f8392366700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c156bf0 con 0x7f838c0ccf90 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.612+0000 7f8392366700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0ae950 con 0x7f838c0b1420 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f838c0ccf90 0x7f838c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:54316/0 (socket says 172.21.15.170:54316) 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838bfff700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f838c0cbf30 0x7f838c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:52358/0 (socket says 172.21.15.170:52358) 2023-10-08T16:10:53.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838b7fe700 1 -- 172.21.15.170:0/1172304607 learned_addr learned my addr 172.21.15.170:0/1172304607 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 -- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 msgr2=0x7f838c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 --2- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 -- 172.21.15.170:0/1172304607 >> v1:172.21.15.191:6789/0 conn(0x7f838c0cbf30 legacy=0x7f838c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838affd700 1 -- 172.21.15.170:0/1172304607 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664422395 0 0) 0x7f838c0cc300 con 0x7f838c0cbf30 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 -- 172.21.15.170:0/1172304607 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0ad370 con 0x7f838c0ccf90 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f8390903700 1 --2- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0ccf90 0x7f838c0c9740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8384009e90 tx=0x7f8384009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84226d614fee941e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838affd700 1 -- 172.21.15.170:0/1172304607 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400e070 con 0x7f838c0ccf90 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838affd700 1 -- 172.21.15.170:0/1172304607 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838400bbf0 con 0x7f838c0ccf90 2023-10-08T16:10:53.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.613+0000 7f838affd700 1 -- 172.21.15.170:0/1172304607 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384012430 con 0x7f838c0ccf90 2023-10-08T16:10:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 -- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0ccf90 msgr2=0x7f838c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 --2- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0ccf90 0x7f838c0c9740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8384009e90 tx=0x7f8384009780 comp rx=0 tx=0).stop 2023-10-08T16:10:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 -- 172.21.15.170:0/1172304607 shutdown_connections 2023-10-08T16:10:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 --2- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0ccf90 0x7f838c0c9740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 --2- 172.21.15.170:0/1172304607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 -- 172.21.15.170:0/1172304607 >> 172.21.15.170:0/1172304607 conn(0x7f838c06a320 msgr2=0x7f838c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 -- 172.21.15.170:0/1172304607 shutdown_connections 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.614+0000 7f8392366700 1 -- 172.21.15.170:0/1172304607 wait complete. 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.615+0000 7f8392366700 1 Processor -- start 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.615+0000 7f8392366700 1 -- start start 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c16b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cbf30 0x7f838c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c16bca0 con 0x7f838c0ccf90 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c16ad10 con 0x7f838c0cbf30 2023-10-08T16:10:53.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8392366700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c16b2d0 con 0x7f838c0b1420 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56456/0 (socket says 172.21.15.170:56456) 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 -- 172.21.15.170:0/3660347861 learned_addr learned my addr 172.21.15.170:0/3660347861 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f838bfff700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c16b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f838b7fe700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cbf30 0x7f838c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 -- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 msgr2=0x7f838c16b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c16b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 -- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cbf30 msgr2=0x7f838c148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cbf30 0x7f838c148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f8390903700 1 -- 172.21.15.170:0/3660347861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384009a20 con 0x7f838c0ccf90 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.616+0000 7f838bfff700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c16b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:53.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8390903700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f838400b6c0 tx=0x7f8384019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838401c610 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838401c790 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c0a98c0 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838401cb50 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.617+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c16af60 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.618+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c04c3c0 con 0x7f838c0ccf90 2023-10-08T16:10:53.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.618+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f838401b400 con 0x7f838c0ccf90 2023-10-08T16:10:53.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.618+0000 7f8373fff700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 0x7f83780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.619+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8384086f60 con 0x7f838c0ccf90 2023-10-08T16:10:53.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.619+0000 7f838bfff700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 0x7f83780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.619+0000 7f838bfff700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 0x7f83780697d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f837c0018d0 tx=0x7f837c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.623+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8384053ae0 con 0x7f838c0ccf90 2023-10-08T16:10:53.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.851+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f838c16b410 con 0x7f838c0ccf90 2023-10-08T16:10:53.851 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:10:53.851+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:10:53.855 INFO:tasks.ceph.mds.d.smithi191.stderr: -2095> 2023-10-08T16:10:29.648+0000 7f33f0238700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T16:10:53.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.855+0000 7f8373fff700 1 -- 172.21.15.170:0/3660347861 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v67) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8384057100 con 0x7f838c0ccf90 2023-10-08T16:10:53.857 INFO:tasks.ceph.mds.d.smithi191.stderr: -924> 2023-10-08T16:10:40.055+0000 7f33ea22c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:10:53.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 msgr2=0x7f83780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 0x7f83780697d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f837c0018d0 tx=0x7f837c009040 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 msgr2=0x7f838c0a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f838400b6c0 tx=0x7f8384019000 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 shutdown_connections 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8378067320 0x7f83780697d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f838c0ccf90 0x7f838c0a8e70 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f838c0b1420 0x7f838c16b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 --2- 172.21.15.170:0/3660347861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f838c0cbf30 0x7f838c148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 >> 172.21.15.170:0/3660347861 conn(0x7f838c06a320 msgr2=0x7f838c0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 shutdown_connections 2023-10-08T16:10:53.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.858+0000 7f8392366700 1 -- 172.21.15.170:0/3660347861 wait complete. 2023-10-08T16:10:53.860 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:10:53.870 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:10:53.879 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:10:53.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.986+0000 7f051972c700 1 Processor -- start 2023-10-08T16:10:53.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.986+0000 7f051972c700 1 -- start start 2023-10-08T16:10:53.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f051972c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f051415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f051972c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0514069560 0x7f05140699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f051972c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f051415a160 con 0x7f0514159c60 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f051972c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0514069f20 con 0x7f0514069560 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f051972c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140d13a0 con 0x7f0514159890 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f05127fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0514159c60 0x7f051415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49566/0 (socket says 172.21.15.170:49566) 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f05127fc700 1 -- 172.21.15.170:0/1810582836 learned_addr learned my addr 172.21.15.170:0/1810582836 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f05137fe700 1 --2- 172.21.15.170:0/1810582836 >> v2:172.21.15.191:3300/0 conn(0x7f0514069560 0x7f05140699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235994623 0 0) 0x7f051415a160 con 0x7f0514159c60 2023-10-08T16:10:53.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04fc003570 con 0x7f0514159c60 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0512ffd700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f051415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299914025 0 0) 0x7f04fc003570 con 0x7f0514159c60 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 msgr2=0x7f051415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f051415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 >> v2:172.21.15.191:3300/0 conn(0x7f0514069560 msgr2=0x7f05140699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 --2- 172.21.15.170:0/1810582836 >> v2:172.21.15.191:3300/0 conn(0x7f0514069560 0x7f05140699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051415e140 con 0x7f0514159c60 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f0508001470 con 0x7f0514159c60 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 >> v1:172.21.15.170:6789/0 conn(0x7f0514159c60 legacy=0x7f051415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04fc003a60 0x7f04fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc004810 0x7f04fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f04fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0514069f20 con 0x7f04fc003a60 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140d13a0 con 0x7f04fc004810 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc00ba60 con 0x7f0514159890 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f0512ffd700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04fc003a60 0x7f04fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.987+0000 7f05127fc700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc004810 0x7f04fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f05137fe700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f04fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 -- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 msgr2=0x7f04fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f04fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 -- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc004810 msgr2=0x7f04fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc004810 0x7f04fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 -- 172.21.15.170:0/1810582836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc003570 con 0x7f04fc003a60 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f05137fe700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0514159890 0x7f04fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0512ffd700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04fc003a60 0x7f04fc003e70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0504009c30 tx=0x7f05040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af4e8d19e2997e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400e710 con 0x7f04fc003a60 2023-10-08T16:10:53.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050400e870 con 0x7f04fc003a60 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f0511ffb700 1 -- 172.21.15.170:0/1810582836 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050400ec30 con 0x7f04fc003a60 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f051972c700 1 -- 172.21.15.170:0/1810582836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04fc003a60 msgr2=0x7f04fc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f051972c700 1 --2- 172.21.15.170:0/1810582836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04fc003a60 0x7f04fc003e70 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0504009c30 tx=0x7f05040091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f05137fe700 1 -- 172.21.15.170:0/1810582836 reap_dead start 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f051972c700 1 -- 172.21.15.170:0/1810582836 shutdown_connections 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f051972c700 1 -- 172.21.15.170:0/1810582836 >> 172.21.15.170:0/1810582836 conn(0x7f0514068e20 msgr2=0x7f05140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.988+0000 7f051972c700 1 -- 172.21.15.170:0/1810582836 shutdown_connections 2023-10-08T16:10:53.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.989+0000 7f051972c700 1 -- 172.21.15.170:0/1810582836 wait complete. 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.990+0000 7f051972c700 1 Processor -- start 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.990+0000 7f051972c700 1 -- start start 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.990+0000 7f051972c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc003a60 0x7f05140b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.990+0000 7f051972c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05140ca350 0x7f05141470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140b1160 con 0x7f05140c8240 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140ca7c0 con 0x7f04fc003a60 2023-10-08T16:10:53.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05141526c0 con 0x7f05140ca350 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56484/0 (socket says 172.21.15.170:56484) 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 -- 172.21.15.170:0/2924632599 learned_addr learned my addr 172.21.15.170:0/2924632599 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f0512ffd700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc003a60 0x7f05140b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05127fc700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05140ca350 0x7f05141470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 -- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05140ca350 msgr2=0x7f05141470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05140ca350 0x7f05141470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 -- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc003a60 msgr2=0x7f05140b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc003a60 0x7f05140b4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 -- 172.21.15.170:0/2924632599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140c8bf0 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f05137fe700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f05140d03b0 tx=0x7f050c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00e050 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c00f040 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05140ca9d0 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c010d40 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.991+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140b1900 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.992+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05140b1460 con 0x7f05140c8240 2023-10-08T16:10:54.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.993+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f050c003020 con 0x7f05140c8240 2023-10-08T16:10:54.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.993+0000 7f04faffd700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 0x7f05000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:54.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.993+0000 7f0512ffd700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 0x7f05000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.993+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f050c084f70 con 0x7f05140c8240 2023-10-08T16:10:54.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.993+0000 7f0512ffd700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 0x7f05000697d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05040091e0 tx=0x7f050401a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:54.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:53.997+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f050c051b50 con 0x7f05140c8240 2023-10-08T16:10:54.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.221+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05140b1b20 con 0x7f05140c8240 2023-10-08T16:10:54.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.856+0000 7f04faffd700 1 -- 172.21.15.170:0/2924632599 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f050c010ea0 con 0x7f05140c8240 2023-10-08T16:10:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 msgr2=0x7f05000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 0x7f05000697d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05040091e0 tx=0x7f050401a000 comp rx=0 tx=0).stop 2023-10-08T16:10:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 msgr2=0x7f05140c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f05140d03b0 tx=0x7f050c00b040 comp rx=0 tx=0).stop 2023-10-08T16:10:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 shutdown_connections 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0500067320 0x7f05000697d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05140c8240 0x7f05140c86b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04fc003a60 0x7f05140b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 --2- 172.21.15.170:0/2924632599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05140ca350 0x7f05141470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 >> 172.21.15.170:0/2924632599 conn(0x7f0514068e20 msgr2=0x7f0514153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 shutdown_connections 2023-10-08T16:10:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.858+0000 7f051972c700 1 -- 172.21.15.170:0/2924632599 wait complete. 2023-10-08T16:10:54.871 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:10:54.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.989+0000 7f8ad067a700 1 Processor -- start 2023-10-08T16:10:54.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.989+0000 7f8ad067a700 1 -- start start 2023-10-08T16:10:54.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8ad067a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8ac80b55a0 0x7f8ac8164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:54.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8ad067a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80cfa60 con 0x7f8ac80cd910 2023-10-08T16:10:54.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8ad067a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80b5b00 con 0x7f8ac8159ec0 2023-10-08T16:10:54.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8ad067a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac80b5da0 con 0x7f8ac80b55a0 2023-10-08T16:10:54.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8acdc15700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8ac80cd910 0x7f8ac80ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49572/0 (socket says 172.21.15.170:49572) 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.990+0000 7f8acdc15700 1 -- 172.21.15.170:0/4218466918 learned_addr learned my addr 172.21.15.170:0/4218466918 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acec17700 1 --2- 172.21.15.170:0/4218466918 >> v2:172.21.15.191:3300/0 conn(0x7f8ac80b55a0 0x7f8ac8164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557705931 0 0) 0x7f8ac80cfa60 con 0x7f8ac80cd910 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab4003570 con 0x7f8ac80cd910 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384669888 0 0) 0x7f8ac80b5b00 con 0x7f8ac8159ec0 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac80cfa60 con 0x7f8ac8159ec0 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3039050960 0 0) 0x7f8ab4003570 con 0x7f8ac80cd910 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acec17700 1 -- 172.21.15.170:0/4218466918 >> v1:172.21.15.191:6789/0 conn(0x7f8ac8159ec0 legacy=0x7f8ac80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acec17700 1 -- 172.21.15.170:0/4218466918 >> v1:172.21.15.170:6789/0 conn(0x7f8ac80cd910 legacy=0x7f8ac80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acec17700 1 -- 172.21.15.170:0/4218466918 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac80b6080 con 0x7f8ac80b55a0 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acec17700 1 --2- 172.21.15.170:0/4218466918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac80b55a0 0x7f8ac8164400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009910 tx=0x7f8ac00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dda633161e467ad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8001470 con 0x7f8ac80b55a0 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac0011420 con 0x7f8ac80b55a0 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.991+0000 7f8acd414700 1 -- 172.21.15.170:0/4218466918 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00117c0 con 0x7f8ac80b55a0 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 -- 172.21.15.170:0/4218466918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac80b55a0 msgr2=0x7f8ac8164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 --2- 172.21.15.170:0/4218466918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac80b55a0 0x7f8ac8164400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009910 tx=0x7f8ac00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 -- 172.21.15.170:0/4218466918 shutdown_connections 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 --2- 172.21.15.170:0/4218466918 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac80b55a0 0x7f8ac8164400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 -- 172.21.15.170:0/4218466918 >> 172.21.15.170:0/4218466918 conn(0x7f8ac80999f0 msgr2=0x7f8ac80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:54.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 -- 172.21.15.170:0/4218466918 shutdown_connections 2023-10-08T16:10:54.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.992+0000 7f8ad067a700 1 -- 172.21.15.170:0/4218466918 wait complete. 2023-10-08T16:10:54.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.993+0000 7f8ad067a700 1 Processor -- start 2023-10-08T16:10:54.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.993+0000 7f8ad067a700 1 -- start start 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac80b55a0 0x7f8ac815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac8068e10 0x7f8ac8156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac816d2c0 con 0x7f8ac8159ec0 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac816d5a0 con 0x7f8ac8068e10 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac816d880 con 0x7f8ac80b55a0 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56500/0 (socket says 172.21.15.170:56500) 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 -- 172.21.15.170:0/3125107449 learned_addr learned my addr 172.21.15.170:0/3125107449 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acec17700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac8068e10 0x7f8ac8156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ace416700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac80b55a0 0x7f8ac815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 -- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac80b55a0 msgr2=0x7f8ac815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac80b55a0 0x7f8ac815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 -- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac8068e10 msgr2=0x7f8ac8156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac8068e10 0x7f8ac8156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 -- 172.21.15.170:0/3125107449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0009420 con 0x7f8ac8159ec0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8acdc15700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80058d0 tx=0x7f8ab8001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800ce20 con 0x7f8ac8159ec0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8005b30 con 0x7f8ac8159ec0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac816db60 con 0x7f8ac8159ec0 2023-10-08T16:10:54.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab80070f0 con 0x7f8ac8159ec0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.994+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac816e120 con 0x7f8ac8159ec0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.995+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac80b29a0 con 0x7f8ac8159ec0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.996+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8ab8007460 con 0x7f8ac8159ec0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.996+0000 7f8abe7fc700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 0x7f8aac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.996+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f8ab8084a10 con 0x7f8ac8159ec0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.996+0000 7f8ace416700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 0x7f8aac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:55.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:54.996+0000 7f8ace416700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 0x7f8aac0697d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400f440 tx=0x7f8ac4002b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:55.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.000+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab8051650 con 0x7f8ac8159ec0 2023-10-08T16:10:55.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.181+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ac80896c0 con 0x7f8ac8159ec0 2023-10-08T16:10:55.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.860+0000 7f8abe7fc700 1 -- 172.21.15.170:0/3125107449 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ab8054c70 con 0x7f8ac8159ec0 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 msgr2=0x7f8aac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 0x7f8aac0697d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8ac400f440 tx=0x7f8ac4002b70 comp rx=0 tx=0).stop 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 msgr2=0x7f8ac80688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80058d0 tx=0x7f8ab8001ad0 comp rx=0 tx=0).stop 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 shutdown_connections 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ac8068e10 0x7f8ac8156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ac80b55a0 0x7f8ac815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8aac067320 0x7f8aac0697d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 --2- 172.21.15.170:0/3125107449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ac8159ec0 0x7f8ac80688d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 >> 172.21.15.170:0/3125107449 conn(0x7f8ac80999f0 msgr2=0x7f8ac80c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 shutdown_connections 2023-10-08T16:10:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.864+0000 7f8ad067a700 1 -- 172.21.15.170:0/3125107449 wait complete. 2023-10-08T16:10:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:10:55.877 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:10:55.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.996+0000 7f6b6a59f700 1 Processor -- start 2023-10-08T16:10:55.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.996+0000 7f6b6a59f700 1 -- start start 2023-10-08T16:10:56.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b6a59f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6b640cca20 0x7f6b640cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b6a59f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640cfa00 con 0x7f6b640b0790 2023-10-08T16:10:56.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b6a59f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640cb580 con 0x7f6b640b03c0 2023-10-08T16:10:56.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b6a59f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640cf160 con 0x7f6b640cca20 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b68d9c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6b640b0790 0x7f6b641591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50498/0 (socket says 172.21.15.170:50498) 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b68d9c700 1 -- 172.21.15.170:0/367405011 learned_addr learned my addr 172.21.15.170:0/367405011 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004312256 0 0) 0x7f6b640cfa00 con 0x7f6b640b0790 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b50003570 con 0x7f6b640b0790 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.997+0000 7f6b69d9e700 1 --2- 172.21.15.170:0/367405011 >> v2:172.21.15.191:3300/0 conn(0x7f6b640cca20 0x7f6b640cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 989967434 0 0) 0x7f6b50003570 con 0x7f6b640b0790 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 >> v2:172.21.15.191:3300/0 conn(0x7f6b640cca20 msgr2=0x7f6b640cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 --2- 172.21.15.170:0/367405011 >> v2:172.21.15.191:3300/0 conn(0x7f6b640cca20 0x7f6b640cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 >> v1:172.21.15.191:6789/0 conn(0x7f6b640b03c0 legacy=0x7f6b640b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640cf440 con 0x7f6b640b0790 2023-10-08T16:10:56.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6b54001470 con 0x7f6b640b0790 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 >> v1:172.21.15.170:6789/0 conn(0x7f6b640b0790 legacy=0x7f6b641591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 0x7f6b50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50004810 0x7f6b50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b500052e0 0x7f6b5000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cf160 con 0x7f6b50003a60 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5000c230 con 0x7f6b50004810 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5000c3f0 con 0x7f6b500052e0 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b69d9e700 1 --2- 172.21.15.170:0/367405011 >> v2:172.21.15.191:3300/0 conn(0x7f6b640cca20 0x7f6b640cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 0x7f6b50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b68d9c700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50004810 0x7f6b50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b69d9e700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b500052e0 0x7f6b5000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 -- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b500052e0 msgr2=0x7f6b5000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b500052e0 0x7f6b5000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 -- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50004810 msgr2=0x7f6b50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50004810 0x7f6b50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 -- 172.21.15.170:0/367405011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50003570 con 0x7f6b50003a60 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b69d9e700 1 -- 172.21.15.170:0/367405011 reap_dead start 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b6959d700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 0x7f6b50003e70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6b600018b0 tx=0x7f6b60007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b0864957c083da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60012420 con 0x7f6b50003a60 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600125a0 con 0x7f6b50003a60 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.998+0000 7f6b5bfff700 1 -- 172.21.15.170:0/367405011 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60012960 con 0x7f6b50003a60 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 -- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 msgr2=0x7f6b50003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 0x7f6b50003e70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6b600018b0 tx=0x7f6b60007d30 comp rx=0 tx=0).stop 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 -- 172.21.15.170:0/367405011 shutdown_connections 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/367405011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b50003a60 0x7f6b50003e70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 -- 172.21.15.170:0/367405011 >> 172.21.15.170:0/367405011 conn(0x7f6b64063230 msgr2=0x7f6b64063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 -- 172.21.15.170:0/367405011 shutdown_connections 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:55.999+0000 7f6b6a59f700 1 -- 172.21.15.170:0/367405011 wait complete. 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.000+0000 7f6b6a59f700 1 Processor -- start 2023-10-08T16:10:56.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.000+0000 7f6b6a59f700 1 -- start start 2023-10-08T16:10:56.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50003a60 0x7f6b64158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b6416ddb0 0x7f6b640d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6416d920 con 0x7f6b640c5260 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64158840 con 0x7f6b50003a60 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6a59f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64158b20 con 0x7f6b6416ddb0 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48574/0 (socket says 172.21.15.170:48574) 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 -- 172.21.15.170:0/2007248921 learned_addr learned my addr 172.21.15.170:0/2007248921 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b6959d700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50003a60 0x7f6b64158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b69d9e700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b6416ddb0 0x7f6b640d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 -- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b6416ddb0 msgr2=0x7f6b640d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b6416ddb0 0x7f6b640d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 -- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50003a60 msgr2=0x7f6b64158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50003a60 0x7f6b64158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 -- 172.21.15.170:0/2007248921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6416d5b0 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.001+0000 7f6b68d9c700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001200 tx=0x7f6b640b1a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.002+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54018030 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.002+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54007740 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.002+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b640c56b0 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.002+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400fac0 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.002+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b64158e00 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.003+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6416dad0 con 0x7f6b640c5260 2023-10-08T16:10:56.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.003+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6b5401b070 con 0x7f6b640c5260 2023-10-08T16:10:56.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.003+0000 7f6b597fa700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 0x7f6b480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:56.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.003+0000 7f6b6959d700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 0x7f6b480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:56.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.003+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f6b54019070 con 0x7f6b640c5260 2023-10-08T16:10:56.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.004+0000 7f6b6959d700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 0x7f6b480697d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6b640b3d50 tx=0x7f6b60015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:56.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.008+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b6416dad0 con 0x7f6b640c5260 2023-10-08T16:10:56.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.187+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b6416dad0 con 0x7f6b640c5260 2023-10-08T16:10:56.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.867+0000 7f6b597fa700 1 -- 172.21.15.170:0/2007248921 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6b6416dad0 con 0x7f6b640c5260 2023-10-08T16:10:56.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 msgr2=0x7f6b480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 0x7f6b480697d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6b640b3d50 tx=0x7f6b60015000 comp rx=0 tx=0).stop 2023-10-08T16:10:56.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 msgr2=0x7f6b641566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6b54001200 tx=0x7f6b640b1a10 comp rx=0 tx=0).stop 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 shutdown_connections 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b6416ddb0 0x7f6b640d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b50003a60 0x7f6b64158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6b48067320 0x7f6b480697d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 --2- 172.21.15.170:0/2007248921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b640c5260 0x7f6b641566a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 >> 172.21.15.170:0/2007248921 conn(0x7f6b64063230 msgr2=0x7f6b640b5ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 shutdown_connections 2023-10-08T16:10:56.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:56.871+0000 7f6b6a59f700 1 -- 172.21.15.170:0/2007248921 wait complete. 2023-10-08T16:10:56.873 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:10:56.884 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:10:57.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.000+0000 7f8d09e97700 1 Processor -- start 2023-10-08T16:10:57.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d09e97700 1 -- start start 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d09e97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8d040699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d09e97700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d04069ee0 con 0x7f8d04159890 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d09e97700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0415a160 con 0x7f8d04159c60 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d09e97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040d13a0 con 0x7f8d04069560 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d037fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8d04159890 0x7f8d0415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50502/0 (socket says 172.21.15.170:50502) 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.001+0000 7f8d037fe700 1 -- 172.21.15.170:0/112894057 learned_addr learned my addr 172.21.15.170:0/112894057 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435112927 0 0) 0x7f8d04069ee0 con 0x7f8d04159890 2023-10-08T16:10:57.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cec003570 con 0x7f8d04159890 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8d040699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1022073967 0 0) 0x7f8cec003570 con 0x7f8d04159890 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 msgr2=0x7f8d040699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8d040699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 >> v1:172.21.15.191:6789/0 conn(0x7f8d04159c60 legacy=0x7f8d0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0415e140 con 0x7f8d04159890 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8cf40017f0 con 0x7f8d04159890 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8d040699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 >> v1:172.21.15.170:6789/0 conn(0x7f8d04159890 legacy=0x7f8d0415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8cec003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cec004160 0x7f8cec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cec004e20 0x7f8cec005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040d13a0 con 0x7f8cec004e20 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00bd10 con 0x7f8cec004160 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec00bed0 con 0x7f8d04069560 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cec004e20 0x7f8cec005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d037fe700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8cec003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.002+0000 7f8d02ffd700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cec004160 0x7f8cec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 -- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 msgr2=0x7f8cec003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8d04069560 0x7f8cec003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 -- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cec004160 msgr2=0x7f8cec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:57.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cec004160 0x7f8cec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 -- 172.21.15.170:0/112894057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec003570 con 0x7f8cec004e20 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cec004e20 0x7f8cec005290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00abf0 tx=0x7f8cfc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395da1147e7ecb91 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc00a530 con 0x7f8cec004e20 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc00c6d0 con 0x7f8cec004e20 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d027fc700 1 -- 172.21.15.170:0/112894057 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc00ca70 con 0x7f8cec004e20 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 -- 172.21.15.170:0/112894057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cec004e20 msgr2=0x7f8cec005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 --2- 172.21.15.170:0/112894057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cec004e20 0x7f8cec005290 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc00abf0 tx=0x7f8cfc008040 comp rx=0 tx=0).stop 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d03fff700 1 -- 172.21.15.170:0/112894057 reap_dead start 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 -- 172.21.15.170:0/112894057 shutdown_connections 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 -- 172.21.15.170:0/112894057 >> 172.21.15.170:0/112894057 conn(0x7f8d04068e20 msgr2=0x7f8d040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:57.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 -- 172.21.15.170:0/112894057 shutdown_connections 2023-10-08T16:10:57.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.003+0000 7f8d09e97700 1 -- 172.21.15.170:0/112894057 wait complete. 2023-10-08T16:10:57.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 Processor -- start 2023-10-08T16:10:57.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 -- start start 2023-10-08T16:10:57.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cec004e20 0x7f8d040b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d040caef0 0x7f8d04169970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d041544c0 con 0x7f8d04169eb0 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d040b5920 con 0x7f8d040caef0 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d09e97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0415b8c0 con 0x7f8cec004e20 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d03fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d03fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48598/0 (socket says 172.21.15.170:48598) 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d03fff700 1 -- 172.21.15.170:0/3870149522 learned_addr learned my addr 172.21.15.170:0/3870149522 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d037fe700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cec004e20 0x7f8d040b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.005+0000 7f8d02ffd700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d040caef0 0x7f8d04169970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 -- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cec004e20 msgr2=0x7f8d040b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cec004e20 0x7f8d040b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 -- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d040caef0 msgr2=0x7f8d04169970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d040caef0 0x7f8d04169970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 -- 172.21.15.170:0/3870149522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0416a870 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d03fff700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00bce0 tx=0x7f8cfc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc01f070 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cfc00c840 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d040b5b50 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc0113f0 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.006+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d040b6110 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.007+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d041544c0 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.007+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8cfc022070 con 0x7f8d04169eb0 2023-10-08T16:10:57.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.008+0000 7f8ceb7fe700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 0x7f8cf00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.008+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8cfc000e40 con 0x7f8d04169eb0 2023-10-08T16:10:57.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.008+0000 7f8d037fe700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 0x7f8cf00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.008+0000 7f8d037fe700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 0x7f8cf00697d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400a0c0 tx=0x7f8cf40021d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.012+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cfc0526b0 con 0x7f8d04169eb0 2023-10-08T16:10:57.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.194+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8d0415ba50 con 0x7f8d04169eb0 2023-10-08T16:10:57.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.198+0000 7f8ceb7fe700 1 -- 172.21.15.170:0/3870149522 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8cfc055cd0 con 0x7f8d04169eb0 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 msgr2=0x7f8cf00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 0x7f8cf00697d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8cf400a0c0 tx=0x7f8cf40021d0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 msgr2=0x7f8d0416a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00bce0 tx=0x7f8cfc01a000 comp rx=0 tx=0).stop 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 shutdown_connections 2023-10-08T16:10:57.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8cf0067320 0x7f8cf00697d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8d04169eb0 0x7f8d0416a330 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cec004e20 0x7f8d040b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 --2- 172.21.15.170:0/3870149522 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8d040caef0 0x7f8d04169970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 >> 172.21.15.170:0/3870149522 conn(0x7f8d04068e20 msgr2=0x7f8d04069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 shutdown_connections 2023-10-08T16:10:57.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.200+0000 7f8d09e97700 1 -- 172.21.15.170:0/3870149522 wait complete. 2023-10-08T16:10:57.213 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup' 2023-10-08T16:10:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.334+0000 7f280cc3d700 1 Processor -- start 2023-10-08T16:10:57.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.334+0000 7f280cc3d700 1 -- start start 2023-10-08T16:10:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f280cc3d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2808159c60 0x7f280815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f280cc3d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280815a160 con 0x7f280815db10 2023-10-08T16:10:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f280cc3d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080d13a0 con 0x7f2808159890 2023-10-08T16:10:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f280cc3d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280815df50 con 0x7f2808159c60 2023-10-08T16:10:57.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2808159c60 0x7f280815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2808159c60 0x7f280815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48606/0 (socket says 172.21.15.170:48606) 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f2806d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f280815db10 0x7f2808069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50516/0 (socket says 172.21.15.170:50516) 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 -- 172.21.15.170:0/325244469 learned_addr learned my addr 172.21.15.170:0/325244469 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 -- 172.21.15.170:0/325244469 >> v1:172.21.15.191:6789/0 conn(0x7f2808159890 legacy=0x7f280815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 -- 172.21.15.170:0/325244469 >> v1:172.21.15.170:6789/0 conn(0x7f280815db10 legacy=0x7f2808069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 -- 172.21.15.170:0/325244469 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280815e140 con 0x7f2808159c60 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f27fdd9b700 1 --2- 172.21.15.170:0/325244469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f280815d5d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f27f00099b0 tx=0x7f27f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68765430bf85d61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f2805d9b700 1 -- 172.21.15.170:0/325244469 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000b820 con 0x7f2808159c60 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f2805d9b700 1 -- 172.21.15.170:0/325244469 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f000b980 con 0x7f2808159c60 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.335+0000 7f2805d9b700 1 -- 172.21.15.170:0/325244469 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000bd20 con 0x7f2808159c60 2023-10-08T16:10:57.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 -- 172.21.15.170:0/325244469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 msgr2=0x7f280815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 --2- 172.21.15.170:0/325244469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f280815d5d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f27f00099b0 tx=0x7f27f0009250 comp rx=0 tx=0).stop 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 -- 172.21.15.170:0/325244469 shutdown_connections 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 --2- 172.21.15.170:0/325244469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f280815d5d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 -- 172.21.15.170:0/325244469 >> 172.21.15.170:0/325244469 conn(0x7f2808068e20 msgr2=0x7f28080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 -- 172.21.15.170:0/325244469 shutdown_connections 2023-10-08T16:10:57.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.336+0000 7f280cc3d700 1 -- 172.21.15.170:0/325244469 wait complete. 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.337+0000 7f280cc3d700 1 Processor -- start 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.337+0000 7f280cc3d700 1 -- start start 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2808159890 0x7f280816b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280815db10 0x7f28080cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2808151f70 con 0x7f2808159c60 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081513a0 con 0x7f2808159890 2023-10-08T16:10:57.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2808151640 con 0x7f280815db10 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48608/0 (socket says 172.21.15.170:48608) 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 -- 172.21.15.170:0/2699957239 learned_addr learned my addr 172.21.15.170:0/2699957239 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280659c700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2808159890 0x7f280816b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 -- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280815db10 msgr2=0x7f28080cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280815db10 0x7f28080cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 -- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2808159890 msgr2=0x7f280816b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2808159890 0x7f280816b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 -- 172.21.15.170:0/2699957239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00094f0 con 0x7f2808159c60 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27fdd9b700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c010 tx=0x7f27f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011e10 con 0x7f2808159c60 2023-10-08T16:10:57.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0020070 con 0x7f2808159c60 2023-10-08T16:10:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2808151920 con 0x7f2808159c60 2023-10-08T16:10:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f001b6a0 con 0x7f2808159c60 2023-10-08T16:10:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.338+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2808150a80 con 0x7f2808159c60 2023-10-08T16:10:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.339+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2808150c10 con 0x7f2808159c60 2023-10-08T16:10:57.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.340+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f27f0022030 con 0x7f2808159c60 2023-10-08T16:10:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.340+0000 7f27feffd700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 0x7f27e80697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.340+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f27f0086820 con 0x7f2808159c60 2023-10-08T16:10:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.340+0000 7f280659c700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 0x7f27e80697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:57.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.341+0000 7f280659c700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 0x7f27e80697a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f27f800a8a0 tx=0x7f27f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:57.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.344+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27f00538c0 con 0x7f2808159c60 2023-10-08T16:10:57.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.605+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]} v 0) v1 -- 0x7f2808150c10 con 0x7f2808159c60 2023-10-08T16:10:57.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.868+0000 7f27feffd700 1 -- 172.21.15.170:0/2699957239 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_info_without_subvolumegroup"]}]=0 v307) v1 ==== 156+0+0 (secure 0 0 0) 0x7f27f0056ee0 con 0x7f2808159c60 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 msgr2=0x7f27e80697a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 0x7f27e80697a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f27f800a8a0 tx=0x7f27f8008040 comp rx=0 tx=0).stop 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 msgr2=0x7f2808150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c010 tx=0x7f27f0016000 comp rx=0 tx=0).stop 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 shutdown_connections 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f280815db10 0x7f28080cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2808159890 0x7f280816b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f27e80672f0 0x7f27e80697a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 --2- 172.21.15.170:0/2699957239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2808159c60 0x7f2808150540 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 >> 172.21.15.170:0/2699957239 conn(0x7f2808068e20 msgr2=0x7f2808155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 shutdown_connections 2023-10-08T16:10:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.870+0000 7f280cc3d700 1 -- 172.21.15.170:0/2699957239 wait complete. 2023-10-08T16:10:57.883 INFO:tasks.cephfs_test_runner:Tests the 'fs volume info' command without subvolume group ... ok 2023-10-08T16:10:57.883 INFO:tasks.cephfs_test_runner:Starting test: test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) That the existing and the newly created volumes can be listed and 2023-10-08T16:10:57.883 INFO:tasks.cephfs_test_runner:test_volume_ls (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:10:57.883 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-08T16:10:57.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.997+0000 7f514959c700 1 Processor -- start 2023-10-08T16:10:57.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f514959c700 1 -- start start 2023-10-08T16:10:58.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f514959c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f514415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:58.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f514959c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514415a160 con 0x7f514415db10 2023-10-08T16:10:58.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f514959c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440d13a0 con 0x7f5144159890 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f514959c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514415e140 con 0x7f5144159c60 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f51437fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f514415db10 0x7f5144069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50518/0 (socket says 172.21.15.170:50518) 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.998+0000 7f51437fe700 1 -- 172.21.15.170:0/4018181007 learned_addr learned my addr 172.21.15.170:0/4018181007 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f513a7fc700 1 --2- 172.21.15.170:0/4018181007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f514415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030967409 0 0) 0x7f514415a160 con 0x7f514415db10 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5130003570 con 0x7f514415db10 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324631460 0 0) 0x7f51440d13a0 con 0x7f5144159890 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f514415a160 con 0x7f5144159890 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f513a7fc700 1 -- 172.21.15.170:0/4018181007 >> v1:172.21.15.191:6789/0 conn(0x7f5144159890 legacy=0x7f514415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f513a7fc700 1 -- 172.21.15.170:0/4018181007 >> v1:172.21.15.170:6789/0 conn(0x7f514415db10 legacy=0x7f5144069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f513a7fc700 1 -- 172.21.15.170:0/4018181007 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514415a3d0 con 0x7f5144159c60 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f513a7fc700 1 --2- 172.21.15.170:0/4018181007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f514415d5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f512c009910 tx=0x7f512c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c295be465faa2a51 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c011630 con 0x7f5144159c60 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c011790 con 0x7f5144159c60 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:57.999+0000 7f51427fc700 1 -- 172.21.15.170:0/4018181007 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c011b30 con 0x7f5144159c60 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 -- 172.21.15.170:0/4018181007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 msgr2=0x7f514415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 --2- 172.21.15.170:0/4018181007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f514415d5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f512c009910 tx=0x7f512c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 -- 172.21.15.170:0/4018181007 shutdown_connections 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 --2- 172.21.15.170:0/4018181007 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f514415d5d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 -- 172.21.15.170:0/4018181007 >> 172.21.15.170:0/4018181007 conn(0x7f5144068e20 msgr2=0x7f51440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 -- 172.21.15.170:0/4018181007 shutdown_connections 2023-10-08T16:10:58.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.000+0000 7f514959c700 1 -- 172.21.15.170:0/4018181007 wait complete. 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.001+0000 7f514959c700 1 Processor -- start 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.001+0000 7f514959c700 1 -- start start 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5144159890 0x7f514416ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f51440ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440cd0e0 con 0x7f514415db10 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440cdc60 con 0x7f5144159890 2023-10-08T16:10:58.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f514959c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440cd330 con 0x7f5144159c60 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48610/0 (socket says 172.21.15.170:48610) 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 -- 172.21.15.170:0/210707301 learned_addr learned my addr 172.21.15.170:0/210707301 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f513a7fc700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f51440ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f5142ffd700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5144159890 0x7f514416ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 -- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 msgr2=0x7f51440ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f51440ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 -- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5144159890 msgr2=0x7f514416ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5144159890 0x7f514416ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 -- 172.21.15.170:0/210707301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009420 con 0x7f514415db10 2023-10-08T16:10:58.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.002+0000 7f51437fe700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f513c00cc20 tx=0x7f513c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.003+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00a960 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.003+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c00aac0 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.003+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51440cd610 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.003+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00ae80 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.003+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514416b150 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.004+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514416b3d0 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.004+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f513c007300 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.004+0000 7f513b7fe700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 0x7f51240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.004+0000 7f5142ffd700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 0x7f51240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.005+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f513c084d40 con 0x7f514415db10 2023-10-08T16:10:58.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.005+0000 7f5142ffd700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 0x7f51240697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f514414da50 tx=0x7f5134009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:58.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.009+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f513c0557e0 con 0x7f514415db10 2023-10-08T16:10:58.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.265+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f51440cddd0 con 0x7f514415db10 2023-10-08T16:10:58.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.872+0000 7f513b7fe700 1 -- 172.21.15.170:0/210707301 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v308) v1 ==== 134+0+0 (secure 0 0 0) 0x7f513c055370 con 0x7f514415db10 2023-10-08T16:10:58.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 msgr2=0x7f51240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 0x7f51240697d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f514414da50 tx=0x7f5134009040 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 msgr2=0x7f51440ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f513c00cc20 tx=0x7f513c001a40 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 shutdown_connections 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5124067320 0x7f51240697d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f514415db10 0x7f51440ce630 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5144159890 0x7f514416ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 --2- 172.21.15.170:0/210707301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5144159c60 0x7f51440ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 >> 172.21.15.170:0/210707301 conn(0x7f5144068e20 msgr2=0x7f51440cf920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 shutdown_connections 2023-10-08T16:10:58.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:58.874+0000 7f514959c700 1 -- 172.21.15.170:0/210707301 wait complete. 2023-10-08T16:10:58.887 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:10:59.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.005+0000 7f6eed04e700 1 Processor -- start 2023-10-08T16:10:59.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.005+0000 7f6eed04e700 1 -- start start 2023-10-08T16:10:59.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6eed04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6eed04e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6eed04e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-08T16:10:59.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6eed04e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80d13a0 con 0x7f6ee815db10 2023-10-08T16:10:59.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6eed04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815e140 con 0x7f6ee8159c60 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee6d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6ee8159890 0x7f6ee815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50532/0 (socket says 172.21.15.170:50532) 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee6d9d700 1 -- 172.21.15.170:0/1829000596 learned_addr learned my addr 172.21.15.170:0/1829000596 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee759e700 1 -- 172.21.15.170:0/1829000596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee759e700 1 --2- 172.21.15.170:0/1829000596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee759e700 1 -- 172.21.15.170:0/1829000596 >> v1:172.21.15.170:6789/0 conn(0x7f6ee8159890 legacy=0x7f6ee815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee759e700 1 -- 172.21.15.170:0/1829000596 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee815a3d0 con 0x7f6ee815db10 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.006+0000 7f6ee5d9b700 1 -- 172.21.15.170:0/1829000596 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280110103 0 0) 0x7f6ee815a160 con 0x7f6ee8159890 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6ee759e700 1 --2- 172.21.15.170:0/1829000596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee8069560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00098e0 tx=0x7f6ee00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663021542af94f58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6ee5d9b700 1 -- 172.21.15.170:0/1829000596 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0009e80 con 0x7f6ee815db10 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6ee5d9b700 1 -- 172.21.15.170:0/1829000596 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000b860 con 0x7f6ee815db10 2023-10-08T16:10:59.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6ee5d9b700 1 -- 172.21.15.170:0/1829000596 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000bbe0 con 0x7f6ee815db10 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 -- 172.21.15.170:0/1829000596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 msgr2=0x7f6ee8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 --2- 172.21.15.170:0/1829000596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee8069560 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00098e0 tx=0x7f6ee00091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 -- 172.21.15.170:0/1829000596 shutdown_connections 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 --2- 172.21.15.170:0/1829000596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee8069560 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 --2- 172.21.15.170:0/1829000596 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159c60 0x7f6ee815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 -- 172.21.15.170:0/1829000596 >> 172.21.15.170:0/1829000596 conn(0x7f6ee8068e20 msgr2=0x7f6ee80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 -- 172.21.15.170:0/1829000596 shutdown_connections 2023-10-08T16:10:59.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.007+0000 7f6eed04e700 1 -- 172.21.15.170:0/1829000596 wait complete. 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 Processor -- start 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 -- start start 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159890 0x7f6ee80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee81512d0 con 0x7f6ee815db10 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8151b70 con 0x7f6ee8159c60 2023-10-08T16:10:59.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6eed04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80cfd90 con 0x7f6ee8159890 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6ee759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6ee759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48618/0 (socket says 172.21.15.170:48618) 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.009+0000 7f6ee759e700 1 -- 172.21.15.170:0/695418035 learned_addr learned my addr 172.21.15.170:0/695418035 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee6d9d700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159890 0x7f6ee80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 -- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159890 msgr2=0x7f6ee80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159890 0x7f6ee80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 -- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 msgr2=0x7f6ee80cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee659c700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 -- 172.21.15.170:0/695418035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0009420 con 0x7f6ee815db10 2023-10-08T16:10:59.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee659c700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6ee759e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b3c0 tx=0x7f6ee0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee001f070 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000b9d0 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8151520 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0025650 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.010+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee814c810 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.011+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee8151ce0 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.011+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6ee0023070 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.012+0000 7f6eceffd700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 0x7f6ed40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.012+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6ee0086060 con 0x7f6ee815db10 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.012+0000 7f6ee6d9d700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 0x7f6ed40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.012+0000 7f6ee6d9d700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 0x7f6ed40697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8001560 tx=0x7f6ed8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.016+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ee00b75a0 con 0x7f6ee815db10 2023-10-08T16:10:59.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.199+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6ee804c3c0 con 0x7f6ee815db10 2023-10-08T16:10:59.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.204+0000 7f6eceffd700 1 -- 172.21.15.170:0/695418035 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ee0052ff0 con 0x7f6ee815db10 2023-10-08T16:10:59.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 msgr2=0x7f6ed40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 0x7f6ed40697d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8001560 tx=0x7f6ed8009040 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 msgr2=0x7f6ee80cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000b3c0 tx=0x7f6ee0017000 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 shutdown_connections 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ed4067320 0x7f6ed40697d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6ee815db10 0x7f6ee80cf710 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6ee8159890 0x7f6ee80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 --2- 172.21.15.170:0/695418035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6ee8159c60 0x7f6ee80cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 >> 172.21.15.170:0/695418035 conn(0x7f6ee8068e20 msgr2=0x7f6ee8157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 shutdown_connections 2023-10-08T16:10:59.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.206+0000 7f6eed04e700 1 -- 172.21.15.170:0/695418035 wait complete. 2023-10-08T16:10:59.218 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:10:59.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:59.256 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:10:59.299 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:10:59.299 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:10:59.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:10:59.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.412+0000 7f471119d700 1 Processor -- start 2023-10-08T16:10:59.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.413+0000 7f471119d700 1 -- start start 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.413+0000 7f471119d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f470c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.413+0000 7f471119d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c0b40f0 con 0x7f470c06c0f0 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.413+0000 7f471119d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f470c069640 con 0x7f470c0ad870 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.413+0000 7f471119d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c069920 con 0x7f470c142ea0 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f471099c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f470c06c0f0 0x7f470c06a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50546/0 (socket says 172.21.15.170:50546) 2023-10-08T16:10:59.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f471099c700 1 -- 172.21.15.170:0/408442878 learned_addr learned my addr 172.21.15.170:0/408442878 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470bfff700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f470c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202399518 0 0) 0x7f470c0b40f0 con 0x7f470c06c0f0 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f4003570 con 0x7f470c06c0f0 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101529182 0 0) 0x7f470c069640 con 0x7f470c0ad870 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470c0b40f0 con 0x7f470c0ad870 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749465937 0 0) 0x7f46f4003570 con 0x7f470c06c0f0 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 msgr2=0x7f470c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f470c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 >> v1:172.21.15.191:6789/0 conn(0x7f470c0ad870 legacy=0x7f470c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c069c00 con 0x7f470c06c0f0 2023-10-08T16:10:59.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f47040017f0 con 0x7f470c06c0f0 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 >> v1:172.21.15.170:6789/0 conn(0x7f470c06c0f0 legacy=0x7f470c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f4003c20 0x7f46f4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f46f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4004d50 0x7f46f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c069920 con 0x7f46f4003c20 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f400bbf0 con 0x7f46f4004d50 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f400bd60 con 0x7f470c142ea0 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470bfff700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f4003c20 0x7f46f4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f471099c700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4004d50 0x7f46f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.414+0000 7f470b7fe700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f46f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 -- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 msgr2=0x7f46f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:59.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c142ea0 0x7f46f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 -- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4004d50 msgr2=0x7f46f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4004d50 0x7f46f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 -- 172.21.15.170:0/408442878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f4003570 con 0x7f46f4003c20 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471099c700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4004d50 0x7f46f40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470bfff700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f4003c20 0x7f46f4004010 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f46fc006f00 tx=0x7f46fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8911bc92b2a1d1fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00e710 con 0x7f46f4003c20 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46fc00e870 con 0x7f46f4003c20 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f470affd700 1 -- 172.21.15.170:0/408442878 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46fc00ec30 con 0x7f46f4003c20 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 -- 172.21.15.170:0/408442878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f4003c20 msgr2=0x7f46f4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 --2- 172.21.15.170:0/408442878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f4003c20 0x7f46f4004010 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f46fc006f00 tx=0x7f46fc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471099c700 1 -- 172.21.15.170:0/408442878 reap_dead start 2023-10-08T16:10:59.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 -- 172.21.15.170:0/408442878 shutdown_connections 2023-10-08T16:10:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 -- 172.21.15.170:0/408442878 >> 172.21.15.170:0/408442878 conn(0x7f470c0ce430 msgr2=0x7f470c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 -- 172.21.15.170:0/408442878 shutdown_connections 2023-10-08T16:10:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.415+0000 7f471119d700 1 -- 172.21.15.170:0/408442878 wait complete. 2023-10-08T16:10:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.416+0000 7f471119d700 1 Processor -- start 2023-10-08T16:10:59.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 -- start start 2023-10-08T16:10:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4003c20 0x7f470c0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 0x7f470c0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c15d100 con 0x7f470c15c740 2023-10-08T16:10:59.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c9580 con 0x7f46f4003c20 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471119d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470c0c9860 con 0x7f470c15ea70 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.417+0000 7f471099c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48640/0 (socket says 172.21.15.170:48640) 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 -- 172.21.15.170:0/1923768929 learned_addr learned my addr 172.21.15.170:0/1923768929 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f470bfff700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4003c20 0x7f470c0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f470b7fe700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 0x7f470c0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 -- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 msgr2=0x7f470c0acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 0x7f470c0acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 -- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4003c20 msgr2=0x7f470c0c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4003c20 0x7f470c0c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 -- 172.21.15.170:0/1923768929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c0c9b40 con 0x7f470c15c740 2023-10-08T16:10:59.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f470b7fe700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 0x7f470c0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471099c700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f470c071240 tx=0x7f47040013e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704007b30 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470400ebc0 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470c0c9e20 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704013420 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.418+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c0ad2c0 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.419+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470c04c3c0 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.419+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4704013850 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.420+0000 7f46f3fff700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 0x7f46f8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.420+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4704084dd0 con 0x7f470c15c740 2023-10-08T16:10:59.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.420+0000 7f470bfff700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 0x7f46f8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.420+0000 7f470bfff700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 0x7f46f8069800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0091e0 tx=0x7f46fc01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.424+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4704051d60 con 0x7f470c15c740 2023-10-08T16:10:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.649+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f470c143270 con 0x7f470c15c740 2023-10-08T16:10:59.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.649+0000 7f46f3fff700 1 -- 172.21.15.170:0/1923768929 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+3541 (secure 0 0 0) 0x7f4704055380 con 0x7f470c15c740 2023-10-08T16:10:59.650 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:10:59.650 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":68,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1316606855","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1316606855},{"type":"v1","addr":"172.21.15.191:6834","nonce":1316606855}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6481,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3595703852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3595703852},{"type":"v1","addr":"172.21.15.191:6835","nonce":3595703852}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":6750,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3797466586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3797466586},{"type":"v1","addr":"172.21.15.170:6836","nonce":3797466586}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1084744652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1084744652},{"type":"v1","addr":"172.21.15.170:6837","nonce":1084744652}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[]} 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 msgr2=0x7f46f8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 0x7f46f8069800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0091e0 tx=0x7f46fc01a000 comp rx=0 tx=0).stop 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 msgr2=0x7f470c15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f470c071240 tx=0x7f47040013e0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 shutdown_connections 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46f8067350 0x7f46f8069800 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f470c15c740 0x7f470c15cbc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f4003c20 0x7f470c0c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 --2- 172.21.15.170:0/1923768929 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f470c15ea70 0x7f470c0acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 >> 172.21.15.170:0/1923768929 conn(0x7f470c0ce430 msgr2=0x7f470c141e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:59.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 shutdown_connections 2023-10-08T16:10:59.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.651+0000 7f471119d700 1 -- 172.21.15.170:0/1923768929 wait complete. 2023-10-08T16:10:59.653 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 68 2023-10-08T16:10:59.666 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:10:59.666 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:10:59.667 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:10:59.667 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:59.667 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:10:59.667 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:10:59.667 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:10:59.667 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:59.668 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:10:59.668 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:10:59.668 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:10:59.668 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:59.668 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:10:59.668 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:10:59.669 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:10:59.669 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:10:59.677 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:10:59.677 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:10:59.678 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:10:59.678 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:10:59.680 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:10:59.680 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:10:59.681 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:10:59.681 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:10:59.719 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:10:59.720 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:59.720 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:10:59.721 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:59.723 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:10:59.723 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:59.724 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:10:59.724 DEBUG:teuthology.parallel:result is None 2023-10-08T16:10:59.724 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:10:59.792 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:10:59.793 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:10:59.793 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:10:59.793+0000 7f1d2b5ae700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:10:59.804 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:10:59.805 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:10:59.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.845+0000 7f8f0a26a700 1 Processor -- start 2023-10-08T16:10:59.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.845+0000 7f8f0a26a700 1 -- start start 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f0a26a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f0a26a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f0406c0a0 0x7f8f0415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f0a26a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0415d7e0 con 0x7f8f0415df50 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f0a26a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0415dac0 con 0x7f8f0406c0a0 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f0a26a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0415aa30 con 0x7f8f0406bcd0 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8f0406c0a0 0x7f8f0415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f08807700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f0415df50 0x7f8f040c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50554/0 (socket says 172.21.15.170:50554) 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f08807700 1 -- 172.21.15.170:0/2672076145 learned_addr learned my addr 172.21.15.170:0/2672076145 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:10:59.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f03fff700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 -- 172.21.15.170:0/2672076145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 msgr2=0x7f8f040caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 -- 172.21.15.170:0/2672076145 >> v1:172.21.15.170:6789/0 conn(0x7f8f0415df50 legacy=0x7f8f040c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 -- 172.21.15.170:0/2672076145 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0415ad10 con 0x7f8f0406c0a0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f02ffd700 1 -- 172.21.15.170:0/2672076145 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422286263 0 0) 0x7f8f0415d7e0 con 0x7f8f0415df50 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.846+0000 7f8f037fe700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0406c0a0 0x7f8f0415a1e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25564b3e7373652a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f02ffd700 1 -- 172.21.15.170:0/2672076145 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4009e80 con 0x7f8f0406c0a0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f02ffd700 1 -- 172.21.15.170:0/2672076145 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400b860 con 0x7f8f0406c0a0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f02ffd700 1 -- 172.21.15.170:0/2672076145 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400bbe0 con 0x7f8f0406c0a0 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 -- 172.21.15.170:0/2672076145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0406c0a0 msgr2=0x7f8f0415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0406c0a0 0x7f8f0415a1e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef40091a0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 -- 172.21.15.170:0/2672076145 shutdown_connections 2023-10-08T16:10:59.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/2672076145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0406c0a0 0x7f8f0415a1e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 -- 172.21.15.170:0/2672076145 >> 172.21.15.170:0/2672076145 conn(0x7f8f040ce600 msgr2=0x7f8f040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:10:59.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 -- 172.21.15.170:0/2672076145 shutdown_connections 2023-10-08T16:10:59.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.847+0000 7f8f0a26a700 1 -- 172.21.15.170:0/2672076145 wait complete. 2023-10-08T16:10:59.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.848+0000 7f8f0a26a700 1 Processor -- start 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.848+0000 7f8f0a26a700 1 -- start start 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 0x7f8f0414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0416d1b0 con 0x7f8f0415df50 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0416a120 con 0x7f8f0406c0a0 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f0a26a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0416a400 con 0x7f8f0406bcd0 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f08807700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f08807700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48700/0 (socket says 172.21.15.170:48700) 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f08807700 1 -- 172.21.15.170:0/1426940634 learned_addr learned my addr 172.21.15.170:0/1426940634 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f03fff700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f037fe700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 0x7f8f0414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f08807700 1 -- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 msgr2=0x7f8f040c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.849+0000 7f8f08807700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f08807700 1 -- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 msgr2=0x7f8f0414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f08807700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 0x7f8f0414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f08807700 1 -- 172.21.15.170:0/1426940634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4009420 con 0x7f8f0415df50 2023-10-08T16:10:59.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f037fe700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 0x7f8f0414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f08807700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00cc90 tx=0x7f8efc001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00cea0 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00ab70 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0070f0 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0416a6e0 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.850+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f040c4a70 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.851+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0404c3c0 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.851+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8efc0074a0 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.852+0000 7f8efbfff700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 0x7f8ee40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.852+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f8efc084e70 con 0x7f8f0415df50 2023-10-08T16:10:59.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.852+0000 7f8f03fff700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 0x7f8ee40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:10:59.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.852+0000 7f8f03fff700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 0x7f8ee40697d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8eec006fd0 tx=0x7f8eec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:10:59.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:10:59.856+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8efc051e00 con 0x7f8f0415df50 2023-10-08T16:11:00.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.057+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8f0415b040 con 0x7f8f0415df50 2023-10-08T16:11:00.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.885+0000 7f8efbfff700 1 -- 172.21.15.170:0/1426940634 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v109) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8efc055420 con 0x7f8f0415df50 2023-10-08T16:11:00.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 msgr2=0x7f8ee40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:00.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 0x7f8ee40697d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8eec006fd0 tx=0x7f8eec008040 comp rx=0 tx=0).stop 2023-10-08T16:11:00.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 msgr2=0x7f8f040b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:00.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00cc90 tx=0x7f8efc001a80 comp rx=0 tx=0).stop 2023-10-08T16:11:00.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 shutdown_connections 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ee4067320 0x7f8ee40697d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f0415df50 0x7f8f040b13a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f0406bcd0 0x7f8f040c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 --2- 172.21.15.170:0/1426940634 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f0406c0a0 0x7f8f0414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 >> 172.21.15.170:0/1426940634 conn(0x7f8f040ce600 msgr2=0x7f8f040cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 shutdown_connections 2023-10-08T16:11:00.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:00.887+0000 7f8f0a26a700 1 -- 172.21.15.170:0/1426940634 wait complete. 2023-10-08T16:11:00.890 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:11:00.901 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:11:01.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.017+0000 7f96b4976700 1 Processor -- start 2023-10-08T16:11:01.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.017+0000 7f96b4976700 1 -- start start 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96b4976700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 0x7f96b00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96b4976700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96b4976700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b015dbb0 con 0x7f96b0159420 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96b4976700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b00b11d0 con 0x7f96b01597f0 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96b4976700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b14b0 con 0x7f96b015d230 2023-10-08T16:11:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96a659c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 0x7f96b00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:50494/0 (socket says 172.21.15.170:50494) 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 -- 172.21.15.170:0/923822890 learned_addr learned my addr 172.21.15.170:0/923822890 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 -- 172.21.15.170:0/923822890 >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 msgr2=0x7f96b00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 --2- 172.21.15.170:0/923822890 >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 0x7f96b00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 -- 172.21.15.170:0/923822890 >> v1:172.21.15.191:6789/0 conn(0x7f96b0159420 legacy=0x7f96b00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96af59e700 1 -- 172.21.15.170:0/923822890 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00b1790 con 0x7f96b015d230 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.018+0000 7f96a659c700 1 --2- 172.21.15.170:0/923822890 >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 0x7f96b00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96af59e700 1 --2- 172.21.15.170:0/923822890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009f20 tx=0x7f96a8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d29fbdcea0751a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96ae59c700 1 -- 172.21.15.170:0/923822890 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800e070 con 0x7f96b015d230 2023-10-08T16:11:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96ae59c700 1 -- 172.21.15.170:0/923822890 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800b9d0 con 0x7f96b015d230 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96ae59c700 1 -- 172.21.15.170:0/923822890 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80092e0 con 0x7f96b015d230 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 -- 172.21.15.170:0/923822890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 msgr2=0x7f96b015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 --2- 172.21.15.170:0/923822890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009f20 tx=0x7f96a8009840 comp rx=0 tx=0).stop 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 -- 172.21.15.170:0/923822890 shutdown_connections 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 --2- 172.21.15.170:0/923822890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b015d670 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 --2- 172.21.15.170:0/923822890 >> v2:172.21.15.191:3300/0 conn(0x7f96b01597f0 0x7f96b00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 -- 172.21.15.170:0/923822890 >> 172.21.15.170:0/923822890 conn(0x7f96b00d0660 msgr2=0x7f96b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 -- 172.21.15.170:0/923822890 shutdown_connections 2023-10-08T16:11:01.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.019+0000 7f96b4976700 1 -- 172.21.15.170:0/923822890 wait complete. 2023-10-08T16:11:01.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.020+0000 7f96b4976700 1 Processor -- start 2023-10-08T16:11:01.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.020+0000 7f96b4976700 1 -- start start 2023-10-08T16:11:01.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b0159420 0x7f96b014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0157390 con 0x7f96b01597f0 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0156dd0 con 0x7f96b0159420 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96b4976700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01570b0 con 0x7f96b015d230 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48714/0 (socket says 172.21.15.170:48714) 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96af59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96a659c700 1 -- 172.21.15.170:0/3546747070 learned_addr learned my addr 172.21.15.170:0/3546747070 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:01.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.021+0000 7f96aed9d700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b0159420 0x7f96b014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 -- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b0159420 msgr2=0x7f96b014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b0159420 0x7f96b014ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 -- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 msgr2=0x7f96b0154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 -- 172.21.15.170:0/3546747070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009420 con 0x7f96b015d230 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96a659c700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96af59e700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b0153f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96a8015000 tx=0x7f96a8018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8008ad0 con 0x7f96b015d230 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8008c30 con 0x7f96b015d230 2023-10-08T16:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a8009ae0 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a801c410 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.022+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b015a990 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.023+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b015ab80 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.023+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f96a8003040 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.024+0000 7f96a77fe700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 0x7f96900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.024+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f96a8086c20 con 0x7f96b015d230 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.024+0000 7f96aed9d700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 0x7f96900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.024+0000 7f96aed9d700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 0x7f96900697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96a0003a80 tx=0x7f96a0007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.028+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96a8086e60 con 0x7f96b015d230 2023-10-08T16:11:01.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.285+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f96b004c3c0 con 0x7f96b015d230 2023-10-08T16:11:01.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.286+0000 7f96a77fe700 1 -- 172.21.15.170:0/3546747070 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f96a8053b70 con 0x7f96b015d230 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:01.287 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:01.288 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.289 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:11:01.290 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.291 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.292 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:11:01.293 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.294 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.295 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.296 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:11:01.297 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:11:01.298 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:11:01.299 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.300 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.301 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:01.302 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:11:01.303 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:11:01.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 msgr2=0x7f96900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 0x7f96900697d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f96a0003a80 tx=0x7f96a0007ec0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 msgr2=0x7f96b0153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b0153f80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f96a8015000 tx=0x7f96a8018000 comp rx=0 tx=0).stop 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 shutdown_connections 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96b015d230 0x7f96b0153f80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96b0159420 0x7f96b014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9690067320 0x7f96900697d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 --2- 172.21.15.170:0/3546747070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96b01597f0 0x7f96b0154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 >> 172.21.15.170:0/3546747070 conn(0x7f96b00d0660 msgr2=0x7f96b0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 shutdown_connections 2023-10-08T16:11:01.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.288+0000 7f96b4976700 1 -- 172.21.15.170:0/3546747070 wait complete. 2023-10-08T16:11:01.307 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:01.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.421+0000 7f308ffff700 1 Processor -- start 2023-10-08T16:11:01.422 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.421+0000 7f308ffff700 1 -- start start 2023-10-08T16:11:01.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3090159420 0x7f30900b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f30901597f0 0x7f30900b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308ffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30900b11d0 con 0x7f309015d230 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308ffff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30900b14b0 con 0x7f3090159420 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308ffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30900b1790 con 0x7f30901597f0 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308effd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3090159420 0x7f30900b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308f7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f309015d230 0x7f309015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50560/0 (socket says 172.21.15.170:50560) 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308f7fe700 1 -- 172.21.15.170:0/317102275 learned_addr learned my addr 172.21.15.170:0/317102275 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f30867fc700 1 --2- 172.21.15.170:0/317102275 >> v2:172.21.15.191:3300/0 conn(0x7f30901597f0 0x7f30900b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308effd700 1 -- 172.21.15.170:0/317102275 >> v2:172.21.15.191:3300/0 conn(0x7f30901597f0 msgr2=0x7f30900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308effd700 1 --2- 172.21.15.170:0/317102275 >> v2:172.21.15.191:3300/0 conn(0x7f30901597f0 0x7f30900b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308effd700 1 -- 172.21.15.170:0/317102275 >> v1:172.21.15.170:6789/0 conn(0x7f309015d230 legacy=0x7f309015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308effd700 1 -- 172.21.15.170:0/317102275 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30900b1a70 con 0x7f3090159420 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.422+0000 7f308e7fc700 1 -- 172.21.15.170:0/317102275 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080023207 0 0) 0x7f30900b11d0 con 0x7f309015d230 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308effd700 1 --2- 172.21.15.170:0/317102275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f30900b5f40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3090089750 tx=0x7f30800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87bc21f2821969 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308e7fc700 1 -- 172.21.15.170:0/317102275 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000b6b0 con 0x7f3090159420 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308e7fc700 1 -- 172.21.15.170:0/317102275 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308000b810 con 0x7f3090159420 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308e7fc700 1 -- 172.21.15.170:0/317102275 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000bbb0 con 0x7f3090159420 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 -- 172.21.15.170:0/317102275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 msgr2=0x7f30900b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 --2- 172.21.15.170:0/317102275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f30900b5f40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3090089750 tx=0x7f30800091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 -- 172.21.15.170:0/317102275 shutdown_connections 2023-10-08T16:11:01.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 --2- 172.21.15.170:0/317102275 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f30900b5f40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 --2- 172.21.15.170:0/317102275 >> v2:172.21.15.191:3300/0 conn(0x7f30901597f0 0x7f30900b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 -- 172.21.15.170:0/317102275 >> 172.21.15.170:0/317102275 conn(0x7f30900d0660 msgr2=0x7f30900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 -- 172.21.15.170:0/317102275 shutdown_connections 2023-10-08T16:11:01.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.423+0000 7f308ffff700 1 -- 172.21.15.170:0/317102275 wait complete. 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.424+0000 7f308ffff700 1 Processor -- start 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 -- start start 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30901597f0 0x7f3090155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f309015d230 0x7f30900c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3090169c10 con 0x7f3090159420 2023-10-08T16:11:01.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309015b8b0 con 0x7f30901597f0 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3090169620 con 0x7f309015d230 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48728/0 (socket says 172.21.15.170:48728) 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308effd700 1 -- 172.21.15.170:0/2440660482 learned_addr learned my addr 172.21.15.170:0/2440660482 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f30867fc700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30901597f0 0x7f3090155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.425+0000 7f308f7fe700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f309015d230 0x7f30900c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 -- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f309015d230 msgr2=0x7f30900c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f309015d230 0x7f30900c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 -- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30901597f0 msgr2=0x7f3090155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30901597f0 0x7f3090155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 -- 172.21.15.170:0/2440660482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009420 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308effd700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f30800091e0 tx=0x7f3080017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000bb90 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080011420 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30900cc010 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30800117c0 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.426+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309015b0a0 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.427+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3090169e80 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.427+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3080011bf0 con 0x7f3090159420 2023-10-08T16:11:01.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.428+0000 7f30877fe700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 0x7f30700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.428+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3080086350 con 0x7f3090159420 2023-10-08T16:11:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.428+0000 7f30867fc700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 0x7f30700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.428+0000 7f30867fc700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 0x7f30700697d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30900af0c0 tx=0x7f3078008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.432+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3080053330 con 0x7f3090159420 2023-10-08T16:11:01.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.617+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f309004c3c0 con 0x7f3090159420 2023-10-08T16:11:01.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.618+0000 7f30877fe700 1 -- 172.21.15.170:0/2440660482 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+12184 (secure 0 0 0) 0x7f3080056950 con 0x7f3090159420 2023-10-08T16:11:01.618 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:01.619 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":109,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:00.883306+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 msgr2=0x7f30700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 0x7f30700697d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f30900af0c0 tx=0x7f3078008040 comp rx=0 tx=0).stop 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 msgr2=0x7f309016a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f30800091e0 tx=0x7f3080017000 comp rx=0 tx=0).stop 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 shutdown_connections 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f309015d230 0x7f30900c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3070067320 0x7f30700697d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3090159420 0x7f309016a030 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 --2- 172.21.15.170:0/2440660482 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f30901597f0 0x7f3090155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 >> 172.21.15.170:0/2440660482 conn(0x7f30900d0660 msgr2=0x7f3090068b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 shutdown_connections 2023-10-08T16:11:01.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.620+0000 7f308ffff700 1 -- 172.21.15.170:0/2440660482 wait complete. 2023-10-08T16:11:01.632 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:11:01.633 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:11:01.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.749+0000 7f41a4bfc700 1 Processor -- start 2023-10-08T16:11:01.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.749+0000 7f41a4bfc700 1 -- start start 2023-10-08T16:11:01.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f41a4bfc700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f41a0159890 0x7f41a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f41a4bfc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 0x7f41a015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f41a4bfc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a015a160 con 0x7f41a015db10 2023-10-08T16:11:01.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f41a4bfc700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a00d13a0 con 0x7f41a0159890 2023-10-08T16:11:01.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f41a4bfc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015e140 con 0x7f41a0159c60 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f41a015db10 0x7f41a0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50566/0 (socket says 172.21.15.170:50566) 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419ed9d700 1 -- 172.21.15.170:0/21341574 learned_addr learned my addr 172.21.15.170:0/21341574 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f4195d9b700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 0x7f41a015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419e59c700 1 --2- 172.21.15.170:0/21341574 >> v2:172.21.15.191:3300/0 conn(0x7f41a0159890 0x7f41a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419dd9b700 1 -- 172.21.15.170:0/21341574 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479091344 0 0) 0x7f41a015a160 con 0x7f41a015db10 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419dd9b700 1 -- 172.21.15.170:0/21341574 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c003570 con 0x7f41a015db10 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419e59c700 1 -- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 msgr2=0x7f41a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419e59c700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 0x7f41a015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419e59c700 1 -- 172.21.15.170:0/21341574 >> v1:172.21.15.170:6789/0 conn(0x7f41a015db10 legacy=0x7f41a0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f419e59c700 1 -- 172.21.15.170:0/21341574 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a015a3d0 con 0x7f41a0159890 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.750+0000 7f4195d9b700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 0x7f41a015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f419e59c700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159890 0x7f41a015abd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f41900098e0 tx=0x7f41900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcaee8e9cbc2847f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f419dd9b700 1 -- 172.21.15.170:0/21341574 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190011630 con 0x7f41a0159890 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f419dd9b700 1 -- 172.21.15.170:0/21341574 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4190011790 con 0x7f41a0159890 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f419dd9b700 1 -- 172.21.15.170:0/21341574 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190011b30 con 0x7f41a0159890 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 -- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159890 msgr2=0x7f41a015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159890 0x7f41a015abd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f41900098e0 tx=0x7f41900091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 -- 172.21.15.170:0/21341574 shutdown_connections 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159890 0x7f41a015abd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/21341574 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159c60 0x7f41a015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 -- 172.21.15.170:0/21341574 >> 172.21.15.170:0/21341574 conn(0x7f41a0068e20 msgr2=0x7f41a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 -- 172.21.15.170:0/21341574 shutdown_connections 2023-10-08T16:11:01.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.751+0000 7f41a4bfc700 1 -- 172.21.15.170:0/21341574 wait complete. 2023-10-08T16:11:01.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 Processor -- start 2023-10-08T16:11:01.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 -- start start 2023-10-08T16:11:01.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159890 0x7f41a016da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159c60 0x7f41a0144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a016df60 con 0x7f41a015db10 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a016d010 con 0x7f41a0159c60 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f41a4bfc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0144790 con 0x7f41a0159890 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f419ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f419ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48734/0 (socket says 172.21.15.170:48734) 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f419ed9d700 1 -- 172.21.15.170:0/2309725955 learned_addr learned my addr 172.21.15.170:0/2309725955 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f419e59c700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159890 0x7f41a016da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.753+0000 7f4195d9b700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159c60 0x7f41a0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 -- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159890 msgr2=0x7f41a016da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159890 0x7f41a016da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 -- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159c60 msgr2=0x7f41a0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159c60 0x7f41a0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 -- 172.21.15.170:0/2309725955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188009420 con 0x7f41a015db10 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f419ed9d700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4198001460 tx=0x7f4198001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800a870 con 0x7f41a015db10 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419800a9d0 con 0x7f41a015db10 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800ad50 con 0x7f41a015db10 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4190009420 con 0x7f41a015db10 2023-10-08T16:11:01.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.754+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a016a790 con 0x7f41a015db10 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.755+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a004c3c0 con 0x7f41a015db10 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.755+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4198007300 con 0x7f41a015db10 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.756+0000 7f4196ffd700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 0x7f41800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.756+0000 7f419e59c700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 0x7f41800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.756+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4198084bc0 con 0x7f41a015db10 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.756+0000 7f419e59c700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 0x7f41800697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41900091e0 tx=0x7f4190017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:01.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.760+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4198051c20 con 0x7f41a015db10 2023-10-08T16:11:01.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.941+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41a006a5b0 con 0x7f41a015db10 2023-10-08T16:11:01.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.941+0000 7f4196ffd700 1 -- 172.21.15.170:0/2309725955 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4198055240 con 0x7f41a015db10 2023-10-08T16:11:01.942 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 msgr2=0x7f41800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 0x7f41800697d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f41900091e0 tx=0x7f4190017000 comp rx=0 tx=0).stop 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 msgr2=0x7f41a01699d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4198001460 tx=0x7f4198001a40 comp rx=0 tx=0).stop 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 shutdown_connections 2023-10-08T16:11:01.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4180067320 0x7f41800697d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41a015db10 0x7f41a01699d0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41a0159890 0x7f41a016da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 --2- 172.21.15.170:0/2309725955 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41a0159c60 0x7f41a0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 >> 172.21.15.170:0/2309725955 conn(0x7f41a0068e20 msgr2=0x7f41a0153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.943+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 shutdown_connections 2023-10-08T16:11:01.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:01.944+0000 7f41a4bfc700 1 -- 172.21.15.170:0/2309725955 wait complete. 2023-10-08T16:11:01.956 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:11:02.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.070+0000 7f9c9db88700 1 Processor -- start 2023-10-08T16:11:02.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.070+0000 7f9c9db88700 1 -- start start 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9db88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c980b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9db88700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c980b11d0 con 0x7f9c98159420 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9db88700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c980b14b0 con 0x7f9c9815d230 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9db88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980b1790 con 0x7f9c981597f0 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9cb86700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9c98159420 0x7f9c980b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50568/0 (socket says 172.21.15.170:50568) 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c9cb86700 1 -- 172.21.15.170:0/2917802934 learned_addr learned my addr 172.21.15.170:0/2917802934 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1253542314 0 0) 0x7f9c980b11d0 con 0x7f9c98159420 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c97fff700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c980b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c84003570 con 0x7f9c98159420 2023-10-08T16:11:02.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897668574 0 0) 0x7f9c84003570 con 0x7f9c98159420 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 msgr2=0x7f9c980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c980b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 >> v1:172.21.15.191:6789/0 conn(0x7f9c9815d230 legacy=0x7f9c980b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980b1a70 con 0x7f9c98159420 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f9c90001470 con 0x7f9c98159420 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.071+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 >> v1:172.21.15.170:6789/0 conn(0x7f9c98159420 legacy=0x7f9c980b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c84003a90 0x7f9c84003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84004d50 0x7f9c840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980b1790 con 0x7f9c84003a90 2023-10-08T16:11:02.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8400bb70 con 0x7f9c84004d50 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8400bd30 con 0x7f9c981597f0 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c84003a90 0x7f9c84003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c97fff700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9d387700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84004d50 0x7f9c840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 -- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 msgr2=0x7f9c84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c981597f0 0x7f9c84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 -- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84004d50 msgr2=0x7f9c840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84004d50 0x7f9c840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 -- 172.21.15.170:0/2917802934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84003570 con 0x7f9c84003a90 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c84003a90 0x7f9c84003ea0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c3c0 tx=0x7f9c90001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5422e0aa2b94b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000cc90 con 0x7f9c84003a90 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9000cdf0 con 0x7f9c84003a90 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.072+0000 7f9c977fe700 1 -- 172.21.15.170:0/2917802934 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90005b70 con 0x7f9c84003a90 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 -- 172.21.15.170:0/2917802934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c84003a90 msgr2=0x7f9c84003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 --2- 172.21.15.170:0/2917802934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c84003a90 0x7f9c84003ea0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000c3c0 tx=0x7f9c90001e50 comp rx=0 tx=0).stop 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9d387700 1 -- 172.21.15.170:0/2917802934 reap_dead start 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 -- 172.21.15.170:0/2917802934 shutdown_connections 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 -- 172.21.15.170:0/2917802934 >> 172.21.15.170:0/2917802934 conn(0x7f9c980d0660 msgr2=0x7f9c980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 -- 172.21.15.170:0/2917802934 shutdown_connections 2023-10-08T16:11:02.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.073+0000 7f9c9db88700 1 -- 172.21.15.170:0/2917802934 wait complete. 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.074+0000 7f9c9db88700 1 Processor -- start 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.074+0000 7f9c9db88700 1 -- start start 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 0x7f9c980c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c980c5470 0x7f9c980c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980c61e0 con 0x7f9c98156b70 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980c64c0 con 0x7f9c84003a90 2023-10-08T16:11:02.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9db88700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980c67a0 con 0x7f9c980c5470 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48752/0 (socket says 172.21.15.170:48752) 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 -- 172.21.15.170:0/1101769367 learned_addr learned my addr 172.21.15.170:0/1101769367 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9d387700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c980c5470 0x7f9c980c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 0x7f9c980c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 -- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c980c5470 msgr2=0x7f9c980c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c980c5470 0x7f9c980c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 -- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 msgr2=0x7f9c980c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 0x7f9c980c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 -- 172.21.15.170:0/1101769367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c980c6a80 con 0x7f9c98156b70 2023-10-08T16:11:02.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 0x7f9c980c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c97fff700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9c88011f80 tx=0x7f9c8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.075+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800e040 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.076+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8800f040 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.076+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c98157520 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.076+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88015430 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.076+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c980c7930 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.077+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c980af980 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.077+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9c880157e0 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.077+0000 7f9c94ff9700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 0x7f9c780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.077+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f9c88085710 con 0x7f9c98156b70 2023-10-08T16:11:02.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.077+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 0x7f9c780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.078+0000 7f9c9cb86700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 0x7f9c780697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9c90001e90 tx=0x7f9c90015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.082+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c88056070 con 0x7f9c98156b70 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.266+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9c9804c3c0 con 0x7f9c98156b70 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.266+0000 7f9c94ff9700 1 -- 172.21.15.170:0/1101769367 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9c88055c00 con 0x7f9c98156b70 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:02.267 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:11:02.268 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:11:02.268 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:02.268 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:02.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 msgr2=0x7f9c780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 0x7f9c780697d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9c90001e90 tx=0x7f9c90015000 comp rx=0 tx=0).stop 2023-10-08T16:11:02.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 msgr2=0x7f9c98156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f9c88011f80 tx=0x7f9c8800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 shutdown_connections 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c980c5470 0x7f9c980c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c84003a90 0x7f9c980c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9c78067320 0x7f9c780697d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 --2- 172.21.15.170:0/1101769367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c98156b70 0x7f9c98156fe0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 >> 172.21.15.170:0/1101769367 conn(0x7f9c980d0660 msgr2=0x7f9c980ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 shutdown_connections 2023-10-08T16:11:02.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.269+0000 7f9c9db88700 1 -- 172.21.15.170:0/1101769367 wait complete. 2023-10-08T16:11:02.281 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:11:02.281 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:11:02.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.399+0000 7f46fb943700 1 Processor -- start 2023-10-08T16:11:02.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.400+0000 7f46fb943700 1 -- start start 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.400+0000 7f46fb943700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f46f406b000 0x7f46f40adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.400+0000 7f46fb943700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f406a0b0 con 0x7f46f41564e0 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.400+0000 7f46fb943700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40694a0 con 0x7f46f41568b0 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.400+0000 7f46fb943700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f4069780 con 0x7f46f406b000 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f96df700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f46f41564e0 0x7f46f406a8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50582/0 (socket says 172.21.15.170:50582) 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f96df700 1 -- 172.21.15.170:0/739608268 learned_addr learned my addr 172.21.15.170:0/739608268 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f8ede700 1 --2- 172.21.15.170:0/739608268 >> v2:172.21.15.191:3300/0 conn(0x7f46f406b000 0x7f46f40adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46ebfff700 1 -- 172.21.15.170:0/739608268 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689307659 0 0) 0x7f46f406a0b0 con 0x7f46f41564e0 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46ebfff700 1 -- 172.21.15.170:0/739608268 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e0003570 con 0x7f46f41564e0 2023-10-08T16:11:02.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f8ede700 1 -- 172.21.15.170:0/739608268 >> v1:172.21.15.191:6789/0 conn(0x7f46f41568b0 legacy=0x7f46f406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f8ede700 1 -- 172.21.15.170:0/739608268 >> v1:172.21.15.170:6789/0 conn(0x7f46f41564e0 legacy=0x7f46f406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f8ede700 1 -- 172.21.15.170:0/739608268 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f4069a60 con 0x7f46f406b000 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46f8ede700 1 --2- 172.21.15.170:0/739608268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f406b000 0x7f46f40adfb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f46e40098e0 tx=0x7f46e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05a519671deff36 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46ebfff700 1 -- 172.21.15.170:0/739608268 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400b640 con 0x7f46f406b000 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46ebfff700 1 -- 172.21.15.170:0/739608268 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400b7a0 con 0x7f46f406b000 2023-10-08T16:11:02.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.401+0000 7f46ebfff700 1 -- 172.21.15.170:0/739608268 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400bb40 con 0x7f46f406b000 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 -- 172.21.15.170:0/739608268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f406b000 msgr2=0x7f46f40adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 --2- 172.21.15.170:0/739608268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f406b000 0x7f46f40adfb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f46e40098e0 tx=0x7f46e40091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 -- 172.21.15.170:0/739608268 shutdown_connections 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 --2- 172.21.15.170:0/739608268 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f406b000 0x7f46f40adfb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 -- 172.21.15.170:0/739608268 >> 172.21.15.170:0/739608268 conn(0x7f46f40cd640 msgr2=0x7f46f40cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 -- 172.21.15.170:0/739608268 shutdown_connections 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.402+0000 7f46fb943700 1 -- 172.21.15.170:0/739608268 wait complete. 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.403+0000 7f46fb943700 1 Processor -- start 2023-10-08T16:11:02.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.403+0000 7f46fb943700 1 -- start start 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f41564e0 0x7f46f40ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f41568b0 0x7f46f4158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40b2560 con 0x7f46f406b000 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4157970 con 0x7f46f41564e0 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46fb943700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4157c50 con 0x7f46f41568b0 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48768/0 (socket says 172.21.15.170:48768) 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 -- 172.21.15.170:0/2949428180 learned_addr learned my addr 172.21.15.170:0/2949428180 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f8ede700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f41564e0 0x7f46f40ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f9ee0700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f41568b0 0x7f46f4158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 -- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f41568b0 msgr2=0x7f46f4158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f41568b0 0x7f46f4158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 -- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f41564e0 msgr2=0x7f46f40ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f41564e0 0x7f46f40ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.404+0000 7f46f96df700 1 -- 172.21.15.170:0/2949428180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4009420 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46f96df700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c7c0 tx=0x7f46f0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0005b30 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f0005cb0 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f4157f30 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0007320 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.405+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f4148080 con 0x7f46f406b000 2023-10-08T16:11:02.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.406+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f41577a0 con 0x7f46f406b000 2023-10-08T16:11:02.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.406+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f46f001c440 con 0x7f46f406b000 2023-10-08T16:11:02.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.406+0000 7f46e97fa700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 0x7f46d8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:02.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.406+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f46f0083dd0 con 0x7f46f406b000 2023-10-08T16:11:02.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.406+0000 7f46f8ede700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 0x7f46d8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:02.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.407+0000 7f46f8ede700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 0x7f46d8069800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f46e40093f0 tx=0x7f46e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:02.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.411+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46f0050db0 con 0x7f46f406b000 2023-10-08T16:11:02.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.594+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46f40718b0 con 0x7f46f406b000 2023-10-08T16:11:02.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.891+0000 7f46e97fa700 1 -- 172.21.15.170:0/2949428180 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) v1 ==== 138+0+0 (secure 0 0 0) 0x7f46f0017070 con 0x7f46f406b000 2023-10-08T16:11:02.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.895+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 msgr2=0x7f46d8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.895+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 0x7f46d8069800 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f46e40093f0 tx=0x7f46e4017000 comp rx=0 tx=0).stop 2023-10-08T16:11:02.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.895+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 msgr2=0x7f46f4147980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.895+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c7c0 tx=0x7f46f0007e00 comp rx=0 tx=0).stop 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 shutdown_connections 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f46f41568b0 0x7f46f4158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f46d8067350 0x7f46d8069800 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46f406b000 0x7f46f4147980 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 --2- 172.21.15.170:0/2949428180 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f46f41564e0 0x7f46f40ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.896+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 >> 172.21.15.170:0/2949428180 conn(0x7f46f40cd640 msgr2=0x7f46f406ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:02.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.897+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 shutdown_connections 2023-10-08T16:11:02.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:02.897+0000 7f46fb943700 1 -- 172.21.15.170:0/2949428180 wait complete. 2023-10-08T16:11:02.898 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:11:02.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:11:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.032+0000 7ff4d5f68700 1 Processor -- start 2023-10-08T16:11:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.032+0000 7ff4d5f68700 1 -- start start 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4d5f68700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff4d0158d40 0x7ff4d0140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4d5f68700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff4d006c0a0 0x7ff4d01412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4d5f68700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4d015b790 con 0x7ff4d006bcd0 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4d5f68700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d015ba70 con 0x7ff4d006c0a0 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4d5f68700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d015bd50 con 0x7ff4d0158d40 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff4d006c0a0 0x7ff4d01412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff4d006c0a0 0x7ff4d01412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48776/0 (socket says 172.21.15.170:48776) 2023-10-08T16:11:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 -- 172.21.15.170:0/3248053396 learned_addr learned my addr 172.21.15.170:0/3248053396 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4ceffd700 1 --2- 172.21.15.170:0/3248053396 >> v2:172.21.15.191:3300/0 conn(0x7ff4d0158d40 0x7ff4d0140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 -- 172.21.15.170:0/3248053396 >> v2:172.21.15.191:3300/0 conn(0x7ff4d0158d40 msgr2=0x7ff4d0140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 --2- 172.21.15.170:0/3248053396 >> v2:172.21.15.191:3300/0 conn(0x7ff4d0158d40 0x7ff4d0140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 -- 172.21.15.170:0/3248053396 >> v1:172.21.15.191:6789/0 conn(0x7ff4d006bcd0 legacy=0x7ff4d00c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 -- 172.21.15.170:0/3248053396 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d015c030 con 0x7ff4d006c0a0 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.033+0000 7ff4cffff700 1 --2- 172.21.15.170:0/3248053396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01412f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00cc140 tx=0x7ff4c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eede5f4ad734663 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4ce7fc700 1 -- 172.21.15.170:0/3248053396 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8009e70 con 0x7ff4d006c0a0 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4ce7fc700 1 -- 172.21.15.170:0/3248053396 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c800b860 con 0x7ff4d006c0a0 2023-10-08T16:11:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4ce7fc700 1 -- 172.21.15.170:0/3248053396 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800bc00 con 0x7ff4d006c0a0 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3248053396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006c0a0 msgr2=0x7ff4d01412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3248053396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01412f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00cc140 tx=0x7ff4c8009250 comp rx=0 tx=0).stop 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3248053396 shutdown_connections 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3248053396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01412f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3248053396 >> v2:172.21.15.191:3300/0 conn(0x7ff4d0158d40 0x7ff4d0140db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3248053396 >> 172.21.15.170:0/3248053396 conn(0x7ff4d00ce5c0 msgr2=0x7ff4d00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3248053396 shutdown_connections 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.034+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3248053396 wait complete. 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.035+0000 7ff4d5f68700 1 Processor -- start 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.035+0000 7ff4d5f68700 1 -- start start 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4d006bcd0 0x7ff4d0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01681c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01691a0 con 0x7ff4d0158d40 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d00b2a90 con 0x7ff4d006c0a0 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4d5f68700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d0168be0 con 0x7ff4d006bcd0 2023-10-08T16:11:03.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48778/0 (socket says 172.21.15.170:48778) 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4cffff700 1 -- 172.21.15.170:0/3225551781 learned_addr learned my addr 172.21.15.170:0/3225551781 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4ceffd700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.036+0000 7ff4cf7fe700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4d006bcd0 0x7ff4d0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 -- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4d006bcd0 msgr2=0x7ff4d0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4d006bcd0 0x7ff4d0156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 -- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4d006c0a0 msgr2=0x7ff4d01681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01681c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 -- 172.21.15.170:0/3225551781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c80094f0 con 0x7ff4d0158d40 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4cffff700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009290 tx=0x7ff4c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800b9d0 con 0x7ff4d0158d40 2023-10-08T16:11:03.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c800bb30 con 0x7ff4d0158d40 2023-10-08T16:11:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8025670 con 0x7ff4d0158d40 2023-10-08T16:11:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d0168ec0 con 0x7ff4d0158d40 2023-10-08T16:11:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.037+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d0156bc0 con 0x7ff4d0158d40 2023-10-08T16:11:03.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.038+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4d00718b0 con 0x7ff4d0158d40 2023-10-08T16:11:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.039+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff4c8023070 con 0x7ff4d0158d40 2023-10-08T16:11:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.039+0000 7ff4b77fe700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 0x7ff4bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.039+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5223+0+0 (secure 0 0 0) 0x7ff4c80864c0 con 0x7ff4d0158d40 2023-10-08T16:11:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.039+0000 7ff4cf7fe700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 0x7ff4bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:03.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.040+0000 7ff4cf7fe700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 0x7ff4bc0697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0007fd0 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:03.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.043+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4c80533b0 con 0x7ff4d0158d40 2023-10-08T16:11:03.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.226+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff4d00cf1f0 con 0x7ff4d0158d40 2023-10-08T16:11:03.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.893+0000 7ff4b77fe700 1 -- 172.21.15.170:0/3225551781 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff4c80569d0 con 0x7ff4d0158d40 2023-10-08T16:11:03.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.897+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 msgr2=0x7ff4bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.897+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 0x7ff4bc0697d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff4c0007fd0 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2023-10-08T16:11:03.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.897+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 msgr2=0x7ff4d015a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:03.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.897+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009290 tx=0x7ff4c8017000 comp rx=0 tx=0).stop 2023-10-08T16:11:03.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 shutdown_connections 2023-10-08T16:11:03.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff4bc067320 0x7ff4bc0697d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4d0158d40 0x7ff4d015a110 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff4d006bcd0 0x7ff4d0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 --2- 172.21.15.170:0/3225551781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff4d006c0a0 0x7ff4d01681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:03.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.899+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 >> 172.21.15.170:0/3225551781 conn(0x7ff4d00ce5c0 msgr2=0x7ff4d0158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:03.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.901+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 shutdown_connections 2023-10-08T16:11:03.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:03.901+0000 7ff4d5f68700 1 -- 172.21.15.170:0/3225551781 wait complete. 2023-10-08T16:11:03.906 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:11:03.921 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:11:04.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.039+0000 7f36c13b1700 1 Processor -- start 2023-10-08T16:11:04.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.039+0000 7f36c13b1700 1 -- start start 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36c13b1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f36bc159890 0x7f36bc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36c13b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc069560 0x7f36bc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36c13b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc069ee0 con 0x7f36bc159c60 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36c13b1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc15a160 con 0x7f36bc159890 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36c13b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0d13a0 con 0x7f36bc069560 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36baffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f36bc159890 0x7f36bc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36baffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f36bc159890 0x7f36bc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48784/0 (socket says 172.21.15.170:48784) 2023-10-08T16:11:04.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36baffd700 1 -- 172.21.15.170:0/1583823924 learned_addr learned my addr 172.21.15.170:0/1583823924 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36baffd700 1 -- 172.21.15.170:0/1583823924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc069560 msgr2=0x7f36bc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.040+0000 7f36bb7fe700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc069560 0x7f36bc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36baffd700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc069560 0x7f36bc0699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36baffd700 1 -- 172.21.15.170:0/1583823924 >> v1:172.21.15.191:6789/0 conn(0x7f36bc159c60 legacy=0x7f36bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36baffd700 1 -- 172.21.15.170:0/1583823924 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc15e140 con 0x7f36bc159890 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36baffd700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc159890 0x7f36bc15abd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0099b0 tx=0x7f36ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30d6ef381ab6154 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36ba7fc700 1 -- 172.21.15.170:0/1583823924 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00b820 con 0x7f36bc159890 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36ba7fc700 1 -- 172.21.15.170:0/1583823924 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac00b980 con 0x7f36bc159890 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36ba7fc700 1 -- 172.21.15.170:0/1583823924 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00bd20 con 0x7f36bc159890 2023-10-08T16:11:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 -- 172.21.15.170:0/1583823924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc159890 msgr2=0x7f36bc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc159890 0x7f36bc15abd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0099b0 tx=0x7f36ac009250 comp rx=0 tx=0).stop 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 -- 172.21.15.170:0/1583823924 shutdown_connections 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc069560 0x7f36bc0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 --2- 172.21.15.170:0/1583823924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc159890 0x7f36bc15abd0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 -- 172.21.15.170:0/1583823924 >> 172.21.15.170:0/1583823924 conn(0x7f36bc068e20 msgr2=0x7f36bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 -- 172.21.15.170:0/1583823924 shutdown_connections 2023-10-08T16:11:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.041+0000 7f36c13b1700 1 -- 172.21.15.170:0/1583823924 wait complete. 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 Processor -- start 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 -- start start 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc159890 0x7f36bc0cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 0x7f36bc0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0c9250 con 0x7f36bc069560 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0b58a0 con 0x7f36bc159c60 2023-10-08T16:11:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36c13b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0b6270 con 0x7f36bc159890 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36baffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36baffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48792/0 (socket says 172.21.15.170:48792) 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.043+0000 7f36baffd700 1 -- 172.21.15.170:0/225452647 learned_addr learned my addr 172.21.15.170:0/225452647 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36b27fc700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc159890 0x7f36bc0cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36bb7fe700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 0x7f36bc0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 -- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc159890 msgr2=0x7f36bc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc159890 0x7f36bc0cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 -- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 msgr2=0x7f36bc0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 0x7f36bc0ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 -- 172.21.15.170:0/225452647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0094f0 con 0x7f36bc069560 2023-10-08T16:11:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36bb7fe700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 0x7f36bc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36baffd700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0899c0 tx=0x7f36ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac011c80 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ac020070 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc0b4c90 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac01b450 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.044+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc0b5250 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.045+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc0cab70 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.046+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f36ac022030 con 0x7f36bc069560 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.046+0000 7f36b37fe700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 0x7f369c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:04.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.046+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f36ac086a00 con 0x7f36bc069560 2023-10-08T16:11:04.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.046+0000 7f36b27fc700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 0x7f369c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:04.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.046+0000 7f36b27fc700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 0x7f369c0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36bc16d590 tx=0x7f36a40090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:04.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.050+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36ac053800 con 0x7f36bc069560 2023-10-08T16:11:04.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.274+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f36bc0cab70 con 0x7f36bc069560 2023-10-08T16:11:04.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.916+0000 7f36b37fe700 1 -- 172.21.15.170:0/225452647 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f36ac056e20 con 0x7f36bc069560 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.925+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 msgr2=0x7f369c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.925+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 0x7f369c0697d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f36bc16d590 tx=0x7f36a40090f0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.925+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 msgr2=0x7f36bc0b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.925+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0899c0 tx=0x7f36ac016000 comp rx=0 tx=0).stop 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 shutdown_connections 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36bc159c60 0x7f36bc0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f369c067320 0x7f369c0697d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36bc069560 0x7f36bc0b5d30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 --2- 172.21.15.170:0/225452647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36bc159890 0x7f36bc0cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:04.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 >> 172.21.15.170:0/225452647 conn(0x7f36bc068e20 msgr2=0x7f36bc0b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:04.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 shutdown_connections 2023-10-08T16:11:04.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:04.926+0000 7f36c13b1700 1 -- 172.21.15.170:0/225452647 wait complete. 2023-10-08T16:11:04.929 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 18 and data pool 19 2023-10-08T16:11:04.942 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:05.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.056+0000 7f2442528700 1 Processor -- start 2023-10-08T16:11:05.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.056+0000 7f2442528700 1 -- start start 2023-10-08T16:11:05.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f2442528700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f243c159c10 0x7f243c15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f2442528700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c15c480 con 0x7f243c0b5ea0 2023-10-08T16:11:05.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f2442528700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c15b5d0 con 0x7f243c15c910 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f2442528700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c0b2a40 con 0x7f243c159c10 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f243bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f243c159c10 0x7f243c15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f243b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f243c0b5ea0 0x7f243c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:50594/0 (socket says 172.21.15.170:50594) 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.057+0000 7f243b7fe700 1 -- 172.21.15.170:0/2275982017 learned_addr learned my addr 172.21.15.170:0/2275982017 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243bfff700 1 -- 172.21.15.170:0/2275982017 >> v1:172.21.15.191:6789/0 conn(0x7f243c15c910 legacy=0x7f243c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243bfff700 1 -- 172.21.15.170:0/2275982017 >> v1:172.21.15.170:6789/0 conn(0x7f243c0b5ea0 legacy=0x7f243c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243bfff700 1 -- 172.21.15.170:0/2275982017 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c15b870 con 0x7f243c159c10 2023-10-08T16:11:05.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243affd700 1 -- 172.21.15.170:0/2275982017 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503827326 0 0) 0x7f243c15c480 con 0x7f243c0b5ea0 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243bfff700 1 --2- 172.21.15.170:0/2275982017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c15b090 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f242c0098e0 tx=0x7f242c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a046c20889c1218d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243affd700 1 -- 172.21.15.170:0/2275982017 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c009e80 con 0x7f243c159c10 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243affd700 1 -- 172.21.15.170:0/2275982017 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c00b860 con 0x7f243c159c10 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f243affd700 1 -- 172.21.15.170:0/2275982017 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c00bbe0 con 0x7f243c159c10 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f2442528700 1 -- 172.21.15.170:0/2275982017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 msgr2=0x7f243c15b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f2442528700 1 --2- 172.21.15.170:0/2275982017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c15b090 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f242c0098e0 tx=0x7f242c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f2442528700 1 -- 172.21.15.170:0/2275982017 shutdown_connections 2023-10-08T16:11:05.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.058+0000 7f2442528700 1 --2- 172.21.15.170:0/2275982017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c15b090 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.059+0000 7f2442528700 1 -- 172.21.15.170:0/2275982017 >> 172.21.15.170:0/2275982017 conn(0x7f243c099a20 msgr2=0x7f243c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:05.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.059+0000 7f2442528700 1 -- 172.21.15.170:0/2275982017 shutdown_connections 2023-10-08T16:11:05.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.059+0000 7f2442528700 1 -- 172.21.15.170:0/2275982017 wait complete. 2023-10-08T16:11:05.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.060+0000 7f2442528700 1 Processor -- start 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.060+0000 7f2442528700 1 -- start start 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243c0b5ea0 0x7f243c16a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 0x7f243c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0ad8d0 con 0x7f243c159c10 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0adb00 con 0x7f243c15c910 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0adde0 con 0x7f243c0b5ea0 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48818/0 (socket says 172.21.15.170:48818) 2023-10-08T16:11:05.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 -- 172.21.15.170:0/3767064168 learned_addr learned my addr 172.21.15.170:0/3767064168 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243bfff700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243c0b5ea0 0x7f243c16a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2440ac5700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 0x7f243c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 -- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243c0b5ea0 msgr2=0x7f243c16a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243c0b5ea0 0x7f243c16a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 -- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 msgr2=0x7f243c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 0x7f243c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 -- 172.21.15.170:0/3767064168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c009420 con 0x7f243c159c10 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2440ac5700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 0x7f243c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:05.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f243b7fe700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f243000c7e0 tx=0x7f2430007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430005b30 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430005c90 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c0ae0c0 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430007300 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.061+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c0ae680 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.062+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c0ae870 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.063+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f24300076b0 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.063+0000 7f2423fff700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 0x7f24280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.063+0000 7f243bfff700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 0x7f24280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.063+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f24300841e0 con 0x7f243c159c10 2023-10-08T16:11:05.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.064+0000 7f243bfff700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 0x7f24280697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f242c0093f0 tx=0x7f242c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.067+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2430050e70 con 0x7f243c159c10 2023-10-08T16:11:05.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.251+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f243c04c3c0 con 0x7f243c159c10 2023-10-08T16:11:05.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.252+0000 7f2423fff700 1 -- 172.21.15.170:0/3767064168 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f2430054490 con 0x7f243c159c10 2023-10-08T16:11:05.252 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:05.253 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:04.275512+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 msgr2=0x7f24280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 0x7f24280697d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f242c0093f0 tx=0x7f242c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 msgr2=0x7f243c0ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f243000c7e0 tx=0x7f2430007df0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 shutdown_connections 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f243c15c910 0x7f243c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f243c0b5ea0 0x7f243c16a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2428067320 0x7f24280697d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 --2- 172.21.15.170:0/3767064168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f243c159c10 0x7f243c0ad390 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 >> 172.21.15.170:0/3767064168 conn(0x7f243c099a20 msgr2=0x7f243c158f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:05.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 shutdown_connections 2023-10-08T16:11:05.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.254+0000 7f2442528700 1 -- 172.21.15.170:0/3767064168 wait complete. 2023-10-08T16:11:05.267 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:05.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.381+0000 7fe4dfe35700 1 Processor -- start 2023-10-08T16:11:05.382 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.381+0000 7fe4dfe35700 1 -- start start 2023-10-08T16:11:05.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4dfe35700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4dfe35700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe4d815db10 0x7fe4d8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4dfe35700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d815a160 con 0x7fe4d8159890 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4dfe35700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d815df50 con 0x7fe4d815db10 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4dfe35700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d80d13a0 con 0x7fe4d8159c60 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4de3d2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe4d815db10 0x7fe4d8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4ddbd1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe4d8159890 0x7fe4d815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59122/0 (socket says 172.21.15.170:59122) 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.382+0000 7fe4ddbd1700 1 -- 172.21.15.170:0/1470783828 learned_addr learned my addr 172.21.15.170:0/1470783828 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dd3d0700 1 --2- 172.21.15.170:0/1470783828 >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4de3d2700 1 -- 172.21.15.170:0/1470783828 >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 msgr2=0x7fe4d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4de3d2700 1 --2- 172.21.15.170:0/1470783828 >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4de3d2700 1 -- 172.21.15.170:0/1470783828 >> v1:172.21.15.170:6789/0 conn(0x7fe4d8159890 legacy=0x7fe4d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dcbcf700 1 -- 172.21.15.170:0/1470783828 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1896877121 0 0) 0x7fe4d815a160 con 0x7fe4d8159890 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4de3d2700 1 -- 172.21.15.170:0/1470783828 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d815e140 con 0x7fe4d815db10 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dd3d0700 1 --2- 172.21.15.170:0/1470783828 >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4de3d2700 1 --2- 172.21.15.170:0/1470783828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d815db10 0x7fe4d8069560 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b67f0 tx=0x7fe4d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d2f6d23b4a7b4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dcbcf700 1 -- 172.21.15.170:0/1470783828 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000b6b0 con 0x7fe4d815db10 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dcbcf700 1 -- 172.21.15.170:0/1470783828 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000b810 con 0x7fe4d815db10 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dcbcf700 1 -- 172.21.15.170:0/1470783828 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000bbb0 con 0x7fe4d815db10 2023-10-08T16:11:05.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dfe35700 1 -- 172.21.15.170:0/1470783828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d815db10 msgr2=0x7fe4d8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.383+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/1470783828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d815db10 0x7fe4d8069560 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b67f0 tx=0x7fe4d00091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 -- 172.21.15.170:0/1470783828 shutdown_connections 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/1470783828 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d815db10 0x7fe4d8069560 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/1470783828 >> v2:172.21.15.191:3300/0 conn(0x7fe4d8159c60 0x7fe4d815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 -- 172.21.15.170:0/1470783828 >> 172.21.15.170:0/1470783828 conn(0x7fe4d8068e20 msgr2=0x7fe4d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 -- 172.21.15.170:0/1470783828 shutdown_connections 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.384+0000 7fe4dfe35700 1 -- 172.21.15.170:0/1470783828 wait complete. 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.385+0000 7fe4dfe35700 1 Processor -- start 2023-10-08T16:11:05.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.385+0000 7fe4dfe35700 1 -- start start 2023-10-08T16:11:05.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4d8159c60 0x7fe4d80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4d815db10 0x7fe4d8151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80cd8e0 con 0x7fe4d8159890 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80cff00 con 0x7fe4d815db10 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80cf630 con 0x7fe4d8159c60 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44580/0 (socket says 172.21.15.170:44580) 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 -- 172.21.15.170:0/3883768384 learned_addr learned my addr 172.21.15.170:0/3883768384 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 -- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4d8159c60 msgr2=0x7fe4d80b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4d8159c60 0x7fe4d80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 -- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4d815db10 msgr2=0x7fe4d8151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4d815db10 0x7fe4d8151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 -- 172.21.15.170:0/3883768384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009420 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4ddbd1700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400cde0 tx=0x7fe4d4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d400ab60 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d400acc0 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d80b2800 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.386+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4007300 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.387+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d80b1be0 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.387+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d804c3c0 con 0x7fe4d8159890 2023-10-08T16:11:05.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.388+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe4d40076f0 con 0x7fe4d8159890 2023-10-08T16:11:05.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.388+0000 7fe4cdffb700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 0x7fe4bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.388+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4d4085240 con 0x7fe4d8159890 2023-10-08T16:11:05.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.388+0000 7fe4dd3d0700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 0x7fe4bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.389+0000 7fe4dd3d0700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 0x7fe4bc0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006fd0 tx=0x7fe4c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.392+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4d4051f50 con 0x7fe4d8159890 2023-10-08T16:11:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.574+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d806a5b0 con 0x7fe4d8159890 2023-10-08T16:11:05.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.575+0000 7fe4cdffb700 1 -- 172.21.15.170:0/3883768384 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fe4d4055570 con 0x7fe4d8159890 2023-10-08T16:11:05.576 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:05.576 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:04.275512+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:05.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 msgr2=0x7fe4bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 0x7fe4bc0697d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8006fd0 tx=0x7fe4c8008040 comp rx=0 tx=0).stop 2023-10-08T16:11:05.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 msgr2=0x7fe4d80cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.578 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400cde0 tx=0x7fe4d4001ae0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 shutdown_connections 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe4d815db10 0x7fe4d8151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe4bc067320 0x7fe4bc0697d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4d8159890 0x7fe4d80cd3a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 --2- 172.21.15.170:0/3883768384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe4d8159c60 0x7fe4d80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 >> 172.21.15.170:0/3883768384 conn(0x7fe4d8068e20 msgr2=0x7fe4d8149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 shutdown_connections 2023-10-08T16:11:05.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.578+0000 7fe4dfe35700 1 -- 172.21.15.170:0/3883768384 wait complete. 2023-10-08T16:11:05.590 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:11:05.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.708+0000 7f83e0a34700 1 Processor -- start 2023-10-08T16:11:05.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.708+0000 7f83e0a34700 1 -- start start 2023-10-08T16:11:05.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83e0a34700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f83dc141500 0x7f83dc0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83e0a34700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f83dc06c0a0 0x7f83dc0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83e0a34700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc15c2a0 con 0x7f83dc06bcd0 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83e0a34700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc0c9880 con 0x7f83dc06c0a0 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83e0a34700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc15b690 con 0x7f83dc141500 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f83dc06c0a0 0x7f83dc0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f83dc06c0a0 0x7f83dc0c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44588/0 (socket says 172.21.15.170:44588) 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 -- 172.21.15.170:0/1250038052 learned_addr learned my addr 172.21.15.170:0/1250038052 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83d9d9b700 1 --2- 172.21.15.170:0/1250038052 >> v2:172.21.15.191:3300/0 conn(0x7f83dc141500 0x7f83dc0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 -- 172.21.15.170:0/1250038052 >> v2:172.21.15.191:3300/0 conn(0x7f83dc141500 msgr2=0x7f83dc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:05.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 --2- 172.21.15.170:0/1250038052 >> v2:172.21.15.191:3300/0 conn(0x7f83dc141500 0x7f83dc0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 -- 172.21.15.170:0/1250038052 >> v1:172.21.15.191:6789/0 conn(0x7f83dc06bcd0 legacy=0x7f83dc140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 -- 172.21.15.170:0/1250038052 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc15b970 con 0x7f83dc06c0a0 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.709+0000 7f83dad9d700 1 --2- 172.21.15.170:0/1250038052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0c9340 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f83dc06c6e0 tx=0x7f83d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748fd17249c1e268 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83d959a700 1 -- 172.21.15.170:0/1250038052 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d400e070 con 0x7f83dc06c0a0 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83d959a700 1 -- 172.21.15.170:0/1250038052 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d400ba50 con 0x7f83dc06c0a0 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83d959a700 1 -- 172.21.15.170:0/1250038052 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d40092e0 con 0x7f83dc06c0a0 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 -- 172.21.15.170:0/1250038052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 msgr2=0x7f83dc0c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 --2- 172.21.15.170:0/1250038052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0c9340 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f83dc06c6e0 tx=0x7f83d4009840 comp rx=0 tx=0).stop 2023-10-08T16:11:05.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 -- 172.21.15.170:0/1250038052 shutdown_connections 2023-10-08T16:11:05.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 --2- 172.21.15.170:0/1250038052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0c9340 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 --2- 172.21.15.170:0/1250038052 >> v2:172.21.15.191:3300/0 conn(0x7f83dc141500 0x7f83dc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 -- 172.21.15.170:0/1250038052 >> 172.21.15.170:0/1250038052 conn(0x7f83dc0ce600 msgr2=0x7f83dc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:05.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 -- 172.21.15.170:0/1250038052 shutdown_connections 2023-10-08T16:11:05.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.710+0000 7f83e0a34700 1 -- 172.21.15.170:0/1250038052 wait complete. 2023-10-08T16:11:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.711+0000 7f83e0a34700 1 Processor -- start 2023-10-08T16:11:05.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.711+0000 7f83e0a34700 1 -- start start 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc06bcd0 0x7f83dc13ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc141500 0x7f83dc15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b4050 con 0x7f83dc06c0a0 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b3440 con 0x7f83dc141500 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83e0a34700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b3720 con 0x7f83dc06bcd0 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44600/0 (socket says 172.21.15.170:44600) 2023-10-08T16:11:05.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 -- 172.21.15.170:0/985774861 learned_addr learned my addr 172.21.15.170:0/985774861 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83da59c700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc06bcd0 0x7f83dc13ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83dad9d700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc141500 0x7f83dc15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 -- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc06bcd0 msgr2=0x7f83dc13ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc06bcd0 0x7f83dc13ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 -- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc141500 msgr2=0x7f83dc15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc141500 0x7f83dc15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.712+0000 7f83d9d9b700 1 -- 172.21.15.170:0/985774861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4009ae0 con 0x7f83dc06c0a0 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83d9d9b700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f83d000d860 tx=0x7f83d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d00125f0 con 0x7f83dc06c0a0 2023-10-08T16:11:05.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d0012750 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc0b3a00 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0012b10 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.713+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc0b30c0 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.714+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc04c3c0 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.714+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f83d001e460 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.715+0000 7f83ca7fc700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 0x7f83bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.715+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83d0085170 con 0x7f83dc06c0a0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.715+0000 7f83da59c700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 0x7f83bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:05.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.715+0000 7f83da59c700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 0x7f83bc0697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0c53b0 tx=0x7f83cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:05.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.719+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83d0055890 con 0x7f83dc06c0a0 2023-10-08T16:11:05.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:05.947+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f83dc15bc50 con 0x7f83dc06c0a0 2023-10-08T16:11:06.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.928+0000 7f83ca7fc700 1 -- 172.21.15.170:0/985774861 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83d0055420 con 0x7f83dc06c0a0 2023-10-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 msgr2=0x7f83bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:06.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 0x7f83bc0697d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0c53b0 tx=0x7f83cc009040 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 msgr2=0x7f83dc0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f83d000d860 tx=0x7f83d0008040 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 shutdown_connections 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc141500 0x7f83dc15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc06bcd0 0x7f83dc13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83bc067320 0x7f83bc0697d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 --2- 172.21.15.170:0/985774861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc06c0a0 0x7f83dc0b2b80 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 >> 172.21.15.170:0/985774861 conn(0x7f83dc0ce600 msgr2=0x7f83dc0ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 shutdown_connections 2023-10-08T16:11:06.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:06.931+0000 7f83e0a34700 1 -- 172.21.15.170:0/985774861 wait complete. 2023-10-08T16:11:06.944 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.063+0000 7f4a1b53b700 1 Processor -- start 2023-10-08T16:11:07.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.063+0000 7f4a1b53b700 1 -- start start 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a1b53b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a14159890 0x7f4a1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a1b53b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a14159c60 0x7f4a1415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a1b53b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1415a160 con 0x7f4a1415db10 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a1b53b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1415df50 con 0x7f4a14159c60 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a1b53b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a140d13a0 con 0x7f4a14159890 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a14159c60 0x7f4a1415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4a14159c60 0x7f4a1415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44604/0 (socket says 172.21.15.170:44604) 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 -- 172.21.15.170:0/1015560631 learned_addr learned my addr 172.21.15.170:0/1015560631 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:07.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a192d7700 1 --2- 172.21.15.170:0/1015560631 >> v2:172.21.15.191:3300/0 conn(0x7f4a14159890 0x7f4a1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 -- 172.21.15.170:0/1015560631 >> v2:172.21.15.191:3300/0 conn(0x7f4a14159890 msgr2=0x7f4a1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 --2- 172.21.15.170:0/1015560631 >> v2:172.21.15.191:3300/0 conn(0x7f4a14159890 0x7f4a1415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.064+0000 7f4a18ad6700 1 -- 172.21.15.170:0/1015560631 >> v1:172.21.15.191:6789/0 conn(0x7f4a1415db10 legacy=0x7f4a14069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a18ad6700 1 -- 172.21.15.170:0/1015560631 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1415e140 con 0x7f4a14159c60 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a18ad6700 1 --2- 172.21.15.170:0/1015560631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a1415d5d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4a140d1090 tx=0x7f4a04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84049b6684dcb217 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a0bfff700 1 -- 172.21.15.170:0/1015560631 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400b860 con 0x7f4a14159c60 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a0bfff700 1 -- 172.21.15.170:0/1015560631 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0400b9c0 con 0x7f4a14159c60 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a0bfff700 1 -- 172.21.15.170:0/1015560631 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400bd60 con 0x7f4a14159c60 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 -- 172.21.15.170:0/1015560631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 msgr2=0x7f4a1415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/1015560631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a1415d5d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4a140d1090 tx=0x7f4a04009250 comp rx=0 tx=0).stop 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 -- 172.21.15.170:0/1015560631 shutdown_connections 2023-10-08T16:11:07.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/1015560631 >> v2:172.21.15.191:3300/0 conn(0x7f4a14159890 0x7f4a1415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/1015560631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a1415d5d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 -- 172.21.15.170:0/1015560631 >> 172.21.15.170:0/1015560631 conn(0x7f4a14068e20 msgr2=0x7f4a140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 -- 172.21.15.170:0/1015560631 shutdown_connections 2023-10-08T16:11:07.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.065+0000 7f4a1b53b700 1 -- 172.21.15.170:0/1015560631 wait complete. 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 Processor -- start 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 -- start start 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 0x7f4a141584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1415db10 0x7f4a140b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140cea90 con 0x7f4a14159c60 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a141589e0 con 0x7f4a1415db10 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.067+0000 7f4a1b53b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140cff20 con 0x7f4a14159890 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44610/0 (socket says 172.21.15.170:44610) 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 -- 172.21.15.170:0/465728514 learned_addr learned my addr 172.21.15.170:0/465728514 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a192d7700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 0x7f4a141584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a19ad8700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1415db10 0x7f4a140b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 -- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 msgr2=0x7f4a141584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 0x7f4a141584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 -- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1415db10 msgr2=0x7f4a140b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1415db10 0x7f4a140b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 -- 172.21.15.170:0/465728514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040094f0 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a192d7700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 0x7f4a141584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a18ad6700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4a14070e20 tx=0x7f4a04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04011e60 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0401a630 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0401a9f0 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a140b52c0 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.068+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a140b47a0 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.069+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1404c3c0 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.069+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4a0402d440 con 0x7f4a14159c60 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.070+0000 7f4a097fa700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:07.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.070+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a04086820 con 0x7f4a14159c60 2023-10-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.070+0000 7f4a192d7700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:07.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.070+0000 7f4a192d7700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 0x7f49f80697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4a10006fd0 tx=0x7f4a10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:07.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.074+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a040534b0 con 0x7f4a14159c60 2023-10-08T16:11:07.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.300+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4a1406a5b0 con 0x7f4a14159c60 2023-10-08T16:11:07.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.930+0000 7f4a097fa700 1 -- 172.21.15.170:0/465728514 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a04056ad0 con 0x7f4a14159c60 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 msgr2=0x7f49f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 0x7f49f80697d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4a10006fd0 tx=0x7f4a10008040 comp rx=0 tx=0).stop 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 msgr2=0x7f4a140cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4a14070e20 tx=0x7f4a04017000 comp rx=0 tx=0).stop 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 shutdown_connections 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1415db10 0x7f4a140b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4a14159890 0x7f4a141584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f8067320 0x7f49f80697d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 --2- 172.21.15.170:0/465728514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a14159c60 0x7f4a140cf9e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:07.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 >> 172.21.15.170:0/465728514 conn(0x7f4a14068e20 msgr2=0x7f4a1415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:07.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 shutdown_connections 2023-10-08T16:11:07.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:07.932+0000 7f4a1b53b700 1 -- 172.21.15.170:0/465728514 wait complete. 2023-10-08T16:11:07.946 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:08.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.060+0000 7fe522917700 1 Processor -- start 2023-10-08T16:11:08.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.060+0000 7fe522917700 1 -- start start 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522917700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe51c15d230 0x7fe51c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522917700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0b2980 con 0x7fe51c1597f0 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522917700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c15db70 con 0x7fe51c159420 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522917700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0b11d0 con 0x7fe51c15d230 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522116700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe51c15d230 0x7fe51c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe521114700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe51c1597f0 0x7fe51c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59132/0 (socket says 172.21.15.170:59132) 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe521114700 1 -- 172.21.15.170:0/390878713 learned_addr learned my addr 172.21.15.170:0/390878713 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.061+0000 7fe522116700 1 -- 172.21.15.170:0/390878713 >> v1:172.21.15.191:6789/0 conn(0x7fe51c159420 legacy=0x7fe51c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522116700 1 -- 172.21.15.170:0/390878713 >> v1:172.21.15.170:6789/0 conn(0x7fe51c1597f0 legacy=0x7fe51c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522116700 1 -- 172.21.15.170:0/390878713 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0b14b0 con 0x7fe51c15d230 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe520913700 1 -- 172.21.15.170:0/390878713 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409815563 0 0) 0x7fe51c0b2980 con 0x7fe51c1597f0 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522116700 1 --2- 172.21.15.170:0/390878713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c15d630 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098e0 tx=0x7fe5140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c2523f484297b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe520913700 1 -- 172.21.15.170:0/390878713 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe514009e80 con 0x7fe51c15d230 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe520913700 1 -- 172.21.15.170:0/390878713 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51400b820 con 0x7fe51c15d230 2023-10-08T16:11:08.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe520913700 1 -- 172.21.15.170:0/390878713 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51400bba0 con 0x7fe51c15d230 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 -- 172.21.15.170:0/390878713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 msgr2=0x7fe51c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 --2- 172.21.15.170:0/390878713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c15d630 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe5140098e0 tx=0x7fe5140091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 -- 172.21.15.170:0/390878713 shutdown_connections 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 --2- 172.21.15.170:0/390878713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c15d630 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 -- 172.21.15.170:0/390878713 >> 172.21.15.170:0/390878713 conn(0x7fe51c0d0660 msgr2=0x7fe51c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.062+0000 7fe522917700 1 -- 172.21.15.170:0/390878713 shutdown_connections 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.063+0000 7fe522917700 1 -- 172.21.15.170:0/390878713 wait complete. 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 Processor -- start 2023-10-08T16:11:08.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 -- start start 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c159420 0x7fe51c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 0x7fe51c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c15bdb0 con 0x7fe51c15d230 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c15c050 con 0x7fe51c1597f0 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522917700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c15c330 con 0x7fe51c159420 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522116700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522116700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44630/0 (socket says 172.21.15.170:44630) 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe522116700 1 -- 172.21.15.170:0/3406569782 learned_addr learned my addr 172.21.15.170:0/3406569782 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe521915700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c159420 0x7fe51c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.064+0000 7fe521114700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 0x7fe51c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 -- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c159420 msgr2=0x7fe51c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c159420 0x7fe51c0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 -- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 msgr2=0x7fe51c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 0x7fe51c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 -- 172.21.15.170:0/3406569782 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe514009420 con 0x7fe51c15d230 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe521114700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 0x7fe51c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522116700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe5140093f0 tx=0x7fe514016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51401b450 con 0x7fe51c15d230 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe514020070 con 0x7fe51c15d230 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c15c610 con 0x7fe51c15d230 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51401b800 con 0x7fe51c15d230 2023-10-08T16:11:08.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.065+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c169be0 con 0x7fe51c15d230 2023-10-08T16:11:08.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.066+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c04c3c0 con 0x7fe51c15d230 2023-10-08T16:11:08.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.067+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe514022030 con 0x7fe51c15d230 2023-10-08T16:11:08.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.067+0000 7fe511ffb700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 0x7fe5000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.067+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe514086c10 con 0x7fe51c15d230 2023-10-08T16:11:08.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.067+0000 7fe521915700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 0x7fe5000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.067+0000 7fe521915700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 0x7fe5000697d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe518007fd0 tx=0x7fe5180026a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.071+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5140539b0 con 0x7fe51c15d230 2023-10-08T16:11:08.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.295+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe51c04c3c0 con 0x7fe51c15d230 2023-10-08T16:11:08.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.296+0000 7fe511ffb700 1 -- 172.21.15.170:0/3406569782 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4599 (secure 0 0 0) 0x7fe514056fd0 con 0x7fe51c15d230 2023-10-08T16:11:08.296 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:08.296 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3558382637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3558382637},{"type":"v1","addr":"172.21.15.170:6836","nonce":3558382637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/707523029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":707523029},{"type":"v1","addr":"172.21.15.170:6837","nonce":707523029}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 msgr2=0x7fe5000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 0x7fe5000697d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe518007fd0 tx=0x7fe5180026a0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 msgr2=0x7fe51c154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe5140093f0 tx=0x7fe514016000 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 shutdown_connections 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe500067320 0x7fe5000697d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe51c15d230 0x7fe51c154d70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe51c159420 0x7fe51c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 --2- 172.21.15.170:0/3406569782 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe51c1597f0 0x7fe51c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 >> 172.21.15.170:0/3406569782 conn(0x7fe51c0d0660 msgr2=0x7fe51c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 shutdown_connections 2023-10-08T16:11:08.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.298+0000 7fe522917700 1 -- 172.21.15.170:0/3406569782 wait complete. 2023-10-08T16:11:08.300 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 74 2023-10-08T16:11:08.311 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:08.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.424+0000 7f304f149700 1 Processor -- start 2023-10-08T16:11:08.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.424+0000 7f304f149700 1 -- start start 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304f149700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f30481597f0 0x7f30480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304f149700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f304815d230 0x7f304815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304f149700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304815dbb0 con 0x7f3048159420 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304f149700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480b11d0 con 0x7f30481597f0 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304f149700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480b14b0 con 0x7f304815d230 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d946700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f30481597f0 0x7f30480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304e147700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3048159420 0x7f30480b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59146/0 (socket says 172.21.15.170:59146) 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304e147700 1 -- 172.21.15.170:0/3989160763 learned_addr learned my addr 172.21.15.170:0/3989160763 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:08.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304e948700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f304815d230 0x7f304815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d946700 1 -- 172.21.15.170:0/3989160763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f304815d230 msgr2=0x7f304815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d946700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f304815d230 0x7f304815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d946700 1 -- 172.21.15.170:0/3989160763 >> v1:172.21.15.170:6789/0 conn(0x7f3048159420 legacy=0x7f30480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d946700 1 -- 172.21.15.170:0/3989160763 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480b1790 con 0x7f30481597f0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.425+0000 7f304d145700 1 -- 172.21.15.170:0/3989160763 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958511077 0 0) 0x7f304815dbb0 con 0x7f3048159420 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304d946700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30481597f0 0x7f30480b2270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098e0 tx=0x7f303c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd3053afaae0a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304d145700 1 -- 172.21.15.170:0/3989160763 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c009e80 con 0x7f30481597f0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304d145700 1 -- 172.21.15.170:0/3989160763 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c00b860 con 0x7f30481597f0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304d145700 1 -- 172.21.15.170:0/3989160763 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00bbe0 con 0x7f30481597f0 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 -- 172.21.15.170:0/3989160763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30481597f0 msgr2=0x7f30480b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30481597f0 0x7f30480b2270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f303c0098e0 tx=0x7f303c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.428 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 -- 172.21.15.170:0/3989160763 shutdown_connections 2023-10-08T16:11:08.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f304815d230 0x7f304815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 --2- 172.21.15.170:0/3989160763 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f30481597f0 0x7f30480b2270 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 -- 172.21.15.170:0/3989160763 >> 172.21.15.170:0/3989160763 conn(0x7f30480d0660 msgr2=0x7f30480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 -- 172.21.15.170:0/3989160763 shutdown_connections 2023-10-08T16:11:08.429 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.426+0000 7f304f149700 1 -- 172.21.15.170:0/3989160763 wait complete. 2023-10-08T16:11:08.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.427+0000 7f304f149700 1 Processor -- start 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.427+0000 7f304f149700 1 -- start start 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 0x7f30480c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f304815d230 0x7f3048154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480c7070 con 0x7f3048159420 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304815ac70 con 0x7f304815d230 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304f149700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304815b230 con 0x7f30481597f0 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44648/0 (socket says 172.21.15.170:44648) 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 -- 172.21.15.170:0/3322448663 learned_addr learned my addr 172.21.15.170:0/3322448663 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304d946700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 0x7f30480c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e948700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f304815d230 0x7f3048154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 -- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 msgr2=0x7f30480c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 0x7f30480c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 -- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f304815d230 msgr2=0x7f3048154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f304815d230 0x7f3048154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304e147700 1 -- 172.21.15.170:0/3322448663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c009420 con 0x7f3048159420 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.428+0000 7f304d946700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 0x7f30480c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f304e147700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f303800cd80 tx=0x7f3038001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800ab10 con 0x7f3048159420 2023-10-08T16:11:08.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303800ac70 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30380070f0 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30480c7280 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.429+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304815aec0 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.430+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304804c3c0 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.430+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f30380074e0 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.431+0000 7f30467fc700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 0x7f30340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.431+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f30380851a0 con 0x7f3048159420 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.431+0000 7f304d946700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 0x7f30340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.431+0000 7f304d946700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 0x7f30340697d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3048159c70 tx=0x7f303c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.435+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3038051eb0 con 0x7f3048159420 2023-10-08T16:11:08.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.620+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30480b1a70 con 0x7f3048159420 2023-10-08T16:11:08.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.620+0000 7f30467fc700 1 -- 172.21.15.170:0/3322448663 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f30380554d0 con 0x7f3048159420 2023-10-08T16:11:08.621 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:08.621 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":113,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:05.909482+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 msgr2=0x7f30340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 0x7f30340697d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3048159c70 tx=0x7f303c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 msgr2=0x7f30480c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f303800cd80 tx=0x7f3038001ae0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 shutdown_connections 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f304815d230 0x7f3048154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.622+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3034067320 0x7f30340697d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.623+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3048159420 0x7f30480c6410 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.623+0000 7f304f149700 1 --2- 172.21.15.170:0/3322448663 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f30481597f0 0x7f30480c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.623+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 >> 172.21.15.170:0/3322448663 conn(0x7f30480d0660 msgr2=0x7f30480b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.623+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 shutdown_connections 2023-10-08T16:11:08.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.623+0000 7f304f149700 1 -- 172.21.15.170:0/3322448663 wait complete. 2023-10-08T16:11:08.635 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:11:08.635 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:08.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.749+0000 7f957f4ec700 1 Processor -- start 2023-10-08T16:11:08.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 -- start start 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f95780af2f0 0x7f95780af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780ad930 con 0x7f9578155c10 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780b0c60 con 0x7f95780af2f0 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.750+0000 7f957f4ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95780acd20 con 0x7f95781578b0 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957ca87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f95780af2f0 0x7f95780af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957ca87700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:50688/0 (socket says 172.21.15.170:50688) 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957ca87700 1 -- 172.21.15.170:0/3740338584 learned_addr learned my addr 172.21.15.170:0/3740338584 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 -- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 msgr2=0x7f9578155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:08.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 -- 172.21.15.170:0/3740338584 >> v1:172.21.15.191:6789/0 conn(0x7f9578155c10 legacy=0x7f95780ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 -- 172.21.15.170:0/3740338584 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780ad000 con 0x7f95780af2f0 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957ca87700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f957da89700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95780af2f0 0x7f95780af710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e60 tx=0x7f9570009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3bae6aff848bdc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f956ffff700 1 -- 172.21.15.170:0/3740338584 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957000e070 con 0x7f95780af2f0 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f956ffff700 1 -- 172.21.15.170:0/3740338584 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957000bb60 con 0x7f95780af2f0 2023-10-08T16:11:08.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.751+0000 7f956ffff700 1 -- 172.21.15.170:0/3740338584 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570012430 con 0x7f95780af2f0 2023-10-08T16:11:08.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 -- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95780af2f0 msgr2=0x7f95780af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95780af2f0 0x7f95780af710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9570009e60 tx=0x7f9570009780 comp rx=0 tx=0).stop 2023-10-08T16:11:08.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 -- 172.21.15.170:0/3740338584 shutdown_connections 2023-10-08T16:11:08.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95780af2f0 0x7f95780af710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3740338584 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95781578b0 0x7f9578155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 -- 172.21.15.170:0/3740338584 >> 172.21.15.170:0/3740338584 conn(0x7f957806a320 msgr2=0x7f95780cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 -- 172.21.15.170:0/3740338584 shutdown_connections 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.752+0000 7f957f4ec700 1 -- 172.21.15.170:0/3740338584 wait complete. 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.753+0000 7f957f4ec700 1 Processor -- start 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.753+0000 7f957f4ec700 1 -- start start 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95780af2f0 0x7f95780cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95781578b0 0x7f9578152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780cc940 con 0x7f9578155c10 2023-10-08T16:11:08.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780cc1f0 con 0x7f95781578b0 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957f4ec700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95780c8ca0 con 0x7f95780af2f0 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44656/0 (socket says 172.21.15.170:44656) 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 -- 172.21.15.170:0/3993059163 learned_addr learned my addr 172.21.15.170:0/3993059163 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957da89700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95781578b0 0x7f9578152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957d288700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95780af2f0 0x7f95780cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 -- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95780af2f0 msgr2=0x7f95780cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95780af2f0 0x7f95780cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 -- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95781578b0 msgr2=0x7f9578152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95781578b0 0x7f9578152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.754+0000 7f957ca87700 1 -- 172.21.15.170:0/3993059163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95680094f0 con 0x7f9578155c10 2023-10-08T16:11:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f957ca87700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f956800ec80 tx=0x7f9568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800c850 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956800c9b0 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9570009a20 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800cd30 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.755+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95780a5f90 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.756+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95780a6120 con 0x7f9578155c10 2023-10-08T16:11:08.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.756+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9568013680 con 0x7f9578155c10 2023-10-08T16:11:08.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.757+0000 7f956d7fa700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 0x7f955c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:08.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.757+0000 7f957d288700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 0x7f955c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:08.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.757+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9568084ec0 con 0x7f9578155c10 2023-10-08T16:11:08.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.757+0000 7f957d288700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 0x7f955c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9574003a80 tx=0x7f9574007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:08.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.761+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9568051b50 con 0x7f9578155c10 2023-10-08T16:11:08.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.940+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f957804c3c0 con 0x7f955c067320 2023-10-08T16:11:08.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.948+0000 7f956d7fa700 1 -- 172.21.15.170:0/3993059163 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287108 (secure 0 0 0) 0x7f957804c3c0 con 0x7f955c067320 2023-10-08T16:11:08.952 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 msgr2=0x7f955c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 0x7f955c0697d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9574003a80 tx=0x7f9574007ec0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 msgr2=0x7f95780c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f956800ec80 tx=0x7f9568009250 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 shutdown_connections 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f95781578b0 0x7f9578152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f95780af2f0 0x7f95780cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f955c067320 0x7f955c0697d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 --2- 172.21.15.170:0/3993059163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9578155c10 0x7f95780c8760 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.951+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 >> 172.21.15.170:0/3993059163 conn(0x7f957806a320 msgr2=0x7f9578154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:08.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.952+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 shutdown_connections 2023-10-08T16:11:08.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:08.952+0000 7f957f4ec700 1 -- 172.21.15.170:0/3993059163 wait complete. 2023-10-08T16:11:08.956 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:08.964 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-10-08T16:11:08.304373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414864,"kb_used_data":11984,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752559856,"statfs":{"total":773094113280,"available":770621292544,"internally_reserved":0,"allocated":12271616,"data_stored":5952034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.414012"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903693+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:48:11.776245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896495+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903281+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901854+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:19.637626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899939+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:49.260895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897848+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897135+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.905669+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903887+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896110+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898238+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:39.892907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902869+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901932+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:14.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896049+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902883+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:33.760866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900115+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898155+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901480+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897822+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:24:47.754512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900169+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899392+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897293+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.904140+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897636+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:36:03.870568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896123+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899760+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902739+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:12.591868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899387+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903063+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:25.494465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902898+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:30.140643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899993+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:25:08.885969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896645+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899008+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911904+0000","last_change":"2023-10-08T16:11:03.911904+0000","last_active":"2023-10-08T16:11:03.911904+0000","last_peered":"2023-10-08T16:11:03.911904+0000","last_clean":"2023-10-08T16:11:03.911904+0000","last_became_active":"2023-10-08T16:11:03.911610+0000","last_became_peered":"2023-10-08T16:11:03.911610+0000","last_unstale":"2023-10-08T16:11:03.911904+0000","last_undegraded":"2023-10-08T16:11:03.911904+0000","last_fullsized":"2023-10-08T16:11:03.911904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:40.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902819+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908616+0000","last_change":"2023-10-08T16:11:03.908616+0000","last_active":"2023-10-08T16:11:03.908616+0000","last_peered":"2023-10-08T16:11:03.908616+0000","last_clean":"2023-10-08T16:11:03.908616+0000","last_became_active":"2023-10-08T16:11:03.908311+0000","last_became_peered":"2023-10-08T16:11:03.908311+0000","last_unstale":"2023-10-08T16:11:03.908616+0000","last_undegraded":"2023-10-08T16:11:03.908616+0000","last_fullsized":"2023-10-08T16:11:03.908616+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899711+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907369+0000","last_change":"2023-10-08T16:11:03.907369+0000","last_active":"2023-10-08T16:11:03.907369+0000","last_peered":"2023-10-08T16:11:03.907369+0000","last_clean":"2023-10-08T16:11:03.907369+0000","last_became_active":"2023-10-08T16:11:03.907119+0000","last_became_peered":"2023-10-08T16:11:03.907119+0000","last_unstale":"2023-10-08T16:11:03.907369+0000","last_undegraded":"2023-10-08T16:11:03.907369+0000","last_fullsized":"2023-10-08T16:11:03.907369+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:41.469949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899690+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.901096+0000","last_change":"2023-10-08T16:11:03.901096+0000","last_active":"2023-10-08T16:11:03.901096+0000","last_peered":"2023-10-08T16:11:03.901096+0000","last_clean":"2023-10-08T16:11:03.901096+0000","last_became_active":"2023-10-08T16:11:03.900784+0000","last_became_peered":"2023-10-08T16:11:03.900784+0000","last_unstale":"2023-10-08T16:11:03.901096+0000","last_undegraded":"2023-10-08T16:11:03.901096+0000","last_fullsized":"2023-10-08T16:11:03.901096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:33.310661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900962+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907483+0000","last_change":"2023-10-08T16:11:03.907483+0000","last_active":"2023-10-08T16:11:03.907483+0000","last_peered":"2023-10-08T16:11:03.907483+0000","last_clean":"2023-10-08T16:11:03.907483+0000","last_became_active":"2023-10-08T16:11:03.906971+0000","last_became_peered":"2023-10-08T16:11:03.906971+0000","last_unstale":"2023-10-08T16:11:03.907483+0000","last_undegraded":"2023-10-08T16:11:03.907483+0000","last_fullsized":"2023-10-08T16:11:03.907483+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:42.596727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897212+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:30.788302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906390+0000","last_change":"2023-10-08T16:11:03.906390+0000","last_active":"2023-10-08T16:11:03.906390+0000","last_peered":"2023-10-08T16:11:03.906390+0000","last_clean":"2023-10-08T16:11:03.906390+0000","last_became_active":"2023-10-08T16:11:03.906138+0000","last_became_peered":"2023-10-08T16:11:03.906138+0000","last_unstale":"2023-10-08T16:11:03.906390+0000","last_undegraded":"2023-10-08T16:11:03.906390+0000","last_fullsized":"2023-10-08T16:11:03.906390+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:48.258705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.906219+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906532+0000","last_change":"2023-10-08T16:11:03.906532+0000","last_active":"2023-10-08T16:11:03.906532+0000","last_peered":"2023-10-08T16:11:03.906532+0000","last_clean":"2023-10-08T16:11:03.906532+0000","last_became_active":"2023-10-08T16:11:03.906222+0000","last_became_peered":"2023-10-08T16:11:03.906222+0000","last_unstale":"2023-10-08T16:11:03.906532+0000","last_undegraded":"2023-10-08T16:11:03.906532+0000","last_fullsized":"2023-10-08T16:11:03.906532+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:47.712149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896598+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909969+0000","last_change":"2023-10-08T16:11:03.909969+0000","last_active":"2023-10-08T16:11:03.909969+0000","last_peered":"2023-10-08T16:11:03.909969+0000","last_clean":"2023-10-08T16:11:03.909969+0000","last_became_active":"2023-10-08T16:11:03.909698+0000","last_became_peered":"2023-10-08T16:11:03.909698+0000","last_unstale":"2023-10-08T16:11:03.909969+0000","last_undegraded":"2023-10-08T16:11:03.909969+0000","last_fullsized":"2023-10-08T16:11:03.909969+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:08.362073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902536+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:00.622457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.912368+0000","last_change":"2023-10-08T16:11:03.912368+0000","last_active":"2023-10-08T16:11:03.912368+0000","last_peered":"2023-10-08T16:11:03.912368+0000","last_clean":"2023-10-08T16:11:03.912368+0000","last_became_active":"2023-10-08T16:11:03.912066+0000","last_became_peered":"2023-10-08T16:11:03.912066+0000","last_unstale":"2023-10-08T16:11:03.912368+0000","last_undegraded":"2023-10-08T16:11:03.912368+0000","last_fullsized":"2023-10-08T16:11:03.912368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:52.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901423+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910705+0000","last_change":"2023-10-08T16:11:03.910705+0000","last_active":"2023-10-08T16:11:03.910705+0000","last_peered":"2023-10-08T16:11:03.910705+0000","last_clean":"2023-10-08T16:11:03.910705+0000","last_became_active":"2023-10-08T16:11:03.910321+0000","last_became_peered":"2023-10-08T16:11:03.910321+0000","last_unstale":"2023-10-08T16:11:03.910705+0000","last_undegraded":"2023-10-08T16:11:03.910705+0000","last_fullsized":"2023-10-08T16:11:03.910705+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:29.783953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911284+0000","last_change":"2023-10-08T16:11:03.911284+0000","last_active":"2023-10-08T16:11:03.911284+0000","last_peered":"2023-10-08T16:11:03.911284+0000","last_clean":"2023-10-08T16:11:03.911284+0000","last_became_active":"2023-10-08T16:11:03.910992+0000","last_became_peered":"2023-10-08T16:11:03.910992+0000","last_unstale":"2023-10-08T16:11:03.911284+0000","last_undegraded":"2023-10-08T16:11:03.911284+0000","last_fullsized":"2023-10-08T16:11:03.911284+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:17.563101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898505+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911078+0000","last_change":"2023-10-08T16:11:03.911078+0000","last_active":"2023-10-08T16:11:03.911078+0000","last_peered":"2023-10-08T16:11:03.911078+0000","last_clean":"2023-10-08T16:11:03.911078+0000","last_became_active":"2023-10-08T16:11:03.910522+0000","last_became_peered":"2023-10-08T16:11:03.910522+0000","last_unstale":"2023-10-08T16:11:03.911078+0000","last_undegraded":"2023-10-08T16:11:03.911078+0000","last_fullsized":"2023-10-08T16:11:03.911078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902657+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:30.954855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913269+0000","last_change":"2023-10-08T16:11:03.913269+0000","last_active":"2023-10-08T16:11:03.913269+0000","last_peered":"2023-10-08T16:11:03.913269+0000","last_clean":"2023-10-08T16:11:03.913269+0000","last_became_active":"2023-10-08T16:11:03.913044+0000","last_became_peered":"2023-10-08T16:11:03.913044+0000","last_unstale":"2023-10-08T16:11:03.913269+0000","last_undegraded":"2023-10-08T16:11:03.913269+0000","last_fullsized":"2023-10-08T16:11:03.913269+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:52.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896011+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905844+0000","last_change":"2023-10-08T16:11:03.905844+0000","last_active":"2023-10-08T16:11:03.905844+0000","last_peered":"2023-10-08T16:11:03.905844+0000","last_clean":"2023-10-08T16:11:03.905844+0000","last_became_active":"2023-10-08T16:11:03.905597+0000","last_became_peered":"2023-10-08T16:11:03.905597+0000","last_unstale":"2023-10-08T16:11:03.905844+0000","last_undegraded":"2023-10-08T16:11:03.905844+0000","last_fullsized":"2023-10-08T16:11:03.905844+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.915073+0000","last_change":"2023-10-08T16:11:03.915073+0000","last_active":"2023-10-08T16:11:03.915073+0000","last_peered":"2023-10-08T16:11:03.915073+0000","last_clean":"2023-10-08T16:11:03.915073+0000","last_became_active":"2023-10-08T16:11:03.914732+0000","last_became_peered":"2023-10-08T16:11:03.914732+0000","last_unstale":"2023-10-08T16:11:03.915073+0000","last_undegraded":"2023-10-08T16:11:03.915073+0000","last_fullsized":"2023-10-08T16:11:03.915073+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902783+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906254+0000","last_change":"2023-10-08T16:11:03.906254+0000","last_active":"2023-10-08T16:11:03.906254+0000","last_peered":"2023-10-08T16:11:03.906254+0000","last_clean":"2023-10-08T16:11:03.906254+0000","last_became_active":"2023-10-08T16:11:03.905790+0000","last_became_peered":"2023-10-08T16:11:03.905790+0000","last_unstale":"2023-10-08T16:11:03.906254+0000","last_undegraded":"2023-10-08T16:11:03.906254+0000","last_fullsized":"2023-10-08T16:11:03.906254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:44.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901908+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910220+0000","last_change":"2023-10-08T16:11:03.910220+0000","last_active":"2023-10-08T16:11:03.910220+0000","last_peered":"2023-10-08T16:11:03.910220+0000","last_clean":"2023-10-08T16:11:03.910220+0000","last_became_active":"2023-10-08T16:11:03.909963+0000","last_became_peered":"2023-10-08T16:11:03.909963+0000","last_unstale":"2023-10-08T16:11:03.910220+0000","last_undegraded":"2023-10-08T16:11:03.910220+0000","last_fullsized":"2023-10-08T16:11:03.910220+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:27.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899130+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907316+0000","last_change":"2023-10-08T16:11:03.907316+0000","last_active":"2023-10-08T16:11:03.907316+0000","last_peered":"2023-10-08T16:11:03.907316+0000","last_clean":"2023-10-08T16:11:03.907316+0000","last_became_active":"2023-10-08T16:11:03.907061+0000","last_became_peered":"2023-10-08T16:11:03.907061+0000","last_unstale":"2023-10-08T16:11:03.907316+0000","last_undegraded":"2023-10-08T16:11:03.907316+0000","last_fullsized":"2023-10-08T16:11:03.907316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:38.380438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895570+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907179+0000","last_change":"2023-10-08T16:11:03.907179+0000","last_active":"2023-10-08T16:11:03.907179+0000","last_peered":"2023-10-08T16:11:03.907179+0000","last_clean":"2023-10-08T16:11:03.907179+0000","last_became_active":"2023-10-08T16:11:03.906946+0000","last_became_peered":"2023-10-08T16:11:03.906946+0000","last_unstale":"2023-10-08T16:11:03.907179+0000","last_undegraded":"2023-10-08T16:11:03.907179+0000","last_fullsized":"2023-10-08T16:11:03.907179+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902425+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:38.941479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908030+0000","last_change":"2023-10-08T16:11:03.908030+0000","last_active":"2023-10-08T16:11:03.908030+0000","last_peered":"2023-10-08T16:11:03.908030+0000","last_clean":"2023-10-08T16:11:03.908030+0000","last_became_active":"2023-10-08T16:11:03.907731+0000","last_became_peered":"2023-10-08T16:11:03.907731+0000","last_unstale":"2023-10-08T16:11:03.908030+0000","last_undegraded":"2023-10-08T16:11:03.908030+0000","last_fullsized":"2023-10-08T16:11:03.908030+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:10.071216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896564+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909202+0000","last_change":"2023-10-08T16:11:03.909202+0000","last_active":"2023-10-08T16:11:03.909202+0000","last_peered":"2023-10-08T16:11:03.909202+0000","last_clean":"2023-10-08T16:11:03.909202+0000","last_became_active":"2023-10-08T16:11:03.908949+0000","last_became_peered":"2023-10-08T16:11:03.908949+0000","last_unstale":"2023-10-08T16:11:03.909202+0000","last_undegraded":"2023-10-08T16:11:03.909202+0000","last_fullsized":"2023-10-08T16:11:03.909202+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:54.146955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907175+0000","last_change":"2023-10-08T16:11:03.907175+0000","last_active":"2023-10-08T16:11:03.907175+0000","last_peered":"2023-10-08T16:11:03.907175+0000","last_clean":"2023-10-08T16:11:03.907175+0000","last_became_active":"2023-10-08T16:11:03.906936+0000","last_became_peered":"2023-10-08T16:11:03.906936+0000","last_unstale":"2023-10-08T16:11:03.907175+0000","last_undegraded":"2023-10-08T16:11:03.907175+0000","last_fullsized":"2023-10-08T16:11:03.907175+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:46.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899442+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910257+0000","last_change":"2023-10-08T16:11:03.910257+0000","last_active":"2023-10-08T16:11:03.910257+0000","last_peered":"2023-10-08T16:11:03.910257+0000","last_clean":"2023-10-08T16:11:03.910257+0000","last_became_active":"2023-10-08T16:11:03.909995+0000","last_became_peered":"2023-10-08T16:11:03.909995+0000","last_unstale":"2023-10-08T16:11:03.910257+0000","last_undegraded":"2023-10-08T16:11:03.910257+0000","last_fullsized":"2023-10-08T16:11:03.910257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:01.296731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897617+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:47.632034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909589+0000","last_change":"2023-10-08T16:11:03.909589+0000","last_active":"2023-10-08T16:11:03.909589+0000","last_peered":"2023-10-08T16:11:03.909589+0000","last_clean":"2023-10-08T16:11:03.909589+0000","last_became_active":"2023-10-08T16:11:03.909342+0000","last_became_peered":"2023-10-08T16:11:03.909342+0000","last_unstale":"2023-10-08T16:11:03.909589+0000","last_undegraded":"2023-10-08T16:11:03.909589+0000","last_fullsized":"2023-10-08T16:11:03.909589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:10.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903232+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:59.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913709+0000","last_change":"2023-10-08T16:11:03.913709+0000","last_active":"2023-10-08T16:11:03.913709+0000","last_peered":"2023-10-08T16:11:03.913709+0000","last_clean":"2023-10-08T16:11:03.913709+0000","last_became_active":"2023-10-08T16:11:03.913436+0000","last_became_peered":"2023-10-08T16:11:03.913436+0000","last_unstale":"2023-10-08T16:11:03.913709+0000","last_undegraded":"2023-10-08T16:11:03.913709+0000","last_fullsized":"2023-10-08T16:11:03.913709+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:07.589542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895429+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909139+0000","last_change":"2023-10-08T16:11:03.909139+0000","last_active":"2023-10-08T16:11:03.909139+0000","last_peered":"2023-10-08T16:11:03.909139+0000","last_clean":"2023-10-08T16:11:03.909139+0000","last_became_active":"2023-10-08T16:11:03.908877+0000","last_became_peered":"2023-10-08T16:11:03.908877+0000","last_unstale":"2023-10-08T16:11:03.909139+0000","last_undegraded":"2023-10-08T16:11:03.909139+0000","last_fullsized":"2023-10-08T16:11:03.909139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:48.689812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897073+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905282+0000","last_change":"2023-10-08T16:11:03.905282+0000","last_active":"2023-10-08T16:11:03.905282+0000","last_peered":"2023-10-08T16:11:03.905282+0000","last_clean":"2023-10-08T16:11:03.905282+0000","last_became_active":"2023-10-08T16:11:03.905030+0000","last_became_peered":"2023-10-08T16:11:03.905030+0000","last_unstale":"2023-10-08T16:11:03.905282+0000","last_undegraded":"2023-10-08T16:11:03.905282+0000","last_fullsized":"2023-10-08T16:11:03.905282+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:54.327772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898252+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907637+0000","last_change":"2023-10-08T16:11:03.907637+0000","last_active":"2023-10-08T16:11:03.907637+0000","last_peered":"2023-10-08T16:11:03.907637+0000","last_clean":"2023-10-08T16:11:03.907637+0000","last_became_active":"2023-10-08T16:11:03.907270+0000","last_became_peered":"2023-10-08T16:11:03.907270+0000","last_unstale":"2023-10-08T16:11:03.907637+0000","last_undegraded":"2023-10-08T16:11:03.907637+0000","last_fullsized":"2023-10-08T16:11:03.907637+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909548+0000","last_change":"2023-10-08T16:11:03.909548+0000","last_active":"2023-10-08T16:11:03.909548+0000","last_peered":"2023-10-08T16:11:03.909548+0000","last_clean":"2023-10-08T16:11:03.909548+0000","last_became_active":"2023-10-08T16:11:03.909225+0000","last_became_peered":"2023-10-08T16:11:03.909225+0000","last_unstale":"2023-10-08T16:11:03.909548+0000","last_undegraded":"2023-10-08T16:11:03.909548+0000","last_fullsized":"2023-10-08T16:11:03.909548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:39.249283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899215+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910568+0000","last_change":"2023-10-08T16:11:03.910568+0000","last_active":"2023-10-08T16:11:03.910568+0000","last_peered":"2023-10-08T16:11:03.910568+0000","last_clean":"2023-10-08T16:11:03.910568+0000","last_became_active":"2023-10-08T16:11:03.910311+0000","last_became_peered":"2023-10-08T16:11:03.910311+0000","last_unstale":"2023-10-08T16:11:03.910568+0000","last_undegraded":"2023-10-08T16:11:03.910568+0000","last_fullsized":"2023-10-08T16:11:03.910568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:56.735575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895513+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906699+0000","last_change":"2023-10-08T16:11:03.906699+0000","last_active":"2023-10-08T16:11:03.906699+0000","last_peered":"2023-10-08T16:11:03.906699+0000","last_clean":"2023-10-08T16:11:03.906699+0000","last_became_active":"2023-10-08T16:11:03.906348+0000","last_became_peered":"2023-10-08T16:11:03.906348+0000","last_unstale":"2023-10-08T16:11:03.906699+0000","last_undegraded":"2023-10-08T16:11:03.906699+0000","last_fullsized":"2023-10-08T16:11:03.906699+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:34.478940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898873+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911957+0000","last_change":"2023-10-08T16:11:03.911957+0000","last_active":"2023-10-08T16:11:03.911957+0000","last_peered":"2023-10-08T16:11:03.911957+0000","last_clean":"2023-10-08T16:11:03.911957+0000","last_became_active":"2023-10-08T16:11:03.911652+0000","last_became_peered":"2023-10-08T16:11:03.911652+0000","last_unstale":"2023-10-08T16:11:03.911957+0000","last_undegraded":"2023-10-08T16:11:03.911957+0000","last_fullsized":"2023-10-08T16:11:03.911957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:21.785467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-08T16:11:02.893767+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:02.893767+0000","last_peered":"2023-10-08T16:11:02.893767+0000","last_clean":"2023-10-08T16:11:02.893767+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:02.893767+0000","last_undegraded":"2023-10-08T16:11:02.893767+0000","last_fullsized":"2023-10-08T16:11:02.893767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1182083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.57899999999999996}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.502}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.378}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.441},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":591715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.68700000000000006,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64700000000000002,"5min":0.75,"15min":0.96599999999999997},"last":0.52800000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.94199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.28399999999999997,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.874},"last":0.61499999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.48899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.309},"min":{"1min":0.219,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.33700000000000002},"min":{"1min":0.219,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.58199999999999996,"15min":0.77500000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.35599999999999998},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.501,"15min":1.2230000000000001},"last":0.38},{"interface":"front","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.72399999999999998},"last":0.45600000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.38600000000000001},"min":{"1min":0.28799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.70399999999999996},"last":0.433},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.35799999999999998},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":1.077},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1.696,"15min":1.696},"last":0.754}]},{"osd":5,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":1.657,"15min":1.657},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.70399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.099},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.50600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.59499999999999997},"min":{"1min":0.48599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.1839999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":1188346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.53400000000000003},"min":{"1min":0.23499999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":3.3220000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.56699999999999995},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":5.6769999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.45500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.47799999999999998},"min":{"1min":0.22700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.434,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.60399999999999998},"min":{"1min":0.29899999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55100000000000005,"5min":1.498,"15min":5.5970000000000004},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.66300000000000003},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.66000000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":1.847,"15min":3.2890000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56200000000000006,"15min":0.77800000000000002},"min":{"1min":0.47199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.80200000000000005},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.754,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.26500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.44},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.40400000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.37},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.46800000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.64300000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.314,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.505,"5min":0.53600000000000003,"15min":1.238},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33200000000000002,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33100000000000002,"15min":0.35599999999999998},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39700000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.497,"5min":0.59299999999999997,"15min":1.3520000000000001},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36299999999999999,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.248},"max":{"1min":0.47599999999999998,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.42099999999999999},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":1.327},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.65700000000000003,"5min":1.099,"15min":1.099},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":1.27},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.91900000000000004,"15min":1.879},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.97599999999999998,"15min":1.825},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":1.0009999999999999,"15min":1.115},"last":0.55900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298644,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073196,"statfs":{"total":96636764160,"available":96330952704,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.56499999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.504}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:08.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.118+0000 7fd2c4add700 1 Processor -- start 2023-10-08T16:11:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.118+0000 7fd2c4add700 1 -- start start 2023-10-08T16:11:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2c4add700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd2c0159890 0x7fd2c015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2c4add700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2c4add700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c0069ee0 con 0x7fd2c0159c60 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2c4add700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c015a160 con 0x7fd2c0159890 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2c4add700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00d13a0 con 0x7fd2c0069560 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2be59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd2c0159890 0x7fd2c015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2bed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2b5d9b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fd2c0159c60 0x7fd2c015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:60216/0 (socket says 172.21.15.170:60216) 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2be59c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd2c0159890 0x7fd2c015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:33748/0 (socket says 172.21.15.170:33748) 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2b5d9b700 1 -- 172.21.15.170:0/758121732 learned_addr learned my addr 172.21.15.170:0/758121732 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.119+0000 7fd2bed9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:50712/0 (socket says 172.21.15.170:50712) 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2be59c700 1 -- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 msgr2=0x7fd2c00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2be59c700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2be59c700 1 -- 172.21.15.170:0/758121732 >> v1:172.21.15.191:6789/0 conn(0x7fd2c0159c60 legacy=0x7fd2c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2be59c700 1 -- 172.21.15.170:0/758121732 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c015e140 con 0x7fd2c0159890 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2bed9d700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2be59c700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c015abd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74f900fb3b7cbb6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2bdd9b700 1 -- 172.21.15.170:0/758121732 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000b640 con 0x7fd2c0159890 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2bdd9b700 1 -- 172.21.15.170:0/758121732 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000b7a0 con 0x7fd2c0159890 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.120+0000 7fd2bdd9b700 1 -- 172.21.15.170:0/758121732 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000bb40 con 0x7fd2c0159890 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 -- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 msgr2=0x7fd2c015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c015abd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b00091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 -- 172.21.15.170:0/758121732 shutdown_connections 2023-10-08T16:11:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0069560 0x7fd2c00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 --2- 172.21.15.170:0/758121732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c015abd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 -- 172.21.15.170:0/758121732 >> 172.21.15.170:0/758121732 conn(0x7fd2c0068e20 msgr2=0x7fd2c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 -- 172.21.15.170:0/758121732 shutdown_connections 2023-10-08T16:11:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.121+0000 7fd2c4add700 1 -- 172.21.15.170:0/758121732 wait complete. 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.122+0000 7fd2c4add700 1 Processor -- start 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.122+0000 7fd2c4add700 1 -- start start 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c00c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0159c60 0x7fd2c00c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00c9e20 con 0x7fd2c0069560 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00c9210 con 0x7fd2c0159890 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00c94f0 con 0x7fd2c0159c60 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44660/0 (socket says 172.21.15.170:44660) 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 -- 172.21.15.170:0/1606419659 learned_addr learned my addr 172.21.15.170:0/1606419659 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:09.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2b5d9b700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c00c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2bed9d700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0159c60 0x7fd2c00c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 -- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0159c60 msgr2=0x7fd2c00c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0159c60 0x7fd2c00c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 -- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 msgr2=0x7fd2c00c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c00c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 -- 172.21.15.170:0/1606419659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b8009a20 con 0x7fd2c0069560 2023-10-08T16:11:09.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2be59c700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00899c0 tx=0x7fd2b0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001bda0 con 0x7fd2c0069560 2023-10-08T16:11:09.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0011420 con 0x7fd2c0069560 2023-10-08T16:11:09.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0009420 con 0x7fd2c0069560 2023-10-08T16:11:09.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00117e0 con 0x7fd2c0069560 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.123+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c00c9ab0 con 0x7fd2c0069560 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.124+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c004c3c0 con 0x7fd2c0069560 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.125+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd2b0011b50 con 0x7fd2c0069560 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.125+0000 7fd2b6ffd700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 0x7fd2a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.125+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd2b0086400 con 0x7fd2c0069560 2023-10-08T16:11:09.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.125+0000 7fd2b5d9b700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 0x7fd2a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.126+0000 7fd2b5d9b700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 0x7fd2a00697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8001780 tx=0x7fd2a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.129+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2b0053090 con 0x7fd2c0069560 2023-10-08T16:11:09.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.309+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2c00c9f60 con 0x7fd2a0067320 2023-10-08T16:11:09.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.316+0000 7fd2b6ffd700 1 -- 172.21.15.170:0/1606419659 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287108 (secure 0 0 0) 0x7fd2a80058e0 con 0x7fd2a0067320 2023-10-08T16:11:09.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:09.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 msgr2=0x7fd2a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 0x7fd2a00697d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8001780 tx=0x7fd2a8009040 comp rx=0 tx=0).stop 2023-10-08T16:11:09.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 msgr2=0x7fd2c01584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00899c0 tx=0x7fd2b0017000 comp rx=0 tx=0).stop 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 shutdown_connections 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd2c0159c60 0x7fd2c00c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd2a0067320 0x7fd2a00697d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd2c0069560 0x7fd2c01584d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 --2- 172.21.15.170:0/1606419659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd2c0159890 0x7fd2c00c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.319+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 >> 172.21.15.170:0/1606419659 conn(0x7fd2c0068e20 msgr2=0x7fd2c015b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.320+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 shutdown_connections 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.320+0000 7fd2c4add700 1 -- 172.21.15.170:0/1606419659 wait complete. 2023-10-08T16:11:09.324 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:09.333 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-10-08T16:11:08.304373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414864,"kb_used_data":11984,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752559856,"statfs":{"total":773094113280,"available":770621292544,"internally_reserved":0,"allocated":12271616,"data_stored":5952034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.414012"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903693+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:48:11.776245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896495+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903281+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901854+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:19.637626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899939+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:49.260895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897848+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897135+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.905669+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903887+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896110+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898238+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:39.892907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902869+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901932+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:14.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896049+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902883+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:33.760866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900115+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898155+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901480+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897822+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:24:47.754512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900169+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899392+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897293+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.904140+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897636+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:36:03.870568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896123+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899760+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902739+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:12.591868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899387+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903063+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:25.494465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902898+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:30.140643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899993+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:25:08.885969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896645+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899008+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911904+0000","last_change":"2023-10-08T16:11:03.911904+0000","last_active":"2023-10-08T16:11:03.911904+0000","last_peered":"2023-10-08T16:11:03.911904+0000","last_clean":"2023-10-08T16:11:03.911904+0000","last_became_active":"2023-10-08T16:11:03.911610+0000","last_became_peered":"2023-10-08T16:11:03.911610+0000","last_unstale":"2023-10-08T16:11:03.911904+0000","last_undegraded":"2023-10-08T16:11:03.911904+0000","last_fullsized":"2023-10-08T16:11:03.911904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:40.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902819+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908616+0000","last_change":"2023-10-08T16:11:03.908616+0000","last_active":"2023-10-08T16:11:03.908616+0000","last_peered":"2023-10-08T16:11:03.908616+0000","last_clean":"2023-10-08T16:11:03.908616+0000","last_became_active":"2023-10-08T16:11:03.908311+0000","last_became_peered":"2023-10-08T16:11:03.908311+0000","last_unstale":"2023-10-08T16:11:03.908616+0000","last_undegraded":"2023-10-08T16:11:03.908616+0000","last_fullsized":"2023-10-08T16:11:03.908616+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899711+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907369+0000","last_change":"2023-10-08T16:11:03.907369+0000","last_active":"2023-10-08T16:11:03.907369+0000","last_peered":"2023-10-08T16:11:03.907369+0000","last_clean":"2023-10-08T16:11:03.907369+0000","last_became_active":"2023-10-08T16:11:03.907119+0000","last_became_peered":"2023-10-08T16:11:03.907119+0000","last_unstale":"2023-10-08T16:11:03.907369+0000","last_undegraded":"2023-10-08T16:11:03.907369+0000","last_fullsized":"2023-10-08T16:11:03.907369+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:41.469949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899690+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.901096+0000","last_change":"2023-10-08T16:11:03.901096+0000","last_active":"2023-10-08T16:11:03.901096+0000","last_peered":"2023-10-08T16:11:03.901096+0000","last_clean":"2023-10-08T16:11:03.901096+0000","last_became_active":"2023-10-08T16:11:03.900784+0000","last_became_peered":"2023-10-08T16:11:03.900784+0000","last_unstale":"2023-10-08T16:11:03.901096+0000","last_undegraded":"2023-10-08T16:11:03.901096+0000","last_fullsized":"2023-10-08T16:11:03.901096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:33.310661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900962+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907483+0000","last_change":"2023-10-08T16:11:03.907483+0000","last_active":"2023-10-08T16:11:03.907483+0000","last_peered":"2023-10-08T16:11:03.907483+0000","last_clean":"2023-10-08T16:11:03.907483+0000","last_became_active":"2023-10-08T16:11:03.906971+0000","last_became_peered":"2023-10-08T16:11:03.906971+0000","last_unstale":"2023-10-08T16:11:03.907483+0000","last_undegraded":"2023-10-08T16:11:03.907483+0000","last_fullsized":"2023-10-08T16:11:03.907483+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:42.596727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897212+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:30.788302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906390+0000","last_change":"2023-10-08T16:11:03.906390+0000","last_active":"2023-10-08T16:11:03.906390+0000","last_peered":"2023-10-08T16:11:03.906390+0000","last_clean":"2023-10-08T16:11:03.906390+0000","last_became_active":"2023-10-08T16:11:03.906138+0000","last_became_peered":"2023-10-08T16:11:03.906138+0000","last_unstale":"2023-10-08T16:11:03.906390+0000","last_undegraded":"2023-10-08T16:11:03.906390+0000","last_fullsized":"2023-10-08T16:11:03.906390+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:48.258705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.906219+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906532+0000","last_change":"2023-10-08T16:11:03.906532+0000","last_active":"2023-10-08T16:11:03.906532+0000","last_peered":"2023-10-08T16:11:03.906532+0000","last_clean":"2023-10-08T16:11:03.906532+0000","last_became_active":"2023-10-08T16:11:03.906222+0000","last_became_peered":"2023-10-08T16:11:03.906222+0000","last_unstale":"2023-10-08T16:11:03.906532+0000","last_undegraded":"2023-10-08T16:11:03.906532+0000","last_fullsized":"2023-10-08T16:11:03.906532+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:47.712149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896598+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909969+0000","last_change":"2023-10-08T16:11:03.909969+0000","last_active":"2023-10-08T16:11:03.909969+0000","last_peered":"2023-10-08T16:11:03.909969+0000","last_clean":"2023-10-08T16:11:03.909969+0000","last_became_active":"2023-10-08T16:11:03.909698+0000","last_became_peered":"2023-10-08T16:11:03.909698+0000","last_unstale":"2023-10-08T16:11:03.909969+0000","last_undegraded":"2023-10-08T16:11:03.909969+0000","last_fullsized":"2023-10-08T16:11:03.909969+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:08.362073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902536+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:00.622457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.912368+0000","last_change":"2023-10-08T16:11:03.912368+0000","last_active":"2023-10-08T16:11:03.912368+0000","last_peered":"2023-10-08T16:11:03.912368+0000","last_clean":"2023-10-08T16:11:03.912368+0000","last_became_active":"2023-10-08T16:11:03.912066+0000","last_became_peered":"2023-10-08T16:11:03.912066+0000","last_unstale":"2023-10-08T16:11:03.912368+0000","last_undegraded":"2023-10-08T16:11:03.912368+0000","last_fullsized":"2023-10-08T16:11:03.912368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:52.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901423+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910705+0000","last_change":"2023-10-08T16:11:03.910705+0000","last_active":"2023-10-08T16:11:03.910705+0000","last_peered":"2023-10-08T16:11:03.910705+0000","last_clean":"2023-10-08T16:11:03.910705+0000","last_became_active":"2023-10-08T16:11:03.910321+0000","last_became_peered":"2023-10-08T16:11:03.910321+0000","last_unstale":"2023-10-08T16:11:03.910705+0000","last_undegraded":"2023-10-08T16:11:03.910705+0000","last_fullsized":"2023-10-08T16:11:03.910705+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:29.783953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911284+0000","last_change":"2023-10-08T16:11:03.911284+0000","last_active":"2023-10-08T16:11:03.911284+0000","last_peered":"2023-10-08T16:11:03.911284+0000","last_clean":"2023-10-08T16:11:03.911284+0000","last_became_active":"2023-10-08T16:11:03.910992+0000","last_became_peered":"2023-10-08T16:11:03.910992+0000","last_unstale":"2023-10-08T16:11:03.911284+0000","last_undegraded":"2023-10-08T16:11:03.911284+0000","last_fullsized":"2023-10-08T16:11:03.911284+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:17.563101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898505+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911078+0000","last_change":"2023-10-08T16:11:03.911078+0000","last_active":"2023-10-08T16:11:03.911078+0000","last_peered":"2023-10-08T16:11:03.911078+0000","last_clean":"2023-10-08T16:11:03.911078+0000","last_became_active":"2023-10-08T16:11:03.910522+0000","last_became_peered":"2023-10-08T16:11:03.910522+0000","last_unstale":"2023-10-08T16:11:03.911078+0000","last_undegraded":"2023-10-08T16:11:03.911078+0000","last_fullsized":"2023-10-08T16:11:03.911078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902657+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:30.954855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913269+0000","last_change":"2023-10-08T16:11:03.913269+0000","last_active":"2023-10-08T16:11:03.913269+0000","last_peered":"2023-10-08T16:11:03.913269+0000","last_clean":"2023-10-08T16:11:03.913269+0000","last_became_active":"2023-10-08T16:11:03.913044+0000","last_became_peered":"2023-10-08T16:11:03.913044+0000","last_unstale":"2023-10-08T16:11:03.913269+0000","last_undegraded":"2023-10-08T16:11:03.913269+0000","last_fullsized":"2023-10-08T16:11:03.913269+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:52.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896011+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905844+0000","last_change":"2023-10-08T16:11:03.905844+0000","last_active":"2023-10-08T16:11:03.905844+0000","last_peered":"2023-10-08T16:11:03.905844+0000","last_clean":"2023-10-08T16:11:03.905844+0000","last_became_active":"2023-10-08T16:11:03.905597+0000","last_became_peered":"2023-10-08T16:11:03.905597+0000","last_unstale":"2023-10-08T16:11:03.905844+0000","last_undegraded":"2023-10-08T16:11:03.905844+0000","last_fullsized":"2023-10-08T16:11:03.905844+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.915073+0000","last_change":"2023-10-08T16:11:03.915073+0000","last_active":"2023-10-08T16:11:03.915073+0000","last_peered":"2023-10-08T16:11:03.915073+0000","last_clean":"2023-10-08T16:11:03.915073+0000","last_became_active":"2023-10-08T16:11:03.914732+0000","last_became_peered":"2023-10-08T16:11:03.914732+0000","last_unstale":"2023-10-08T16:11:03.915073+0000","last_undegraded":"2023-10-08T16:11:03.915073+0000","last_fullsized":"2023-10-08T16:11:03.915073+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902783+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906254+0000","last_change":"2023-10-08T16:11:03.906254+0000","last_active":"2023-10-08T16:11:03.906254+0000","last_peered":"2023-10-08T16:11:03.906254+0000","last_clean":"2023-10-08T16:11:03.906254+0000","last_became_active":"2023-10-08T16:11:03.905790+0000","last_became_peered":"2023-10-08T16:11:03.905790+0000","last_unstale":"2023-10-08T16:11:03.906254+0000","last_undegraded":"2023-10-08T16:11:03.906254+0000","last_fullsized":"2023-10-08T16:11:03.906254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:44.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901908+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910220+0000","last_change":"2023-10-08T16:11:03.910220+0000","last_active":"2023-10-08T16:11:03.910220+0000","last_peered":"2023-10-08T16:11:03.910220+0000","last_clean":"2023-10-08T16:11:03.910220+0000","last_became_active":"2023-10-08T16:11:03.909963+0000","last_became_peered":"2023-10-08T16:11:03.909963+0000","last_unstale":"2023-10-08T16:11:03.910220+0000","last_undegraded":"2023-10-08T16:11:03.910220+0000","last_fullsized":"2023-10-08T16:11:03.910220+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:27.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899130+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907316+0000","last_change":"2023-10-08T16:11:03.907316+0000","last_active":"2023-10-08T16:11:03.907316+0000","last_peered":"2023-10-08T16:11:03.907316+0000","last_clean":"2023-10-08T16:11:03.907316+0000","last_became_active":"2023-10-08T16:11:03.907061+0000","last_became_peered":"2023-10-08T16:11:03.907061+0000","last_unstale":"2023-10-08T16:11:03.907316+0000","last_undegraded":"2023-10-08T16:11:03.907316+0000","last_fullsized":"2023-10-08T16:11:03.907316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:38.380438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895570+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907179+0000","last_change":"2023-10-08T16:11:03.907179+0000","last_active":"2023-10-08T16:11:03.907179+0000","last_peered":"2023-10-08T16:11:03.907179+0000","last_clean":"2023-10-08T16:11:03.907179+0000","last_became_active":"2023-10-08T16:11:03.906946+0000","last_became_peered":"2023-10-08T16:11:03.906946+0000","last_unstale":"2023-10-08T16:11:03.907179+0000","last_undegraded":"2023-10-08T16:11:03.907179+0000","last_fullsized":"2023-10-08T16:11:03.907179+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902425+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:38.941479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908030+0000","last_change":"2023-10-08T16:11:03.908030+0000","last_active":"2023-10-08T16:11:03.908030+0000","last_peered":"2023-10-08T16:11:03.908030+0000","last_clean":"2023-10-08T16:11:03.908030+0000","last_became_active":"2023-10-08T16:11:03.907731+0000","last_became_peered":"2023-10-08T16:11:03.907731+0000","last_unstale":"2023-10-08T16:11:03.908030+0000","last_undegraded":"2023-10-08T16:11:03.908030+0000","last_fullsized":"2023-10-08T16:11:03.908030+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:10.071216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896564+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909202+0000","last_change":"2023-10-08T16:11:03.909202+0000","last_active":"2023-10-08T16:11:03.909202+0000","last_peered":"2023-10-08T16:11:03.909202+0000","last_clean":"2023-10-08T16:11:03.909202+0000","last_became_active":"2023-10-08T16:11:03.908949+0000","last_became_peered":"2023-10-08T16:11:03.908949+0000","last_unstale":"2023-10-08T16:11:03.909202+0000","last_undegraded":"2023-10-08T16:11:03.909202+0000","last_fullsized":"2023-10-08T16:11:03.909202+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:54.146955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907175+0000","last_change":"2023-10-08T16:11:03.907175+0000","last_active":"2023-10-08T16:11:03.907175+0000","last_peered":"2023-10-08T16:11:03.907175+0000","last_clean":"2023-10-08T16:11:03.907175+0000","last_became_active":"2023-10-08T16:11:03.906936+0000","last_became_peered":"2023-10-08T16:11:03.906936+0000","last_unstale":"2023-10-08T16:11:03.907175+0000","last_undegraded":"2023-10-08T16:11:03.907175+0000","last_fullsized":"2023-10-08T16:11:03.907175+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:46.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899442+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910257+0000","last_change":"2023-10-08T16:11:03.910257+0000","last_active":"2023-10-08T16:11:03.910257+0000","last_peered":"2023-10-08T16:11:03.910257+0000","last_clean":"2023-10-08T16:11:03.910257+0000","last_became_active":"2023-10-08T16:11:03.909995+0000","last_became_peered":"2023-10-08T16:11:03.909995+0000","last_unstale":"2023-10-08T16:11:03.910257+0000","last_undegraded":"2023-10-08T16:11:03.910257+0000","last_fullsized":"2023-10-08T16:11:03.910257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:01.296731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897617+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:47.632034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909589+0000","last_change":"2023-10-08T16:11:03.909589+0000","last_active":"2023-10-08T16:11:03.909589+0000","last_peered":"2023-10-08T16:11:03.909589+0000","last_clean":"2023-10-08T16:11:03.909589+0000","last_became_active":"2023-10-08T16:11:03.909342+0000","last_became_peered":"2023-10-08T16:11:03.909342+0000","last_unstale":"2023-10-08T16:11:03.909589+0000","last_undegraded":"2023-10-08T16:11:03.909589+0000","last_fullsized":"2023-10-08T16:11:03.909589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:10.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903232+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:59.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913709+0000","last_change":"2023-10-08T16:11:03.913709+0000","last_active":"2023-10-08T16:11:03.913709+0000","last_peered":"2023-10-08T16:11:03.913709+0000","last_clean":"2023-10-08T16:11:03.913709+0000","last_became_active":"2023-10-08T16:11:03.913436+0000","last_became_peered":"2023-10-08T16:11:03.913436+0000","last_unstale":"2023-10-08T16:11:03.913709+0000","last_undegraded":"2023-10-08T16:11:03.913709+0000","last_fullsized":"2023-10-08T16:11:03.913709+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:07.589542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895429+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909139+0000","last_change":"2023-10-08T16:11:03.909139+0000","last_active":"2023-10-08T16:11:03.909139+0000","last_peered":"2023-10-08T16:11:03.909139+0000","last_clean":"2023-10-08T16:11:03.909139+0000","last_became_active":"2023-10-08T16:11:03.908877+0000","last_became_peered":"2023-10-08T16:11:03.908877+0000","last_unstale":"2023-10-08T16:11:03.909139+0000","last_undegraded":"2023-10-08T16:11:03.909139+0000","last_fullsized":"2023-10-08T16:11:03.909139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:48.689812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897073+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905282+0000","last_change":"2023-10-08T16:11:03.905282+0000","last_active":"2023-10-08T16:11:03.905282+0000","last_peered":"2023-10-08T16:11:03.905282+0000","last_clean":"2023-10-08T16:11:03.905282+0000","last_became_active":"2023-10-08T16:11:03.905030+0000","last_became_peered":"2023-10-08T16:11:03.905030+0000","last_unstale":"2023-10-08T16:11:03.905282+0000","last_undegraded":"2023-10-08T16:11:03.905282+0000","last_fullsized":"2023-10-08T16:11:03.905282+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:54.327772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898252+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907637+0000","last_change":"2023-10-08T16:11:03.907637+0000","last_active":"2023-10-08T16:11:03.907637+0000","last_peered":"2023-10-08T16:11:03.907637+0000","last_clean":"2023-10-08T16:11:03.907637+0000","last_became_active":"2023-10-08T16:11:03.907270+0000","last_became_peered":"2023-10-08T16:11:03.907270+0000","last_unstale":"2023-10-08T16:11:03.907637+0000","last_undegraded":"2023-10-08T16:11:03.907637+0000","last_fullsized":"2023-10-08T16:11:03.907637+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909548+0000","last_change":"2023-10-08T16:11:03.909548+0000","last_active":"2023-10-08T16:11:03.909548+0000","last_peered":"2023-10-08T16:11:03.909548+0000","last_clean":"2023-10-08T16:11:03.909548+0000","last_became_active":"2023-10-08T16:11:03.909225+0000","last_became_peered":"2023-10-08T16:11:03.909225+0000","last_unstale":"2023-10-08T16:11:03.909548+0000","last_undegraded":"2023-10-08T16:11:03.909548+0000","last_fullsized":"2023-10-08T16:11:03.909548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:39.249283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899215+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910568+0000","last_change":"2023-10-08T16:11:03.910568+0000","last_active":"2023-10-08T16:11:03.910568+0000","last_peered":"2023-10-08T16:11:03.910568+0000","last_clean":"2023-10-08T16:11:03.910568+0000","last_became_active":"2023-10-08T16:11:03.910311+0000","last_became_peered":"2023-10-08T16:11:03.910311+0000","last_unstale":"2023-10-08T16:11:03.910568+0000","last_undegraded":"2023-10-08T16:11:03.910568+0000","last_fullsized":"2023-10-08T16:11:03.910568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:56.735575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895513+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906699+0000","last_change":"2023-10-08T16:11:03.906699+0000","last_active":"2023-10-08T16:11:03.906699+0000","last_peered":"2023-10-08T16:11:03.906699+0000","last_clean":"2023-10-08T16:11:03.906699+0000","last_became_active":"2023-10-08T16:11:03.906348+0000","last_became_peered":"2023-10-08T16:11:03.906348+0000","last_unstale":"2023-10-08T16:11:03.906699+0000","last_undegraded":"2023-10-08T16:11:03.906699+0000","last_fullsized":"2023-10-08T16:11:03.906699+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:34.478940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898873+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911957+0000","last_change":"2023-10-08T16:11:03.911957+0000","last_active":"2023-10-08T16:11:03.911957+0000","last_peered":"2023-10-08T16:11:03.911957+0000","last_clean":"2023-10-08T16:11:03.911957+0000","last_became_active":"2023-10-08T16:11:03.911652+0000","last_became_peered":"2023-10-08T16:11:03.911652+0000","last_unstale":"2023-10-08T16:11:03.911957+0000","last_undegraded":"2023-10-08T16:11:03.911957+0000","last_fullsized":"2023-10-08T16:11:03.911957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:21.785467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-08T16:11:02.893767+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:02.893767+0000","last_peered":"2023-10-08T16:11:02.893767+0000","last_clean":"2023-10-08T16:11:02.893767+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:02.893767+0000","last_undegraded":"2023-10-08T16:11:02.893767+0000","last_fullsized":"2023-10-08T16:11:02.893767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1182083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.57899999999999996}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.502}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.378}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.441},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":591715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.68700000000000006,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64700000000000002,"5min":0.75,"15min":0.96599999999999997},"last":0.52800000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.94199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.28399999999999997,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.874},"last":0.61499999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.48899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.309},"min":{"1min":0.219,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.33700000000000002},"min":{"1min":0.219,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.58199999999999996,"15min":0.77500000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.35599999999999998},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.501,"15min":1.2230000000000001},"last":0.38},{"interface":"front","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.72399999999999998},"last":0.45600000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.38600000000000001},"min":{"1min":0.28799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.70399999999999996},"last":0.433},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.35799999999999998},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":1.077},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1.696,"15min":1.696},"last":0.754}]},{"osd":5,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":1.657,"15min":1.657},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.70399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.099},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.50600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.59499999999999997},"min":{"1min":0.48599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.1839999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":1188346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.53400000000000003},"min":{"1min":0.23499999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":3.3220000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.56699999999999995},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":5.6769999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.45500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.47799999999999998},"min":{"1min":0.22700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.434,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.60399999999999998},"min":{"1min":0.29899999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55100000000000005,"5min":1.498,"15min":5.5970000000000004},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.66300000000000003},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.66000000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":1.847,"15min":3.2890000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56200000000000006,"15min":0.77800000000000002},"min":{"1min":0.47199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.80200000000000005},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.754,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.26500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.44},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.40400000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.37},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.46800000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.64300000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.314,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.505,"5min":0.53600000000000003,"15min":1.238},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33200000000000002,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33100000000000002,"15min":0.35599999999999998},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39700000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.497,"5min":0.59299999999999997,"15min":1.3520000000000001},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36299999999999999,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.248},"max":{"1min":0.47599999999999998,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.42099999999999999},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":1.327},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.65700000000000003,"5min":1.099,"15min":1.099},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":1.27},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.91900000000000004,"15min":1.879},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.97599999999999998,"15min":1.825},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":1.0009999999999999,"15min":1.115},"last":0.55900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298644,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073196,"statfs":{"total":96636764160,"available":96330952704,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.56499999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.504}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:09.341 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:09.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.487+0000 7f8be1cca700 1 Processor -- start 2023-10-08T16:11:09.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.487+0000 7f8be1cca700 1 -- start start 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc1597f0 0x7f8bdc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 0x7f8bdc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc15dbd0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bdc0b11d0 con 0x7f8bdc15d230 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be1cca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0b14b0 con 0x7f8bdc159420 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc1597f0 0x7f8bdc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8bdc1597f0 0x7f8bdc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44666/0 (socket says 172.21.15.170:44666) 2023-10-08T16:11:09.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 -- 172.21.15.170:0/1445939122 learned_addr learned my addr 172.21.15.170:0/1445939122 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be14c9700 1 --2- 172.21.15.170:0/1445939122 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 0x7f8bdc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 -- 172.21.15.170:0/1445939122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 msgr2=0x7f8bdc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 -- 172.21.15.170:0/1445939122 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 msgr2=0x7f8bdc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/1445939122 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 0x7f8bdc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 -- 172.21.15.170:0/1445939122 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc0b1790 con 0x7f8bdc1597f0 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be14c9700 1 --2- 172.21.15.170:0/1445939122 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 0x7f8bdc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0b2270 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d1720 tx=0x7f8bcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e2cf999bcddfa5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdb7fe700 1 -- 172.21.15.170:0/1445939122 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00e070 con 0x7f8bdc1597f0 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdb7fe700 1 -- 172.21.15.170:0/1445939122 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc00ba50 con 0x7f8bdc1597f0 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.488+0000 7f8bdb7fe700 1 -- 172.21.15.170:0/1445939122 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0092e0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 -- 172.21.15.170:0/1445939122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 msgr2=0x7f8bdc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0b2270 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc0d1720 tx=0x7f8bcc009840 comp rx=0 tx=0).stop 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 -- 172.21.15.170:0/1445939122 shutdown_connections 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 --2- 172.21.15.170:0/1445939122 >> v2:172.21.15.191:3300/0 conn(0x7f8bdc15d230 0x7f8bdc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 --2- 172.21.15.170:0/1445939122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0b2270 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 -- 172.21.15.170:0/1445939122 >> 172.21.15.170:0/1445939122 conn(0x7f8bdc0d0660 msgr2=0x7f8bdc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 -- 172.21.15.170:0/1445939122 shutdown_connections 2023-10-08T16:11:09.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.489+0000 7f8be1cca700 1 -- 172.21.15.170:0/1445939122 wait complete. 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.490+0000 7f8be1cca700 1 Processor -- start 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.490+0000 7f8be1cca700 1 -- start start 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc15c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 0x7f8bdc153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc0c65f0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc15a730 con 0x7f8bdc15d230 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be1cca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc159e00 con 0x7f8bdc159420 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44676/0 (socket says 172.21.15.170:44676) 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 -- 172.21.15.170:0/638906659 learned_addr learned my addr 172.21.15.170:0/638906659 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be14c9700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 0x7f8bdc153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 -- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 msgr2=0x7f8bdc15c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc15c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 -- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 msgr2=0x7f8bdc153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 0x7f8bdc153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8bdbfff700 1 -- 172.21.15.170:0/638906659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009ae0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.491+0000 7f8be14c9700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 0x7f8bdc153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8bdbfff700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc089340 tx=0x7f8bcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc008b70 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc008cd0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc15a0e0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0266b0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.492+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc0c62f0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.493+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc15a8c0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.493+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8bcc003040 con 0x7f8bdc1597f0 2023-10-08T16:11:09.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.493+0000 7f8bd8ff9700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 0x7f8bbc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.493+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8bcc086cd0 con 0x7f8bdc1597f0 2023-10-08T16:11:09.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.493+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 0x7f8bbc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.494+0000 7f8be0cc8700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 0x7f8bbc0697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ab50 tx=0x7f8bd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.498+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bcc053850 con 0x7f8bdc1597f0 2023-10-08T16:11:09.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.676+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8bdc04c3c0 con 0x7f8bbc067320 2023-10-08T16:11:09.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.684+0000 7f8bd8ff9700 1 -- 172.21.15.170:0/638906659 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287108 (secure 0 0 0) 0x7f8bdc04c3c0 con 0x7f8bbc067320 2023-10-08T16:11:09.688 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 msgr2=0x7f8bbc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 0x7f8bbc0697d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400ab50 tx=0x7f8bd4008040 comp rx=0 tx=0).stop 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 msgr2=0x7f8bdc0c5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc089340 tx=0x7f8bcc018000 comp rx=0 tx=0).stop 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 shutdown_connections 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8bdc15d230 0x7f8bdc153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8bdc159420 0x7f8bdc15c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bbc067320 0x7f8bbc0697d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 --2- 172.21.15.170:0/638906659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8bdc1597f0 0x7f8bdc0c5db0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 >> 172.21.15.170:0/638906659 conn(0x7f8bdc0d0660 msgr2=0x7f8bdc0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:09.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 shutdown_connections 2023-10-08T16:11:09.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.687+0000 7f8be1cca700 1 -- 172.21.15.170:0/638906659 wait complete. 2023-10-08T16:11:09.692 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:09.700 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2023-10-08T16:11:08.304373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2414864,"kb_used_data":11984,"kb_used_omap":0,"kb_used_meta":2402816,"kb_avail":752559856,"statfs":{"total":773094113280,"available":770621292544,"internally_reserved":0,"allocated":12271616,"data_stored":5952034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2460483584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.414012"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903693+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:48:11.776245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896495+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903281+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901854+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:19.637626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899939+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:49:49.260895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897848+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897135+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.905669+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903887+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896110+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898238+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:39.892907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902869+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901932+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:43:14.536400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896049+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902883+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:33.760866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900115+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898155+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901480+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897822+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:24:47.754512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900169+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899392+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897293+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.904140+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897636+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:36:03.870568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896123+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899760+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902739+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:12.591868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899387+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903063+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:53:25.494465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902898+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:30.140643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899993+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:25:08.885969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896645+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899008+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911904+0000","last_change":"2023-10-08T16:11:03.911904+0000","last_active":"2023-10-08T16:11:03.911904+0000","last_peered":"2023-10-08T16:11:03.911904+0000","last_clean":"2023-10-08T16:11:03.911904+0000","last_became_active":"2023-10-08T16:11:03.911610+0000","last_became_peered":"2023-10-08T16:11:03.911610+0000","last_unstale":"2023-10-08T16:11:03.911904+0000","last_undegraded":"2023-10-08T16:11:03.911904+0000","last_fullsized":"2023-10-08T16:11:03.911904+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:40.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902819+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908616+0000","last_change":"2023-10-08T16:11:03.908616+0000","last_active":"2023-10-08T16:11:03.908616+0000","last_peered":"2023-10-08T16:11:03.908616+0000","last_clean":"2023-10-08T16:11:03.908616+0000","last_became_active":"2023-10-08T16:11:03.908311+0000","last_became_peered":"2023-10-08T16:11:03.908311+0000","last_unstale":"2023-10-08T16:11:03.908616+0000","last_undegraded":"2023-10-08T16:11:03.908616+0000","last_fullsized":"2023-10-08T16:11:03.908616+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899711+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907369+0000","last_change":"2023-10-08T16:11:03.907369+0000","last_active":"2023-10-08T16:11:03.907369+0000","last_peered":"2023-10-08T16:11:03.907369+0000","last_clean":"2023-10-08T16:11:03.907369+0000","last_became_active":"2023-10-08T16:11:03.907119+0000","last_became_peered":"2023-10-08T16:11:03.907119+0000","last_unstale":"2023-10-08T16:11:03.907369+0000","last_undegraded":"2023-10-08T16:11:03.907369+0000","last_fullsized":"2023-10-08T16:11:03.907369+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:41.469949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899690+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.901096+0000","last_change":"2023-10-08T16:11:03.901096+0000","last_active":"2023-10-08T16:11:03.901096+0000","last_peered":"2023-10-08T16:11:03.901096+0000","last_clean":"2023-10-08T16:11:03.901096+0000","last_became_active":"2023-10-08T16:11:03.900784+0000","last_became_peered":"2023-10-08T16:11:03.900784+0000","last_unstale":"2023-10-08T16:11:03.901096+0000","last_undegraded":"2023-10-08T16:11:03.901096+0000","last_fullsized":"2023-10-08T16:11:03.901096+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:33.310661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.900962+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907483+0000","last_change":"2023-10-08T16:11:03.907483+0000","last_active":"2023-10-08T16:11:03.907483+0000","last_peered":"2023-10-08T16:11:03.907483+0000","last_clean":"2023-10-08T16:11:03.907483+0000","last_became_active":"2023-10-08T16:11:03.906971+0000","last_became_peered":"2023-10-08T16:11:03.906971+0000","last_unstale":"2023-10-08T16:11:03.907483+0000","last_undegraded":"2023-10-08T16:11:03.907483+0000","last_fullsized":"2023-10-08T16:11:03.907483+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:42.596727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897212+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:30.788302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906390+0000","last_change":"2023-10-08T16:11:03.906390+0000","last_active":"2023-10-08T16:11:03.906390+0000","last_peered":"2023-10-08T16:11:03.906390+0000","last_clean":"2023-10-08T16:11:03.906390+0000","last_became_active":"2023-10-08T16:11:03.906138+0000","last_became_peered":"2023-10-08T16:11:03.906138+0000","last_unstale":"2023-10-08T16:11:03.906390+0000","last_undegraded":"2023-10-08T16:11:03.906390+0000","last_fullsized":"2023-10-08T16:11:03.906390+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:48.258705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.906219+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906532+0000","last_change":"2023-10-08T16:11:03.906532+0000","last_active":"2023-10-08T16:11:03.906532+0000","last_peered":"2023-10-08T16:11:03.906532+0000","last_clean":"2023-10-08T16:11:03.906532+0000","last_became_active":"2023-10-08T16:11:03.906222+0000","last_became_peered":"2023-10-08T16:11:03.906222+0000","last_unstale":"2023-10-08T16:11:03.906532+0000","last_undegraded":"2023-10-08T16:11:03.906532+0000","last_fullsized":"2023-10-08T16:11:03.906532+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:47.712149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896598+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909969+0000","last_change":"2023-10-08T16:11:03.909969+0000","last_active":"2023-10-08T16:11:03.909969+0000","last_peered":"2023-10-08T16:11:03.909969+0000","last_clean":"2023-10-08T16:11:03.909969+0000","last_became_active":"2023-10-08T16:11:03.909698+0000","last_became_peered":"2023-10-08T16:11:03.909698+0000","last_unstale":"2023-10-08T16:11:03.909969+0000","last_undegraded":"2023-10-08T16:11:03.909969+0000","last_fullsized":"2023-10-08T16:11:03.909969+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:08.362073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902536+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:36:00.622457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.912368+0000","last_change":"2023-10-08T16:11:03.912368+0000","last_active":"2023-10-08T16:11:03.912368+0000","last_peered":"2023-10-08T16:11:03.912368+0000","last_clean":"2023-10-08T16:11:03.912368+0000","last_became_active":"2023-10-08T16:11:03.912066+0000","last_became_peered":"2023-10-08T16:11:03.912066+0000","last_unstale":"2023-10-08T16:11:03.912368+0000","last_undegraded":"2023-10-08T16:11:03.912368+0000","last_fullsized":"2023-10-08T16:11:03.912368+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:52.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901423+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910705+0000","last_change":"2023-10-08T16:11:03.910705+0000","last_active":"2023-10-08T16:11:03.910705+0000","last_peered":"2023-10-08T16:11:03.910705+0000","last_clean":"2023-10-08T16:11:03.910705+0000","last_became_active":"2023-10-08T16:11:03.910321+0000","last_became_peered":"2023-10-08T16:11:03.910321+0000","last_unstale":"2023-10-08T16:11:03.910705+0000","last_undegraded":"2023-10-08T16:11:03.910705+0000","last_fullsized":"2023-10-08T16:11:03.910705+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:29.783953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911284+0000","last_change":"2023-10-08T16:11:03.911284+0000","last_active":"2023-10-08T16:11:03.911284+0000","last_peered":"2023-10-08T16:11:03.911284+0000","last_clean":"2023-10-08T16:11:03.911284+0000","last_became_active":"2023-10-08T16:11:03.910992+0000","last_became_peered":"2023-10-08T16:11:03.910992+0000","last_unstale":"2023-10-08T16:11:03.911284+0000","last_undegraded":"2023-10-08T16:11:03.911284+0000","last_fullsized":"2023-10-08T16:11:03.911284+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:17.563101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898505+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911078+0000","last_change":"2023-10-08T16:11:03.911078+0000","last_active":"2023-10-08T16:11:03.911078+0000","last_peered":"2023-10-08T16:11:03.911078+0000","last_clean":"2023-10-08T16:11:03.911078+0000","last_became_active":"2023-10-08T16:11:03.910522+0000","last_became_peered":"2023-10-08T16:11:03.910522+0000","last_unstale":"2023-10-08T16:11:03.911078+0000","last_undegraded":"2023-10-08T16:11:03.911078+0000","last_fullsized":"2023-10-08T16:11:03.911078+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902657+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:30.954855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913269+0000","last_change":"2023-10-08T16:11:03.913269+0000","last_active":"2023-10-08T16:11:03.913269+0000","last_peered":"2023-10-08T16:11:03.913269+0000","last_clean":"2023-10-08T16:11:03.913269+0000","last_became_active":"2023-10-08T16:11:03.913044+0000","last_became_peered":"2023-10-08T16:11:03.913044+0000","last_unstale":"2023-10-08T16:11:03.913269+0000","last_undegraded":"2023-10-08T16:11:03.913269+0000","last_fullsized":"2023-10-08T16:11:03.913269+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:52.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896011+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905844+0000","last_change":"2023-10-08T16:11:03.905844+0000","last_active":"2023-10-08T16:11:03.905844+0000","last_peered":"2023-10-08T16:11:03.905844+0000","last_clean":"2023-10-08T16:11:03.905844+0000","last_became_active":"2023-10-08T16:11:03.905597+0000","last_became_peered":"2023-10-08T16:11:03.905597+0000","last_unstale":"2023-10-08T16:11:03.905844+0000","last_undegraded":"2023-10-08T16:11:03.905844+0000","last_fullsized":"2023-10-08T16:11:03.905844+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.915073+0000","last_change":"2023-10-08T16:11:03.915073+0000","last_active":"2023-10-08T16:11:03.915073+0000","last_peered":"2023-10-08T16:11:03.915073+0000","last_clean":"2023-10-08T16:11:03.915073+0000","last_became_active":"2023-10-08T16:11:03.914732+0000","last_became_peered":"2023-10-08T16:11:03.914732+0000","last_unstale":"2023-10-08T16:11:03.915073+0000","last_undegraded":"2023-10-08T16:11:03.915073+0000","last_fullsized":"2023-10-08T16:11:03.915073+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902783+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906254+0000","last_change":"2023-10-08T16:11:03.906254+0000","last_active":"2023-10-08T16:11:03.906254+0000","last_peered":"2023-10-08T16:11:03.906254+0000","last_clean":"2023-10-08T16:11:03.906254+0000","last_became_active":"2023-10-08T16:11:03.905790+0000","last_became_peered":"2023-10-08T16:11:03.905790+0000","last_unstale":"2023-10-08T16:11:03.906254+0000","last_undegraded":"2023-10-08T16:11:03.906254+0000","last_fullsized":"2023-10-08T16:11:03.906254+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:44.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901908+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910220+0000","last_change":"2023-10-08T16:11:03.910220+0000","last_active":"2023-10-08T16:11:03.910220+0000","last_peered":"2023-10-08T16:11:03.910220+0000","last_clean":"2023-10-08T16:11:03.910220+0000","last_became_active":"2023-10-08T16:11:03.909963+0000","last_became_peered":"2023-10-08T16:11:03.909963+0000","last_unstale":"2023-10-08T16:11:03.910220+0000","last_undegraded":"2023-10-08T16:11:03.910220+0000","last_fullsized":"2023-10-08T16:11:03.910220+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:27.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899130+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907316+0000","last_change":"2023-10-08T16:11:03.907316+0000","last_active":"2023-10-08T16:11:03.907316+0000","last_peered":"2023-10-08T16:11:03.907316+0000","last_clean":"2023-10-08T16:11:03.907316+0000","last_became_active":"2023-10-08T16:11:03.907061+0000","last_became_peered":"2023-10-08T16:11:03.907061+0000","last_unstale":"2023-10-08T16:11:03.907316+0000","last_undegraded":"2023-10-08T16:11:03.907316+0000","last_fullsized":"2023-10-08T16:11:03.907316+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:38.380438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895570+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907179+0000","last_change":"2023-10-08T16:11:03.907179+0000","last_active":"2023-10-08T16:11:03.907179+0000","last_peered":"2023-10-08T16:11:03.907179+0000","last_clean":"2023-10-08T16:11:03.907179+0000","last_became_active":"2023-10-08T16:11:03.906946+0000","last_became_peered":"2023-10-08T16:11:03.906946+0000","last_unstale":"2023-10-08T16:11:03.907179+0000","last_undegraded":"2023-10-08T16:11:03.907179+0000","last_fullsized":"2023-10-08T16:11:03.907179+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.902425+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:38.941479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.908030+0000","last_change":"2023-10-08T16:11:03.908030+0000","last_active":"2023-10-08T16:11:03.908030+0000","last_peered":"2023-10-08T16:11:03.908030+0000","last_clean":"2023-10-08T16:11:03.908030+0000","last_became_active":"2023-10-08T16:11:03.907731+0000","last_became_peered":"2023-10-08T16:11:03.907731+0000","last_unstale":"2023-10-08T16:11:03.908030+0000","last_undegraded":"2023-10-08T16:11:03.908030+0000","last_fullsized":"2023-10-08T16:11:03.908030+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:10.071216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.896564+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909202+0000","last_change":"2023-10-08T16:11:03.909202+0000","last_active":"2023-10-08T16:11:03.909202+0000","last_peered":"2023-10-08T16:11:03.909202+0000","last_clean":"2023-10-08T16:11:03.909202+0000","last_became_active":"2023-10-08T16:11:03.908949+0000","last_became_peered":"2023-10-08T16:11:03.908949+0000","last_unstale":"2023-10-08T16:11:03.909202+0000","last_undegraded":"2023-10-08T16:11:03.909202+0000","last_fullsized":"2023-10-08T16:11:03.909202+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:54.146955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895524+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907175+0000","last_change":"2023-10-08T16:11:03.907175+0000","last_active":"2023-10-08T16:11:03.907175+0000","last_peered":"2023-10-08T16:11:03.907175+0000","last_clean":"2023-10-08T16:11:03.907175+0000","last_became_active":"2023-10-08T16:11:03.906936+0000","last_became_peered":"2023-10-08T16:11:03.906936+0000","last_unstale":"2023-10-08T16:11:03.907175+0000","last_undegraded":"2023-10-08T16:11:03.907175+0000","last_fullsized":"2023-10-08T16:11:03.907175+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:46.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899442+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910257+0000","last_change":"2023-10-08T16:11:03.910257+0000","last_active":"2023-10-08T16:11:03.910257+0000","last_peered":"2023-10-08T16:11:03.910257+0000","last_clean":"2023-10-08T16:11:03.910257+0000","last_became_active":"2023-10-08T16:11:03.909995+0000","last_became_peered":"2023-10-08T16:11:03.909995+0000","last_unstale":"2023-10-08T16:11:03.910257+0000","last_undegraded":"2023-10-08T16:11:03.910257+0000","last_fullsized":"2023-10-08T16:11:03.910257+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:01.296731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897617+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:44:47.632034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909589+0000","last_change":"2023-10-08T16:11:03.909589+0000","last_active":"2023-10-08T16:11:03.909589+0000","last_peered":"2023-10-08T16:11:03.909589+0000","last_clean":"2023-10-08T16:11:03.909589+0000","last_became_active":"2023-10-08T16:11:03.909342+0000","last_became_peered":"2023-10-08T16:11:03.909342+0000","last_unstale":"2023-10-08T16:11:03.909589+0000","last_undegraded":"2023-10-08T16:11:03.909589+0000","last_fullsized":"2023-10-08T16:11:03.909589+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:10.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.903232+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:59.542888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.913709+0000","last_change":"2023-10-08T16:11:03.913709+0000","last_active":"2023-10-08T16:11:03.913709+0000","last_peered":"2023-10-08T16:11:03.913709+0000","last_clean":"2023-10-08T16:11:03.913709+0000","last_became_active":"2023-10-08T16:11:03.913436+0000","last_became_peered":"2023-10-08T16:11:03.913436+0000","last_unstale":"2023-10-08T16:11:03.913709+0000","last_undegraded":"2023-10-08T16:11:03.913709+0000","last_fullsized":"2023-10-08T16:11:03.913709+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:07.589542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895429+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909139+0000","last_change":"2023-10-08T16:11:03.909139+0000","last_active":"2023-10-08T16:11:03.909139+0000","last_peered":"2023-10-08T16:11:03.909139+0000","last_clean":"2023-10-08T16:11:03.909139+0000","last_became_active":"2023-10-08T16:11:03.908877+0000","last_became_peered":"2023-10-08T16:11:03.908877+0000","last_unstale":"2023-10-08T16:11:03.909139+0000","last_undegraded":"2023-10-08T16:11:03.909139+0000","last_fullsized":"2023-10-08T16:11:03.909139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:48.689812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.897073+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.905282+0000","last_change":"2023-10-08T16:11:03.905282+0000","last_active":"2023-10-08T16:11:03.905282+0000","last_peered":"2023-10-08T16:11:03.905282+0000","last_clean":"2023-10-08T16:11:03.905282+0000","last_became_active":"2023-10-08T16:11:03.905030+0000","last_became_peered":"2023-10-08T16:11:03.905030+0000","last_unstale":"2023-10-08T16:11:03.905282+0000","last_undegraded":"2023-10-08T16:11:03.905282+0000","last_fullsized":"2023-10-08T16:11:03.905282+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:54.327772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898252+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.907637+0000","last_change":"2023-10-08T16:11:03.907637+0000","last_active":"2023-10-08T16:11:03.907637+0000","last_peered":"2023-10-08T16:11:03.907637+0000","last_clean":"2023-10-08T16:11:03.907637+0000","last_became_active":"2023-10-08T16:11:03.907270+0000","last_became_peered":"2023-10-08T16:11:03.907270+0000","last_unstale":"2023-10-08T16:11:03.907637+0000","last_undegraded":"2023-10-08T16:11:03.907637+0000","last_fullsized":"2023-10-08T16:11:03.907637+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.901126+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.909548+0000","last_change":"2023-10-08T16:11:03.909548+0000","last_active":"2023-10-08T16:11:03.909548+0000","last_peered":"2023-10-08T16:11:03.909548+0000","last_clean":"2023-10-08T16:11:03.909548+0000","last_became_active":"2023-10-08T16:11:03.909225+0000","last_became_peered":"2023-10-08T16:11:03.909225+0000","last_unstale":"2023-10-08T16:11:03.909548+0000","last_undegraded":"2023-10-08T16:11:03.909548+0000","last_fullsized":"2023-10-08T16:11:03.909548+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:39.249283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.899215+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.910568+0000","last_change":"2023-10-08T16:11:03.910568+0000","last_active":"2023-10-08T16:11:03.910568+0000","last_peered":"2023-10-08T16:11:03.910568+0000","last_clean":"2023-10-08T16:11:03.910568+0000","last_became_active":"2023-10-08T16:11:03.910311+0000","last_became_peered":"2023-10-08T16:11:03.910311+0000","last_unstale":"2023-10-08T16:11:03.910568+0000","last_undegraded":"2023-10-08T16:11:03.910568+0000","last_fullsized":"2023-10-08T16:11:03.910568+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:56.735575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.895513+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.906699+0000","last_change":"2023-10-08T16:11:03.906699+0000","last_active":"2023-10-08T16:11:03.906699+0000","last_peered":"2023-10-08T16:11:03.906699+0000","last_clean":"2023-10-08T16:11:03.906699+0000","last_became_active":"2023-10-08T16:11:03.906348+0000","last_became_peered":"2023-10-08T16:11:03.906348+0000","last_unstale":"2023-10-08T16:11:03.906699+0000","last_undegraded":"2023-10-08T16:11:03.906699+0000","last_fullsized":"2023-10-08T16:11:03.906699+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:34.478940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T16:11:03.890066+0000","last_change":"2023-10-08T16:11:03.898873+0000","last_active":"2023-10-08T16:11:03.890066+0000","last_peered":"2023-10-08T16:11:03.890066+0000","last_clean":"2023-10-08T16:11:03.890066+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T16:11:03.890066+0000","last_undegraded":"2023-10-08T16:11:03.890066+0000","last_fullsized":"2023-10-08T16:11:03.890066+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":111,"state":"active+clean","last_fresh":"2023-10-08T16:11:03.911957+0000","last_change":"2023-10-08T16:11:03.911957+0000","last_active":"2023-10-08T16:11:03.911957+0000","last_peered":"2023-10-08T16:11:03.911957+0000","last_clean":"2023-10-08T16:11:03.911957+0000","last_became_active":"2023-10-08T16:11:03.911652+0000","last_became_peered":"2023-10-08T16:11:03.911652+0000","last_unstale":"2023-10-08T16:11:03.911957+0000","last_undegraded":"2023-10-08T16:11:03.911957+0000","last_fullsized":"2023-10-08T16:11:03.911957+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:21.785467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":237,"reported_epoch":110,"state":"active+clean","last_fresh":"2023-10-08T16:11:02.893767+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:02.893767+0000","last_peered":"2023-10-08T16:11:02.893767+0000","last_clean":"2023-10-08T16:11:02.893767+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:02.893767+0000","last_undegraded":"2023-10-08T16:11:02.893767+0000","last_fullsized":"2023-10-08T16:11:02.893767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303308,"kb_used_data":1924,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068532,"statfs":{"total":96636764160,"available":96326176768,"internally_reserved":0,"allocated":1970176,"data_stored":1182083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.51700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.57899999999999996}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.502}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.41499999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.378}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.441},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298632,"kb_used_data":1344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073208,"statfs":{"total":96636764160,"available":96330964992,"internally_reserved":0,"allocated":1376256,"data_stored":591715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.68700000000000006,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.307},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.40400000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64700000000000002,"5min":0.75,"15min":0.96599999999999997},"last":0.52800000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.94199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.28399999999999997,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.63200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.874},"last":0.61499999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.48899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.309},"min":{"1min":0.219,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.33700000000000002},"min":{"1min":0.219,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.58199999999999996,"15min":0.77500000000000002},"last":0.36099999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.35599999999999998},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.501,"15min":1.2230000000000001},"last":0.38},{"interface":"front","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.72399999999999998},"last":0.45600000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.38600000000000001},"min":{"1min":0.28799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.70399999999999996},"last":0.433},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.35799999999999998},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.42199999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":1.077},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1.696,"15min":1.696},"last":0.754}]},{"osd":5,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":1.657,"15min":1.657},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.70399999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.099},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.50600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.59499999999999997},"min":{"1min":0.48599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.1839999999999999},"last":0.76600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574931,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303320,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068520,"statfs":{"total":96636764160,"available":96326164480,"internally_reserved":0,"allocated":1982464,"data_stored":1188346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.53400000000000003},"min":{"1min":0.23499999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":3.3220000000000001},"last":0.33700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.56699999999999995},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":5.6769999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.45500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.47799999999999998},"min":{"1min":0.22700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.434,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.60399999999999998},"min":{"1min":0.29899999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55100000000000005,"5min":1.498,"15min":5.5970000000000004},"last":0.38600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.66300000000000003},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.66000000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":1.847,"15min":3.2890000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56200000000000006,"15min":0.77800000000000002},"min":{"1min":0.47199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.45400000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.80200000000000005},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.754,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.26500000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.44},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.40400000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.37},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.46800000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.64300000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.314,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.505,"5min":0.53600000000000003,"15min":1.238},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33200000000000002,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.28699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33100000000000002,"15min":0.35599999999999998},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39700000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.497,"5min":0.59299999999999997,"15min":1.3520000000000001},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36299999999999999,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.248},"max":{"1min":0.47599999999999998,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.42099999999999999},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":1.327},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.65700000000000003,"5min":1.099,"15min":1.099},"last":0.52200000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":1.27},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.91900000000000004,"15min":1.879},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.60899999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.97599999999999998,"15min":1.825},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":1.0009999999999999,"15min":1.115},"last":0.55900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298644,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073196,"statfs":{"total":96636764160,"available":96330952704,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.499},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54700000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302740,"kb_used_data":1356,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069100,"statfs":{"total":96636764160,"available":96326758400,"internally_reserved":0,"allocated":1388544,"data_stored":597978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.56499999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.504}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.51700000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.63100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:09.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3f is not active+clean 2023-10-08T16:11:09.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903693+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:48:11.776245+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-10-08T16:11:09.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896495+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903281+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3c is not active+clean 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901854+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:08:19.637626+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899939+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:49:49.260895+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897848+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897135+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.905669+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903887+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896110+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-10-08T16:11:09.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.898238+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:12:39.892907+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902869+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.33 is not active+clean 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901932+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:43:14.536400+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896049+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.31 is not active+clean 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902883+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:35:33.760866+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.900115+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.898155+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901480+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897822+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:24:47.754512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-10-08T16:11:09.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.900169+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899392+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897293+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.904140+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897636+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:36:03.870568+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T16:11:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896123+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899760+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.25 is not active+clean 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902739+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:43:12.591868+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-10-08T16:11:09.712 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899387+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.23 is not active+clean 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903063+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:53:25.494465+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.22 is not active+clean 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902898+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:09:30.140643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899993+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:25:08.885969+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-10-08T16:11:09.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896645+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899008+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902819+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899711+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-08T16:11:09.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899690+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.900962+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897212+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:57:30.788302+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.906219+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:57:57.927578+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896598+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2023-10-08T16:11:09.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902536+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T20:36:00.622457+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901423+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899126+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.898505+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902657+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:31:30.954855+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896011+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903524+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:27:43.601076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902783+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T03:43:49.694162+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901908+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:52:40.333467+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-10-08T16:11:09.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899130+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:42:01.797530+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.895570+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:26:34.890024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.902425+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:44:38.941479+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-08T16:11:09.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.896564+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-08T16:11:09.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.895524+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-08T16:11:09.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899442+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897617+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T18:44:47.632034+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.903232+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T16:18:59.542888+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.895429+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.897073+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.898252+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.901126+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-10T01:54:09.366423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T16:11:09.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-10-08T16:11:09.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.899215+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T21:37:09.828688+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T16:11:09.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-08T16:11:09.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.895513+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T16:11:09.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-08T16:11:09.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T16:11:03.890066+0000', 'last_change': '2023-10-08T16:11:03.898873+0000', 'last_active': '2023-10-08T16:11:03.890066+0000', 'last_peered': '2023-10-08T16:11:03.890066+0000', 'last_clean': '2023-10-08T16:11:03.890066+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T16:11:03.890066+0000', 'last_undegraded': '2023-10-08T16:11:03.890066+0000', 'last_fullsized': '2023-10-08T16:11:03.890066+0000', 'mapping_epoch': 111, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 111, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'last_clean_scrub_stamp': '2023-10-08T16:11:03.890066+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T16:11:09.723 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T16:11:09.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.852+0000 7f2c33827700 1 Processor -- start 2023-10-08T16:11:09.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.852+0000 7f2c33827700 1 -- start start 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c33827700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 0x7f2c2c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c33827700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2c2c0b53d0 0x7f2c2c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c33827700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c0b5b00 con 0x7f2c2c0cce40 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c33827700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c0b5850 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c33827700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c0b5da0 con 0x7f2c2c159ec0 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c30dc2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2c2c0b53d0 0x7f2c2c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c31dc4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2c2c0cce40 0x7f2c2c0cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59150/0 (socket says 172.21.15.170:59150) 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.853+0000 7f2c31dc4700 1 -- 172.21.15.170:0/4175939366 learned_addr learned my addr 172.21.15.170:0/4175939366 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c315c3700 1 --2- 172.21.15.170:0/4175939366 >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 0x7f2c2c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c30dc2700 1 -- 172.21.15.170:0/4175939366 >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 msgr2=0x7f2c2c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c30dc2700 1 --2- 172.21.15.170:0/4175939366 >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 0x7f2c2c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c30dc2700 1 -- 172.21.15.170:0/4175939366 >> v1:172.21.15.170:6789/0 conn(0x7f2c2c0cce40 legacy=0x7f2c2c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c30dc2700 1 -- 172.21.15.170:0/4175939366 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c0b6080 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c315c3700 1 --2- 172.21.15.170:0/4175939366 >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 0x7f2c2c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c23fff700 1 -- 172.21.15.170:0/4175939366 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377967783 0 0) 0x7f2c2c0b5b00 con 0x7f2c2c0cce40 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c30dc2700 1 --2- 172.21.15.170:0/4175939366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c0cc900 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c089120 tx=0x7f2c1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43b46f1ce74144 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c23fff700 1 -- 172.21.15.170:0/4175939366 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00b6b0 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c23fff700 1 -- 172.21.15.170:0/4175939366 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c00b810 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c23fff700 1 -- 172.21.15.170:0/4175939366 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00bbb0 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c33827700 1 -- 172.21.15.170:0/4175939366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0b53d0 msgr2=0x7f2c2c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c33827700 1 --2- 172.21.15.170:0/4175939366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c0cc900 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c089120 tx=0x7f2c1c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c33827700 1 -- 172.21.15.170:0/4175939366 shutdown_connections 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c33827700 1 --2- 172.21.15.170:0/4175939366 >> v2:172.21.15.191:3300/0 conn(0x7f2c2c159ec0 0x7f2c2c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.854+0000 7f2c33827700 1 --2- 172.21.15.170:0/4175939366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c0cc900 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.855+0000 7f2c33827700 1 -- 172.21.15.170:0/4175939366 >> 172.21.15.170:0/4175939366 conn(0x7f2c2c0999f0 msgr2=0x7f2c2c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.855+0000 7f2c33827700 1 -- 172.21.15.170:0/4175939366 shutdown_connections 2023-10-08T16:11:09.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.855+0000 7f2c33827700 1 -- 172.21.15.170:0/4175939366 wait complete. 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.856+0000 7f2c33827700 1 Processor -- start 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.856+0000 7f2c33827700 1 -- start start 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.856+0000 7f2c33827700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c16dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.856+0000 7f2c33827700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44692/0 (socket says 172.21.15.170:44692) 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c2c159ec0 0x7f2c2c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 -- 172.21.15.170:0/50825854 learned_addr learned my addr 172.21.15.170:0/50825854 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c16d620 con 0x7f2c2c0cce40 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c16a4d0 con 0x7f2c2c0b53d0 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c16a7b0 con 0x7f2c2c159ec0 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c315c3700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c16dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c31dc4700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c2c159ec0 0x7f2c2c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 -- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c2c159ec0 msgr2=0x7f2c2c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c2c159ec0 0x7f2c2c156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 -- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c2c0b53d0 msgr2=0x7f2c2c16dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c16dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c009420 con 0x7f2c2c0cce40 2023-10-08T16:11:09.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c30dc2700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c089390 tx=0x7f2c1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c00bb90 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1c011420 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c16aa90 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0117a0 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.857+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c16dff0 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.858+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c063c90 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.859+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2c1c011b90 con 0x7f2c2c0cce40 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.859+0000 7f2c217fa700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 0x7f2c100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.859+0000 7f2c315c3700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 0x7f2c100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:09.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.859+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c1c086550 con 0x7f2c2c0cce40 2023-10-08T16:11:09.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.859+0000 7f2c315c3700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 0x7f2c100697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c280077a0 tx=0x7f2c28009a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:09.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:09.863+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c1c0531e0 con 0x7f2c2c0cce40 2023-10-08T16:11:10.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.124+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2c2c04c3c0 con 0x7f2c2c0cce40 2023-10-08T16:11:10.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.124+0000 7f2c217fa700 1 -- 172.21.15.170:0/50825854 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1457 (secure 0 0 0) 0x7f2c1c056800 con 0x7f2c2c0cce40 2023-10-08T16:11:10.126 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:10.126 INFO:teuthology.orchestra.run.smithi170.stdout:{"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":411,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1696781068,"num_in_osds":8,"osd_in_since":1696781061,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2472820736,"bytes_avail":770621292544,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":74,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":7125}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-08T16:10:54.302049+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-08T16:11:10.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 msgr2=0x7f2c100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 0x7f2c100697d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c280077a0 tx=0x7f2c28009a20 comp rx=0 tx=0).stop 2023-10-08T16:11:10.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 msgr2=0x7f2c2c169c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c089390 tx=0x7f2c1c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 shutdown_connections 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2c2c159ec0 0x7f2c2c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2c2c0b53d0 0x7f2c2c16dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2c10067320 0x7f2c100697d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 --2- 172.21.15.170:0/50825854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2c2c0cce40 0x7f2c2c169c80 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 >> 172.21.15.170:0/50825854 conn(0x7f2c2c0999f0 msgr2=0x7f2c2c151fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 shutdown_connections 2023-10-08T16:11:10.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.126+0000 7f2c33827700 1 -- 172.21.15.170:0/50825854 wait complete. 2023-10-08T16:11:10.140 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T16:11:10.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:10.211 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:11:10.211+0000 7fb7eb3bc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:11:10.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.253+0000 7fd69f44e700 1 Processor -- start 2023-10-08T16:11:10.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.253+0000 7fd69f44e700 1 -- start start 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69f44e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd698069560 0x7fd6980699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69f44e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd698069ee0 con 0x7fd698159890 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69f44e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69815a160 con 0x7fd698159c60 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69f44e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980d13a0 con 0x7fd698069560 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d9eb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd698069560 0x7fd6980699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d1ea700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd698159890 0x7fd69815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59156/0 (socket says 172.21.15.170:59156) 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d1ea700 1 -- 172.21.15.170:0/3718664816 learned_addr learned my addr 172.21.15.170:0/3718664816 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:10.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d9eb700 1 -- 172.21.15.170:0/3718664816 >> v1:172.21.15.191:6789/0 conn(0x7fd698159c60 legacy=0x7fd69815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d9eb700 1 -- 172.21.15.170:0/3718664816 >> v1:172.21.15.170:6789/0 conn(0x7fd698159890 legacy=0x7fd69815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d9eb700 1 -- 172.21.15.170:0/3718664816 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69815e140 con 0x7fd698069560 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd697fff700 1 -- 172.21.15.170:0/3718664816 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051672002 0 0) 0x7fd698069ee0 con 0x7fd698159890 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd69d9eb700 1 --2- 172.21.15.170:0/3718664816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698069560 0x7fd6980699a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd6880098e0 tx=0x7fd6880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd2b47d7144c505 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.254+0000 7fd697fff700 1 -- 172.21.15.170:0/3718664816 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688009e80 con 0x7fd698069560 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd697fff700 1 -- 172.21.15.170:0/3718664816 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68800b820 con 0x7fd698069560 2023-10-08T16:11:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd697fff700 1 -- 172.21.15.170:0/3718664816 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800bba0 con 0x7fd698069560 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 -- 172.21.15.170:0/3718664816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698069560 msgr2=0x7fd6980699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 --2- 172.21.15.170:0/3718664816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698069560 0x7fd6980699a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd6880098e0 tx=0x7fd6880091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 -- 172.21.15.170:0/3718664816 shutdown_connections 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 --2- 172.21.15.170:0/3718664816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698069560 0x7fd6980699a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 -- 172.21.15.170:0/3718664816 >> 172.21.15.170:0/3718664816 conn(0x7fd698068e20 msgr2=0x7fd6980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 -- 172.21.15.170:0/3718664816 shutdown_connections 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.255+0000 7fd69f44e700 1 -- 172.21.15.170:0/3718664816 wait complete. 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.256+0000 7fd69f44e700 1 Processor -- start 2023-10-08T16:11:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.256+0000 7fd69f44e700 1 -- start start 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd698069560 0x7fd6981534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd698159890 0x7fd6980b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980cd0c0 con 0x7fd698159c60 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980cd960 con 0x7fd698159890 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69f44e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd698153a30 con 0x7fd698069560 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69d9eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69d9eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44708/0 (socket says 172.21.15.170:44708) 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69d9eb700 1 -- 172.21.15.170:0/2447651821 learned_addr learned my addr 172.21.15.170:0/2447651821 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69d1ea700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd698069560 0x7fd6981534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.257+0000 7fd69c9e9700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd698159890 0x7fd6980b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 -- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd698069560 msgr2=0x7fd6981534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd698069560 0x7fd6981534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 -- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd698159890 msgr2=0x7fd6980b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd698159890 0x7fd6980b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 -- 172.21.15.170:0/2447651821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009420 con 0x7fd698159c60 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69d9eb700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd69806a070 tx=0x7fd688016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800bb60 con 0x7fd698159c60 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688020070 con 0x7fd698159c60 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6980b62c0 con 0x7fd698159c60 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68801c680 con 0x7fd698159c60 2023-10-08T16:11:10.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.258+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6980cd5f0 con 0x7fd698159c60 2023-10-08T16:11:10.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.259+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6980cdad0 con 0x7fd698159c60 2023-10-08T16:11:10.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.259+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd688022030 con 0x7fd698159c60 2023-10-08T16:11:10.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.259+0000 7fd6957fa700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 0x7fd6840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:10.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.260+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd688086bd0 con 0x7fd698159c60 2023-10-08T16:11:10.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.260+0000 7fd69d1ea700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 0x7fd6840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:10.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.260+0000 7fd69d1ea700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 0x7fd6840697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd690002250 tx=0x7fd690001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:10.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.264+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd688053970 con 0x7fd698159c60 2023-10-08T16:11:10.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.441+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6980cdad0 con 0x7fd684067320 2023-10-08T16:11:10.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.449+0000 7fd6957fa700 1 -- 172.21.15.170:0/2447651821 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290136 (secure 0 0 0) 0x7fd6980cdad0 con 0x7fd684067320 2023-10-08T16:11:10.454 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:10.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.452+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 msgr2=0x7fd6840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.452+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 0x7fd6840697d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd690002250 tx=0x7fd690001a20 comp rx=0 tx=0).stop 2023-10-08T16:11:10.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.452+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 msgr2=0x7fd698154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.452+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd69806a070 tx=0x7fd688016000 comp rx=0 tx=0).stop 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 shutdown_connections 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd684067320 0x7fd6840697d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd698159c60 0x7fd698154060 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd698069560 0x7fd6981534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 --2- 172.21.15.170:0/2447651821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd698159890 0x7fd6980b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 >> 172.21.15.170:0/2447651821 conn(0x7fd698068e20 msgr2=0x7fd6980d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 shutdown_connections 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:10.453+0000 7fd69f44e700 1 -- 172.21.15.170:0/2447651821 wait complete. 2023-10-08T16:11:10.457 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:10.466 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2023-10-08T16:11:10.304678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423336,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551384,"statfs":{"total":773094113280,"available":770612617216,"internally_reserved":0,"allocated":12558336,"data_stored":6073580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.412381"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926081+0000","last_change":"2023-10-08T16:11:04.922905+0000","last_active":"2023-10-08T16:11:04.926081+0000","last_peered":"2023-10-08T16:11:04.926081+0000","last_clean":"2023-10-08T16:11:04.926081+0000","last_became_active":"2023-10-08T16:11:04.922592+0000","last_became_peered":"2023-10-08T16:11:04.922592+0000","last_unstale":"2023-10-08T16:11:04.926081+0000","last_undegraded":"2023-10-08T16:11:04.926081+0000","last_fullsized":"2023-10-08T16:11:04.926081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:49.633201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920743+0000","last_change":"2023-10-08T16:11:04.916819+0000","last_active":"2023-10-08T16:11:04.920743+0000","last_peered":"2023-10-08T16:11:04.920743+0000","last_clean":"2023-10-08T16:11:04.920743+0000","last_became_active":"2023-10-08T16:11:04.915080+0000","last_became_peered":"2023-10-08T16:11:04.915080+0000","last_unstale":"2023-10-08T16:11:04.920743+0000","last_undegraded":"2023-10-08T16:11:04.920743+0000","last_fullsized":"2023-10-08T16:11:04.920743+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:50.336238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931731+0000","last_change":"2023-10-08T16:11:04.921766+0000","last_active":"2023-10-08T16:11:04.931731+0000","last_peered":"2023-10-08T16:11:04.931731+0000","last_clean":"2023-10-08T16:11:04.931731+0000","last_became_active":"2023-10-08T16:11:04.920227+0000","last_became_peered":"2023-10-08T16:11:04.920227+0000","last_unstale":"2023-10-08T16:11:04.931731+0000","last_undegraded":"2023-10-08T16:11:04.931731+0000","last_fullsized":"2023-10-08T16:11:04.931731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930736+0000","last_change":"2023-10-08T16:11:04.927630+0000","last_active":"2023-10-08T16:11:04.930736+0000","last_peered":"2023-10-08T16:11:04.930736+0000","last_clean":"2023-10-08T16:11:04.930736+0000","last_became_active":"2023-10-08T16:11:04.927334+0000","last_became_peered":"2023-10-08T16:11:04.927334+0000","last_unstale":"2023-10-08T16:11:04.930736+0000","last_undegraded":"2023-10-08T16:11:04.930736+0000","last_fullsized":"2023-10-08T16:11:04.930736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:11.344791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.939953+0000","last_change":"2023-10-08T16:11:04.931826+0000","last_active":"2023-10-08T16:11:04.939953+0000","last_peered":"2023-10-08T16:11:04.939953+0000","last_clean":"2023-10-08T16:11:04.939953+0000","last_became_active":"2023-10-08T16:11:04.931357+0000","last_became_peered":"2023-10-08T16:11:04.931357+0000","last_unstale":"2023-10-08T16:11:04.939953+0000","last_undegraded":"2023-10-08T16:11:04.939953+0000","last_fullsized":"2023-10-08T16:11:04.939953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:20.856257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921401+0000","last_change":"2023-10-08T16:11:04.915094+0000","last_active":"2023-10-08T16:11:04.921401+0000","last_peered":"2023-10-08T16:11:04.921401+0000","last_clean":"2023-10-08T16:11:04.921401+0000","last_became_active":"2023-10-08T16:11:04.913963+0000","last_became_peered":"2023-10-08T16:11:04.913963+0000","last_unstale":"2023-10-08T16:11:04.921401+0000","last_undegraded":"2023-10-08T16:11:04.921401+0000","last_fullsized":"2023-10-08T16:11:04.921401+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921646+0000","last_change":"2023-10-08T16:11:04.916445+0000","last_active":"2023-10-08T16:11:04.921646+0000","last_peered":"2023-10-08T16:11:04.921646+0000","last_clean":"2023-10-08T16:11:04.921646+0000","last_became_active":"2023-10-08T16:11:04.916207+0000","last_became_peered":"2023-10-08T16:11:04.916207+0000","last_unstale":"2023-10-08T16:11:04.921646+0000","last_undegraded":"2023-10-08T16:11:04.921646+0000","last_fullsized":"2023-10-08T16:11:04.921646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.932047+0000","last_change":"2023-10-08T16:11:04.922747+0000","last_active":"2023-10-08T16:11:04.932047+0000","last_peered":"2023-10-08T16:11:04.932047+0000","last_clean":"2023-10-08T16:11:04.932047+0000","last_became_active":"2023-10-08T16:11:04.922021+0000","last_became_peered":"2023-10-08T16:11:04.922021+0000","last_unstale":"2023-10-08T16:11:04.932047+0000","last_undegraded":"2023-10-08T16:11:04.932047+0000","last_fullsized":"2023-10-08T16:11:04.932047+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:50.336238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927244+0000","last_change":"2023-10-08T16:11:04.918705+0000","last_active":"2023-10-08T16:11:04.927244+0000","last_peered":"2023-10-08T16:11:04.927244+0000","last_clean":"2023-10-08T16:11:04.927244+0000","last_became_active":"2023-10-08T16:11:04.918384+0000","last_became_peered":"2023-10-08T16:11:04.918384+0000","last_unstale":"2023-10-08T16:11:04.927244+0000","last_undegraded":"2023-10-08T16:11:04.927244+0000","last_fullsized":"2023-10-08T16:11:04.927244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.916897+0000","last_change":"2023-10-08T16:11:04.915034+0000","last_active":"2023-10-08T16:11:04.916897+0000","last_peered":"2023-10-08T16:11:04.916897+0000","last_clean":"2023-10-08T16:11:04.916897+0000","last_became_active":"2023-10-08T16:11:04.914784+0000","last_became_peered":"2023-10-08T16:11:04.914784+0000","last_unstale":"2023-10-08T16:11:04.916897+0000","last_undegraded":"2023-10-08T16:11:04.916897+0000","last_fullsized":"2023-10-08T16:11:04.916897+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:07.668042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919840+0000","last_change":"2023-10-08T16:11:04.919116+0000","last_active":"2023-10-08T16:11:04.919840+0000","last_peered":"2023-10-08T16:11:04.919840+0000","last_clean":"2023-10-08T16:11:04.919840+0000","last_became_active":"2023-10-08T16:11:04.918610+0000","last_became_peered":"2023-10-08T16:11:04.918610+0000","last_unstale":"2023-10-08T16:11:04.919840+0000","last_undegraded":"2023-10-08T16:11:04.919840+0000","last_fullsized":"2023-10-08T16:11:04.919840+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:55:45.661956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926966+0000","last_change":"2023-10-08T16:11:04.918875+0000","last_active":"2023-10-08T16:11:04.926966+0000","last_peered":"2023-10-08T16:11:04.926966+0000","last_clean":"2023-10-08T16:11:04.926966+0000","last_became_active":"2023-10-08T16:11:04.918515+0000","last_became_peered":"2023-10-08T16:11:04.918515+0000","last_unstale":"2023-10-08T16:11:04.926966+0000","last_undegraded":"2023-10-08T16:11:04.926966+0000","last_fullsized":"2023-10-08T16:11:04.926966+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926210+0000","last_change":"2023-10-08T16:11:04.921695+0000","last_active":"2023-10-08T16:11:04.926210+0000","last_peered":"2023-10-08T16:11:04.926210+0000","last_clean":"2023-10-08T16:11:04.926210+0000","last_became_active":"2023-10-08T16:11:04.921213+0000","last_became_peered":"2023-10-08T16:11:04.921213+0000","last_unstale":"2023-10-08T16:11:04.926210+0000","last_undegraded":"2023-10-08T16:11:04.926210+0000","last_fullsized":"2023-10-08T16:11:04.926210+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:45.144197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.914566+0000","last_change":"2023-10-08T16:11:04.912547+0000","last_active":"2023-10-08T16:11:04.914566+0000","last_peered":"2023-10-08T16:11:04.914566+0000","last_clean":"2023-10-08T16:11:04.914566+0000","last_became_active":"2023-10-08T16:11:04.912294+0000","last_became_peered":"2023-10-08T16:11:04.912294+0000","last_unstale":"2023-10-08T16:11:04.914566+0000","last_undegraded":"2023-10-08T16:11:04.914566+0000","last_fullsized":"2023-10-08T16:11:04.914566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920938+0000","last_change":"2023-10-08T16:11:04.918170+0000","last_active":"2023-10-08T16:11:04.920938+0000","last_peered":"2023-10-08T16:11:04.920938+0000","last_clean":"2023-10-08T16:11:04.920938+0000","last_became_active":"2023-10-08T16:11:04.917772+0000","last_became_peered":"2023-10-08T16:11:04.917772+0000","last_unstale":"2023-10-08T16:11:04.920938+0000","last_undegraded":"2023-10-08T16:11:04.920938+0000","last_fullsized":"2023-10-08T16:11:04.920938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:19:57.595042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919382+0000","last_change":"2023-10-08T16:11:04.914772+0000","last_active":"2023-10-08T16:11:04.919382+0000","last_peered":"2023-10-08T16:11:04.919382+0000","last_clean":"2023-10-08T16:11:04.919382+0000","last_became_active":"2023-10-08T16:11:04.914516+0000","last_became_peered":"2023-10-08T16:11:04.914516+0000","last_unstale":"2023-10-08T16:11:04.919382+0000","last_undegraded":"2023-10-08T16:11:04.919382+0000","last_fullsized":"2023-10-08T16:11:04.919382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921734+0000","last_change":"2023-10-08T16:11:04.914199+0000","last_active":"2023-10-08T16:11:04.921734+0000","last_peered":"2023-10-08T16:11:04.921734+0000","last_clean":"2023-10-08T16:11:04.921734+0000","last_became_active":"2023-10-08T16:11:04.913909+0000","last_became_peered":"2023-10-08T16:11:04.913909+0000","last_unstale":"2023-10-08T16:11:04.921734+0000","last_undegraded":"2023-10-08T16:11:04.921734+0000","last_fullsized":"2023-10-08T16:11:04.921734+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926864+0000","last_change":"2023-10-08T16:11:04.922659+0000","last_active":"2023-10-08T16:11:04.926864+0000","last_peered":"2023-10-08T16:11:04.926864+0000","last_clean":"2023-10-08T16:11:04.926864+0000","last_became_active":"2023-10-08T16:11:04.922417+0000","last_became_peered":"2023-10-08T16:11:04.922417+0000","last_unstale":"2023-10-08T16:11:04.926864+0000","last_undegraded":"2023-10-08T16:11:04.926864+0000","last_fullsized":"2023-10-08T16:11:04.926864+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.925709+0000","last_change":"2023-10-08T16:11:04.922107+0000","last_active":"2023-10-08T16:11:04.925709+0000","last_peered":"2023-10-08T16:11:04.925709+0000","last_clean":"2023-10-08T16:11:04.925709+0000","last_became_active":"2023-10-08T16:11:04.921812+0000","last_became_peered":"2023-10-08T16:11:04.921812+0000","last_unstale":"2023-10-08T16:11:04.925709+0000","last_undegraded":"2023-10-08T16:11:04.925709+0000","last_fullsized":"2023-10-08T16:11:04.925709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:46.172756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919497+0000","last_change":"2023-10-08T16:11:04.916903+0000","last_active":"2023-10-08T16:11:04.919497+0000","last_peered":"2023-10-08T16:11:04.919497+0000","last_clean":"2023-10-08T16:11:04.919497+0000","last_became_active":"2023-10-08T16:11:04.916650+0000","last_became_peered":"2023-10-08T16:11:04.916650+0000","last_unstale":"2023-10-08T16:11:04.919497+0000","last_undegraded":"2023-10-08T16:11:04.919497+0000","last_fullsized":"2023-10-08T16:11:04.919497+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.940056+0000","last_change":"2023-10-08T16:11:04.931918+0000","last_active":"2023-10-08T16:11:04.940056+0000","last_peered":"2023-10-08T16:11:04.940056+0000","last_clean":"2023-10-08T16:11:04.940056+0000","last_became_active":"2023-10-08T16:11:04.931651+0000","last_became_peered":"2023-10-08T16:11:04.931651+0000","last_unstale":"2023-10-08T16:11:04.940056+0000","last_undegraded":"2023-10-08T16:11:04.940056+0000","last_fullsized":"2023-10-08T16:11:04.940056+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:12.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921995+0000","last_change":"2023-10-08T16:11:04.920165+0000","last_active":"2023-10-08T16:11:04.921995+0000","last_peered":"2023-10-08T16:11:04.921995+0000","last_clean":"2023-10-08T16:11:04.921995+0000","last_became_active":"2023-10-08T16:11:04.919646+0000","last_became_peered":"2023-10-08T16:11:04.919646+0000","last_unstale":"2023-10-08T16:11:04.921995+0000","last_undegraded":"2023-10-08T16:11:04.921995+0000","last_fullsized":"2023-10-08T16:11:04.921995+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:09.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927218+0000","last_change":"2023-10-08T16:11:04.918212+0000","last_active":"2023-10-08T16:11:04.927218+0000","last_peered":"2023-10-08T16:11:04.927218+0000","last_clean":"2023-10-08T16:11:04.927218+0000","last_became_active":"2023-10-08T16:11:04.917959+0000","last_became_peered":"2023-10-08T16:11:04.917959+0000","last_unstale":"2023-10-08T16:11:04.927218+0000","last_undegraded":"2023-10-08T16:11:04.927218+0000","last_fullsized":"2023-10-08T16:11:04.927218+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919326+0000","last_change":"2023-10-08T16:11:04.917917+0000","last_active":"2023-10-08T16:11:04.919326+0000","last_peered":"2023-10-08T16:11:04.919326+0000","last_clean":"2023-10-08T16:11:04.919326+0000","last_became_active":"2023-10-08T16:11:04.917657+0000","last_became_peered":"2023-10-08T16:11:04.917657+0000","last_unstale":"2023-10-08T16:11:04.919326+0000","last_undegraded":"2023-10-08T16:11:04.919326+0000","last_fullsized":"2023-10-08T16:11:04.919326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:46:58.792657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928478+0000","last_change":"2023-10-08T16:11:04.928086+0000","last_active":"2023-10-08T16:11:04.928478+0000","last_peered":"2023-10-08T16:11:04.928478+0000","last_clean":"2023-10-08T16:11:04.928478+0000","last_became_active":"2023-10-08T16:11:04.927812+0000","last_became_peered":"2023-10-08T16:11:04.927812+0000","last_unstale":"2023-10-08T16:11:04.928478+0000","last_undegraded":"2023-10-08T16:11:04.928478+0000","last_fullsized":"2023-10-08T16:11:04.928478+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:39.384788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919523+0000","last_change":"2023-10-08T16:11:04.915488+0000","last_active":"2023-10-08T16:11:04.919523+0000","last_peered":"2023-10-08T16:11:04.919523+0000","last_clean":"2023-10-08T16:11:04.919523+0000","last_became_active":"2023-10-08T16:11:04.915243+0000","last_became_peered":"2023-10-08T16:11:04.915243+0000","last_unstale":"2023-10-08T16:11:04.919523+0000","last_undegraded":"2023-10-08T16:11:04.919523+0000","last_fullsized":"2023-10-08T16:11:04.919523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920719+0000","last_change":"2023-10-08T16:11:04.915681+0000","last_active":"2023-10-08T16:11:04.920719+0000","last_peered":"2023-10-08T16:11:04.920719+0000","last_clean":"2023-10-08T16:11:04.920719+0000","last_became_active":"2023-10-08T16:11:04.915282+0000","last_became_peered":"2023-10-08T16:11:04.915282+0000","last_unstale":"2023-10-08T16:11:04.920719+0000","last_undegraded":"2023-10-08T16:11:04.920719+0000","last_fullsized":"2023-10-08T16:11:04.920719+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:21.385845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926854+0000","last_change":"2023-10-08T16:11:04.918001+0000","last_active":"2023-10-08T16:11:04.926854+0000","last_peered":"2023-10-08T16:11:04.926854+0000","last_clean":"2023-10-08T16:11:04.926854+0000","last_became_active":"2023-10-08T16:11:04.917731+0000","last_became_peered":"2023-10-08T16:11:04.917731+0000","last_unstale":"2023-10-08T16:11:04.926854+0000","last_undegraded":"2023-10-08T16:11:04.926854+0000","last_fullsized":"2023-10-08T16:11:04.926854+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920993+0000","last_change":"2023-10-08T16:11:04.918673+0000","last_active":"2023-10-08T16:11:04.920993+0000","last_peered":"2023-10-08T16:11:04.920993+0000","last_clean":"2023-10-08T16:11:04.920993+0000","last_became_active":"2023-10-08T16:11:04.918200+0000","last_became_peered":"2023-10-08T16:11:04.918200+0000","last_unstale":"2023-10-08T16:11:04.920993+0000","last_undegraded":"2023-10-08T16:11:04.920993+0000","last_fullsized":"2023-10-08T16:11:04.920993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:35.606792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926074+0000","last_change":"2023-10-08T16:11:04.919852+0000","last_active":"2023-10-08T16:11:04.926074+0000","last_peered":"2023-10-08T16:11:04.926074+0000","last_clean":"2023-10-08T16:11:04.926074+0000","last_became_active":"2023-10-08T16:11:04.918698+0000","last_became_peered":"2023-10-08T16:11:04.918698+0000","last_unstale":"2023-10-08T16:11:04.926074+0000","last_undegraded":"2023-10-08T16:11:04.926074+0000","last_fullsized":"2023-10-08T16:11:04.926074+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:08:37.183227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930864+0000","last_change":"2023-10-08T16:11:04.925095+0000","last_active":"2023-10-08T16:11:04.930864+0000","last_peered":"2023-10-08T16:11:04.930864+0000","last_clean":"2023-10-08T16:11:04.930864+0000","last_became_active":"2023-10-08T16:11:04.924795+0000","last_became_peered":"2023-10-08T16:11:04.924795+0000","last_unstale":"2023-10-08T16:11:04.930864+0000","last_undegraded":"2023-10-08T16:11:04.930864+0000","last_fullsized":"2023-10-08T16:11:04.930864+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:02.301080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921979+0000","last_change":"2023-10-08T16:11:04.915631+0000","last_active":"2023-10-08T16:11:04.921979+0000","last_peered":"2023-10-08T16:11:04.921979+0000","last_clean":"2023-10-08T16:11:04.921979+0000","last_became_active":"2023-10-08T16:11:04.915357+0000","last_became_peered":"2023-10-08T16:11:04.915357+0000","last_unstale":"2023-10-08T16:11:04.921979+0000","last_undegraded":"2023-10-08T16:11:04.921979+0000","last_fullsized":"2023-10-08T16:11:04.921979+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919299+0000","last_change":"2023-10-08T16:11:04.912221+0000","last_active":"2023-10-08T16:11:04.919299+0000","last_peered":"2023-10-08T16:11:04.919299+0000","last_clean":"2023-10-08T16:11:04.919299+0000","last_became_active":"2023-10-08T16:11:04.911981+0000","last_became_peered":"2023-10-08T16:11:04.911981+0000","last_unstale":"2023-10-08T16:11:04.919299+0000","last_undegraded":"2023-10-08T16:11:04.919299+0000","last_fullsized":"2023-10-08T16:11:04.919299+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"112'4","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.939993+0000","last_change":"2023-10-08T16:11:03.911904+0000","last_active":"2023-10-08T16:11:04.939993+0000","last_peered":"2023-10-08T16:11:04.939993+0000","last_clean":"2023-10-08T16:11:04.939993+0000","last_became_active":"2023-10-08T16:11:03.911610+0000","last_became_peered":"2023-10-08T16:11:03.911610+0000","last_unstale":"2023-10-08T16:11:04.939993+0000","last_undegraded":"2023-10-08T16:11:04.939993+0000","last_fullsized":"2023-10-08T16:11:04.939993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:40.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931783+0000","last_change":"2023-10-08T16:11:04.921001+0000","last_active":"2023-10-08T16:11:04.931783+0000","last_peered":"2023-10-08T16:11:04.931783+0000","last_clean":"2023-10-08T16:11:04.931783+0000","last_became_active":"2023-10-08T16:11:04.918800+0000","last_became_peered":"2023-10-08T16:11:04.918800+0000","last_unstale":"2023-10-08T16:11:04.931783+0000","last_undegraded":"2023-10-08T16:11:04.931783+0000","last_fullsized":"2023-10-08T16:11:04.931783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929001+0000","last_change":"2023-10-08T16:11:03.908616+0000","last_active":"2023-10-08T16:11:04.929001+0000","last_peered":"2023-10-08T16:11:04.929001+0000","last_clean":"2023-10-08T16:11:04.929001+0000","last_became_active":"2023-10-08T16:11:03.908311+0000","last_became_peered":"2023-10-08T16:11:03.908311+0000","last_unstale":"2023-10-08T16:11:04.929001+0000","last_undegraded":"2023-10-08T16:11:04.929001+0000","last_fullsized":"2023-10-08T16:11:04.929001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934059+0000","last_change":"2023-10-08T16:11:04.927373+0000","last_active":"2023-10-08T16:11:04.934059+0000","last_peered":"2023-10-08T16:11:04.934059+0000","last_clean":"2023-10-08T16:11:04.934059+0000","last_became_active":"2023-10-08T16:11:04.926985+0000","last_became_peered":"2023-10-08T16:11:04.926985+0000","last_unstale":"2023-10-08T16:11:04.934059+0000","last_undegraded":"2023-10-08T16:11:04.934059+0000","last_fullsized":"2023-10-08T16:11:04.934059+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:55.332121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928276+0000","last_change":"2023-10-08T16:11:03.907369+0000","last_active":"2023-10-08T16:11:04.928276+0000","last_peered":"2023-10-08T16:11:04.928276+0000","last_clean":"2023-10-08T16:11:04.928276+0000","last_became_active":"2023-10-08T16:11:03.907119+0000","last_became_peered":"2023-10-08T16:11:03.907119+0000","last_unstale":"2023-10-08T16:11:04.928276+0000","last_undegraded":"2023-10-08T16:11:04.928276+0000","last_fullsized":"2023-10-08T16:11:04.928276+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:41.469949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930714+0000","last_change":"2023-10-08T16:11:04.924240+0000","last_active":"2023-10-08T16:11:04.930714+0000","last_peered":"2023-10-08T16:11:04.930714+0000","last_clean":"2023-10-08T16:11:04.930714+0000","last_became_active":"2023-10-08T16:11:04.923879+0000","last_became_peered":"2023-10-08T16:11:04.923879+0000","last_unstale":"2023-10-08T16:11:04.930714+0000","last_undegraded":"2023-10-08T16:11:04.930714+0000","last_fullsized":"2023-10-08T16:11:04.930714+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:57.647866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927909+0000","last_change":"2023-10-08T16:11:03.901096+0000","last_active":"2023-10-08T16:11:04.927909+0000","last_peered":"2023-10-08T16:11:04.927909+0000","last_clean":"2023-10-08T16:11:04.927909+0000","last_became_active":"2023-10-08T16:11:03.900784+0000","last_became_peered":"2023-10-08T16:11:03.900784+0000","last_unstale":"2023-10-08T16:11:04.927909+0000","last_undegraded":"2023-10-08T16:11:04.927909+0000","last_fullsized":"2023-10-08T16:11:04.927909+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:33.310661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919537+0000","last_change":"2023-10-08T16:11:04.916963+0000","last_active":"2023-10-08T16:11:04.919537+0000","last_peered":"2023-10-08T16:11:04.919537+0000","last_clean":"2023-10-08T16:11:04.919537+0000","last_became_active":"2023-10-08T16:11:04.916728+0000","last_became_peered":"2023-10-08T16:11:04.916728+0000","last_unstale":"2023-10-08T16:11:04.919537+0000","last_undegraded":"2023-10-08T16:11:04.919537+0000","last_fullsized":"2023-10-08T16:11:04.919537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926986+0000","last_change":"2023-10-08T16:11:03.907483+0000","last_active":"2023-10-08T16:11:04.926986+0000","last_peered":"2023-10-08T16:11:04.926986+0000","last_clean":"2023-10-08T16:11:04.926986+0000","last_became_active":"2023-10-08T16:11:03.906971+0000","last_became_peered":"2023-10-08T16:11:03.906971+0000","last_unstale":"2023-10-08T16:11:04.926986+0000","last_undegraded":"2023-10-08T16:11:04.926986+0000","last_fullsized":"2023-10-08T16:11:04.926986+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:42.596727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.925200+0000","last_change":"2023-10-08T16:11:04.919204+0000","last_active":"2023-10-08T16:11:04.925200+0000","last_peered":"2023-10-08T16:11:04.925200+0000","last_clean":"2023-10-08T16:11:04.925200+0000","last_became_active":"2023-10-08T16:11:04.918917+0000","last_became_peered":"2023-10-08T16:11:04.918917+0000","last_unstale":"2023-10-08T16:11:04.925200+0000","last_undegraded":"2023-10-08T16:11:04.925200+0000","last_fullsized":"2023-10-08T16:11:04.925200+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927423+0000","last_change":"2023-10-08T16:11:03.906390+0000","last_active":"2023-10-08T16:11:04.927423+0000","last_peered":"2023-10-08T16:11:04.927423+0000","last_clean":"2023-10-08T16:11:04.927423+0000","last_became_active":"2023-10-08T16:11:03.906138+0000","last_became_peered":"2023-10-08T16:11:03.906138+0000","last_unstale":"2023-10-08T16:11:04.927423+0000","last_undegraded":"2023-10-08T16:11:04.927423+0000","last_fullsized":"2023-10-08T16:11:04.927423+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:48.258705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931882+0000","last_change":"2023-10-08T16:11:04.922183+0000","last_active":"2023-10-08T16:11:04.931882+0000","last_peered":"2023-10-08T16:11:04.931882+0000","last_clean":"2023-10-08T16:11:04.931882+0000","last_became_active":"2023-10-08T16:11:04.921875+0000","last_became_peered":"2023-10-08T16:11:04.921875+0000","last_unstale":"2023-10-08T16:11:04.931882+0000","last_undegraded":"2023-10-08T16:11:04.931882+0000","last_fullsized":"2023-10-08T16:11:04.931882+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:48.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904946+0000","last_change":"2023-10-08T16:11:03.906532+0000","last_active":"2023-10-08T16:11:04.904946+0000","last_peered":"2023-10-08T16:11:04.904946+0000","last_clean":"2023-10-08T16:11:04.904946+0000","last_became_active":"2023-10-08T16:11:03.906222+0000","last_became_peered":"2023-10-08T16:11:03.906222+0000","last_unstale":"2023-10-08T16:11:04.904946+0000","last_undegraded":"2023-10-08T16:11:04.904946+0000","last_fullsized":"2023-10-08T16:11:04.904946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:47.712149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.915463+0000","last_change":"2023-10-08T16:11:04.913786+0000","last_active":"2023-10-08T16:11:04.915463+0000","last_peered":"2023-10-08T16:11:04.915463+0000","last_clean":"2023-10-08T16:11:04.915463+0000","last_became_active":"2023-10-08T16:11:04.913480+0000","last_became_peered":"2023-10-08T16:11:04.913480+0000","last_unstale":"2023-10-08T16:11:04.915463+0000","last_undegraded":"2023-10-08T16:11:04.915463+0000","last_fullsized":"2023-10-08T16:11:04.915463+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904058+0000","last_change":"2023-10-08T16:11:03.909969+0000","last_active":"2023-10-08T16:11:04.904058+0000","last_peered":"2023-10-08T16:11:04.904058+0000","last_clean":"2023-10-08T16:11:04.904058+0000","last_became_active":"2023-10-08T16:11:03.909698+0000","last_became_peered":"2023-10-08T16:11:03.909698+0000","last_unstale":"2023-10-08T16:11:04.904058+0000","last_undegraded":"2023-10-08T16:11:04.904058+0000","last_fullsized":"2023-10-08T16:11:04.904058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:08.362073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926962+0000","last_change":"2023-10-08T16:11:04.921139+0000","last_active":"2023-10-08T16:11:04.926962+0000","last_peered":"2023-10-08T16:11:04.926962+0000","last_clean":"2023-10-08T16:11:04.926962+0000","last_became_active":"2023-10-08T16:11:04.918955+0000","last_became_peered":"2023-10-08T16:11:04.918955+0000","last_unstale":"2023-10-08T16:11:04.926962+0000","last_undegraded":"2023-10-08T16:11:04.926962+0000","last_fullsized":"2023-10-08T16:11:04.926962+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:45.264721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.905959+0000","last_change":"2023-10-08T16:11:03.912368+0000","last_active":"2023-10-08T16:11:04.905959+0000","last_peered":"2023-10-08T16:11:04.905959+0000","last_clean":"2023-10-08T16:11:04.905959+0000","last_became_active":"2023-10-08T16:11:03.912066+0000","last_became_peered":"2023-10-08T16:11:03.912066+0000","last_unstale":"2023-10-08T16:11:04.905959+0000","last_undegraded":"2023-10-08T16:11:04.905959+0000","last_fullsized":"2023-10-08T16:11:04.905959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:52.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919308+0000","last_change":"2023-10-08T16:11:04.916646+0000","last_active":"2023-10-08T16:11:04.919308+0000","last_peered":"2023-10-08T16:11:04.919308+0000","last_clean":"2023-10-08T16:11:04.919308+0000","last_became_active":"2023-10-08T16:11:04.914613+0000","last_became_peered":"2023-10-08T16:11:04.914613+0000","last_unstale":"2023-10-08T16:11:04.919308+0000","last_undegraded":"2023-10-08T16:11:04.919308+0000","last_fullsized":"2023-10-08T16:11:04.919308+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911708+0000","last_change":"2023-10-08T16:11:03.910705+0000","last_active":"2023-10-08T16:11:04.911708+0000","last_peered":"2023-10-08T16:11:04.911708+0000","last_clean":"2023-10-08T16:11:04.911708+0000","last_became_active":"2023-10-08T16:11:03.910321+0000","last_became_peered":"2023-10-08T16:11:03.910321+0000","last_unstale":"2023-10-08T16:11:04.911708+0000","last_undegraded":"2023-10-08T16:11:04.911708+0000","last_fullsized":"2023-10-08T16:11:04.911708+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:29.783953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.933986+0000","last_change":"2023-10-08T16:11:04.926768+0000","last_active":"2023-10-08T16:11:04.933986+0000","last_peered":"2023-10-08T16:11:04.933986+0000","last_clean":"2023-10-08T16:11:04.933986+0000","last_became_active":"2023-10-08T16:11:04.926469+0000","last_became_peered":"2023-10-08T16:11:04.926469+0000","last_unstale":"2023-10-08T16:11:04.933986+0000","last_undegraded":"2023-10-08T16:11:04.933986+0000","last_fullsized":"2023-10-08T16:11:04.933986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:42.474298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"112'3","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934136+0000","last_change":"2023-10-08T16:11:03.911284+0000","last_active":"2023-10-08T16:11:04.934136+0000","last_peered":"2023-10-08T16:11:04.934136+0000","last_clean":"2023-10-08T16:11:04.934136+0000","last_became_active":"2023-10-08T16:11:03.910992+0000","last_became_peered":"2023-10-08T16:11:03.910992+0000","last_unstale":"2023-10-08T16:11:04.934136+0000","last_undegraded":"2023-10-08T16:11:04.934136+0000","last_fullsized":"2023-10-08T16:11:04.934136+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:17.563101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929122+0000","last_change":"2023-10-08T16:11:04.924103+0000","last_active":"2023-10-08T16:11:04.929122+0000","last_peered":"2023-10-08T16:11:04.929122+0000","last_clean":"2023-10-08T16:11:04.929122+0000","last_became_active":"2023-10-08T16:11:04.923604+0000","last_became_peered":"2023-10-08T16:11:04.923604+0000","last_unstale":"2023-10-08T16:11:04.929122+0000","last_undegraded":"2023-10-08T16:11:04.929122+0000","last_fullsized":"2023-10-08T16:11:04.929122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"112'6","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.941751+0000","last_change":"2023-10-08T16:11:03.911078+0000","last_active":"2023-10-08T16:11:04.941751+0000","last_peered":"2023-10-08T16:11:04.941751+0000","last_clean":"2023-10-08T16:11:04.941751+0000","last_became_active":"2023-10-08T16:11:03.910522+0000","last_became_peered":"2023-10-08T16:11:03.910522+0000","last_unstale":"2023-10-08T16:11:04.941751+0000","last_undegraded":"2023-10-08T16:11:04.941751+0000","last_fullsized":"2023-10-08T16:11:04.941751+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920977+0000","last_change":"2023-10-08T16:11:04.918512+0000","last_active":"2023-10-08T16:11:04.920977+0000","last_peered":"2023-10-08T16:11:04.920977+0000","last_clean":"2023-10-08T16:11:04.920977+0000","last_became_active":"2023-10-08T16:11:04.917852+0000","last_became_peered":"2023-10-08T16:11:04.917852+0000","last_unstale":"2023-10-08T16:11:04.920977+0000","last_undegraded":"2023-10-08T16:11:04.920977+0000","last_fullsized":"2023-10-08T16:11:04.920977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:26.927620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909933+0000","last_change":"2023-10-08T16:11:03.913269+0000","last_active":"2023-10-08T16:11:04.909933+0000","last_peered":"2023-10-08T16:11:04.909933+0000","last_clean":"2023-10-08T16:11:04.909933+0000","last_became_active":"2023-10-08T16:11:03.913044+0000","last_became_peered":"2023-10-08T16:11:03.913044+0000","last_unstale":"2023-10-08T16:11:04.909933+0000","last_undegraded":"2023-10-08T16:11:04.909933+0000","last_fullsized":"2023-10-08T16:11:04.909933+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:52.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920806+0000","last_change":"2023-10-08T16:11:04.916638+0000","last_active":"2023-10-08T16:11:04.920806+0000","last_peered":"2023-10-08T16:11:04.920806+0000","last_clean":"2023-10-08T16:11:04.920806+0000","last_became_active":"2023-10-08T16:11:04.916386+0000","last_became_peered":"2023-10-08T16:11:04.916386+0000","last_unstale":"2023-10-08T16:11:04.920806+0000","last_undegraded":"2023-10-08T16:11:04.920806+0000","last_fullsized":"2023-10-08T16:11:04.920806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904127+0000","last_change":"2023-10-08T16:11:03.905844+0000","last_active":"2023-10-08T16:11:04.904127+0000","last_peered":"2023-10-08T16:11:04.904127+0000","last_clean":"2023-10-08T16:11:04.904127+0000","last_became_active":"2023-10-08T16:11:03.905597+0000","last_became_peered":"2023-10-08T16:11:03.905597+0000","last_unstale":"2023-10-08T16:11:04.904127+0000","last_undegraded":"2023-10-08T16:11:04.904127+0000","last_fullsized":"2023-10-08T16:11:04.904127+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927195+0000","last_change":"2023-10-08T16:11:04.918112+0000","last_active":"2023-10-08T16:11:04.927195+0000","last_peered":"2023-10-08T16:11:04.927195+0000","last_clean":"2023-10-08T16:11:04.927195+0000","last_became_active":"2023-10-08T16:11:04.917865+0000","last_became_peered":"2023-10-08T16:11:04.917865+0000","last_unstale":"2023-10-08T16:11:04.927195+0000","last_undegraded":"2023-10-08T16:11:04.927195+0000","last_fullsized":"2023-10-08T16:11:04.927195+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:07.668042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"112'8","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.936878+0000","last_change":"2023-10-08T16:11:03.915073+0000","last_active":"2023-10-08T16:11:04.936878+0000","last_peered":"2023-10-08T16:11:04.936878+0000","last_clean":"2023-10-08T16:11:04.936878+0000","last_became_active":"2023-10-08T16:11:03.914732+0000","last_became_peered":"2023-10-08T16:11:03.914732+0000","last_unstale":"2023-10-08T16:11:04.936878+0000","last_undegraded":"2023-10-08T16:11:04.936878+0000","last_fullsized":"2023-10-08T16:11:04.936878+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920447+0000","last_change":"2023-10-08T16:11:04.915424+0000","last_active":"2023-10-08T16:11:04.920447+0000","last_peered":"2023-10-08T16:11:04.920447+0000","last_clean":"2023-10-08T16:11:04.920447+0000","last_became_active":"2023-10-08T16:11:04.915153+0000","last_became_peered":"2023-10-08T16:11:04.915153+0000","last_unstale":"2023-10-08T16:11:04.920447+0000","last_undegraded":"2023-10-08T16:11:04.920447+0000","last_fullsized":"2023-10-08T16:11:04.920447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934223+0000","last_change":"2023-10-08T16:11:03.906254+0000","last_active":"2023-10-08T16:11:04.934223+0000","last_peered":"2023-10-08T16:11:04.934223+0000","last_clean":"2023-10-08T16:11:04.934223+0000","last_became_active":"2023-10-08T16:11:03.905790+0000","last_became_peered":"2023-10-08T16:11:03.905790+0000","last_unstale":"2023-10-08T16:11:04.934223+0000","last_undegraded":"2023-10-08T16:11:04.934223+0000","last_fullsized":"2023-10-08T16:11:04.934223+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:44.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919329+0000","last_change":"2023-10-08T16:11:04.916958+0000","last_active":"2023-10-08T16:11:04.919329+0000","last_peered":"2023-10-08T16:11:04.919329+0000","last_clean":"2023-10-08T16:11:04.919329+0000","last_became_active":"2023-10-08T16:11:04.916700+0000","last_became_peered":"2023-10-08T16:11:04.916700+0000","last_unstale":"2023-10-08T16:11:04.919329+0000","last_undegraded":"2023-10-08T16:11:04.919329+0000","last_fullsized":"2023-10-08T16:11:04.919329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.936723+0000","last_change":"2023-10-08T16:11:03.910220+0000","last_active":"2023-10-08T16:11:04.936723+0000","last_peered":"2023-10-08T16:11:04.936723+0000","last_clean":"2023-10-08T16:11:04.936723+0000","last_became_active":"2023-10-08T16:11:03.909963+0000","last_became_peered":"2023-10-08T16:11:03.909963+0000","last_unstale":"2023-10-08T16:11:04.936723+0000","last_undegraded":"2023-10-08T16:11:04.936723+0000","last_fullsized":"2023-10-08T16:11:04.936723+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:27.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919386+0000","last_change":"2023-10-08T16:11:04.914718+0000","last_active":"2023-10-08T16:11:04.919386+0000","last_peered":"2023-10-08T16:11:04.919386+0000","last_clean":"2023-10-08T16:11:04.919386+0000","last_became_active":"2023-10-08T16:11:04.914452+0000","last_became_peered":"2023-10-08T16:11:04.914452+0000","last_unstale":"2023-10-08T16:11:04.919386+0000","last_undegraded":"2023-10-08T16:11:04.919386+0000","last_fullsized":"2023-10-08T16:11:04.919386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929412+0000","last_change":"2023-10-08T16:11:03.907316+0000","last_active":"2023-10-08T16:11:04.929412+0000","last_peered":"2023-10-08T16:11:04.929412+0000","last_clean":"2023-10-08T16:11:04.929412+0000","last_became_active":"2023-10-08T16:11:03.907061+0000","last_became_peered":"2023-10-08T16:11:03.907061+0000","last_unstale":"2023-10-08T16:11:04.929412+0000","last_undegraded":"2023-10-08T16:11:04.929412+0000","last_fullsized":"2023-10-08T16:11:04.929412+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:38.380438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.915591+0000","last_change":"2023-10-08T16:11:04.912949+0000","last_active":"2023-10-08T16:11:04.915591+0000","last_peered":"2023-10-08T16:11:04.915591+0000","last_clean":"2023-10-08T16:11:04.915591+0000","last_became_active":"2023-10-08T16:11:04.912691+0000","last_became_peered":"2023-10-08T16:11:04.912691+0000","last_unstale":"2023-10-08T16:11:04.915591+0000","last_undegraded":"2023-10-08T16:11:04.915591+0000","last_fullsized":"2023-10-08T16:11:04.915591+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.906021+0000","last_change":"2023-10-08T16:11:03.907179+0000","last_active":"2023-10-08T16:11:04.906021+0000","last_peered":"2023-10-08T16:11:04.906021+0000","last_clean":"2023-10-08T16:11:04.906021+0000","last_became_active":"2023-10-08T16:11:03.906946+0000","last_became_peered":"2023-10-08T16:11:03.906946+0000","last_unstale":"2023-10-08T16:11:04.906021+0000","last_undegraded":"2023-10-08T16:11:04.906021+0000","last_fullsized":"2023-10-08T16:11:04.906021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926096+0000","last_change":"2023-10-08T16:11:04.920813+0000","last_active":"2023-10-08T16:11:04.926096+0000","last_peered":"2023-10-08T16:11:04.926096+0000","last_clean":"2023-10-08T16:11:04.926096+0000","last_became_active":"2023-10-08T16:11:04.918345+0000","last_became_peered":"2023-10-08T16:11:04.918345+0000","last_unstale":"2023-10-08T16:11:04.926096+0000","last_undegraded":"2023-10-08T16:11:04.926096+0000","last_fullsized":"2023-10-08T16:11:04.926096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:53.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930891+0000","last_change":"2023-10-08T16:11:03.908030+0000","last_active":"2023-10-08T16:11:04.930891+0000","last_peered":"2023-10-08T16:11:04.930891+0000","last_clean":"2023-10-08T16:11:04.930891+0000","last_became_active":"2023-10-08T16:11:03.907731+0000","last_became_peered":"2023-10-08T16:11:03.907731+0000","last_unstale":"2023-10-08T16:11:04.930891+0000","last_undegraded":"2023-10-08T16:11:04.930891+0000","last_fullsized":"2023-10-08T16:11:04.930891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:10.071216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921606+0000","last_change":"2023-10-08T16:11:04.916141+0000","last_active":"2023-10-08T16:11:04.921606+0000","last_peered":"2023-10-08T16:11:04.921606+0000","last_clean":"2023-10-08T16:11:04.921606+0000","last_became_active":"2023-10-08T16:11:04.915845+0000","last_became_peered":"2023-10-08T16:11:04.915845+0000","last_unstale":"2023-10-08T16:11:04.921606+0000","last_undegraded":"2023-10-08T16:11:04.921606+0000","last_fullsized":"2023-10-08T16:11:04.921606+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909697+0000","last_change":"2023-10-08T16:11:03.909202+0000","last_active":"2023-10-08T16:11:04.909697+0000","last_peered":"2023-10-08T16:11:04.909697+0000","last_clean":"2023-10-08T16:11:04.909697+0000","last_became_active":"2023-10-08T16:11:03.908949+0000","last_became_peered":"2023-10-08T16:11:03.908949+0000","last_unstale":"2023-10-08T16:11:04.909697+0000","last_undegraded":"2023-10-08T16:11:04.909697+0000","last_fullsized":"2023-10-08T16:11:04.909697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:54.146955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.916973+0000","last_change":"2023-10-08T16:11:04.915617+0000","last_active":"2023-10-08T16:11:04.916973+0000","last_peered":"2023-10-08T16:11:04.916973+0000","last_clean":"2023-10-08T16:11:04.916973+0000","last_became_active":"2023-10-08T16:11:04.915382+0000","last_became_peered":"2023-10-08T16:11:04.915382+0000","last_unstale":"2023-10-08T16:11:04.916973+0000","last_undegraded":"2023-10-08T16:11:04.916973+0000","last_fullsized":"2023-10-08T16:11:04.916973+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:48.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.908074+0000","last_change":"2023-10-08T16:11:03.907175+0000","last_active":"2023-10-08T16:11:04.908074+0000","last_peered":"2023-10-08T16:11:04.908074+0000","last_clean":"2023-10-08T16:11:04.908074+0000","last_became_active":"2023-10-08T16:11:03.906936+0000","last_became_peered":"2023-10-08T16:11:03.906936+0000","last_unstale":"2023-10-08T16:11:04.908074+0000","last_undegraded":"2023-10-08T16:11:04.908074+0000","last_fullsized":"2023-10-08T16:11:04.908074+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:46.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.933884+0000","last_change":"2023-10-08T16:11:04.925982+0000","last_active":"2023-10-08T16:11:04.933884+0000","last_peered":"2023-10-08T16:11:04.933884+0000","last_clean":"2023-10-08T16:11:04.933884+0000","last_became_active":"2023-10-08T16:11:04.925304+0000","last_became_peered":"2023-10-08T16:11:04.925304+0000","last_unstale":"2023-10-08T16:11:04.933884+0000","last_undegraded":"2023-10-08T16:11:04.933884+0000","last_fullsized":"2023-10-08T16:11:04.933884+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:28.533612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.910499+0000","last_change":"2023-10-08T16:11:03.910257+0000","last_active":"2023-10-08T16:11:04.910499+0000","last_peered":"2023-10-08T16:11:04.910499+0000","last_clean":"2023-10-08T16:11:04.910499+0000","last_became_active":"2023-10-08T16:11:03.909995+0000","last_became_peered":"2023-10-08T16:11:03.909995+0000","last_unstale":"2023-10-08T16:11:04.910499+0000","last_undegraded":"2023-10-08T16:11:04.910499+0000","last_fullsized":"2023-10-08T16:11:04.910499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:01.296731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920887+0000","last_change":"2023-10-08T16:11:04.913585+0000","last_active":"2023-10-08T16:11:04.920887+0000","last_peered":"2023-10-08T16:11:04.920887+0000","last_clean":"2023-10-08T16:11:04.920887+0000","last_became_active":"2023-10-08T16:11:04.912650+0000","last_became_peered":"2023-10-08T16:11:04.912650+0000","last_unstale":"2023-10-08T16:11:04.920887+0000","last_undegraded":"2023-10-08T16:11:04.920887+0000","last_fullsized":"2023-10-08T16:11:04.920887+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934304+0000","last_change":"2023-10-08T16:11:03.909589+0000","last_active":"2023-10-08T16:11:04.934304+0000","last_peered":"2023-10-08T16:11:04.934304+0000","last_clean":"2023-10-08T16:11:04.934304+0000","last_became_active":"2023-10-08T16:11:03.909342+0000","last_became_peered":"2023-10-08T16:11:03.909342+0000","last_unstale":"2023-10-08T16:11:04.934304+0000","last_undegraded":"2023-10-08T16:11:04.934304+0000","last_fullsized":"2023-10-08T16:11:04.934304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:10.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926061+0000","last_change":"2023-10-08T16:11:04.921542+0000","last_active":"2023-10-08T16:11:04.926061+0000","last_peered":"2023-10-08T16:11:04.926061+0000","last_clean":"2023-10-08T16:11:04.926061+0000","last_became_active":"2023-10-08T16:11:04.921199+0000","last_became_peered":"2023-10-08T16:11:04.921199+0000","last_unstale":"2023-10-08T16:11:04.926061+0000","last_undegraded":"2023-10-08T16:11:04.926061+0000","last_fullsized":"2023-10-08T16:11:04.926061+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:38.890416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911910+0000","last_change":"2023-10-08T16:11:03.913709+0000","last_active":"2023-10-08T16:11:04.911910+0000","last_peered":"2023-10-08T16:11:04.911910+0000","last_clean":"2023-10-08T16:11:04.911910+0000","last_became_active":"2023-10-08T16:11:03.913436+0000","last_became_peered":"2023-10-08T16:11:03.913436+0000","last_unstale":"2023-10-08T16:11:04.911910+0000","last_undegraded":"2023-10-08T16:11:04.911910+0000","last_fullsized":"2023-10-08T16:11:04.911910+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:07.589542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928446+0000","last_change":"2023-10-08T16:11:04.926028+0000","last_active":"2023-10-08T16:11:04.928446+0000","last_peered":"2023-10-08T16:11:04.928446+0000","last_clean":"2023-10-08T16:11:04.928446+0000","last_became_active":"2023-10-08T16:11:04.925768+0000","last_became_peered":"2023-10-08T16:11:04.925768+0000","last_unstale":"2023-10-08T16:11:04.928446+0000","last_undegraded":"2023-10-08T16:11:04.928446+0000","last_fullsized":"2023-10-08T16:11:04.928446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:44.041214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904545+0000","last_change":"2023-10-08T16:11:03.909139+0000","last_active":"2023-10-08T16:11:04.904545+0000","last_peered":"2023-10-08T16:11:04.904545+0000","last_clean":"2023-10-08T16:11:04.904545+0000","last_became_active":"2023-10-08T16:11:03.908877+0000","last_became_peered":"2023-10-08T16:11:03.908877+0000","last_unstale":"2023-10-08T16:11:04.904545+0000","last_undegraded":"2023-10-08T16:11:04.904545+0000","last_fullsized":"2023-10-08T16:11:04.904545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:48.689812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921672+0000","last_change":"2023-10-08T16:11:04.916524+0000","last_active":"2023-10-08T16:11:04.921672+0000","last_peered":"2023-10-08T16:11:04.921672+0000","last_clean":"2023-10-08T16:11:04.921672+0000","last_became_active":"2023-10-08T16:11:04.916095+0000","last_became_peered":"2023-10-08T16:11:04.916095+0000","last_unstale":"2023-10-08T16:11:04.921672+0000","last_undegraded":"2023-10-08T16:11:04.921672+0000","last_fullsized":"2023-10-08T16:11:04.921672+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.913045+0000","last_change":"2023-10-08T16:11:03.905282+0000","last_active":"2023-10-08T16:11:04.913045+0000","last_peered":"2023-10-08T16:11:04.913045+0000","last_clean":"2023-10-08T16:11:04.913045+0000","last_became_active":"2023-10-08T16:11:03.905030+0000","last_became_peered":"2023-10-08T16:11:03.905030+0000","last_unstale":"2023-10-08T16:11:04.913045+0000","last_undegraded":"2023-10-08T16:11:04.913045+0000","last_fullsized":"2023-10-08T16:11:04.913045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:54.327772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930785+0000","last_change":"2023-10-08T16:11:04.924999+0000","last_active":"2023-10-08T16:11:04.930785+0000","last_peered":"2023-10-08T16:11:04.930785+0000","last_clean":"2023-10-08T16:11:04.930785+0000","last_became_active":"2023-10-08T16:11:04.924577+0000","last_became_peered":"2023-10-08T16:11:04.924577+0000","last_unstale":"2023-10-08T16:11:04.930785+0000","last_undegraded":"2023-10-08T16:11:04.930785+0000","last_fullsized":"2023-10-08T16:11:04.930785+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:53.038890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.903825+0000","last_change":"2023-10-08T16:11:03.907637+0000","last_active":"2023-10-08T16:11:04.903825+0000","last_peered":"2023-10-08T16:11:04.903825+0000","last_clean":"2023-10-08T16:11:04.903825+0000","last_became_active":"2023-10-08T16:11:03.907270+0000","last_became_peered":"2023-10-08T16:11:03.907270+0000","last_unstale":"2023-10-08T16:11:04.903825+0000","last_undegraded":"2023-10-08T16:11:04.903825+0000","last_fullsized":"2023-10-08T16:11:04.903825+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928972+0000","last_change":"2023-10-08T16:11:04.923256+0000","last_active":"2023-10-08T16:11:04.928972+0000","last_peered":"2023-10-08T16:11:04.928972+0000","last_clean":"2023-10-08T16:11:04.928972+0000","last_became_active":"2023-10-08T16:11:04.923012+0000","last_became_peered":"2023-10-08T16:11:04.923012+0000","last_unstale":"2023-10-08T16:11:04.928972+0000","last_undegraded":"2023-10-08T16:11:04.928972+0000","last_fullsized":"2023-10-08T16:11:04.928972+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911880+0000","last_change":"2023-10-08T16:11:03.909548+0000","last_active":"2023-10-08T16:11:04.911880+0000","last_peered":"2023-10-08T16:11:04.911880+0000","last_clean":"2023-10-08T16:11:04.911880+0000","last_became_active":"2023-10-08T16:11:03.909225+0000","last_became_peered":"2023-10-08T16:11:03.909225+0000","last_unstale":"2023-10-08T16:11:04.911880+0000","last_undegraded":"2023-10-08T16:11:04.911880+0000","last_fullsized":"2023-10-08T16:11:04.911880+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:39.249283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920179+0000","last_change":"2023-10-08T16:11:04.916787+0000","last_active":"2023-10-08T16:11:04.920179+0000","last_peered":"2023-10-08T16:11:04.920179+0000","last_clean":"2023-10-08T16:11:04.920179+0000","last_became_active":"2023-10-08T16:11:04.916360+0000","last_became_peered":"2023-10-08T16:11:04.916360+0000","last_unstale":"2023-10-08T16:11:04.920179+0000","last_undegraded":"2023-10-08T16:11:04.920179+0000","last_fullsized":"2023-10-08T16:11:04.920179+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909235+0000","last_change":"2023-10-08T16:11:03.910568+0000","last_active":"2023-10-08T16:11:04.909235+0000","last_peered":"2023-10-08T16:11:04.909235+0000","last_clean":"2023-10-08T16:11:04.909235+0000","last_became_active":"2023-10-08T16:11:03.910311+0000","last_became_peered":"2023-10-08T16:11:03.910311+0000","last_unstale":"2023-10-08T16:11:04.909235+0000","last_undegraded":"2023-10-08T16:11:04.909235+0000","last_fullsized":"2023-10-08T16:11:04.909235+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:56.735575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928227+0000","last_change":"2023-10-08T16:11:04.926281+0000","last_active":"2023-10-08T16:11:04.928227+0000","last_peered":"2023-10-08T16:11:04.928227+0000","last_clean":"2023-10-08T16:11:04.928227+0000","last_became_active":"2023-10-08T16:11:04.925981+0000","last_became_peered":"2023-10-08T16:11:04.925981+0000","last_unstale":"2023-10-08T16:11:04.928227+0000","last_undegraded":"2023-10-08T16:11:04.928227+0000","last_fullsized":"2023-10-08T16:11:04.928227+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:23.716868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.910596+0000","last_change":"2023-10-08T16:11:03.906699+0000","last_active":"2023-10-08T16:11:04.910596+0000","last_peered":"2023-10-08T16:11:04.910596+0000","last_clean":"2023-10-08T16:11:04.910596+0000","last_became_active":"2023-10-08T16:11:03.906348+0000","last_became_peered":"2023-10-08T16:11:03.906348+0000","last_unstale":"2023-10-08T16:11:04.910596+0000","last_undegraded":"2023-10-08T16:11:04.910596+0000","last_fullsized":"2023-10-08T16:11:04.910596+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:34.478940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930763+0000","last_change":"2023-10-08T16:11:04.924954+0000","last_active":"2023-10-08T16:11:04.930763+0000","last_peered":"2023-10-08T16:11:04.930763+0000","last_clean":"2023-10-08T16:11:04.930763+0000","last_became_active":"2023-10-08T16:11:04.924604+0000","last_became_peered":"2023-10-08T16:11:04.924604+0000","last_unstale":"2023-10-08T16:11:04.930763+0000","last_undegraded":"2023-10-08T16:11:04.930763+0000","last_fullsized":"2023-10-08T16:11:04.930763+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:35.483289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.940007+0000","last_change":"2023-10-08T16:11:03.911957+0000","last_active":"2023-10-08T16:11:04.940007+0000","last_peered":"2023-10-08T16:11:04.940007+0000","last_clean":"2023-10-08T16:11:04.940007+0000","last_became_active":"2023-10-08T16:11:03.911652+0000","last_became_peered":"2023-10-08T16:11:03.911652+0000","last_unstale":"2023-10-08T16:11:04.940007+0000","last_undegraded":"2023-10-08T16:11:04.940007+0000","last_fullsized":"2023-10-08T16:11:04.940007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:21.785467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":239,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911425+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:04.911425+0000","last_peered":"2023-10-08T16:11:04.911425+0000","last_clean":"2023-10-08T16:11:04.911425+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:04.911425+0000","last_undegraded":"2023-10-08T16:11:04.911425+0000","last_fullsized":"2023-10-08T16:11:04.911425+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303368,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068472,"statfs":{"total":96636764160,"available":96326115328,"internally_reserved":0,"allocated":2031616,"data_stored":1203543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.436}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.38200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.48799999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.314},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.497}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302776,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069064,"statfs":{"total":96636764160,"available":96326721536,"internally_reserved":0,"allocated":1425408,"data_stored":612565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.68700000000000006,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64700000000000002,"5min":0.75,"15min":0.96599999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.94199999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.28399999999999997,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.874},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.48899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":611029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.309},"min":{"1min":0.219,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.33700000000000002},"min":{"1min":0.219,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.58199999999999996,"15min":0.77500000000000002},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.35599999999999998},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.501,"15min":1.2230000000000001},"last":0.376},{"interface":"front","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.72399999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.38600000000000001},"min":{"1min":0.28799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.70399999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.35799999999999998},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.30299999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":1.077},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1.696,"15min":1.696},"last":0.46500000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":1.657,"15min":1.657},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.52000000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.099},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.50600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.59499999999999997},"min":{"1min":0.48599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.1839999999999999},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303344,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068496,"statfs":{"total":96636764160,"available":96326139904,"internally_reserved":0,"allocated":2007040,"data_stored":1201397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.53400000000000003},"min":{"1min":0.23499999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":3.3220000000000001},"last":0.33100000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.56699999999999995},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":5.6769999999999996},"last":0.376},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.45500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36799999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.47799999999999998},"min":{"1min":0.22700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.434,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.317},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.60399999999999998},"min":{"1min":0.29899999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55100000000000005,"5min":1.498,"15min":5.5970000000000004},"last":0.40200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.66300000000000003},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.66000000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":1.847,"15min":3.2890000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56200000000000006,"15min":0.77800000000000002},"min":{"1min":0.47199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.54900000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.80200000000000005},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.754,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.379}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.25},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.314,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.505,"5min":0.53600000000000003,"15min":1.238},"last":0.251},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33200000000000002,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33100000000000002,"15min":0.35599999999999998},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39700000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.497,"5min":0.59299999999999997,"15min":1.3520000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36299999999999999,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.248},"max":{"1min":0.47599999999999998,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.42099999999999999},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":1.327},"last":0.29699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.65700000000000003,"5min":1.099,"15min":1.099},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":1.27},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.91900000000000004,"15min":1.879},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.97599999999999998,"15min":1.825},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":1.0009999999999999,"15min":1.115},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1417216,"data_stored":611099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.501},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55000000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.35899999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.61699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64500000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:13.594 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:13.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.720+0000 7ff53e8a4700 1 Processor -- start 2023-10-08T16:11:13.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.720+0000 7ff53e8a4700 1 -- start start 2023-10-08T16:11:13.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53e8a4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff538159c60 0x7ff53815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53e8a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff53815db10 0x7ff538069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53e8a4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53815a160 con 0x7ff538159890 2023-10-08T16:11:13.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53e8a4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53815df50 con 0x7ff538159c60 2023-10-08T16:11:13.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53e8a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5380d13a0 con 0x7ff53815db10 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff5377fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff538159c60 0x7ff53815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff537fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff538159890 0x7ff53815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59162/0 (socket says 172.21.15.170:59162) 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff537fff700 1 -- 172.21.15.170:0/3812983631 learned_addr learned my addr 172.21.15.170:0/3812983631 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff53ce41700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff53815db10 0x7ff538069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff5377fe700 1 -- 172.21.15.170:0/3812983631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff53815db10 msgr2=0x7ff538069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff5377fe700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff53815db10 0x7ff538069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff5377fe700 1 -- 172.21.15.170:0/3812983631 >> v1:172.21.15.170:6789/0 conn(0x7ff538159890 legacy=0x7ff53815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.721+0000 7ff536ffd700 1 -- 172.21.15.170:0/3812983631 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663226600 0 0) 0x7ff53815a160 con 0x7ff538159890 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff5377fe700 1 -- 172.21.15.170:0/3812983631 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53815e140 con 0x7ff538159c60 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff5377fe700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159c60 0x7ff53815d5d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0098e0 tx=0x7ff52c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6fbf39c52b6a72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff536ffd700 1 -- 172.21.15.170:0/3812983631 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00b640 con 0x7ff538159c60 2023-10-08T16:11:13.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff536ffd700 1 -- 172.21.15.170:0/3812983631 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c00b7a0 con 0x7ff538159c60 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff536ffd700 1 -- 172.21.15.170:0/3812983631 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c00bb40 con 0x7ff538159c60 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 -- 172.21.15.170:0/3812983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159c60 msgr2=0x7ff53815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159c60 0x7ff53815d5d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0098e0 tx=0x7ff52c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 -- 172.21.15.170:0/3812983631 shutdown_connections 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff53815db10 0x7ff538069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/3812983631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159c60 0x7ff53815d5d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 -- 172.21.15.170:0/3812983631 >> 172.21.15.170:0/3812983631 conn(0x7ff538068e20 msgr2=0x7ff5380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 -- 172.21.15.170:0/3812983631 shutdown_connections 2023-10-08T16:11:13.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.722+0000 7ff53e8a4700 1 -- 172.21.15.170:0/3812983631 wait complete. 2023-10-08T16:11:13.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.723+0000 7ff53e8a4700 1 Processor -- start 2023-10-08T16:11:13.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 -- start start 2023-10-08T16:11:13.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 0x7ff53814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 0x7ff5381525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538152140 con 0x7ff538159890 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff538151530 con 0x7ff53815db10 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff53e8a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff538151810 con 0x7ff538159c60 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff537fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.724+0000 7ff537fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44724/0 (socket says 172.21.15.170:44724) 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 -- 172.21.15.170:0/1690907855 learned_addr learned my addr 172.21.15.170:0/1690907855 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff53ce41700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 0x7ff5381525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff5377fe700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 0x7ff53814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 -- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 msgr2=0x7ff53814b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 0x7ff53814b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 -- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 msgr2=0x7ff5381525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 0x7ff5381525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 -- 172.21.15.170:0/1690907855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c009420 con 0x7ff538159890 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff53ce41700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 0x7ff5381525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff5377fe700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 0x7ff53814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff537fff700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c7e0 tx=0x7ff528007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528005b30 con 0x7ff538159890 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff528005c90 con 0x7ff538159890 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528007300 con 0x7ff538159890 2023-10-08T16:11:13.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff538151af0 con 0x7ff538159890 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.725+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53814a990 con 0x7ff538159890 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.726+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53804c3c0 con 0x7ff538159890 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.727+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff52801b400 con 0x7ff538159890 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.727+0000 7ff51ffff700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 0x7ff5240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.727+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff528084670 con 0x7ff538159890 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.727+0000 7ff5377fe700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 0x7ff5240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.728+0000 7ff5377fe700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 0x7ff5240697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff538151360 tx=0x7ff52c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:13.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.731+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff528051480 con 0x7ff538159890 2023-10-08T16:11:13.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.910+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff538152280 con 0x7ff524067320 2023-10-08T16:11:13.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.918+0000 7ff51ffff700 1 -- 172.21.15.170:0/1690907855 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290136 (secure 0 0 0) 0x7ff52c00b910 con 0x7ff524067320 2023-10-08T16:11:13.922 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 msgr2=0x7ff5240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 0x7ff5240697d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff538151360 tx=0x7ff52c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:13.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 msgr2=0x7ff53814a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c7e0 tx=0x7ff528007df0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 shutdown_connections 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff53815db10 0x7ff5381525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff524067320 0x7ff5240697d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff538159890 0x7ff53814a450 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 --2- 172.21.15.170:0/1690907855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff538159c60 0x7ff53814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 >> 172.21.15.170:0/1690907855 conn(0x7ff538068e20 msgr2=0x7ff538153e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:13.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 shutdown_connections 2023-10-08T16:11:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:13.921+0000 7ff53e8a4700 1 -- 172.21.15.170:0/1690907855 wait complete. 2023-10-08T16:11:13.926 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:13.934 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2023-10-08T16:11:12.305228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2423336,"kb_used_data":12264,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752551384,"statfs":{"total":773094113280,"available":770612617216,"internally_reserved":0,"allocated":12558336,"data_stored":6073580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001584"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926081+0000","last_change":"2023-10-08T16:11:04.922905+0000","last_active":"2023-10-08T16:11:04.926081+0000","last_peered":"2023-10-08T16:11:04.926081+0000","last_clean":"2023-10-08T16:11:04.926081+0000","last_became_active":"2023-10-08T16:11:04.922592+0000","last_became_peered":"2023-10-08T16:11:04.922592+0000","last_unstale":"2023-10-08T16:11:04.926081+0000","last_undegraded":"2023-10-08T16:11:04.926081+0000","last_fullsized":"2023-10-08T16:11:04.926081+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:49.633201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920743+0000","last_change":"2023-10-08T16:11:04.916819+0000","last_active":"2023-10-08T16:11:04.920743+0000","last_peered":"2023-10-08T16:11:04.920743+0000","last_clean":"2023-10-08T16:11:04.920743+0000","last_became_active":"2023-10-08T16:11:04.915080+0000","last_became_peered":"2023-10-08T16:11:04.915080+0000","last_unstale":"2023-10-08T16:11:04.920743+0000","last_undegraded":"2023-10-08T16:11:04.920743+0000","last_fullsized":"2023-10-08T16:11:04.920743+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:50.336238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931731+0000","last_change":"2023-10-08T16:11:04.921766+0000","last_active":"2023-10-08T16:11:04.931731+0000","last_peered":"2023-10-08T16:11:04.931731+0000","last_clean":"2023-10-08T16:11:04.931731+0000","last_became_active":"2023-10-08T16:11:04.920227+0000","last_became_peered":"2023-10-08T16:11:04.920227+0000","last_unstale":"2023-10-08T16:11:04.931731+0000","last_undegraded":"2023-10-08T16:11:04.931731+0000","last_fullsized":"2023-10-08T16:11:04.931731+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930736+0000","last_change":"2023-10-08T16:11:04.927630+0000","last_active":"2023-10-08T16:11:04.930736+0000","last_peered":"2023-10-08T16:11:04.930736+0000","last_clean":"2023-10-08T16:11:04.930736+0000","last_became_active":"2023-10-08T16:11:04.927334+0000","last_became_peered":"2023-10-08T16:11:04.927334+0000","last_unstale":"2023-10-08T16:11:04.930736+0000","last_undegraded":"2023-10-08T16:11:04.930736+0000","last_fullsized":"2023-10-08T16:11:04.930736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:11.344791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.939953+0000","last_change":"2023-10-08T16:11:04.931826+0000","last_active":"2023-10-08T16:11:04.939953+0000","last_peered":"2023-10-08T16:11:04.939953+0000","last_clean":"2023-10-08T16:11:04.939953+0000","last_became_active":"2023-10-08T16:11:04.931357+0000","last_became_peered":"2023-10-08T16:11:04.931357+0000","last_unstale":"2023-10-08T16:11:04.939953+0000","last_undegraded":"2023-10-08T16:11:04.939953+0000","last_fullsized":"2023-10-08T16:11:04.939953+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:38:20.856257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921401+0000","last_change":"2023-10-08T16:11:04.915094+0000","last_active":"2023-10-08T16:11:04.921401+0000","last_peered":"2023-10-08T16:11:04.921401+0000","last_clean":"2023-10-08T16:11:04.921401+0000","last_became_active":"2023-10-08T16:11:04.913963+0000","last_became_peered":"2023-10-08T16:11:04.913963+0000","last_unstale":"2023-10-08T16:11:04.921401+0000","last_undegraded":"2023-10-08T16:11:04.921401+0000","last_fullsized":"2023-10-08T16:11:04.921401+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:40:17.951940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921646+0000","last_change":"2023-10-08T16:11:04.916445+0000","last_active":"2023-10-08T16:11:04.921646+0000","last_peered":"2023-10-08T16:11:04.921646+0000","last_clean":"2023-10-08T16:11:04.921646+0000","last_became_active":"2023-10-08T16:11:04.916207+0000","last_became_peered":"2023-10-08T16:11:04.916207+0000","last_unstale":"2023-10-08T16:11:04.921646+0000","last_undegraded":"2023-10-08T16:11:04.921646+0000","last_fullsized":"2023-10-08T16:11:04.921646+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.932047+0000","last_change":"2023-10-08T16:11:04.922747+0000","last_active":"2023-10-08T16:11:04.932047+0000","last_peered":"2023-10-08T16:11:04.932047+0000","last_clean":"2023-10-08T16:11:04.932047+0000","last_became_active":"2023-10-08T16:11:04.922021+0000","last_became_peered":"2023-10-08T16:11:04.922021+0000","last_unstale":"2023-10-08T16:11:04.932047+0000","last_undegraded":"2023-10-08T16:11:04.932047+0000","last_fullsized":"2023-10-08T16:11:04.932047+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:57:50.336238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927244+0000","last_change":"2023-10-08T16:11:04.918705+0000","last_active":"2023-10-08T16:11:04.927244+0000","last_peered":"2023-10-08T16:11:04.927244+0000","last_clean":"2023-10-08T16:11:04.927244+0000","last_became_active":"2023-10-08T16:11:04.918384+0000","last_became_peered":"2023-10-08T16:11:04.918384+0000","last_unstale":"2023-10-08T16:11:04.927244+0000","last_undegraded":"2023-10-08T16:11:04.927244+0000","last_fullsized":"2023-10-08T16:11:04.927244+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:00.992096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.916897+0000","last_change":"2023-10-08T16:11:04.915034+0000","last_active":"2023-10-08T16:11:04.916897+0000","last_peered":"2023-10-08T16:11:04.916897+0000","last_clean":"2023-10-08T16:11:04.916897+0000","last_became_active":"2023-10-08T16:11:04.914784+0000","last_became_peered":"2023-10-08T16:11:04.914784+0000","last_unstale":"2023-10-08T16:11:04.916897+0000","last_undegraded":"2023-10-08T16:11:04.916897+0000","last_fullsized":"2023-10-08T16:11:04.916897+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:07.668042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919840+0000","last_change":"2023-10-08T16:11:04.919116+0000","last_active":"2023-10-08T16:11:04.919840+0000","last_peered":"2023-10-08T16:11:04.919840+0000","last_clean":"2023-10-08T16:11:04.919840+0000","last_became_active":"2023-10-08T16:11:04.918610+0000","last_became_peered":"2023-10-08T16:11:04.918610+0000","last_unstale":"2023-10-08T16:11:04.919840+0000","last_undegraded":"2023-10-08T16:11:04.919840+0000","last_fullsized":"2023-10-08T16:11:04.919840+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:55:45.661956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926966+0000","last_change":"2023-10-08T16:11:04.918875+0000","last_active":"2023-10-08T16:11:04.926966+0000","last_peered":"2023-10-08T16:11:04.926966+0000","last_clean":"2023-10-08T16:11:04.926966+0000","last_became_active":"2023-10-08T16:11:04.918515+0000","last_became_peered":"2023-10-08T16:11:04.918515+0000","last_unstale":"2023-10-08T16:11:04.926966+0000","last_undegraded":"2023-10-08T16:11:04.926966+0000","last_fullsized":"2023-10-08T16:11:04.926966+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:23:21.924927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926210+0000","last_change":"2023-10-08T16:11:04.921695+0000","last_active":"2023-10-08T16:11:04.926210+0000","last_peered":"2023-10-08T16:11:04.926210+0000","last_clean":"2023-10-08T16:11:04.926210+0000","last_became_active":"2023-10-08T16:11:04.921213+0000","last_became_peered":"2023-10-08T16:11:04.921213+0000","last_unstale":"2023-10-08T16:11:04.926210+0000","last_undegraded":"2023-10-08T16:11:04.926210+0000","last_fullsized":"2023-10-08T16:11:04.926210+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:32:45.144197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.914566+0000","last_change":"2023-10-08T16:11:04.912547+0000","last_active":"2023-10-08T16:11:04.914566+0000","last_peered":"2023-10-08T16:11:04.914566+0000","last_clean":"2023-10-08T16:11:04.914566+0000","last_became_active":"2023-10-08T16:11:04.912294+0000","last_became_peered":"2023-10-08T16:11:04.912294+0000","last_unstale":"2023-10-08T16:11:04.914566+0000","last_undegraded":"2023-10-08T16:11:04.914566+0000","last_fullsized":"2023-10-08T16:11:04.914566+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:57.739925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920938+0000","last_change":"2023-10-08T16:11:04.918170+0000","last_active":"2023-10-08T16:11:04.920938+0000","last_peered":"2023-10-08T16:11:04.920938+0000","last_clean":"2023-10-08T16:11:04.920938+0000","last_became_active":"2023-10-08T16:11:04.917772+0000","last_became_peered":"2023-10-08T16:11:04.917772+0000","last_unstale":"2023-10-08T16:11:04.920938+0000","last_undegraded":"2023-10-08T16:11:04.920938+0000","last_fullsized":"2023-10-08T16:11:04.920938+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:19:57.595042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919382+0000","last_change":"2023-10-08T16:11:04.914772+0000","last_active":"2023-10-08T16:11:04.919382+0000","last_peered":"2023-10-08T16:11:04.919382+0000","last_clean":"2023-10-08T16:11:04.919382+0000","last_became_active":"2023-10-08T16:11:04.914516+0000","last_became_peered":"2023-10-08T16:11:04.914516+0000","last_unstale":"2023-10-08T16:11:04.919382+0000","last_undegraded":"2023-10-08T16:11:04.919382+0000","last_fullsized":"2023-10-08T16:11:04.919382+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:20:20.645918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921734+0000","last_change":"2023-10-08T16:11:04.914199+0000","last_active":"2023-10-08T16:11:04.921734+0000","last_peered":"2023-10-08T16:11:04.921734+0000","last_clean":"2023-10-08T16:11:04.921734+0000","last_became_active":"2023-10-08T16:11:04.913909+0000","last_became_peered":"2023-10-08T16:11:04.913909+0000","last_unstale":"2023-10-08T16:11:04.921734+0000","last_undegraded":"2023-10-08T16:11:04.921734+0000","last_fullsized":"2023-10-08T16:11:04.921734+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926864+0000","last_change":"2023-10-08T16:11:04.922659+0000","last_active":"2023-10-08T16:11:04.926864+0000","last_peered":"2023-10-08T16:11:04.926864+0000","last_clean":"2023-10-08T16:11:04.926864+0000","last_became_active":"2023-10-08T16:11:04.922417+0000","last_became_peered":"2023-10-08T16:11:04.922417+0000","last_unstale":"2023-10-08T16:11:04.926864+0000","last_undegraded":"2023-10-08T16:11:04.926864+0000","last_fullsized":"2023-10-08T16:11:04.926864+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:55.936020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.925709+0000","last_change":"2023-10-08T16:11:04.922107+0000","last_active":"2023-10-08T16:11:04.925709+0000","last_peered":"2023-10-08T16:11:04.925709+0000","last_clean":"2023-10-08T16:11:04.925709+0000","last_became_active":"2023-10-08T16:11:04.921812+0000","last_became_peered":"2023-10-08T16:11:04.921812+0000","last_unstale":"2023-10-08T16:11:04.925709+0000","last_undegraded":"2023-10-08T16:11:04.925709+0000","last_fullsized":"2023-10-08T16:11:04.925709+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:46.172756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919497+0000","last_change":"2023-10-08T16:11:04.916903+0000","last_active":"2023-10-08T16:11:04.919497+0000","last_peered":"2023-10-08T16:11:04.919497+0000","last_clean":"2023-10-08T16:11:04.919497+0000","last_became_active":"2023-10-08T16:11:04.916650+0000","last_became_peered":"2023-10-08T16:11:04.916650+0000","last_unstale":"2023-10-08T16:11:04.919497+0000","last_undegraded":"2023-10-08T16:11:04.919497+0000","last_fullsized":"2023-10-08T16:11:04.919497+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:11:21.462776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.940056+0000","last_change":"2023-10-08T16:11:04.931918+0000","last_active":"2023-10-08T16:11:04.940056+0000","last_peered":"2023-10-08T16:11:04.940056+0000","last_clean":"2023-10-08T16:11:04.940056+0000","last_became_active":"2023-10-08T16:11:04.931651+0000","last_became_peered":"2023-10-08T16:11:04.931651+0000","last_unstale":"2023-10-08T16:11:04.940056+0000","last_undegraded":"2023-10-08T16:11:04.940056+0000","last_fullsized":"2023-10-08T16:11:04.940056+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:22:12.609330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921995+0000","last_change":"2023-10-08T16:11:04.920165+0000","last_active":"2023-10-08T16:11:04.921995+0000","last_peered":"2023-10-08T16:11:04.921995+0000","last_clean":"2023-10-08T16:11:04.921995+0000","last_became_active":"2023-10-08T16:11:04.919646+0000","last_became_peered":"2023-10-08T16:11:04.919646+0000","last_unstale":"2023-10-08T16:11:04.921995+0000","last_undegraded":"2023-10-08T16:11:04.921995+0000","last_fullsized":"2023-10-08T16:11:04.921995+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:02:09.809190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927218+0000","last_change":"2023-10-08T16:11:04.918212+0000","last_active":"2023-10-08T16:11:04.927218+0000","last_peered":"2023-10-08T16:11:04.927218+0000","last_clean":"2023-10-08T16:11:04.927218+0000","last_became_active":"2023-10-08T16:11:04.917959+0000","last_became_peered":"2023-10-08T16:11:04.917959+0000","last_unstale":"2023-10-08T16:11:04.927218+0000","last_undegraded":"2023-10-08T16:11:04.927218+0000","last_fullsized":"2023-10-08T16:11:04.927218+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:15.423357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919326+0000","last_change":"2023-10-08T16:11:04.917917+0000","last_active":"2023-10-08T16:11:04.919326+0000","last_peered":"2023-10-08T16:11:04.919326+0000","last_clean":"2023-10-08T16:11:04.919326+0000","last_became_active":"2023-10-08T16:11:04.917657+0000","last_became_peered":"2023-10-08T16:11:04.917657+0000","last_unstale":"2023-10-08T16:11:04.919326+0000","last_undegraded":"2023-10-08T16:11:04.919326+0000","last_fullsized":"2023-10-08T16:11:04.919326+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:46:58.792657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928478+0000","last_change":"2023-10-08T16:11:04.928086+0000","last_active":"2023-10-08T16:11:04.928478+0000","last_peered":"2023-10-08T16:11:04.928478+0000","last_clean":"2023-10-08T16:11:04.928478+0000","last_became_active":"2023-10-08T16:11:04.927812+0000","last_became_peered":"2023-10-08T16:11:04.927812+0000","last_unstale":"2023-10-08T16:11:04.928478+0000","last_undegraded":"2023-10-08T16:11:04.928478+0000","last_fullsized":"2023-10-08T16:11:04.928478+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:39.384788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919523+0000","last_change":"2023-10-08T16:11:04.915488+0000","last_active":"2023-10-08T16:11:04.919523+0000","last_peered":"2023-10-08T16:11:04.919523+0000","last_clean":"2023-10-08T16:11:04.919523+0000","last_became_active":"2023-10-08T16:11:04.915243+0000","last_became_peered":"2023-10-08T16:11:04.915243+0000","last_unstale":"2023-10-08T16:11:04.919523+0000","last_undegraded":"2023-10-08T16:11:04.919523+0000","last_fullsized":"2023-10-08T16:11:04.919523+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:55:11.783919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920719+0000","last_change":"2023-10-08T16:11:04.915681+0000","last_active":"2023-10-08T16:11:04.920719+0000","last_peered":"2023-10-08T16:11:04.920719+0000","last_clean":"2023-10-08T16:11:04.920719+0000","last_became_active":"2023-10-08T16:11:04.915282+0000","last_became_peered":"2023-10-08T16:11:04.915282+0000","last_unstale":"2023-10-08T16:11:04.920719+0000","last_undegraded":"2023-10-08T16:11:04.920719+0000","last_fullsized":"2023-10-08T16:11:04.920719+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:21.385845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926854+0000","last_change":"2023-10-08T16:11:04.918001+0000","last_active":"2023-10-08T16:11:04.926854+0000","last_peered":"2023-10-08T16:11:04.926854+0000","last_clean":"2023-10-08T16:11:04.926854+0000","last_became_active":"2023-10-08T16:11:04.917731+0000","last_became_peered":"2023-10-08T16:11:04.917731+0000","last_unstale":"2023-10-08T16:11:04.926854+0000","last_undegraded":"2023-10-08T16:11:04.926854+0000","last_fullsized":"2023-10-08T16:11:04.926854+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:04:16.758931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920993+0000","last_change":"2023-10-08T16:11:04.918673+0000","last_active":"2023-10-08T16:11:04.920993+0000","last_peered":"2023-10-08T16:11:04.920993+0000","last_clean":"2023-10-08T16:11:04.920993+0000","last_became_active":"2023-10-08T16:11:04.918200+0000","last_became_peered":"2023-10-08T16:11:04.918200+0000","last_unstale":"2023-10-08T16:11:04.920993+0000","last_undegraded":"2023-10-08T16:11:04.920993+0000","last_fullsized":"2023-10-08T16:11:04.920993+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:11:35.606792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926074+0000","last_change":"2023-10-08T16:11:04.919852+0000","last_active":"2023-10-08T16:11:04.926074+0000","last_peered":"2023-10-08T16:11:04.926074+0000","last_clean":"2023-10-08T16:11:04.926074+0000","last_became_active":"2023-10-08T16:11:04.918698+0000","last_became_peered":"2023-10-08T16:11:04.918698+0000","last_unstale":"2023-10-08T16:11:04.926074+0000","last_undegraded":"2023-10-08T16:11:04.926074+0000","last_fullsized":"2023-10-08T16:11:04.926074+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:08:37.183227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930864+0000","last_change":"2023-10-08T16:11:04.925095+0000","last_active":"2023-10-08T16:11:04.930864+0000","last_peered":"2023-10-08T16:11:04.930864+0000","last_clean":"2023-10-08T16:11:04.930864+0000","last_became_active":"2023-10-08T16:11:04.924795+0000","last_became_peered":"2023-10-08T16:11:04.924795+0000","last_unstale":"2023-10-08T16:11:04.930864+0000","last_undegraded":"2023-10-08T16:11:04.930864+0000","last_fullsized":"2023-10-08T16:11:04.930864+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:02.301080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921979+0000","last_change":"2023-10-08T16:11:04.915631+0000","last_active":"2023-10-08T16:11:04.921979+0000","last_peered":"2023-10-08T16:11:04.921979+0000","last_clean":"2023-10-08T16:11:04.921979+0000","last_became_active":"2023-10-08T16:11:04.915357+0000","last_became_peered":"2023-10-08T16:11:04.915357+0000","last_unstale":"2023-10-08T16:11:04.921979+0000","last_undegraded":"2023-10-08T16:11:04.921979+0000","last_fullsized":"2023-10-08T16:11:04.921979+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:08.593892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919299+0000","last_change":"2023-10-08T16:11:04.912221+0000","last_active":"2023-10-08T16:11:04.919299+0000","last_peered":"2023-10-08T16:11:04.919299+0000","last_clean":"2023-10-08T16:11:04.919299+0000","last_became_active":"2023-10-08T16:11:04.911981+0000","last_became_peered":"2023-10-08T16:11:04.911981+0000","last_unstale":"2023-10-08T16:11:04.919299+0000","last_undegraded":"2023-10-08T16:11:04.919299+0000","last_fullsized":"2023-10-08T16:11:04.919299+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:31.226054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"112'4","reported_seq":14,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.939993+0000","last_change":"2023-10-08T16:11:03.911904+0000","last_active":"2023-10-08T16:11:04.939993+0000","last_peered":"2023-10-08T16:11:04.939993+0000","last_clean":"2023-10-08T16:11:04.939993+0000","last_became_active":"2023-10-08T16:11:03.911610+0000","last_became_peered":"2023-10-08T16:11:03.911610+0000","last_unstale":"2023-10-08T16:11:04.939993+0000","last_undegraded":"2023-10-08T16:11:04.939993+0000","last_fullsized":"2023-10-08T16:11:04.939993+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:40.504190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931783+0000","last_change":"2023-10-08T16:11:04.921001+0000","last_active":"2023-10-08T16:11:04.931783+0000","last_peered":"2023-10-08T16:11:04.931783+0000","last_clean":"2023-10-08T16:11:04.931783+0000","last_became_active":"2023-10-08T16:11:04.918800+0000","last_became_peered":"2023-10-08T16:11:04.918800+0000","last_unstale":"2023-10-08T16:11:04.931783+0000","last_undegraded":"2023-10-08T16:11:04.931783+0000","last_fullsized":"2023-10-08T16:11:04.931783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:28:14.470202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929001+0000","last_change":"2023-10-08T16:11:03.908616+0000","last_active":"2023-10-08T16:11:04.929001+0000","last_peered":"2023-10-08T16:11:04.929001+0000","last_clean":"2023-10-08T16:11:04.929001+0000","last_became_active":"2023-10-08T16:11:03.908311+0000","last_became_peered":"2023-10-08T16:11:03.908311+0000","last_unstale":"2023-10-08T16:11:04.929001+0000","last_undegraded":"2023-10-08T16:11:04.929001+0000","last_fullsized":"2023-10-08T16:11:04.929001+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934059+0000","last_change":"2023-10-08T16:11:04.927373+0000","last_active":"2023-10-08T16:11:04.934059+0000","last_peered":"2023-10-08T16:11:04.934059+0000","last_clean":"2023-10-08T16:11:04.934059+0000","last_became_active":"2023-10-08T16:11:04.926985+0000","last_became_peered":"2023-10-08T16:11:04.926985+0000","last_unstale":"2023-10-08T16:11:04.934059+0000","last_undegraded":"2023-10-08T16:11:04.934059+0000","last_fullsized":"2023-10-08T16:11:04.934059+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:55.332121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928276+0000","last_change":"2023-10-08T16:11:03.907369+0000","last_active":"2023-10-08T16:11:04.928276+0000","last_peered":"2023-10-08T16:11:04.928276+0000","last_clean":"2023-10-08T16:11:04.928276+0000","last_became_active":"2023-10-08T16:11:03.907119+0000","last_became_peered":"2023-10-08T16:11:03.907119+0000","last_unstale":"2023-10-08T16:11:04.928276+0000","last_undegraded":"2023-10-08T16:11:04.928276+0000","last_fullsized":"2023-10-08T16:11:04.928276+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:41.469949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930714+0000","last_change":"2023-10-08T16:11:04.924240+0000","last_active":"2023-10-08T16:11:04.930714+0000","last_peered":"2023-10-08T16:11:04.930714+0000","last_clean":"2023-10-08T16:11:04.930714+0000","last_became_active":"2023-10-08T16:11:04.923879+0000","last_became_peered":"2023-10-08T16:11:04.923879+0000","last_unstale":"2023-10-08T16:11:04.930714+0000","last_undegraded":"2023-10-08T16:11:04.930714+0000","last_fullsized":"2023-10-08T16:11:04.930714+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:22:57.647866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927909+0000","last_change":"2023-10-08T16:11:03.901096+0000","last_active":"2023-10-08T16:11:04.927909+0000","last_peered":"2023-10-08T16:11:04.927909+0000","last_clean":"2023-10-08T16:11:04.927909+0000","last_became_active":"2023-10-08T16:11:03.900784+0000","last_became_peered":"2023-10-08T16:11:03.900784+0000","last_unstale":"2023-10-08T16:11:04.927909+0000","last_undegraded":"2023-10-08T16:11:04.927909+0000","last_fullsized":"2023-10-08T16:11:04.927909+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:20:33.310661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919537+0000","last_change":"2023-10-08T16:11:04.916963+0000","last_active":"2023-10-08T16:11:04.919537+0000","last_peered":"2023-10-08T16:11:04.919537+0000","last_clean":"2023-10-08T16:11:04.919537+0000","last_became_active":"2023-10-08T16:11:04.916728+0000","last_became_peered":"2023-10-08T16:11:04.916728+0000","last_unstale":"2023-10-08T16:11:04.919537+0000","last_undegraded":"2023-10-08T16:11:04.919537+0000","last_fullsized":"2023-10-08T16:11:04.919537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:37:21.077125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926986+0000","last_change":"2023-10-08T16:11:03.907483+0000","last_active":"2023-10-08T16:11:04.926986+0000","last_peered":"2023-10-08T16:11:04.926986+0000","last_clean":"2023-10-08T16:11:04.926986+0000","last_became_active":"2023-10-08T16:11:03.906971+0000","last_became_peered":"2023-10-08T16:11:03.906971+0000","last_unstale":"2023-10-08T16:11:04.926986+0000","last_undegraded":"2023-10-08T16:11:04.926986+0000","last_fullsized":"2023-10-08T16:11:04.926986+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:27:42.596727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.925200+0000","last_change":"2023-10-08T16:11:04.919204+0000","last_active":"2023-10-08T16:11:04.925200+0000","last_peered":"2023-10-08T16:11:04.925200+0000","last_clean":"2023-10-08T16:11:04.925200+0000","last_became_active":"2023-10-08T16:11:04.918917+0000","last_became_peered":"2023-10-08T16:11:04.918917+0000","last_unstale":"2023-10-08T16:11:04.925200+0000","last_undegraded":"2023-10-08T16:11:04.925200+0000","last_fullsized":"2023-10-08T16:11:04.925200+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:22.789816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927423+0000","last_change":"2023-10-08T16:11:03.906390+0000","last_active":"2023-10-08T16:11:04.927423+0000","last_peered":"2023-10-08T16:11:04.927423+0000","last_clean":"2023-10-08T16:11:04.927423+0000","last_became_active":"2023-10-08T16:11:03.906138+0000","last_became_peered":"2023-10-08T16:11:03.906138+0000","last_unstale":"2023-10-08T16:11:04.927423+0000","last_undegraded":"2023-10-08T16:11:04.927423+0000","last_fullsized":"2023-10-08T16:11:04.927423+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:48.258705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.931882+0000","last_change":"2023-10-08T16:11:04.922183+0000","last_active":"2023-10-08T16:11:04.931882+0000","last_peered":"2023-10-08T16:11:04.931882+0000","last_clean":"2023-10-08T16:11:04.931882+0000","last_became_active":"2023-10-08T16:11:04.921875+0000","last_became_peered":"2023-10-08T16:11:04.921875+0000","last_unstale":"2023-10-08T16:11:04.931882+0000","last_undegraded":"2023-10-08T16:11:04.931882+0000","last_fullsized":"2023-10-08T16:11:04.931882+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:48.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904946+0000","last_change":"2023-10-08T16:11:03.906532+0000","last_active":"2023-10-08T16:11:04.904946+0000","last_peered":"2023-10-08T16:11:04.904946+0000","last_clean":"2023-10-08T16:11:04.904946+0000","last_became_active":"2023-10-08T16:11:03.906222+0000","last_became_peered":"2023-10-08T16:11:03.906222+0000","last_unstale":"2023-10-08T16:11:04.904946+0000","last_undegraded":"2023-10-08T16:11:04.904946+0000","last_fullsized":"2023-10-08T16:11:04.904946+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:47.712149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.915463+0000","last_change":"2023-10-08T16:11:04.913786+0000","last_active":"2023-10-08T16:11:04.915463+0000","last_peered":"2023-10-08T16:11:04.915463+0000","last_clean":"2023-10-08T16:11:04.915463+0000","last_became_active":"2023-10-08T16:11:04.913480+0000","last_became_peered":"2023-10-08T16:11:04.913480+0000","last_unstale":"2023-10-08T16:11:04.915463+0000","last_undegraded":"2023-10-08T16:11:04.915463+0000","last_fullsized":"2023-10-08T16:11:04.915463+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:47.965784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904058+0000","last_change":"2023-10-08T16:11:03.909969+0000","last_active":"2023-10-08T16:11:04.904058+0000","last_peered":"2023-10-08T16:11:04.904058+0000","last_clean":"2023-10-08T16:11:04.904058+0000","last_became_active":"2023-10-08T16:11:03.909698+0000","last_became_peered":"2023-10-08T16:11:03.909698+0000","last_unstale":"2023-10-08T16:11:04.904058+0000","last_undegraded":"2023-10-08T16:11:04.904058+0000","last_fullsized":"2023-10-08T16:11:04.904058+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:08.362073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926962+0000","last_change":"2023-10-08T16:11:04.921139+0000","last_active":"2023-10-08T16:11:04.926962+0000","last_peered":"2023-10-08T16:11:04.926962+0000","last_clean":"2023-10-08T16:11:04.926962+0000","last_became_active":"2023-10-08T16:11:04.918955+0000","last_became_peered":"2023-10-08T16:11:04.918955+0000","last_unstale":"2023-10-08T16:11:04.926962+0000","last_undegraded":"2023-10-08T16:11:04.926962+0000","last_fullsized":"2023-10-08T16:11:04.926962+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:45.264721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.905959+0000","last_change":"2023-10-08T16:11:03.912368+0000","last_active":"2023-10-08T16:11:04.905959+0000","last_peered":"2023-10-08T16:11:04.905959+0000","last_clean":"2023-10-08T16:11:04.905959+0000","last_became_active":"2023-10-08T16:11:03.912066+0000","last_became_peered":"2023-10-08T16:11:03.912066+0000","last_unstale":"2023-10-08T16:11:04.905959+0000","last_undegraded":"2023-10-08T16:11:04.905959+0000","last_fullsized":"2023-10-08T16:11:04.905959+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:24:52.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919308+0000","last_change":"2023-10-08T16:11:04.916646+0000","last_active":"2023-10-08T16:11:04.919308+0000","last_peered":"2023-10-08T16:11:04.919308+0000","last_clean":"2023-10-08T16:11:04.919308+0000","last_became_active":"2023-10-08T16:11:04.914613+0000","last_became_peered":"2023-10-08T16:11:04.914613+0000","last_unstale":"2023-10-08T16:11:04.919308+0000","last_undegraded":"2023-10-08T16:11:04.919308+0000","last_fullsized":"2023-10-08T16:11:04.919308+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:46:38.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911708+0000","last_change":"2023-10-08T16:11:03.910705+0000","last_active":"2023-10-08T16:11:04.911708+0000","last_peered":"2023-10-08T16:11:04.911708+0000","last_clean":"2023-10-08T16:11:04.911708+0000","last_became_active":"2023-10-08T16:11:03.910321+0000","last_became_peered":"2023-10-08T16:11:03.910321+0000","last_unstale":"2023-10-08T16:11:04.911708+0000","last_undegraded":"2023-10-08T16:11:04.911708+0000","last_fullsized":"2023-10-08T16:11:04.911708+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:57:29.783953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.933986+0000","last_change":"2023-10-08T16:11:04.926768+0000","last_active":"2023-10-08T16:11:04.933986+0000","last_peered":"2023-10-08T16:11:04.933986+0000","last_clean":"2023-10-08T16:11:04.933986+0000","last_became_active":"2023-10-08T16:11:04.926469+0000","last_became_peered":"2023-10-08T16:11:04.926469+0000","last_unstale":"2023-10-08T16:11:04.933986+0000","last_undegraded":"2023-10-08T16:11:04.933986+0000","last_fullsized":"2023-10-08T16:11:04.933986+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:03:42.474298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"112'3","reported_seq":13,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934136+0000","last_change":"2023-10-08T16:11:03.911284+0000","last_active":"2023-10-08T16:11:04.934136+0000","last_peered":"2023-10-08T16:11:04.934136+0000","last_clean":"2023-10-08T16:11:04.934136+0000","last_became_active":"2023-10-08T16:11:03.910992+0000","last_became_peered":"2023-10-08T16:11:03.910992+0000","last_unstale":"2023-10-08T16:11:04.934136+0000","last_undegraded":"2023-10-08T16:11:04.934136+0000","last_fullsized":"2023-10-08T16:11:04.934136+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:23:17.563101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929122+0000","last_change":"2023-10-08T16:11:04.924103+0000","last_active":"2023-10-08T16:11:04.929122+0000","last_peered":"2023-10-08T16:11:04.929122+0000","last_clean":"2023-10-08T16:11:04.929122+0000","last_became_active":"2023-10-08T16:11:04.923604+0000","last_became_peered":"2023-10-08T16:11:04.923604+0000","last_unstale":"2023-10-08T16:11:04.929122+0000","last_undegraded":"2023-10-08T16:11:04.929122+0000","last_fullsized":"2023-10-08T16:11:04.929122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:56.150939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"112'6","reported_seq":16,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.941751+0000","last_change":"2023-10-08T16:11:03.911078+0000","last_active":"2023-10-08T16:11:04.941751+0000","last_peered":"2023-10-08T16:11:04.941751+0000","last_clean":"2023-10-08T16:11:04.941751+0000","last_became_active":"2023-10-08T16:11:03.910522+0000","last_became_peered":"2023-10-08T16:11:03.910522+0000","last_unstale":"2023-10-08T16:11:04.941751+0000","last_undegraded":"2023-10-08T16:11:04.941751+0000","last_fullsized":"2023-10-08T16:11:04.941751+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920977+0000","last_change":"2023-10-08T16:11:04.918512+0000","last_active":"2023-10-08T16:11:04.920977+0000","last_peered":"2023-10-08T16:11:04.920977+0000","last_clean":"2023-10-08T16:11:04.920977+0000","last_became_active":"2023-10-08T16:11:04.917852+0000","last_became_peered":"2023-10-08T16:11:04.917852+0000","last_unstale":"2023-10-08T16:11:04.920977+0000","last_undegraded":"2023-10-08T16:11:04.920977+0000","last_fullsized":"2023-10-08T16:11:04.920977+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:26.927620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909933+0000","last_change":"2023-10-08T16:11:03.913269+0000","last_active":"2023-10-08T16:11:04.909933+0000","last_peered":"2023-10-08T16:11:04.909933+0000","last_clean":"2023-10-08T16:11:04.909933+0000","last_became_active":"2023-10-08T16:11:03.913044+0000","last_became_peered":"2023-10-08T16:11:03.913044+0000","last_unstale":"2023-10-08T16:11:04.909933+0000","last_undegraded":"2023-10-08T16:11:04.909933+0000","last_fullsized":"2023-10-08T16:11:04.909933+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:39:52.369697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920806+0000","last_change":"2023-10-08T16:11:04.916638+0000","last_active":"2023-10-08T16:11:04.920806+0000","last_peered":"2023-10-08T16:11:04.920806+0000","last_clean":"2023-10-08T16:11:04.920806+0000","last_became_active":"2023-10-08T16:11:04.916386+0000","last_became_peered":"2023-10-08T16:11:04.916386+0000","last_unstale":"2023-10-08T16:11:04.920806+0000","last_undegraded":"2023-10-08T16:11:04.920806+0000","last_fullsized":"2023-10-08T16:11:04.920806+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:55.151304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904127+0000","last_change":"2023-10-08T16:11:03.905844+0000","last_active":"2023-10-08T16:11:04.904127+0000","last_peered":"2023-10-08T16:11:04.904127+0000","last_clean":"2023-10-08T16:11:04.904127+0000","last_became_active":"2023-10-08T16:11:03.905597+0000","last_became_peered":"2023-10-08T16:11:03.905597+0000","last_unstale":"2023-10-08T16:11:04.904127+0000","last_undegraded":"2023-10-08T16:11:04.904127+0000","last_fullsized":"2023-10-08T16:11:04.904127+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.927195+0000","last_change":"2023-10-08T16:11:04.918112+0000","last_active":"2023-10-08T16:11:04.927195+0000","last_peered":"2023-10-08T16:11:04.927195+0000","last_clean":"2023-10-08T16:11:04.927195+0000","last_became_active":"2023-10-08T16:11:04.917865+0000","last_became_peered":"2023-10-08T16:11:04.917865+0000","last_unstale":"2023-10-08T16:11:04.927195+0000","last_undegraded":"2023-10-08T16:11:04.927195+0000","last_fullsized":"2023-10-08T16:11:04.927195+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:07.668042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"112'8","reported_seq":18,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.936878+0000","last_change":"2023-10-08T16:11:03.915073+0000","last_active":"2023-10-08T16:11:04.936878+0000","last_peered":"2023-10-08T16:11:04.936878+0000","last_clean":"2023-10-08T16:11:04.936878+0000","last_became_active":"2023-10-08T16:11:03.914732+0000","last_became_peered":"2023-10-08T16:11:03.914732+0000","last_unstale":"2023-10-08T16:11:04.936878+0000","last_undegraded":"2023-10-08T16:11:04.936878+0000","last_fullsized":"2023-10-08T16:11:04.936878+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920447+0000","last_change":"2023-10-08T16:11:04.915424+0000","last_active":"2023-10-08T16:11:04.920447+0000","last_peered":"2023-10-08T16:11:04.920447+0000","last_clean":"2023-10-08T16:11:04.920447+0000","last_became_active":"2023-10-08T16:11:04.915153+0000","last_became_peered":"2023-10-08T16:11:04.915153+0000","last_unstale":"2023-10-08T16:11:04.920447+0000","last_undegraded":"2023-10-08T16:11:04.920447+0000","last_fullsized":"2023-10-08T16:11:04.920447+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:19:19.737985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934223+0000","last_change":"2023-10-08T16:11:03.906254+0000","last_active":"2023-10-08T16:11:04.934223+0000","last_peered":"2023-10-08T16:11:04.934223+0000","last_clean":"2023-10-08T16:11:04.934223+0000","last_became_active":"2023-10-08T16:11:03.905790+0000","last_became_peered":"2023-10-08T16:11:03.905790+0000","last_unstale":"2023-10-08T16:11:04.934223+0000","last_undegraded":"2023-10-08T16:11:04.934223+0000","last_fullsized":"2023-10-08T16:11:04.934223+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:26:44.410985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919329+0000","last_change":"2023-10-08T16:11:04.916958+0000","last_active":"2023-10-08T16:11:04.919329+0000","last_peered":"2023-10-08T16:11:04.919329+0000","last_clean":"2023-10-08T16:11:04.919329+0000","last_became_active":"2023-10-08T16:11:04.916700+0000","last_became_peered":"2023-10-08T16:11:04.916700+0000","last_unstale":"2023-10-08T16:11:04.919329+0000","last_undegraded":"2023-10-08T16:11:04.919329+0000","last_fullsized":"2023-10-08T16:11:04.919329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:18:39.237203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.936723+0000","last_change":"2023-10-08T16:11:03.910220+0000","last_active":"2023-10-08T16:11:04.936723+0000","last_peered":"2023-10-08T16:11:04.936723+0000","last_clean":"2023-10-08T16:11:04.936723+0000","last_became_active":"2023-10-08T16:11:03.909963+0000","last_became_peered":"2023-10-08T16:11:03.909963+0000","last_unstale":"2023-10-08T16:11:04.936723+0000","last_undegraded":"2023-10-08T16:11:04.936723+0000","last_fullsized":"2023-10-08T16:11:04.936723+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:27.529263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.919386+0000","last_change":"2023-10-08T16:11:04.914718+0000","last_active":"2023-10-08T16:11:04.919386+0000","last_peered":"2023-10-08T16:11:04.919386+0000","last_clean":"2023-10-08T16:11:04.919386+0000","last_became_active":"2023-10-08T16:11:04.914452+0000","last_became_peered":"2023-10-08T16:11:04.914452+0000","last_unstale":"2023-10-08T16:11:04.919386+0000","last_undegraded":"2023-10-08T16:11:04.919386+0000","last_fullsized":"2023-10-08T16:11:04.919386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:52:52.077063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"112'1","reported_seq":11,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.929412+0000","last_change":"2023-10-08T16:11:03.907316+0000","last_active":"2023-10-08T16:11:04.929412+0000","last_peered":"2023-10-08T16:11:04.929412+0000","last_clean":"2023-10-08T16:11:04.929412+0000","last_became_active":"2023-10-08T16:11:03.907061+0000","last_became_peered":"2023-10-08T16:11:03.907061+0000","last_unstale":"2023-10-08T16:11:04.929412+0000","last_undegraded":"2023-10-08T16:11:04.929412+0000","last_fullsized":"2023-10-08T16:11:04.929412+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:38.380438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.915591+0000","last_change":"2023-10-08T16:11:04.912949+0000","last_active":"2023-10-08T16:11:04.915591+0000","last_peered":"2023-10-08T16:11:04.915591+0000","last_clean":"2023-10-08T16:11:04.915591+0000","last_became_active":"2023-10-08T16:11:04.912691+0000","last_became_peered":"2023-10-08T16:11:04.912691+0000","last_unstale":"2023-10-08T16:11:04.915591+0000","last_undegraded":"2023-10-08T16:11:04.915591+0000","last_fullsized":"2023-10-08T16:11:04.915591+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:41:49.263054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.906021+0000","last_change":"2023-10-08T16:11:03.907179+0000","last_active":"2023-10-08T16:11:04.906021+0000","last_peered":"2023-10-08T16:11:04.906021+0000","last_clean":"2023-10-08T16:11:04.906021+0000","last_became_active":"2023-10-08T16:11:03.906946+0000","last_became_peered":"2023-10-08T16:11:03.906946+0000","last_unstale":"2023-10-08T16:11:04.906021+0000","last_undegraded":"2023-10-08T16:11:04.906021+0000","last_fullsized":"2023-10-08T16:11:04.906021+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:33.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926096+0000","last_change":"2023-10-08T16:11:04.920813+0000","last_active":"2023-10-08T16:11:04.926096+0000","last_peered":"2023-10-08T16:11:04.926096+0000","last_clean":"2023-10-08T16:11:04.926096+0000","last_became_active":"2023-10-08T16:11:04.918345+0000","last_became_peered":"2023-10-08T16:11:04.918345+0000","last_unstale":"2023-10-08T16:11:04.926096+0000","last_undegraded":"2023-10-08T16:11:04.926096+0000","last_fullsized":"2023-10-08T16:11:04.926096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:53.234208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930891+0000","last_change":"2023-10-08T16:11:03.908030+0000","last_active":"2023-10-08T16:11:04.930891+0000","last_peered":"2023-10-08T16:11:04.930891+0000","last_clean":"2023-10-08T16:11:04.930891+0000","last_became_active":"2023-10-08T16:11:03.907731+0000","last_became_peered":"2023-10-08T16:11:03.907731+0000","last_unstale":"2023-10-08T16:11:04.930891+0000","last_undegraded":"2023-10-08T16:11:04.930891+0000","last_fullsized":"2023-10-08T16:11:04.930891+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:07:10.071216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921606+0000","last_change":"2023-10-08T16:11:04.916141+0000","last_active":"2023-10-08T16:11:04.921606+0000","last_peered":"2023-10-08T16:11:04.921606+0000","last_clean":"2023-10-08T16:11:04.921606+0000","last_became_active":"2023-10-08T16:11:04.915845+0000","last_became_peered":"2023-10-08T16:11:04.915845+0000","last_unstale":"2023-10-08T16:11:04.921606+0000","last_undegraded":"2023-10-08T16:11:04.921606+0000","last_fullsized":"2023-10-08T16:11:04.921606+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:41.508540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909697+0000","last_change":"2023-10-08T16:11:03.909202+0000","last_active":"2023-10-08T16:11:04.909697+0000","last_peered":"2023-10-08T16:11:04.909697+0000","last_clean":"2023-10-08T16:11:04.909697+0000","last_became_active":"2023-10-08T16:11:03.908949+0000","last_became_peered":"2023-10-08T16:11:03.908949+0000","last_unstale":"2023-10-08T16:11:04.909697+0000","last_undegraded":"2023-10-08T16:11:04.909697+0000","last_fullsized":"2023-10-08T16:11:04.909697+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:24:54.146955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.916973+0000","last_change":"2023-10-08T16:11:04.915617+0000","last_active":"2023-10-08T16:11:04.916973+0000","last_peered":"2023-10-08T16:11:04.916973+0000","last_clean":"2023-10-08T16:11:04.916973+0000","last_became_active":"2023-10-08T16:11:04.915382+0000","last_became_peered":"2023-10-08T16:11:04.915382+0000","last_unstale":"2023-10-08T16:11:04.916973+0000","last_undegraded":"2023-10-08T16:11:04.916973+0000","last_fullsized":"2023-10-08T16:11:04.916973+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:51:48.716498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.908074+0000","last_change":"2023-10-08T16:11:03.907175+0000","last_active":"2023-10-08T16:11:04.908074+0000","last_peered":"2023-10-08T16:11:04.908074+0000","last_clean":"2023-10-08T16:11:04.908074+0000","last_became_active":"2023-10-08T16:11:03.906936+0000","last_became_peered":"2023-10-08T16:11:03.906936+0000","last_unstale":"2023-10-08T16:11:04.908074+0000","last_undegraded":"2023-10-08T16:11:04.908074+0000","last_fullsized":"2023-10-08T16:11:04.908074+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:28:46.961435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.933884+0000","last_change":"2023-10-08T16:11:04.925982+0000","last_active":"2023-10-08T16:11:04.933884+0000","last_peered":"2023-10-08T16:11:04.933884+0000","last_clean":"2023-10-08T16:11:04.933884+0000","last_became_active":"2023-10-08T16:11:04.925304+0000","last_became_peered":"2023-10-08T16:11:04.925304+0000","last_unstale":"2023-10-08T16:11:04.933884+0000","last_undegraded":"2023-10-08T16:11:04.933884+0000","last_fullsized":"2023-10-08T16:11:04.933884+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:28.533612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.910499+0000","last_change":"2023-10-08T16:11:03.910257+0000","last_active":"2023-10-08T16:11:04.910499+0000","last_peered":"2023-10-08T16:11:04.910499+0000","last_clean":"2023-10-08T16:11:04.910499+0000","last_became_active":"2023-10-08T16:11:03.909995+0000","last_became_peered":"2023-10-08T16:11:03.909995+0000","last_unstale":"2023-10-08T16:11:04.910499+0000","last_undegraded":"2023-10-08T16:11:04.910499+0000","last_fullsized":"2023-10-08T16:11:04.910499+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:01.296731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920887+0000","last_change":"2023-10-08T16:11:04.913585+0000","last_active":"2023-10-08T16:11:04.920887+0000","last_peered":"2023-10-08T16:11:04.920887+0000","last_clean":"2023-10-08T16:11:04.920887+0000","last_became_active":"2023-10-08T16:11:04.912650+0000","last_became_peered":"2023-10-08T16:11:04.912650+0000","last_unstale":"2023-10-08T16:11:04.920887+0000","last_undegraded":"2023-10-08T16:11:04.920887+0000","last_fullsized":"2023-10-08T16:11:04.920887+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:40.253632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.934304+0000","last_change":"2023-10-08T16:11:03.909589+0000","last_active":"2023-10-08T16:11:04.934304+0000","last_peered":"2023-10-08T16:11:04.934304+0000","last_clean":"2023-10-08T16:11:04.934304+0000","last_became_active":"2023-10-08T16:11:03.909342+0000","last_became_peered":"2023-10-08T16:11:03.909342+0000","last_unstale":"2023-10-08T16:11:04.934304+0000","last_undegraded":"2023-10-08T16:11:04.934304+0000","last_fullsized":"2023-10-08T16:11:04.934304+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:27:10.377601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.926061+0000","last_change":"2023-10-08T16:11:04.921542+0000","last_active":"2023-10-08T16:11:04.926061+0000","last_peered":"2023-10-08T16:11:04.926061+0000","last_clean":"2023-10-08T16:11:04.926061+0000","last_became_active":"2023-10-08T16:11:04.921199+0000","last_became_peered":"2023-10-08T16:11:04.921199+0000","last_unstale":"2023-10-08T16:11:04.926061+0000","last_undegraded":"2023-10-08T16:11:04.926061+0000","last_fullsized":"2023-10-08T16:11:04.926061+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:45:38.890416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911910+0000","last_change":"2023-10-08T16:11:03.913709+0000","last_active":"2023-10-08T16:11:04.911910+0000","last_peered":"2023-10-08T16:11:04.911910+0000","last_clean":"2023-10-08T16:11:04.911910+0000","last_became_active":"2023-10-08T16:11:03.913436+0000","last_became_peered":"2023-10-08T16:11:03.913436+0000","last_unstale":"2023-10-08T16:11:04.911910+0000","last_undegraded":"2023-10-08T16:11:04.911910+0000","last_fullsized":"2023-10-08T16:11:04.911910+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:18:07.589542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928446+0000","last_change":"2023-10-08T16:11:04.926028+0000","last_active":"2023-10-08T16:11:04.928446+0000","last_peered":"2023-10-08T16:11:04.928446+0000","last_clean":"2023-10-08T16:11:04.928446+0000","last_became_active":"2023-10-08T16:11:04.925768+0000","last_became_peered":"2023-10-08T16:11:04.925768+0000","last_unstale":"2023-10-08T16:11:04.928446+0000","last_undegraded":"2023-10-08T16:11:04.928446+0000","last_fullsized":"2023-10-08T16:11:04.928446+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:06:44.041214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.904545+0000","last_change":"2023-10-08T16:11:03.909139+0000","last_active":"2023-10-08T16:11:04.904545+0000","last_peered":"2023-10-08T16:11:04.904545+0000","last_clean":"2023-10-08T16:11:04.904545+0000","last_became_active":"2023-10-08T16:11:03.908877+0000","last_became_peered":"2023-10-08T16:11:03.908877+0000","last_unstale":"2023-10-08T16:11:04.904545+0000","last_undegraded":"2023-10-08T16:11:04.904545+0000","last_fullsized":"2023-10-08T16:11:04.904545+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:43:48.689812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.921672+0000","last_change":"2023-10-08T16:11:04.916524+0000","last_active":"2023-10-08T16:11:04.921672+0000","last_peered":"2023-10-08T16:11:04.921672+0000","last_clean":"2023-10-08T16:11:04.921672+0000","last_became_active":"2023-10-08T16:11:04.916095+0000","last_became_peered":"2023-10-08T16:11:04.916095+0000","last_unstale":"2023-10-08T16:11:04.921672+0000","last_undegraded":"2023-10-08T16:11:04.921672+0000","last_fullsized":"2023-10-08T16:11:04.921672+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:29:45.729968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.913045+0000","last_change":"2023-10-08T16:11:03.905282+0000","last_active":"2023-10-08T16:11:04.913045+0000","last_peered":"2023-10-08T16:11:04.913045+0000","last_clean":"2023-10-08T16:11:04.913045+0000","last_became_active":"2023-10-08T16:11:03.905030+0000","last_became_peered":"2023-10-08T16:11:03.905030+0000","last_unstale":"2023-10-08T16:11:04.913045+0000","last_undegraded":"2023-10-08T16:11:04.913045+0000","last_fullsized":"2023-10-08T16:11:04.913045+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:11:54.327772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930785+0000","last_change":"2023-10-08T16:11:04.924999+0000","last_active":"2023-10-08T16:11:04.930785+0000","last_peered":"2023-10-08T16:11:04.930785+0000","last_clean":"2023-10-08T16:11:04.930785+0000","last_became_active":"2023-10-08T16:11:04.924577+0000","last_became_peered":"2023-10-08T16:11:04.924577+0000","last_unstale":"2023-10-08T16:11:04.930785+0000","last_undegraded":"2023-10-08T16:11:04.930785+0000","last_fullsized":"2023-10-08T16:11:04.930785+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:56:53.038890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.903825+0000","last_change":"2023-10-08T16:11:03.907637+0000","last_active":"2023-10-08T16:11:04.903825+0000","last_peered":"2023-10-08T16:11:04.903825+0000","last_clean":"2023-10-08T16:11:04.903825+0000","last_became_active":"2023-10-08T16:11:03.907270+0000","last_became_peered":"2023-10-08T16:11:03.907270+0000","last_unstale":"2023-10-08T16:11:04.903825+0000","last_undegraded":"2023-10-08T16:11:04.903825+0000","last_fullsized":"2023-10-08T16:11:04.903825+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:38.043479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928972+0000","last_change":"2023-10-08T16:11:04.923256+0000","last_active":"2023-10-08T16:11:04.928972+0000","last_peered":"2023-10-08T16:11:04.928972+0000","last_clean":"2023-10-08T16:11:04.928972+0000","last_became_active":"2023-10-08T16:11:04.923012+0000","last_became_peered":"2023-10-08T16:11:04.923012+0000","last_unstale":"2023-10-08T16:11:04.928972+0000","last_undegraded":"2023-10-08T16:11:04.928972+0000","last_fullsized":"2023-10-08T16:11:04.928972+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:06.584044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911880+0000","last_change":"2023-10-08T16:11:03.909548+0000","last_active":"2023-10-08T16:11:04.911880+0000","last_peered":"2023-10-08T16:11:04.911880+0000","last_clean":"2023-10-08T16:11:04.911880+0000","last_became_active":"2023-10-08T16:11:03.909225+0000","last_became_peered":"2023-10-08T16:11:03.909225+0000","last_unstale":"2023-10-08T16:11:04.911880+0000","last_undegraded":"2023-10-08T16:11:04.911880+0000","last_fullsized":"2023-10-08T16:11:04.911880+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:46:39.249283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.920179+0000","last_change":"2023-10-08T16:11:04.916787+0000","last_active":"2023-10-08T16:11:04.920179+0000","last_peered":"2023-10-08T16:11:04.920179+0000","last_clean":"2023-10-08T16:11:04.920179+0000","last_became_active":"2023-10-08T16:11:04.916360+0000","last_became_peered":"2023-10-08T16:11:04.916360+0000","last_unstale":"2023-10-08T16:11:04.920179+0000","last_undegraded":"2023-10-08T16:11:04.920179+0000","last_fullsized":"2023-10-08T16:11:04.920179+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:46.114575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.909235+0000","last_change":"2023-10-08T16:11:03.910568+0000","last_active":"2023-10-08T16:11:04.909235+0000","last_peered":"2023-10-08T16:11:04.909235+0000","last_clean":"2023-10-08T16:11:04.909235+0000","last_became_active":"2023-10-08T16:11:03.910311+0000","last_became_peered":"2023-10-08T16:11:03.910311+0000","last_unstale":"2023-10-08T16:11:04.909235+0000","last_undegraded":"2023-10-08T16:11:04.909235+0000","last_fullsized":"2023-10-08T16:11:04.909235+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:30:56.735575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.928227+0000","last_change":"2023-10-08T16:11:04.926281+0000","last_active":"2023-10-08T16:11:04.928227+0000","last_peered":"2023-10-08T16:11:04.928227+0000","last_clean":"2023-10-08T16:11:04.928227+0000","last_became_active":"2023-10-08T16:11:04.925981+0000","last_became_peered":"2023-10-08T16:11:04.925981+0000","last_unstale":"2023-10-08T16:11:04.928227+0000","last_undegraded":"2023-10-08T16:11:04.928227+0000","last_fullsized":"2023-10-08T16:11:04.928227+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:23.716868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.910596+0000","last_change":"2023-10-08T16:11:03.906699+0000","last_active":"2023-10-08T16:11:04.910596+0000","last_peered":"2023-10-08T16:11:04.910596+0000","last_clean":"2023-10-08T16:11:04.910596+0000","last_became_active":"2023-10-08T16:11:03.906348+0000","last_became_peered":"2023-10-08T16:11:03.906348+0000","last_unstale":"2023-10-08T16:11:04.910596+0000","last_undegraded":"2023-10-08T16:11:04.910596+0000","last_fullsized":"2023-10-08T16:11:04.910596+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:34.478940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.930763+0000","last_change":"2023-10-08T16:11:04.924954+0000","last_active":"2023-10-08T16:11:04.930763+0000","last_peered":"2023-10-08T16:11:04.930763+0000","last_clean":"2023-10-08T16:11:04.930763+0000","last_became_active":"2023-10-08T16:11:04.924604+0000","last_became_peered":"2023-10-08T16:11:04.924604+0000","last_unstale":"2023-10-08T16:11:04.930763+0000","last_undegraded":"2023-10-08T16:11:04.930763+0000","last_fullsized":"2023-10-08T16:11:04.930763+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:03.890066+0000","last_clean_scrub_stamp":"2023-10-08T16:11:03.890066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:07:35.483289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"112'2","reported_seq":12,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.940007+0000","last_change":"2023-10-08T16:11:03.911957+0000","last_active":"2023-10-08T16:11:04.940007+0000","last_peered":"2023-10-08T16:11:04.940007+0000","last_clean":"2023-10-08T16:11:04.940007+0000","last_became_active":"2023-10-08T16:11:03.911652+0000","last_became_peered":"2023-10-08T16:11:03.911652+0000","last_unstale":"2023-10-08T16:11:04.940007+0000","last_undegraded":"2023-10-08T16:11:04.940007+0000","last_fullsized":"2023-10-08T16:11:04.940007+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:02.885717+0000","last_clean_scrub_stamp":"2023-10-08T16:11:02.885717+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:45:21.785467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":239,"reported_epoch":112,"state":"active+clean","last_fresh":"2023-10-08T16:11:04.911425+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:04.911425+0000","last_peered":"2023-10-08T16:11:04.911425+0000","last_clean":"2023-10-08T16:11:04.911425+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:04.911425+0000","last_undegraded":"2023-10-08T16:11:04.911425+0000","last_fullsized":"2023-10-08T16:11:04.911425+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303368,"kb_used_data":1984,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068472,"statfs":{"total":96636764160,"available":96326115328,"internally_reserved":0,"allocated":2031616,"data_stored":1203543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.50700000000000001,"15min":0.78100000000000003},"min":{"1min":0.38700000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.436}]},{"osd":1,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52500000000000002,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":1.083},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.81699999999999995},"min":{"1min":0.38,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.63100000000000001,"5min":0.88800000000000001,"15min":8.2040000000000006},"last":0.38200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.498,"15min":0.54100000000000004},"min":{"1min":0.40400000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68300000000000005,"5min":0.77800000000000002,"15min":1.175},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53500000000000003,"15min":0.84299999999999997},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":8.4339999999999993},"last":0.48799999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52300000000000002,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.214},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.84599999999999997},"min":{"1min":0.41399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.94999999999999996,"15min":8.3409999999999993},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45300000000000001,"15min":0.48899999999999999},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.314},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.55000000000000004},"min":{"1min":0.44600000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.73299999999999998,"15min":0.752},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.38400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.52800000000000002,"15min":0.55800000000000005},"min":{"1min":0.38600000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.748,"5min":0.752,"15min":0.752},"last":0.497}]},{"osd":6,"last update":"Sun Oct 8 16:10:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.77400000000000002,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.47299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302776,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069064,"statfs":{"total":96636764160,"available":96326721536,"internally_reserved":0,"allocated":1425408,"data_stored":612565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.48999999999999999},"min":{"1min":0.36099999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.65300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.52400000000000002,"15min":0.52600000000000002},"min":{"1min":0.32600000000000001,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.68700000000000006,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51300000000000001,"15min":0.49399999999999999},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.68200000000000005,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.35899999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70799999999999996,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.35499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.34999999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54000000000000004,"15min":0.53200000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64700000000000002,"5min":0.75,"15min":0.96599999999999997},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.55700000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69799999999999995,"5min":0.745,"15min":0.94199999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42299999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.72099999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.51600000000000001},"min":{"1min":0.28399999999999997,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51600000000000001,"15min":0.53400000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.58499999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.52400000000000002},"min":{"1min":0.36599999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.74199999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56000000000000005,"15min":0.58499999999999996},"min":{"1min":0.38100000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.874},"last":0.53200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56899999999999995,"15min":0.59499999999999997},"min":{"1min":0.38500000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.75,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.60899999999999999},"min":{"1min":0.48899999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302764,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069076,"statfs":{"total":96636764160,"available":96326733824,"internally_reserved":0,"allocated":1413120,"data_stored":611029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.30399999999999999,"15min":0.309},"min":{"1min":0.219,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.42199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.32900000000000001,"15min":0.33700000000000002},"min":{"1min":0.219,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.52100000000000002,"5min":0.58199999999999996,"15min":0.77500000000000002},"last":0.32200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.32200000000000001,"15min":0.35599999999999998},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.501,"15min":1.2230000000000001},"last":0.376},{"interface":"front","average":{"1min":0.378,"5min":0.35299999999999998,"15min":0.372},"min":{"1min":0.28199999999999997,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.48399999999999999,"5min":0.56299999999999994,"15min":0.72399999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.38600000000000001},"min":{"1min":0.28799999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.70399999999999996},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.35799999999999998},"min":{"1min":0.255,"5min":0.215,"15min":0.215},"max":{"1min":0.51200000000000001,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.30299999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":1.077},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55300000000000005,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":1.696,"15min":1.696},"last":0.46500000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.55200000000000005,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.77000000000000002,"5min":1.657,"15min":1.657},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.52000000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.099},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.40699999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83099999999999996,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 16:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.50600000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.56100000000000005,"15min":0.59499999999999997},"min":{"1min":0.48599999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.1839999999999999},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303344,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068496,"statfs":{"total":96636764160,"available":96326139904,"internally_reserved":0,"allocated":2007040,"data_stored":1201397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.318,"15min":0.53400000000000003},"min":{"1min":0.23499999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.32700000000000001,"15min":0.46899999999999997},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":3.3220000000000001},"last":0.33100000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.56699999999999995},"min":{"1min":0.25800000000000001,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.51900000000000002,"5min":0.53400000000000003,"15min":5.6769999999999996},"last":0.376},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.379},"min":{"1min":0.26800000000000002,"5min":0.219,"15min":0.219},"max":{"1min":0.45500000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.36799999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34000000000000002,"15min":0.47799999999999998},"min":{"1min":0.22700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.434,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.317},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.60399999999999998},"min":{"1min":0.29899999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55100000000000005,"5min":1.498,"15min":5.5970000000000004},"last":0.40200000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.35899999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55500000000000005,"15min":0.66300000000000003},"min":{"1min":0.46400000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70499999999999996,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.55600000000000005,"15min":0.66000000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75700000000000001,"5min":1.847,"15min":3.2890000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56200000000000006,"15min":0.77800000000000002},"min":{"1min":0.47199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76300000000000001,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.54900000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.59999999999999998,"15min":0.80200000000000005},"min":{"1min":0.43099999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.753,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58899999999999997},"min":{"1min":0.42999999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.78200000000000003,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60599999999999998,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.754,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.60799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574931,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.27900000000000003,"5min":0.308,"15min":0.30599999999999999},"min":{"1min":0.22500000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.34300000000000003,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.32600000000000001,"15min":0.34899999999999998},"min":{"1min":0.25900000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.38700000000000001,"5min":0.52900000000000003,"15min":0.64300000000000002},"last":0.379}]},{"osd":1,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33700000000000002,"15min":0.34899999999999998},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.371,"5min":0.57699999999999996,"15min":0.58499999999999996},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.34799999999999998,"15min":0.35899999999999999},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.42299999999999999,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.35199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33100000000000002,"15min":0.34599999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.35899999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.25},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.379,"15min":0.38700000000000001},"min":{"1min":0.29299999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46100000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.39100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.78500000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73899999999999999,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.67100000000000004},"min":{"1min":0.42899999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58999999999999997,"15min":0.57899999999999996},"min":{"1min":0.45200000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.58399999999999996},"min":{"1min":0.44,"5min":0.39900000000000002,"15min":0.39300000000000002},"max":{"1min":0.80200000000000005,"5min":0.871,"15min":0.871},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.42499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.61099999999999999,"15min":0.623},"min":{"1min":0.46800000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83199999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.314,"15min":0.35199999999999998},"min":{"1min":0.19800000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.505,"5min":0.53600000000000003,"15min":1.238},"last":0.251},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.33200000000000002,"15min":0.38800000000000001},"min":{"1min":0.22900000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.33100000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.33100000000000002,"15min":0.35599999999999998},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.39700000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.29499999999999998,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.497,"5min":0.59299999999999997,"15min":1.3520000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36299999999999999,"15min":0.39500000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.248},"max":{"1min":0.47599999999999998,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.42099999999999999},"min":{"1min":0.34699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.46999999999999997,"5min":0.56200000000000006,"15min":1.327},"last":0.29699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54300000000000004,"15min":0.60199999999999998},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.314},"max":{"1min":0.72799999999999998,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.40400000000000003,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.65700000000000003,"5min":1.099,"15min":1.099},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54100000000000004,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73799999999999999,"5min":0.77900000000000003,"15min":1.27},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.70299999999999996,"5min":0.91900000000000004,"15min":1.879},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.57199999999999995,"15min":0.60599999999999998},"min":{"1min":0.41499999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.432,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78200000000000003,"5min":0.97599999999999998,"15min":1.825},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60299999999999998,"15min":0.63},"min":{"1min":0.496,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":1.0009999999999999,"15min":1.115},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302768,"kb_used_data":1384,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069072,"statfs":{"total":96636764160,"available":96326729728,"internally_reserved":0,"allocated":1417216,"data_stored":611099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.5,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65700000000000003,"5min":0.746,"15min":1.579},"last":0.501},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52400000000000002,"15min":0.50800000000000001},"min":{"1min":0.379,"5min":0.32200000000000001,"15min":0.31900000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55000000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51000000000000001,"15min":0.48899999999999999},"min":{"1min":0.35099999999999998,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72599999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.36399999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.70599999999999996,"15min":0.80700000000000005},"last":0.57199999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.58699999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.43099999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58799999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.40899999999999997,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72699999999999998,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56100000000000005}]},{"osd":5,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.502,"15min":0.52900000000000003},"min":{"1min":0.33900000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.80000000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.35899999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.56399999999999995},"min":{"1min":0.35299999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56599999999999995,"15min":0.58999999999999997},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53600000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:10:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.60899999999999999},"min":{"1min":0.52000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61099999999999999,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.47699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302772,"kb_used_data":1388,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069068,"statfs":{"total":96636764160,"available":96326725632,"internally_reserved":0,"allocated":1421312,"data_stored":611141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53200000000000003,"15min":0.64800000000000002},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":3.7770000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.51900000000000002,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.247,"15min":0.247},"max":{"1min":0.80400000000000005,"5min":0.83799999999999997,"15min":4.0119999999999996},"last":0.41699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.50900000000000001,"15min":1.01},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.503},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.55200000000000005,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":1.252},"last":0.46000000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.53700000000000003,"15min":0.66500000000000004},"min":{"1min":0.35599999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.77400000000000002,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.747,"5min":0.76100000000000001,"15min":1.444},"last":0.442}]},{"osd":3,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53700000000000003,"15min":0.96299999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.754,"5min":0.86499999999999999,"15min":9.3170000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56699999999999995,"15min":0.70199999999999996},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":3.8719999999999999},"last":0.61699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.49199999999999999,"15min":0.496},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67200000000000004,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51900000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68700000000000006,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64500000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.58099999999999996},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77900000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.41899999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:13.941 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:11:13.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.093+0000 7fcd93b79700 1 Processor -- start 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.093+0000 7fcd93b79700 1 -- start start 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd93b79700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcd8c069c70 0x7fcd8c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd93b79700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0ce380 con 0x7fcd8c0af320 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd93b79700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0cec20 con 0x7fcd8c159950 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd93b79700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c0afa90 con 0x7fcd8c069c70 2023-10-08T16:11:14.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd92116700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcd8c0af320 0x7fcd8c0cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59172/0 (socket says 172.21.15.170:59172) 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd92116700 1 -- 172.21.15.170:0/1705148103 learned_addr learned my addr 172.21.15.170:0/1705148103 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd91114700 1 --2- 172.21.15.170:0/1705148103 >> v2:172.21.15.191:3300/0 conn(0x7fcd8c069c70 0x7fcd8c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073696692 0 0) 0x7fcd8c0ce380 con 0x7fcd8c0af320 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.094+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd78003570 con 0x7fcd8c0af320 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355664726 0 0) 0x7fcd78003570 con 0x7fcd8c0af320 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 >> v2:172.21.15.191:3300/0 conn(0x7fcd8c069c70 msgr2=0x7fcd8c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 --2- 172.21.15.170:0/1705148103 >> v2:172.21.15.191:3300/0 conn(0x7fcd8c069c70 0x7fcd8c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 >> v1:172.21.15.191:6789/0 conn(0x7fcd8c159950 legacy=0x7fcd8c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0ce5d0 con 0x7fcd8c0af320 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fcd84001470 con 0x7fcd8c0af320 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 >> v1:172.21.15.170:6789/0 conn(0x7fcd8c0af320 legacy=0x7fcd8c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd78003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd78004810 0x7fcd78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd780051c0 0x7fcd7800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0afa90 con 0x7fcd78003a60 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800c160 con 0x7fcd78004810 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800c320 con 0x7fcd780051c0 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd78003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91114700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd78004810 0x7fcd78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd92116700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd780051c0 0x7fcd7800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 -- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd780051c0 msgr2=0x7fcd7800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd780051c0 0x7fcd7800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 -- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd78004810 msgr2=0x7fcd78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd78004810 0x7fcd78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 -- 172.21.15.170:0/1705148103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78003570 con 0x7fcd78003a60 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd92116700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd780051c0 0x7fcd7800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd92116700 1 -- 172.21.15.170:0/1705148103 reap_dead start 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd91915700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd78003e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd88001fc0 tx=0x7fcd88001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0b54bab3a54d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88001950 con 0x7fcd78003a60 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8800c8b0 con 0x7fcd78003a60 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.095+0000 7fcd90913700 1 -- 172.21.15.170:0/1705148103 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8800cc50 con 0x7fcd78003a60 2023-10-08T16:11:14.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 -- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 msgr2=0x7fcd78003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd78003e70 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd88001fc0 tx=0x7fcd88001640 comp rx=0 tx=0).stop 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 -- 172.21.15.170:0/1705148103 shutdown_connections 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1705148103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd78003e70 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 -- 172.21.15.170:0/1705148103 >> 172.21.15.170:0/1705148103 conn(0x7fcd8c068e00 msgr2=0x7fcd8c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 -- 172.21.15.170:0/1705148103 shutdown_connections 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.096+0000 7fcd93b79700 1 -- 172.21.15.170:0/1705148103 wait complete. 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.097+0000 7fcd93b79700 1 Processor -- start 2023-10-08T16:11:14.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.097+0000 7fcd93b79700 1 -- start start 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd8c0ca0f0 0x7fcd8c16dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd8c0b4340 0x7fcd8c0b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1514c0 con 0x7fcd78003a60 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b3eb0 con 0x7fcd8c0b4340 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd93b79700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b35f0 con 0x7fcd8c0ca0f0 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44750/0 (socket says 172.21.15.170:44750) 2023-10-08T16:11:14.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 -- 172.21.15.170:0/1621714484 learned_addr learned my addr 172.21.15.170:0/1621714484 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91114700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd8c0ca0f0 0x7fcd8c16dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd92116700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd8c0b4340 0x7fcd8c0b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 -- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd8c0ca0f0 msgr2=0x7fcd8c16dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd8c0ca0f0 0x7fcd8c16dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 -- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd8c0b4340 msgr2=0x7fcd8c0b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd8c0b4340 0x7fcd8c0b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 -- 172.21.15.170:0/1621714484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0c91d0 con 0x7fcd78003a60 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.098+0000 7fcd91915700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd88012960 tx=0x7fcd88015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.099+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88001950 con 0x7fcd78003a60 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.099+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8800c6d0 con 0x7fcd78003a60 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.099+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c16e2c0 con 0x7fcd78003a60 2023-10-08T16:11:14.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.099+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88026630 con 0x7fcd78003a60 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.099+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c0b3b40 con 0x7fcd78003a60 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.100+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c0718b0 con 0x7fcd78003a60 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.101+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd88019030 con 0x7fcd78003a60 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.101+0000 7fcd81ffb700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 0x7fcd700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.101+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd880869b0 con 0x7fcd78003a60 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.101+0000 7fcd91114700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 0x7fcd700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.102+0000 7fcd91114700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 0x7fcd700697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0b6670 tx=0x7fcd7c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.104+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd880536c0 con 0x7fcd78003a60 2023-10-08T16:11:14.294 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:11:14.297 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:11:14.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.365+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcd8c0ce8b0 con 0x7fcd78003a60 2023-10-08T16:11:14.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.368+0000 7fcd81ffb700 1 -- 172.21.15.170:0/1621714484 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcd88017020 con 0x7fcd78003a60 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 msgr2=0x7fcd700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 0x7fcd700697d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0b6670 tx=0x7fcd7c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 msgr2=0x7fcd8c0c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcd88012960 tx=0x7fcd88015000 comp rx=0 tx=0).stop 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 shutdown_connections 2023-10-08T16:11:14.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd8c0b4340 0x7fcd8c0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd70067320 0x7fcd700697d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78003a60 0x7fcd8c0c8c90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 --2- 172.21.15.170:0/1621714484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd8c0ca0f0 0x7fcd8c16dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.370+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 >> 172.21.15.170:0/1621714484 conn(0x7fcd8c068e00 msgr2=0x7fcd8c152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.371+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 shutdown_connections 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.371+0000 7fcd93b79700 1 -- 172.21.15.170:0/1621714484 wait complete. 2023-10-08T16:11:14.373 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:11:14.384 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.539+0000 7f8382c99700 1 Processor -- start 2023-10-08T16:11:14.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.539+0000 7f8382c99700 1 -- start start 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8382c99700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8382c99700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f837c069560 0x7f837c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8382c99700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c15a160 con 0x7f837c159c60 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8382c99700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c069f20 con 0x7f837c069560 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8382c99700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0d13a0 con 0x7f837c159890 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f837c069560 0x7f837c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f837c069560 0x7f837c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44756/0 (socket says 172.21.15.170:44756) 2023-10-08T16:11:14.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 -- 172.21.15.170:0/890480236 learned_addr learned my addr 172.21.15.170:0/890480236 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8380a35700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 -- 172.21.15.170:0/890480236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 msgr2=0x7f837c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 -- 172.21.15.170:0/890480236 >> v1:172.21.15.191:6789/0 conn(0x7f837c159c60 legacy=0x7f837c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.540+0000 7f8381236700 1 -- 172.21.15.170:0/890480236 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c15e140 con 0x7f837c069560 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8381236700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c069560 0x7f837c0699e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f83740099b0 tx=0x7f8374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7072b34f36835a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f83737fe700 1 -- 172.21.15.170:0/890480236 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374009ea0 con 0x7f837c069560 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f83737fe700 1 -- 172.21.15.170:0/890480236 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400b860 con 0x7f837c069560 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f83737fe700 1 -- 172.21.15.170:0/890480236 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400bc00 con 0x7f837c069560 2023-10-08T16:11:14.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 -- 172.21.15.170:0/890480236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c069560 msgr2=0x7f837c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c069560 0x7f837c0699e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f83740099b0 tx=0x7f8374009250 comp rx=0 tx=0).stop 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 -- 172.21.15.170:0/890480236 shutdown_connections 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c069560 0x7f837c0699e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 --2- 172.21.15.170:0/890480236 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 -- 172.21.15.170:0/890480236 >> 172.21.15.170:0/890480236 conn(0x7f837c068e20 msgr2=0x7f837c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 -- 172.21.15.170:0/890480236 shutdown_connections 2023-10-08T16:11:14.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.541+0000 7f8382c99700 1 -- 172.21.15.170:0/890480236 wait complete. 2023-10-08T16:11:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.542+0000 7f8382c99700 1 Processor -- start 2023-10-08T16:11:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 -- start start 2023-10-08T16:11:14.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f837c069560 0x7f837c140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c16dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0cfc00 con 0x7f837c159c60 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c16e2b0 con 0x7f837c069560 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8382c99700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c16ad30 con 0x7f837c159890 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8381236700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8381236700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44764/0 (socket says 172.21.15.170:44764) 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8381236700 1 -- 172.21.15.170:0/685972175 learned_addr learned my addr 172.21.15.170:0/685972175 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8380a35700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f837c069560 0x7f837c140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.543+0000 7f8373fff700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c16dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 -- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 msgr2=0x7f837c16dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c16dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 -- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f837c069560 msgr2=0x7f837c140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f837c069560 0x7f837c140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 -- 172.21.15.170:0/685972175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83740094f0 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8381236700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f837c0899c0 tx=0x7f8374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400baa0 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374025420 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c16a120 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400bc00 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.544+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c16a6e0 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.545+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c064ed0 con 0x7f837c159c60 2023-10-08T16:11:14.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.545+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8374023070 con 0x7f837c159c60 2023-10-08T16:11:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.546+0000 7f8370ff9700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 0x7f83680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.546+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f83740861f0 con 0x7f837c159c60 2023-10-08T16:11:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.546+0000 7f8380a35700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 0x7f83680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:14.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.546+0000 7f8380a35700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 0x7f83680697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f837800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:14.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.550+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8374052e80 con 0x7f837c159c60 2023-10-08T16:11:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.774+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f837c04c3c0 con 0x7f837c159c60 2023-10-08T16:11:14.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.774+0000 7f8370ff9700 1 -- 172.21.15.170:0/685972175 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f83740564a0 con 0x7f837c159c60 2023-10-08T16:11:14.776 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:14.776 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7113,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3558382637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3558382637},{"type":"v1","addr":"172.21.15.170:6836","nonce":3558382637}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7116,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/707523029","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":707523029},{"type":"v1","addr":"172.21.15.170:6837","nonce":707523029}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 msgr2=0x7f83680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 0x7f83680697d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8378009fd0 tx=0x7f837800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 msgr2=0x7f837c0b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f837c0899c0 tx=0x7f8374017000 comp rx=0 tx=0).stop 2023-10-08T16:11:14.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 shutdown_connections 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8368067320 0x7f83680697d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f837c159c60 0x7f837c0b15a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f837c069560 0x7f837c140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 --2- 172.21.15.170:0/685972175 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f837c159890 0x7f837c16dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.776+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 >> 172.21.15.170:0/685972175 conn(0x7f837c068e20 msgr2=0x7f837c155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.777+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 shutdown_connections 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:14.777+0000 7f8382c99700 1 -- 172.21.15.170:0/685972175 wait complete. 2023-10-08T16:11:14.779 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 74 2023-10-08T16:11:15.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:11:04.275853+0000', 'modified': '2023-10-08T16:11:07.928111+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7125}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7125': {'gid': 7125, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6834/47051258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6832', 'nonce': 47051258}, {'type': 'v1', 'addr': '172.21.15.191:6834', 'nonce': 47051258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:11:15.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:11:15.193 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T16:11:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.307+0000 7f547ab8d700 1 Processor -- start 2023-10-08T16:11:15.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.307+0000 7f547ab8d700 1 -- start start 2023-10-08T16:11:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f547ab8d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f5474069560 0x7f54740699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f547ab8d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5474069ee0 con 0x7f5474159890 2023-10-08T16:11:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f547ab8d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547415a160 con 0x7f5474159c60 2023-10-08T16:11:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f547ab8d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54740d13a0 con 0x7f5474069560 2023-10-08T16:11:15.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f5478929700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5474159890 0x7f547415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57534/0 (socket says 172.21.15.170:57534) 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.308+0000 7f5478929700 1 -- 172.21.15.170:0/2409275754 learned_addr learned my addr 172.21.15.170:0/2409275754 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354599483 0 0) 0x7f5474069ee0 con 0x7f5474159890 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f545c003570 con 0x7f5474159890 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f547912a700 1 --2- 172.21.15.170:0/2409275754 >> v2:172.21.15.191:3300/0 conn(0x7f5474069560 0x7f54740699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 868933652 0 0) 0x7f545c003570 con 0x7f5474159890 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 >> v2:172.21.15.191:3300/0 conn(0x7f5474069560 msgr2=0x7f54740699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 --2- 172.21.15.170:0/2409275754 >> v2:172.21.15.191:3300/0 conn(0x7f5474069560 0x7f54740699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 >> v1:172.21.15.191:6789/0 conn(0x7f5474159c60 legacy=0x7f547415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547415e140 con 0x7f5474159890 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f54700017f0 con 0x7f5474159890 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 >> v1:172.21.15.170:6789/0 conn(0x7f5474159890 legacy=0x7f547415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f545c003a60 0x7f545c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 0x7f545c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c0051c0 0x7f545c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54740d13a0 con 0x7f545c004810 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c00c0d0 con 0x7f545c0051c0 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c00c290 con 0x7f545c003a60 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 0x7f545c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f5478929700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f545c003a60 0x7f545c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f547912a700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c0051c0 0x7f545c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 -- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f545c003a60 msgr2=0x7f545c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f545c003a60 0x7f545c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 -- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c0051c0 msgr2=0x7f545c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c0051c0 0x7f545c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f546bfff700 1 -- 172.21.15.170:0/2409275754 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c003570 con 0x7f545c004810 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f547912a700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c0051c0 0x7f545c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.309+0000 7f547912a700 1 -- 172.21.15.170:0/2409275754 reap_dead start 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f546bfff700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 0x7f545c004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f54640018b0 tx=0x7f5464007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e762c9746fcd861 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464012420 con 0x7f545c004810 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54640125a0 con 0x7f545c004810 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f546b7fe700 1 -- 172.21.15.170:0/2409275754 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464012960 con 0x7f545c004810 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 -- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 msgr2=0x7f545c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 0x7f545c004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f54640018b0 tx=0x7f5464007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 -- 172.21.15.170:0/2409275754 shutdown_connections 2023-10-08T16:11:15.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 --2- 172.21.15.170:0/2409275754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f545c004810 0x7f545c004c80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 -- 172.21.15.170:0/2409275754 >> 172.21.15.170:0/2409275754 conn(0x7f5474068e20 msgr2=0x7f54740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 -- 172.21.15.170:0/2409275754 shutdown_connections 2023-10-08T16:11:15.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.310+0000 7f547ab8d700 1 -- 172.21.15.170:0/2409275754 wait complete. 2023-10-08T16:11:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.311+0000 7f547ab8d700 1 Processor -- start 2023-10-08T16:11:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 -- start start 2023-10-08T16:11:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c004810 0x7f54740c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547414d1f0 0x7f547414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54740c60d0 con 0x7f547414a7b0 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54740c63b0 con 0x7f545c004810 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547ab8d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54740c6690 con 0x7f547414d1f0 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547912a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547912a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40510/0 (socket says 172.21.15.170:40510) 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f547912a700 1 -- 172.21.15.170:0/195497565 learned_addr learned my addr 172.21.15.170:0/195497565 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f5478929700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c004810 0x7f54740c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.312+0000 7f546bfff700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547414d1f0 0x7f547414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 -- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547414d1f0 msgr2=0x7f547414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547414d1f0 0x7f547414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 -- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c004810 msgr2=0x7f54740c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c004810 0x7f54740c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 -- 172.21.15.170:0/195497565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54740c6970 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547912a700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f54740d03b0 tx=0x7f546c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f5468ff9700 1 -- 172.21.15.170:0/195497565 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c011ed0 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f5468ff9700 1 -- 172.21.15.170:0/195497565 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c015670 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54740c6c50 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f5468ff9700 1 -- 172.21.15.170:0/195497565 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c015a10 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.313+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54740c7170 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.314+0000 7f5468ff9700 1 -- 172.21.15.170:0/195497565 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f546c01f410 con 0x7f547414a7b0 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.315+0000 7f5468ff9700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 0x7f54600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.315+0000 7f5468ff9700 1 -- 172.21.15.170:0/195497565 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f546c084a20 con 0x7f547414a7b0 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.315+0000 7f5478929700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 0x7f54600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.315+0000 7f5478929700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 0x7f54600697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f54700013d0 tx=0x7f5470006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.316+0000 7f5483e85b80 1 Processor -- start 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 -- start start 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f1425a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1353a70 con 0x5568f13fca50 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1351270 con 0x5568f1425f60 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f134d060 con 0x5568f1446d00 2023-10-08T16:11:15.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40526/0 (socket says 172.21.15.170:40526) 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.317+0000 7f5478929700 1 -- 172.21.15.170:0/2855208821 learned_addr learned my addr 172.21.15.170:0/2855208821 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f546bfff700 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f1425a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f547912a700 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 -- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 msgr2=0x5568f1425a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f1425a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 -- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 msgr2=0x5568f13f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 -- 172.21.15.170:0/2855208821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f11e7270 con 0x5568f13fca50 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5478929700 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f54700155a0 tx=0x7f5470018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab48dc404bc72762 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5459ffb700 1 -- 172.21.15.170:0/2855208821 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470015ea0 con 0x5568f13fca50 2023-10-08T16:11:15.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5459ffb700 1 -- 172.21.15.170:0/2855208821 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470021620 con 0x5568f13fca50 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5459ffb700 1 -- 172.21.15.170:0/2855208821 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700219a0 con 0x5568f13fca50 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5483e85b80 1 -- 172.21.15.170:0/2855208821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 msgr2=0x5568f13fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.318+0000 7f5483e85b80 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f54700155a0 tx=0x7f5470018000 comp rx=0 tx=0).stop 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 -- 172.21.15.170:0/2855208821 shutdown_connections 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13fce20 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 --2- 172.21.15.170:0/2855208821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f1425a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 -- 172.21.15.170:0/2855208821 >> 172.21.15.170:0/2855208821 conn(0x5568f13fe370 msgr2=0x5568f13fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 -- 172.21.15.170:0/2855208821 shutdown_connections 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 -- 172.21.15.170:0/2855208821 wait complete. 2023-10-08T16:11:15.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 -- start start 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.319+0000 7f5483e85b80 1 Processor -- start 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f13fca50 0x5568f1422060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f143ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f12f8890 con 0x5568f1425f60 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f1313ea0 con 0x5568f13fca50 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f11cfe30 con 0x5568f1446d00 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5483e85b80 10 client.0 _ll_register_callbacks cb 0x5568f13c0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:11:15.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f546bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f546bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40534/0 (socket says 172.21.15.170:40534) 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f546bfff700 1 -- 172.21.15.170:0/3895851905 learned_addr learned my addr 172.21.15.170:0/3895851905 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f5478929700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f13fca50 0x5568f1422060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f547912a700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f143ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f546bfff700 1 -- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 msgr2=0x5568f143ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.320+0000 7f546bfff700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f143ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f546bfff700 1 -- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f13fca50 msgr2=0x5568f1422060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f546bfff700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f13fca50 0x5568f1422060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f546bfff700 1 -- 172.21.15.170:0/3895851905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547415e140 con 0x5568f1425f60 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f546bfff700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f546400c3f0 tx=0x7f5464015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbf608107b75c3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464001200 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f546401f410 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568f13f1b70 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5464012920 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 10 client.7290 fetch_fsmap learned FSMap version 74 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5568f1189fe0 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568f13f1b70 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f54640018e0 con 0x5568f1425f60 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 10 client.7290 fetch_fsmap finished waiting for FSMap version 74 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 10 client.7290 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 20 client.7290 populate_metadata read hostname 'smithi170' 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f1456be0 0x5568f1456fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 4 client.7290 mds_command: new command op to 7125 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.321+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 --> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568f13ceaa0 con 0x5568f1456be0 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.322+0000 7f5478929700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f1456be0 0x5568f1456fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.322+0000 7f5452ffd700 10 client.7290 ms_handle_connect on v2:172.21.15.191:6832/47051258 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.322+0000 7f5478929700 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f1456be0 0x5568f1456fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40048855d314717a server_cookie=ebdef33244e8a688 in_seq=0 out_seq=0 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5452ffd700 1 -- 172.21.15.170:0/3895851905 <== mds.0 v2:172.21.15.191:6832/47051258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5568f13ceaa0 con 0x5568f1456be0 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5452ffd700 10 client.7290 handle_command_reply: tid=0 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 client.7290 shutdown 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 msgr2=0x5568f1453a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f546400c3f0 tx=0x7f5464015000 comp rx=0 tx=0).stop 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 shutdown_connections 2023-10-08T16:11:15.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f1456be0 0x5568f1456fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f143ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f13fca50 0x5568f1422060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5452ffd700 0 client.7290 ms_handle_reset on v2:172.21.15.191:6832/47051258 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3895851905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f1425f60 0x5568f1453a80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 >> 172.21.15.170:0/3895851905 conn(0x5568f13fe370 msgr2=0x5568f13f7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 shutdown_connections 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 1 -- 172.21.15.170:0/3895851905 wait complete. 2023-10-08T16:11:15.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.323+0000 7f5483e85b80 20 client.7290 trim_cache size 0 max 16384 2023-10-08T16:11:15.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 Processor -- start 2023-10-08T16:11:15.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 -- start start 2023-10-08T16:11:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1425f60 0x5568f1450120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1446d00 0x5568f14249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f13f7830 con 0x5568f13fca50 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f142c930 con 0x5568f1446d00 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f13faa70 con 0x5568f1425f60 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40544/0 (socket says 172.21.15.170:40544) 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.340+0000 7f5478929700 1 -- 172.21.15.170:0/3939333523 learned_addr learned my addr 172.21.15.170:0/3939333523 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f546bfff700 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1425f60 0x5568f1450120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f547912a700 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1446d00 0x5568f14249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.344 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 -- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1425f60 msgr2=0x5568f1450120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1425f60 0x5568f1450120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 -- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1446d00 msgr2=0x5568f14249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1446d00 0x5568f14249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 -- 172.21.15.170:0/3939333523 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568f131b780 con 0x5568f13fca50 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5478929700 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5470000f80 tx=0x7f547002d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34057305d3700a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f545a7fc700 1 -- 172.21.15.170:0/3939333523 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470031040 con 0x5568f13fca50 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f545a7fc700 1 -- 172.21.15.170:0/3939333523 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470021d10 con 0x5568f13fca50 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f545a7fc700 1 -- 172.21.15.170:0/3939333523 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470035250 con 0x5568f13fca50 2023-10-08T16:11:15.345 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5483e85b80 1 -- 172.21.15.170:0/3939333523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 msgr2=0x5568f13f8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.341+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f5470000f80 tx=0x7f547002d040 comp rx=0 tx=0).stop 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 -- 172.21.15.170:0/3939333523 shutdown_connections 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1446d00 0x5568f14249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13f8350 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3939333523 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1425f60 0x5568f1450120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 -- 172.21.15.170:0/3939333523 >> 172.21.15.170:0/3939333523 conn(0x5568f13fe370 msgr2=0x5568f13f54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 -- 172.21.15.170:0/3939333523 shutdown_connections 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 -- 172.21.15.170:0/3939333523 wait complete. 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 -- start start 2023-10-08T16:11:15.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.342+0000 7f5483e85b80 1 Processor -- start 2023-10-08T16:11:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f144f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f14a9d10 con 0x5568f13fca50 2023-10-08T16:11:15.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568f143a100 con 0x5568f1425f60 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568f1445a70 con 0x5568f1446d00 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5483e85b80 10 client.0 _ll_register_callbacks cb 0x5568f13c0110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5478929700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40550/0 (socket says 172.21.15.170:40550) 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5478929700 1 -- 172.21.15.170:0/3320075234 learned_addr learned my addr 172.21.15.170:0/3320075234 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f547912a700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f144f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f546bfff700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.343+0000 7f5478929700 1 -- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 msgr2=0x5568f144f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5478929700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f144f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5478929700 1 -- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 msgr2=0x5568f13ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5478929700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5478929700 1 -- 172.21.15.170:0/3320075234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568f11e7270 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f547912a700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f144f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5478929700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3cdcd04e9afed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470031040 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5470015e20 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5470021620 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f547004acc0 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 10 client.7296 fetch_fsmap learned FSMap version 74 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5568f1302470 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470021620 con 0x5568f13fca50 2023-10-08T16:11:15.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f547004a020 con 0x5568f13fca50 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 10 client.7296 fetch_fsmap finished waiting for FSMap version 74 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 10 client.7296 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 20 client.7296 populate_metadata read hostname 'smithi170' 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f144f650 0x5568f144fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 4 client.7296 mds_command: new command op to 7125 tid=0[{"prefix": "status"}] 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 --> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568f12efe00 con 0x5568f144f650 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.344+0000 7f546bfff700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f144f650 0x5568f144fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f546bfff700 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f144f650 0x5568f144fa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16741c3de83b5f1 server_cookie=b05dbd61730464c8 in_seq=0 out_seq=0 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5452ffd700 10 client.7296 ms_handle_connect on v2:172.21.15.191:6832/47051258 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5452ffd700 1 -- 172.21.15.170:0/3320075234 <== mds.0 v2:172.21.15.191:6832/47051258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5568f12efe00 con 0x5568f144f650 2023-10-08T16:11:15.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5452ffd700 10 client.7296 handle_command_reply: tid=0 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 client.7296 shutdown 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 msgr2=0x5568f13ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470036000 comp rx=0 tx=0).stop 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 shutdown_connections 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:6832/47051258,v1:172.21.15.191:6834/47051258] conn(0x5568f144f650 0x5568f144fa20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5568f1446d00 0x5568f144f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5452ffd700 0 client.7296 ms_handle_reset on v2:172.21.15.191:6832/47051258 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f13fca50 0x5568f13ff950 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 --2- 172.21.15.170:0/3320075234 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5568f1425f60 0x5568f13ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 >> 172.21.15.170:0/3320075234 conn(0x5568f13fe370 msgr2=0x5568f143cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 shutdown_connections 2023-10-08T16:11:15.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.345+0000 7f5483e85b80 1 -- 172.21.15.170:0/3320075234 wait complete. 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f5483e85b80 20 client.7296 trim_cache size 0 max 16384 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7125, 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:11:15.355 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 10.436505681, 2023-10-08T16:11:15.356 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 74, 2023-10-08T16:11:15.356 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 113, 2023-10-08T16:11:15.356 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 113, 2023-10-08T16:11:15.356 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 15.540427001999999 2023-10-08T16:11:15.356 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 msgr2=0x7f54600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 0x7f54600697d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f54700013d0 tx=0x7f5470006040 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 msgr2=0x7f547415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f54740d03b0 tx=0x7f546c008040 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 shutdown_connections 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5460067320 0x7f54600697d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f547414a7b0 0x7f547415f780 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f545c004810 0x7f54740c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 --2- 172.21.15.170:0/195497565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f547414d1f0 0x7f547414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 >> 172.21.15.170:0/195497565 conn(0x7f5474068e20 msgr2=0x7f5474158b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 shutdown_connections 2023-10-08T16:11:15.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.346+0000 7f547ab8d700 1 -- 172.21.15.170:0/195497565 wait complete. 2023-10-08T16:11:15.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:11:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.472+0000 7f6e5be78700 1 Processor -- start 2023-10-08T16:11:15.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5be78700 1 -- start start 2023-10-08T16:11:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5be78700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6e54159890 0x7f6e5415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5be78700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5415a160 con 0x7f6e54069560 2023-10-08T16:11:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5be78700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540d13a0 con 0x7f6e54159c60 2023-10-08T16:11:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5be78700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5415e140 con 0x7f6e54159890 2023-10-08T16:11:15.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5a415700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6e54069560 0x7f6e540ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57536/0 (socket says 172.21.15.170:57536) 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.473+0000 7f6e5a415700 1 -- 172.21.15.170:0/4124788661 learned_addr learned my addr 172.21.15.170:0/4124788661 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 --2- 172.21.15.170:0/4124788661 >> v2:172.21.15.191:3300/0 conn(0x7f6e54159890 0x7f6e5415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617488417 0 0) 0x7f6e5415a160 con 0x7f6e54069560 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e40003570 con 0x7f6e54069560 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3710074908 0 0) 0x7f6e40003570 con 0x7f6e54069560 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 >> v2:172.21.15.191:3300/0 conn(0x7f6e54159890 msgr2=0x7f6e5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 --2- 172.21.15.170:0/4124788661 >> v2:172.21.15.191:3300/0 conn(0x7f6e54159890 0x7f6e5415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 >> v1:172.21.15.191:6789/0 conn(0x7f6e54159c60 legacy=0x7f6e5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5415a3d0 con 0x7f6e54069560 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6e4c001470 con 0x7f6e54069560 2023-10-08T16:11:15.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 >> v1:172.21.15.170:6789/0 conn(0x7f6e54069560 legacy=0x7f6e540ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 0x7f6e40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e40004810 0x7f6e40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e400051c0 0x7f6e4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5415e140 con 0x7f6e40003a60 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4000c0d0 con 0x7f6e40004810 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4000c290 con 0x7f6e400051c0 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 0x7f6e40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59413700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e40004810 0x7f6e40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e5a415700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e400051c0 0x7f6e4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 -- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e400051c0 msgr2=0x7f6e4000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e400051c0 0x7f6e4000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 -- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e40004810 msgr2=0x7f6e40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e40004810 0x7f6e40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e59c14700 1 -- 172.21.15.170:0/4124788661 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40003570 con 0x7f6e40003a60 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.474+0000 7f6e5a415700 1 -- 172.21.15.170:0/4124788661 reap_dead start 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e59c14700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 0x7f6e40003e70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e210 tx=0x7f6e5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642365548f1736e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000eac0 con 0x7f6e40003a60 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000f040 con 0x7f6e40003a60 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e58c12700 1 -- 172.21.15.170:0/4124788661 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000ee70 con 0x7f6e40003a60 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 -- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 msgr2=0x7f6e40003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 0x7f6e40003e70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e210 tx=0x7f6e5000b040 comp rx=0 tx=0).stop 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 -- 172.21.15.170:0/4124788661 shutdown_connections 2023-10-08T16:11:15.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 --2- 172.21.15.170:0/4124788661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e40003a60 0x7f6e40003e70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 -- 172.21.15.170:0/4124788661 >> 172.21.15.170:0/4124788661 conn(0x7f6e54068e20 msgr2=0x7f6e540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 -- 172.21.15.170:0/4124788661 shutdown_connections 2023-10-08T16:11:15.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.475+0000 7f6e5be78700 1 -- 172.21.15.170:0/4124788661 wait complete. 2023-10-08T16:11:15.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.476+0000 7f6e5be78700 1 Processor -- start 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.476+0000 7f6e5be78700 1 -- start start 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e40003a60 0x7f6e5414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 0x7f6e540b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540b40d0 con 0x7f6e540b6070 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540cf8a0 con 0x7f6e540ce180 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5be78700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540b37a0 con 0x7f6e40003a60 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5a415700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5a415700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40572/0 (socket says 172.21.15.170:40572) 2023-10-08T16:11:15.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e5a415700 1 -- 172.21.15.170:0/1819444474 learned_addr learned my addr 172.21.15.170:0/1819444474 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e59c14700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e40003a60 0x7f6e5414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.477+0000 7f6e59413700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 0x7f6e540b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 -- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e40003a60 msgr2=0x7f6e5414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e40003a60 0x7f6e5414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 -- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 msgr2=0x7f6e540b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 0x7f6e540b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 -- 172.21.15.170:0/1819444474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540b3a80 con 0x7f6e540b6070 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e59413700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 0x7f6e540b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5a415700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6e540cadc0 tx=0x7f6e4c000fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c012590 con 0x7f6e540b6070 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c012710 con 0x7f6e540b6070 2023-10-08T16:11:15.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540b3d60 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c012ad0 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.478+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e540b4aa0 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.479+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540b4240 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.479+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6e4c01d460 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.480+0000 7f6e49ffb700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 0x7f6e380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.480+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6e4c084a60 con 0x7f6e540b6070 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.480+0000 7f6e59c14700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 0x7f6e380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:15.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.480+0000 7f6e59c14700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 0x7f6e380697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000afd0 tx=0x7f6e5001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:15.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.484+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e4c0516f0 con 0x7f6e540b6070 2023-10-08T16:11:15.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.710+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e5404c3c0 con 0x7f6e540b6070 2023-10-08T16:11:15.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.711+0000 7f6e49ffb700 1 -- 172.21.15.170:0/1819444474 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e4c054d10 con 0x7f6e540b6070 2023-10-08T16:11:15.713 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:11:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 msgr2=0x7f6e380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 0x7f6e380697d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000afd0 tx=0x7f6e5001a000 comp rx=0 tx=0).stop 2023-10-08T16:11:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 msgr2=0x7f6e54152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:15.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6e540cadc0 tx=0x7f6e4c000fd0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 shutdown_connections 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6e38067320 0x7f6e380697d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e540b6070 0x7f6e54152180 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6e40003a60 0x7f6e5414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 --2- 172.21.15.170:0/1819444474 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6e540ce180 0x7f6e540b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 >> 172.21.15.170:0/1819444474 conn(0x7f6e54068e20 msgr2=0x7f6e5415cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 shutdown_connections 2023-10-08T16:11:15.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:15.713+0000 7f6e5be78700 1 -- 172.21.15.170:0/1819444474 wait complete. 2023-10-08T16:11:15.726 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:11:15.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:11:15.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:11:15.726 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:11:15.727 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:15.727 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:11:15.727 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:11:15.727 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:11:15.727 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:15.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.727 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:11:15.742 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:11:15.742 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:11:15.743 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42075sec preferred_lft 42075sec 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:11:15.744 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:11:15.745 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:15.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.746 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:11:15.806 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:11:15.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.807 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:11:15.862 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:11:15.863 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:11:15.863 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:11:15.863 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:11:15.863 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:11:15.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.863 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:11:15.863 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:11:15.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:11:15.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:11:15.864 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:11:15.864 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:11:15.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.963 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:11:15.977 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:11:15.977 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:11:15.977 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:15.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:15.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:11:16.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:16.041 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:16.096 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:16.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:16.096 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:16.150 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:11:16.150 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:11:16.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:16.230 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:16.230 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:16.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:16.271 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:16.272 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:16.273 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:11:16.273 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:11:16.273 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:11:16.274 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:11:16.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:16.274 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:16.339 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:11:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.493+0000 7fa674268700 1 Processor -- start 2023-10-08T16:11:16.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.493+0000 7fa674268700 1 -- start start 2023-10-08T16:11:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa66c0ac230 0x7fa66c0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa66c0ccc40 0x7fa66c0af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c0ad930 con 0x7fa66c0ac230 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c0acd20 con 0x7fa66c0ccc40 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa674268700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0ad000 con 0x7fa66c155c10 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.494+0000 7fa671803700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa66c0ac230 0x7fa66c0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa672805700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa66c0ccc40 0x7fa66c0af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa66c0ac230 0x7fa66c0ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40576/0 (socket says 172.21.15.170:40576) 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 -- 172.21.15.170:0/617483292 learned_addr learned my addr 172.21.15.170:0/617483292 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa672004700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 -- 172.21.15.170:0/617483292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 msgr2=0x7fa66c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 -- 172.21.15.170:0/617483292 >> v2:172.21.15.191:3300/0 conn(0x7fa66c0ccc40 msgr2=0x7fa66c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 --2- 172.21.15.170:0/617483292 >> v2:172.21.15.191:3300/0 conn(0x7fa66c0ccc40 0x7fa66c0af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 -- 172.21.15.170:0/617483292 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0ad2e0 con 0x7fa66c0ac230 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671803700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ac230 0x7fa66c0ac6a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0c8c60 tx=0x7fa65c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803d0192c972baee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671002700 1 -- 172.21.15.170:0/617483292 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00b640 con 0x7fa66c0ac230 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671002700 1 -- 172.21.15.170:0/617483292 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00b7a0 con 0x7fa66c0ac230 2023-10-08T16:11:16.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.495+0000 7fa671002700 1 -- 172.21.15.170:0/617483292 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c00bb60 con 0x7fa66c0ac230 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 -- 172.21.15.170:0/617483292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ac230 msgr2=0x7fa66c0ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ac230 0x7fa66c0ac6a0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0c8c60 tx=0x7fa65c009250 comp rx=0 tx=0).stop 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 -- 172.21.15.170:0/617483292 shutdown_connections 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 --2- 172.21.15.170:0/617483292 >> v2:172.21.15.191:3300/0 conn(0x7fa66c0ccc40 0x7fa66c0af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 --2- 172.21.15.170:0/617483292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ac230 0x7fa66c0ac6a0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 -- 172.21.15.170:0/617483292 >> 172.21.15.170:0/617483292 conn(0x7fa66c06a320 msgr2=0x7fa66c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 -- 172.21.15.170:0/617483292 shutdown_connections 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.496+0000 7fa674268700 1 -- 172.21.15.170:0/617483292 wait complete. 2023-10-08T16:11:16.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.497+0000 7fa674268700 1 Processor -- start 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.497+0000 7fa674268700 1 -- start start 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66c0ac230 0x7fa66c0aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c147940 con 0x7fa66c0ccc40 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c14ae70 con 0x7fa66c0ac230 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c0a6ea0 con 0x7fa66c155c10 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40586/0 (socket says 172.21.15.170:40586) 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 -- 172.21.15.170:0/2626877755 learned_addr learned my addr 172.21.15.170:0/2626877755 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:16.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa672004700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66c0ac230 0x7fa66c0aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 -- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 msgr2=0x7fa66c14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 -- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66c0ac230 msgr2=0x7fa66c0aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66c0ac230 0x7fa66c0aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 -- 172.21.15.170:0/2626877755 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0094f0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa671803700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa66c064290 tx=0x7fa65c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c000d10 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c00bb20 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0a8390 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c011670 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.498+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c0a8950 con 0x7fa66c0ccc40 2023-10-08T16:11:16.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.499+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c04c3c0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.500+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa65c0119e0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.501+0000 7fa6627fc700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 0x7fa6500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:16.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.501+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa65c087110 con 0x7fa66c0ccc40 2023-10-08T16:11:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.501+0000 7fa672004700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 0x7fa6500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:16.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.501+0000 7fa672004700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 0x7fa6500697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa668007620 tx=0x7fa66800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:16.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.506+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa65c053da0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.733+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa66c0678d0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.734+0000 7fa6627fc700 1 -- 172.21.15.170:0/2626877755 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa65c0573c0 con 0x7fa66c0ccc40 2023-10-08T16:11:16.735 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 msgr2=0x7fa6500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 0x7fa6500697d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa668007620 tx=0x7fa66800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 msgr2=0x7fa66c14a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa66c064290 tx=0x7fa65c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 shutdown_connections 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa66c155c10 0x7fa66c14a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa66c0ac230 0x7fa66c0aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa650067320 0x7fa6500697d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 --2- 172.21.15.170:0/2626877755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa66c0ccc40 0x7fa66c14a3f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 >> 172.21.15.170:0/2626877755 conn(0x7fa66c06a320 msgr2=0x7fa66c06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 shutdown_connections 2023-10-08T16:11:16.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:16.736+0000 7fa674268700 1 -- 172.21.15.170:0/2626877755 wait complete. 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:11:16.750 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:16.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:16.751 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:11:16.769 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:11:16.769 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:11:16.769 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:16.770 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42075sec preferred_lft 42075sec 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:11:16.771 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:16.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:16.773 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:11:16.837 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:11:16.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:16.839 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:11:16.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:11:16.898 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:11:17.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:17.003 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:11:17.017 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:11:17.018 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:11:17.018 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:17.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:17.018 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:11:17.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:17.083 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:17.138 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:17.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:17.138 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:17.192 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:11:17.192 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:11:17.192 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:17.271 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:17.272 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:17.290 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:17.291 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:17.312 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:11:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:11:17.313 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:11:17.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:11:17.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:17.314 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:17.378 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:11:17.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.498+0000 7f069f90e700 1 Processor -- start 2023-10-08T16:11:17.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.498+0000 7f069f90e700 1 -- start start 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069f90e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 0x7f06980ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069f90e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980b5b00 con 0x7f0698159ec0 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069f90e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06980b5da0 con 0x7f06980b55a0 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069f90e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06980b6080 con 0x7f06980cd910 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069d6aa700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0698159ec0 0x7f06980b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57542/0 (socket says 172.21.15.170:57542) 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069d6aa700 1 -- 172.21.15.170:0/2427515042 learned_addr learned my addr 172.21.15.170:0/2427515042 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069cea9700 1 --2- 172.21.15.170:0/2427515042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 0x7f06980ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742141721 0 0) 0x7f06980b5b00 con 0x7f0698159ec0 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0680003570 con 0x7f0698159ec0 2023-10-08T16:11:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210397057 0 0) 0x7f06980b5da0 con 0x7f06980b55a0 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06980b5b00 con 0x7f06980b55a0 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698699188 0 0) 0x7f0680003570 con 0x7f0698159ec0 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069cea9700 1 -- 172.21.15.170:0/2427515042 >> v1:172.21.15.191:6789/0 conn(0x7f06980b55a0 legacy=0x7f06980ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069cea9700 1 -- 172.21.15.170:0/2427515042 >> v1:172.21.15.170:6789/0 conn(0x7f0698159ec0 legacy=0x7f06980b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.499+0000 7f069cea9700 1 -- 172.21.15.170:0/2427515042 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980cc100 con 0x7f06980cd910 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069cea9700 1 --2- 172.21.15.170:0/2427515042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 0x7f06980ce340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f068c009e90 tx=0x7f068c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d93a768a359292d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00e070 con 0x7f06980cd910 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00b870 con 0x7f06980cd910 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f0697fff700 1 -- 172.21.15.170:0/2427515042 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06900015f0 con 0x7f06980cd910 2023-10-08T16:11:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 -- 172.21.15.170:0/2427515042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 msgr2=0x7f06980ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 --2- 172.21.15.170:0/2427515042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 0x7f06980ce340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f068c009e90 tx=0x7f068c009780 comp rx=0 tx=0).stop 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 -- 172.21.15.170:0/2427515042 shutdown_connections 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 --2- 172.21.15.170:0/2427515042 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd910 0x7f06980ce340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 -- 172.21.15.170:0/2427515042 >> 172.21.15.170:0/2427515042 conn(0x7f06980999f0 msgr2=0x7f06980b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 -- 172.21.15.170:0/2427515042 shutdown_connections 2023-10-08T16:11:17.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.500+0000 7f069f90e700 1 -- 172.21.15.170:0/2427515042 wait complete. 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.501+0000 7f069f90e700 1 Processor -- start 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.501+0000 7f069f90e700 1 -- start start 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980b55a0 0x7f0698155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c8a80 0x7f06980c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06980c9460 con 0x7f0698159ec0 2023-10-08T16:11:17.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698068390 con 0x7f06980c8a80 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069f90e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698068670 con 0x7f06980b55a0 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069cea9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069cea9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40602/0 (socket says 172.21.15.170:40602) 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069cea9700 1 -- 172.21.15.170:0/379854759 learned_addr learned my addr 172.21.15.170:0/379854759 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.502+0000 7f069d6aa700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980b55a0 0x7f0698155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:17.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069deab700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c8a80 0x7f06980c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 -- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980b55a0 msgr2=0x7f0698155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980b55a0 0x7f0698155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 -- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c8a80 msgr2=0x7f06980c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c8a80 0x7f06980c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 -- 172.21.15.170:0/379854759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c009a20 con 0x7f0698159ec0 2023-10-08T16:11:17.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069cea9700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0698089390 tx=0x7f068c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c001090 con 0x7f0698159ec0 2023-10-08T16:11:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c012b30 con 0x7f0698159ec0 2023-10-08T16:11:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069809d6f0 con 0x7f0698159ec0 2023-10-08T16:11:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c01e450 con 0x7f0698159ec0 2023-10-08T16:11:17.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.503+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069809dcb0 con 0x7f0698159ec0 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.504+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698063c90 con 0x7f0698159ec0 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.504+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f068c01e800 con 0x7f0698159ec0 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.505+0000 7f06957fa700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 0x7f06840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.505+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f068c086f50 con 0x7f0698159ec0 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.505+0000 7f069d6aa700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 0x7f06840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:17.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.505+0000 7f069d6aa700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 0x7f06840697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0690000c00 tx=0x7f06900016e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:17.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.509+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f068c053ad0 con 0x7f0698159ec0 2023-10-08T16:11:17.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:17.732+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06980718b0 con 0x7f0698159ec0 2023-10-08T16:11:18.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.515+0000 7f06957fa700 1 -- 172.21.15.170:0/379854759 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 136+0+0 (secure 0 0 0) 0x7f068c0570f0 con 0x7f0698159ec0 2023-10-08T16:11:18.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 msgr2=0x7f06840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 0x7f06840697d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0690000c00 tx=0x7f06900016e0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 msgr2=0x7f06981450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0698089390 tx=0x7f068c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 shutdown_connections 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f06980c8a80 0x7f06980c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980b55a0 0x7f0698155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0684067320 0x7f06840697d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 --2- 172.21.15.170:0/379854759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0698159ec0 0x7f06981450a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 >> 172.21.15.170:0/379854759 conn(0x7f06980999f0 msgr2=0x7f0698158be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:18.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 shutdown_connections 2023-10-08T16:11:18.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.518+0000 7f069f90e700 1 -- 172.21.15.170:0/379854759 wait complete. 2023-10-08T16:11:18.533 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:11:18.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.649+0000 7f5ca9e6b700 1 Processor -- start 2023-10-08T16:11:18.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.649+0000 7f5ca9e6b700 1 -- start start 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca9e6b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5ca4155c10 0x7f5ca4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca9e6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca9e6b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca40acac0 con 0x7f5ca40ac230 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca9e6b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca40ad930 con 0x7f5ca4155c10 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca9e6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40acd20 con 0x7f5ca40ac6d0 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5ca4155c10 0x7f5ca4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5ca4155c10 0x7f5ca4155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40612/0 (socket says 172.21.15.170:40612) 2023-10-08T16:11:18.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5ca40ac230 0x7f5ca41568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57552/0 (socket says 172.21.15.170:57552) 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 -- 172.21.15.170:0/3995639728 learned_addr learned my addr 172.21.15.170:0/3995639728 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca3fff700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 -- 172.21.15.170:0/3995639728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 msgr2=0x7f5ca40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 -- 172.21.15.170:0/3995639728 >> v1:172.21.15.170:6789/0 conn(0x7f5ca40ac230 legacy=0x7f5ca41568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca27fc700 1 -- 172.21.15.170:0/3995639728 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156177106 0 0) 0x7f5ca40acac0 con 0x7f5ca40ac230 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 -- 172.21.15.170:0/3995639728 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40ad000 con 0x7f5ca4155c10 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.650+0000 7f5ca37fe700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4155fe0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40acc50 tx=0x7f5c9400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35722cc3dfe9f6c7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca27fc700 1 -- 172.21.15.170:0/3995639728 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400ca40 con 0x7f5ca4155c10 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca27fc700 1 -- 172.21.15.170:0/3995639728 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400cba0 con 0x7f5ca4155c10 2023-10-08T16:11:18.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca27fc700 1 -- 172.21.15.170:0/3995639728 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940183f0 con 0x7f5ca4155c10 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/3995639728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 msgr2=0x7f5ca4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4155fe0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40acc50 tx=0x7f5c9400a3f0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/3995639728 shutdown_connections 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/3995639728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4155fe0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/3995639728 >> 172.21.15.170:0/3995639728 conn(0x7f5ca406a320 msgr2=0x7f5ca40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/3995639728 shutdown_connections 2023-10-08T16:11:18.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.651+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/3995639728 wait complete. 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.652+0000 7f5ca9e6b700 1 Processor -- start 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.652+0000 7f5ca9e6b700 1 -- start start 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ca40ac230 0x7f5ca414b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40c9380 con 0x7f5ca4155c10 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40c9940 con 0x7f5ca40ac230 2023-10-08T16:11:18.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca9e6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4169780 con 0x7f5ca40ac6d0 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40620/0 (socket says 172.21.15.170:40620) 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca3fff700 1 -- 172.21.15.170:0/1547115169 learned_addr learned my addr 172.21.15.170:0/1547115169 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca37fe700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ca40ac230 0x7f5ca414b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.653+0000 7f5ca2ffd700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 -- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 msgr2=0x7f5ca40c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 -- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ca40ac230 msgr2=0x7f5ca414b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ca40ac230 0x7f5ca414b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 -- 172.21.15.170:0/1547115169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9400a690 con 0x7f5ca4155c10 2023-10-08T16:11:18.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca3fff700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c00ebf0 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00ed50 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca4169a60 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c014670 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.654+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca414b9b0 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.655+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca40adc20 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.655+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5c9c014a20 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.656+0000 7f5c8b7fe700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 0x7f5c90069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.656+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5c9c0852a0 con 0x7f5ca4155c10 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.656+0000 7f5ca37fe700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 0x7f5c90069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:18.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.656+0000 7f5ca37fe700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 0x7f5c90069850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40acc50 tx=0x7f5c94017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:18.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.660+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c9c051ef0 con 0x7f5ca4155c10 2023-10-08T16:11:18.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.874+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ca404c3c0 con 0x7f5c900673a0 2023-10-08T16:11:18.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.875+0000 7f5c8b7fe700 1 -- 172.21.15.170:0/1547115169 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f5ca404c3c0 con 0x7f5c900673a0 2023-10-08T16:11:18.877 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:18.877 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:18.877 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:11:18.877 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:18.877 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:18.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 msgr2=0x7f5c90069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 0x7f5c90069850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40acc50 tx=0x7f5c94017000 comp rx=0 tx=0).stop 2023-10-08T16:11:18.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 msgr2=0x7f5ca4168cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c009900 comp rx=0 tx=0).stop 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 shutdown_connections 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5c900673a0 0x7f5c90069850 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ca4155c10 0x7f5ca4168cc0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5ca40ac230 0x7f5ca414b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 --2- 172.21.15.170:0/1547115169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5ca40ac6d0 0x7f5ca40c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 >> 172.21.15.170:0/1547115169 conn(0x7f5ca406a320 msgr2=0x7f5ca40ae030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 shutdown_connections 2023-10-08T16:11:18.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:18.877+0000 7f5ca9e6b700 1 -- 172.21.15.170:0/1547115169 wait complete. 2023-10-08T16:11:18.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:11:19.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.059+0000 7fd8b9e9b700 1 Processor -- start 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.059+0000 7fd8b9e9b700 1 -- start start 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b9e9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 0x7fd8b415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b9e9b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 0x7fd8b4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b9e9b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b415a160 con 0x7fd8b4159890 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b9e9b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40d13a0 con 0x7fd8b415db10 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b9e9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b415e140 con 0x7fd8b4159c60 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 0x7fd8b415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b37fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fd8b4159890 0x7fd8b415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:46070/0 (socket says 172.21.15.170:46070) 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b37fe700 1 -- 172.21.15.170:0/107031254 learned_addr learned my addr 172.21.15.170:0/107031254 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b3fff700 1 --2- 172.21.15.170:0/107031254 >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 0x7fd8b4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b27fc700 1 -- 172.21.15.170:0/107031254 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544425496 0 0) 0x7fd8b415a160 con 0x7fd8b4159890 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.060+0000 7fd8b27fc700 1 -- 172.21.15.170:0/107031254 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c003570 con 0x7fd8b4159890 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b2ffd700 1 -- 172.21.15.170:0/107031254 >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 msgr2=0x7fd8b4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b2ffd700 1 --2- 172.21.15.170:0/107031254 >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 0x7fd8b4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b2ffd700 1 -- 172.21.15.170:0/107031254 >> v1:172.21.15.191:6789/0 conn(0x7fd8b4159890 legacy=0x7fd8b415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b2ffd700 1 -- 172.21.15.170:0/107031254 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b415a3d0 con 0x7fd8b4159c60 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b3fff700 1 --2- 172.21.15.170:0/107031254 >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 0x7fd8b4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b2ffd700 1 --2- 172.21.15.170:0/107031254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 0x7fd8b415d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098e0 tx=0x7fd8a80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7145db14fd4ecf81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b27fc700 1 -- 172.21.15.170:0/107031254 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011630 con 0x7fd8b4159c60 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b27fc700 1 -- 172.21.15.170:0/107031254 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8011790 con 0x7fd8b4159c60 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b27fc700 1 -- 172.21.15.170:0/107031254 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8011b10 con 0x7fd8b4159c60 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/107031254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 msgr2=0x7fd8b415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/107031254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 0x7fd8b415d5d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098e0 tx=0x7fd8a80091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/107031254 shutdown_connections 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/107031254 >> v2:172.21.15.191:3300/0 conn(0x7fd8b415db10 0x7fd8b4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/107031254 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159c60 0x7fd8b415d5d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/107031254 >> 172.21.15.170:0/107031254 conn(0x7fd8b4068e20 msgr2=0x7fd8b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/107031254 shutdown_connections 2023-10-08T16:11:19.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.061+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/107031254 wait complete. 2023-10-08T16:11:19.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 Processor -- start 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 -- start start 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 0x7fd8b40c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40b4a40 con 0x7fd8b415db10 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40b4d20 con 0x7fd8b4159c60 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.063+0000 7fd8b9e9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40b3ad0 con 0x7fd8b4159890 2023-10-08T16:11:19.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 0x7fd8b40c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40626/0 (socket says 172.21.15.170:40626) 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b2ffd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:41514/0 (socket says 172.21.15.170:41514) 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b3fff700 1 -- 172.21.15.170:0/3452198547 learned_addr learned my addr 172.21.15.170:0/3452198547 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 -- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 msgr2=0x7fd8b40b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 -- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 msgr2=0x7fd8b4147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 -- 172.21.15.170:0/3452198547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac009420 con 0x7fd8b4159890 2023-10-08T16:11:19.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b2ffd700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b3fff700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b37fe700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 0x7fd8b40c7aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400cc40 tx=0x7fd8a4001a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400a850 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a400a9d0 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a8009420 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ad90 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.064+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b40b4050 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.065+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b40d1710 con 0x7fd8b4159890 2023-10-08T16:11:19.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.066+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd8a4007300 con 0x7fd8b4159890 2023-10-08T16:11:19.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.066+0000 7fd89b7fe700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 0x7fd8a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.066+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd8a4084f00 con 0x7fd8b4159890 2023-10-08T16:11:19.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.066+0000 7fd8b2ffd700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 0x7fd8a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.066+0000 7fd8b2ffd700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 0x7fd8a00697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415a090 tx=0x7fd8a8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.070+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8a4051c10 con 0x7fd8b4159890 2023-10-08T16:11:19.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.252+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd8b404c3c0 con 0x7fd8b4159890 2023-10-08T16:11:19.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.254+0000 7fd89b7fe700 1 -- 172.21.15.170:0/3452198547 <== mon.2 v2:172.21.15.191:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8a4055230 con 0x7fd8b4159890 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 msgr2=0x7fd8a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 0x7fd8a00697d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd8b415a090 tx=0x7fd8a8017000 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 msgr2=0x7fd8b40c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 0x7fd8b40c7aa0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400cc40 tx=0x7fd8a4001a20 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 shutdown_connections 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd8a0067320 0x7fd8a00697d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b415db10 0x7fd8b4147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd8b4159890 0x7fd8b40c7aa0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 --2- 172.21.15.170:0/3452198547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd8b4159c60 0x7fd8b40b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 >> 172.21.15.170:0/3452198547 conn(0x7fd8b4068e20 msgr2=0x7fd8b40c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:19.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 shutdown_connections 2023-10-08T16:11:19.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.256+0000 7fd8b9e9b700 1 -- 172.21.15.170:0/3452198547 wait complete. 2023-10-08T16:11:19.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:11:19.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.444+0000 7fe89ccef700 1 Processor -- start 2023-10-08T16:11:19.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.444+0000 7fe89ccef700 1 -- start start 2023-10-08T16:11:19.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89ccef700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe8981579f0 0x7fe898162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89ccef700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8980cb300 con 0x7fe8980b1850 2023-10-08T16:11:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89ccef700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8980cb5e0 con 0x7fe8980b1c20 2023-10-08T16:11:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89ccef700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89806fc30 con 0x7fe8981579f0 2023-10-08T16:11:19.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe8980b1850 0x7fe8980ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57562/0 (socket says 172.21.15.170:57562) 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe89659c700 1 -- 172.21.15.170:0/2462204328 learned_addr learned my addr 172.21.15.170:0/2462204328 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe896d9d700 1 --2- 172.21.15.170:0/2462204328 >> v2:172.21.15.191:3300/0 conn(0x7fe8981579f0 0x7fe898162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127411419 0 0) 0x7fe8980cb300 con 0x7fe8980b1850 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.445+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe884003570 con 0x7fe8980b1850 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917817379 0 0) 0x7fe8980cb5e0 con 0x7fe8980b1c20 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8980cb300 con 0x7fe8980b1c20 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3735576475 0 0) 0x7fe884003570 con 0x7fe8980b1850 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 >> v2:172.21.15.191:3300/0 conn(0x7fe8981579f0 msgr2=0x7fe898162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 --2- 172.21.15.170:0/2462204328 >> v2:172.21.15.191:3300/0 conn(0x7fe8981579f0 0x7fe898162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 >> v1:172.21.15.191:6789/0 conn(0x7fe8980b1c20 legacy=0x7fe8981572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980cb8c0 con 0x7fe8980b1850 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fe888001470 con 0x7fe8980b1850 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe896d9d700 1 --2- 172.21.15.170:0/2462204328 >> v2:172.21.15.191:3300/0 conn(0x7fe8981579f0 0x7fe898162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 >> v1:172.21.15.170:6789/0 conn(0x7fe8980b1850 legacy=0x7fe8980ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884003c40 0x7fe884004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 0x7fe884004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe884005210 0x7fe88400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89806fc30 con 0x7fe884004810 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88400c130 con 0x7fe884005210 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88400c2c0 con 0x7fe884003c40 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 0x7fe884004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe896d9d700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe884005210 0x7fe88400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe89659c700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884003c40 0x7fe884004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 -- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884003c40 msgr2=0x7fe884004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884003c40 0x7fe884004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 -- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe884005210 msgr2=0x7fe88400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe884005210 0x7fe88400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe896d9d700 1 -- 172.21.15.170:0/2462204328 reap_dead start 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 -- 172.21.15.170:0/2462204328 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884003570 con 0x7fe884004810 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 0x7fe884004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe880001a20 tx=0x7fe880007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d5a47eceef422e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.446+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000c550 con 0x7fe884004810 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880001e90 con 0x7fe884004810 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe895d9b700 1 -- 172.21.15.170:0/2462204328 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000c8e0 con 0x7fe884004810 2023-10-08T16:11:19.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 -- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 msgr2=0x7fe884004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 0x7fe884004c80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe880001a20 tx=0x7fe880007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 -- 172.21.15.170:0/2462204328 shutdown_connections 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2462204328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe884004810 0x7fe884004c80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 -- 172.21.15.170:0/2462204328 >> 172.21.15.170:0/2462204328 conn(0x7fe89806a320 msgr2=0x7fe8980cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 -- 172.21.15.170:0/2462204328 shutdown_connections 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.447+0000 7fe89ccef700 1 -- 172.21.15.170:0/2462204328 wait complete. 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.448+0000 7fe89ccef700 1 Processor -- start 2023-10-08T16:11:19.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.448+0000 7fe89ccef700 1 -- start start 2023-10-08T16:11:19.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 0x7fe8980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe89806ed40 0x7fe89806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980cc260 con 0x7fe898145200 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89806f6f0 con 0x7fe89806ed40 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89ccef700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89815c9e0 con 0x7fe884004810 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe88dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 0x7fe8980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe88dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40654/0 (socket says 172.21.15.170:40654) 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe88dd9b700 1 -- 172.21.15.170:0/2698928960 learned_addr learned my addr 172.21.15.170:0/2698928960 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe896d9d700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe89806ed40 0x7fe89806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 -- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe89806ed40 msgr2=0x7fe89806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe89806ed40 0x7fe89806f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 -- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 msgr2=0x7fe89815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe89659c700 1 -- 172.21.15.170:0/2698928960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89815bdd0 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.449+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe89659c700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 0x7fe8980a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8980cd080 tx=0x7fe888001d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888012560 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880126e0 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89815c0b0 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888012aa0 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.450+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89815c670 con 0x7fe884004810 2023-10-08T16:11:19.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.451+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8980ad560 con 0x7fe884004810 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.451+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe88801d460 con 0x7fe884004810 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.451+0000 7fe88effd700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 0x7fe8780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.452+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 0x7fe8780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.452+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe888084bf0 con 0x7fe884004810 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.452+0000 7fe88dd9b700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 0x7fe8780697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe880001830 tx=0x7fe880015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.456+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mon.2 v2:172.21.15.191:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe888051880 con 0x7fe884004810 2023-10-08T16:11:19.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.670+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe89815cd70 con 0x7fe878067320 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.672+0000 7fe88effd700 1 -- 172.21.15.170:0/2698928960 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe89815cd70 con 0x7fe878067320 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:19.673 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.673+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 msgr2=0x7fe8780697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.673+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 0x7fe8780697d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe880001830 tx=0x7fe880015000 comp rx=0 tx=0).stop 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.673+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 msgr2=0x7fe8980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.673+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 0x7fe8980a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe8980cd080 tx=0x7fe888001d20 comp rx=0 tx=0).stop 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 shutdown_connections 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe89806ed40 0x7fe89806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe884004810 0x7fe8980a6280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe878067320 0x7fe8780697d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 --2- 172.21.15.170:0/2698928960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898145200 0x7fe89815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 >> 172.21.15.170:0/2698928960 conn(0x7fe89806a320 msgr2=0x7fe898153140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 shutdown_connections 2023-10-08T16:11:19.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.674+0000 7fe89ccef700 1 -- 172.21.15.170:0/2698928960 wait complete. 2023-10-08T16:11:19.689 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000085327 2023-10-08T16:11:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.804+0000 7f16a9b25700 1 Processor -- start 2023-10-08T16:11:19.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.804+0000 7f16a9b25700 1 -- start start 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a9b25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f16a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a9b25700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a4069ee0 con 0x7f16a4159c60 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a9b25700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a415a160 con 0x7f16a4159890 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a9b25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d13a0 con 0x7f16a4069560 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f16a4159c60 0x7f16a415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57574/0 (socket says 172.21.15.170:57574) 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a2ffd700 1 -- 172.21.15.170:0/1718024166 learned_addr learned my addr 172.21.15.170:0/1718024166 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a3fff700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f16a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704665354 0 0) 0x7f16a4069ee0 con 0x7f16a4159c60 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.805+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f168c003570 con 0x7f16a4159c60 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060090243 0 0) 0x7f16a415a160 con 0x7f16a4159890 2023-10-08T16:11:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a4069ee0 con 0x7f16a4159890 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679908878 0 0) 0x7f168c003570 con 0x7f16a4159c60 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 msgr2=0x7f16a40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f16a40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 >> v1:172.21.15.191:6789/0 conn(0x7f16a4159890 legacy=0x7f16a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a415e140 con 0x7f16a4159c60 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f1698001470 con 0x7f16a4159c60 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 >> v1:172.21.15.170:6789/0 conn(0x7f16a4159c60 legacy=0x7f16a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f168c003c20 0x7f168c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f168c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f168c004d50 0x7f168c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a40d13a0 con 0x7f168c004d50 2023-10-08T16:11:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00bb70 con 0x7f168c003c20 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168c00bd30 con 0x7f16a4069560 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f168c004d50 0x7f168c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a2ffd700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f168c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a37fe700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f168c003c20 0x7f168c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 -- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 msgr2=0x7f168c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a4069560 0x7f168c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 -- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f168c003c20 msgr2=0x7f168c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f168c003c20 0x7f168c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 -- 172.21.15.170:0/1718024166 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f168c003570 con 0x7f168c004d50 2023-10-08T16:11:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a3fff700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f168c004d50 0x7f168c0051c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f169c00e210 tx=0x7f169c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e57d4b05764cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00eac0 con 0x7f168c004d50 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c00f040 con 0x7f168c004d50 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.806+0000 7f16a27fc700 1 -- 172.21.15.170:0/1718024166 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00ee70 con 0x7f168c004d50 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 -- 172.21.15.170:0/1718024166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f168c004d50 msgr2=0x7f168c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 --2- 172.21.15.170:0/1718024166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f168c004d50 0x7f168c0051c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f169c00e210 tx=0x7f169c00b040 comp rx=0 tx=0).stop 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a3fff700 1 -- 172.21.15.170:0/1718024166 reap_dead start 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 -- 172.21.15.170:0/1718024166 shutdown_connections 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 -- 172.21.15.170:0/1718024166 >> 172.21.15.170:0/1718024166 conn(0x7f16a4068e20 msgr2=0x7f16a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 -- 172.21.15.170:0/1718024166 shutdown_connections 2023-10-08T16:11:19.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.807+0000 7f16a9b25700 1 -- 172.21.15.170:0/1718024166 wait complete. 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.808+0000 7f16a9b25700 1 Processor -- start 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.808+0000 7f16a9b25700 1 -- start start 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a414d370 0x7f16a414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4151120 0x7f16a4169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415f180 con 0x7f16a414d740 2023-10-08T16:11:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415e850 con 0x7f16a4151120 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a9b25700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a415eb30 con 0x7f16a414d370 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40672/0 (socket says 172.21.15.170:40672) 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a2ffd700 1 -- 172.21.15.170:0/333296812 learned_addr learned my addr 172.21.15.170:0/333296812 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a37fe700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a414d370 0x7f16a414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.809+0000 7f16a3fff700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4151120 0x7f16a4169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 -- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a414d370 msgr2=0x7f16a414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a414d370 0x7f16a414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 -- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4151120 msgr2=0x7f16a4169db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4151120 0x7f16a4169db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 -- 172.21.15.170:0/333296812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a415ee10 con 0x7f16a414d740 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a2ffd700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f16980073c0 tx=0x7f1698001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800f620 con 0x7f16a414d740 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800f7a0 con 0x7f16a414d740 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a41515c0 con 0x7f16a414d740 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800fb60 con 0x7f16a414d740 2023-10-08T16:11:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.810+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a416a2f0 con 0x7f16a414d740 2023-10-08T16:11:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.811+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a415f2f0 con 0x7f16a414d740 2023-10-08T16:11:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.811+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f169801e3b0 con 0x7f16a414d740 2023-10-08T16:11:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.812+0000 7f168b7fe700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 0x7f16900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.812+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1698084850 con 0x7f16a414d740 2023-10-08T16:11:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.812+0000 7f16a37fe700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 0x7f16900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.812+0000 7f16a37fe700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 0x7f16900697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1694001220 tx=0x7f1694007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:19.816+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16980515f0 con 0x7f16a414d740 2023-10-08T16:11:20.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:20.030+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000085327", "target": ["mon-mgr", ""]}) v1 -- 0x7f16a404c3c0 con 0x7f1690067320 2023-10-08T16:11:22.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.529+0000 7f168b7fe700 1 -- 172.21.15.170:0/333296812 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f16a404c3c0 con 0x7f1690067320 2023-10-08T16:11:22.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 msgr2=0x7f16900697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:22.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 0x7f16900697d0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1694001220 tx=0x7f1694007d10 comp rx=0 tx=0).stop 2023-10-08T16:11:22.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 msgr2=0x7f16a40ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f16980073c0 tx=0x7f1698001e10 comp rx=0 tx=0).stop 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 shutdown_connections 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f16a4151120 0x7f16a4169db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f16a414d370 0x7f16a414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1690067320 0x7f16900697d0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 --2- 172.21.15.170:0/333296812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16a414d740 0x7f16a40ceed0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 >> 172.21.15.170:0/333296812 conn(0x7f16a4068e20 msgr2=0x7f16a40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 shutdown_connections 2023-10-08T16:11:22.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.534+0000 7f16a9b25700 1 -- 172.21.15.170:0/333296812 wait complete. 2023-10-08T16:11:22.536 INFO:teuthology.orchestra.run.smithi170.stderr:Volume created successfully (no MDS daemons created) 2023-10-08T16:11:22.553 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create volume_0000000000085328 2023-10-08T16:11:22.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.678+0000 7ff7c54e0700 1 Processor -- start 2023-10-08T16:11:22.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.678+0000 7ff7c54e0700 1 -- start start 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff7c015a3b0 0x7ff7c015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff7c006e470 0x7ff7c006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c00aea70 con 0x7ff7c006e470 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c006ee00 con 0x7ff7c015a3b0 2023-10-08T16:11:22.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c54e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00ae480 con 0x7ff7c00aef00 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff7c006e470 0x7ff7c006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff7c006e470 0x7ff7c006e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40688/0 (socket says 172.21.15.170:40688) 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 -- 172.21.15.170:0/2351597700 learned_addr learned my addr 172.21.15.170:0/2351597700 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 -- 172.21.15.170:0/2351597700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 msgr2=0x7ff7c015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 --2- 172.21.15.170:0/2351597700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 -- 172.21.15.170:0/2351597700 >> v2:172.21.15.191:3300/0 conn(0x7ff7c015a3b0 msgr2=0x7ff7c015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 --2- 172.21.15.170:0/2351597700 >> v2:172.21.15.191:3300/0 conn(0x7ff7c015a3b0 0x7ff7c015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.679+0000 7ff7c4cdf700 1 -- 172.21.15.170:0/2351597700 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c015acc0 con 0x7ff7c006e470 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c4cdf700 1 --2- 172.21.15.170:0/2351597700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c006e8c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0063350 tx=0x7ff7b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa23b17f92296012 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7beffd700 1 -- 172.21.15.170:0/2351597700 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b800e070 con 0x7ff7c006e470 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7beffd700 1 -- 172.21.15.170:0/2351597700 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b800ba50 con 0x7ff7c006e470 2023-10-08T16:11:22.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7beffd700 1 -- 172.21.15.170:0/2351597700 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80092e0 con 0x7ff7c006e470 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 -- 172.21.15.170:0/2351597700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 msgr2=0x7ff7c006e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/2351597700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c006e8c0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0063350 tx=0x7ff7b8009840 comp rx=0 tx=0).stop 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 -- 172.21.15.170:0/2351597700 shutdown_connections 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/2351597700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c006e8c0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/2351597700 >> v2:172.21.15.191:3300/0 conn(0x7ff7c015a3b0 0x7ff7c015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/2351597700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 -- 172.21.15.170:0/2351597700 >> 172.21.15.170:0/2351597700 conn(0x7ff7c00c9f70 msgr2=0x7ff7c006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 -- 172.21.15.170:0/2351597700 shutdown_connections 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.680+0000 7ff7c54e0700 1 -- 172.21.15.170:0/2351597700 wait complete. 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.681+0000 7ff7c54e0700 1 Processor -- start 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 -- start start 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c01552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c015a3b0 0x7ff7c0155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01526b0 con 0x7ff7c006e470 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00c5c20 con 0x7ff7c015a3b0 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7c54e0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c00c5f00 con 0x7ff7c00aef00 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:22.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7bffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40694/0 (socket says 172.21.15.170:40694) 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.682+0000 7ff7bffff700 1 -- 172.21.15.170:0/580654775 learned_addr learned my addr 172.21.15.170:0/580654775 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7c4cdf700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c015a3b0 0x7ff7c0155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bf7fe700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c01552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 -- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 msgr2=0x7ff7c01552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c01552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 -- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c015a3b0 msgr2=0x7ff7c0155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c015a3b0 0x7ff7c0155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 -- 172.21.15.170:0/580654775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8009ae0 con 0x7ff7c006e470 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bf7fe700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:22.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7bffff700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b590 tx=0x7ff7b0006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000f5e0 con 0x7ff7c006e470 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000f740 con 0x7ff7c006e470 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c00c61e0 con 0x7ff7c006e470 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000fb00 con 0x7ff7c006e470 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.683+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c00c5890 con 0x7ff7c006e470 2023-10-08T16:11:22.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.684+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c00c6810 con 0x7ff7c006e470 2023-10-08T16:11:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.684+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff7b001a400 con 0x7ff7c006e470 2023-10-08T16:11:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.685+0000 7ff7a7fff700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 0x7ff7ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.685+0000 7ff7bf7fe700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 0x7ff7ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.685+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 6603+0+0 (secure 0 0 0) 0x7ff7b0084a90 con 0x7ff7c006e470 2023-10-08T16:11:22.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.685+0000 7ff7bf7fe700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 0x7ff7ac0697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0159c60 tx=0x7ff7b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:22.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.689+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7b00514a0 con 0x7ff7c006e470 2023-10-08T16:11:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:22.899+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "volume_0000000000085328", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7c015dd30 con 0x7ff7ac067320 2023-10-08T16:11:25.210 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:11:25.210+0000 7f1d255a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:11:25.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.576+0000 7ff7a7fff700 1 -- 172.21.15.170:0/580654775 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) v1 ==== 60+0+0 (secure 0 0 0) 0x7ff7c015dd30 con 0x7ff7ac067320 2023-10-08T16:11:25.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 msgr2=0x7ff7ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 0x7ff7ac0697d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0159c60 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-10-08T16:11:25.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 msgr2=0x7ff7c0152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b590 tx=0x7ff7b0006f90 comp rx=0 tx=0).stop 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 shutdown_connections 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7c015a3b0 0x7ff7c0155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7ac067320 0x7ff7ac0697d0 secure :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0159c60 tx=0x7ff7b4008040 comp rx=0 tx=0).stop 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7c006e470 0x7ff7c0152170 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 --2- 172.21.15.170:0/580654775 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7c00aef00 0x7ff7c01552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 >> 172.21.15.170:0/580654775 conn(0x7ff7c00c9f70 msgr2=0x7ff7c00c8720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:25.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 shutdown_connections 2023-10-08T16:11:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.580+0000 7ff7c54e0700 1 -- 172.21.15.170:0/580654775 wait complete. 2023-10-08T16:11:25.582 INFO:teuthology.orchestra.run.smithi170.stderr:Volume created successfully (no MDS daemons created) 2023-10-08T16:11:25.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:11:25.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.717+0000 7fb24ac40700 1 Processor -- start 2023-10-08T16:11:25.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.717+0000 7fb24ac40700 1 -- start start 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb24ac40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb2440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb24ac40700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb244069ee0 con 0x7fb244159c60 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb24ac40700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb24415a160 con 0x7fb244159890 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb24ac40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440d13a0 con 0x7fb244069560 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb23bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb244159c60 0x7fb24415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58698/0 (socket says 172.21.15.170:58698) 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb23bfff700 1 -- 172.21.15.170:0/424507835 learned_addr learned my addr 172.21.15.170:0/424507835 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764208171 0 0) 0x7fb244069ee0 con 0x7fb244159c60 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb2491dd700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb2440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.718+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb22c003570 con 0x7fb244159c60 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190951310 0 0) 0x7fb22c003570 con 0x7fb244159c60 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 msgr2=0x7fb2440699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb2440699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 >> v1:172.21.15.191:6789/0 conn(0x7fb244159890 legacy=0x7fb24415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb24415e140 con 0x7fb244159c60 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb234001470 con 0x7fb244159c60 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 >> v1:172.21.15.170:6789/0 conn(0x7fb244159c60 legacy=0x7fb24415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c003a90 0x7fb22c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb22c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c004d50 0x7fb22c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440d13a0 con 0x7fb22c003a90 2023-10-08T16:11:25.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c00bba0 con 0x7fb22c004d50 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c00bd60 con 0x7fb244069560 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c003a90 0x7fb22c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23bfff700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb22c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2491dd700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c004d50 0x7fb22c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 -- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 msgr2=0x7fb22c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb22c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 -- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c004d50 msgr2=0x7fb22c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c004d50 0x7fb22c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 -- 172.21.15.170:0/424507835 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c003570 con 0x7fb22c003a90 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23bfff700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb244069560 0x7fb22c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb2489dc700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c003a90 0x7fb22c003ea0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb240001870 tx=0x7fb240007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca333474a6bceca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240012400 con 0x7fb22c003a90 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb240012580 con 0x7fb22c003a90 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.719+0000 7fb23b7fe700 1 -- 172.21.15.170:0/424507835 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240012940 con 0x7fb22c003a90 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 -- 172.21.15.170:0/424507835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c003a90 msgr2=0x7fb22c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 --2- 172.21.15.170:0/424507835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb22c003a90 0x7fb22c003ea0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fb240001870 tx=0x7fb240007d10 comp rx=0 tx=0).stop 2023-10-08T16:11:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb2491dd700 1 -- 172.21.15.170:0/424507835 reap_dead start 2023-10-08T16:11:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 -- 172.21.15.170:0/424507835 shutdown_connections 2023-10-08T16:11:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 -- 172.21.15.170:0/424507835 >> 172.21.15.170:0/424507835 conn(0x7fb244068e20 msgr2=0x7fb2440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 -- 172.21.15.170:0/424507835 shutdown_connections 2023-10-08T16:11:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.720+0000 7fb24ac40700 1 -- 172.21.15.170:0/424507835 wait complete. 2023-10-08T16:11:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.721+0000 7fb24ac40700 1 Processor -- start 2023-10-08T16:11:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.721+0000 7fb24ac40700 1 -- start start 2023-10-08T16:11:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c003a90 0x7fb2440cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440d05f0 0x7fb2440d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440d0fd0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440ce720 con 0x7fb22c003a90 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb24ac40700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2440ceed0 con 0x7fb2440d05f0 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47012/0 (socket says 172.21.15.170:47012) 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 -- 172.21.15.170:0/1324185265 learned_addr learned my addr 172.21.15.170:0/1324185265 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb2489dc700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c003a90 0x7fb2440cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb2491dd700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440d05f0 0x7fb2440d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 -- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440d05f0 msgr2=0x7fb2440d0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440d05f0 0x7fb2440d0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 -- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c003a90 msgr2=0x7fb2440cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c003a90 0x7fb2440cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.722+0000 7fb23bfff700 1 -- 172.21.15.170:0/1324185265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2440cd680 con 0x7fb2440cd0c0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb23bfff700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb2340163e0 tx=0x7fb234000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb234016df0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb234018040 con 0x7fb2440cd0c0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2440cd960 con 0x7fb2440cd0c0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2340122e0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.723+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2440cdf20 con 0x7fb2440cd0c0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.724+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24404c3c0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.724+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb234010850 con 0x7fb2440cd0c0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.724+0000 7fb238ff9700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 0x7fb2300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.725+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7501+0+0 (secure 0 0 0) 0x7fb23408cee0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.725+0000 7fb2489dc700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 0x7fb2300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.725+0000 7fb2489dc700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 0x7fb2300697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb240001eb0 tx=0x7fb240015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:25.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.729+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2340555a0 con 0x7fb2440cd0c0 2023-10-08T16:11:25.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.942+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2440c72b0 con 0x7fb230067320 2023-10-08T16:11:25.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.943+0000 7fb238ff9700 1 -- 172.21.15.170:0/1324185265 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+151 (secure 0 0 0) 0x7fb2440c72b0 con 0x7fb230067320 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:25.944 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000085327" 2023-10-08T16:11:25.945 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:25.945 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:25.945 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000085328" 2023-10-08T16:11:25.945 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:25.945 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 msgr2=0x7fb2300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 0x7fb2300697d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb240001eb0 tx=0x7fb240015000 comp rx=0 tx=0).stop 2023-10-08T16:11:25.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 msgr2=0x7fb24414c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb2340163e0 tx=0x7fb234000f90 comp rx=0 tx=0).stop 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 shutdown_connections 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb2440d05f0 0x7fb2440d0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb22c003a90 0x7fb2440cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb230067320 0x7fb2300697d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 --2- 172.21.15.170:0/1324185265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2440cd0c0 0x7fb24414c1d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 >> 172.21.15.170:0/1324185265 conn(0x7fb244068e20 msgr2=0x7fb2440b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:25.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 shutdown_connections 2023-10-08T16:11:25.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:25.945+0000 7fb24ac40700 1 -- 172.21.15.170:0/1324185265 wait complete. 2023-10-08T16:11:25.958 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000085327 --yes-i-really-mean-it 2023-10-08T16:11:26.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.075+0000 7f0a57471700 1 Processor -- start 2023-10-08T16:11:26.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.075+0000 7f0a57471700 1 -- start start 2023-10-08T16:11:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a57471700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a500b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a57471700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a500b2080 con 0x7f0a500b4e70 2023-10-08T16:11:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a57471700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a500cfb00 con 0x7f0a500b33b0 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a57471700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500b5c40 con 0x7f0a500b16d0 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a5646f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0a500b4e70 0x7f0a500ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58710/0 (socket says 172.21.15.170:58710) 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a5646f700 1 -- 172.21.15.170:0/1731873307 learned_addr learned my addr 172.21.15.170:0/1731873307 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217965646 0 0) 0x7f0a500b2080 con 0x7f0a500b4e70 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a38003570 con 0x7f0a500b4e70 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.076+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a500b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3154950819 0 0) 0x7f0a38003570 con 0x7f0a500b4e70 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 msgr2=0x7f0a500b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a500b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 >> v1:172.21.15.191:6789/0 conn(0x7f0a500b33b0 legacy=0x7f0a500b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500b5f20 con 0x7f0a500b4e70 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f0a400017f0 con 0x7f0a500b4e70 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a500b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 >> v1:172.21.15.170:6789/0 conn(0x7f0a500b4e70 legacy=0x7f0a500ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38003a90 0x7f0a38003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a38004810 0x7f0a38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a3800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500b5c40 con 0x7f0a38004810 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a38005500 con 0x7f0a38003a90 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a380056c0 con 0x7f0a500b16d0 2023-10-08T16:11:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a38004810 0x7f0a38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5646f700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38003a90 0x7f0a38003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a56c70700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a3800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 msgr2=0x7f0a3800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500b16d0 0x7f0a3800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38003a90 msgr2=0x7f0a38003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38003a90 0x7f0a38003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1731873307 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a38003570 con 0x7f0a38004810 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a38004810 0x7f0a38004c80 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009ba0 tx=0x7f0a44008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d378bbbae8fa5e76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400c070 con 0x7f0a38004810 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400e5e0 con 0x7f0a38004810 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.077+0000 7f0a5546d700 1 -- 172.21.15.170:0/1731873307 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a44009510 con 0x7f0a38004810 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 -- 172.21.15.170:0/1731873307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a38004810 msgr2=0x7f0a38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 --2- 172.21.15.170:0/1731873307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a38004810 0x7f0a38004c80 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009ba0 tx=0x7f0a44008040 comp rx=0 tx=0).stop 2023-10-08T16:11:26.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a56c70700 1 -- 172.21.15.170:0/1731873307 reap_dead start 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 -- 172.21.15.170:0/1731873307 shutdown_connections 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 -- 172.21.15.170:0/1731873307 >> 172.21.15.170:0/1731873307 conn(0x7f0a500d10c0 msgr2=0x7f0a500affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 -- 172.21.15.170:0/1731873307 shutdown_connections 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.078+0000 7f0a57471700 1 -- 172.21.15.170:0/1731873307 wait complete. 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.079+0000 7f0a57471700 1 Processor -- start 2023-10-08T16:11:26.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.079+0000 7f0a57471700 1 -- start start 2023-10-08T16:11:26.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38004810 0x7f0a500a7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500aa700 0x7f0a5014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500cc0c0 con 0x7f0a5014ea30 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a500a8dc0 con 0x7f0a38004810 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a57471700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5014eea0 con 0x7f0a500aa700 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47034/0 (socket says 172.21.15.170:47034) 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1295106077 learned_addr learned my addr 172.21.15.170:0/1295106077 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a56c70700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500aa700 0x7f0a5014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a5646f700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38004810 0x7f0a500a7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500aa700 msgr2=0x7f0a5014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500aa700 0x7f0a5014dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:26.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38004810 msgr2=0x7f0a500a7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38004810 0x7f0a500a7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 -- 172.21.15.170:0/1295106077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a500a81b0 con 0x7f0a5014ea30 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.080+0000 7f0a55c6e700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e450 tx=0x7f0a4401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.081+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400ee80 con 0x7f0a5014ea30 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.081+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a44013bc0 con 0x7f0a5014ea30 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.081+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a500a8490 con 0x7f0a5014ea30 2023-10-08T16:11:26.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.081+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4401c440 con 0x7f0a5014ea30 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.081+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a500a8a50 con 0x7f0a5014ea30 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.082+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a500a8f50 con 0x7f0a5014ea30 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.082+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0a4401c7f0 con 0x7f0a5014ea30 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.083+0000 7f0a4e7fc700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 0x7f0a3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.083+0000 7f0a5646f700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 0x7f0a3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.083+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 7501+0+0 (secure 0 0 0) 0x7f0a44086d50 con 0x7f0a5014ea30 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.083+0000 7f0a5646f700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 0x7f0a3c0697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0a400014a0 tx=0x7f0a40006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:26.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.087+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a440b85a0 con 0x7f0a5014ea30 2023-10-08T16:11:26.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:26.298+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000085327", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a5004bb80 con 0x7f0a3c067320 2023-10-08T16:11:26.301 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:11:26.301+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:11:26.304 INFO:tasks.ceph.mds.c.smithi170.stderr: -1257> 2023-10-08T16:10:59.793+0000 7f1d2b5ae700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:11:26.306 INFO:tasks.ceph.mds.c.smithi170.stderr: -37> 2023-10-08T16:11:25.210+0000 7f1d255a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:11:26.331 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:11:29.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.327+0000 7f0a4e7fc700 1 -- 172.21.15.170:0/1295106077 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f0a5004bb80 con 0x7f0a3c067320 2023-10-08T16:11:29.328 INFO:teuthology.orchestra.run.smithi170.stdout:metadata pool: cephfs.volume_0000000000085327.meta data pool: ['cephfs.volume_0000000000085327.data'] removed 2023-10-08T16:11:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 msgr2=0x7f0a3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 0x7f0a3c0697d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0a400014a0 tx=0x7f0a40006040 comp rx=0 tx=0).stop 2023-10-08T16:11:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 msgr2=0x7f0a500cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400e450 tx=0x7f0a4401a000 comp rx=0 tx=0).stop 2023-10-08T16:11:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 shutdown_connections 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0a500aa700 0x7f0a5014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0a38004810 0x7f0a500a7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0a3c067320 0x7f0a3c0697d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 --2- 172.21.15.170:0/1295106077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a5014ea30 0x7f0a500cbb80 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 >> 172.21.15.170:0/1295106077 conn(0x7f0a500d10c0 msgr2=0x7f0a50069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 shutdown_connections 2023-10-08T16:11:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.330+0000 7f0a57471700 1 -- 172.21.15.170:0/1295106077 wait complete. 2023-10-08T16:11:29.346 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm volume_0000000000085328 --yes-i-really-mean-it 2023-10-08T16:11:29.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.469+0000 7f8f96906700 1 Processor -- start 2023-10-08T16:11:29.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.469+0000 7f8f96906700 1 -- start start 2023-10-08T16:11:29.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96906700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8f90157680 0x7f8f900cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96906700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f900c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96906700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f900cd240 con 0x7f8f90157a50 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96906700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f900cc700 con 0x7f8f90157680 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96906700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c9a20 con 0x7f8f900b1450 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f95103700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8f90157a50 0x7f8f900c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58726/0 (socket says 172.21.15.170:58726) 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f95103700 1 -- 172.21.15.170:0/1935304301 learned_addr learned my addr 172.21.15.170:0/1935304301 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f95904700 1 --2- 172.21.15.170:0/1935304301 >> v2:172.21.15.191:3300/0 conn(0x7f8f90157680 0x7f8f900cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f96105700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f900c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370754644 0 0) 0x7f8f900cd240 con 0x7f8f90157a50 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.470+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c003570 con 0x7f8f90157a50 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742981366 0 0) 0x7f8f7c003570 con 0x7f8f90157a50 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 msgr2=0x7f8f900c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f900c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 >> v2:172.21.15.191:3300/0 conn(0x7f8f90157680 msgr2=0x7f8f900cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 --2- 172.21.15.170:0/1935304301 >> v2:172.21.15.191:3300/0 conn(0x7f8f90157680 0x7f8f900cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900cc910 con 0x7f8f90157a50 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8f80001470 con 0x7f8f90157a50 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 >> v1:172.21.15.170:6789/0 conn(0x7f8f90157a50 legacy=0x7f8f900c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f7c003a60 0x7f8f7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f7c004d50 0x7f8f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900cc700 con 0x7f8f7c003a60 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c9a20 con 0x7f8f7c004d50 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c00ba60 con 0x7f8f900b1450 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f7c003a60 0x7f8f7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95103700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f96105700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f7c004d50 0x7f8f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 -- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 msgr2=0x7f8f7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f900b1450 0x7f8f7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 -- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f7c004d50 msgr2=0x7f8f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:29.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f7c004d50 0x7f8f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 -- 172.21.15.170:0/1935304301 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c003570 con 0x7f8f7c003a60 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f96105700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f7c004d50 0x7f8f7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f95904700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f7c003a60 0x7f8f7c003e70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009c00 tx=0x7f8f8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5a4b126487f280 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.471+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00e6b0 con 0x7f8f7c003a60 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00e810 con 0x7f8f7c003a60 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f94902700 1 -- 172.21.15.170:0/1935304301 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00ebd0 con 0x7f8f7c003a60 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 -- 172.21.15.170:0/1935304301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f7c003a60 msgr2=0x7f8f7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 --2- 172.21.15.170:0/1935304301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f7c003a60 0x7f8f7c003e70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009c00 tx=0x7f8f8c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96105700 1 -- 172.21.15.170:0/1935304301 reap_dead start 2023-10-08T16:11:29.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 -- 172.21.15.170:0/1935304301 shutdown_connections 2023-10-08T16:11:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 -- 172.21.15.170:0/1935304301 >> 172.21.15.170:0/1935304301 conn(0x7f8f9006baa0 msgr2=0x7f8f9006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 -- 172.21.15.170:0/1935304301 shutdown_connections 2023-10-08T16:11:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.472+0000 7f8f96906700 1 -- 172.21.15.170:0/1935304301 wait complete. 2023-10-08T16:11:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.473+0000 7f8f96906700 1 Processor -- start 2023-10-08T16:11:29.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.473+0000 7f8f96906700 1 -- start start 2023-10-08T16:11:29.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7c003a60 0x7f8f900c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f901512e0 0x7f8f90159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c6c40 con 0x7f8f90151750 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900c6370 con 0x7f8f901512e0 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96906700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9015a3e0 con 0x7f8f7c003a60 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47082/0 (socket says 172.21.15.170:47082) 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 -- 172.21.15.170:0/2987853960 learned_addr learned my addr 172.21.15.170:0/2987853960 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f95103700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f901512e0 0x7f8f90159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f95904700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7c003a60 0x7f8f900c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 -- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7c003a60 msgr2=0x7f8f900c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7c003a60 0x7f8f900c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 -- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f901512e0 msgr2=0x7f8f90159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:29.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f901512e0 0x7f8f90159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.474+0000 7f8f96105700 1 -- 172.21.15.170:0/2987853960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900c65f0 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f96105700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006cd60 tx=0x7f8f8800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800e050 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8800f040 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f900c68d0 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f88010ec0 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.475+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900c83d0 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.476+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f900c6df0 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.476+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8f88003020 con 0x7f8f90151750 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.477+0000 7f8f85ffb700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 0x7f8f740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:29.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.477+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6689+0+0 (secure 0 0 0) 0x7f8f88085510 con 0x7f8f90151750 2023-10-08T16:11:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.477+0000 7f8f95904700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 0x7f8f740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.477+0000 7f8f95904700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 0x7f8f740697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0091e0 tx=0x7f8f8c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:29.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.481+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f900c6df0 con 0x7f8f90151750 2023-10-08T16:11:29.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:29.694+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "volume_0000000000085328", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f900c6d80 con 0x7f8f74067320 2023-10-08T16:11:29.697 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:11:29.697+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:11:29.722 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:11:32.714 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.714+0000 7f8f85ffb700 1 -- 172.21.15.170:0/2987853960 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+110 (secure 0 0 0) 0x7f8f900c6d80 con 0x7f8f74067320 2023-10-08T16:11:32.716 INFO:teuthology.orchestra.run.smithi170.stdout:metadata pool: cephfs.volume_0000000000085328.meta data pool: ['cephfs.volume_0000000000085328.data'] removed 2023-10-08T16:11:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.717+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 msgr2=0x7f8f740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.717+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 0x7f8f740697d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0091e0 tx=0x7f8f8c01a000 comp rx=0 tx=0).stop 2023-10-08T16:11:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.717+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 msgr2=0x7f8f900b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:32.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.717+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f8f9006cd60 tx=0x7f8f8800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 shutdown_connections 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8f74067320 0x7f8f740697d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f90151750 0x7f8f900b3660 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8f7c003a60 0x7f8f900c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 --2- 172.21.15.170:0/2987853960 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8f901512e0 0x7f8f90159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 >> 172.21.15.170:0/2987853960 conn(0x7f8f9006baa0 msgr2=0x7f8f900cae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 shutdown_connections 2023-10-08T16:11:32.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:32.718+0000 7f8f96906700 1 -- 172.21.15.170:0/2987853960 wait complete. 2023-10-08T16:11:32.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:11:32.733 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:11:32.773 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:11:32.817 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:11:32.818 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:11:32.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:11:32.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:32.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:11:32.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:32.870 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:11:32.885 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:11:32.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:11:32.885 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:32.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:11:32.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:32.935 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:11:32.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:33.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.069+0000 7f4159032700 1 Processor -- start 2023-10-08T16:11:33.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.069+0000 7f4159032700 1 -- start start 2023-10-08T16:11:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.070+0000 7f4159032700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.070+0000 7f4159032700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 0x7f41540699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.070+0000 7f4159032700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415415a160 con 0x7f4154159c60 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.070+0000 7f4159032700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4154069f20 con 0x7f4154069560 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.070+0000 7f4159032700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540d13a0 con 0x7f4154159890 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:50486/0 (socket says 172.21.15.170:50486) 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 -- 172.21.15.170:0/3791643938 learned_addr learned my addr 172.21.15.170:0/3791643938 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415359e700 1 --2- 172.21.15.170:0/3791643938 >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 0x7f41540699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415259c700 1 -- 172.21.15.170:0/3791643938 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389035403 0 0) 0x7f415415a160 con 0x7f4154159c60 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415259c700 1 -- 172.21.15.170:0/3791643938 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4140003570 con 0x7f4154159c60 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 -- 172.21.15.170:0/3791643938 >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 msgr2=0x7f41540699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 --2- 172.21.15.170:0/3791643938 >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 0x7f41540699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 -- 172.21.15.170:0/3791643938 >> v1:172.21.15.191:6789/0 conn(0x7f4154159c60 legacy=0x7f415415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 -- 172.21.15.170:0/3791643938 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415415e140 con 0x7f4154159890 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415359e700 1 --2- 172.21.15.170:0/3791643938 >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 0x7f41540699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f4152d9d700 1 --2- 172.21.15.170:0/3791643938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41440099b0 tx=0x7f4144009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dc61398d74b5232 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415259c700 1 -- 172.21.15.170:0/3791643938 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400b680 con 0x7f4154159890 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415259c700 1 -- 172.21.15.170:0/3791643938 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400b7e0 con 0x7f4154159890 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.071+0000 7f415259c700 1 -- 172.21.15.170:0/3791643938 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400bb60 con 0x7f4154159890 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 -- 172.21.15.170:0/3791643938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 msgr2=0x7f415415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 --2- 172.21.15.170:0/3791643938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f41440099b0 tx=0x7f4144009250 comp rx=0 tx=0).stop 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 -- 172.21.15.170:0/3791643938 shutdown_connections 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 --2- 172.21.15.170:0/3791643938 >> v2:172.21.15.191:3300/0 conn(0x7f4154069560 0x7f41540699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 --2- 172.21.15.170:0/3791643938 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f415415abd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 -- 172.21.15.170:0/3791643938 >> 172.21.15.170:0/3791643938 conn(0x7f4154068e20 msgr2=0x7f41540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 -- 172.21.15.170:0/3791643938 shutdown_connections 2023-10-08T16:11:33.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.072+0000 7f4159032700 1 -- 172.21.15.170:0/3791643938 wait complete. 2023-10-08T16:11:33.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.073+0000 7f4159032700 1 Processor -- start 2023-10-08T16:11:33.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.073+0000 7f4159032700 1 -- start start 2023-10-08T16:11:33.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f41540cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4154159c60 0x7f41540c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cd960 con 0x7f4154069560 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cdc40 con 0x7f4154159c60 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4159032700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cdf20 con 0x7f4154159890 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47108/0 (socket says 172.21.15.170:47108) 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 -- 172.21.15.170:0/760361098 learned_addr learned my addr 172.21.15.170:0/760361098 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:33.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f415359e700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4154159c60 0x7f41540c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f414a59c700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f41540cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 -- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 msgr2=0x7f41540cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f41540cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 -- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4154159c60 msgr2=0x7f41540c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4154159c60 0x7f41540c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 -- 172.21.15.170:0/760361098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f414c009a20 con 0x7f4154069560 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f414a59c700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f41540cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.074+0000 7f4152d9d700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4144009290 tx=0x7f4144017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.075+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414400bb60 con 0x7f4154069560 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.075+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144000c90 con 0x7f4154069560 2023-10-08T16:11:33.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.075+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41440094f0 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.075+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414401a660 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.075+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41540cd600 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.076+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415404c3c0 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.076+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f414401aa90 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.076+0000 7f414b7fe700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 0x7f41340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.076+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f4144086970 con 0x7f4154069560 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.076+0000 7f414a59c700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 0x7f41340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.077+0000 7f414a59c700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 0x7f41340697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41540c7120 tx=0x7f413c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.081+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4144053610 con 0x7f4154069560 2023-10-08T16:11:33.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.304+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41540ce900 con 0x7f4154069560 2023-10-08T16:11:33.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.304+0000 7f414b7fe700 1 -- 172.21.15.170:0/760361098 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f4144056c30 con 0x7f4154069560 2023-10-08T16:11:33.305 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:33.305 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/309861718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":309861718},{"type":"v1","addr":"172.21.15.170:6837","nonce":309861718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1899300397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1899300397},{"type":"v1","addr":"172.21.15.170:6836","nonce":1899300397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 msgr2=0x7f41340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 0x7f41340697d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f41540c7120 tx=0x7f413c009040 comp rx=0 tx=0).stop 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 msgr2=0x7f4154153770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4144009290 tx=0x7f4144017000 comp rx=0 tx=0).stop 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 shutdown_connections 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4154159c60 0x7f41540c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4134067320 0x7f41340697d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4154069560 0x7f4154153770 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 --2- 172.21.15.170:0/760361098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4154159890 0x7f41540cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 >> 172.21.15.170:0/760361098 conn(0x7f4154068e20 msgr2=0x7f41540b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 shutdown_connections 2023-10-08T16:11:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.306+0000 7f4159032700 1 -- 172.21.15.170:0/760361098 wait complete. 2023-10-08T16:11:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 83 2023-10-08T16:11:33.319 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.432+0000 7f89f6ab4700 1 Processor -- start 2023-10-08T16:11:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.432+0000 7f89f6ab4700 1 -- start start 2023-10-08T16:11:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f6ab4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89f006a1c0 0x7f89f01615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f6ab4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f00b00b0 con 0x7f89f00cb760 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f6ab4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f00acdb0 con 0x7f89f00cbb30 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f6ab4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00c8a10 con 0x7f89f006a1c0 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f4850700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89f00cb760 0x7f89f00c8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58756/0 (socket says 172.21.15.170:58756) 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.433+0000 7f89f4850700 1 -- 172.21.15.170:0/2295113805 learned_addr learned my addr 172.21.15.170:0/2295113805 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f5051700 1 --2- 172.21.15.170:0/2295113805 >> v2:172.21.15.191:3300/0 conn(0x7f89f006a1c0 0x7f89f01615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743423587 0 0) 0x7f89f00b00b0 con 0x7f89f00cb760 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d8003570 con 0x7f89f00cb760 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 949180677 0 0) 0x7f89d8003570 con 0x7f89f00cb760 2023-10-08T16:11:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 >> v2:172.21.15.191:3300/0 conn(0x7f89f006a1c0 msgr2=0x7f89f01615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 --2- 172.21.15.170:0/2295113805 >> v2:172.21.15.191:3300/0 conn(0x7f89f006a1c0 0x7f89f01615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 >> v1:172.21.15.191:6789/0 conn(0x7f89f00cbb30 legacy=0x7f89f0069ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00ad000 con 0x7f89f00cb760 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f89ec0017f0 con 0x7f89f00cb760 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f5051700 1 --2- 172.21.15.170:0/2295113805 >> v2:172.21.15.191:3300/0 conn(0x7f89f006a1c0 0x7f89f01615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 >> v1:172.21.15.170:6789/0 conn(0x7f89f00cb760 legacy=0x7f89f00c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 0x7f89d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8004810 0x7f89d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d80053c0 0x7f89d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c8a10 con 0x7f89d8003a60 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d800c2d0 con 0x7f89d8004810 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d800c490 con 0x7f89d80053c0 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 0x7f89d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89e7fff700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8004810 0x7f89d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f5051700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d80053c0 0x7f89d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 -- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d80053c0 msgr2=0x7f89d800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89d80053c0 0x7f89d800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 -- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8004810 msgr2=0x7f89d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8004810 0x7f89d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 -- 172.21.15.170:0/2295113805 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d8003570 con 0x7f89d8003a60 2023-10-08T16:11:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f5051700 1 -- 172.21.15.170:0/2295113805 reap_dead start 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.434+0000 7f89f4850700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 0x7f89d8003e70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00a900 tx=0x7f89f00b22b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67c2f318c0f654e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0070d0 con 0x7f89d8003a60 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec007230 con 0x7f89d8003a60 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89e77fe700 1 -- 172.21.15.170:0/2295113805 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0075f0 con 0x7f89d8003a60 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 -- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 msgr2=0x7f89d8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 0x7f89d8003e70 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00a900 tx=0x7f89f00b22b0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 -- 172.21.15.170:0/2295113805 shutdown_connections 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/2295113805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89d8003a60 0x7f89d8003e70 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 -- 172.21.15.170:0/2295113805 >> 172.21.15.170:0/2295113805 conn(0x7f89f006d5a0 msgr2=0x7f89f0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 -- 172.21.15.170:0/2295113805 shutdown_connections 2023-10-08T16:11:33.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.435+0000 7f89f6ab4700 1 -- 172.21.15.170:0/2295113805 wait complete. 2023-10-08T16:11:33.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.436+0000 7f89f6ab4700 1 Processor -- start 2023-10-08T16:11:33.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.436+0000 7f89f6ab4700 1 -- start start 2023-10-08T16:11:33.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8003a60 0x7f89f0169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f00c17f0 0x7f89f00c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c1080 con 0x7f89f0097510 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c1360 con 0x7f89d8003a60 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f6ab4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f016a190 con 0x7f89f00c17f0 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47126/0 (socket says 172.21.15.170:47126) 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 -- 172.21.15.170:0/3318378583 learned_addr learned my addr 172.21.15.170:0/3318378583 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f4850700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8003a60 0x7f89f0169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89f5051700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f00c17f0 0x7f89f00c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 -- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f00c17f0 msgr2=0x7f89f00c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f00c17f0 0x7f89f00c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 -- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8003a60 msgr2=0x7f89f0169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8003a60 0x7f89f0169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.437+0000 7f89e7fff700 1 -- 172.21.15.170:0/3318378583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f016a470 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89e7fff700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f89e0002950 tx=0x7f89e0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000f670 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000f7d0 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f016a750 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000fb90 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.438+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00abca0 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.439+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f004c3c0 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.439+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89e001c400 con 0x7f89f0097510 2023-10-08T16:11:33.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.439+0000 7f89e4ff9700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 0x7f89dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.440+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f89e0085160 con 0x7f89f0097510 2023-10-08T16:11:33.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.440+0000 7f89f4850700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 0x7f89dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.440+0000 7f89f4850700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 0x7f89dc0697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00a3f0 tx=0x7f89ec015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.444+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e0051df0 con 0x7f89f0097510 2023-10-08T16:11:33.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.630+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89f00c14a0 con 0x7f89f0097510 2023-10-08T16:11:33.631 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.631+0000 7f89e4ff9700 1 -- 172.21.15.170:0/3318378583 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15673 (secure 0 0 0) 0x7f89e0055410 con 0x7f89f0097510 2023-10-08T16:11:33.632 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:33.632 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":127,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:32.704282+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6834/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6837/707523029":"2023-10-09T16:11:26.301707+0000","172.21.15.170:6835/707523029":"2023-10-09T16:11:26.301707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 msgr2=0x7f89dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 0x7f89dc0697d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00a3f0 tx=0x7f89ec015000 comp rx=0 tx=0).stop 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 msgr2=0x7f89f0097980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f89e0002950 tx=0x7f89e0007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 shutdown_connections 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89f00c17f0 0x7f89f00c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89d8003a60 0x7f89f0169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89dc067320 0x7f89dc0697d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 --2- 172.21.15.170:0/3318378583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0097510 0x7f89f0097980 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 >> 172.21.15.170:0/3318378583 conn(0x7f89f006d5a0 msgr2=0x7f89f00ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 shutdown_connections 2023-10-08T16:11:33.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.633+0000 7f89f6ab4700 1 -- 172.21.15.170:0/3318378583 wait complete. 2023-10-08T16:11:33.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:11:02.613155+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:11:03.244744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:11:33.646 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:11:33.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.764+0000 7ff6ad4a6700 1 Processor -- start 2023-10-08T16:11:33.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.764+0000 7ff6ad4a6700 1 -- start start 2023-10-08T16:11:33.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 0x7ff6a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 0x7ff6a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6a815db10 0x7ff6a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.767 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a815a160 con 0x7ff6a815db10 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a80d13a0 con 0x7ff6a8159c60 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6ad4a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a815e140 con 0x7ff6a8159890 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6a77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6a815db10 0x7ff6a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6a77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6a815db10 0x7ff6a8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47142/0 (socket says 172.21.15.170:47142) 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.765+0000 7ff6a77fe700 1 -- 172.21.15.170:0/2848015528 learned_addr learned my addr 172.21.15.170:0/2848015528 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 0x7ff6a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff69e7fc700 1 --2- 172.21.15.170:0/2848015528 >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 0x7ff6a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 -- 172.21.15.170:0/2848015528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 msgr2=0x7ff6a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 0x7ff6a815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 -- 172.21.15.170:0/2848015528 >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 msgr2=0x7ff6a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 --2- 172.21.15.170:0/2848015528 >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 0x7ff6a815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 -- 172.21.15.170:0/2848015528 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a815a3d0 con 0x7ff6a815db10 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 0x7ff6a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff69e7fc700 1 --2- 172.21.15.170:0/2848015528 >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 0x7ff6a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a77fe700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a815db10 0x7ff6a8069560 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80650f0 tx=0x7ff6a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c4beed4495d0ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a67fc700 1 -- 172.21.15.170:0/2848015528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a000e070 con 0x7ff6a815db10 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a67fc700 1 -- 172.21.15.170:0/2848015528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a000ba50 con 0x7ff6a815db10 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.766+0000 7ff6a67fc700 1 -- 172.21.15.170:0/2848015528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00092e0 con 0x7ff6a815db10 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2848015528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a815db10 msgr2=0x7ff6a8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a815db10 0x7ff6a8069560 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff6a80650f0 tx=0x7ff6a0009840 comp rx=0 tx=0).stop 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2848015528 shutdown_connections 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a815db10 0x7ff6a8069560 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2848015528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a8159890 0x7ff6a815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2848015528 >> v2:172.21.15.191:3300/0 conn(0x7ff6a8159c60 0x7ff6a815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2848015528 >> 172.21.15.170:0/2848015528 conn(0x7ff6a8068e20 msgr2=0x7ff6a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2848015528 shutdown_connections 2023-10-08T16:11:33.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.767+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2848015528 wait complete. 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.768+0000 7ff6ad4a6700 1 Processor -- start 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.768+0000 7ff6ad4a6700 1 -- start start 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 0x7ff6a80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 0x7ff6a80cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b1410 con 0x7ff6a8159c60 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a8144790 con 0x7ff6a8159890 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a80b1600 con 0x7ff6a815db10 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47144/0 (socket says 172.21.15.170:47144) 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 -- 172.21.15.170:0/2972292848 learned_addr learned my addr 172.21.15.170:0/2972292848 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6a77fe700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 0x7ff6a80cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 0x7ff6a80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 -- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 msgr2=0x7ff6a80cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 0x7ff6a80cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 -- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 msgr2=0x7ff6a80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 0x7ff6a80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 -- 172.21.15.170:0/2972292848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a0009ae0 con 0x7ff6a8159c60 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 0x7ff6a80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6a77fe700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 0x7ff6a80cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69e7fc700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c8b0 tx=0x7ff690008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690011600 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff690011760 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a80b18e0 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690011b20 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.769+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a80b1ea0 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.770+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a804c3c0 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.771+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff6900073d0 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.771+0000 7ff69f7fe700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 0x7ff6880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.771+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7ff690084e00 con 0x7ff6a8159c60 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.771+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 0x7ff6880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.771+0000 7ff6a6ffd700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 0x7ff6880697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ab50 tx=0x7ff698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.775+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6900518d0 con 0x7ff6a8159c60 2023-10-08T16:11:33.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.960+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6a806a5b0 con 0x7ff6a8159c60 2023-10-08T16:11:33.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.960+0000 7ff69f7fe700 1 -- 172.21.15.170:0/2972292848 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6900150a0 con 0x7ff6a8159c60 2023-10-08T16:11:33.961 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 msgr2=0x7ff6880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 0x7ff6880697d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff69800ab50 tx=0x7ff698008040 comp rx=0 tx=0).stop 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 msgr2=0x7ff6a80cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c8b0 tx=0x7ff690008040 comp rx=0 tx=0).stop 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 shutdown_connections 2023-10-08T16:11:33.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6a815db10 0x7ff6a80cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6a8159890 0x7ff6a80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff688067320 0x7ff6880697d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 --2- 172.21.15.170:0/2972292848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6a8159c60 0x7ff6a80cd700 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 >> 172.21.15.170:0/2972292848 conn(0x7ff6a8068e20 msgr2=0x7ff6a81552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 shutdown_connections 2023-10-08T16:11:33.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:33.962+0000 7ff6ad4a6700 1 -- 172.21.15.170:0/2972292848 wait complete. 2023-10-08T16:11:33.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:11:34.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.091+0000 7ff8254e7700 1 Processor -- start 2023-10-08T16:11:34.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.091+0000 7ff8254e7700 1 -- start start 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff8254e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8200b4e70 0x7ff8200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff8254e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff82015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff8254e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8200cfb00 con 0x7ff8200b16d0 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff8254e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200b5c40 con 0x7ff8200b4e70 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff8254e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200b5f20 con 0x7ff8200b1b40 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8200b4e70 0x7ff8200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8200b4e70 0x7ff8200ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47156/0 (socket says 172.21.15.170:47156) 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 -- 172.21.15.170:0/368537479 learned_addr learned my addr 172.21.15.170:0/368537479 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:34.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff824ce6700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff82015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 -- 172.21.15.170:0/368537479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 msgr2=0x7ff82015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff82015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 -- 172.21.15.170:0/368537479 >> v1:172.21.15.191:6789/0 conn(0x7ff8200b16d0 legacy=0x7ff82015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.092+0000 7ff81ffff700 1 -- 172.21.15.170:0/368537479 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200b6200 con 0x7ff8200b4e70 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff81ffff700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff8200ccff0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff8100099b0 tx=0x7ff810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17cbf4e23f6f9cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff81effd700 1 -- 172.21.15.170:0/368537479 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000b820 con 0x7ff8200b4e70 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff81effd700 1 -- 172.21.15.170:0/368537479 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000b980 con 0x7ff8200b4e70 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff81effd700 1 -- 172.21.15.170:0/368537479 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000bd20 con 0x7ff8200b4e70 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 -- 172.21.15.170:0/368537479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 msgr2=0x7ff8200ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff8200ccff0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff8100099b0 tx=0x7ff810009250 comp rx=0 tx=0).stop 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 -- 172.21.15.170:0/368537479 shutdown_connections 2023-10-08T16:11:34.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff82015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 --2- 172.21.15.170:0/368537479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff8200ccff0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 -- 172.21.15.170:0/368537479 >> 172.21.15.170:0/368537479 conn(0x7ff8200d10c0 msgr2=0x7ff8200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 -- 172.21.15.170:0/368537479 shutdown_connections 2023-10-08T16:11:34.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.093+0000 7ff8254e7700 1 -- 172.21.15.170:0/368537479 wait complete. 2023-10-08T16:11:34.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.094+0000 7ff8254e7700 1 Processor -- start 2023-10-08T16:11:34.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 -- start start 2023-10-08T16:11:34.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8200b16d0 0x7ff8200cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff8200c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200cc310 con 0x7ff8200b4e70 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82016b180 con 0x7ff8200b16d0 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff8254e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200c90e0 con 0x7ff8200b1b40 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff824ce6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff824ce6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47170/0 (socket says 172.21.15.170:47170) 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.095+0000 7ff824ce6700 1 -- 172.21.15.170:0/3876497832 learned_addr learned my addr 172.21.15.170:0/3876497832 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff81ffff700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8200b16d0 0x7ff8200cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 -- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 msgr2=0x7ff8200c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff81f7fe700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff8200c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff8200c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 -- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8200b16d0 msgr2=0x7ff8200cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8200b16d0 0x7ff8200cbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 -- 172.21.15.170:0/3876497832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100094f0 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff824ce6700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ceb0 tx=0x7ff818007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800ae40 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff818016430 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820147970 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8180167d0 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.096+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8200c8b20 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.097+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82016b2f0 con 0x7ff8200b4e70 2023-10-08T16:11:34.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.097+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff818003020 con 0x7ff8200b4e70 2023-10-08T16:11:34.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.098+0000 7ff807fff700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 0x7ff80c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.098+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7ff818012070 con 0x7ff8200b4e70 2023-10-08T16:11:34.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.098+0000 7ff81ffff700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 0x7ff80c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.098+0000 7ff81ffff700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 0x7ff80c0697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff8100099b0 tx=0x7ff810016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.102+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8180526a0 con 0x7ff8200b4e70 2023-10-08T16:11:34.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.284+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff820147970 con 0x7ff8200b4e70 2023-10-08T16:11:34.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.285+0000 7ff807fff700 1 -- 172.21.15.170:0/3876497832 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff820147970 con 0x7ff8200b4e70 2023-10-08T16:11:34.285 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:11:34.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 msgr2=0x7ff80c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 0x7ff80c0697d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff8100099b0 tx=0x7ff810016000 comp rx=0 tx=0).stop 2023-10-08T16:11:34.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 msgr2=0x7ff82016d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ceb0 tx=0x7ff818007870 comp rx=0 tx=0).stop 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 shutdown_connections 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff80c067320 0x7ff80c0697d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8200b4e70 0x7ff82016d420 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff8200b16d0 0x7ff8200cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 --2- 172.21.15.170:0/3876497832 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff8200b1b40 0x7ff8200c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 >> 172.21.15.170:0/3876497832 conn(0x7ff8200d10c0 msgr2=0x7ff82015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 shutdown_connections 2023-10-08T16:11:34.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.287+0000 7ff8254e7700 1 -- 172.21.15.170:0/3876497832 wait complete. 2023-10-08T16:11:34.299 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:11:34.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.416+0000 7fc7534e4700 1 Processor -- start 2023-10-08T16:11:34.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.416+0000 7fc7534e4700 1 -- start start 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc7534e4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc74c0b16d0 0x7fc74c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc7534e4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc74c0b33b0 0x7fc74c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc7534e4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74c0b2080 con 0x7fc74c0b4e70 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc7534e4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0cfb00 con 0x7fc74c0b33b0 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc7534e4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0b5c40 con 0x7fc74c0b16d0 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc74c0b33b0 0x7fc74c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc74c0b33b0 0x7fc74c0b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47178/0 (socket says 172.21.15.170:47178) 2023-10-08T16:11:34.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 -- 172.21.15.170:0/1624091784 learned_addr learned my addr 172.21.15.170:0/1624091784 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc751ce1700 1 --2- 172.21.15.170:0/1624091784 >> v2:172.21.15.191:3300/0 conn(0x7fc74c0b16d0 0x7fc74c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 -- 172.21.15.170:0/1624091784 >> v2:172.21.15.191:3300/0 conn(0x7fc74c0b16d0 msgr2=0x7fc74c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 --2- 172.21.15.170:0/1624091784 >> v2:172.21.15.191:3300/0 conn(0x7fc74c0b16d0 0x7fc74c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.417+0000 7fc752ce3700 1 -- 172.21.15.170:0/1624091784 >> v1:172.21.15.191:6789/0 conn(0x7fc74c0b4e70 legacy=0x7fc74c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc752ce3700 1 -- 172.21.15.170:0/1624091784 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c0b5f20 con 0x7fc74c0b33b0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc752ce3700 1 --2- 172.21.15.170:0/1624091784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b33b0 0x7fc74c0b37d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc74c089830 tx=0x7fc744009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240bae4d65927187 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7514e0700 1 -- 172.21.15.170:0/1624091784 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400e070 con 0x7fc74c0b33b0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7514e0700 1 -- 172.21.15.170:0/1624091784 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400ba50 con 0x7fc74c0b33b0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7514e0700 1 -- 172.21.15.170:0/1624091784 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440092e0 con 0x7fc74c0b33b0 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 -- 172.21.15.170:0/1624091784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b33b0 msgr2=0x7fc74c0b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1624091784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b33b0 0x7fc74c0b37d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc74c089830 tx=0x7fc744009840 comp rx=0 tx=0).stop 2023-10-08T16:11:34.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 -- 172.21.15.170:0/1624091784 shutdown_connections 2023-10-08T16:11:34.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1624091784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b33b0 0x7fc74c0b37d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1624091784 >> v2:172.21.15.191:3300/0 conn(0x7fc74c0b16d0 0x7fc74c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 -- 172.21.15.170:0/1624091784 >> 172.21.15.170:0/1624091784 conn(0x7fc74c0d10c0 msgr2=0x7fc74c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 -- 172.21.15.170:0/1624091784 shutdown_connections 2023-10-08T16:11:34.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.418+0000 7fc7534e4700 1 -- 172.21.15.170:0/1624091784 wait complete. 2023-10-08T16:11:34.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.419+0000 7fc7534e4700 1 Processor -- start 2023-10-08T16:11:34.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 -- start start 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74c0b16d0 0x7fc74c0af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74c0b33b0 0x7fc74c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c16e0c0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0a83d0 con 0x7fc74c0b16d0 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7534e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0cb5e0 con 0x7fc74c0b33b0 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc752ce3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc752ce3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47184/0 (socket says 172.21.15.170:47184) 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc752ce3700 1 -- 172.21.15.170:0/1003260856 learned_addr learned my addr 172.21.15.170:0/1003260856 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc751ce1700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74c0b33b0 0x7fc74c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.424 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.420+0000 7fc7524e2700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74c0b16d0 0x7fc74c0af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 -- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74c0b33b0 msgr2=0x7fc74c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74c0b33b0 0x7fc74c0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 -- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74c0b16d0 msgr2=0x7fc74c0af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74c0b16d0 0x7fc74c0af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 -- 172.21.15.170:0/1003260856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc744009ae0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc752ce3700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc744009fd0 tx=0x7fc744018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400be80 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744007000 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c0ca9d0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74401d420 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.421+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c0cb270 con 0x7fc74c0b4e70 2023-10-08T16:11:34.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.422+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c0cadc0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.422+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc744003050 con 0x7fc74c0b4e70 2023-10-08T16:11:34.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.423+0000 7fc7427fc700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.423+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fc7440869f0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.423+0000 7fc7524e2700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.423+0000 7fc7524e2700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 0x7fc7300697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc74c068a30 tx=0x7fc748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.427+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7440534c0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.608+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc74c04bb80 con 0x7fc74c0b4e70 2023-10-08T16:11:34.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.609+0000 7fc7427fc700 1 -- 172.21.15.170:0/1003260856 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc744056ae0 con 0x7fc74c0b4e70 2023-10-08T16:11:34.609 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:11:34.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 msgr2=0x7fc7300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 0x7fc7300697d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc74c068a30 tx=0x7fc748009040 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 msgr2=0x7fc74c16d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fc744009fd0 tx=0x7fc744018000 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 shutdown_connections 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc730067320 0x7fc7300697d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74c0b4e70 0x7fc74c16d1b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc74c0b16d0 0x7fc74c0af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 --2- 172.21.15.170:0/1003260856 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74c0b33b0 0x7fc74c0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 >> 172.21.15.170:0/1003260856 conn(0x7fc74c0d10c0 msgr2=0x7fc74c158170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 shutdown_connections 2023-10-08T16:11:34.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.611+0000 7fc7534e4700 1 -- 172.21.15.170:0/1003260856 wait complete. 2023-10-08T16:11:34.623 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:34.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.739+0000 7f450629a700 1 Processor -- start 2023-10-08T16:11:34.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.739+0000 7f450629a700 1 -- start start 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f450629a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 0x7f45000b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f450629a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f450015d230 0x7f450015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f450629a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450015dbd0 con 0x7f45001597f0 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f450629a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45000b2980 con 0x7f450015d230 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f450629a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45000b11d0 con 0x7f4500159420 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f4504a97700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f45001597f0 0x7f45000b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58760/0 (socket says 172.21.15.170:58760) 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.740+0000 7f4504a97700 1 -- 172.21.15.170:0/2309841592 learned_addr learned my addr 172.21.15.170:0/2309841592 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 --2- 172.21.15.170:0/2309841592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 0x7f45000b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505a99700 1 --2- 172.21.15.170:0/2309841592 >> v2:172.21.15.191:3300/0 conn(0x7f450015d230 0x7f450015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f44f7fff700 1 -- 172.21.15.170:0/2309841592 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202000069 0 0) 0x7f44f0001110 con 0x7f45001597f0 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f44f7fff700 1 -- 172.21.15.170:0/2309841592 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ec003570 con 0x7f45001597f0 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 -- 172.21.15.170:0/2309841592 >> v2:172.21.15.191:3300/0 conn(0x7f450015d230 msgr2=0x7f450015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 --2- 172.21.15.170:0/2309841592 >> v2:172.21.15.191:3300/0 conn(0x7f450015d230 0x7f450015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 -- 172.21.15.170:0/2309841592 >> v1:172.21.15.170:6789/0 conn(0x7f45001597f0 legacy=0x7f45000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 -- 172.21.15.170:0/2309841592 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45000b14b0 con 0x7f4500159420 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f4505298700 1 --2- 172.21.15.170:0/2309841592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 0x7f45000b5f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e60 tx=0x7f44fc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b6fcdc124b0e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f44f7fff700 1 -- 172.21.15.170:0/2309841592 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00e070 con 0x7f4500159420 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f44f7fff700 1 -- 172.21.15.170:0/2309841592 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc00bab0 con 0x7f4500159420 2023-10-08T16:11:34.743 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.741+0000 7f44f7fff700 1 -- 172.21.15.170:0/2309841592 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00be30 con 0x7f4500159420 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 -- 172.21.15.170:0/2309841592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 msgr2=0x7f45000b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 --2- 172.21.15.170:0/2309841592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 0x7f45000b5f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009e60 tx=0x7f44fc009780 comp rx=0 tx=0).stop 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 -- 172.21.15.170:0/2309841592 shutdown_connections 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 --2- 172.21.15.170:0/2309841592 >> v2:172.21.15.191:3300/0 conn(0x7f450015d230 0x7f450015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 --2- 172.21.15.170:0/2309841592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4500159420 0x7f45000b5f40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 -- 172.21.15.170:0/2309841592 >> 172.21.15.170:0/2309841592 conn(0x7f45000d0660 msgr2=0x7f45000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 -- 172.21.15.170:0/2309841592 shutdown_connections 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.742+0000 7f450629a700 1 -- 172.21.15.170:0/2309841592 wait complete. 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.743+0000 7f450629a700 1 Processor -- start 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.743+0000 7f450629a700 1 -- start start 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4500159420 0x7f45000c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f450015d230 0x7f4500069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500069d90 con 0x7f45001597f0 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45000c6fd0 con 0x7f4500159420 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f450629a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45000c63c0 con 0x7f450015d230 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47194/0 (socket says 172.21.15.170:47194) 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 -- 172.21.15.170:0/2214034046 learned_addr learned my addr 172.21.15.170:0/2214034046 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4505a99700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f450015d230 0x7f4500069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4505298700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4500159420 0x7f45000c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 -- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f450015d230 msgr2=0x7f4500069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f450015d230 0x7f4500069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 -- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4500159420 msgr2=0x7f45000c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4500159420 0x7f45000c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 -- 172.21.15.170:0/2214034046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009a20 con 0x7f45001597f0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.744+0000 7f4504a97700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c7c0 tx=0x7f44f0007c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.745+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0005b30 con 0x7f45001597f0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.745+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f0005c90 con 0x7f45001597f0 2023-10-08T16:11:34.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.745+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45000c66a0 con 0x7f45001597f0 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.745+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f0007300 con 0x7f45001597f0 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.745+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45000c6c60 con 0x7f45001597f0 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.746+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450004c3c0 con 0x7f45001597f0 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.746+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f44f001b3b0 con 0x7f45001597f0 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.746+0000 7f44f57fa700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 0x7f44e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:34.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.746+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f44f0083ff0 con 0x7f45001597f0 2023-10-08T16:11:34.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.746+0000 7f4505298700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 0x7f44e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:34.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.747+0000 7f4505298700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 0x7f44e40697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:34.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.750+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44f0050bd0 con 0x7f45001597f0 2023-10-08T16:11:34.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.974+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f450015dbd0 con 0x7f45001597f0 2023-10-08T16:11:34.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.975+0000 7f44f57fa700 1 -- 172.21.15.170:0/2214034046 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f450015dbd0 con 0x7f45001597f0 2023-10-08T16:11:34.975 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:34.975 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/309861718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":309861718},{"type":"v1","addr":"172.21.15.170:6837","nonce":309861718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1899300397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1899300397},{"type":"v1","addr":"172.21.15.170:6836","nonce":1899300397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 msgr2=0x7f44e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 0x7f44e40697d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0097c0 tx=0x7f44fc019000 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 msgr2=0x7f45000c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f44f000c7c0 tx=0x7f44f0007c30 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 shutdown_connections 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f450015d230 0x7f4500069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4500159420 0x7f45000c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f44e4067320 0x7f44e40697d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 --2- 172.21.15.170:0/2214034046 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45001597f0 0x7f45000c5cb0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:34.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 >> 172.21.15.170:0/2214034046 conn(0x7f45000d0660 msgr2=0x7f45000b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 shutdown_connections 2023-10-08T16:11:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:34.977+0000 7f450629a700 1 -- 172.21.15.170:0/2214034046 wait complete. 2023-10-08T16:11:34.979 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 83 2023-10-08T16:11:34.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:35.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.104+0000 7f83e0edb700 1 Processor -- start 2023-10-08T16:11:35.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.104+0000 7f83e0edb700 1 -- start start 2023-10-08T16:11:35.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83e0edb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83dc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83e0edb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f83dc15d230 0x7f83dc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83e0edb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc15dbd0 con 0x7f83dc1597f0 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83e0edb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc0b2980 con 0x7f83dc15d230 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83e0edb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b11d0 con 0x7f83dc159420 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83daffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f83dc1597f0 0x7f83dc0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58774/0 (socket says 172.21.15.170:58774) 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83daffd700 1 -- 172.21.15.170:0/715088831 learned_addr learned my addr 172.21.15.170:0/715088831 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434030413 0 0) 0x7f83d0001110 con 0x7f83dc1597f0 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83db7fe700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83dc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c4003570 con 0x7f83dc1597f0 2023-10-08T16:11:35.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83dbfff700 1 --2- 172.21.15.170:0/715088831 >> v2:172.21.15.191:3300/0 conn(0x7f83dc15d230 0x7f83dc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 232837738 0 0) 0x7f83c4003570 con 0x7f83dc1597f0 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 msgr2=0x7f83dc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83dc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 >> v2:172.21.15.191:3300/0 conn(0x7f83dc15d230 msgr2=0x7f83dc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 --2- 172.21.15.170:0/715088831 >> v2:172.21.15.191:3300/0 conn(0x7f83dc15d230 0x7f83dc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.105+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0b14b0 con 0x7f83dc1597f0 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f83d0001600 con 0x7f83dc1597f0 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 >> v1:172.21.15.170:6789/0 conn(0x7f83dc1597f0 legacy=0x7f83dc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83c4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83c4003fe0 0x7f83c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4004d50 0x7f83c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b2980 con 0x7f83c4004d50 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc0b11d0 con 0x7f83c4003fe0 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c400ba90 con 0x7f83dc159420 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4004d50 0x7f83c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83db7fe700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83c4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83daffd700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83c4003fe0 0x7f83c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 -- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 msgr2=0x7f83c4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc159420 0x7f83c4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 -- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83c4003fe0 msgr2=0x7f83c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83c4003fe0 0x7f83c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 -- 172.21.15.170:0/715088831 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4003570 con 0x7f83c4004d50 2023-10-08T16:11:35.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83dbfff700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4004d50 0x7f83c40051c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009c00 tx=0x7f83d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e7e4a109083bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d400e6b0 con 0x7f83c4004d50 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d400e810 con 0x7f83c4004d50 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.106+0000 7f83da7fc700 1 -- 172.21.15.170:0/715088831 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d400ebd0 con 0x7f83c4004d50 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 -- 172.21.15.170:0/715088831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4004d50 msgr2=0x7f83c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 --2- 172.21.15.170:0/715088831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83c4004d50 0x7f83c40051c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009c00 tx=0x7f83d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83dbfff700 1 -- 172.21.15.170:0/715088831 reap_dead start 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 -- 172.21.15.170:0/715088831 shutdown_connections 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 -- 172.21.15.170:0/715088831 >> 172.21.15.170:0/715088831 conn(0x7f83dc0d0660 msgr2=0x7f83dc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 -- 172.21.15.170:0/715088831 shutdown_connections 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.107+0000 7f83e0edb700 1 -- 172.21.15.170:0/715088831 wait complete. 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.108+0000 7f83e0edb700 1 Processor -- start 2023-10-08T16:11:35.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.108+0000 7f83e0edb700 1 -- start start 2023-10-08T16:11:35.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 0x7f83dc153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 0x7f83dc155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc15c3b0 con 0x7f83dc15b2e0 2023-10-08T16:11:35.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc15b7a0 con 0x7f83dc14c970 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83e0edb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc15ba80 con 0x7f83dc14c5a0 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47214/0 (socket says 172.21.15.170:47214) 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 -- 172.21.15.170:0/2173667344 learned_addr learned my addr 172.21.15.170:0/2173667344 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83dbfff700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 0x7f83dc155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83db7fe700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 0x7f83dc153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 -- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 msgr2=0x7f83dc153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 0x7f83dc153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 -- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 msgr2=0x7f83dc155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 0x7f83dc155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 -- 172.21.15.170:0/2173667344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc15bd60 con 0x7f83dc15b2e0 2023-10-08T16:11:35.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83db7fe700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 0x7f83dc153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83dbfff700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 0x7f83dc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83daffd700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000fd0 tx=0x7f83d0007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.109+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d000fc60 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.110+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d000fdc0 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.110+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc15c040 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.110+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d0013620 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.110+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc15c840 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.110+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc15c520 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.111+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f83d0013990 con 0x7f83dc15b2e0 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.111+0000 7f83c37fe700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 0x7f83c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.111+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f83d0084fa0 con 0x7f83dc15b2e0 2023-10-08T16:11:35.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.111+0000 7f83db7fe700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 0x7f83c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.112+0000 7f83db7fe700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 0x7f83c80697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009fd0 tx=0x7f83cc00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.116+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83d0051c90 con 0x7f83dc15b2e0 2023-10-08T16:11:35.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.300+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83dc15dbd0 con 0x7f83dc15b2e0 2023-10-08T16:11:35.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.301+0000 7f83c37fe700 1 -- 172.21.15.170:0/2173667344 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15673 (secure 0 0 0) 0x7f83dc15dbd0 con 0x7f83dc15b2e0 2023-10-08T16:11:35.302 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:35.303 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":127,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:32.704282+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6834/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6837/707523029":"2023-10-09T16:11:26.301707+0000","172.21.15.170:6835/707523029":"2023-10-09T16:11:26.301707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:35.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 msgr2=0x7f83c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 0x7f83c80697d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009fd0 tx=0x7f83cc00b040 comp rx=0 tx=0).stop 2023-10-08T16:11:35.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 msgr2=0x7f83dc1557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f83d0000fd0 tx=0x7f83d0007a80 comp rx=0 tx=0).stop 2023-10-08T16:11:35.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 shutdown_connections 2023-10-08T16:11:35.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83dc14c970 0x7f83dc155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83dc14c5a0 0x7f83dc153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f83c8067320 0x7f83c80697d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 --2- 172.21.15.170:0/2173667344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83dc15b2e0 0x7f83dc1557d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 >> 172.21.15.170:0/2173667344 conn(0x7f83dc0d0660 msgr2=0x7f83dc14d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:35.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 shutdown_connections 2023-10-08T16:11:35.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.304+0000 7f83e0edb700 1 -- 172.21.15.170:0/2173667344 wait complete. 2023-10-08T16:11:35.317 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:11:35.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:35.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.430+0000 7f6d93c8e700 1 Processor -- start 2023-10-08T16:11:35.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.430+0000 7f6d93c8e700 1 -- start start 2023-10-08T16:11:35.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d93c8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c069560 0x7f6d8c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d93c8e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c069ee0 con 0x7f6d8c159890 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d93c8e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c15a160 con 0x7f6d8c159c60 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d93c8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0d13a0 con 0x7f6d8c069560 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d91a2a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6d8c159890 0x7f6d8c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57976/0 (socket says 172.21.15.170:57976) 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d91a2a700 1 -- 172.21.15.170:0/2716593762 learned_addr learned my addr 172.21.15.170:0/2716593762 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d9222b700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c069560 0x7f6d8c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.431+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880150675 0 0) 0x7f6d8c069ee0 con 0x7f6d8c159890 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d78003570 con 0x7f6d8c159890 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3603768731 0 0) 0x7f6d78003570 con 0x7f6d8c159890 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c069560 msgr2=0x7f6d8c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c069560 0x7f6d8c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 >> v1:172.21.15.191:6789/0 conn(0x7f6d8c159c60 legacy=0x7f6d8c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c15e140 con 0x7f6d8c159890 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6d88001470 con 0x7f6d8c159890 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 >> v1:172.21.15.170:6789/0 conn(0x7f6d8c159890 legacy=0x7f6d8c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d78003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0d13a0 con 0x7f6d78003a90 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7800c050 con 0x7f6d78004810 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7800c1e0 con 0x7f6d780051c0 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d9222b700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c069560 0x7f6d8c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:35.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d78003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91229700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d9222b700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 -- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d780051c0 msgr2=0x7f6d7800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d780051c0 0x7f6d7800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 -- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d78004810 msgr2=0x7f6d78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 -- 172.21.15.170:0/2716593762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78003570 con 0x7f6d78003a90 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91229700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d78004810 0x7f6d78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d78003ea0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c3c0 tx=0x7f6d88001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e4edc9b1dd2987 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.432+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800cc90 con 0x7f6d78003a90 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8800cdf0 con 0x7f6d78003a90 2023-10-08T16:11:35.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d90a28700 1 -- 172.21.15.170:0/2716593762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88005b50 con 0x7f6d78003a90 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 -- 172.21.15.170:0/2716593762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 msgr2=0x7f6d78003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/2716593762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d78003ea0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800c3c0 tx=0x7f6d88001e50 comp rx=0 tx=0).stop 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d9222b700 1 -- 172.21.15.170:0/2716593762 reap_dead start 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 -- 172.21.15.170:0/2716593762 shutdown_connections 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 -- 172.21.15.170:0/2716593762 >> 172.21.15.170:0/2716593762 conn(0x7f6d8c068e20 msgr2=0x7f6d8c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 -- 172.21.15.170:0/2716593762 shutdown_connections 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.433+0000 7f6d93c8e700 1 -- 172.21.15.170:0/2716593762 wait complete. 2023-10-08T16:11:35.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.434+0000 7f6d93c8e700 1 Processor -- start 2023-10-08T16:11:35.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.434+0000 7f6d93c8e700 1 -- start start 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c152e00 0x7f6d8c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 0x7f6d8c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c16d940 con 0x7f6d78003a90 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c14bb90 con 0x7f6d8c151940 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d93c8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c16db50 con 0x7f6d8c152e00 2023-10-08T16:11:35.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51674/0 (socket says 172.21.15.170:51674) 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 -- 172.21.15.170:0/1068091520 learned_addr learned my addr 172.21.15.170:0/1068091520 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91229700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c152e00 0x7f6d8c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d9222b700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 0x7f6d8c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 -- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c152e00 msgr2=0x7f6d8c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c152e00 0x7f6d8c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 -- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 msgr2=0x7f6d8c151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 0x7f6d8c151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 -- 172.21.15.170:0/1068091520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c16de30 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d9222b700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 0x7f6d8c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.435+0000 7f6d91a2a700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6d88007cd0 tx=0x7f6d88015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.436+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800cc90 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.436+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88000ed0 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.436+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c16e110 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.436+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88007530 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.436+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c15c7d0 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.437+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c151280 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.437+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6d88019070 con 0x7f6d78003a90 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.437+0000 7f6d81ffb700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.443 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.438+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f6d88086150 con 0x7f6d78003a90 2023-10-08T16:11:35.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.438+0000 7f6d91229700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.438+0000 7f6d91229700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 0x7f6d700697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0b39f0 tx=0x7f6d7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.442+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d8c151280 con 0x7f6d78003a90 2023-10-08T16:11:35.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.663+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d8c151280 con 0x7f6d78003a90 2023-10-08T16:11:35.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.664+0000 7f6d81ffb700 1 -- 172.21.15.170:0/1068091520 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f6d8c151280 con 0x7f6d78003a90 2023-10-08T16:11:35.664 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:35.665 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/309861718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":309861718},{"type":"v1","addr":"172.21.15.170:6837","nonce":309861718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1899300397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1899300397},{"type":"v1","addr":"172.21.15.170:6836","nonce":1899300397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 msgr2=0x7f6d700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 0x7f6d700697d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c0b39f0 tx=0x7f6d7c009040 comp rx=0 tx=0).stop 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 msgr2=0x7f6d8c16d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f6d88007cd0 tx=0x7f6d88015000 comp rx=0 tx=0).stop 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 shutdown_connections 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6d8c151940 0x7f6d8c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6d70067320 0x7f6d700697d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d78003a90 0x7f6d8c16d400 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 --2- 172.21.15.170:0/1068091520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6d8c152e00 0x7f6d8c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 >> 172.21.15.170:0/1068091520 conn(0x7f6d8c068e20 msgr2=0x7f6d8c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 shutdown_connections 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.666+0000 7f6d93c8e700 1 -- 172.21.15.170:0/1068091520 wait complete. 2023-10-08T16:11:35.668 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 83 2023-10-08T16:11:35.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:35.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.793+0000 7f8cc693d700 1 Processor -- start 2023-10-08T16:11:35.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.793+0000 7f8cc693d700 1 -- start start 2023-10-08T16:11:35.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc693d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc0155c10 0x7f8cc0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc693d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 0x7f8cc00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc693d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc00acac0 con 0x7f8cc00ac230 2023-10-08T16:11:35.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc693d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00ad930 con 0x7f8cc0155c10 2023-10-08T16:11:35.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc693d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00acd20 con 0x7f8cc00ac6d0 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cbffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cc0155c10 0x7f8cc0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cb7fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8cc00ac230 0x7f8cc01568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57982/0 (socket says 172.21.15.170:57982) 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cb7fff700 1 -- 172.21.15.170:0/3299007350 learned_addr learned my addr 172.21.15.170:0/3299007350 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cc4eda700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 0x7f8cc00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.794+0000 7f8cbffff700 1 -- 172.21.15.170:0/3299007350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 msgr2=0x7f8cc00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbffff700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 0x7f8cc00b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbffff700 1 -- 172.21.15.170:0/3299007350 >> v1:172.21.15.170:6789/0 conn(0x7f8cc00ac230 legacy=0x7f8cc01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3299007350 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452429296 0 0) 0x7f8cc00acac0 con 0x7f8cc00ac230 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbffff700 1 -- 172.21.15.170:0/3299007350 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00ad000 con 0x7f8cc0155c10 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc4eda700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 0x7f8cc00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:35.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbffff700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc0155fe0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00acc50 tx=0x7f8cb000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ac678a5cd4e724 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3299007350 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000c970 con 0x7f8cc0155c10 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3299007350 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb000cad0 con 0x7f8cc0155c10 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cbf7fe700 1 -- 172.21.15.170:0/3299007350 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000ce90 con 0x7f8cc0155c10 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 -- 172.21.15.170:0/3299007350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 msgr2=0x7f8cc0155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc0155fe0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00acc50 tx=0x7f8cb000a330 comp rx=0 tx=0).stop 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 -- 172.21.15.170:0/3299007350 shutdown_connections 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac6d0 0x7f8cc00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 --2- 172.21.15.170:0/3299007350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc0155fe0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.795+0000 7f8cc693d700 1 -- 172.21.15.170:0/3299007350 >> 172.21.15.170:0/3299007350 conn(0x7f8cc006a320 msgr2=0x7f8cc00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.796+0000 7f8cc693d700 1 -- 172.21.15.170:0/3299007350 shutdown_connections 2023-10-08T16:11:35.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.796+0000 7f8cc693d700 1 -- 172.21.15.170:0/3299007350 wait complete. 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 Processor -- start 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 -- start start 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac230 0x7f8cc013aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 0x7f8cc0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc013b020 con 0x7f8cc0155c10 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0152520 con 0x7f8cc00ac6d0 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc693d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0151b50 con 0x7f8cc00ac230 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc4eda700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc4eda700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51704/0 (socket says 172.21.15.170:51704) 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.797+0000 7f8cc4eda700 1 -- 172.21.15.170:0/4146516672 learned_addr learned my addr 172.21.15.170:0/4146516672 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cbffff700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac230 0x7f8cc013aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cb7fff700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 0x7f8cc0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 -- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac230 msgr2=0x7f8cc013aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac230 0x7f8cc013aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 -- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 msgr2=0x7f8cc0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:35.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 0x7f8cc0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 -- 172.21.15.170:0/4146516672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb000a5d0 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cb7fff700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 0x7f8cc0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc4eda700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800d180 tx=0x7f8cb8009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ddc0 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800f040 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc0150f40 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8015690 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.798+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc0151500 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.799+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc004c3c0 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.800+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8cb801e3b0 con 0x7f8cc0155c10 2023-10-08T16:11:35.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.800+0000 7f8cbcff9700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 0x7f8ca00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.800+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f8cb80853e0 con 0x7f8cc0155c10 2023-10-08T16:11:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.800+0000 7f8cbffff700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 0x7f8ca00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.800+0000 7f8cbffff700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 0x7f8ca00697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000a1c0 tx=0x7f8cb0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:35.804+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8cb8051eb0 con 0x7f8cc0155c10 2023-10-08T16:11:36.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.031+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cc00ad2e0 con 0x7f8cc0155c10 2023-10-08T16:11:36.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.031+0000 7f8cbcff9700 1 -- 172.21.15.170:0/4146516672 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4599 (secure 0 0 0) 0x7f8cb80554d0 con 0x7f8cc0155c10 2023-10-08T16:11:36.032 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:36.032 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":83,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/309861718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":309861718},{"type":"v1","addr":"172.21.15.170:6837","nonce":309861718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1899300397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1899300397},{"type":"v1","addr":"172.21.15.170:6836","nonce":1899300397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:04.275853+0000","modified":"2023-10-08T16:11:07.928111+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7125},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7125":{"gid":7125,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6834/47051258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":47051258},{"type":"v1","addr":"172.21.15.191:6834","nonce":47051258}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 msgr2=0x7f8ca00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 0x7f8ca00697d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000a1c0 tx=0x7f8cb0019000 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 msgr2=0x7f8cc00c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800d180 tx=0x7f8cb8009a50 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 shutdown_connections 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ca0067320 0x7f8ca00697d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cc0155c10 0x7f8cc00c9030 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8cc00ac230 0x7f8cc013aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 --2- 172.21.15.170:0/4146516672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8cc00ac6d0 0x7f8cc0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 >> 172.21.15.170:0/4146516672 conn(0x7f8cc006a320 msgr2=0x7f8cc00b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 shutdown_connections 2023-10-08T16:11:36.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.034+0000 7f8cc693d700 1 -- 172.21.15.170:0/4146516672 wait complete. 2023-10-08T16:11:36.036 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 83 2023-10-08T16:11:36.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:36.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.160+0000 7fc797fff700 1 Processor -- start 2023-10-08T16:11:36.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.160+0000 7fc797fff700 1 -- start start 2023-10-08T16:11:36.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc797fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc798159420 0x7fc7980b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc797fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980b2980 con 0x7fc79815d230 2023-10-08T16:11:36.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc797fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980b11d0 con 0x7fc7981597f0 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc797fff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7980b14b0 con 0x7fc798159420 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc7977fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc79815d230 0x7fc79815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:57992/0 (socket says 172.21.15.170:57992) 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc7977fe700 1 -- 172.21.15.170:0/3693861473 learned_addr learned my addr 172.21.15.170:0/3693861473 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc796ffd700 1 --2- 172.21.15.170:0/3693861473 >> v2:172.21.15.191:3300/0 conn(0x7fc798159420 0x7fc7980b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825639209 0 0) 0x7fc7980b2980 con 0x7fc79815d230 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.161+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc784003570 con 0x7fc79815d230 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24345398 0 0) 0x7fc784003570 con 0x7fc79815d230 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 >> v2:172.21.15.191:3300/0 conn(0x7fc798159420 msgr2=0x7fc7980b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 --2- 172.21.15.170:0/3693861473 >> v2:172.21.15.191:3300/0 conn(0x7fc798159420 0x7fc7980b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 >> v1:172.21.15.191:6789/0 conn(0x7fc7981597f0 legacy=0x7fc7980b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980b1790 con 0x7fc79815d230 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fc7900017f0 con 0x7fc79815d230 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 >> v1:172.21.15.170:6789/0 conn(0x7fc79815d230 legacy=0x7fc79815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc784003a60 0x7fc784003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc784004810 0x7fc784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc78400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b14b0 con 0x7fc7840051c0 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c0d0 con 0x7fc784003a60 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78400c290 con 0x7fc784004810 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc78400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc796ffd700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc784003a60 0x7fc784003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc78e7fc700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc784004810 0x7fc784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 -- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc784004810 msgr2=0x7fc784004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc784004810 0x7fc784004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 -- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc784003a60 msgr2=0x7fc784003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc784003a60 0x7fc784003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 -- 172.21.15.170:0/3693861473 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784003570 con 0x7fc7840051c0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 -- 172.21.15.170:0/3693861473 reap_dead start 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7977fe700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc78400ba60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc79000a900 tx=0x7fc7900021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadb22d2062c4326 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900070d0 con 0x7fc7840051c0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc790007230 con 0x7fc7840051c0 2023-10-08T16:11:36.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.162+0000 7fc7967fc700 1 -- 172.21.15.170:0/3693861473 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7900075d0 con 0x7fc7840051c0 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 -- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 msgr2=0x7fc78400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc78400ba60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc79000a900 tx=0x7fc7900021d0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 -- 172.21.15.170:0/3693861473 shutdown_connections 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 --2- 172.21.15.170:0/3693861473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc78400ba60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 -- 172.21.15.170:0/3693861473 >> 172.21.15.170:0/3693861473 conn(0x7fc7980d0660 msgr2=0x7fc7980d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 -- 172.21.15.170:0/3693861473 shutdown_connections 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.163+0000 7fc797fff700 1 -- 172.21.15.170:0/3693861473 wait complete. 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.164+0000 7fc797fff700 1 Processor -- start 2023-10-08T16:11:36.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.164+0000 7fc797fff700 1 -- start start 2023-10-08T16:11:36.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc79814f310 0x7fc798157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc798157b40 0x7fc798169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc798157fb0 con 0x7fc7840051c0 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79816b650 con 0x7fc798157b40 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc797fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798159ca0 con 0x7fc79814f310 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51722/0 (socket says 172.21.15.170:51722) 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 -- 172.21.15.170:0/918186157 learned_addr learned my addr 172.21.15.170:0/918186157 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc7977fe700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc798157b40 0x7fc798169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc78e7fc700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc79814f310 0x7fc798157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 -- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc79814f310 msgr2=0x7fc798157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc79814f310 0x7fc798157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 -- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc798157b40 msgr2=0x7fc798169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc798157b40 0x7fc798169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 -- 172.21.15.170:0/918186157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79816aa30 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.165+0000 7fc796ffd700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc788006ac0 tx=0x7fc7880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.166+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788011ec0 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.166+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78800f940 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.166+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79816ad10 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.166+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78800fcc0 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.166+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79815a240 con 0x7fc7840051c0 2023-10-08T16:11:36.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.167+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79815a430 con 0x7fc7840051c0 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.167+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc788014650 con 0x7fc7840051c0 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.167+0000 7fc78f7fe700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 0x7fc778069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.167+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7fc788018030 con 0x7fc7840051c0 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.167+0000 7fc78e7fc700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 0x7fc778069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.168+0000 7fc78e7fc700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 0x7fc778069800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc79816d170 tx=0x7fc780007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.171+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc788055c50 con 0x7fc7840051c0 2023-10-08T16:11:36.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.357+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc79804c3c0 con 0x7fc7840051c0 2023-10-08T16:11:36.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.358+0000 7fc78f7fe700 1 -- 172.21.15.170:0/918186157 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+15673 (secure 0 0 0) 0x7fc7880557e0 con 0x7fc7840051c0 2023-10-08T16:11:36.358 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:36.358 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":127,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:32.704282+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:02.613155+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:03.244744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6836/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6834/3558382637":"2023-10-09T16:11:29.697526+0000","172.21.15.170:6837/707523029":"2023-10-09T16:11:26.301707+0000","172.21.15.170:6835/707523029":"2023-10-09T16:11:26.301707+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 msgr2=0x7fc778069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 0x7fc778069800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc79816d170 tx=0x7fc780007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 msgr2=0x7fc79816b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fc788006ac0 tx=0x7fc7880091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 shutdown_connections 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc798157b40 0x7fc798169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc778067350 0x7fc778069800 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7840051c0 0x7fc79816b110 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 --2- 172.21.15.170:0/918186157 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc79814f310 0x7fc798157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 >> 172.21.15.170:0/918186157 conn(0x7fc7980d0660 msgr2=0x7fc798069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 shutdown_connections 2023-10-08T16:11:36.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.360+0000 7fc797fff700 1 -- 172.21.15.170:0/918186157 wait complete. 2023-10-08T16:11:36.372 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:11:36.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.487+0000 7f0920a64700 1 Processor -- start 2023-10-08T16:11:36.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.487+0000 7f0920a64700 1 -- start start 2023-10-08T16:11:36.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f0920a64700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f091c159c60 0x7f091c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f0920a64700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c15a160 con 0x7f091c15db10 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f0920a64700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0d13a0 con 0x7f091c159890 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f0920a64700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c15df50 con 0x7f091c159c60 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091ad9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f091c15db10 0x7f091c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58000/0 (socket says 172.21.15.170:58000) 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091ad9d700 1 -- 172.21.15.170:0/1664835815 learned_addr learned my addr 172.21.15.170:0/1664835815 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f0919d9b700 1 --2- 172.21.15.170:0/1664835815 >> v2:172.21.15.191:3300/0 conn(0x7f091c159c60 0x7f091c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311048517 0 0) 0x7f091c15a160 con 0x7f091c15db10 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0908003570 con 0x7f091c15db10 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1585203118 0 0) 0x7f0908003570 con 0x7f091c15db10 2023-10-08T16:11:36.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 >> v2:172.21.15.191:3300/0 conn(0x7f091c159c60 msgr2=0x7f091c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 --2- 172.21.15.170:0/1664835815 >> v2:172.21.15.191:3300/0 conn(0x7f091c159c60 0x7f091c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 >> v1:172.21.15.191:6789/0 conn(0x7f091c159890 legacy=0x7f091c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c15e140 con 0x7f091c15db10 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f090c001470 con 0x7f091c15db10 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 >> v1:172.21.15.170:6789/0 conn(0x7f091c15db10 legacy=0x7f091c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f0908003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0908004810 0x7f0908004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09080051c0 0x7f090800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090800c050 con 0x7f0908003a60 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090800c190 con 0x7f0908004810 2023-10-08T16:11:36.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.488+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090800c350 con 0x7f09080051c0 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f0908003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f0919d9b700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0908004810 0x7f0908004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091ad9d700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09080051c0 0x7f090800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 -- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09080051c0 msgr2=0x7f090800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09080051c0 0x7f090800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 -- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0908004810 msgr2=0x7f0908004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0908004810 0x7f0908004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 -- 172.21.15.170:0/1664835815 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908003570 con 0x7f0908003a60 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091ad9d700 1 -- 172.21.15.170:0/1664835815 reap_dead start 2023-10-08T16:11:36.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091a59c700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f0908003e30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f09140020a0 tx=0x7f09140016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73a2a4bfa788b3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140019e0 con 0x7f0908003a60 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400c8b0 con 0x7f0908003a60 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.489+0000 7f091959a700 1 -- 172.21.15.170:0/1664835815 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400cc50 con 0x7f0908003a60 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 -- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 msgr2=0x7f0908003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f0908003e30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f09140020a0 tx=0x7f09140016d0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 -- 172.21.15.170:0/1664835815 shutdown_connections 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 --2- 172.21.15.170:0/1664835815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f0908003e30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 -- 172.21.15.170:0/1664835815 >> 172.21.15.170:0/1664835815 conn(0x7f091c068e20 msgr2=0x7f091c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 -- 172.21.15.170:0/1664835815 shutdown_connections 2023-10-08T16:11:36.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.490+0000 7f0920a64700 1 -- 172.21.15.170:0/1664835815 wait complete. 2023-10-08T16:11:36.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.491+0000 7f0920a64700 1 Processor -- start 2023-10-08T16:11:36.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.491+0000 7f0920a64700 1 -- start start 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f091c150970 0x7f091c152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 0x7f091c152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0cd0c0 con 0x7f0908003a60 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0cd3a0 con 0x7f091c0b5760 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0cf5c0 con 0x7f091c150970 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51742/0 (socket says 172.21.15.170:51742) 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 -- 172.21.15.170:0/1753272702 learned_addr learned my addr 172.21.15.170:0/1753272702 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0919d9b700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f091c150970 0x7f091c152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091ad9d700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 0x7f091c152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 -- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f091c150970 msgr2=0x7f091c152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f091c150970 0x7f091c152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 -- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 msgr2=0x7f091c152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 0x7f091c152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 -- 172.21.15.170:0/1753272702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0cf8a0 con 0x7f0908003a60 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091ad9d700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 0x7f091c152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f091a59c700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f091400cd50 tx=0x7f0914016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140073d0 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914007530 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140078d0 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c0cfb80 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.492+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c15b4d0 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.493+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c04c3c0 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.494+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f091401e430 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.494+0000 7f09127fc700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 0x7f08fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.494+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5877+0+0 (secure 0 0 0) 0x7f0914086e70 con 0x7f0908003a60 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.494+0000 7f0919d9b700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 0x7f08fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.494+0000 7f0919d9b700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 0x7f08fc0697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0904006dc0 tx=0x7f0904008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.498+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0914053a50 con 0x7f0908003a60 2023-10-08T16:11:36.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.721+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f091c0cf030 con 0x7f0908003a60 2023-10-08T16:11:36.722 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:11:36.722+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:11:36.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.725+0000 7f09127fc700 1 -- 172.21.15.170:0/1753272702 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0914057070 con 0x7f0908003a60 2023-10-08T16:11:36.727 INFO:tasks.ceph.mds.d.smithi191.stderr: -1499> 2023-10-08T16:11:10.211+0000 7fb7eb3bc700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:11:36.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.728+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 msgr2=0x7f08fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.728+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 0x7f08fc0697d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0904006dc0 tx=0x7f0904008040 comp rx=0 tx=0).stop 2023-10-08T16:11:36.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.728+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 msgr2=0x7f091c0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.728+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f091400cd50 tx=0x7f0914016000 comp rx=0 tx=0).stop 2023-10-08T16:11:36.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 shutdown_connections 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f091c0b5760 0x7f091c152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08fc067320 0x7f08fc0697d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0908003a60 0x7f091c0b5220 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 --2- 172.21.15.170:0/1753272702 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f091c150970 0x7f091c152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 >> 172.21.15.170:0/1753272702 conn(0x7f091c068e20 msgr2=0x7f091c154370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 shutdown_connections 2023-10-08T16:11:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.729+0000 7f0920a64700 1 -- 172.21.15.170:0/1753272702 wait complete. 2023-10-08T16:11:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:11:36.741 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:11:36.751 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:11:36.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.859+0000 7f2fa65a1700 1 Processor -- start 2023-10-08T16:11:36.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.859+0000 7f2fa65a1700 1 -- start start 2023-10-08T16:11:36.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa65a1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2fa006f060 0x7f2fa00ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa65a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa65a1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa006ebd0 con 0x7f2fa00cb1a0 2023-10-08T16:11:36.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa65a1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa015b2e0 con 0x7f2fa006f060 2023-10-08T16:11:36.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa65a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa015aa70 con 0x7f2fa006f4b0 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa5da0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa5da0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:33836/0 (socket says 172.21.15.170:33836) 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.860+0000 7f2fa5da0700 1 -- 172.21.15.170:0/2591224907 learned_addr learned my addr 172.21.15.170:0/2591224907 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2591224907 >> v2:172.21.15.191:3300/0 conn(0x7f2fa006f060 0x7f2fa00ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2f97fff700 1 -- 172.21.15.170:0/2591224907 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293962782 0 0) 0x7f2fa006ebd0 con 0x7f2fa00cb1a0 2023-10-08T16:11:36.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2f97fff700 1 -- 172.21.15.170:0/2591224907 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f8c003570 con 0x7f2fa00cb1a0 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 msgr2=0x7f2fa00ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2591224907 >> v1:172.21.15.191:6789/0 conn(0x7f2fa00cb1a0 legacy=0x7f2fa015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2591224907 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa015ad20 con 0x7f2fa006f060 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa5da0700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:36.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006f060 0x7f2fa00ae0d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098e0 tx=0x7f2f900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951496e1a12b6625 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2f97fff700 1 -- 172.21.15.170:0/2591224907 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90011630 con 0x7f2fa006f060 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2f97fff700 1 -- 172.21.15.170:0/2591224907 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90011790 con 0x7f2fa006f060 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.861+0000 7f2f97fff700 1 -- 172.21.15.170:0/2591224907 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90011b30 con 0x7f2fa006f060 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006f060 msgr2=0x7f2fa00ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006f060 0x7f2fa00ae0d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2f900098e0 tx=0x7f2f900091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2591224907 shutdown_connections 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f4b0 0x7f2fa00ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2591224907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa006f060 0x7f2fa00ae0d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2591224907 >> 172.21.15.170:0/2591224907 conn(0x7f2fa00cc1e0 msgr2=0x7f2fa00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2591224907 shutdown_connections 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.862+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2591224907 wait complete. 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.863+0000 7f2fa65a1700 1 Processor -- start 2023-10-08T16:11:36.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.863+0000 7f2fa65a1700 1 -- start start 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f060 0x7f2fa014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 0x7f2fa00b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0155820 con 0x7f2fa006f4b0 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0154940 con 0x7f2fa00cb1a0 2023-10-08T16:11:36.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0154c10 con 0x7f2fa006f060 2023-10-08T16:11:36.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51748/0 (socket says 172.21.15.170:51748) 2023-10-08T16:11:36.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2528612158 learned_addr learned my addr 172.21.15.170:0/2528612158 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:36.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa559f700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f060 0x7f2fa014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa5da0700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 0x7f2fa00b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f060 msgr2=0x7f2fa014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f060 0x7f2fa014dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 msgr2=0x7f2fa00b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 0x7f2fa00b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 -- 172.21.15.170:0/2528612158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9800b790 con 0x7f2fa006f4b0 2023-10-08T16:11:36.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa5da0700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 0x7f2fa00b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa4d9e700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006ce80 tx=0x7f2f90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90011b10 con 0x7f2fa006f4b0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f900253f0 con 0x7f2fa006f4b0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f90009420 con 0x7f2fa006f4b0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.864+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90025790 con 0x7f2fa006f4b0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.865+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa01551d0 con 0x7f2fa006f4b0 2023-10-08T16:11:36.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.865+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa006c450 con 0x7f2fa006f4b0 2023-10-08T16:11:36.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.866+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f90025b80 con 0x7f2fa006f4b0 2023-10-08T16:11:36.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.866+0000 7f2f957fa700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 0x7f2f840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:36.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.866+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f2f90086640 con 0x7f2fa006f4b0 2023-10-08T16:11:36.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.866+0000 7f2fa559f700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 0x7f2f840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:36.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.867+0000 7f2fa559f700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 0x7f2f840697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:36.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:36.870+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f900531d0 con 0x7f2fa006f4b0 2023-10-08T16:11:37.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.102+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2fa015b000 con 0x7f2fa006f4b0 2023-10-08T16:11:37.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.726+0000 7f2f957fa700 1 -- 172.21.15.170:0/2528612158 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2f900567f0 con 0x7f2fa006f4b0 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 msgr2=0x7f2f840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 0x7f2f840697d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c007fd0 tx=0x7f2f9c009040 comp rx=0 tx=0).stop 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 msgr2=0x7f2fa00b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006ce80 tx=0x7f2f90017000 comp rx=0 tx=0).stop 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 shutdown_connections 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa00cb1a0 0x7f2fa00b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa006f060 0x7f2fa014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f84067320 0x7f2f840697d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 --2- 172.21.15.170:0/2528612158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa006f4b0 0x7f2fa00b0cd0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 >> 172.21.15.170:0/2528612158 conn(0x7f2fa00cc1e0 msgr2=0x7f2fa00c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 shutdown_connections 2023-10-08T16:11:37.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.728+0000 7f2fa65a1700 1 -- 172.21.15.170:0/2528612158 wait complete. 2023-10-08T16:11:37.741 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:11:37.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.859+0000 7fb3de1f6700 1 Processor -- start 2023-10-08T16:11:37.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.860+0000 7fb3de1f6700 1 -- start start 2023-10-08T16:11:37.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3de1f6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb3d80b03c0 0x7fb3d80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3de1f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3d80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3de1f6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d80cfa00 con 0x7fb3d81591e0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3de1f6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80cb580 con 0x7fb3d80b03c0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3de1f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80b2de0 con 0x7fb3d80b0790 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3dd9f5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb3d81591e0 0x7fb3d8159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58016/0 (socket says 172.21.15.170:58016) 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/2991919726 learned_addr learned my addr 172.21.15.170:0/2991919726 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3dc9f3700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3d80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/2991919726 >> v2:172.21.15.191:3300/0 conn(0x7fb3d80b03c0 0x7fb3d80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408827004 0 0) 0x7fb3d80cfa00 con 0x7fb3d81591e0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c4003570 con 0x7fb3d81591e0 2023-10-08T16:11:37.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697118000 0 0) 0x7fb3c4003570 con 0x7fb3d81591e0 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 msgr2=0x7fb3d80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3d80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 >> v2:172.21.15.191:3300/0 conn(0x7fb3d80b03c0 msgr2=0x7fb3d80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 --2- 172.21.15.170:0/2991919726 >> v2:172.21.15.191:3300/0 conn(0x7fb3d80b03c0 0x7fb3d80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80cf160 con 0x7fb3d81591e0 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb3d0001470 con 0x7fb3d81591e0 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/2991919726 >> v2:172.21.15.191:3300/0 conn(0x7fb3d80b03c0 0x7fb3d80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 >> v1:172.21.15.170:6789/0 conn(0x7fb3d81591e0 legacy=0x7fb3d8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3c4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3c4003fe0 0x7fb3c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3c4004d50 0x7fb3c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80cb580 con 0x7fb3c4004d50 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80b2de0 con 0x7fb3c4003fe0 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.861+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c400ba90 con 0x7fb3d80b0790 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3c4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3c4004d50 0x7fb3c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dc9f3700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3c4003fe0 0x7fb3c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 msgr2=0x7fb3c4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3d80b0790 0x7fb3c4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3c4003fe0 msgr2=0x7fb3c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3c4003fe0 0x7fb3c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/2991919726 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c4003570 con 0x7fb3c4004d50 2023-10-08T16:11:37.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3c4004d50 0x7fb3c40051c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d0001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e79f37df6dba666 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000cc30 con 0x7fb3c4004d50 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000cd90 con 0x7fb3c4004d50 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.862+0000 7fb3cffff700 1 -- 172.21.15.170:0/2991919726 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0005b70 con 0x7fb3c4004d50 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 -- 172.21.15.170:0/2991919726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3c4004d50 msgr2=0x7fb3c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/2991919726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3c4004d50 0x7fb3c40051c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d0001e90 comp rx=0 tx=0).stop 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/2991919726 reap_dead start 2023-10-08T16:11:37.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 -- 172.21.15.170:0/2991919726 shutdown_connections 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 -- 172.21.15.170:0/2991919726 >> 172.21.15.170:0/2991919726 conn(0x7fb3d8063230 msgr2=0x7fb3d8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 -- 172.21.15.170:0/2991919726 shutdown_connections 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.863+0000 7fb3de1f6700 1 -- 172.21.15.170:0/2991919726 wait complete. 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.864+0000 7fb3de1f6700 1 Processor -- start 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.864+0000 7fb3de1f6700 1 -- start start 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3c4004d50 0x7fb3d80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8145850 0x7fb3d80c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c92d0 con 0x7fb3d8145cc0 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c75a0 con 0x7fb3d8145850 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3de1f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c7880 con 0x7fb3c4004d50 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51778/0 (socket says 172.21.15.170:51778) 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/551190937 learned_addr learned my addr 172.21.15.170:0/551190937 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:37.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dc9f3700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8145850 0x7fb3d80c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3c4004d50 0x7fb3d80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3c4004d50 msgr2=0x7fb3d80b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3c4004d50 0x7fb3d80b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8145850 msgr2=0x7fb3d80c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8145850 0x7fb3d80c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 -- 172.21.15.170:0/551190937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80c9540 con 0x7fb3d8145cc0 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3dd9f5700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005ca0 tx=0x7fb3d0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:37.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000c9e0 con 0x7fb3d8145cc0 2023-10-08T16:11:37.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.865+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0001430 con 0x7fb3d8145cc0 2023-10-08T16:11:37.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.866+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d80c9820 con 0x7fb3d8145cc0 2023-10-08T16:11:37.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.866+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0007530 con 0x7fb3d8145cc0 2023-10-08T16:11:37.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.866+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d81565a0 con 0x7fb3d8145cc0 2023-10-08T16:11:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.867+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d80c9130 con 0x7fb3d8145cc0 2023-10-08T16:11:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.867+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb3d0019030 con 0x7fb3d8145cc0 2023-10-08T16:11:37.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.867+0000 7fb3cd7fa700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.867+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fb3d0085140 con 0x7fb3d8145cc0 2023-10-08T16:11:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.867+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.868+0000 7fb3dd1f4700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 0x7fb3bc0697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40094d0 tx=0x7fb3d400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:37.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:37.871+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3d80c9130 con 0x7fb3d8145cc0 2023-10-08T16:11:38.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.054+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb3d80c9130 con 0x7fb3d8145cc0 2023-10-08T16:11:38.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.730+0000 7fb3cd7fa700 1 -- 172.21.15.170:0/551190937 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb3d80c9130 con 0x7fb3d8145cc0 2023-10-08T16:11:38.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 msgr2=0x7fb3bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 0x7fb3bc0697d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40094d0 tx=0x7fb3d400b040 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 msgr2=0x7fb3d8157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb3d0005ca0 tx=0x7fb3d0015000 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 shutdown_connections 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3bc067320 0x7fb3bc0697d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3d8145cc0 0x7fb3d8157b70 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3c4004d50 0x7fb3d80b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 --2- 172.21.15.170:0/551190937 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3d8145850 0x7fb3d80c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 >> 172.21.15.170:0/551190937 conn(0x7fb3d8063230 msgr2=0x7fb3d8157350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:38.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.733+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 shutdown_connections 2023-10-08T16:11:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.734+0000 7fb3de1f6700 1 -- 172.21.15.170:0/551190937 wait complete. 2023-10-08T16:11:38.735 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:11:38.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:11:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.869+0000 7fefe5884700 1 Processor -- start 2023-10-08T16:11:38.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.869+0000 7fefe5884700 1 -- start start 2023-10-08T16:11:38.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefe5884700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fefe0157020 0x7fefe006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefe5884700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00693a0 con 0x7fefe0156540 2023-10-08T16:11:38.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefe5884700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00afdf0 con 0x7fefe00b1290 2023-10-08T16:11:38.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefe5884700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe00b2970 con 0x7fefe0157020 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefdeffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fefe0156540 0x7fefe0156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58024/0 (socket says 172.21.15.170:58024) 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.870+0000 7fefdeffd700 1 -- 172.21.15.170:0/3244564762 learned_addr learned my addr 172.21.15.170:0/3244564762 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdf7fe700 1 --2- 172.21.15.170:0/3244564762 >> v2:172.21.15.191:3300/0 conn(0x7fefe0157020 0x7fefe006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204415999 0 0) 0x7fefe00693a0 con 0x7fefe0156540 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefcc003570 con 0x7fefe0156540 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246825319 0 0) 0x7fefcc003570 con 0x7fefe0156540 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 >> v2:172.21.15.191:3300/0 conn(0x7fefe0157020 msgr2=0x7fefe006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 --2- 172.21.15.170:0/3244564762 >> v2:172.21.15.191:3300/0 conn(0x7fefe0157020 0x7fefe006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 >> v1:172.21.15.191:6789/0 conn(0x7fefe00b1290 legacy=0x7fefe015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0068b00 con 0x7fefe0156540 2023-10-08T16:11:38.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fefd0001470 con 0x7fefe0156540 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 >> v1:172.21.15.170:6789/0 conn(0x7fefe0156540 legacy=0x7fefe0156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 0x7fefcc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc004810 0x7fefcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefcc0051c0 0x7fefcc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00b2970 con 0x7fefcc003a60 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc00c160 con 0x7fefcc0051c0 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc00c320 con 0x7fefcc004810 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 0x7fefcc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdf7fe700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefcc0051c0 0x7fefcc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefd67fc700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc004810 0x7fefcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 -- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc004810 msgr2=0x7fefcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc004810 0x7fefcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 -- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefcc0051c0 msgr2=0x7fefcc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefcc0051c0 0x7fefcc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 -- 172.21.15.170:0/3244564762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc003570 con 0x7fefcc003a60 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdf7fe700 1 -- 172.21.15.170:0/3244564762 reap_dead start 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.871+0000 7fefdeffd700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 0x7fefcc003e70 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c3c0 tx=0x7fefd0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca790ae77794fd98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000cc10 con 0x7fefcc003a60 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000cd70 con 0x7fefcc003a60 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefde7fc700 1 -- 172.21.15.170:0/3244564762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0005b50 con 0x7fefcc003a60 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 -- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 msgr2=0x7fefcc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 0x7fefcc003e70 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c3c0 tx=0x7fefd0001e50 comp rx=0 tx=0).stop 2023-10-08T16:11:38.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 -- 172.21.15.170:0/3244564762 shutdown_connections 2023-10-08T16:11:38.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 --2- 172.21.15.170:0/3244564762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefcc003a60 0x7fefcc003e70 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 -- 172.21.15.170:0/3244564762 >> 172.21.15.170:0/3244564762 conn(0x7fefe00cd880 msgr2=0x7fefe00ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:38.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 -- 172.21.15.170:0/3244564762 shutdown_connections 2023-10-08T16:11:38.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.872+0000 7fefe5884700 1 -- 172.21.15.170:0/3244564762 wait complete. 2023-10-08T16:11:38.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.873+0000 7fefe5884700 1 Processor -- start 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.873+0000 7fefe5884700 1 -- start start 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc003a60 0x7fefe01470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefe00c8fd0 0x7fefe016ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe016b630 con 0x7fefe016b1c0 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe0144410 con 0x7fefe00c8fd0 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefe5884700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01475e0 con 0x7fefcc003a60 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51796/0 (socket says 172.21.15.170:51796) 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefdeffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc003a60 0x7fefe01470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefdf7fe700 1 -- 172.21.15.170:0/2853616483 learned_addr learned my addr 172.21.15.170:0/2853616483 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.874+0000 7fefd67fc700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefe00c8fd0 0x7fefe016ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 -- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc003a60 msgr2=0x7fefe01470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc003a60 0x7fefe01470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 -- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefe00c8fd0 msgr2=0x7fefe016ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefe00c8fd0 0x7fefe016ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 -- 172.21.15.170:0/2853616483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0146000 con 0x7fefe016b1c0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefdf7fe700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fefe00ac8d0 tx=0x7fefd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800e040 con 0x7fefe016b1c0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800f040 con 0x7fefe016b1c0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe01462e0 con 0x7fefe016b1c0 2023-10-08T16:11:38.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800fd70 con 0x7fefe016b1c0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.875+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe01468a0 con 0x7fefe016b1c0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.876+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe00718b0 con 0x7fefe016b1c0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.876+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fefd8015640 con 0x7fefe016b1c0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.877+0000 7fefd77fe700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 0x7fefc00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.877+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5525+0+0 (secure 0 0 0) 0x7fefd8085420 con 0x7fefe016b1c0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.877+0000 7fefdeffd700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 0x7fefc00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.877+0000 7fefdeffd700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 0x7fefc00697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fefd0007cd0 tx=0x7fefd0015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:38.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:38.881+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefd8052010 con 0x7fefe016b1c0 2023-10-08T16:11:39.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.065+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fefe00678d0 con 0x7fefe016b1c0 2023-10-08T16:11:39.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.731+0000 7fefd77fe700 1 -- 172.21.15.170:0/2853616483 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7fefd8055630 con 0x7fefe016b1c0 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 msgr2=0x7fefc00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 0x7fefc00697d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fefd0007cd0 tx=0x7fefd0015000 comp rx=0 tx=0).stop 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 msgr2=0x7fefe0145290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fefe00ac8d0 tx=0x7fefd800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 shutdown_connections 2023-10-08T16:11:39.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fefc0067320 0x7fefc00697d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefe016b1c0 0x7fefe0145290 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fefcc003a60 0x7fefe01470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 --2- 172.21.15.170:0/2853616483 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fefe00c8fd0 0x7fefe016ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 >> 172.21.15.170:0/2853616483 conn(0x7fefe00cd880 msgr2=0x7fefe0153f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 shutdown_connections 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.735+0000 7fefe5884700 1 -- 172.21.15.170:0/2853616483 wait complete. 2023-10-08T16:11:39.736 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:11:39.749 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:11:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.865+0000 7fa0c8add700 1 Processor -- start 2023-10-08T16:11:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.865+0000 7fa0c8add700 1 -- start start 2023-10-08T16:11:39.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c8add700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa0c4159890 0x7fa0c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c8add700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c4069560 0x7fa0c40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c8add700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c4069ee0 con 0x7fa0c4159c60 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c8add700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415a160 con 0x7fa0c4069560 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c8add700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40d13a0 con 0x7fa0c4159890 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa0c4069560 0x7fa0c40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c1d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa0c4159c60 0x7fa0c415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58034/0 (socket says 172.21.15.170:58034) 2023-10-08T16:11:39.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c1d9b700 1 -- 172.21.15.170:0/2637317490 learned_addr learned my addr 172.21.15.170:0/2637317490 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c259c700 1 --2- 172.21.15.170:0/2637317490 >> v2:172.21.15.191:3300/0 conn(0x7fa0c4159890 0x7fa0c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 -- 172.21.15.170:0/2637317490 >> v2:172.21.15.191:3300/0 conn(0x7fa0c4159890 msgr2=0x7fa0c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 --2- 172.21.15.170:0/2637317490 >> v2:172.21.15.191:3300/0 conn(0x7fa0c4159890 0x7fa0c415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 -- 172.21.15.170:0/2637317490 >> v1:172.21.15.170:6789/0 conn(0x7fa0c4159c60 legacy=0x7fa0c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 -- 172.21.15.170:0/2637317490 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c415e140 con 0x7fa0c4069560 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c159a700 1 -- 172.21.15.170:0/2637317490 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573097137 0 0) 0x7fa0c4069ee0 con 0x7fa0c4159c60 2023-10-08T16:11:39.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c2d9d700 1 --2- 172.21.15.170:0/2637317490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4069560 0x7fa0c40699a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40d1090 tx=0x7fa0bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2913ab1cf067fc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c159a700 1 -- 172.21.15.170:0/2637317490 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00e070 con 0x7fa0c4069560 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c159a700 1 -- 172.21.15.170:0/2637317490 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc00bb60 con 0x7fa0c4069560 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.866+0000 7fa0c159a700 1 -- 172.21.15.170:0/2637317490 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc012430 con 0x7fa0c4069560 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 -- 172.21.15.170:0/2637317490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4069560 msgr2=0x7fa0c40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 --2- 172.21.15.170:0/2637317490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4069560 0x7fa0c40699a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40d1090 tx=0x7fa0bc009780 comp rx=0 tx=0).stop 2023-10-08T16:11:39.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 -- 172.21.15.170:0/2637317490 shutdown_connections 2023-10-08T16:11:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 --2- 172.21.15.170:0/2637317490 >> v2:172.21.15.191:3300/0 conn(0x7fa0c4159890 0x7fa0c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 --2- 172.21.15.170:0/2637317490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4069560 0x7fa0c40699a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 -- 172.21.15.170:0/2637317490 >> 172.21.15.170:0/2637317490 conn(0x7fa0c4068e20 msgr2=0x7fa0c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 -- 172.21.15.170:0/2637317490 shutdown_connections 2023-10-08T16:11:39.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.867+0000 7fa0c8add700 1 -- 172.21.15.170:0/2637317490 wait complete. 2023-10-08T16:11:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.868+0000 7fa0c8add700 1 Processor -- start 2023-10-08T16:11:39.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.868+0000 7fa0c8add700 1 -- start start 2023-10-08T16:11:39.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c4069560 0x7fa0c40b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4159c60 0x7fa0c41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cf210 con 0x7fa0c4159890 2023-10-08T16:11:39.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cf4f0 con 0x7fa0c4159c60 2023-10-08T16:11:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c8add700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cf7d0 con 0x7fa0c4069560 2023-10-08T16:11:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51808/0 (socket says 172.21.15.170:51808) 2023-10-08T16:11:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 -- 172.21.15.170:0/559552109 learned_addr learned my addr 172.21.15.170:0/559552109 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:39.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c259c700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c4069560 0x7fa0c40b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c2d9d700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4159c60 0x7fa0c41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 -- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c4069560 msgr2=0x7fa0c40b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c4069560 0x7fa0c40b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 -- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4159c60 msgr2=0x7fa0c41552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:39.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4159c60 0x7fa0c41552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 -- 172.21.15.170:0/559552109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009a20 con 0x7fa0c4159890 2023-10-08T16:11:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.869+0000 7fa0c1d9b700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800c5b0 tx=0x7fa0b8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.870+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8005b30 con 0x7fa0c4159890 2023-10-08T16:11:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.870+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8005c90 con 0x7fa0c4159890 2023-10-08T16:11:39.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.870+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8007300 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.870+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40cfab0 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.870+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40ceec0 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.871+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40cf350 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.871+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa0b801b3b0 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.871+0000 7fa0b27fc700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 0x7fa0a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.871+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7fa0b8083e50 con 0x7fa0c4159890 2023-10-08T16:11:39.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.871+0000 7fa0c259c700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 0x7fa0a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.872+0000 7fa0c259c700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 0x7fa0a40697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009db0 tx=0x7fa0b40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:39.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:39.876+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0c40cf350 con 0x7fa0c4159890 2023-10-08T16:11:40.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.060+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0c40cf350 con 0x7fa0c4159890 2023-10-08T16:11:40.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.063+0000 7fa0b27fc700 1 -- 172.21.15.170:0/559552109 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0c40cf350 con 0x7fa0c4159890 2023-10-08T16:11:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 msgr2=0x7fa0a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 0x7fa0a40697d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009db0 tx=0x7fa0b40091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 msgr2=0x7fa0c4154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800c5b0 tx=0x7fa0b8001070 comp rx=0 tx=0).stop 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 shutdown_connections 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa0c4159c60 0x7fa0c41552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa0c4069560 0x7fa0c40b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0a4067320 0x7fa0a40697d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 --2- 172.21.15.170:0/559552109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0c4159890 0x7fa0c4154d60 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 >> 172.21.15.170:0/559552109 conn(0x7fa0c4068e20 msgr2=0x7fa0c40cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 shutdown_connections 2023-10-08T16:11:40.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.066+0000 7fa0c8add700 1 -- 172.21.15.170:0/559552109 wait complete. 2023-10-08T16:11:40.078 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls' 2023-10-08T16:11:40.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.192+0000 7f801761c700 1 Processor -- start 2023-10-08T16:11:40.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 -- start start 2023-10-08T16:11:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8010159c60 0x7f801015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f801015db10 0x7f8010069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801015a160 con 0x7f8010159890 2023-10-08T16:11:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801015df50 con 0x7f8010159c60 2023-10-08T16:11:40.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.193+0000 7f801761c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80100d13a0 con 0x7f801015db10 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8010159c60 0x7f801015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f80153b8700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8010159890 0x7f801015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58044/0 (socket says 172.21.15.170:58044) 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f80153b8700 1 -- 172.21.15.170:0/2558593301 learned_addr learned my addr 172.21.15.170:0/2558593301 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8015bb9700 1 --2- 172.21.15.170:0/2558593301 >> v2:172.21.15.191:3300/0 conn(0x7f801015db10 0x7f8010069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 -- 172.21.15.170:0/2558593301 >> v2:172.21.15.191:3300/0 conn(0x7f801015db10 msgr2=0x7f8010069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 --2- 172.21.15.170:0/2558593301 >> v2:172.21.15.191:3300/0 conn(0x7f801015db10 0x7f8010069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 -- 172.21.15.170:0/2558593301 >> v1:172.21.15.170:6789/0 conn(0x7f8010159890 legacy=0x7f801015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 -- 172.21.15.170:0/2558593301 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801015e140 con 0x7f8010159c60 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8014bb7700 1 --2- 172.21.15.170:0/2558593301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8010159c60 0x7f801015d5d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f801006a890 tx=0x7f80000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58159cec858dc2d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8007fff700 1 -- 172.21.15.170:0/2558593301 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000011630 con 0x7f8010159c60 2023-10-08T16:11:40.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8007fff700 1 -- 172.21.15.170:0/2558593301 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000011790 con 0x7f8010159c60 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.194+0000 7f8007fff700 1 -- 172.21.15.170:0/2558593301 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000011b30 con 0x7f8010159c60 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 -- 172.21.15.170:0/2558593301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8010159c60 msgr2=0x7f801015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 --2- 172.21.15.170:0/2558593301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8010159c60 0x7f801015d5d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f801006a890 tx=0x7f80000091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 -- 172.21.15.170:0/2558593301 shutdown_connections 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 --2- 172.21.15.170:0/2558593301 >> v2:172.21.15.191:3300/0 conn(0x7f801015db10 0x7f8010069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 --2- 172.21.15.170:0/2558593301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8010159c60 0x7f801015d5d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 -- 172.21.15.170:0/2558593301 >> 172.21.15.170:0/2558593301 conn(0x7f8010068e20 msgr2=0x7f80100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 -- 172.21.15.170:0/2558593301 shutdown_connections 2023-10-08T16:11:40.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.195+0000 7f801761c700 1 -- 172.21.15.170:0/2558593301 wait complete. 2023-10-08T16:11:40.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.196+0000 7f801761c700 1 Processor -- start 2023-10-08T16:11:40.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.196+0000 7f801761c700 1 -- start start 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 0x7f801016a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8010159c60 0x7f80100b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b1280 con 0x7f801015db10 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b0670 con 0x7f8010159890 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f801761c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80100b0950 con 0x7f8010159c60 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51814/0 (socket says 172.21.15.170:51814) 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 -- 172.21.15.170:0/78922249 learned_addr learned my addr 172.21.15.170:0/78922249 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8014bb7700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8010159c60 0x7f80100b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f80153b8700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 0x7f801016a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 -- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8010159c60 msgr2=0x7f80100b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8010159c60 0x7f80100b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 -- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 msgr2=0x7f801016a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 0x7f801016a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 -- 172.21.15.170:0/78922249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009420 con 0x7f801015db10 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f80153b8700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 0x7f801016a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.197+0000 7f8015bb9700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f800800c880 tx=0x7f8008008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.198+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011600 con 0x7f801015db10 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.198+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008011760 con 0x7f801015db10 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.198+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80100b0c30 con 0x7f801015db10 2023-10-08T16:11:40.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.198+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008011b00 con 0x7f801015db10 2023-10-08T16:11:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.198+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f801016a6f0 con 0x7f801015db10 2023-10-08T16:11:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.199+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80100b13f0 con 0x7f801015db10 2023-10-08T16:11:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.199+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8008018360 con 0x7f801015db10 2023-10-08T16:11:40.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.200+0000 7f80057fa700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.200+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f8008084890 con 0x7f801015db10 2023-10-08T16:11:40.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.200+0000 7f80153b8700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.200+0000 7f80153b8700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f800c002230 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.204+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80080550b0 con 0x7f801015db10 2023-10-08T16:11:40.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.467+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]} v 0) v1 -- 0x7f801004c3c0 con 0x7f801015db10 2023-10-08T16:11:40.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.740+0000 7f80057fa700 1 -- 172.21.15.170:0/78922249 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_ls"]}]=0 v347) v1 ==== 131+0+0 (secure 0 0 0) 0x7f8008054c40 con 0x7f801015db10 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 msgr2=0x7f7ff40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f800c002230 tx=0x7f800c009040 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 msgr2=0x7f801016acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f800800c880 tx=0x7f8008008040 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 shutdown_connections 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f801015db10 0x7f801016acf0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8010159890 0x7f801016a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 --2- 172.21.15.170:0/78922249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8010159c60 0x7f80100b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 >> 172.21.15.170:0/78922249 conn(0x7f8010068e20 msgr2=0x7f8010156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:40.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 shutdown_connections 2023-10-08T16:11:40.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.743+0000 7f801761c700 1 -- 172.21.15.170:0/78922249 wait complete. 2023-10-08T16:11:40.756 INFO:tasks.cephfs_test_runner:That the existing and the newly created volumes can be listed and ... ok 2023-10-08T16:11:40.756 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) That volume, its file system and pools, can be renamed. 2023-10-08T16:11:40.756 INFO:tasks.cephfs_test_runner:test_volume_rename (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:11:40.756 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-08T16:11:40.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.872+0000 7f480e196700 1 Processor -- start 2023-10-08T16:11:40.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.872+0000 7f480e196700 1 -- start start 2023-10-08T16:11:40.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f480e196700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4808159890 0x7f480815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f480e196700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f480e196700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4808069ee0 con 0x7f4808159c60 2023-10-08T16:11:40.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f480e196700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480815a160 con 0x7f4808159890 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f480e196700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080d13a0 con 0x7f4808069560 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.873+0000 7f4807fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:33924/0 (socket says 172.21.15.170:33924) 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 -- 172.21.15.170:0/3892749491 learned_addr learned my addr 172.21.15.170:0/3892749491 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48077fe700 1 --2- 172.21.15.170:0/3892749491 >> v2:172.21.15.191:3300/0 conn(0x7f4808159890 0x7f480815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48067fc700 1 -- 172.21.15.170:0/3892749491 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764276608 0 0) 0x7f4808069ee0 con 0x7f4808159c60 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48067fc700 1 -- 172.21.15.170:0/3892749491 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f0003570 con 0x7f4808159c60 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 -- 172.21.15.170:0/3892749491 >> v2:172.21.15.191:3300/0 conn(0x7f4808159890 msgr2=0x7f480815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 --2- 172.21.15.170:0/3892749491 >> v2:172.21.15.191:3300/0 conn(0x7f4808159890 0x7f480815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 -- 172.21.15.170:0/3892749491 >> v1:172.21.15.191:6789/0 conn(0x7f4808159c60 legacy=0x7f480815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 -- 172.21.15.170:0/3892749491 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480815e140 con 0x7f4808069560 2023-10-08T16:11:40.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f4807fff700 1 --2- 172.21.15.170:0/3892749491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4800009f20 tx=0x7f4800009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d0bb73dc45a85c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48067fc700 1 -- 172.21.15.170:0/3892749491 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000e070 con 0x7f4808069560 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48067fc700 1 -- 172.21.15.170:0/3892749491 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000b970 con 0x7f4808069560 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.874+0000 7f48067fc700 1 -- 172.21.15.170:0/3892749491 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48000092e0 con 0x7f4808069560 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 -- 172.21.15.170:0/3892749491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 msgr2=0x7f48080699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 --2- 172.21.15.170:0/3892749491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4800009f20 tx=0x7f4800009840 comp rx=0 tx=0).stop 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 -- 172.21.15.170:0/3892749491 shutdown_connections 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 --2- 172.21.15.170:0/3892749491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808069560 0x7f48080699a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 --2- 172.21.15.170:0/3892749491 >> v2:172.21.15.191:3300/0 conn(0x7f4808159890 0x7f480815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 -- 172.21.15.170:0/3892749491 >> 172.21.15.170:0/3892749491 conn(0x7f4808068e20 msgr2=0x7f48080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 -- 172.21.15.170:0/3892749491 shutdown_connections 2023-10-08T16:11:40.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.875+0000 7f480e196700 1 -- 172.21.15.170:0/3892749491 wait complete. 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.876+0000 7f480e196700 1 Processor -- start 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.876+0000 7f480e196700 1 -- start start 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808159890 0x7f48080ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4808159c60 0x7f48081421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48081698f0 con 0x7f4808069560 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480814d4c0 con 0x7f4808159c60 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f480e196700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480816de20 con 0x7f4808159890 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51820/0 (socket says 172.21.15.170:51820) 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 -- 172.21.15.170:0/2372519820 learned_addr learned my addr 172.21.15.170:0/2372519820 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f4807fff700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4808159c60 0x7f48081421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f4806ffd700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808159890 0x7f48080ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 -- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808159890 msgr2=0x7f48080ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808159890 0x7f48080ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 -- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4808159c60 msgr2=0x7f48081421a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4808159c60 0x7f48081421a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:40.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 -- 172.21.15.170:0/2372519820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4800009ae0 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.877+0000 7f48077fe700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f47f800ec80 tx=0x7f47f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.878+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800c9d0 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.878+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800cb30 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.878+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4808169b60 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.878+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8013470 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.878+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480816d860 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.879+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4808169e40 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.879+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f47f8013820 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.879+0000 7f47ef7fe700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.880+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f47f8084db0 con 0x7f4808069560 2023-10-08T16:11:40.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.880+0000 7f4806ffd700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:40.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.880+0000 7f4806ffd700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 0x7f47f40697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4808153510 tx=0x7f47fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:40.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:40.884+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47f8089070 con 0x7f4808069560 2023-10-08T16:11:41.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.141+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f480804c3c0 con 0x7f4808069560 2023-10-08T16:11:41.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.742+0000 7f47ef7fe700 1 -- 172.21.15.170:0/2372519820 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v348) v1 ==== 138+0+0 (secure 0 0 0) 0x7f47f8051c90 con 0x7f4808069560 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 msgr2=0x7f47f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 0x7f47f40697d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4808153510 tx=0x7f47fc009040 comp rx=0 tx=0).stop 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 msgr2=0x7f480816cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f47f800ec80 tx=0x7f47f80091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 shutdown_connections 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4808159c60 0x7f48081421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f47f4067320 0x7f47f40697d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4808069560 0x7f480816cf30 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 --2- 172.21.15.170:0/2372519820 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4808159890 0x7f48080ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 >> 172.21.15.170:0/2372519820 conn(0x7f4808068e20 msgr2=0x7f4808153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:41.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 shutdown_connections 2023-10-08T16:11:41.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.744+0000 7f480e196700 1 -- 172.21.15.170:0/2372519820 wait complete. 2023-10-08T16:11:41.757 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:11:41.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.874+0000 7fca6c3c0700 1 Processor -- start 2023-10-08T16:11:41.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.874+0000 7fca6c3c0700 1 -- start start 2023-10-08T16:11:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6c3c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159c60 0x7fca6415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6c3c0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fca6415db10 0x7fca64069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6c3c0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6415a160 con 0x7fca64159890 2023-10-08T16:11:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6c3c0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca640d13a0 con 0x7fca6415db10 2023-10-08T16:11:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6c3c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6415e140 con 0x7fca64159c60 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a15c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fca64159890 0x7fca6415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58050/0 (socket says 172.21.15.170:58050) 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a15c700 1 -- 172.21.15.170:0/315330176 learned_addr learned my addr 172.21.15.170:0/315330176 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a95d700 1 --2- 172.21.15.170:0/315330176 >> v2:172.21.15.170:3300/0 conn(0x7fca6415db10 0x7fca64069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6995b700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159c60 0x7fca6415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a95d700 1 -- 172.21.15.170:0/315330176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159c60 msgr2=0x7fca6415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a95d700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159c60 0x7fca6415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6a95d700 1 -- 172.21.15.170:0/315330176 >> v1:172.21.15.170:6789/0 conn(0x7fca64159890 legacy=0x7fca6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.875+0000 7fca6915a700 1 -- 172.21.15.170:0/315330176 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379132104 0 0) 0x7fca6415a160 con 0x7fca64159890 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6a95d700 1 -- 172.21.15.170:0/315330176 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6415a3d0 con 0x7fca6415db10 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6a95d700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca6415db10 0x7fca64069560 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009e60 tx=0x7fca5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc31e2ddedaae9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6915a700 1 -- 172.21.15.170:0/315330176 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00e070 con 0x7fca6415db10 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6915a700 1 -- 172.21.15.170:0/315330176 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00bc30 con 0x7fca6415db10 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6915a700 1 -- 172.21.15.170:0/315330176 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c012430 con 0x7fca6415db10 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 -- 172.21.15.170:0/315330176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca6415db10 msgr2=0x7fca64069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca6415db10 0x7fca64069560 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009e60 tx=0x7fca5c009780 comp rx=0 tx=0).stop 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 -- 172.21.15.170:0/315330176 shutdown_connections 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca6415db10 0x7fca64069560 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/315330176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca64159c60 0x7fca6415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 -- 172.21.15.170:0/315330176 >> 172.21.15.170:0/315330176 conn(0x7fca64068e20 msgr2=0x7fca640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 -- 172.21.15.170:0/315330176 shutdown_connections 2023-10-08T16:11:41.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.876+0000 7fca6c3c0700 1 -- 172.21.15.170:0/315330176 wait complete. 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.877+0000 7fca6c3c0700 1 Processor -- start 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 -- start start 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca64159890 0x7fca6415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415db10 0x7fca64160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca64160fd0 con 0x7fca64159c60 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6415d2a0 con 0x7fca64159890 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6c3c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6415c690 con 0x7fca6415db10 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6995b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6995b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51848/0 (socket says 172.21.15.170:51848) 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6995b700 1 -- 172.21.15.170:0/4177517057 learned_addr learned my addr 172.21.15.170:0/4177517057 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6a95d700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415db10 0x7fca64160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.878+0000 7fca6a15c700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca64159890 0x7fca6415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 -- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415db10 msgr2=0x7fca64160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415db10 0x7fca64160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 -- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca64159890 msgr2=0x7fca6415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:41.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca64159890 0x7fca6415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 -- 172.21.15.170:0/4177517057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c009a20 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6995b700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fca5400d860 tx=0x7fca54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca540125f0 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca54012750 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6415c970 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54012ad0 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.879+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6415cf30 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.880+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6415d410 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.880+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fca5401e460 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.881+0000 7fca5a7fc700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 0x7fca480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.881+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7fca540850c0 con 0x7fca64159c60 2023-10-08T16:11:41.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.881+0000 7fca6a15c700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 0x7fca480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:41.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.881+0000 7fca6a15c700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 0x7fca480697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca600011e0 tx=0x7fca60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:41.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:41.885+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca540559f0 con 0x7fca64159c60 2023-10-08T16:11:42.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.070+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca6404c3c0 con 0x7fca64159c60 2023-10-08T16:11:42.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.074+0000 7fca5a7fc700 1 -- 172.21.15.170:0/4177517057 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca54055580 con 0x7fca64159c60 2023-10-08T16:11:42.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 msgr2=0x7fca480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 0x7fca480697d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fca600011e0 tx=0x7fca60009040 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 msgr2=0x7fca64160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fca5400d860 tx=0x7fca54008040 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 shutdown_connections 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca6415db10 0x7fca64160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca64159890 0x7fca6415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca48067320 0x7fca480697d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 --2- 172.21.15.170:0/4177517057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca64159c60 0x7fca64160550 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 >> 172.21.15.170:0/4177517057 conn(0x7fca64068e20 msgr2=0x7fca64153d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 shutdown_connections 2023-10-08T16:11:42.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.076+0000 7fca6c3c0700 1 -- 172.21.15.170:0/4177517057 wait complete. 2023-10-08T16:11:42.088 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:11:42.088 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:11:42.126 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:11:42.170 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:11:42.171 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:11:42.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:42.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.285+0000 7f9d3f59e700 1 Processor -- start 2023-10-08T16:11:42.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.285+0000 7f9d3f59e700 1 -- start start 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3f59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9d4006c0f0 0x7f9d401620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3f59e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400b16a0 con 0x7f9d400cc3c0 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3f59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d400b1c60 con 0x7f9d401574d0 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3f59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d400cd980 con 0x7f9d4006c0f0 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3dd9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9d400cc3c0 0x7f9d4006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58056/0 (socket says 172.21.15.170:58056) 2023-10-08T16:11:42.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3dd9b700 1 -- 172.21.15.170:0/2953502451 learned_addr learned my addr 172.21.15.170:0/2953502451 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3ed9d700 1 --2- 172.21.15.170:0/2953502451 >> v2:172.21.15.191:3300/0 conn(0x7f9d4006c0f0 0x7f9d401620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3d59a700 1 -- 172.21.15.170:0/2953502451 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597181537 0 0) 0x7f9d400b16a0 con 0x7f9d400cc3c0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3d59a700 1 -- 172.21.15.170:0/2953502451 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d28003570 con 0x7f9d400cc3c0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3ed9d700 1 -- 172.21.15.170:0/2953502451 >> v1:172.21.15.191:6789/0 conn(0x7f9d401574d0 legacy=0x7f9d400cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3ed9d700 1 -- 172.21.15.170:0/2953502451 >> v1:172.21.15.170:6789/0 conn(0x7f9d400cc3c0 legacy=0x7f9d4006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.286+0000 7f9d3ed9d700 1 -- 172.21.15.170:0/2953502451 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d401578a0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3ed9d700 1 --2- 172.21.15.170:0/2953502451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4006c0f0 0x7f9d401620a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009e60 tx=0x7f9d38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eec457057d50564 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3d59a700 1 -- 172.21.15.170:0/2953502451 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800e070 con 0x7f9d4006c0f0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3d59a700 1 -- 172.21.15.170:0/2953502451 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d3800bb60 con 0x7f9d4006c0f0 2023-10-08T16:11:42.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3d59a700 1 -- 172.21.15.170:0/2953502451 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38012430 con 0x7f9d4006c0f0 2023-10-08T16:11:42.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2953502451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4006c0f0 msgr2=0x7f9d401620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2953502451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4006c0f0 0x7f9d401620a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009e60 tx=0x7f9d38009780 comp rx=0 tx=0).stop 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2953502451 shutdown_connections 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2953502451 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d4006c0f0 0x7f9d401620a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2953502451 >> 172.21.15.170:0/2953502451 conn(0x7f9d400ce430 msgr2=0x7f9d400ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2953502451 shutdown_connections 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.287+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2953502451 wait complete. 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.288+0000 7f9d3f59e700 1 Processor -- start 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.288+0000 7f9d3f59e700 1 -- start start 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d400cc3c0 0x7f9d400c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 0x7f9d4015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400b0120 con 0x7f9d4006c0f0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400af510 con 0x7f9d400cc3c0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d400af7f0 con 0x7f9d401574d0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51856/0 (socket says 172.21.15.170:51856) 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3e59c700 1 -- 172.21.15.170:0/2075600506 learned_addr learned my addr 172.21.15.170:0/2075600506 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3dd9b700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d400cc3c0 0x7f9d400c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.289+0000 7f9d3ed9d700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 0x7f9d4015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 -- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 msgr2=0x7f9d4015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 0x7f9d4015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 -- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d400cc3c0 msgr2=0x7f9d400c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d400cc3c0 0x7f9d400c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 -- 172.21.15.170:0/2075600506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38009a20 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3ed9d700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 0x7f9d4015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3e59c700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c4c0 tx=0x7f9d3000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30011590 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30011710 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d400afad0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30011ad0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.290+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d40146530 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.291+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4004c3c0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.291+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9d3001b3b0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.292+0000 7f9d2e7fc700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 0x7f9d200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.292+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f9d30083d50 con 0x7f9d4006c0f0 2023-10-08T16:11:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.292+0000 7f9d3dd9b700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 0x7f9d200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.292+0000 7f9d3dd9b700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 0x7f9d200697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d4016ae50 tx=0x7f9d34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.296+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d30050ce0 con 0x7f9d4006c0f0 2023-10-08T16:11:42.520 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.520+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d400cdb70 con 0x7f9d4006c0f0 2023-10-08T16:11:42.521 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.520+0000 7f9d2e7fc700 1 -- 172.21.15.170:0/2075600506 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f9d30017070 con 0x7f9d4006c0f0 2023-10-08T16:11:42.521 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:42.521 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6550,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/652073062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":652073062},{"type":"v1","addr":"172.21.15.191:6835","nonce":652073062}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":69},{"gid":6823,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/3849564308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":3849564308},{"type":"v1","addr":"172.21.15.191:6834","nonce":3849564308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85},{"gid":7365,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/309861718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":309861718},{"type":"v1","addr":"172.21.15.170:6837","nonce":309861718}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1899300397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1899300397},{"type":"v1","addr":"172.21.15.170:6836","nonce":1899300397}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[]} 2023-10-08T16:11:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 msgr2=0x7f9d200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 0x7f9d200697d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d4016ae50 tx=0x7f9d34009040 comp rx=0 tx=0).stop 2023-10-08T16:11:42.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 msgr2=0x7f9d40145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000c4c0 tx=0x7f9d3000c7d0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 shutdown_connections 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9d401574d0 0x7f9d4015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9d20067320 0x7f9d200697d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d4006c0f0 0x7f9d40145ff0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 --2- 172.21.15.170:0/2075600506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9d400cc3c0 0x7f9d400c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 >> 172.21.15.170:0/2075600506 conn(0x7f9d400ce430 msgr2=0x7f9d401551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 shutdown_connections 2023-10-08T16:11:42.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.523+0000 7f9d3f59e700 1 -- 172.21.15.170:0/2075600506 wait complete. 2023-10-08T16:11:42.525 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 85 2023-10-08T16:11:42.535 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:11:42.535 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:11:42.536 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:11:42.536 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:11:42.536 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:11:42.536 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:11:42.536 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:11:42.536 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:11:42.536 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:11:42.536 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:11:42.537 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:11:42.537 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:11:42.537 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:11:42.537 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:11:42.537 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:11:42.537 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:11:42.544 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:11:42.544 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:11:42.545 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:11:42.545 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:11:42.547 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:11:42.547 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:11:42.548 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:11:42.548 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:11:42.586 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:11:42.587 DEBUG:teuthology.parallel:result is None 2023-10-08T16:11:42.587 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:11:42.587 DEBUG:teuthology.parallel:result is None 2023-10-08T16:11:42.590 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:11:42.590 DEBUG:teuthology.parallel:result is None 2023-10-08T16:11:42.591 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:11:42.591 DEBUG:teuthology.parallel:result is None 2023-10-08T16:11:42.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:11:42.659 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:11:42.660 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:11:42.669 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:11:42.670 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:11:42.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.712+0000 7f181b6fb700 1 Processor -- start 2023-10-08T16:11:42.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.713+0000 7f181b6fb700 1 -- start start 2023-10-08T16:11:42.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.713+0000 7f181b6fb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f181406bcd0 0x7f18140caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.713+0000 7f181b6fb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f181406c0a0 0x7f181415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.713+0000 7f181b6fb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181415d7e0 con 0x7f181415df50 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.713+0000 7f181b6fb700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181415dac0 con 0x7f181406c0a0 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f181b6fb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181415aa30 con 0x7f181406bcd0 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f181406c0a0 0x7f181415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f181406c0a0 0x7f181415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51878/0 (socket says 172.21.15.170:51878) 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 -- 172.21.15.170:0/741928061 learned_addr learned my addr 172.21.15.170:0/741928061 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1819497700 1 --2- 172.21.15.170:0/741928061 >> v2:172.21.15.191:3300/0 conn(0x7f181406bcd0 0x7f18140caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 -- 172.21.15.170:0/741928061 >> v2:172.21.15.191:3300/0 conn(0x7f181406bcd0 msgr2=0x7f18140caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 --2- 172.21.15.170:0/741928061 >> v2:172.21.15.191:3300/0 conn(0x7f181406bcd0 0x7f18140caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 -- 172.21.15.170:0/741928061 >> v1:172.21.15.191:6789/0 conn(0x7f181415df50 legacy=0x7f18140c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 -- 172.21.15.170:0/741928061 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181415ad10 con 0x7f181406c0a0 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f1818c96700 1 --2- 172.21.15.170:0/741928061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181406c0a0 0x7f181415a1e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f18140691a0 tx=0x7f1804009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b380829225f93984 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f180bfff700 1 -- 172.21.15.170:0/741928061 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804009e70 con 0x7f181406c0a0 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f180bfff700 1 -- 172.21.15.170:0/741928061 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180400b860 con 0x7f181406c0a0 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.714+0000 7f180bfff700 1 -- 172.21.15.170:0/741928061 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400bc20 con 0x7f181406c0a0 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 -- 172.21.15.170:0/741928061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181406c0a0 msgr2=0x7f181415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 --2- 172.21.15.170:0/741928061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181406c0a0 0x7f181415a1e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f18140691a0 tx=0x7f1804009250 comp rx=0 tx=0).stop 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 -- 172.21.15.170:0/741928061 shutdown_connections 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 --2- 172.21.15.170:0/741928061 >> v2:172.21.15.191:3300/0 conn(0x7f181406bcd0 0x7f18140caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 --2- 172.21.15.170:0/741928061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181406c0a0 0x7f181415a1e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 -- 172.21.15.170:0/741928061 >> 172.21.15.170:0/741928061 conn(0x7f18140ce600 msgr2=0x7f18140cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 -- 172.21.15.170:0/741928061 shutdown_connections 2023-10-08T16:11:42.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.715+0000 7f181b6fb700 1 -- 172.21.15.170:0/741928061 wait complete. 2023-10-08T16:11:42.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.716+0000 7f181b6fb700 1 Processor -- start 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.716+0000 7f181b6fb700 1 -- start start 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181406bcd0 0x7f18140c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181406c0a0 0x7f181416a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c7470 con 0x7f181415df50 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c7900 con 0x7f181406c0a0 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f181b6fb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c6860 con 0x7f181406bcd0 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51890/0 (socket says 172.21.15.170:51890) 2023-10-08T16:11:42.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 -- 172.21.15.170:0/1716729912 learned_addr learned my addr 172.21.15.170:0/1716729912 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819497700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181406bcd0 0x7f18140c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1818c96700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181406c0a0 0x7f181416a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 -- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181406bcd0 msgr2=0x7f18140c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181406bcd0 0x7f18140c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 -- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181406c0a0 msgr2=0x7f181416a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181406c0a0 0x7f181416a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.717+0000 7f1819c98700 1 -- 172.21.15.170:0/1716729912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040094f0 con 0x7f181415df50 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f1819c98700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c8e0 tx=0x7f180c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c012630 con 0x7f181415df50 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c012790 con 0x7f181415df50 2023-10-08T16:11:42.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18140c6b40 con 0x7f181415df50 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c012b50 con 0x7f181415df50 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.718+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18140c7100 con 0x7f181415df50 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.719+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181404c3c0 con 0x7f181415df50 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.719+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f180c01a3a0 con 0x7f181415df50 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.719+0000 7f18097fa700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 0x7f17f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:42.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.720+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5111+0+0 (secure 0 0 0) 0x7f180c084c30 con 0x7f181415df50 2023-10-08T16:11:42.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.720+0000 7f1819497700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 0x7f17f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:42.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.720+0000 7f1819497700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 0x7f17f80697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1810009d10 tx=0x7f18100091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:42.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.724+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f180c051b10 con 0x7f181415df50 2023-10-08T16:11:42.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:42.927+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18140c75b0 con 0x7f181415df50 2023-10-08T16:11:43.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.753+0000 7f18097fa700 1 -- 172.21.15.170:0/1716729912 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7f180c055130 con 0x7f181415df50 2023-10-08T16:11:43.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 msgr2=0x7f17f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 0x7f17f80697d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1810009d10 tx=0x7f18100091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 msgr2=0x7f18140afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f180c00c8e0 tx=0x7f180c001710 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 shutdown_connections 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17f8067320 0x7f17f80697d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f181415df50 0x7f18140afe20 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f181406bcd0 0x7f18140c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 --2- 172.21.15.170:0/1716729912 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f181406c0a0 0x7f181416a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 >> 172.21.15.170:0/1716729912 conn(0x7f18140ce600 msgr2=0x7f18140cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:43.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 shutdown_connections 2023-10-08T16:11:43.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.756+0000 7f181b6fb700 1 -- 172.21.15.170:0/1716729912 wait complete. 2023-10-08T16:11:43.758 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:11:43.769 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:11:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.883+0000 7f148d0e4700 1 Processor -- start 2023-10-08T16:11:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.883+0000 7f148d0e4700 1 -- start start 2023-10-08T16:11:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.884+0000 7f148d0e4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.884+0000 7f148d0e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.884+0000 7f148d0e4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f148815a160 con 0x7f148815db10 2023-10-08T16:11:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.884+0000 7f148d0e4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14880d13a0 con 0x7f1488159890 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.884+0000 7f148d0e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148815e140 con 0x7f1488159c60 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f1486d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:34010/0 (socket says 172.21.15.170:34010) 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f1486d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:60212/0 (socket says 172.21.15.170:60212) 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 -- 172.21.15.170:0/3236053543 learned_addr learned my addr 172.21.15.170:0/3236053543 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f148759e700 1 --1- 172.21.15.170:0/3236053543 >> v1:172.21.15.191:6789/0 conn(0x7f148815db10 0x7f1488069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:46864/0 (socket says 172.21.15.170:46864) 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 -- 172.21.15.170:0/3236053543 >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 msgr2=0x7f148815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 --2- 172.21.15.170:0/3236053543 >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 -- 172.21.15.170:0/3236053543 >> v1:172.21.15.191:6789/0 conn(0x7f148815db10 legacy=0x7f1488069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 -- 172.21.15.170:0/3236053543 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148815a3d0 con 0x7f1488159c60 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f1486d9d700 1 --2- 172.21.15.170:0/3236053543 >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f147e59c700 1 --2- 172.21.15.170:0/3236053543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14700099b0 tx=0x7f1470009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58370ecf2c7fd791 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f148659c700 1 -- 172.21.15.170:0/3236053543 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470009ea0 con 0x7f1488159c60 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f148659c700 1 -- 172.21.15.170:0/3236053543 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000b860 con 0x7f1488159c60 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.885+0000 7f148659c700 1 -- 172.21.15.170:0/3236053543 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000bc00 con 0x7f1488159c60 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 -- 172.21.15.170:0/3236053543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 msgr2=0x7f148815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 --2- 172.21.15.170:0/3236053543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14700099b0 tx=0x7f1470009250 comp rx=0 tx=0).stop 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 -- 172.21.15.170:0/3236053543 shutdown_connections 2023-10-08T16:11:43.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 --2- 172.21.15.170:0/3236053543 >> v2:172.21.15.191:3300/0 conn(0x7f1488159890 0x7f148815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 --2- 172.21.15.170:0/3236053543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815d5d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 -- 172.21.15.170:0/3236053543 >> 172.21.15.170:0/3236053543 conn(0x7f1488068e20 msgr2=0x7f14880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 -- 172.21.15.170:0/3236053543 shutdown_connections 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.886+0000 7f148d0e4700 1 -- 172.21.15.170:0/3236053543 wait complete. 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.887+0000 7f148d0e4700 1 Processor -- start 2023-10-08T16:11:43.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.887+0000 7f148d0e4700 1 -- start start 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f148815db10 0x7f1488143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1488152a80 con 0x7f1488159890 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880b3ad0 con 0x7f148815db10 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148d0e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14880ce2a0 con 0x7f1488159c60 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51902/0 (socket says 172.21.15.170:51902) 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 -- 172.21.15.170:0/1427677155 learned_addr learned my addr 172.21.15.170:0/1427677155 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:43.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f147e59c700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f148759e700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f148815db10 0x7f1488143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 -- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 msgr2=0x7f148815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 -- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f148815db10 msgr2=0x7f1488143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f148815db10 0x7f1488143dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 -- 172.21.15.170:0/1427677155 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009a20 con 0x7f1488159890 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f1486d9d700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f147800c170 tx=0x7f147800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800ce10 con 0x7f1488159890 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.888+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147800f040 con 0x7f1488159890 2023-10-08T16:11:43.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.889+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14700094f0 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.889+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478014690 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.889+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14880cd710 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.889+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14880ce580 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.890+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1478014a00 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.890+0000 7f147f7fe700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 0x7f14680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.890+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1478085a00 con 0x7f1488159890 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.890+0000 7f147e59c700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 0x7f14680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.891+0000 7f147e59c700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 0x7f14680697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14700099b0 tx=0x7f1470017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:43.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:43.895+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14780528d0 con 0x7f1488159890 2023-10-08T16:11:44.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.150+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f148804c3c0 con 0x7f1488159890 2023-10-08T16:11:44.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.150+0000 7f147f7fe700 1 -- 172.21.15.170:0/1427677155 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1478014cb0 con 0x7f1488159890 2023-10-08T16:11:44.151 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:44.151 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:11:44.151 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:11:44.151 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:44.152 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.153 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:11:44.154 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.155 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.156 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.156 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:11:44.156 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:11:44.156 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.157 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.158 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.159 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.160 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.161 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.162 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.163 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.164 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.165 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.166 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.166 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.166 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:11:44.166 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:11:44.166 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.167 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:11:44.168 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.169 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:11:44.170 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.171 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:44.172 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:11:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:44.174 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:11:44.175 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:11:44.176 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:11:44.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.177 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:44.177 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:11:44.177 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.152+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 msgr2=0x7f14680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.152+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 0x7f14680697d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f14700099b0 tx=0x7f1470017000 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.152+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 msgr2=0x7f14880b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.152+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f147800c170 tx=0x7f147800b040 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 shutdown_connections 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f148815db10 0x7f1488143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1468067320 0x7f14680697d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1488159890 0x7f14880b3590 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 --2- 172.21.15.170:0/1427677155 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1488159c60 0x7f148815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 >> 172.21.15.170:0/1427677155 conn(0x7f1488068e20 msgr2=0x7f14880d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 shutdown_connections 2023-10-08T16:11:44.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.153+0000 7f148d0e4700 1 -- 172.21.15.170:0/1427677155 wait complete. 2023-10-08T16:11:44.180 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:44.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.295+0000 7ff77447f700 1 Processor -- start 2023-10-08T16:11:44.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.295+0000 7ff77447f700 1 -- start start 2023-10-08T16:11:44.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff77447f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 0x7ff76c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff77447f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff76c1579f0 0x7ff76c0abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff77447f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c0cb300 con 0x7ff76c0ae680 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff77447f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c06fc30 con 0x7ff76c1579f0 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff77447f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0cc260 con 0x7ff76c0b1850 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff772a1c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff76c1579f0 0x7ff76c0abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff772a1c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff76c1579f0 0x7ff76c0abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51906/0 (socket says 172.21.15.170:51906) 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.296+0000 7ff772a1c700 1 -- 172.21.15.170:0/3742626732 learned_addr learned my addr 172.21.15.170:0/3742626732 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff77221b700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 0x7ff76c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff772a1c700 1 -- 172.21.15.170:0/3742626732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 msgr2=0x7ff76c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff772a1c700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 0x7ff76c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff772a1c700 1 -- 172.21.15.170:0/3742626732 >> v1:172.21.15.170:6789/0 conn(0x7ff76c0ae680 legacy=0x7ff76c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff771219700 1 -- 172.21.15.170:0/3742626732 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659621063 0 0) 0x7ff76c0cb300 con 0x7ff76c0ae680 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff772a1c700 1 -- 172.21.15.170:0/3742626732 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c0cb5e0 con 0x7ff76c1579f0 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff77221b700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 0x7ff76c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff772a1c700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c1579f0 0x7ff76c0abf60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff7640099b0 tx=0x7ff764009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407407a6017570c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff771219700 1 -- 172.21.15.170:0/3742626732 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76400b860 con 0x7ff76c1579f0 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff771219700 1 -- 172.21.15.170:0/3742626732 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76400b9c0 con 0x7ff76c1579f0 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.297+0000 7ff771219700 1 -- 172.21.15.170:0/3742626732 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76400bd60 con 0x7ff76c1579f0 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 -- 172.21.15.170:0/3742626732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c1579f0 msgr2=0x7ff76c0abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c1579f0 0x7ff76c0abf60 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7ff7640099b0 tx=0x7ff764009250 comp rx=0 tx=0).stop 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 -- 172.21.15.170:0/3742626732 shutdown_connections 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c0b1850 0x7ff76c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 --2- 172.21.15.170:0/3742626732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c1579f0 0x7ff76c0abf60 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 -- 172.21.15.170:0/3742626732 >> 172.21.15.170:0/3742626732 conn(0x7ff76c06a320 msgr2=0x7ff76c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 -- 172.21.15.170:0/3742626732 shutdown_connections 2023-10-08T16:11:44.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.298+0000 7ff77447f700 1 -- 172.21.15.170:0/3742626732 wait complete. 2023-10-08T16:11:44.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.299+0000 7ff77447f700 1 Processor -- start 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.299+0000 7ff77447f700 1 -- start start 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 0x7ff76c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c1579f0 0x7ff76c0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0a98d0 con 0x7ff76c0ae680 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0aa730 con 0x7ff76c0b1850 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0a9b20 con 0x7ff76c1579f0 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51920/0 (socket says 172.21.15.170:51920) 2023-10-08T16:11:44.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 -- 172.21.15.170:0/1286169019 learned_addr learned my addr 172.21.15.170:0/1286169019 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff772a1c700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c1579f0 0x7ff76c0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff771a1a700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 0x7ff76c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 -- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c1579f0 msgr2=0x7ff76c0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c1579f0 0x7ff76c0aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 -- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 msgr2=0x7ff76c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 0x7ff76c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 -- 172.21.15.170:0/1286169019 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640094f0 con 0x7ff76c0ae680 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff771a1a700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 0x7ff76c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77221b700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff76800a430 tx=0x7ff768007910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800d650 con 0x7ff76c0ae680 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800d7b0 con 0x7ff76c0ae680 2023-10-08T16:11:44.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76c0a9e00 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800db70 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.300+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c0aa3c0 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.301+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c04c3c0 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.302+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff768015440 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.302+0000 7ff7627fc700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 0x7ff7500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.302+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff7680851a0 con 0x7ff76c0ae680 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.302+0000 7ff771a1a700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 0x7ff7500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.302+0000 7ff771a1a700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 0x7ff7500697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0011e0 tx=0x7ff75c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.306+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff76804e5a0 con 0x7ff76c0ae680 2023-10-08T16:11:44.375 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:11:44.377 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:11:44.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.494+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff76c0cb8c0 con 0x7ff76c0ae680 2023-10-08T16:11:44.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.495+0000 7ff7627fc700 1 -- 172.21.15.170:0/1286169019 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+12192 (secure 0 0 0) 0x7ff768052230 con 0x7ff76c0ae680 2023-10-08T16:11:44.496 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:44.496 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":135,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:43.751735+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:44.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 msgr2=0x7ff7500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 0x7ff7500697d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0011e0 tx=0x7ff75c009040 comp rx=0 tx=0).stop 2023-10-08T16:11:44.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 msgr2=0x7ff76c0b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7ff76800a430 tx=0x7ff768007910 comp rx=0 tx=0).stop 2023-10-08T16:11:44.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 shutdown_connections 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c1579f0 0x7ff76c0aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff750067320 0x7ff7500697d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c0ae680 0x7ff76c0b0310 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 --2- 172.21.15.170:0/1286169019 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c0b1850 0x7ff76c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.497+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 >> 172.21.15.170:0/1286169019 conn(0x7ff76c06a320 msgr2=0x7ff76c152ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.498+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 shutdown_connections 2023-10-08T16:11:44.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.498+0000 7ff77447f700 1 -- 172.21.15.170:0/1286169019 wait complete. 2023-10-08T16:11:44.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:11:44.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:11:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.626+0000 7f463d683700 1 Processor -- start 2023-10-08T16:11:44.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.626+0000 7f463d683700 1 -- start start 2023-10-08T16:11:44.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f463d683700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4638159890 0x7f463815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f463d683700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 0x7f46380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f463d683700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4638069ee0 con 0x7f4638159c60 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f463d683700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463815a160 con 0x7f4638159890 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f463d683700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380d13a0 con 0x7f4638069560 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4638159890 0x7f463815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4638159890 0x7f463815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51926/0 (socket says 172.21.15.170:51926) 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 -- 172.21.15.170:0/743366467 learned_addr learned my addr 172.21.15.170:0/743366467 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f46377fe700 1 --2- 172.21.15.170:0/743366467 >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 0x7f46380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 -- 172.21.15.170:0/743366467 >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 msgr2=0x7f46380699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 --2- 172.21.15.170:0/743366467 >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 0x7f46380699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 -- 172.21.15.170:0/743366467 >> v1:172.21.15.191:6789/0 conn(0x7f4638159c60 legacy=0x7f463815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f4636ffd700 1 -- 172.21.15.170:0/743366467 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463815e140 con 0x7f4638159890 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.627+0000 7f46377fe700 1 --2- 172.21.15.170:0/743366467 >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 0x7f46380699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f4636ffd700 1 --2- 172.21.15.170:0/743366467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159890 0x7f463815abd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f463806a890 tx=0x7f4628009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e37d599eead3a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f4635ffb700 1 -- 172.21.15.170:0/743366467 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462800e070 con 0x7f4638159890 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f4635ffb700 1 -- 172.21.15.170:0/743366467 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462800ba10 con 0x7f4638159890 2023-10-08T16:11:44.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f4635ffb700 1 -- 172.21.15.170:0/743366467 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46280092e0 con 0x7f4638159890 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 -- 172.21.15.170:0/743366467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159890 msgr2=0x7f463815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 --2- 172.21.15.170:0/743366467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159890 0x7f463815abd0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f463806a890 tx=0x7f4628009840 comp rx=0 tx=0).stop 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 -- 172.21.15.170:0/743366467 shutdown_connections 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 --2- 172.21.15.170:0/743366467 >> v2:172.21.15.191:3300/0 conn(0x7f4638069560 0x7f46380699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 --2- 172.21.15.170:0/743366467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159890 0x7f463815abd0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 -- 172.21.15.170:0/743366467 >> 172.21.15.170:0/743366467 conn(0x7f4638068e20 msgr2=0x7f46380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 -- 172.21.15.170:0/743366467 shutdown_connections 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.628+0000 7f463d683700 1 -- 172.21.15.170:0/743366467 wait complete. 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 Processor -- start 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 -- start start 2023-10-08T16:11:44.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4638069560 0x7f4638157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4638159890 0x7f46381450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638156a80 con 0x7f4638159c60 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4638156d60 con 0x7f4638069560 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f463d683700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4638157040 con 0x7f4638159890 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f46377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f46377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51940/0 (socket says 172.21.15.170:51940) 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f46377fe700 1 -- 172.21.15.170:0/740019154 learned_addr learned my addr 172.21.15.170:0/740019154 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f4636ffd700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4638069560 0x7f4638157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.630+0000 7f46367fc700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4638159890 0x7f46381450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 -- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4638159890 msgr2=0x7f46381450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4638159890 0x7f46381450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 -- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4638069560 msgr2=0x7f4638157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4638069560 0x7f4638157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 -- 172.21.15.170:0/740019154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4628009ae0 con 0x7f4638159c60 2023-10-08T16:11:44.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f46377fe700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f463000c880 tx=0x7f4630008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630011600 con 0x7f4638159c60 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4630011760 con 0x7f4638159c60 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4638157320 con 0x7f4638159c60 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630011b20 con 0x7f4638159c60 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.631+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4638145600 con 0x7f4638159c60 2023-10-08T16:11:44.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.632+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4638157840 con 0x7f4638159c60 2023-10-08T16:11:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.632+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f46300193b0 con 0x7f4638159c60 2023-10-08T16:11:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.633+0000 7f461effd700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 0x7f46240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.633+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4630084620 con 0x7f4638159c60 2023-10-08T16:11:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.633+0000 7f4636ffd700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 0x7f46240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.633+0000 7f4636ffd700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 0x7f46240697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4628009880 tx=0x7f4628017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.637+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4630054f80 con 0x7f4638159c60 2023-10-08T16:11:44.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.820+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f463804c3c0 con 0x7f4638159c60 2023-10-08T16:11:44.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.820+0000 7f461effd700 1 -- 172.21.15.170:0/740019154 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4630054b10 con 0x7f4638159c60 2023-10-08T16:11:44.821 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 msgr2=0x7f46240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 0x7f46240697d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4628009880 tx=0x7f4628017000 comp rx=0 tx=0).stop 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 msgr2=0x7f46380b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f463000c880 tx=0x7f4630008040 comp rx=0 tx=0).stop 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 shutdown_connections 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4624067320 0x7f46240697d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4638159c60 0x7f46380b37a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4638069560 0x7f4638157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 --2- 172.21.15.170:0/740019154 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4638159890 0x7f46381450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 >> 172.21.15.170:0/740019154 conn(0x7f4638068e20 msgr2=0x7f46380d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 shutdown_connections 2023-10-08T16:11:44.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.822+0000 7f463d683700 1 -- 172.21.15.170:0/740019154 wait complete. 2023-10-08T16:11:44.835 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:11:44.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.951+0000 7f8a02da3700 1 Processor -- start 2023-10-08T16:11:44.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.951+0000 7f8a02da3700 1 -- start start 2023-10-08T16:11:44.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f8a02da3700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89fc159890 0x7f89fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f8a02da3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15a160 con 0x7f89fc159c60 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f8a02da3700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0d13a0 con 0x7f89fc069560 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f8a02da3700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc15e140 con 0x7f89fc159890 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f89f3fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89fc159c60 0x7f89fc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:58074/0 (socket says 172.21.15.170:58074) 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.952+0000 7f89f3fff700 1 -- 172.21.15.170:0/3998396902 learned_addr learned my addr 172.21.15.170:0/3998396902 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274084596 0 0) 0x7f89fc15a160 con 0x7f89fc159c60 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/3998396902 >> v2:172.21.15.191:3300/0 conn(0x7f89fc159890 0x7f89fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003570 con 0x7f89fc159c60 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4132104202 0 0) 0x7f89e4003570 con 0x7f89fc159c60 2023-10-08T16:11:44.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 >> v2:172.21.15.191:3300/0 conn(0x7f89fc159890 msgr2=0x7f89fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 --2- 172.21.15.170:0/3998396902 >> v2:172.21.15.191:3300/0 conn(0x7f89fc159890 0x7f89fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 >> v1:172.21.15.191:6789/0 conn(0x7f89fc069560 legacy=0x7f89fc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc15a3d0 con 0x7f89fc159c60 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f89ec001470 con 0x7f89fc159c60 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/3998396902 >> v2:172.21.15.191:3300/0 conn(0x7f89fc159890 0x7f89fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 >> v1:172.21.15.170:6789/0 conn(0x7f89fc159c60 legacy=0x7f89fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e4003a60 0x7f89e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40052e0 0x7f89e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc15e140 con 0x7f89e4004810 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c190 con 0x7f89e4003a60 2023-10-08T16:11:44.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c350 con 0x7f89e40052e0 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e4003a60 0x7f89e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f8a01340700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40052e0 0x7f89e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 -- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40052e0 msgr2=0x7f89e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e40052e0 0x7f89e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 -- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e4003a60 msgr2=0x7f89e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e4003a60 0x7f89e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 -- 172.21.15.170:0/3998396902 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003570 con 0x7f89e4004810 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f8a01340700 1 -- 172.21.15.170:0/3998396902 reap_dead start 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.953+0000 7f89f3fff700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c3c0 tx=0x7f89ec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d4fce2acfb55cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00cc50 con 0x7f89e4004810 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec00cdb0 con 0x7f89e4004810 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f89f37fe700 1 -- 172.21.15.170:0/3998396902 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec005b70 con 0x7f89e4004810 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 -- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 msgr2=0x7f89e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00c3c0 tx=0x7f89ec001e50 comp rx=0 tx=0).stop 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 -- 172.21.15.170:0/3998396902 shutdown_connections 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 --2- 172.21.15.170:0/3998396902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e4004810 0x7f89e4004c80 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 -- 172.21.15.170:0/3998396902 >> 172.21.15.170:0/3998396902 conn(0x7f89fc068e20 msgr2=0x7f89fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 -- 172.21.15.170:0/3998396902 shutdown_connections 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.954+0000 7f8a02da3700 1 -- 172.21.15.170:0/3998396902 wait complete. 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.955+0000 7f8a02da3700 1 Processor -- start 2023-10-08T16:11:44.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.955+0000 7f8a02da3700 1 -- start start 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e4004810 0x7f89fc1400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89fc0b1b70 0x7f89fc0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cda80 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc157230 con 0x7f89fc0b1b70 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a02da3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b25f0 con 0x7f89e4004810 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51964/0 (socket says 172.21.15.170:51964) 2023-10-08T16:11:44.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 -- 172.21.15.170:0/1259813660 learned_addr learned my addr 172.21.15.170:0/1259813660 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e4004810 0x7f89fc1400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f89f3fff700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89fc0b1b70 0x7f89fc0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 -- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e4004810 msgr2=0x7f89fc1400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e4004810 0x7f89fc1400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.956+0000 7f8a01340700 1 -- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89fc0b1b70 msgr2=0x7f89fc0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f8a01340700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89fc0b1b70 0x7f89fc0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f8a01340700 1 -- 172.21.15.170:0/1259813660 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc156900 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f8a01340700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06a300 tx=0x7f89f4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400eec0 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f4006db0 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f400c820 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc156be0 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.957+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc140620 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.958+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc04c3c0 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.958+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89f400cbd0 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.959+0000 7f89f0ff9700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 0x7f89e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.959+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f89f4084030 con 0x7f89fc0cd0c0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.959+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 0x7f89e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.959+0000 7f8a00b3f700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 0x7f89e80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f89fc155a60 tx=0x7f89f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:44.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:44.963+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f4051090 con 0x7f89fc0cd0c0 2023-10-08T16:11:45.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.147+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f89fc157370 con 0x7f89fc0cd0c0 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.147+0000 7f89f0ff9700 1 -- 172.21.15.170:0/1259813660 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+65 (secure 0 0 0) 0x7f89f4019060 con 0x7f89fc0cd0c0 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:11:45.148 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:11:45.149 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:45.149 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 msgr2=0x7f89e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 0x7f89e80697d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f89fc155a60 tx=0x7f89f8008040 comp rx=0 tx=0).stop 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 msgr2=0x7f89fc0cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f89fc06a300 tx=0x7f89f4001640 comp rx=0 tx=0).stop 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 shutdown_connections 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e8067320 0x7f89e80697d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89fc0cd0c0 0x7f89fc0cd540 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e4004810 0x7f89fc1400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 --2- 172.21.15.170:0/1259813660 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89fc0b1b70 0x7f89fc0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 >> 172.21.15.170:0/1259813660 conn(0x7f89fc068e20 msgr2=0x7f89fc0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:45.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 shutdown_connections 2023-10-08T16:11:45.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.150+0000 7f8a02da3700 1 -- 172.21.15.170:0/1259813660 wait complete. 2023-10-08T16:11:45.162 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:11:45.162 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:11:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.280+0000 7f040bee7700 1 Processor -- start 2023-10-08T16:11:45.281 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.280+0000 7f040bee7700 1 -- start start 2023-10-08T16:11:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f040bee7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.283 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f040bee7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04040b53d0 0x7f04040cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f040bee7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04040b5b00 con 0x7f04040cce40 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f040bee7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040b5da0 con 0x7f04040b53d0 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f040bee7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04040b6080 con 0x7f0404159ec0 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.281+0000 7f0409c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409482700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f04040b53d0 0x7f04040cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f040a484700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f04040cce40 0x7f04040cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:44384/0 (socket says 172.21.15.170:44384) 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:50658/0 (socket says 172.21.15.170:50658) 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f040a484700 1 -- 172.21.15.170:0/1843065170 learned_addr learned my addr 172.21.15.170:0/1843065170 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:45.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 -- 172.21.15.170:0/1843065170 >> v2:172.21.15.191:3300/0 conn(0x7f04040b53d0 msgr2=0x7f04040cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 --2- 172.21.15.170:0/1843065170 >> v2:172.21.15.191:3300/0 conn(0x7f04040b53d0 0x7f04040cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 -- 172.21.15.170:0/1843065170 >> v1:172.21.15.191:6789/0 conn(0x7f04040cce40 legacy=0x7f04040cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 -- 172.21.15.170:0/1843065170 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040cc100 con 0x7f0404159ec0 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0409c83700 1 --2- 172.21.15.170:0/1843065170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04000098d0 tx=0x7f0400009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c78b0221d060756 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0408c81700 1 -- 172.21.15.170:0/1843065170 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400005c10 con 0x7f0404159ec0 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0408c81700 1 -- 172.21.15.170:0/1843065170 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0400005d70 con 0x7f0404159ec0 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.282+0000 7f0408c81700 1 -- 172.21.15.170:0/1843065170 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400006280 con 0x7f0404159ec0 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 -- 172.21.15.170:0/1843065170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 msgr2=0x7f04040b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 --2- 172.21.15.170:0/1843065170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f04000098d0 tx=0x7f0400009120 comp rx=0 tx=0).stop 2023-10-08T16:11:45.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 -- 172.21.15.170:0/1843065170 shutdown_connections 2023-10-08T16:11:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 --2- 172.21.15.170:0/1843065170 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f04040b4e90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 --2- 172.21.15.170:0/1843065170 >> v2:172.21.15.191:3300/0 conn(0x7f04040b53d0 0x7f04040cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 -- 172.21.15.170:0/1843065170 >> 172.21.15.170:0/1843065170 conn(0x7f04040999f0 msgr2=0x7f04040b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 -- 172.21.15.170:0/1843065170 shutdown_connections 2023-10-08T16:11:45.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.283+0000 7f040bee7700 1 -- 172.21.15.170:0/1843065170 wait complete. 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.284+0000 7f040bee7700 1 Processor -- start 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.284+0000 7f040bee7700 1 -- start start 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04040b53d0 0x7f040409a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f0404157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404157690 con 0x7f04040cce40 2023-10-08T16:11:45.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040409b390 con 0x7f04040b53d0 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0404169bd0 con 0x7f0404159ec0 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43236/0 (socket says 172.21.15.170:43236) 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 -- 172.21.15.170:0/2428912277 learned_addr learned my addr 172.21.15.170:0/2428912277 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040a484700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f0404157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409c83700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04040b53d0 0x7f040409a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 -- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 msgr2=0x7f0404157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f0404157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 -- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04040b53d0 msgr2=0x7f040409a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04040b53d0 0x7f040409a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 -- 172.21.15.170:0/2428912277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000093c0 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f0409482700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ec80 tx=0x7f03f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400c9d0 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f400cb30 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040409aaf0 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.285+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4013470 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.286+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040409b0b0 con 0x7f04040cce40 2023-10-08T16:11:45.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.287+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04041699a0 con 0x7f04040cce40 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.287+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f03f4013820 con 0x7f04040cce40 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.287+0000 7f03f9ffb700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 0x7f03e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.287+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f03f4084c90 con 0x7f04040cce40 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.287+0000 7f0409c83700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 0x7f03e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.288+0000 7f0409c83700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 0x7f03e80697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f04040c6bf0 tx=0x7f0400016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.292+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04041699a0 con 0x7f04040cce40 2023-10-08T16:11:45.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.475+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04041699a0 con 0x7f04040cce40 2023-10-08T16:11:45.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.765+0000 7f03f9ffb700 1 -- 172.21.15.170:0/2428912277 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04041699a0 con 0x7f04040cce40 2023-10-08T16:11:45.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 msgr2=0x7f03e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 0x7f03e80697d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f04040c6bf0 tx=0x7f0400016000 comp rx=0 tx=0).stop 2023-10-08T16:11:45.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 msgr2=0x7f040406ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ec80 tx=0x7f03f40091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 shutdown_connections 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0404159ec0 0x7f0404157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f04040b53d0 0x7f040409a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f03e8067320 0x7f03e80697d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 --2- 172.21.15.170:0/2428912277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04040cce40 0x7f040406ac10 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.769+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 >> 172.21.15.170:0/2428912277 conn(0x7f04040999f0 msgr2=0x7f04040b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.770+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 shutdown_connections 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.770+0000 7f040bee7700 1 -- 172.21.15.170:0/2428912277 wait complete. 2023-10-08T16:11:45.771 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:11:45.783 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:11:45.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.904+0000 7f852246d700 1 Processor -- start 2023-10-08T16:11:45.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.904+0000 7f852246d700 1 -- start start 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852246d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f851c0cf480 0x7f851c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852246d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15d760 0x7f851c0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852246d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f851c15dbc0 con 0x7f851c1595d0 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852246d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851c0b10f0 con 0x7f851c0cf480 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852246d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c15e000 con 0x7f851c15d760 2023-10-08T16:11:45.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f851c0cf480 0x7f851c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852146b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f851c1595d0 0x7f851c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49122/0 (socket says 172.21.15.170:49122) 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f852146b700 1 -- 172.21.15.170:0/3591332388 learned_addr learned my addr 172.21.15.170:0/3591332388 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8521c6c700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15d760 0x7f851c0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 -- 172.21.15.170:0/3591332388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15d760 msgr2=0x7f851c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15d760 0x7f851c0b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 -- 172.21.15.170:0/3591332388 >> v1:172.21.15.170:6789/0 conn(0x7f851c1595d0 legacy=0x7f851c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8513fff700 1 -- 172.21.15.170:0/3591332388 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573460205 0 0) 0x7f8518001290 con 0x7f851c1595d0 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 -- 172.21.15.170:0/3591332388 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0cf990 con 0x7f851c0cf480 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.905+0000 7f8520c6a700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c0cf480 0x7f851c15d220 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098e0 tx=0x7f850c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8fbb97228e978c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f8513fff700 1 -- 172.21.15.170:0/3591332388 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c009e80 con 0x7f851c0cf480 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f8513fff700 1 -- 172.21.15.170:0/3591332388 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c00b860 con 0x7f851c0cf480 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f8513fff700 1 -- 172.21.15.170:0/3591332388 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c00bbe0 con 0x7f851c0cf480 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 -- 172.21.15.170:0/3591332388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c0cf480 msgr2=0x7f851c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c0cf480 0x7f851c15d220 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f850c0098e0 tx=0x7f850c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 -- 172.21.15.170:0/3591332388 shutdown_connections 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c15d760 0x7f851c0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.909 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 --2- 172.21.15.170:0/3591332388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c0cf480 0x7f851c15d220 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 -- 172.21.15.170:0/3591332388 >> 172.21.15.170:0/3591332388 conn(0x7f851c0d05f0 msgr2=0x7f851c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:45.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 -- 172.21.15.170:0/3591332388 shutdown_connections 2023-10-08T16:11:45.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.906+0000 7f852246d700 1 -- 172.21.15.170:0/3591332388 wait complete. 2023-10-08T16:11:45.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.907+0000 7f852246d700 1 Processor -- start 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.907+0000 7f852246d700 1 -- start start 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0cf480 0x7f851c16a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 0x7f851c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c16abb0 con 0x7f851c15d760 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851c155010 con 0x7f851c1595d0 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852246d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851c1552f0 con 0x7f851c0cf480 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f8521c6c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f8521c6c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43252/0 (socket says 172.21.15.170:43252) 2023-10-08T16:11:45.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f8521c6c700 1 -- 172.21.15.170:0/3935348700 learned_addr learned my addr 172.21.15.170:0/3935348700 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f852146b700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0cf480 0x7f851c16a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.908+0000 7f8520c6a700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 0x7f851c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 -- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0cf480 msgr2=0x7f851c16a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0cf480 0x7f851c16a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 -- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 msgr2=0x7f851c15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 0x7f851c15c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 -- 172.21.15.170:0/3935348700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009420 con 0x7f851c15d760 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8520c6a700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 0x7f851c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:45.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f8521c6c700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fa0 tx=0x7f8514009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851400ec70 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851400edd0 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851c1555d0 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8514014670 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.909+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851c16a880 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.910+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851c155af0 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.911+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f85140149e0 con 0x7f851c15d760 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.911+0000 7f85117fa700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 0x7f85000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:45.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.911+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f8514086100 con 0x7f851c15d760 2023-10-08T16:11:45.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.911+0000 7f852146b700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 0x7f85000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:45.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.912+0000 7f852146b700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 0x7f85000697d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f851c15cdf0 tx=0x7f8518009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:45.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:45.915+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8514052fb0 con 0x7f851c15d760 2023-10-08T16:11:46.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.097+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f851c04c3c0 con 0x7f851c15d760 2023-10-08T16:11:46.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.771+0000 7f85117fa700 1 -- 172.21.15.170:0/3935348700 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7f85140565d0 con 0x7f851c15d760 2023-10-08T16:11:46.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.776+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 msgr2=0x7f85000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:46.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.776+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 0x7f85000697d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f851c15cdf0 tx=0x7f8518009040 comp rx=0 tx=0).stop 2023-10-08T16:11:46.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.776+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 msgr2=0x7f851c0c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:46.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.776+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8514009fa0 tx=0x7f8514009900 comp rx=0 tx=0).stop 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 shutdown_connections 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8500067320 0x7f85000697d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f851c15d760 0x7f851c0c67a0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f851c0cf480 0x7f851c16a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 --2- 172.21.15.170:0/3935348700 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f851c1595d0 0x7f851c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.777+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 >> 172.21.15.170:0/3935348700 conn(0x7f851c0d05f0 msgr2=0x7f851c1543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:46.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.778+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 shutdown_connections 2023-10-08T16:11:46.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.779+0000 7f852246d700 1 -- 172.21.15.170:0/3935348700 wait complete. 2023-10-08T16:11:46.790 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:11:46.802 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:11:46.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.920+0000 7f0cf608a700 1 Processor -- start 2023-10-08T16:11:46.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 -- start start 2023-10-08T16:11:46.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0cf00cb1a0 0x7f0cf015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf015a8a0 0x7f0cf006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf015b2e0 con 0x7f0cf006f5a0 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf006ebd0 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.921+0000 7f0cf608a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015ad20 con 0x7f0cf015a8a0 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43266/0 (socket says 172.21.15.170:43266) 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 -- 172.21.15.170:0/3504550647 learned_addr learned my addr 172.21.15.170:0/3504550647 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf4887700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf015a8a0 0x7f0cf006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5088700 1 --2- 172.21.15.170:0/3504550647 >> v2:172.21.15.191:3300/0 conn(0x7f0cf00cb1a0 0x7f0cf015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 -- 172.21.15.170:0/3504550647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf015a8a0 msgr2=0x7f0cf006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf015a8a0 0x7f0cf006f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 -- 172.21.15.170:0/3504550647 >> v2:172.21.15.191:3300/0 conn(0x7f0cf00cb1a0 msgr2=0x7f0cf015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 --2- 172.21.15.170:0/3504550647 >> v2:172.21.15.191:3300/0 conn(0x7f0cf00cb1a0 0x7f0cf015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 -- 172.21.15.170:0/3504550647 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf015b000 con 0x7f0cf006f5a0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0cf5889700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00cce10 tx=0x7f0ce8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcfd1758243634c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0ce7fff700 1 -- 172.21.15.170:0/3504550647 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800b8f0 con 0x7f0cf006f5a0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0ce7fff700 1 -- 172.21.15.170:0/3504550647 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800ba50 con 0x7f0cf006f5a0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.922+0000 7f0ce7fff700 1 -- 172.21.15.170:0/3504550647 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800bdf0 con 0x7f0cf006f5a0 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 -- 172.21.15.170:0/3504550647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf006f5a0 msgr2=0x7f0cf00ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00cce10 tx=0x7f0ce8009250 comp rx=0 tx=0).stop 2023-10-08T16:11:46.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 -- 172.21.15.170:0/3504550647 shutdown_connections 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf006f5a0 0x7f0cf00ae0d0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3504550647 >> v2:172.21.15.191:3300/0 conn(0x7f0cf00cb1a0 0x7f0cf015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3504550647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf015a8a0 0x7f0cf006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 -- 172.21.15.170:0/3504550647 >> 172.21.15.170:0/3504550647 conn(0x7f0cf00cc1e0 msgr2=0x7f0cf00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 -- 172.21.15.170:0/3504550647 shutdown_connections 2023-10-08T16:11:46.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.923+0000 7f0cf608a700 1 -- 172.21.15.170:0/3504550647 wait complete. 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.924+0000 7f0cf608a700 1 Processor -- start 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.924+0000 7f0cf608a700 1 -- start start 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0cf006f5a0 0x7f0cf0154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf0155ef0 0x7f0cf00b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf01553f0 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0153e10 con 0x7f0cf006f5a0 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf608a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf01540f0 con 0x7f0cf0155ef0 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43270/0 (socket says 172.21.15.170:43270) 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 -- 172.21.15.170:0/3764164890 learned_addr learned my addr 172.21.15.170:0/3764164890 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf5889700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf0155ef0 0x7f0cf00b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 -- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf0155ef0 msgr2=0x7f0cf00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf0155ef0 0x7f0cf00b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 -- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0cf006f5a0 msgr2=0x7f0cf0154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0cf006f5a0 0x7f0cf0154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 -- 172.21.15.170:0/3764164890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80094f0 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.925+0000 7f0cf4887700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000d860 tx=0x7f0ce0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.926+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0009610 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.926+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0009770 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.926+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf01543d0 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.926+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0009b30 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.926+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf0154990 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.927+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf0154b50 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.927+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0ce001e460 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.927+0000 7f0ce57fa700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 0x7f0cd40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.927+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f0ce00852b0 con 0x7f0cf00cb1a0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.927+0000 7f0cf5088700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 0x7f0cd40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:46.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.928+0000 7f0cf5088700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 0x7f0cd40697d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0cec007460 tx=0x7f0cec009a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:46.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:46.932+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ce0051f60 con 0x7f0cf00cb1a0 2023-10-08T16:11:47.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.158+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0cf0154d00 con 0x7f0cf00cb1a0 2023-10-08T16:11:47.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.782+0000 7f0ce57fa700 1 -- 172.21.15.170:0/3764164890 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0ce0055580 con 0x7f0cf00cb1a0 2023-10-08T16:11:47.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 msgr2=0x7f0cd40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:47.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 0x7f0cd40697d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0cec007460 tx=0x7f0cec009a10 comp rx=0 tx=0).stop 2023-10-08T16:11:47.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 msgr2=0x7f0cf0153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:47.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000d860 tx=0x7f0ce0008040 comp rx=0 tx=0).stop 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 shutdown_connections 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0cf0155ef0 0x7f0cf00b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0cf006f5a0 0x7f0cf0154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0cd4067320 0x7f0cd40697d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 --2- 172.21.15.170:0/3764164890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0cf00cb1a0 0x7f0cf0153510 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.786+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 >> 172.21.15.170:0/3764164890 conn(0x7f0cf00cc1e0 msgr2=0x7f0cf0069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.787+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 shutdown_connections 2023-10-08T16:11:47.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.787+0000 7f0cf608a700 1 -- 172.21.15.170:0/3764164890 wait complete. 2023-10-08T16:11:47.796 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 24 and data pool 25 2023-10-08T16:11:47.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:47.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.922+0000 7f936af9b700 1 Processor -- start 2023-10-08T16:11:47.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.922+0000 7f936af9b700 1 -- start start 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f936af9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9364159890 0x7f936415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f936af9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9364159c60 0x7f936415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f936af9b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936415a160 con 0x7f936415db10 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f936af9b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640d13a0 con 0x7f9364159890 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f936af9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936415e140 con 0x7f9364159c60 2023-10-08T16:11:47.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f9368d37700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9364159890 0x7f936415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f9368d37700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9364159890 0x7f936415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43272/0 (socket says 172.21.15.170:43272) 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.923+0000 7f9368d37700 1 -- 172.21.15.170:0/2566262260 learned_addr learned my addr 172.21.15.170:0/2566262260 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f935bfff700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9364159c60 0x7f936415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f9368d37700 1 -- 172.21.15.170:0/2566262260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9364159c60 msgr2=0x7f936415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f9368d37700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9364159c60 0x7f936415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f9368d37700 1 -- 172.21.15.170:0/2566262260 >> v1:172.21.15.191:6789/0 conn(0x7f936415db10 legacy=0x7f9364069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f9368d37700 1 -- 172.21.15.170:0/2566262260 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936415a3d0 con 0x7f9364159890 2023-10-08T16:11:47.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f9368d37700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159890 0x7f936415abd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f93600099b0 tx=0x7f9360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fb14bff834a1cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f935b7fe700 1 -- 172.21.15.170:0/2566262260 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000b820 con 0x7f9364159890 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f935b7fe700 1 -- 172.21.15.170:0/2566262260 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000b980 con 0x7f9364159890 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f935b7fe700 1 -- 172.21.15.170:0/2566262260 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000bd20 con 0x7f9364159890 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 -- 172.21.15.170:0/2566262260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159890 msgr2=0x7f936415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159890 0x7f936415abd0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f93600099b0 tx=0x7f9360009250 comp rx=0 tx=0).stop 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 -- 172.21.15.170:0/2566262260 shutdown_connections 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159890 0x7f936415abd0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 --2- 172.21.15.170:0/2566262260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9364159c60 0x7f936415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 -- 172.21.15.170:0/2566262260 >> 172.21.15.170:0/2566262260 conn(0x7f9364068e20 msgr2=0x7f93640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:47.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 -- 172.21.15.170:0/2566262260 shutdown_connections 2023-10-08T16:11:47.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.924+0000 7f936af9b700 1 -- 172.21.15.170:0/2566262260 wait complete. 2023-10-08T16:11:47.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.925+0000 7f936af9b700 1 Processor -- start 2023-10-08T16:11:47.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 -- start start 2023-10-08T16:11:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364159890 0x7f9364169f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f936415db10 0x7f93640cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cde70 con 0x7f9364159c60 2023-10-08T16:11:47.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936416a460 con 0x7f9364159890 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.926+0000 7f936af9b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93640b4a50 con 0x7f936415db10 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43280/0 (socket says 172.21.15.170:43280) 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 -- 172.21.15.170:0/3253473980 learned_addr learned my addr 172.21.15.170:0/3253473980 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f9368d37700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364159890 0x7f9364169f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f9369538700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f936415db10 0x7f93640cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 -- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f936415db10 msgr2=0x7f93640cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f936415db10 0x7f93640cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 -- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364159890 msgr2=0x7f9364169f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:47.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364159890 0x7f9364169f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 -- 172.21.15.170:0/3253473980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600094f0 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f935bfff700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f935400c880 tx=0x7f9354008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354011600 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354011780 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93640cd260 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354011b40 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.927+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93640cd820 con 0x7f9364159c60 2023-10-08T16:11:47.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.928+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936404c3c0 con 0x7f9364159c60 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.929+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9354010350 con 0x7f9364159c60 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.929+0000 7f9358ff9700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 0x7f93500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.929+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9354084ae0 con 0x7f9364159c60 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.929+0000 7f9368d37700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 0x7f93500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.929+0000 7f9368d37700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 0x7f93500697d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f93600099b0 tx=0x7f9360016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:47.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:47.933+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9354051660 con 0x7f9364159c60 2023-10-08T16:11:48.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.119+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93641536c0 con 0x7f9364159c60 2023-10-08T16:11:48.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.119+0000 7f9358ff9700 1 -- 172.21.15.170:0/3253473980 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f9354015090 con 0x7f9364159c60 2023-10-08T16:11:48.120 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:48.121 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":138,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:47.159230+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:46.115982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:48.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 msgr2=0x7f93500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 0x7f93500697d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f93600099b0 tx=0x7f9360016000 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 msgr2=0x7f93640b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f935400c880 tx=0x7f9354008040 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 shutdown_connections 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f936415db10 0x7f93640cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9364159890 0x7f9364169f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9350067320 0x7f93500697d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 --2- 172.21.15.170:0/3253473980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9364159c60 0x7f93640b5390 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 >> 172.21.15.170:0/3253473980 conn(0x7f9364068e20 msgr2=0x7f93640b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 shutdown_connections 2023-10-08T16:11:48.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.122+0000 7f936af9b700 1 -- 172.21.15.170:0/3253473980 wait complete. 2023-10-08T16:11:48.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:48.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.249+0000 7f07dd770700 1 Processor -- start 2023-10-08T16:11:48.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.249+0000 7f07dd770700 1 -- start start 2023-10-08T16:11:48.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.250+0000 7f07dd770700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f07d81597c0 0x7f07d8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.250+0000 7f07dd770700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80ae390 con 0x7f07d80af210 2023-10-08T16:11:48.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.250+0000 7f07dd770700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d80ae930 con 0x7f07d80aedc0 2023-10-08T16:11:48.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.250+0000 7f07dd770700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d815e390 con 0x7f07d81597c0 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.250+0000 7f07d77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f07d80af210 0x7f07d806e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49134/0 (socket says 172.21.15.170:49134) 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d77fe700 1 -- 172.21.15.170:0/2165069307 learned_addr learned my addr 172.21.15.170:0/2165069307 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d6ffd700 1 --2- 172.21.15.170:0/2165069307 >> v2:172.21.15.191:3300/0 conn(0x7f07d81597c0 0x7f07d8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987540237 0 0) 0x7f07d80ae390 con 0x7f07d80af210 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4003570 con 0x7f07d80af210 2023-10-08T16:11:48.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704623003 0 0) 0x7f07c4003570 con 0x7f07d80af210 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 >> v2:172.21.15.191:3300/0 conn(0x7f07d81597c0 msgr2=0x7f07d8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 --2- 172.21.15.170:0/2165069307 >> v2:172.21.15.191:3300/0 conn(0x7f07d81597c0 0x7f07d8159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 >> v1:172.21.15.191:6789/0 conn(0x7f07d80aedc0 legacy=0x7f07d815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d815db00 con 0x7f07d80af210 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f07d00017f0 con 0x7f07d80af210 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 >> v1:172.21.15.170:6789/0 conn(0x7f07d80af210 legacy=0x7f07d806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c4003a60 0x7f07c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c40051c0 0x7f07c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d815e390 con 0x7f07c4004810 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400c0d0 con 0x7f07c4003a60 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400c290 con 0x7f07c40051c0 2023-10-08T16:11:48.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d6ffd700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c4003a60 0x7f07c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07d77fe700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c40051c0 0x7f07c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07ce7fc700 1 -- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c40051c0 msgr2=0x7f07c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c40051c0 0x7f07c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07ce7fc700 1 -- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c4003a60 msgr2=0x7f07c4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.251+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07c4003a60 0x7f07c4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07ce7fc700 1 -- 172.21.15.170:0/2165069307 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4003570 con 0x7f07c4004810 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07d77fe700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07c40051c0 0x7f07c400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:48.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07d77fe700 1 -- 172.21.15.170:0/2165069307 reap_dead start 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07c4004c80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f07c00018b0 tx=0x7f07c0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c96921450471667 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0012420 con 0x7f07c4004810 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00125a0 con 0x7f07c4004810 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07d67fc700 1 -- 172.21.15.170:0/2165069307 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0012960 con 0x7f07c4004810 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 -- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 msgr2=0x7f07c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07c4004c80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f07c00018b0 tx=0x7f07c0007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 -- 172.21.15.170:0/2165069307 shutdown_connections 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 --2- 172.21.15.170:0/2165069307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07c4004c80 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 -- 172.21.15.170:0/2165069307 >> 172.21.15.170:0/2165069307 conn(0x7f07d80c9f30 msgr2=0x7f07d806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 -- 172.21.15.170:0/2165069307 shutdown_connections 2023-10-08T16:11:48.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.252+0000 7f07dd770700 1 -- 172.21.15.170:0/2165069307 wait complete. 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 Processor -- start 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 -- start start 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07d80b0770 0x7f07d8169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07d816a1f0 0x7f07d816a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d8144560 con 0x7f07c4004810 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d80b1220 con 0x7f07d816a1f0 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07dd770700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d816abd0 con 0x7f07d80b0770 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07d6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43302/0 (socket says 172.21.15.170:43302) 2023-10-08T16:11:48.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07d6ffd700 1 -- 172.21.15.170:0/4200977265 learned_addr learned my addr 172.21.15.170:0/4200977265 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07d80b0770 0x7f07d8169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.254+0000 7f07d77fe700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07d816a1f0 0x7f07d816a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 -- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07d80b0770 msgr2=0x7f07d8169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07d80b0770 0x7f07d8169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 -- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07d816a1f0 msgr2=0x7f07d816a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07d816a1f0 0x7f07d816a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 -- 172.21.15.170:0/4200977265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d816b2f0 con 0x7f07c4004810 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07d6ffd700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f07c8011d70 tx=0x7f07c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800fac0 con 0x7f07c4004810 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c800fc20 con 0x7f07c4004810 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d80b0c60 con 0x7f07c4004810 2023-10-08T16:11:48.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8014440 con 0x7f07c4004810 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.255+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d816af80 con 0x7f07c4004810 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.256+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d80ad3b0 con 0x7f07c4004810 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.256+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f07c8014870 con 0x7f07c4004810 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.257+0000 7f07cf7fe700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 0x7f07b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.257+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 0x7f07b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.257+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f07c8018030 con 0x7f07c4004810 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.257+0000 7f07ce7fc700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 0x7f07b80697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f07d80b24d0 tx=0x7f07c0015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.261+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07c8052410 con 0x7f07c4004810 2023-10-08T16:11:48.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.447+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d804c3c0 con 0x7f07c4004810 2023-10-08T16:11:48.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.448+0000 7f07cf7fe700 1 -- 172.21.15.170:0/4200977265 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f07c8055a30 con 0x7f07c4004810 2023-10-08T16:11:48.448 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:48.448 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":138,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:47.159230+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:46.115982+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:48.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 msgr2=0x7f07b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 0x7f07b80697d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f07d80b24d0 tx=0x7f07c0015000 comp rx=0 tx=0).stop 2023-10-08T16:11:48.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 msgr2=0x7f07d80b0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f07c8011d70 tx=0x7f07c80091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 shutdown_connections 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f07d816a1f0 0x7f07d816a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f07b8067320 0x7f07b80697d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07c4004810 0x7f07d80b0230 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 --2- 172.21.15.170:0/4200977265 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f07d80b0770 0x7f07d8169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 >> 172.21.15.170:0/4200977265 conn(0x7f07d80c9f30 msgr2=0x7f07d8156200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 shutdown_connections 2023-10-08T16:11:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.450+0000 7f07dd770700 1 -- 172.21.15.170:0/4200977265 wait complete. 2023-10-08T16:11:48.463 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:11:48.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 Processor -- start 2023-10-08T16:11:48.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 -- start start 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f17f0159890 0x7f17f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f015a160 con 0x7f17f015db10 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f00d13a0 con 0x7f17f0159890 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.581+0000 7f17f4a75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f015e140 con 0x7f17f0159c60 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17eed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f17f015db10 0x7f17f0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49140/0 (socket says 172.21.15.170:49140) 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17eed9d700 1 -- 172.21.15.170:0/1312795295 learned_addr learned my addr 172.21.15.170:0/1312795295 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314939839 0 0) 0x7f17f015a160 con 0x7f17f015db10 2023-10-08T16:11:48.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 --2- 172.21.15.170:0/1312795295 >> v2:172.21.15.191:3300/0 conn(0x7f17f0159890 0x7f17f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17dc003570 con 0x7f17f015db10 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17e5d9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 581647208 0 0) 0x7f17dc003570 con 0x7f17f015db10 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 msgr2=0x7f17f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 >> v2:172.21.15.191:3300/0 conn(0x7f17f0159890 msgr2=0x7f17f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 --2- 172.21.15.170:0/1312795295 >> v2:172.21.15.191:3300/0 conn(0x7f17f0159890 0x7f17f015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f015a3d0 con 0x7f17f015db10 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f17e80017f0 con 0x7f17f015db10 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 >> v1:172.21.15.170:6789/0 conn(0x7f17f015db10 legacy=0x7f17f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17dc003a60 0x7f17dc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17dc004810 0x7f17dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00d13a0 con 0x7f17dc003a60 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f015e140 con 0x7f17dc004810 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc00ba60 con 0x7f17f0159c60 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17dc003a60 0x7f17dc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17e5d9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17dc004810 0x7f17dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17eed9d700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 -- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 msgr2=0x7f17dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17f0159c60 0x7f17dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 -- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17dc004810 msgr2=0x7f17dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17dc004810 0x7f17dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17ee59c700 1 -- 172.21.15.170:0/1312795295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc003570 con 0x7f17dc003a60 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.582+0000 7f17e5d9b700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17dc004810 0x7f17dc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17ee59c700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17dc003a60 0x7f17dc003e70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009c30 tx=0x7f17e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b635179dd68b1aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000e710 con 0x7f17dc003a60 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e000e870 con 0x7f17dc003a60 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17edd9b700 1 -- 172.21.15.170:0/1312795295 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e000ec30 con 0x7f17dc003a60 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 -- 172.21.15.170:0/1312795295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17dc003a60 msgr2=0x7f17dc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 --2- 172.21.15.170:0/1312795295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17dc003a60 0x7f17dc003e70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009c30 tx=0x7f17e00091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17eed9d700 1 -- 172.21.15.170:0/1312795295 reap_dead start 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 -- 172.21.15.170:0/1312795295 shutdown_connections 2023-10-08T16:11:48.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 -- 172.21.15.170:0/1312795295 >> 172.21.15.170:0/1312795295 conn(0x7f17f0068e20 msgr2=0x7f17f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:48.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 -- 172.21.15.170:0/1312795295 shutdown_connections 2023-10-08T16:11:48.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.583+0000 7f17f4a75700 1 -- 172.21.15.170:0/1312795295 wait complete. 2023-10-08T16:11:48.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.584+0000 7f17f4a75700 1 Processor -- start 2023-10-08T16:11:48.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 -- start start 2023-10-08T16:11:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17dc003a60 0x7f17f00ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f00ce130 0x7f17f016cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f016d920 con 0x7f17f016d4b0 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00b4840 con 0x7f17f00ce130 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17f4a75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f00b3f50 con 0x7f17dc003a60 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17eed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17eed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43324/0 (socket says 172.21.15.170:43324) 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17eed9d700 1 -- 172.21.15.170:0/641121528 learned_addr learned my addr 172.21.15.170:0/641121528 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17ee59c700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17dc003a60 0x7f17f00ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.585+0000 7f17e5d9b700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f00ce130 0x7f17f016cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 -- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17dc003a60 msgr2=0x7f17f00ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17dc003a60 0x7f17f00ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 -- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f00ce130 msgr2=0x7f17f016cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:48.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f00ce130 0x7f17f016cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 -- 172.21.15.170:0/641121528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f00ce5a0 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17eed9d700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f17f015dee0 tx=0x7f17e8001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e800eb10 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e800ec70 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f00b62c0 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8013420 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.586+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f00b44d0 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.587+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f00b49b0 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.587+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f17e8013870 con 0x7f17f016d4b0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.588+0000 7f17e6ffd700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 0x7f17d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.588+0000 7f17ee59c700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 0x7f17d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:48.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.588+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f17e80850d0 con 0x7f17f016d4b0 2023-10-08T16:11:48.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.588+0000 7f17ee59c700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 0x7f17d00697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:48.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.592+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17e8051d60 con 0x7f17f016d4b0 2023-10-08T16:11:48.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:48.821+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f17f004c3c0 con 0x7f17f016d4b0 2023-10-08T16:11:49.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.782+0000 7f17e6ffd700 1 -- 172.21.15.170:0/641121528 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7f17e8055380 con 0x7f17f016d4b0 2023-10-08T16:11:49.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 msgr2=0x7f17d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 0x7f17d00697d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e001a000 comp rx=0 tx=0).stop 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 msgr2=0x7f17f00ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f17f015dee0 tx=0x7f17e8001330 comp rx=0 tx=0).stop 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 shutdown_connections 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f17d0067320 0x7f17d00697d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f17f016d4b0 0x7f17f00ce7e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f17dc003a60 0x7f17f00ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 --2- 172.21.15.170:0/641121528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f17f00ce130 0x7f17f016cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 >> 172.21.15.170:0/641121528 conn(0x7f17f0068e20 msgr2=0x7f17f015bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:49.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 shutdown_connections 2023-10-08T16:11:49.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.785+0000 7f17f4a75700 1 -- 172.21.15.170:0/641121528 wait complete. 2023-10-08T16:11:49.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:11:49.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.923+0000 7f634f59e700 1 Processor -- start 2023-10-08T16:11:49.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.923+0000 7f634f59e700 1 -- start start 2023-10-08T16:11:49.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634f59e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f63500b4e70 0x7f63500ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634f59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63500b1b40 0x7f635015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634f59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63500cfb00 con 0x7f63500b16d0 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634f59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500b5c40 con 0x7f63500b1b40 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634f59e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63500b5f20 con 0x7f63500b4e70 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63500b1b40 0x7f635015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63500b1b40 0x7f635015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43332/0 (socket says 172.21.15.170:43332) 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 -- 172.21.15.170:0/2287210420 learned_addr learned my addr 172.21.15.170:0/2287210420 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:49.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634e59c700 1 --2- 172.21.15.170:0/2287210420 >> v2:172.21.15.191:3300/0 conn(0x7f63500b4e70 0x7f63500ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 -- 172.21.15.170:0/2287210420 >> v2:172.21.15.191:3300/0 conn(0x7f63500b4e70 msgr2=0x7f63500ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 --2- 172.21.15.170:0/2287210420 >> v2:172.21.15.191:3300/0 conn(0x7f63500b4e70 0x7f63500ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 -- 172.21.15.170:0/2287210420 >> v1:172.21.15.191:6789/0 conn(0x7f63500b16d0 legacy=0x7f635015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 -- 172.21.15.170:0/2287210420 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63500b6200 con 0x7f63500b1b40 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.924+0000 7f634ed9d700 1 --2- 172.21.15.170:0/2287210420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f635015d950 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6350064910 tx=0x7f6348009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62e3bb571ee7ebd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:49.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634dd9b700 1 -- 172.21.15.170:0/2287210420 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634800e070 con 0x7f63500b1b40 2023-10-08T16:11:49.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634dd9b700 1 -- 172.21.15.170:0/2287210420 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634800ba10 con 0x7f63500b1b40 2023-10-08T16:11:49.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634dd9b700 1 -- 172.21.15.170:0/2287210420 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63480092e0 con 0x7f63500b1b40 2023-10-08T16:11:49.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 -- 172.21.15.170:0/2287210420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 msgr2=0x7f635015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:49.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 --2- 172.21.15.170:0/2287210420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f635015d950 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6350064910 tx=0x7f6348009840 comp rx=0 tx=0).stop 2023-10-08T16:11:49.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 -- 172.21.15.170:0/2287210420 shutdown_connections 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 --2- 172.21.15.170:0/2287210420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f635015d950 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 --2- 172.21.15.170:0/2287210420 >> v2:172.21.15.191:3300/0 conn(0x7f63500b4e70 0x7f63500ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 -- 172.21.15.170:0/2287210420 >> 172.21.15.170:0/2287210420 conn(0x7f63500d10c0 msgr2=0x7f63500affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 -- 172.21.15.170:0/2287210420 shutdown_connections 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.925+0000 7f634f59e700 1 -- 172.21.15.170:0/2287210420 wait complete. 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.926+0000 7f634f59e700 1 Processor -- start 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.926+0000 7f634f59e700 1 -- start start 2023-10-08T16:11:49.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 0x7f63500cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 0x7f6350144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63500c7110 con 0x7f63500b1b40 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6350169f80 con 0x7f63500b4e70 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63500cbb80 con 0x7f63500b16d0 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f6345d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f6345d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43340/0 (socket says 172.21.15.170:43340) 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f6345d9b700 1 -- 172.21.15.170:0/1762497016 learned_addr learned my addr 172.21.15.170:0/1762497016 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634ed9d700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 0x7f6350144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.927+0000 7f634e59c700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 0x7f63500cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 -- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 msgr2=0x7f63500cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 0x7f63500cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 -- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 msgr2=0x7f6350144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 0x7f6350144780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 -- 172.21.15.170:0/1762497016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348009ae0 con 0x7f63500b1b40 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f634ed9d700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 0x7f6350144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f634e59c700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 0x7f63500cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6345d9b700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f633800c5d0 tx=0x7f6338007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011590 con 0x7f63500b1b40 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6338011710 con 0x7f63500b1b40 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6338011ad0 con 0x7f63500b1b40 2023-10-08T16:11:49.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63500cbe60 con 0x7f63500b1b40 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.928+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6350169c00 con 0x7f63500b1b40 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.929+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6350088990 con 0x7f63500b1b40 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.929+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f633801c460 con 0x7f63500b1b40 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.930+0000 7f6346ffd700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 0x7f63300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.930+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6338084a70 con 0x7f63500b1b40 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.930+0000 7f634e59c700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 0x7f63300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.930+0000 7f634e59c700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 0x7f63300697d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6340006fd0 tx=0x7f6340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:49.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:49.934+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6338051700 con 0x7f63500b1b40 2023-10-08T16:11:50.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.158+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f635016a0c0 con 0x7f63500b1b40 2023-10-08T16:11:50.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.789+0000 7f6346ffd700 1 -- 172.21.15.170:0/1762497016 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6338054d20 con 0x7f63500b1b40 2023-10-08T16:11:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.791+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 msgr2=0x7f63300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 0x7f63300697d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6340006fd0 tx=0x7f6340008040 comp rx=0 tx=0).stop 2023-10-08T16:11:50.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 msgr2=0x7f63500c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f633800c5d0 tx=0x7f6338007f60 comp rx=0 tx=0).stop 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 shutdown_connections 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63500b4e70 0x7f6350144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63500b16d0 0x7f63500cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6330067320 0x7f63300697d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 --2- 172.21.15.170:0/1762497016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63500b1b40 0x7f63500c6bd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 >> 172.21.15.170:0/1762497016 conn(0x7f63500d10c0 msgr2=0x7f63501590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:50.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 shutdown_connections 2023-10-08T16:11:50.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.792+0000 7f634f59e700 1 -- 172.21.15.170:0/1762497016 wait complete. 2023-10-08T16:11:50.805 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.920+0000 7f35a3e93700 1 Processor -- start 2023-10-08T16:11:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.920+0000 7f35a3e93700 1 -- start start 2023-10-08T16:11:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a3e93700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a3e93700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f359c15db10 0x7f359c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a3e93700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c15a160 con 0x7f359c159890 2023-10-08T16:11:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a3e93700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c0d13a0 con 0x7f359c15db10 2023-10-08T16:11:50.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a3e93700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c15e140 con 0x7f359c159c60 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f359c15db10 0x7f359c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f359c15db10 0x7f359c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43344/0 (socket says 172.21.15.170:43344) 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 -- 172.21.15.170:0/2640484136 learned_addr learned my addr 172.21.15.170:0/2640484136 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a142e700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 -- 172.21.15.170:0/2640484136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 msgr2=0x7f359c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 -- 172.21.15.170:0/2640484136 >> v1:172.21.15.191:6789/0 conn(0x7f359c159890 legacy=0x7f359c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 -- 172.21.15.170:0/2640484136 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c15a3d0 con 0x7f359c15db10 2023-10-08T16:11:50.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a142e700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.921+0000 7f35a2430700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c15db10 0x7f359c069560 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3594009f20 tx=0x7f3594009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccff3345d9050700 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a0c2d700 1 -- 172.21.15.170:0/2640484136 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400e070 con 0x7f359c15db10 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a0c2d700 1 -- 172.21.15.170:0/2640484136 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359400ba80 con 0x7f359c15db10 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a0c2d700 1 -- 172.21.15.170:0/2640484136 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940092e0 con 0x7f359c15db10 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 -- 172.21.15.170:0/2640484136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c15db10 msgr2=0x7f359c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c15db10 0x7f359c069560 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3594009f20 tx=0x7f3594009840 comp rx=0 tx=0).stop 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 -- 172.21.15.170:0/2640484136 shutdown_connections 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c15db10 0x7f359c069560 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 --2- 172.21.15.170:0/2640484136 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 -- 172.21.15.170:0/2640484136 >> 172.21.15.170:0/2640484136 conn(0x7f359c068e20 msgr2=0x7f359c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 -- 172.21.15.170:0/2640484136 shutdown_connections 2023-10-08T16:11:50.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.922+0000 7f35a3e93700 1 -- 172.21.15.170:0/2640484136 wait complete. 2023-10-08T16:11:50.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.923+0000 7f35a3e93700 1 Processor -- start 2023-10-08T16:11:50.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.923+0000 7f35a3e93700 1 -- start start 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f359c15db10 0x7f359c153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0b4750 con 0x7f359c159890 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c16d660 con 0x7f359c15db10 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a3e93700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c16d940 con 0x7f359c159c60 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43356/0 (socket says 172.21.15.170:43356) 2023-10-08T16:11:50.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 -- 172.21.15.170:0/3864444078 learned_addr learned my addr 172.21.15.170:0/3864444078 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a142e700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a2430700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f359c15db10 0x7f359c153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 -- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 msgr2=0x7f359c147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 -- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f359c15db10 msgr2=0x7f359c153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f359c15db10 0x7f359c153850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.924+0000 7f35a1c2f700 1 -- 172.21.15.170:0/3864444078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594009ae0 con 0x7f359c159890 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f35a1c2f700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f359800c590 tx=0x7f359800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:50.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011590 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598011710 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c16dc20 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598011ad0 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.925+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c1477c0 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.926+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c04c3c0 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.926+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f359801b3b0 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.926+0000 7f3591ffb700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 0x7f35800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.926+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3598084180 con 0x7f359c159890 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.926+0000 7f35a142e700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 0x7f35800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:50.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.927+0000 7f35a142e700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 0x7f35800697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f359c154bc0 tx=0x7f358c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:50.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:50.931+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3598050e10 con 0x7f359c159890 2023-10-08T16:11:51.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.154+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f359c15b0c0 con 0x7f359c159890 2023-10-08T16:11:51.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.154+0000 7f3591ffb700 1 -- 172.21.15.170:0/3864444078 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f3598017090 con 0x7f359c159890 2023-10-08T16:11:51.155 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:51.155 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":90,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:50.781439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:11:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.156+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 msgr2=0x7f35800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.156+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 0x7f35800697d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f359c154bc0 tx=0x7f358c008040 comp rx=0 tx=0).stop 2023-10-08T16:11:51.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 msgr2=0x7f359c0ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f359800c590 tx=0x7f359800c8a0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 shutdown_connections 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f359c15db10 0x7f359c153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3580067320 0x7f35800697d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f359c159890 0x7f359c0ca310 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 --2- 172.21.15.170:0/3864444078 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f359c159c60 0x7f359c147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 >> 172.21.15.170:0/3864444078 conn(0x7f359c068e20 msgr2=0x7f359c15b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 shutdown_connections 2023-10-08T16:11:51.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.157+0000 7f35a3e93700 1 -- 172.21.15.170:0/3864444078 wait complete. 2023-10-08T16:11:51.159 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 90 2023-10-08T16:11:51.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:11:51.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.283+0000 7f810dc60700 1 Processor -- start 2023-10-08T16:11:51.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.283+0000 7f810dc60700 1 -- start start 2023-10-08T16:11:51.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f810dc60700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8108159890 0x7f810815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f810dc60700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 0x7f81080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f810dc60700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8108069ee0 con 0x7f8108159c60 2023-10-08T16:11:51.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f810dc60700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810815a160 con 0x7f8108159890 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f810dc60700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81080d13a0 con 0x7f8108069560 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f81077fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8108159890 0x7f810815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f8106ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8108159c60 0x7f810815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49154/0 (socket says 172.21.15.170:49154) 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f8106ffd700 1 -- 172.21.15.170:0/17758249 learned_addr learned my addr 172.21.15.170:0/17758249 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.284+0000 7f81077fe700 1 --2- 172.21.15.170:0/17758249 >> v2:172.21.15.170:3300/0 conn(0x7f8108159890 0x7f810815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43364/0 (socket says 172.21.15.170:43364) 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f8107fff700 1 --2- 172.21.15.170:0/17758249 >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 0x7f81080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81077fe700 1 -- 172.21.15.170:0/17758249 >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 msgr2=0x7f81080699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81077fe700 1 --2- 172.21.15.170:0/17758249 >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 0x7f81080699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81077fe700 1 -- 172.21.15.170:0/17758249 >> v1:172.21.15.170:6789/0 conn(0x7f8108159c60 legacy=0x7f810815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81077fe700 1 -- 172.21.15.170:0/17758249 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810815e140 con 0x7f8108159890 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81067fc700 1 -- 172.21.15.170:0/17758249 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685002817 0 0) 0x7f8108069ee0 con 0x7f8108159c60 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f8107fff700 1 --2- 172.21.15.170:0/17758249 >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 0x7f81080699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81077fe700 1 --2- 172.21.15.170:0/17758249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159890 0x7f810815abd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f810806a890 tx=0x7f80f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e773f5633ee322 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81067fc700 1 -- 172.21.15.170:0/17758249 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800e070 con 0x7f8108159890 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81067fc700 1 -- 172.21.15.170:0/17758249 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800bc30 con 0x7f8108159890 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f81067fc700 1 -- 172.21.15.170:0/17758249 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8012430 con 0x7f8108159890 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f810dc60700 1 -- 172.21.15.170:0/17758249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159890 msgr2=0x7f810815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.285+0000 7f810dc60700 1 --2- 172.21.15.170:0/17758249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159890 0x7f810815abd0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f810806a890 tx=0x7f80f8009780 comp rx=0 tx=0).stop 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 -- 172.21.15.170:0/17758249 shutdown_connections 2023-10-08T16:11:51.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 --2- 172.21.15.170:0/17758249 >> v2:172.21.15.191:3300/0 conn(0x7f8108069560 0x7f81080699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 --2- 172.21.15.170:0/17758249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159890 0x7f810815abd0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 -- 172.21.15.170:0/17758249 >> 172.21.15.170:0/17758249 conn(0x7f8108068e20 msgr2=0x7f81080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 -- 172.21.15.170:0/17758249 shutdown_connections 2023-10-08T16:11:51.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.286+0000 7f810dc60700 1 -- 172.21.15.170:0/17758249 wait complete. 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.287+0000 7f810dc60700 1 Processor -- start 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.287+0000 7f810dc60700 1 -- start start 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8108069560 0x7f8108151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108159890 0x7f810816a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810816a8c0 con 0x7f8108159c60 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810814d090 con 0x7f8108159890 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8108151990 con 0x7f8108069560 2023-10-08T16:11:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43368/0 (socket says 172.21.15.170:43368) 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 -- 172.21.15.170:0/1362353836 learned_addr learned my addr 172.21.15.170:0/1362353836 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f81077fe700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8108069560 0x7f8108151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8106ffd700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108159890 0x7f810816a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 -- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8108069560 msgr2=0x7f8108151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8108069560 0x7f8108151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 -- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108159890 msgr2=0x7f810816a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108159890 0x7f810816a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 -- 172.21.15.170:0/1362353836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f8009a20 con 0x7f8108159c60 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f8107fff700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f810000e8a0 tx=0x7f8100008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8100013410 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8100013570 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8108152c60 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81000138f0 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.288+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8108151620 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.289+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8108151b00 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.290+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8100013ce0 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.290+0000 7f80ef7fe700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.290+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8100084b70 con 0x7f8108159c60 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.290+0000 7f81077fe700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.290+0000 7f81077fe700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 0x7f80f40697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f810806a890 tx=0x7f80f8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.294+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81000516f0 con 0x7f8108159c60 2023-10-08T16:11:51.480 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.480+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f810804c3c0 con 0x7f8108159c60 2023-10-08T16:11:51.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.480+0000 7f80ef7fe700 1 -- 172.21.15.170:0/1362353836 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f8100054d10 con 0x7f8108159c60 2023-10-08T16:11:51.481 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:51.481 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":139,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:11:48.770958+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T16:11:46.115982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.482+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 msgr2=0x7f80f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.482+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 0x7f80f40697d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f810806a890 tx=0x7f80f8019000 comp rx=0 tx=0).stop 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.482+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 msgr2=0x7f81081499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.482+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f810000e8a0 tx=0x7f8100008040 comp rx=0 tx=0).stop 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 shutdown_connections 2023-10-08T16:11:51.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f80f4067320 0x7f80f40697d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8108159c60 0x7f81081499d0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8108069560 0x7f8108151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 --2- 172.21.15.170:0/1362353836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8108159890 0x7f810816a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 >> 172.21.15.170:0/1362353836 conn(0x7f8108068e20 msgr2=0x7f81080b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 shutdown_connections 2023-10-08T16:11:51.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.483+0000 7f810dc60700 1 -- 172.21.15.170:0/1362353836 wait complete. 2023-10-08T16:11:51.496 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:11:51.496 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.610+0000 7fe182abd700 1 Processor -- start 2023-10-08T16:11:51.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.610+0000 7fe182abd700 1 -- start start 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe182abd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 0x7fe17c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe182abd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe17c1597f0 0x7fe17c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe182abd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0b11d0 con 0x7fe17c15d230 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe182abd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0b14b0 con 0x7fe17c1597f0 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe182abd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0b1790 con 0x7fe17c159420 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe17c1597f0 0x7fe17c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe17c1597f0 0x7fe17c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43370/0 (socket says 172.21.15.170:43370) 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 -- 172.21.15.170:0/67068903 learned_addr learned my addr 172.21.15.170:0/67068903 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe181abb700 1 --2- 172.21.15.170:0/67068903 >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 0x7fe17c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 -- 172.21.15.170:0/67068903 >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 msgr2=0x7fe17c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 --2- 172.21.15.170:0/67068903 >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 0x7fe17c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 -- 172.21.15.170:0/67068903 >> v1:172.21.15.191:6789/0 conn(0x7fe17c15d230 legacy=0x7fe17c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe1812ba700 1 -- 172.21.15.170:0/67068903 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0b1a70 con 0x7fe17c1597f0 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.611+0000 7fe181abb700 1 --2- 172.21.15.170:0/67068903 >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 0x7fe17c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe1812ba700 1 --2- 172.21.15.170:0/67068903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c0b2270 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0b2f20 tx=0x7fe16c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6462804c6c21a92e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe180ab9700 1 -- 172.21.15.170:0/67068903 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00b820 con 0x7fe17c1597f0 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe180ab9700 1 -- 172.21.15.170:0/67068903 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c00b980 con 0x7fe17c1597f0 2023-10-08T16:11:51.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe180ab9700 1 -- 172.21.15.170:0/67068903 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00bd20 con 0x7fe17c1597f0 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 -- 172.21.15.170:0/67068903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 msgr2=0x7fe17c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 --2- 172.21.15.170:0/67068903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c0b2270 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0b2f20 tx=0x7fe16c009250 comp rx=0 tx=0).stop 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 -- 172.21.15.170:0/67068903 shutdown_connections 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 --2- 172.21.15.170:0/67068903 >> v2:172.21.15.191:3300/0 conn(0x7fe17c159420 0x7fe17c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 --2- 172.21.15.170:0/67068903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c0b2270 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 -- 172.21.15.170:0/67068903 >> 172.21.15.170:0/67068903 conn(0x7fe17c0d0660 msgr2=0x7fe17c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 -- 172.21.15.170:0/67068903 shutdown_connections 2023-10-08T16:11:51.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.612+0000 7fe182abd700 1 -- 172.21.15.170:0/67068903 wait complete. 2023-10-08T16:11:51.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.613+0000 7fe182abd700 1 Processor -- start 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 -- start start 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe17c159420 0x7fe17c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe17c15d230 0x7fe17c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15c200 con 0x7fe17c1597f0 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1534a0 con 0x7fe17c15d230 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe182abd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15b5f0 con 0x7fe17c159420 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe1812ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe1812ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43382/0 (socket says 172.21.15.170:43382) 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe1812ba700 1 -- 172.21.15.170:0/1131617586 learned_addr learned my addr 172.21.15.170:0/1131617586 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:51.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe181abb700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe17c159420 0x7fe17c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.614+0000 7fe1822bc700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe17c15d230 0x7fe17c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 -- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe17c159420 msgr2=0x7fe17c152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe17c159420 0x7fe17c152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 -- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe17c15d230 msgr2=0x7fe17c15ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe17c15d230 0x7fe17c15ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 -- 172.21.15.170:0/1131617586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c0094f0 con 0x7fe17c1597f0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe1812ba700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009290 tx=0x7fe16c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c011cc0 con 0x7fe17c1597f0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c020070 con 0x7fe17c1597f0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c15b8d0 con 0x7fe17c1597f0 2023-10-08T16:11:51.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c01b6a0 con 0x7fe17c1597f0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.615+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c15be90 con 0x7fe17c1597f0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.616+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c04c3c0 con 0x7fe17c1597f0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.616+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe16c022030 con 0x7fe17c1597f0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.617+0000 7fe171ffb700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 0x7fe1600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.617+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe16c086a10 con 0x7fe17c1597f0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.617+0000 7fe181abb700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 0x7fe1600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.617+0000 7fe181abb700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 0x7fe1600697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d6e0 tx=0x7fe178007850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.621+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe16c0537b0 con 0x7fe17c1597f0 2023-10-08T16:11:51.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.799+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe17c04c3c0 con 0x7fe160067320 2023-10-08T16:11:51.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.807+0000 7fe171ffb700 1 -- 172.21.15.170:0/1131617586 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289607 (secure 0 0 0) 0x7fe17c04c3c0 con 0x7fe160067320 2023-10-08T16:11:51.810 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 msgr2=0x7fe1600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 0x7fe1600697d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d6e0 tx=0x7fe178007850 comp rx=0 tx=0).stop 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 msgr2=0x7fe17c15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009290 tx=0x7fe16c016000 comp rx=0 tx=0).stop 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 shutdown_connections 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe17c15d230 0x7fe17c15ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe17c159420 0x7fe17c152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe160067320 0x7fe1600697d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 --2- 172.21.15.170:0/1131617586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe17c1597f0 0x7fe17c15c690 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 >> 172.21.15.170:0/1131617586 conn(0x7fe17c0d0660 msgr2=0x7fe17c153b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.810+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 shutdown_connections 2023-10-08T16:11:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.811+0000 7fe182abd700 1 -- 172.21.15.170:0/1131617586 wait complete. 2023-10-08T16:11:51.814 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:51.823 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-10-08T16:11:50.312711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425976,"kb_used_data":14904,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548744,"statfs":{"total":773094113280,"available":770609913856,"internally_reserved":0,"allocated":15261696,"data_stored":7447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548289"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806752+0000","last_change":"2023-10-08T16:11:47.791142+0000","last_active":"2023-10-08T16:11:47.806752+0000","last_peered":"2023-10-08T16:11:47.806752+0000","last_clean":"2023-10-08T16:11:47.806752+0000","last_became_active":"2023-10-08T16:11:47.790340+0000","last_became_peered":"2023-10-08T16:11:47.790340+0000","last_unstale":"2023-10-08T16:11:47.806752+0000","last_undegraded":"2023-10-08T16:11:47.806752+0000","last_fullsized":"2023-10-08T16:11:47.806752+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794856+0000","last_change":"2023-10-08T16:11:47.787461+0000","last_active":"2023-10-08T16:11:47.794856+0000","last_peered":"2023-10-08T16:11:47.794856+0000","last_clean":"2023-10-08T16:11:47.794856+0000","last_became_active":"2023-10-08T16:11:47.787158+0000","last_became_peered":"2023-10-08T16:11:47.787158+0000","last_unstale":"2023-10-08T16:11:47.794856+0000","last_undegraded":"2023-10-08T16:11:47.794856+0000","last_fullsized":"2023-10-08T16:11:47.794856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:52.224326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796229+0000","last_change":"2023-10-08T16:11:47.792439+0000","last_active":"2023-10-08T16:11:47.796229+0000","last_peered":"2023-10-08T16:11:47.796229+0000","last_clean":"2023-10-08T16:11:47.796229+0000","last_became_active":"2023-10-08T16:11:47.792177+0000","last_became_peered":"2023-10-08T16:11:47.792177+0000","last_unstale":"2023-10-08T16:11:47.796229+0000","last_undegraded":"2023-10-08T16:11:47.796229+0000","last_fullsized":"2023-10-08T16:11:47.796229+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:53.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795211+0000","last_change":"2023-10-08T16:11:47.788835+0000","last_active":"2023-10-08T16:11:47.795211+0000","last_peered":"2023-10-08T16:11:47.795211+0000","last_clean":"2023-10-08T16:11:47.795211+0000","last_became_active":"2023-10-08T16:11:47.787243+0000","last_became_peered":"2023-10-08T16:11:47.787243+0000","last_unstale":"2023-10-08T16:11:47.795211+0000","last_undegraded":"2023-10-08T16:11:47.795211+0000","last_fullsized":"2023-10-08T16:11:47.795211+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:05:17.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795807+0000","last_change":"2023-10-08T16:11:47.792354+0000","last_active":"2023-10-08T16:11:47.795807+0000","last_peered":"2023-10-08T16:11:47.795807+0000","last_clean":"2023-10-08T16:11:47.795807+0000","last_became_active":"2023-10-08T16:11:47.792060+0000","last_became_peered":"2023-10-08T16:11:47.792060+0000","last_unstale":"2023-10-08T16:11:47.795807+0000","last_undegraded":"2023-10-08T16:11:47.795807+0000","last_fullsized":"2023-10-08T16:11:47.795807+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:14:15.112013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794044+0000","last_change":"2023-10-08T16:11:47.791030+0000","last_active":"2023-10-08T16:11:47.794044+0000","last_peered":"2023-10-08T16:11:47.794044+0000","last_clean":"2023-10-08T16:11:47.794044+0000","last_became_active":"2023-10-08T16:11:47.790800+0000","last_became_peered":"2023-10-08T16:11:47.790800+0000","last_unstale":"2023-10-08T16:11:47.794044+0000","last_undegraded":"2023-10-08T16:11:47.794044+0000","last_fullsized":"2023-10-08T16:11:47.794044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:36:15.948835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799321+0000","last_change":"2023-10-08T16:11:47.792384+0000","last_active":"2023-10-08T16:11:47.799321+0000","last_peered":"2023-10-08T16:11:47.799321+0000","last_clean":"2023-10-08T16:11:47.799321+0000","last_became_active":"2023-10-08T16:11:47.791926+0000","last_became_peered":"2023-10-08T16:11:47.791926+0000","last_unstale":"2023-10-08T16:11:47.799321+0000","last_undegraded":"2023-10-08T16:11:47.799321+0000","last_fullsized":"2023-10-08T16:11:47.799321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:46:09.131918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798192+0000","last_change":"2023-10-08T16:11:47.797252+0000","last_active":"2023-10-08T16:11:47.798192+0000","last_peered":"2023-10-08T16:11:47.798192+0000","last_clean":"2023-10-08T16:11:47.798192+0000","last_became_active":"2023-10-08T16:11:47.796759+0000","last_became_peered":"2023-10-08T16:11:47.796759+0000","last_unstale":"2023-10-08T16:11:47.798192+0000","last_undegraded":"2023-10-08T16:11:47.798192+0000","last_fullsized":"2023-10-08T16:11:47.798192+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806472+0000","last_change":"2023-10-08T16:11:47.795870+0000","last_active":"2023-10-08T16:11:47.806472+0000","last_peered":"2023-10-08T16:11:47.806472+0000","last_clean":"2023-10-08T16:11:47.806472+0000","last_became_active":"2023-10-08T16:11:47.795624+0000","last_became_peered":"2023-10-08T16:11:47.795624+0000","last_unstale":"2023-10-08T16:11:47.806472+0000","last_undegraded":"2023-10-08T16:11:47.806472+0000","last_fullsized":"2023-10-08T16:11:47.806472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:58.472655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787289+0000","last_change":"2023-10-08T16:11:47.784233+0000","last_active":"2023-10-08T16:11:47.787289+0000","last_peered":"2023-10-08T16:11:47.787289+0000","last_clean":"2023-10-08T16:11:47.787289+0000","last_became_active":"2023-10-08T16:11:47.783992+0000","last_became_peered":"2023-10-08T16:11:47.783992+0000","last_unstale":"2023-10-08T16:11:47.787289+0000","last_undegraded":"2023-10-08T16:11:47.787289+0000","last_fullsized":"2023-10-08T16:11:47.787289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:20.354451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801175+0000","last_change":"2023-10-08T16:11:47.796160+0000","last_active":"2023-10-08T16:11:47.801175+0000","last_peered":"2023-10-08T16:11:47.801175+0000","last_clean":"2023-10-08T16:11:47.801175+0000","last_became_active":"2023-10-08T16:11:47.795902+0000","last_became_peered":"2023-10-08T16:11:47.795902+0000","last_unstale":"2023-10-08T16:11:47.801175+0000","last_undegraded":"2023-10-08T16:11:47.801175+0000","last_fullsized":"2023-10-08T16:11:47.801175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:43.673722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794848+0000","last_change":"2023-10-08T16:11:47.787350+0000","last_active":"2023-10-08T16:11:47.794848+0000","last_peered":"2023-10-08T16:11:47.794848+0000","last_clean":"2023-10-08T16:11:47.794848+0000","last_became_active":"2023-10-08T16:11:47.786923+0000","last_became_peered":"2023-10-08T16:11:47.786923+0000","last_unstale":"2023-10-08T16:11:47.794848+0000","last_undegraded":"2023-10-08T16:11:47.794848+0000","last_fullsized":"2023-10-08T16:11:47.794848+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:48.957415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801226+0000","last_change":"2023-10-08T16:11:47.797763+0000","last_active":"2023-10-08T16:11:47.801226+0000","last_peered":"2023-10-08T16:11:47.801226+0000","last_clean":"2023-10-08T16:11:47.801226+0000","last_became_active":"2023-10-08T16:11:47.797463+0000","last_became_peered":"2023-10-08T16:11:47.797463+0000","last_unstale":"2023-10-08T16:11:47.801226+0000","last_undegraded":"2023-10-08T16:11:47.801226+0000","last_fullsized":"2023-10-08T16:11:47.801226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:53.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806407+0000","last_change":"2023-10-08T16:11:47.794341+0000","last_active":"2023-10-08T16:11:47.806407+0000","last_peered":"2023-10-08T16:11:47.806407+0000","last_clean":"2023-10-08T16:11:47.806407+0000","last_became_active":"2023-10-08T16:11:47.794007+0000","last_became_peered":"2023-10-08T16:11:47.794007+0000","last_unstale":"2023-10-08T16:11:47.806407+0000","last_undegraded":"2023-10-08T16:11:47.806407+0000","last_fullsized":"2023-10-08T16:11:47.806407+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:04.512790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798062+0000","last_change":"2023-10-08T16:11:47.793050+0000","last_active":"2023-10-08T16:11:47.798062+0000","last_peered":"2023-10-08T16:11:47.798062+0000","last_clean":"2023-10-08T16:11:47.798062+0000","last_became_active":"2023-10-08T16:11:47.792781+0000","last_became_peered":"2023-10-08T16:11:47.792781+0000","last_unstale":"2023-10-08T16:11:47.798062+0000","last_undegraded":"2023-10-08T16:11:47.798062+0000","last_fullsized":"2023-10-08T16:11:47.798062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:44.453346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796813+0000","last_change":"2023-10-08T16:11:47.794967+0000","last_active":"2023-10-08T16:11:47.796813+0000","last_peered":"2023-10-08T16:11:47.796813+0000","last_clean":"2023-10-08T16:11:47.796813+0000","last_became_active":"2023-10-08T16:11:47.794738+0000","last_became_peered":"2023-10-08T16:11:47.794738+0000","last_unstale":"2023-10-08T16:11:47.796813+0000","last_undegraded":"2023-10-08T16:11:47.796813+0000","last_fullsized":"2023-10-08T16:11:47.796813+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:46.504886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795186+0000","last_change":"2023-10-08T16:11:47.788280+0000","last_active":"2023-10-08T16:11:47.795186+0000","last_peered":"2023-10-08T16:11:47.795186+0000","last_clean":"2023-10-08T16:11:47.795186+0000","last_became_active":"2023-10-08T16:11:47.788004+0000","last_became_peered":"2023-10-08T16:11:47.788004+0000","last_unstale":"2023-10-08T16:11:47.795186+0000","last_undegraded":"2023-10-08T16:11:47.795186+0000","last_fullsized":"2023-10-08T16:11:47.795186+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:57.567976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807193+0000","last_change":"2023-10-08T16:11:47.796583+0000","last_active":"2023-10-08T16:11:47.807193+0000","last_peered":"2023-10-08T16:11:47.807193+0000","last_clean":"2023-10-08T16:11:47.807193+0000","last_became_active":"2023-10-08T16:11:47.796297+0000","last_became_peered":"2023-10-08T16:11:47.796297+0000","last_unstale":"2023-10-08T16:11:47.807193+0000","last_undegraded":"2023-10-08T16:11:47.807193+0000","last_fullsized":"2023-10-08T16:11:47.807193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:00.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796441+0000","last_change":"2023-10-08T16:11:47.794993+0000","last_active":"2023-10-08T16:11:47.796441+0000","last_peered":"2023-10-08T16:11:47.796441+0000","last_clean":"2023-10-08T16:11:47.796441+0000","last_became_active":"2023-10-08T16:11:47.794748+0000","last_became_peered":"2023-10-08T16:11:47.794748+0000","last_unstale":"2023-10-08T16:11:47.796441+0000","last_undegraded":"2023-10-08T16:11:47.796441+0000","last_fullsized":"2023-10-08T16:11:47.796441+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:43.732366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795991+0000","last_change":"2023-10-08T16:11:47.795538+0000","last_active":"2023-10-08T16:11:47.795991+0000","last_peered":"2023-10-08T16:11:47.795991+0000","last_clean":"2023-10-08T16:11:47.795991+0000","last_became_active":"2023-10-08T16:11:47.795319+0000","last_became_peered":"2023-10-08T16:11:47.795319+0000","last_unstale":"2023-10-08T16:11:47.795991+0000","last_undegraded":"2023-10-08T16:11:47.795991+0000","last_fullsized":"2023-10-08T16:11:47.795991+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:35.302393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795787+0000","last_change":"2023-10-08T16:11:47.792318+0000","last_active":"2023-10-08T16:11:47.795787+0000","last_peered":"2023-10-08T16:11:47.795787+0000","last_clean":"2023-10-08T16:11:47.795787+0000","last_became_active":"2023-10-08T16:11:47.791802+0000","last_became_peered":"2023-10-08T16:11:47.791802+0000","last_unstale":"2023-10-08T16:11:47.795787+0000","last_undegraded":"2023-10-08T16:11:47.795787+0000","last_fullsized":"2023-10-08T16:11:47.795787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:09.190734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787116+0000","last_change":"2023-10-08T16:11:47.785313+0000","last_active":"2023-10-08T16:11:47.787116+0000","last_peered":"2023-10-08T16:11:47.787116+0000","last_clean":"2023-10-08T16:11:47.787116+0000","last_became_active":"2023-10-08T16:11:47.784754+0000","last_became_peered":"2023-10-08T16:11:47.784754+0000","last_unstale":"2023-10-08T16:11:47.787116+0000","last_undegraded":"2023-10-08T16:11:47.787116+0000","last_fullsized":"2023-10-08T16:11:47.787116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:40.641363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792889+0000","last_change":"2023-10-08T16:11:47.789615+0000","last_active":"2023-10-08T16:11:47.792889+0000","last_peered":"2023-10-08T16:11:47.792889+0000","last_clean":"2023-10-08T16:11:47.792889+0000","last_became_active":"2023-10-08T16:11:47.789382+0000","last_became_peered":"2023-10-08T16:11:47.789382+0000","last_unstale":"2023-10-08T16:11:47.792889+0000","last_undegraded":"2023-10-08T16:11:47.792889+0000","last_fullsized":"2023-10-08T16:11:47.792889+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:33.715587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.777107+0000","last_change":"2023-10-08T16:11:47.776665+0000","last_active":"2023-10-08T16:11:47.777107+0000","last_peered":"2023-10-08T16:11:47.777107+0000","last_clean":"2023-10-08T16:11:47.777107+0000","last_became_active":"2023-10-08T16:11:47.776413+0000","last_became_peered":"2023-10-08T16:11:47.776413+0000","last_unstale":"2023-10-08T16:11:47.777107+0000","last_undegraded":"2023-10-08T16:11:47.777107+0000","last_fullsized":"2023-10-08T16:11:47.777107+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:27:35.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795099+0000","last_change":"2023-10-08T16:11:47.788253+0000","last_active":"2023-10-08T16:11:47.795099+0000","last_peered":"2023-10-08T16:11:47.795099+0000","last_clean":"2023-10-08T16:11:47.795099+0000","last_became_active":"2023-10-08T16:11:47.788030+0000","last_became_peered":"2023-10-08T16:11:47.788030+0000","last_unstale":"2023-10-08T16:11:47.795099+0000","last_undegraded":"2023-10-08T16:11:47.795099+0000","last_fullsized":"2023-10-08T16:11:47.795099+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:12.205907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798073+0000","last_change":"2023-10-08T16:11:47.791649+0000","last_active":"2023-10-08T16:11:47.798073+0000","last_peered":"2023-10-08T16:11:47.798073+0000","last_clean":"2023-10-08T16:11:47.798073+0000","last_became_active":"2023-10-08T16:11:47.791418+0000","last_became_peered":"2023-10-08T16:11:47.791418+0000","last_unstale":"2023-10-08T16:11:47.798073+0000","last_undegraded":"2023-10-08T16:11:47.798073+0000","last_fullsized":"2023-10-08T16:11:47.798073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:43.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792851+0000","last_change":"2023-10-08T16:11:47.789429+0000","last_active":"2023-10-08T16:11:47.792851+0000","last_peered":"2023-10-08T16:11:47.792851+0000","last_clean":"2023-10-08T16:11:47.792851+0000","last_became_active":"2023-10-08T16:11:47.788973+0000","last_became_peered":"2023-10-08T16:11:47.788973+0000","last_unstale":"2023-10-08T16:11:47.792851+0000","last_undegraded":"2023-10-08T16:11:47.792851+0000","last_fullsized":"2023-10-08T16:11:47.792851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:43.673722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786172+0000","last_change":"2023-10-08T16:11:47.785028+0000","last_active":"2023-10-08T16:11:47.786172+0000","last_peered":"2023-10-08T16:11:47.786172+0000","last_clean":"2023-10-08T16:11:47.786172+0000","last_became_active":"2023-10-08T16:11:47.784770+0000","last_became_peered":"2023-10-08T16:11:47.784770+0000","last_unstale":"2023-10-08T16:11:47.786172+0000","last_undegraded":"2023-10-08T16:11:47.786172+0000","last_fullsized":"2023-10-08T16:11:47.786172+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:07.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798197+0000","last_change":"2023-10-08T16:11:47.796004+0000","last_active":"2023-10-08T16:11:47.798197+0000","last_peered":"2023-10-08T16:11:47.798197+0000","last_clean":"2023-10-08T16:11:47.798197+0000","last_became_active":"2023-10-08T16:11:47.795761+0000","last_became_peered":"2023-10-08T16:11:47.795761+0000","last_unstale":"2023-10-08T16:11:47.798197+0000","last_undegraded":"2023-10-08T16:11:47.798197+0000","last_fullsized":"2023-10-08T16:11:47.798197+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:30:56.356410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807239+0000","last_change":"2023-10-08T16:11:47.804605+0000","last_active":"2023-10-08T16:11:47.807239+0000","last_peered":"2023-10-08T16:11:47.807239+0000","last_clean":"2023-10-08T16:11:47.807239+0000","last_became_active":"2023-10-08T16:11:47.804308+0000","last_became_peered":"2023-10-08T16:11:47.804308+0000","last_unstale":"2023-10-08T16:11:47.807239+0000","last_undegraded":"2023-10-08T16:11:47.807239+0000","last_fullsized":"2023-10-08T16:11:47.807239+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:44.453346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.789692+0000","last_change":"2023-10-08T16:11:47.788468+0000","last_active":"2023-10-08T16:11:47.789692+0000","last_peered":"2023-10-08T16:11:47.789692+0000","last_clean":"2023-10-08T16:11:47.789692+0000","last_became_active":"2023-10-08T16:11:47.788222+0000","last_became_peered":"2023-10-08T16:11:47.788222+0000","last_unstale":"2023-10-08T16:11:47.789692+0000","last_undegraded":"2023-10-08T16:11:47.789692+0000","last_fullsized":"2023-10-08T16:11:47.789692+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:43.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796634+0000","last_change":"2023-10-08T16:11:47.796422+0000","last_active":"2023-10-08T16:11:47.796634+0000","last_peered":"2023-10-08T16:11:47.796634+0000","last_clean":"2023-10-08T16:11:47.796634+0000","last_became_active":"2023-10-08T16:11:47.796198+0000","last_became_peered":"2023-10-08T16:11:47.796198+0000","last_unstale":"2023-10-08T16:11:47.796634+0000","last_undegraded":"2023-10-08T16:11:47.796634+0000","last_fullsized":"2023-10-08T16:11:47.796634+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:05.805765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.797848+0000","last_change":"2023-10-08T16:11:47.795840+0000","last_active":"2023-10-08T16:11:47.797848+0000","last_peered":"2023-10-08T16:11:47.797848+0000","last_clean":"2023-10-08T16:11:47.797848+0000","last_became_active":"2023-10-08T16:11:47.795619+0000","last_became_peered":"2023-10-08T16:11:47.795619+0000","last_unstale":"2023-10-08T16:11:47.797848+0000","last_undegraded":"2023-10-08T16:11:47.797848+0000","last_fullsized":"2023-10-08T16:11:47.797848+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:21:34.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799284+0000","last_change":"2023-10-08T16:11:46.789501+0000","last_active":"2023-10-08T16:11:47.799284+0000","last_peered":"2023-10-08T16:11:47.799284+0000","last_clean":"2023-10-08T16:11:47.799284+0000","last_became_active":"2023-10-08T16:11:46.789223+0000","last_became_peered":"2023-10-08T16:11:46.789223+0000","last_unstale":"2023-10-08T16:11:47.799284+0000","last_undegraded":"2023-10-08T16:11:47.799284+0000","last_fullsized":"2023-10-08T16:11:47.799284+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:29.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792957+0000","last_change":"2023-10-08T16:11:47.787336+0000","last_active":"2023-10-08T16:11:47.792957+0000","last_peered":"2023-10-08T16:11:47.792957+0000","last_clean":"2023-10-08T16:11:47.792957+0000","last_became_active":"2023-10-08T16:11:47.786963+0000","last_became_peered":"2023-10-08T16:11:47.786963+0000","last_unstale":"2023-10-08T16:11:47.792957+0000","last_undegraded":"2023-10-08T16:11:47.792957+0000","last_fullsized":"2023-10-08T16:11:47.792957+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:47.748612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794578+0000","last_change":"2023-10-08T16:11:46.785401+0000","last_active":"2023-10-08T16:11:47.794578+0000","last_peered":"2023-10-08T16:11:47.794578+0000","last_clean":"2023-10-08T16:11:47.794578+0000","last_became_active":"2023-10-08T16:11:46.785115+0000","last_became_peered":"2023-10-08T16:11:46.785115+0000","last_unstale":"2023-10-08T16:11:47.794578+0000","last_undegraded":"2023-10-08T16:11:47.794578+0000","last_fullsized":"2023-10-08T16:11:47.794578+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:13:46.016861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.803878+0000","last_change":"2023-10-08T16:11:47.798633+0000","last_active":"2023-10-08T16:11:47.803878+0000","last_peered":"2023-10-08T16:11:47.803878+0000","last_clean":"2023-10-08T16:11:47.803878+0000","last_became_active":"2023-10-08T16:11:47.798322+0000","last_became_peered":"2023-10-08T16:11:47.798322+0000","last_unstale":"2023-10-08T16:11:47.803878+0000","last_undegraded":"2023-10-08T16:11:47.803878+0000","last_fullsized":"2023-10-08T16:11:47.803878+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:53.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794091+0000","last_change":"2023-10-08T16:11:46.791077+0000","last_active":"2023-10-08T16:11:47.794091+0000","last_peered":"2023-10-08T16:11:47.794091+0000","last_clean":"2023-10-08T16:11:47.794091+0000","last_became_active":"2023-10-08T16:11:46.790732+0000","last_became_peered":"2023-10-08T16:11:46.790732+0000","last_unstale":"2023-10-08T16:11:47.794091+0000","last_undegraded":"2023-10-08T16:11:47.794091+0000","last_fullsized":"2023-10-08T16:11:47.794091+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:58.331890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806614+0000","last_change":"2023-10-08T16:11:47.797444+0000","last_active":"2023-10-08T16:11:47.806614+0000","last_peered":"2023-10-08T16:11:47.806614+0000","last_clean":"2023-10-08T16:11:47.806614+0000","last_became_active":"2023-10-08T16:11:47.797196+0000","last_became_peered":"2023-10-08T16:11:47.797196+0000","last_unstale":"2023-10-08T16:11:47.806614+0000","last_undegraded":"2023-10-08T16:11:47.806614+0000","last_fullsized":"2023-10-08T16:11:47.806614+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:19.039859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807179+0000","last_change":"2023-10-08T16:11:46.787321+0000","last_active":"2023-10-08T16:11:47.807179+0000","last_peered":"2023-10-08T16:11:47.807179+0000","last_clean":"2023-10-08T16:11:47.807179+0000","last_became_active":"2023-10-08T16:11:46.786924+0000","last_became_peered":"2023-10-08T16:11:46.786924+0000","last_unstale":"2023-10-08T16:11:47.807179+0000","last_undegraded":"2023-10-08T16:11:47.807179+0000","last_fullsized":"2023-10-08T16:11:47.807179+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:56.561267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786648+0000","last_change":"2023-10-08T16:11:47.783278+0000","last_active":"2023-10-08T16:11:47.786648+0000","last_peered":"2023-10-08T16:11:47.786648+0000","last_clean":"2023-10-08T16:11:47.786648+0000","last_became_active":"2023-10-08T16:11:47.783024+0000","last_became_peered":"2023-10-08T16:11:47.783024+0000","last_unstale":"2023-10-08T16:11:47.786648+0000","last_undegraded":"2023-10-08T16:11:47.786648+0000","last_fullsized":"2023-10-08T16:11:47.786648+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795253+0000","last_change":"2023-10-08T16:11:46.785437+0000","last_active":"2023-10-08T16:11:47.795253+0000","last_peered":"2023-10-08T16:11:47.795253+0000","last_clean":"2023-10-08T16:11:47.795253+0000","last_became_active":"2023-10-08T16:11:46.785184+0000","last_became_peered":"2023-10-08T16:11:46.785184+0000","last_unstale":"2023-10-08T16:11:47.795253+0000","last_undegraded":"2023-10-08T16:11:47.795253+0000","last_fullsized":"2023-10-08T16:11:47.795253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:11.184537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.797787+0000","last_change":"2023-10-08T16:11:47.795058+0000","last_active":"2023-10-08T16:11:47.797787+0000","last_peered":"2023-10-08T16:11:47.797787+0000","last_clean":"2023-10-08T16:11:47.797787+0000","last_became_active":"2023-10-08T16:11:47.794654+0000","last_became_peered":"2023-10-08T16:11:47.794654+0000","last_unstale":"2023-10-08T16:11:47.797787+0000","last_undegraded":"2023-10-08T16:11:47.797787+0000","last_fullsized":"2023-10-08T16:11:47.797787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:03.766135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796012+0000","last_change":"2023-10-08T16:11:46.784141+0000","last_active":"2023-10-08T16:11:47.796012+0000","last_peered":"2023-10-08T16:11:47.796012+0000","last_clean":"2023-10-08T16:11:47.796012+0000","last_became_active":"2023-10-08T16:11:46.783870+0000","last_became_peered":"2023-10-08T16:11:46.783870+0000","last_unstale":"2023-10-08T16:11:47.796012+0000","last_undegraded":"2023-10-08T16:11:47.796012+0000","last_fullsized":"2023-10-08T16:11:47.796012+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:09.035001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799334+0000","last_change":"2023-10-08T16:11:47.793434+0000","last_active":"2023-10-08T16:11:47.799334+0000","last_peered":"2023-10-08T16:11:47.799334+0000","last_clean":"2023-10-08T16:11:47.799334+0000","last_became_active":"2023-10-08T16:11:47.791899+0000","last_became_peered":"2023-10-08T16:11:47.791899+0000","last_unstale":"2023-10-08T16:11:47.799334+0000","last_undegraded":"2023-10-08T16:11:47.799334+0000","last_fullsized":"2023-10-08T16:11:47.799334+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:33.573488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778779+0000","last_change":"2023-10-08T16:11:46.785300+0000","last_active":"2023-10-08T16:11:47.778779+0000","last_peered":"2023-10-08T16:11:47.778779+0000","last_clean":"2023-10-08T16:11:47.778779+0000","last_became_active":"2023-10-08T16:11:46.785023+0000","last_became_peered":"2023-10-08T16:11:46.785023+0000","last_unstale":"2023-10-08T16:11:47.778779+0000","last_undegraded":"2023-10-08T16:11:47.778779+0000","last_fullsized":"2023-10-08T16:11:47.778779+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:31.724283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778449+0000","last_change":"2023-10-08T16:11:47.775923+0000","last_active":"2023-10-08T16:11:47.778449+0000","last_peered":"2023-10-08T16:11:47.778449+0000","last_clean":"2023-10-08T16:11:47.778449+0000","last_became_active":"2023-10-08T16:11:47.775652+0000","last_became_peered":"2023-10-08T16:11:47.775652+0000","last_unstale":"2023-10-08T16:11:47.778449+0000","last_undegraded":"2023-10-08T16:11:47.778449+0000","last_fullsized":"2023-10-08T16:11:47.778449+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:33.573488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778152+0000","last_change":"2023-10-08T16:11:46.782694+0000","last_active":"2023-10-08T16:11:47.778152+0000","last_peered":"2023-10-08T16:11:47.778152+0000","last_clean":"2023-10-08T16:11:47.778152+0000","last_became_active":"2023-10-08T16:11:46.782446+0000","last_became_peered":"2023-10-08T16:11:46.782446+0000","last_unstale":"2023-10-08T16:11:47.778152+0000","last_undegraded":"2023-10-08T16:11:47.778152+0000","last_fullsized":"2023-10-08T16:11:47.778152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:39.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794956+0000","last_change":"2023-10-08T16:11:47.790850+0000","last_active":"2023-10-08T16:11:47.794956+0000","last_peered":"2023-10-08T16:11:47.794956+0000","last_clean":"2023-10-08T16:11:47.794956+0000","last_became_active":"2023-10-08T16:11:47.787554+0000","last_became_peered":"2023-10-08T16:11:47.787554+0000","last_unstale":"2023-10-08T16:11:47.794956+0000","last_undegraded":"2023-10-08T16:11:47.794956+0000","last_fullsized":"2023-10-08T16:11:47.794956+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.777698+0000","last_change":"2023-10-08T16:11:46.787439+0000","last_active":"2023-10-08T16:11:47.777698+0000","last_peered":"2023-10-08T16:11:47.777698+0000","last_clean":"2023-10-08T16:11:47.777698+0000","last_became_active":"2023-10-08T16:11:46.787155+0000","last_became_peered":"2023-10-08T16:11:46.787155+0000","last_unstale":"2023-10-08T16:11:47.777698+0000","last_undegraded":"2023-10-08T16:11:47.777698+0000","last_fullsized":"2023-10-08T16:11:47.777698+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:30.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.803857+0000","last_change":"2023-10-08T16:11:47.798323+0000","last_active":"2023-10-08T16:11:47.803857+0000","last_peered":"2023-10-08T16:11:47.803857+0000","last_clean":"2023-10-08T16:11:47.803857+0000","last_became_active":"2023-10-08T16:11:47.797802+0000","last_became_peered":"2023-10-08T16:11:47.797802+0000","last_unstale":"2023-10-08T16:11:47.803857+0000","last_undegraded":"2023-10-08T16:11:47.803857+0000","last_fullsized":"2023-10-08T16:11:47.803857+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:33.715587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795368+0000","last_change":"2023-10-08T16:11:46.785257+0000","last_active":"2023-10-08T16:11:47.795368+0000","last_peered":"2023-10-08T16:11:47.795368+0000","last_clean":"2023-10-08T16:11:47.795368+0000","last_became_active":"2023-10-08T16:11:46.785006+0000","last_became_peered":"2023-10-08T16:11:46.785006+0000","last_unstale":"2023-10-08T16:11:47.795368+0000","last_undegraded":"2023-10-08T16:11:47.795368+0000","last_fullsized":"2023-10-08T16:11:47.795368+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.314888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786858+0000","last_change":"2023-10-08T16:11:47.783690+0000","last_active":"2023-10-08T16:11:47.786858+0000","last_peered":"2023-10-08T16:11:47.786858+0000","last_clean":"2023-10-08T16:11:47.786858+0000","last_became_active":"2023-10-08T16:11:47.782652+0000","last_became_peered":"2023-10-08T16:11:47.782652+0000","last_unstale":"2023-10-08T16:11:47.786858+0000","last_undegraded":"2023-10-08T16:11:47.786858+0000","last_fullsized":"2023-10-08T16:11:47.786858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:10:37.538839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807440+0000","last_change":"2023-10-08T16:11:46.781686+0000","last_active":"2023-10-08T16:11:47.807440+0000","last_peered":"2023-10-08T16:11:47.807440+0000","last_clean":"2023-10-08T16:11:47.807440+0000","last_became_active":"2023-10-08T16:11:46.781440+0000","last_became_peered":"2023-10-08T16:11:46.781440+0000","last_unstale":"2023-10-08T16:11:47.807440+0000","last_undegraded":"2023-10-08T16:11:47.807440+0000","last_fullsized":"2023-10-08T16:11:47.807440+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:18.937283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.791388+0000","last_change":"2023-10-08T16:11:47.789178+0000","last_active":"2023-10-08T16:11:47.791388+0000","last_peered":"2023-10-08T16:11:47.791388+0000","last_clean":"2023-10-08T16:11:47.791388+0000","last_became_active":"2023-10-08T16:11:47.788882+0000","last_became_peered":"2023-10-08T16:11:47.788882+0000","last_unstale":"2023-10-08T16:11:47.791388+0000","last_undegraded":"2023-10-08T16:11:47.791388+0000","last_fullsized":"2023-10-08T16:11:47.791388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:39.175148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807168+0000","last_change":"2023-10-08T16:11:46.785794+0000","last_active":"2023-10-08T16:11:47.807168+0000","last_peered":"2023-10-08T16:11:47.807168+0000","last_clean":"2023-10-08T16:11:47.807168+0000","last_became_active":"2023-10-08T16:11:46.785555+0000","last_became_peered":"2023-10-08T16:11:46.785555+0000","last_unstale":"2023-10-08T16:11:47.807168+0000","last_undegraded":"2023-10-08T16:11:47.807168+0000","last_fullsized":"2023-10-08T16:11:47.807168+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:29.580179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801112+0000","last_change":"2023-10-08T16:11:47.797147+0000","last_active":"2023-10-08T16:11:47.801112+0000","last_peered":"2023-10-08T16:11:47.801112+0000","last_clean":"2023-10-08T16:11:47.801112+0000","last_became_active":"2023-10-08T16:11:47.796766+0000","last_became_peered":"2023-10-08T16:11:47.796766+0000","last_unstale":"2023-10-08T16:11:47.801112+0000","last_undegraded":"2023-10-08T16:11:47.801112+0000","last_fullsized":"2023-10-08T16:11:47.801112+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:55:53.872812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782126+0000","last_change":"2023-10-08T16:11:46.788116+0000","last_active":"2023-10-08T16:11:47.782126+0000","last_peered":"2023-10-08T16:11:47.782126+0000","last_clean":"2023-10-08T16:11:47.782126+0000","last_became_active":"2023-10-08T16:11:46.787871+0000","last_became_peered":"2023-10-08T16:11:46.787871+0000","last_unstale":"2023-10-08T16:11:47.782126+0000","last_undegraded":"2023-10-08T16:11:47.782126+0000","last_fullsized":"2023-10-08T16:11:47.782126+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801085+0000","last_change":"2023-10-08T16:11:47.792912+0000","last_active":"2023-10-08T16:11:47.801085+0000","last_peered":"2023-10-08T16:11:47.801085+0000","last_clean":"2023-10-08T16:11:47.801085+0000","last_became_active":"2023-10-08T16:11:47.792682+0000","last_became_peered":"2023-10-08T16:11:47.792682+0000","last_unstale":"2023-10-08T16:11:47.801085+0000","last_undegraded":"2023-10-08T16:11:47.801085+0000","last_fullsized":"2023-10-08T16:11:47.801085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:53.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.774071+0000","last_change":"2023-10-08T16:11:46.782750+0000","last_active":"2023-10-08T16:11:47.774071+0000","last_peered":"2023-10-08T16:11:47.774071+0000","last_clean":"2023-10-08T16:11:47.774071+0000","last_became_active":"2023-10-08T16:11:46.782468+0000","last_became_peered":"2023-10-08T16:11:46.782468+0000","last_unstale":"2023-10-08T16:11:47.774071+0000","last_undegraded":"2023-10-08T16:11:47.774071+0000","last_fullsized":"2023-10-08T16:11:47.774071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:39.700318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.800887+0000","last_change":"2023-10-08T16:11:47.800217+0000","last_active":"2023-10-08T16:11:47.800887+0000","last_peered":"2023-10-08T16:11:47.800887+0000","last_clean":"2023-10-08T16:11:47.800887+0000","last_became_active":"2023-10-08T16:11:47.799975+0000","last_became_peered":"2023-10-08T16:11:47.799975+0000","last_unstale":"2023-10-08T16:11:47.800887+0000","last_undegraded":"2023-10-08T16:11:47.800887+0000","last_fullsized":"2023-10-08T16:11:47.800887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:30.586888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807403+0000","last_change":"2023-10-08T16:11:46.785349+0000","last_active":"2023-10-08T16:11:47.807403+0000","last_peered":"2023-10-08T16:11:47.807403+0000","last_clean":"2023-10-08T16:11:47.807403+0000","last_became_active":"2023-10-08T16:11:46.785083+0000","last_became_peered":"2023-10-08T16:11:46.785083+0000","last_unstale":"2023-10-08T16:11:47.807403+0000","last_undegraded":"2023-10-08T16:11:47.807403+0000","last_fullsized":"2023-10-08T16:11:47.807403+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:45:16.996074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.800574+0000","last_change":"2023-10-08T16:11:47.799266+0000","last_active":"2023-10-08T16:11:47.800574+0000","last_peered":"2023-10-08T16:11:47.800574+0000","last_clean":"2023-10-08T16:11:47.800574+0000","last_became_active":"2023-10-08T16:11:47.799023+0000","last_became_peered":"2023-10-08T16:11:47.799023+0000","last_unstale":"2023-10-08T16:11:47.800574+0000","last_undegraded":"2023-10-08T16:11:47.800574+0000","last_fullsized":"2023-10-08T16:11:47.800574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:07.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795728+0000","last_change":"2023-10-08T16:11:46.790388+0000","last_active":"2023-10-08T16:11:47.795728+0000","last_peered":"2023-10-08T16:11:47.795728+0000","last_clean":"2023-10-08T16:11:47.795728+0000","last_became_active":"2023-10-08T16:11:46.790197+0000","last_became_peered":"2023-10-08T16:11:46.790197+0000","last_unstale":"2023-10-08T16:11:47.795728+0000","last_undegraded":"2023-10-08T16:11:47.795728+0000","last_fullsized":"2023-10-08T16:11:47.795728+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:28.232219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801010+0000","last_change":"2023-10-08T16:11:47.791538+0000","last_active":"2023-10-08T16:11:47.801010+0000","last_peered":"2023-10-08T16:11:47.801010+0000","last_clean":"2023-10-08T16:11:47.801010+0000","last_became_active":"2023-10-08T16:11:47.791262+0000","last_became_peered":"2023-10-08T16:11:47.791262+0000","last_unstale":"2023-10-08T16:11:47.801010+0000","last_undegraded":"2023-10-08T16:11:47.801010+0000","last_fullsized":"2023-10-08T16:11:47.801010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:05:17.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799819+0000","last_change":"2023-10-08T16:11:46.783954+0000","last_active":"2023-10-08T16:11:47.799819+0000","last_peered":"2023-10-08T16:11:47.799819+0000","last_clean":"2023-10-08T16:11:47.799819+0000","last_became_active":"2023-10-08T16:11:46.783731+0000","last_became_peered":"2023-10-08T16:11:46.783731+0000","last_unstale":"2023-10-08T16:11:47.799819+0000","last_undegraded":"2023-10-08T16:11:47.799819+0000","last_fullsized":"2023-10-08T16:11:47.799819+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:21.952273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807227+0000","last_change":"2023-10-08T16:11:47.804663+0000","last_active":"2023-10-08T16:11:47.807227+0000","last_peered":"2023-10-08T16:11:47.807227+0000","last_clean":"2023-10-08T16:11:47.807227+0000","last_became_active":"2023-10-08T16:11:47.804375+0000","last_became_peered":"2023-10-08T16:11:47.804375+0000","last_unstale":"2023-10-08T16:11:47.807227+0000","last_undegraded":"2023-10-08T16:11:47.807227+0000","last_fullsized":"2023-10-08T16:11:47.807227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:52.224326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807135+0000","last_change":"2023-10-08T16:11:46.785752+0000","last_active":"2023-10-08T16:11:47.807135+0000","last_peered":"2023-10-08T16:11:47.807135+0000","last_clean":"2023-10-08T16:11:47.807135+0000","last_became_active":"2023-10-08T16:11:46.784455+0000","last_became_peered":"2023-10-08T16:11:46.784455+0000","last_unstale":"2023-10-08T16:11:47.807135+0000","last_undegraded":"2023-10-08T16:11:47.807135+0000","last_fullsized":"2023-10-08T16:11:47.807135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.793477+0000","last_change":"2023-10-08T16:11:47.783605+0000","last_active":"2023-10-08T16:11:47.793477+0000","last_peered":"2023-10-08T16:11:47.793477+0000","last_clean":"2023-10-08T16:11:47.793477+0000","last_became_active":"2023-10-08T16:11:47.783335+0000","last_became_peered":"2023-10-08T16:11:47.783335+0000","last_unstale":"2023-10-08T16:11:47.793477+0000","last_undegraded":"2023-10-08T16:11:47.793477+0000","last_fullsized":"2023-10-08T16:11:47.793477+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:19.039859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.780019+0000","last_change":"2023-10-08T16:11:46.785071+0000","last_active":"2023-10-08T16:11:47.780019+0000","last_peered":"2023-10-08T16:11:47.780019+0000","last_clean":"2023-10-08T16:11:47.780019+0000","last_became_active":"2023-10-08T16:11:46.784793+0000","last_became_peered":"2023-10-08T16:11:46.784793+0000","last_unstale":"2023-10-08T16:11:47.780019+0000","last_undegraded":"2023-10-08T16:11:47.780019+0000","last_fullsized":"2023-10-08T16:11:47.780019+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:14.393450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787137+0000","last_change":"2023-10-08T16:11:47.785463+0000","last_active":"2023-10-08T16:11:47.787137+0000","last_peered":"2023-10-08T16:11:47.787137+0000","last_clean":"2023-10-08T16:11:47.787137+0000","last_became_active":"2023-10-08T16:11:47.785026+0000","last_became_peered":"2023-10-08T16:11:47.785026+0000","last_unstale":"2023-10-08T16:11:47.787137+0000","last_undegraded":"2023-10-08T16:11:47.787137+0000","last_fullsized":"2023-10-08T16:11:47.787137+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:32:43.670275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796152+0000","last_change":"2023-10-08T16:11:46.782131+0000","last_active":"2023-10-08T16:11:47.796152+0000","last_peered":"2023-10-08T16:11:47.796152+0000","last_clean":"2023-10-08T16:11:47.796152+0000","last_became_active":"2023-10-08T16:11:46.781873+0000","last_became_peered":"2023-10-08T16:11:46.781873+0000","last_unstale":"2023-10-08T16:11:47.796152+0000","last_undegraded":"2023-10-08T16:11:47.796152+0000","last_fullsized":"2023-10-08T16:11:47.796152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:39.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798643+0000","last_change":"2023-10-08T16:11:47.792095+0000","last_active":"2023-10-08T16:11:47.798643+0000","last_peered":"2023-10-08T16:11:47.798643+0000","last_clean":"2023-10-08T16:11:47.798643+0000","last_became_active":"2023-10-08T16:11:47.791819+0000","last_became_peered":"2023-10-08T16:11:47.791819+0000","last_unstale":"2023-10-08T16:11:47.798643+0000","last_undegraded":"2023-10-08T16:11:47.798643+0000","last_fullsized":"2023-10-08T16:11:47.798643+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:02:30.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.776502+0000","last_change":"2023-10-08T16:11:46.787514+0000","last_active":"2023-10-08T16:11:47.776502+0000","last_peered":"2023-10-08T16:11:47.776502+0000","last_clean":"2023-10-08T16:11:47.776502+0000","last_became_active":"2023-10-08T16:11:46.787189+0000","last_became_peered":"2023-10-08T16:11:46.787189+0000","last_unstale":"2023-10-08T16:11:47.776502+0000","last_undegraded":"2023-10-08T16:11:47.776502+0000","last_fullsized":"2023-10-08T16:11:47.776502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:07:02.465272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796899+0000","last_change":"2023-10-08T16:11:47.788734+0000","last_active":"2023-10-08T16:11:47.796899+0000","last_peered":"2023-10-08T16:11:47.796899+0000","last_clean":"2023-10-08T16:11:47.796899+0000","last_became_active":"2023-10-08T16:11:47.787445+0000","last_became_peered":"2023-10-08T16:11:47.787445+0000","last_unstale":"2023-10-08T16:11:47.796899+0000","last_undegraded":"2023-10-08T16:11:47.796899+0000","last_fullsized":"2023-10-08T16:11:47.796899+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:04.512790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.773197+0000","last_change":"2023-10-08T16:11:46.785212+0000","last_active":"2023-10-08T16:11:47.773197+0000","last_peered":"2023-10-08T16:11:47.773197+0000","last_clean":"2023-10-08T16:11:47.773197+0000","last_became_active":"2023-10-08T16:11:46.784928+0000","last_became_peered":"2023-10-08T16:11:46.784928+0000","last_unstale":"2023-10-08T16:11:47.773197+0000","last_undegraded":"2023-10-08T16:11:47.773197+0000","last_fullsized":"2023-10-08T16:11:47.773197+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:14.393450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782836+0000","last_change":"2023-10-08T16:11:47.782364+0000","last_active":"2023-10-08T16:11:47.782836+0000","last_peered":"2023-10-08T16:11:47.782836+0000","last_clean":"2023-10-08T16:11:47.782836+0000","last_became_active":"2023-10-08T16:11:47.782102+0000","last_became_peered":"2023-10-08T16:11:47.782102+0000","last_unstale":"2023-10-08T16:11:47.782836+0000","last_undegraded":"2023-10-08T16:11:47.782836+0000","last_fullsized":"2023-10-08T16:11:47.782836+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:33.907690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786474+0000","last_change":"2023-10-08T16:11:46.781294+0000","last_active":"2023-10-08T16:11:47.786474+0000","last_peered":"2023-10-08T16:11:47.786474+0000","last_clean":"2023-10-08T16:11:47.786474+0000","last_became_active":"2023-10-08T16:11:46.780870+0000","last_became_peered":"2023-10-08T16:11:46.780870+0000","last_unstale":"2023-10-08T16:11:47.786474+0000","last_undegraded":"2023-10-08T16:11:47.786474+0000","last_fullsized":"2023-10-08T16:11:47.786474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:34.295684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786163+0000","last_change":"2023-10-08T16:11:47.784309+0000","last_active":"2023-10-08T16:11:47.786163+0000","last_peered":"2023-10-08T16:11:47.786163+0000","last_clean":"2023-10-08T16:11:47.786163+0000","last_became_active":"2023-10-08T16:11:47.784018+0000","last_became_peered":"2023-10-08T16:11:47.784018+0000","last_unstale":"2023-10-08T16:11:47.786163+0000","last_undegraded":"2023-10-08T16:11:47.786163+0000","last_fullsized":"2023-10-08T16:11:47.786163+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:57.502642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806782+0000","last_change":"2023-10-08T16:11:46.789607+0000","last_active":"2023-10-08T16:11:47.806782+0000","last_peered":"2023-10-08T16:11:47.806782+0000","last_clean":"2023-10-08T16:11:47.806782+0000","last_became_active":"2023-10-08T16:11:46.789315+0000","last_became_peered":"2023-10-08T16:11:46.789315+0000","last_unstale":"2023-10-08T16:11:47.806782+0000","last_undegraded":"2023-10-08T16:11:47.806782+0000","last_fullsized":"2023-10-08T16:11:47.806782+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:03.088849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796474+0000","last_change":"2023-10-08T16:11:47.795194+0000","last_active":"2023-10-08T16:11:47.796474+0000","last_peered":"2023-10-08T16:11:47.796474+0000","last_clean":"2023-10-08T16:11:47.796474+0000","last_became_active":"2023-10-08T16:11:47.794957+0000","last_became_peered":"2023-10-08T16:11:47.794957+0000","last_unstale":"2023-10-08T16:11:47.796474+0000","last_undegraded":"2023-10-08T16:11:47.796474+0000","last_fullsized":"2023-10-08T16:11:47.796474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782342+0000","last_change":"2023-10-08T16:11:46.786702+0000","last_active":"2023-10-08T16:11:47.782342+0000","last_peered":"2023-10-08T16:11:47.782342+0000","last_clean":"2023-10-08T16:11:47.782342+0000","last_became_active":"2023-10-08T16:11:46.786459+0000","last_became_peered":"2023-10-08T16:11:46.786459+0000","last_unstale":"2023-10-08T16:11:47.782342+0000","last_undegraded":"2023-10-08T16:11:47.782342+0000","last_fullsized":"2023-10-08T16:11:47.782342+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:09.035001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.783002+0000","last_change":"2023-10-08T16:11:47.781663+0000","last_active":"2023-10-08T16:11:47.783002+0000","last_peered":"2023-10-08T16:11:47.783002+0000","last_clean":"2023-10-08T16:11:47.783002+0000","last_became_active":"2023-10-08T16:11:47.781422+0000","last_became_peered":"2023-10-08T16:11:47.781422+0000","last_unstale":"2023-10-08T16:11:47.783002+0000","last_undegraded":"2023-10-08T16:11:47.783002+0000","last_fullsized":"2023-10-08T16:11:47.783002+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:32:43.670275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.781976+0000","last_change":"2023-10-08T16:11:46.788969+0000","last_active":"2023-10-08T16:11:47.781976+0000","last_peered":"2023-10-08T16:11:47.781976+0000","last_clean":"2023-10-08T16:11:47.781976+0000","last_became_active":"2023-10-08T16:11:46.788162+0000","last_became_peered":"2023-10-08T16:11:46.788162+0000","last_unstale":"2023-10-08T16:11:47.781976+0000","last_undegraded":"2023-10-08T16:11:47.781976+0000","last_fullsized":"2023-10-08T16:11:47.781976+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:21.952273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796850+0000","last_change":"2023-10-08T16:11:47.794930+0000","last_active":"2023-10-08T16:11:47.796850+0000","last_peered":"2023-10-08T16:11:47.796850+0000","last_clean":"2023-10-08T16:11:47.796850+0000","last_became_active":"2023-10-08T16:11:47.794668+0000","last_became_peered":"2023-10-08T16:11:47.794668+0000","last_unstale":"2023-10-08T16:11:47.796850+0000","last_undegraded":"2023-10-08T16:11:47.796850+0000","last_fullsized":"2023-10-08T16:11:47.796850+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:00.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.775733+0000","last_change":"2023-10-08T16:11:46.784219+0000","last_active":"2023-10-08T16:11:47.775733+0000","last_peered":"2023-10-08T16:11:47.775733+0000","last_clean":"2023-10-08T16:11:47.775733+0000","last_became_active":"2023-10-08T16:11:46.783745+0000","last_became_peered":"2023-10-08T16:11:46.783745+0000","last_unstale":"2023-10-08T16:11:47.775733+0000","last_undegraded":"2023-10-08T16:11:47.775733+0000","last_fullsized":"2023-10-08T16:11:47.775733+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806700+0000","last_change":"2023-10-08T16:11:47.792212+0000","last_active":"2023-10-08T16:11:47.806700+0000","last_peered":"2023-10-08T16:11:47.806700+0000","last_clean":"2023-10-08T16:11:47.806700+0000","last_became_active":"2023-10-08T16:11:47.792010+0000","last_became_peered":"2023-10-08T16:11:47.792010+0000","last_unstale":"2023-10-08T16:11:47.806700+0000","last_undegraded":"2023-10-08T16:11:47.806700+0000","last_fullsized":"2023-10-08T16:11:47.806700+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.785859+0000","last_change":"2023-10-08T16:11:46.784781+0000","last_active":"2023-10-08T16:11:47.785859+0000","last_peered":"2023-10-08T16:11:47.785859+0000","last_clean":"2023-10-08T16:11:47.785859+0000","last_became_active":"2023-10-08T16:11:46.784496+0000","last_became_peered":"2023-10-08T16:11:46.784496+0000","last_unstale":"2023-10-08T16:11:47.785859+0000","last_undegraded":"2023-10-08T16:11:47.785859+0000","last_fullsized":"2023-10-08T16:11:47.785859+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:14:14.105304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807350+0000","last_change":"2023-10-08T16:11:47.801711+0000","last_active":"2023-10-08T16:11:47.807350+0000","last_peered":"2023-10-08T16:11:47.807350+0000","last_clean":"2023-10-08T16:11:47.807350+0000","last_became_active":"2023-10-08T16:11:47.798092+0000","last_became_peered":"2023-10-08T16:11:47.798092+0000","last_unstale":"2023-10-08T16:11:47.807350+0000","last_undegraded":"2023-10-08T16:11:47.807350+0000","last_fullsized":"2023-10-08T16:11:47.807350+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:27:35.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.785943+0000","last_change":"2023-10-08T16:11:46.782692+0000","last_active":"2023-10-08T16:11:47.785943+0000","last_peered":"2023-10-08T16:11:47.785943+0000","last_clean":"2023-10-08T16:11:47.785943+0000","last_became_active":"2023-10-08T16:11:46.782421+0000","last_became_peered":"2023-10-08T16:11:46.782421+0000","last_unstale":"2023-10-08T16:11:47.785943+0000","last_undegraded":"2023-10-08T16:11:47.785943+0000","last_fullsized":"2023-10-08T16:11:47.785943+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:40:34.527999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796879+0000","last_change":"2023-10-08T16:11:47.788600+0000","last_active":"2023-10-08T16:11:47.796879+0000","last_peered":"2023-10-08T16:11:47.796879+0000","last_clean":"2023-10-08T16:11:47.796879+0000","last_became_active":"2023-10-08T16:11:47.787132+0000","last_became_peered":"2023-10-08T16:11:47.787132+0000","last_unstale":"2023-10-08T16:11:47.796879+0000","last_undegraded":"2023-10-08T16:11:47.796879+0000","last_fullsized":"2023-10-08T16:11:47.796879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:58.472655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.780256+0000","last_change":"2023-10-08T16:11:46.784110+0000","last_active":"2023-10-08T16:11:47.780256+0000","last_peered":"2023-10-08T16:11:47.780256+0000","last_clean":"2023-10-08T16:11:47.780256+0000","last_became_active":"2023-10-08T16:11:46.783643+0000","last_became_peered":"2023-10-08T16:11:46.783643+0000","last_unstale":"2023-10-08T16:11:47.780256+0000","last_undegraded":"2023-10-08T16:11:47.780256+0000","last_fullsized":"2023-10-08T16:11:47.780256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:37:17.409105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786362+0000","last_change":"2023-10-08T16:11:47.785590+0000","last_active":"2023-10-08T16:11:47.786362+0000","last_peered":"2023-10-08T16:11:47.786362+0000","last_clean":"2023-10-08T16:11:47.786362+0000","last_became_active":"2023-10-08T16:11:47.784857+0000","last_became_peered":"2023-10-08T16:11:47.784857+0000","last_unstale":"2023-10-08T16:11:47.786362+0000","last_undegraded":"2023-10-08T16:11:47.786362+0000","last_fullsized":"2023-10-08T16:11:47.786362+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:37:18.415815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.772646+0000","last_change":"2023-10-08T16:11:46.785049+0000","last_active":"2023-10-08T16:11:47.772646+0000","last_peered":"2023-10-08T16:11:47.772646+0000","last_clean":"2023-10-08T16:11:47.772646+0000","last_became_active":"2023-10-08T16:11:46.784787+0000","last_became_peered":"2023-10-08T16:11:46.784787+0000","last_unstale":"2023-10-08T16:11:47.772646+0000","last_undegraded":"2023-10-08T16:11:47.772646+0000","last_fullsized":"2023-10-08T16:11:47.772646+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:36.819458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792467+0000","last_change":"2023-10-08T16:11:47.790578+0000","last_active":"2023-10-08T16:11:47.792467+0000","last_peered":"2023-10-08T16:11:47.792467+0000","last_clean":"2023-10-08T16:11:47.792467+0000","last_became_active":"2023-10-08T16:11:47.790347+0000","last_became_peered":"2023-10-08T16:11:47.790347+0000","last_unstale":"2023-10-08T16:11:47.792467+0000","last_undegraded":"2023-10-08T16:11:47.792467+0000","last_fullsized":"2023-10-08T16:11:47.792467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:50.588017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.779759+0000","last_change":"2023-10-08T16:11:46.784792+0000","last_active":"2023-10-08T16:11:47.779759+0000","last_peered":"2023-10-08T16:11:47.779759+0000","last_clean":"2023-10-08T16:11:47.779759+0000","last_became_active":"2023-10-08T16:11:46.784505+0000","last_became_peered":"2023-10-08T16:11:46.784505+0000","last_unstale":"2023-10-08T16:11:47.779759+0000","last_undegraded":"2023-10-08T16:11:47.779759+0000","last_fullsized":"2023-10-08T16:11:47.779759+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:27.029115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":261,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778985+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:47.778985+0000","last_peered":"2023-10-08T16:11:47.778985+0000","last_clean":"2023-10-08T16:11:47.778985+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:47.778985+0000","last_undegraded":"2023-10-08T16:11:47.778985+0000","last_fullsized":"2023-10-08T16:11:47.778985+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1370150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.86199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.754},"min":{"1min":0.38,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.55500000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":0.82199999999999995,"15min":1.083},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.78500000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.626,"5min":0.70299999999999996,"15min":1.175},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.80900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.442},{"interface":"front","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66100000000000003,"5min":0.71399999999999997,"15min":8.3409999999999993},"last":0.54100000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":779678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.52100000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.626}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":787502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.373},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.57699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1370068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.249},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28100000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.36699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":780344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.30599999999999999},"min":{"1min":0.23400000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.36799999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.311},{"interface":"front","average":{"1min":0.315,"5min":0.31900000000000001,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.189,"15min":0.189},"max":{"1min":0.40899999999999997,"5min":0.49299999999999999,"15min":0.64300000000000002},"last":0.436}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.44,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.35799999999999998},"min":{"1min":0.28699999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46200000000000002,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.34399999999999997},"min":{"1min":0.25,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.38500000000000001},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46800000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.254,"15min":0.254},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.65900000000000003},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.63},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.871},"last":0.59399999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82399999999999995,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":786160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.317},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.254}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.376}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.77800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303116,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068724,"statfs":{"total":96636764160,"available":96326373376,"internally_reserved":0,"allocated":1773568,"data_stored":788056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.432},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.80700000000000005},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83099999999999996,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.30499999999999999},"max":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.77100000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.55900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56100000000000005,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.78700000000000003,"15min":0.83399999999999996},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.89200000000000002},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":785966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.61899999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.504},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.57299999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.59699999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:51.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:11:51.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.977+0000 7fa5c0192700 1 Processor -- start 2023-10-08T16:11:51.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.977+0000 7fa5c0192700 1 -- start start 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5c0192700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5c0192700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b815a160 con 0x7fa5b8159890 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5c0192700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b80d13a0 con 0x7fa5b815db10 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5c0192700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b815df50 con 0x7fa5b8159c60 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bd72d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bd72d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43392/0 (socket says 172.21.15.170:43392) 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bd72d700 1 -- 172.21.15.170:0/1316115297 learned_addr learned my addr 172.21.15.170:0/1316115297 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:51.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bdf2e700 1 --1- 172.21.15.170:0/1316115297 >> v1:172.21.15.170:6789/0 conn(0x7fa5b8159890 0x7fa5b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49156/0 (socket says 172.21.15.170:49156) 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bd72d700 1 -- 172.21.15.170:0/1316115297 >> v1:172.21.15.191:6789/0 conn(0x7fa5b815db10 legacy=0x7fa5b8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bd72d700 1 -- 172.21.15.170:0/1316115297 >> v1:172.21.15.170:6789/0 conn(0x7fa5b8159890 legacy=0x7fa5b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.978+0000 7fa5bcf2c700 1 -- 172.21.15.170:0/1316115297 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757062059 0 0) 0x7fa5b815a160 con 0x7fa5b8159890 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5bd72d700 1 -- 172.21.15.170:0/1316115297 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b815e140 con 0x7fa5b8159c60 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5bd72d700 1 --2- 172.21.15.170:0/1316115297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b8159c60 0x7fa5b815d5d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80099b0 tx=0x7fa5a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753d521a44bf0baa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5bcf2c700 1 -- 172.21.15.170:0/1316115297 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800b860 con 0x7fa5b8159c60 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5bcf2c700 1 -- 172.21.15.170:0/1316115297 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800b9c0 con 0x7fa5b8159c60 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5bcf2c700 1 -- 172.21.15.170:0/1316115297 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800bd60 con 0x7fa5b8159c60 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 -- 172.21.15.170:0/1316115297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b8159c60 msgr2=0x7fa5b815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1316115297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b8159c60 0x7fa5b815d5d0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80099b0 tx=0x7fa5a8009250 comp rx=0 tx=0).stop 2023-10-08T16:11:51.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 -- 172.21.15.170:0/1316115297 shutdown_connections 2023-10-08T16:11:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1316115297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b8159c60 0x7fa5b815d5d0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 -- 172.21.15.170:0/1316115297 >> 172.21.15.170:0/1316115297 conn(0x7fa5b8068e20 msgr2=0x7fa5b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 -- 172.21.15.170:0/1316115297 shutdown_connections 2023-10-08T16:11:51.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.979+0000 7fa5c0192700 1 -- 172.21.15.170:0/1316115297 wait complete. 2023-10-08T16:11:51.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.980+0000 7fa5c0192700 1 Processor -- start 2023-10-08T16:11:51.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 -- start start 2023-10-08T16:11:51.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b8159890 0x7fa5b816dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8159c60 0x7fa5b8152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815bfa0 con 0x7fa5b815db10 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80ca7a0 con 0x7fa5b8159c60 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5c0192700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b815b670 con 0x7fa5b8159890 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5be72f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5be72f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43400/0 (socket says 172.21.15.170:43400) 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5be72f700 1 -- 172.21.15.170:0/1029597360 learned_addr learned my addr 172.21.15.170:0/1029597360 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5bd72d700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8159c60 0x7fa5b8152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.981+0000 7fa5bdf2e700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b8159890 0x7fa5b816dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 -- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b8159890 msgr2=0x7fa5b816dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b8159890 0x7fa5b816dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 -- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8159c60 msgr2=0x7fa5b8152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8159c60 0x7fa5b8152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 -- 172.21.15.170:0/1029597360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80094f0 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5be72f700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c4c0 tx=0x7fa5b0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000ce70 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0011590 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b815b950 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0011970 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.982+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b80b0770 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.983+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b804c3c0 con 0x7fa5b815db10 2023-10-08T16:11:51.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.983+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa5b0011d20 con 0x7fa5b815db10 2023-10-08T16:11:51.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.984+0000 7fa5adffb700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 0x7fa59c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:51.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.984+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa5b0083f00 con 0x7fa5b815db10 2023-10-08T16:11:51.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.984+0000 7fa5bdf2e700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 0x7fa59c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:51.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.984+0000 7fa5bdf2e700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 0x7fa59c0697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4001560 tx=0x7fa5b4001ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:51.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:51.988+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5b0050b90 con 0x7fa5b815db10 2023-10-08T16:11:52.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.166+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5b815c0e0 con 0x7fa59c067320 2023-10-08T16:11:52.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.174+0000 7fa5adffb700 1 -- 172.21.15.170:0/1029597360 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289607 (secure 0 0 0) 0x7fa5b815c0e0 con 0x7fa59c067320 2023-10-08T16:11:52.178 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 msgr2=0x7fa59c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 0x7fa59c0697d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4001560 tx=0x7fa5b4001ae0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 msgr2=0x7fa5b80ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c4c0 tx=0x7fa5b0007f50 comp rx=0 tx=0).stop 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 shutdown_connections 2023-10-08T16:11:52.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa59c067320 0x7fa59c0697d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa5b815db10 0x7fa5b80ca260 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa5b8159890 0x7fa5b816dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 --2- 172.21.15.170:0/1029597360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa5b8159c60 0x7fa5b8152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 >> 172.21.15.170:0/1029597360 conn(0x7fa5b8068e20 msgr2=0x7fa5b8153fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 shutdown_connections 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.178+0000 7fa5c0192700 1 -- 172.21.15.170:0/1029597360 wait complete. 2023-10-08T16:11:52.181 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:11:52.190 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-10-08T16:11:50.312711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425976,"kb_used_data":14904,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752548744,"statfs":{"total":773094113280,"available":770609913856,"internally_reserved":0,"allocated":15261696,"data_stored":7447924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548289"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806752+0000","last_change":"2023-10-08T16:11:47.791142+0000","last_active":"2023-10-08T16:11:47.806752+0000","last_peered":"2023-10-08T16:11:47.806752+0000","last_clean":"2023-10-08T16:11:47.806752+0000","last_became_active":"2023-10-08T16:11:47.790340+0000","last_became_peered":"2023-10-08T16:11:47.790340+0000","last_unstale":"2023-10-08T16:11:47.806752+0000","last_undegraded":"2023-10-08T16:11:47.806752+0000","last_fullsized":"2023-10-08T16:11:47.806752+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794856+0000","last_change":"2023-10-08T16:11:47.787461+0000","last_active":"2023-10-08T16:11:47.794856+0000","last_peered":"2023-10-08T16:11:47.794856+0000","last_clean":"2023-10-08T16:11:47.794856+0000","last_became_active":"2023-10-08T16:11:47.787158+0000","last_became_peered":"2023-10-08T16:11:47.787158+0000","last_unstale":"2023-10-08T16:11:47.794856+0000","last_undegraded":"2023-10-08T16:11:47.794856+0000","last_fullsized":"2023-10-08T16:11:47.794856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:52.224326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796229+0000","last_change":"2023-10-08T16:11:47.792439+0000","last_active":"2023-10-08T16:11:47.796229+0000","last_peered":"2023-10-08T16:11:47.796229+0000","last_clean":"2023-10-08T16:11:47.796229+0000","last_became_active":"2023-10-08T16:11:47.792177+0000","last_became_peered":"2023-10-08T16:11:47.792177+0000","last_unstale":"2023-10-08T16:11:47.796229+0000","last_undegraded":"2023-10-08T16:11:47.796229+0000","last_fullsized":"2023-10-08T16:11:47.796229+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:53.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795211+0000","last_change":"2023-10-08T16:11:47.788835+0000","last_active":"2023-10-08T16:11:47.795211+0000","last_peered":"2023-10-08T16:11:47.795211+0000","last_clean":"2023-10-08T16:11:47.795211+0000","last_became_active":"2023-10-08T16:11:47.787243+0000","last_became_peered":"2023-10-08T16:11:47.787243+0000","last_unstale":"2023-10-08T16:11:47.795211+0000","last_undegraded":"2023-10-08T16:11:47.795211+0000","last_fullsized":"2023-10-08T16:11:47.795211+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:05:17.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795807+0000","last_change":"2023-10-08T16:11:47.792354+0000","last_active":"2023-10-08T16:11:47.795807+0000","last_peered":"2023-10-08T16:11:47.795807+0000","last_clean":"2023-10-08T16:11:47.795807+0000","last_became_active":"2023-10-08T16:11:47.792060+0000","last_became_peered":"2023-10-08T16:11:47.792060+0000","last_unstale":"2023-10-08T16:11:47.795807+0000","last_undegraded":"2023-10-08T16:11:47.795807+0000","last_fullsized":"2023-10-08T16:11:47.795807+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:14:15.112013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794044+0000","last_change":"2023-10-08T16:11:47.791030+0000","last_active":"2023-10-08T16:11:47.794044+0000","last_peered":"2023-10-08T16:11:47.794044+0000","last_clean":"2023-10-08T16:11:47.794044+0000","last_became_active":"2023-10-08T16:11:47.790800+0000","last_became_peered":"2023-10-08T16:11:47.790800+0000","last_unstale":"2023-10-08T16:11:47.794044+0000","last_undegraded":"2023-10-08T16:11:47.794044+0000","last_fullsized":"2023-10-08T16:11:47.794044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:36:15.948835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799321+0000","last_change":"2023-10-08T16:11:47.792384+0000","last_active":"2023-10-08T16:11:47.799321+0000","last_peered":"2023-10-08T16:11:47.799321+0000","last_clean":"2023-10-08T16:11:47.799321+0000","last_became_active":"2023-10-08T16:11:47.791926+0000","last_became_peered":"2023-10-08T16:11:47.791926+0000","last_unstale":"2023-10-08T16:11:47.799321+0000","last_undegraded":"2023-10-08T16:11:47.799321+0000","last_fullsized":"2023-10-08T16:11:47.799321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:46:09.131918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798192+0000","last_change":"2023-10-08T16:11:47.797252+0000","last_active":"2023-10-08T16:11:47.798192+0000","last_peered":"2023-10-08T16:11:47.798192+0000","last_clean":"2023-10-08T16:11:47.798192+0000","last_became_active":"2023-10-08T16:11:47.796759+0000","last_became_peered":"2023-10-08T16:11:47.796759+0000","last_unstale":"2023-10-08T16:11:47.798192+0000","last_undegraded":"2023-10-08T16:11:47.798192+0000","last_fullsized":"2023-10-08T16:11:47.798192+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806472+0000","last_change":"2023-10-08T16:11:47.795870+0000","last_active":"2023-10-08T16:11:47.806472+0000","last_peered":"2023-10-08T16:11:47.806472+0000","last_clean":"2023-10-08T16:11:47.806472+0000","last_became_active":"2023-10-08T16:11:47.795624+0000","last_became_peered":"2023-10-08T16:11:47.795624+0000","last_unstale":"2023-10-08T16:11:47.806472+0000","last_undegraded":"2023-10-08T16:11:47.806472+0000","last_fullsized":"2023-10-08T16:11:47.806472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:58.472655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787289+0000","last_change":"2023-10-08T16:11:47.784233+0000","last_active":"2023-10-08T16:11:47.787289+0000","last_peered":"2023-10-08T16:11:47.787289+0000","last_clean":"2023-10-08T16:11:47.787289+0000","last_became_active":"2023-10-08T16:11:47.783992+0000","last_became_peered":"2023-10-08T16:11:47.783992+0000","last_unstale":"2023-10-08T16:11:47.787289+0000","last_undegraded":"2023-10-08T16:11:47.787289+0000","last_fullsized":"2023-10-08T16:11:47.787289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:20.354451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801175+0000","last_change":"2023-10-08T16:11:47.796160+0000","last_active":"2023-10-08T16:11:47.801175+0000","last_peered":"2023-10-08T16:11:47.801175+0000","last_clean":"2023-10-08T16:11:47.801175+0000","last_became_active":"2023-10-08T16:11:47.795902+0000","last_became_peered":"2023-10-08T16:11:47.795902+0000","last_unstale":"2023-10-08T16:11:47.801175+0000","last_undegraded":"2023-10-08T16:11:47.801175+0000","last_fullsized":"2023-10-08T16:11:47.801175+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:43.673722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794848+0000","last_change":"2023-10-08T16:11:47.787350+0000","last_active":"2023-10-08T16:11:47.794848+0000","last_peered":"2023-10-08T16:11:47.794848+0000","last_clean":"2023-10-08T16:11:47.794848+0000","last_became_active":"2023-10-08T16:11:47.786923+0000","last_became_peered":"2023-10-08T16:11:47.786923+0000","last_unstale":"2023-10-08T16:11:47.794848+0000","last_undegraded":"2023-10-08T16:11:47.794848+0000","last_fullsized":"2023-10-08T16:11:47.794848+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:48.957415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801226+0000","last_change":"2023-10-08T16:11:47.797763+0000","last_active":"2023-10-08T16:11:47.801226+0000","last_peered":"2023-10-08T16:11:47.801226+0000","last_clean":"2023-10-08T16:11:47.801226+0000","last_became_active":"2023-10-08T16:11:47.797463+0000","last_became_peered":"2023-10-08T16:11:47.797463+0000","last_unstale":"2023-10-08T16:11:47.801226+0000","last_undegraded":"2023-10-08T16:11:47.801226+0000","last_fullsized":"2023-10-08T16:11:47.801226+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:53.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806407+0000","last_change":"2023-10-08T16:11:47.794341+0000","last_active":"2023-10-08T16:11:47.806407+0000","last_peered":"2023-10-08T16:11:47.806407+0000","last_clean":"2023-10-08T16:11:47.806407+0000","last_became_active":"2023-10-08T16:11:47.794007+0000","last_became_peered":"2023-10-08T16:11:47.794007+0000","last_unstale":"2023-10-08T16:11:47.806407+0000","last_undegraded":"2023-10-08T16:11:47.806407+0000","last_fullsized":"2023-10-08T16:11:47.806407+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:04.512790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798062+0000","last_change":"2023-10-08T16:11:47.793050+0000","last_active":"2023-10-08T16:11:47.798062+0000","last_peered":"2023-10-08T16:11:47.798062+0000","last_clean":"2023-10-08T16:11:47.798062+0000","last_became_active":"2023-10-08T16:11:47.792781+0000","last_became_peered":"2023-10-08T16:11:47.792781+0000","last_unstale":"2023-10-08T16:11:47.798062+0000","last_undegraded":"2023-10-08T16:11:47.798062+0000","last_fullsized":"2023-10-08T16:11:47.798062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:44.453346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796813+0000","last_change":"2023-10-08T16:11:47.794967+0000","last_active":"2023-10-08T16:11:47.796813+0000","last_peered":"2023-10-08T16:11:47.796813+0000","last_clean":"2023-10-08T16:11:47.796813+0000","last_became_active":"2023-10-08T16:11:47.794738+0000","last_became_peered":"2023-10-08T16:11:47.794738+0000","last_unstale":"2023-10-08T16:11:47.796813+0000","last_undegraded":"2023-10-08T16:11:47.796813+0000","last_fullsized":"2023-10-08T16:11:47.796813+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:25:46.504886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795186+0000","last_change":"2023-10-08T16:11:47.788280+0000","last_active":"2023-10-08T16:11:47.795186+0000","last_peered":"2023-10-08T16:11:47.795186+0000","last_clean":"2023-10-08T16:11:47.795186+0000","last_became_active":"2023-10-08T16:11:47.788004+0000","last_became_peered":"2023-10-08T16:11:47.788004+0000","last_unstale":"2023-10-08T16:11:47.795186+0000","last_undegraded":"2023-10-08T16:11:47.795186+0000","last_fullsized":"2023-10-08T16:11:47.795186+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:57.567976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807193+0000","last_change":"2023-10-08T16:11:47.796583+0000","last_active":"2023-10-08T16:11:47.807193+0000","last_peered":"2023-10-08T16:11:47.807193+0000","last_clean":"2023-10-08T16:11:47.807193+0000","last_became_active":"2023-10-08T16:11:47.796297+0000","last_became_peered":"2023-10-08T16:11:47.796297+0000","last_unstale":"2023-10-08T16:11:47.807193+0000","last_undegraded":"2023-10-08T16:11:47.807193+0000","last_fullsized":"2023-10-08T16:11:47.807193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:00.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796441+0000","last_change":"2023-10-08T16:11:47.794993+0000","last_active":"2023-10-08T16:11:47.796441+0000","last_peered":"2023-10-08T16:11:47.796441+0000","last_clean":"2023-10-08T16:11:47.796441+0000","last_became_active":"2023-10-08T16:11:47.794748+0000","last_became_peered":"2023-10-08T16:11:47.794748+0000","last_unstale":"2023-10-08T16:11:47.796441+0000","last_undegraded":"2023-10-08T16:11:47.796441+0000","last_fullsized":"2023-10-08T16:11:47.796441+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:19:43.732366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795991+0000","last_change":"2023-10-08T16:11:47.795538+0000","last_active":"2023-10-08T16:11:47.795991+0000","last_peered":"2023-10-08T16:11:47.795991+0000","last_clean":"2023-10-08T16:11:47.795991+0000","last_became_active":"2023-10-08T16:11:47.795319+0000","last_became_peered":"2023-10-08T16:11:47.795319+0000","last_unstale":"2023-10-08T16:11:47.795991+0000","last_undegraded":"2023-10-08T16:11:47.795991+0000","last_fullsized":"2023-10-08T16:11:47.795991+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:35.302393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795787+0000","last_change":"2023-10-08T16:11:47.792318+0000","last_active":"2023-10-08T16:11:47.795787+0000","last_peered":"2023-10-08T16:11:47.795787+0000","last_clean":"2023-10-08T16:11:47.795787+0000","last_became_active":"2023-10-08T16:11:47.791802+0000","last_became_peered":"2023-10-08T16:11:47.791802+0000","last_unstale":"2023-10-08T16:11:47.795787+0000","last_undegraded":"2023-10-08T16:11:47.795787+0000","last_fullsized":"2023-10-08T16:11:47.795787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:18:09.190734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787116+0000","last_change":"2023-10-08T16:11:47.785313+0000","last_active":"2023-10-08T16:11:47.787116+0000","last_peered":"2023-10-08T16:11:47.787116+0000","last_clean":"2023-10-08T16:11:47.787116+0000","last_became_active":"2023-10-08T16:11:47.784754+0000","last_became_peered":"2023-10-08T16:11:47.784754+0000","last_unstale":"2023-10-08T16:11:47.787116+0000","last_undegraded":"2023-10-08T16:11:47.787116+0000","last_fullsized":"2023-10-08T16:11:47.787116+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:06:40.641363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792889+0000","last_change":"2023-10-08T16:11:47.789615+0000","last_active":"2023-10-08T16:11:47.792889+0000","last_peered":"2023-10-08T16:11:47.792889+0000","last_clean":"2023-10-08T16:11:47.792889+0000","last_became_active":"2023-10-08T16:11:47.789382+0000","last_became_peered":"2023-10-08T16:11:47.789382+0000","last_unstale":"2023-10-08T16:11:47.792889+0000","last_undegraded":"2023-10-08T16:11:47.792889+0000","last_fullsized":"2023-10-08T16:11:47.792889+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:33.715587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.777107+0000","last_change":"2023-10-08T16:11:47.776665+0000","last_active":"2023-10-08T16:11:47.777107+0000","last_peered":"2023-10-08T16:11:47.777107+0000","last_clean":"2023-10-08T16:11:47.777107+0000","last_became_active":"2023-10-08T16:11:47.776413+0000","last_became_peered":"2023-10-08T16:11:47.776413+0000","last_unstale":"2023-10-08T16:11:47.777107+0000","last_undegraded":"2023-10-08T16:11:47.777107+0000","last_fullsized":"2023-10-08T16:11:47.777107+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:27:35.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795099+0000","last_change":"2023-10-08T16:11:47.788253+0000","last_active":"2023-10-08T16:11:47.795099+0000","last_peered":"2023-10-08T16:11:47.795099+0000","last_clean":"2023-10-08T16:11:47.795099+0000","last_became_active":"2023-10-08T16:11:47.788030+0000","last_became_peered":"2023-10-08T16:11:47.788030+0000","last_unstale":"2023-10-08T16:11:47.795099+0000","last_undegraded":"2023-10-08T16:11:47.795099+0000","last_fullsized":"2023-10-08T16:11:47.795099+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:49:12.205907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798073+0000","last_change":"2023-10-08T16:11:47.791649+0000","last_active":"2023-10-08T16:11:47.798073+0000","last_peered":"2023-10-08T16:11:47.798073+0000","last_clean":"2023-10-08T16:11:47.798073+0000","last_became_active":"2023-10-08T16:11:47.791418+0000","last_became_peered":"2023-10-08T16:11:47.791418+0000","last_unstale":"2023-10-08T16:11:47.798073+0000","last_undegraded":"2023-10-08T16:11:47.798073+0000","last_fullsized":"2023-10-08T16:11:47.798073+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:43.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792851+0000","last_change":"2023-10-08T16:11:47.789429+0000","last_active":"2023-10-08T16:11:47.792851+0000","last_peered":"2023-10-08T16:11:47.792851+0000","last_clean":"2023-10-08T16:11:47.792851+0000","last_became_active":"2023-10-08T16:11:47.788973+0000","last_became_peered":"2023-10-08T16:11:47.788973+0000","last_unstale":"2023-10-08T16:11:47.792851+0000","last_undegraded":"2023-10-08T16:11:47.792851+0000","last_fullsized":"2023-10-08T16:11:47.792851+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:43.673722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786172+0000","last_change":"2023-10-08T16:11:47.785028+0000","last_active":"2023-10-08T16:11:47.786172+0000","last_peered":"2023-10-08T16:11:47.786172+0000","last_clean":"2023-10-08T16:11:47.786172+0000","last_became_active":"2023-10-08T16:11:47.784770+0000","last_became_peered":"2023-10-08T16:11:47.784770+0000","last_unstale":"2023-10-08T16:11:47.786172+0000","last_undegraded":"2023-10-08T16:11:47.786172+0000","last_fullsized":"2023-10-08T16:11:47.786172+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:26:07.359907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798197+0000","last_change":"2023-10-08T16:11:47.796004+0000","last_active":"2023-10-08T16:11:47.798197+0000","last_peered":"2023-10-08T16:11:47.798197+0000","last_clean":"2023-10-08T16:11:47.798197+0000","last_became_active":"2023-10-08T16:11:47.795761+0000","last_became_peered":"2023-10-08T16:11:47.795761+0000","last_unstale":"2023-10-08T16:11:47.798197+0000","last_undegraded":"2023-10-08T16:11:47.798197+0000","last_fullsized":"2023-10-08T16:11:47.798197+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:30:56.356410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807239+0000","last_change":"2023-10-08T16:11:47.804605+0000","last_active":"2023-10-08T16:11:47.807239+0000","last_peered":"2023-10-08T16:11:47.807239+0000","last_clean":"2023-10-08T16:11:47.807239+0000","last_became_active":"2023-10-08T16:11:47.804308+0000","last_became_peered":"2023-10-08T16:11:47.804308+0000","last_unstale":"2023-10-08T16:11:47.807239+0000","last_undegraded":"2023-10-08T16:11:47.807239+0000","last_fullsized":"2023-10-08T16:11:47.807239+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:44.453346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.789692+0000","last_change":"2023-10-08T16:11:47.788468+0000","last_active":"2023-10-08T16:11:47.789692+0000","last_peered":"2023-10-08T16:11:47.789692+0000","last_clean":"2023-10-08T16:11:47.789692+0000","last_became_active":"2023-10-08T16:11:47.788222+0000","last_became_peered":"2023-10-08T16:11:47.788222+0000","last_unstale":"2023-10-08T16:11:47.789692+0000","last_undegraded":"2023-10-08T16:11:47.789692+0000","last_fullsized":"2023-10-08T16:11:47.789692+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:43.165133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796634+0000","last_change":"2023-10-08T16:11:47.796422+0000","last_active":"2023-10-08T16:11:47.796634+0000","last_peered":"2023-10-08T16:11:47.796634+0000","last_clean":"2023-10-08T16:11:47.796634+0000","last_became_active":"2023-10-08T16:11:47.796198+0000","last_became_peered":"2023-10-08T16:11:47.796198+0000","last_unstale":"2023-10-08T16:11:47.796634+0000","last_undegraded":"2023-10-08T16:11:47.796634+0000","last_fullsized":"2023-10-08T16:11:47.796634+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:05.805765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.797848+0000","last_change":"2023-10-08T16:11:47.795840+0000","last_active":"2023-10-08T16:11:47.797848+0000","last_peered":"2023-10-08T16:11:47.797848+0000","last_clean":"2023-10-08T16:11:47.797848+0000","last_became_active":"2023-10-08T16:11:47.795619+0000","last_became_peered":"2023-10-08T16:11:47.795619+0000","last_unstale":"2023-10-08T16:11:47.797848+0000","last_undegraded":"2023-10-08T16:11:47.797848+0000","last_fullsized":"2023-10-08T16:11:47.797848+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:21:34.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"138'4","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799284+0000","last_change":"2023-10-08T16:11:46.789501+0000","last_active":"2023-10-08T16:11:47.799284+0000","last_peered":"2023-10-08T16:11:47.799284+0000","last_clean":"2023-10-08T16:11:47.799284+0000","last_became_active":"2023-10-08T16:11:46.789223+0000","last_became_peered":"2023-10-08T16:11:46.789223+0000","last_unstale":"2023-10-08T16:11:47.799284+0000","last_undegraded":"2023-10-08T16:11:47.799284+0000","last_fullsized":"2023-10-08T16:11:47.799284+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:29.821225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792957+0000","last_change":"2023-10-08T16:11:47.787336+0000","last_active":"2023-10-08T16:11:47.792957+0000","last_peered":"2023-10-08T16:11:47.792957+0000","last_clean":"2023-10-08T16:11:47.792957+0000","last_became_active":"2023-10-08T16:11:47.786963+0000","last_became_peered":"2023-10-08T16:11:47.786963+0000","last_unstale":"2023-10-08T16:11:47.792957+0000","last_undegraded":"2023-10-08T16:11:47.792957+0000","last_fullsized":"2023-10-08T16:11:47.792957+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:09:47.748612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794578+0000","last_change":"2023-10-08T16:11:46.785401+0000","last_active":"2023-10-08T16:11:47.794578+0000","last_peered":"2023-10-08T16:11:47.794578+0000","last_clean":"2023-10-08T16:11:47.794578+0000","last_became_active":"2023-10-08T16:11:46.785115+0000","last_became_peered":"2023-10-08T16:11:46.785115+0000","last_unstale":"2023-10-08T16:11:47.794578+0000","last_undegraded":"2023-10-08T16:11:47.794578+0000","last_fullsized":"2023-10-08T16:11:47.794578+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:13:46.016861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.803878+0000","last_change":"2023-10-08T16:11:47.798633+0000","last_active":"2023-10-08T16:11:47.803878+0000","last_peered":"2023-10-08T16:11:47.803878+0000","last_clean":"2023-10-08T16:11:47.803878+0000","last_became_active":"2023-10-08T16:11:47.798322+0000","last_became_peered":"2023-10-08T16:11:47.798322+0000","last_unstale":"2023-10-08T16:11:47.803878+0000","last_undegraded":"2023-10-08T16:11:47.803878+0000","last_fullsized":"2023-10-08T16:11:47.803878+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:22:53.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794091+0000","last_change":"2023-10-08T16:11:46.791077+0000","last_active":"2023-10-08T16:11:47.794091+0000","last_peered":"2023-10-08T16:11:47.794091+0000","last_clean":"2023-10-08T16:11:47.794091+0000","last_became_active":"2023-10-08T16:11:46.790732+0000","last_became_peered":"2023-10-08T16:11:46.790732+0000","last_unstale":"2023-10-08T16:11:47.794091+0000","last_undegraded":"2023-10-08T16:11:47.794091+0000","last_fullsized":"2023-10-08T16:11:47.794091+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:58.331890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806614+0000","last_change":"2023-10-08T16:11:47.797444+0000","last_active":"2023-10-08T16:11:47.806614+0000","last_peered":"2023-10-08T16:11:47.806614+0000","last_clean":"2023-10-08T16:11:47.806614+0000","last_became_active":"2023-10-08T16:11:47.797196+0000","last_became_peered":"2023-10-08T16:11:47.797196+0000","last_unstale":"2023-10-08T16:11:47.806614+0000","last_undegraded":"2023-10-08T16:11:47.806614+0000","last_fullsized":"2023-10-08T16:11:47.806614+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:19.039859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807179+0000","last_change":"2023-10-08T16:11:46.787321+0000","last_active":"2023-10-08T16:11:47.807179+0000","last_peered":"2023-10-08T16:11:47.807179+0000","last_clean":"2023-10-08T16:11:47.807179+0000","last_became_active":"2023-10-08T16:11:46.786924+0000","last_became_peered":"2023-10-08T16:11:46.786924+0000","last_unstale":"2023-10-08T16:11:47.807179+0000","last_undegraded":"2023-10-08T16:11:47.807179+0000","last_fullsized":"2023-10-08T16:11:47.807179+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:56.561267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786648+0000","last_change":"2023-10-08T16:11:47.783278+0000","last_active":"2023-10-08T16:11:47.786648+0000","last_peered":"2023-10-08T16:11:47.786648+0000","last_clean":"2023-10-08T16:11:47.786648+0000","last_became_active":"2023-10-08T16:11:47.783024+0000","last_became_peered":"2023-10-08T16:11:47.783024+0000","last_unstale":"2023-10-08T16:11:47.786648+0000","last_undegraded":"2023-10-08T16:11:47.786648+0000","last_fullsized":"2023-10-08T16:11:47.786648+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:35:36.624320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795253+0000","last_change":"2023-10-08T16:11:46.785437+0000","last_active":"2023-10-08T16:11:47.795253+0000","last_peered":"2023-10-08T16:11:47.795253+0000","last_clean":"2023-10-08T16:11:47.795253+0000","last_became_active":"2023-10-08T16:11:46.785184+0000","last_became_peered":"2023-10-08T16:11:46.785184+0000","last_unstale":"2023-10-08T16:11:47.795253+0000","last_undegraded":"2023-10-08T16:11:47.795253+0000","last_fullsized":"2023-10-08T16:11:47.795253+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:18:11.184537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.797787+0000","last_change":"2023-10-08T16:11:47.795058+0000","last_active":"2023-10-08T16:11:47.797787+0000","last_peered":"2023-10-08T16:11:47.797787+0000","last_clean":"2023-10-08T16:11:47.797787+0000","last_became_active":"2023-10-08T16:11:47.794654+0000","last_became_peered":"2023-10-08T16:11:47.794654+0000","last_unstale":"2023-10-08T16:11:47.797787+0000","last_undegraded":"2023-10-08T16:11:47.797787+0000","last_fullsized":"2023-10-08T16:11:47.797787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:40:03.766135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796012+0000","last_change":"2023-10-08T16:11:46.784141+0000","last_active":"2023-10-08T16:11:47.796012+0000","last_peered":"2023-10-08T16:11:47.796012+0000","last_clean":"2023-10-08T16:11:47.796012+0000","last_became_active":"2023-10-08T16:11:46.783870+0000","last_became_peered":"2023-10-08T16:11:46.783870+0000","last_unstale":"2023-10-08T16:11:47.796012+0000","last_undegraded":"2023-10-08T16:11:47.796012+0000","last_fullsized":"2023-10-08T16:11:47.796012+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:09.035001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799334+0000","last_change":"2023-10-08T16:11:47.793434+0000","last_active":"2023-10-08T16:11:47.799334+0000","last_peered":"2023-10-08T16:11:47.799334+0000","last_clean":"2023-10-08T16:11:47.799334+0000","last_became_active":"2023-10-08T16:11:47.791899+0000","last_became_peered":"2023-10-08T16:11:47.791899+0000","last_unstale":"2023-10-08T16:11:47.799334+0000","last_undegraded":"2023-10-08T16:11:47.799334+0000","last_fullsized":"2023-10-08T16:11:47.799334+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:33.573488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778779+0000","last_change":"2023-10-08T16:11:46.785300+0000","last_active":"2023-10-08T16:11:47.778779+0000","last_peered":"2023-10-08T16:11:47.778779+0000","last_clean":"2023-10-08T16:11:47.778779+0000","last_became_active":"2023-10-08T16:11:46.785023+0000","last_became_peered":"2023-10-08T16:11:46.785023+0000","last_unstale":"2023-10-08T16:11:47.778779+0000","last_undegraded":"2023-10-08T16:11:47.778779+0000","last_fullsized":"2023-10-08T16:11:47.778779+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:31.724283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778449+0000","last_change":"2023-10-08T16:11:47.775923+0000","last_active":"2023-10-08T16:11:47.778449+0000","last_peered":"2023-10-08T16:11:47.778449+0000","last_clean":"2023-10-08T16:11:47.778449+0000","last_became_active":"2023-10-08T16:11:47.775652+0000","last_became_peered":"2023-10-08T16:11:47.775652+0000","last_unstale":"2023-10-08T16:11:47.778449+0000","last_undegraded":"2023-10-08T16:11:47.778449+0000","last_fullsized":"2023-10-08T16:11:47.778449+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:50:33.573488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778152+0000","last_change":"2023-10-08T16:11:46.782694+0000","last_active":"2023-10-08T16:11:47.778152+0000","last_peered":"2023-10-08T16:11:47.778152+0000","last_clean":"2023-10-08T16:11:47.778152+0000","last_became_active":"2023-10-08T16:11:46.782446+0000","last_became_peered":"2023-10-08T16:11:46.782446+0000","last_unstale":"2023-10-08T16:11:47.778152+0000","last_undegraded":"2023-10-08T16:11:47.778152+0000","last_fullsized":"2023-10-08T16:11:47.778152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:39.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.794956+0000","last_change":"2023-10-08T16:11:47.790850+0000","last_active":"2023-10-08T16:11:47.794956+0000","last_peered":"2023-10-08T16:11:47.794956+0000","last_clean":"2023-10-08T16:11:47.794956+0000","last_became_active":"2023-10-08T16:11:47.787554+0000","last_became_peered":"2023-10-08T16:11:47.787554+0000","last_unstale":"2023-10-08T16:11:47.794956+0000","last_undegraded":"2023-10-08T16:11:47.794956+0000","last_fullsized":"2023-10-08T16:11:47.794956+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.777698+0000","last_change":"2023-10-08T16:11:46.787439+0000","last_active":"2023-10-08T16:11:47.777698+0000","last_peered":"2023-10-08T16:11:47.777698+0000","last_clean":"2023-10-08T16:11:47.777698+0000","last_became_active":"2023-10-08T16:11:46.787155+0000","last_became_peered":"2023-10-08T16:11:46.787155+0000","last_unstale":"2023-10-08T16:11:47.777698+0000","last_undegraded":"2023-10-08T16:11:47.777698+0000","last_fullsized":"2023-10-08T16:11:47.777698+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:26:30.908090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.803857+0000","last_change":"2023-10-08T16:11:47.798323+0000","last_active":"2023-10-08T16:11:47.803857+0000","last_peered":"2023-10-08T16:11:47.803857+0000","last_clean":"2023-10-08T16:11:47.803857+0000","last_became_active":"2023-10-08T16:11:47.797802+0000","last_became_peered":"2023-10-08T16:11:47.797802+0000","last_unstale":"2023-10-08T16:11:47.803857+0000","last_undegraded":"2023-10-08T16:11:47.803857+0000","last_fullsized":"2023-10-08T16:11:47.803857+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:38:33.715587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"138'3","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795368+0000","last_change":"2023-10-08T16:11:46.785257+0000","last_active":"2023-10-08T16:11:47.795368+0000","last_peered":"2023-10-08T16:11:47.795368+0000","last_clean":"2023-10-08T16:11:47.795368+0000","last_became_active":"2023-10-08T16:11:46.785006+0000","last_became_peered":"2023-10-08T16:11:46.785006+0000","last_unstale":"2023-10-08T16:11:47.795368+0000","last_undegraded":"2023-10-08T16:11:47.795368+0000","last_fullsized":"2023-10-08T16:11:47.795368+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:29:59.314888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786858+0000","last_change":"2023-10-08T16:11:47.783690+0000","last_active":"2023-10-08T16:11:47.786858+0000","last_peered":"2023-10-08T16:11:47.786858+0000","last_clean":"2023-10-08T16:11:47.786858+0000","last_became_active":"2023-10-08T16:11:47.782652+0000","last_became_peered":"2023-10-08T16:11:47.782652+0000","last_unstale":"2023-10-08T16:11:47.786858+0000","last_undegraded":"2023-10-08T16:11:47.786858+0000","last_fullsized":"2023-10-08T16:11:47.786858+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:10:37.538839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"138'6","reported_seq":16,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807440+0000","last_change":"2023-10-08T16:11:46.781686+0000","last_active":"2023-10-08T16:11:47.807440+0000","last_peered":"2023-10-08T16:11:47.807440+0000","last_clean":"2023-10-08T16:11:47.807440+0000","last_became_active":"2023-10-08T16:11:46.781440+0000","last_became_peered":"2023-10-08T16:11:46.781440+0000","last_unstale":"2023-10-08T16:11:47.807440+0000","last_undegraded":"2023-10-08T16:11:47.807440+0000","last_fullsized":"2023-10-08T16:11:47.807440+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:18.937283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.791388+0000","last_change":"2023-10-08T16:11:47.789178+0000","last_active":"2023-10-08T16:11:47.791388+0000","last_peered":"2023-10-08T16:11:47.791388+0000","last_clean":"2023-10-08T16:11:47.791388+0000","last_became_active":"2023-10-08T16:11:47.788882+0000","last_became_peered":"2023-10-08T16:11:47.788882+0000","last_unstale":"2023-10-08T16:11:47.791388+0000","last_undegraded":"2023-10-08T16:11:47.791388+0000","last_fullsized":"2023-10-08T16:11:47.791388+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:00:39.175148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807168+0000","last_change":"2023-10-08T16:11:46.785794+0000","last_active":"2023-10-08T16:11:47.807168+0000","last_peered":"2023-10-08T16:11:47.807168+0000","last_clean":"2023-10-08T16:11:47.807168+0000","last_became_active":"2023-10-08T16:11:46.785555+0000","last_became_peered":"2023-10-08T16:11:46.785555+0000","last_unstale":"2023-10-08T16:11:47.807168+0000","last_undegraded":"2023-10-08T16:11:47.807168+0000","last_fullsized":"2023-10-08T16:11:47.807168+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:29.580179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801112+0000","last_change":"2023-10-08T16:11:47.797147+0000","last_active":"2023-10-08T16:11:47.801112+0000","last_peered":"2023-10-08T16:11:47.801112+0000","last_clean":"2023-10-08T16:11:47.801112+0000","last_became_active":"2023-10-08T16:11:47.796766+0000","last_became_peered":"2023-10-08T16:11:47.796766+0000","last_unstale":"2023-10-08T16:11:47.801112+0000","last_undegraded":"2023-10-08T16:11:47.801112+0000","last_fullsized":"2023-10-08T16:11:47.801112+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:55:53.872812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782126+0000","last_change":"2023-10-08T16:11:46.788116+0000","last_active":"2023-10-08T16:11:47.782126+0000","last_peered":"2023-10-08T16:11:47.782126+0000","last_clean":"2023-10-08T16:11:47.782126+0000","last_became_active":"2023-10-08T16:11:46.787871+0000","last_became_peered":"2023-10-08T16:11:46.787871+0000","last_unstale":"2023-10-08T16:11:47.782126+0000","last_undegraded":"2023-10-08T16:11:47.782126+0000","last_fullsized":"2023-10-08T16:11:47.782126+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801085+0000","last_change":"2023-10-08T16:11:47.792912+0000","last_active":"2023-10-08T16:11:47.801085+0000","last_peered":"2023-10-08T16:11:47.801085+0000","last_clean":"2023-10-08T16:11:47.801085+0000","last_became_active":"2023-10-08T16:11:47.792682+0000","last_became_peered":"2023-10-08T16:11:47.792682+0000","last_unstale":"2023-10-08T16:11:47.801085+0000","last_undegraded":"2023-10-08T16:11:47.801085+0000","last_fullsized":"2023-10-08T16:11:47.801085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:53.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.774071+0000","last_change":"2023-10-08T16:11:46.782750+0000","last_active":"2023-10-08T16:11:47.774071+0000","last_peered":"2023-10-08T16:11:47.774071+0000","last_clean":"2023-10-08T16:11:47.774071+0000","last_became_active":"2023-10-08T16:11:46.782468+0000","last_became_peered":"2023-10-08T16:11:46.782468+0000","last_unstale":"2023-10-08T16:11:47.774071+0000","last_undegraded":"2023-10-08T16:11:47.774071+0000","last_fullsized":"2023-10-08T16:11:47.774071+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:00:39.700318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.800887+0000","last_change":"2023-10-08T16:11:47.800217+0000","last_active":"2023-10-08T16:11:47.800887+0000","last_peered":"2023-10-08T16:11:47.800887+0000","last_clean":"2023-10-08T16:11:47.800887+0000","last_became_active":"2023-10-08T16:11:47.799975+0000","last_became_peered":"2023-10-08T16:11:47.799975+0000","last_unstale":"2023-10-08T16:11:47.800887+0000","last_undegraded":"2023-10-08T16:11:47.800887+0000","last_fullsized":"2023-10-08T16:11:47.800887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:30.586888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"138'8","reported_seq":18,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807403+0000","last_change":"2023-10-08T16:11:46.785349+0000","last_active":"2023-10-08T16:11:47.807403+0000","last_peered":"2023-10-08T16:11:47.807403+0000","last_clean":"2023-10-08T16:11:47.807403+0000","last_became_active":"2023-10-08T16:11:46.785083+0000","last_became_peered":"2023-10-08T16:11:46.785083+0000","last_unstale":"2023-10-08T16:11:47.807403+0000","last_undegraded":"2023-10-08T16:11:47.807403+0000","last_fullsized":"2023-10-08T16:11:47.807403+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:45:16.996074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.800574+0000","last_change":"2023-10-08T16:11:47.799266+0000","last_active":"2023-10-08T16:11:47.800574+0000","last_peered":"2023-10-08T16:11:47.800574+0000","last_clean":"2023-10-08T16:11:47.800574+0000","last_became_active":"2023-10-08T16:11:47.799023+0000","last_became_peered":"2023-10-08T16:11:47.799023+0000","last_unstale":"2023-10-08T16:11:47.800574+0000","last_undegraded":"2023-10-08T16:11:47.800574+0000","last_fullsized":"2023-10-08T16:11:47.800574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:07.163174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.795728+0000","last_change":"2023-10-08T16:11:46.790388+0000","last_active":"2023-10-08T16:11:47.795728+0000","last_peered":"2023-10-08T16:11:47.795728+0000","last_clean":"2023-10-08T16:11:47.795728+0000","last_became_active":"2023-10-08T16:11:46.790197+0000","last_became_peered":"2023-10-08T16:11:46.790197+0000","last_unstale":"2023-10-08T16:11:47.795728+0000","last_undegraded":"2023-10-08T16:11:47.795728+0000","last_fullsized":"2023-10-08T16:11:47.795728+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:46:28.232219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.801010+0000","last_change":"2023-10-08T16:11:47.791538+0000","last_active":"2023-10-08T16:11:47.801010+0000","last_peered":"2023-10-08T16:11:47.801010+0000","last_clean":"2023-10-08T16:11:47.801010+0000","last_became_active":"2023-10-08T16:11:47.791262+0000","last_became_peered":"2023-10-08T16:11:47.791262+0000","last_unstale":"2023-10-08T16:11:47.801010+0000","last_undegraded":"2023-10-08T16:11:47.801010+0000","last_fullsized":"2023-10-08T16:11:47.801010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:05:17.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.799819+0000","last_change":"2023-10-08T16:11:46.783954+0000","last_active":"2023-10-08T16:11:47.799819+0000","last_peered":"2023-10-08T16:11:47.799819+0000","last_clean":"2023-10-08T16:11:47.799819+0000","last_became_active":"2023-10-08T16:11:46.783731+0000","last_became_peered":"2023-10-08T16:11:46.783731+0000","last_unstale":"2023-10-08T16:11:47.799819+0000","last_undegraded":"2023-10-08T16:11:47.799819+0000","last_fullsized":"2023-10-08T16:11:47.799819+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:21.952273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807227+0000","last_change":"2023-10-08T16:11:47.804663+0000","last_active":"2023-10-08T16:11:47.807227+0000","last_peered":"2023-10-08T16:11:47.807227+0000","last_clean":"2023-10-08T16:11:47.807227+0000","last_became_active":"2023-10-08T16:11:47.804375+0000","last_became_peered":"2023-10-08T16:11:47.804375+0000","last_unstale":"2023-10-08T16:11:47.807227+0000","last_undegraded":"2023-10-08T16:11:47.807227+0000","last_fullsized":"2023-10-08T16:11:47.807227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:33:52.224326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"138'1","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807135+0000","last_change":"2023-10-08T16:11:46.785752+0000","last_active":"2023-10-08T16:11:47.807135+0000","last_peered":"2023-10-08T16:11:47.807135+0000","last_clean":"2023-10-08T16:11:47.807135+0000","last_became_active":"2023-10-08T16:11:46.784455+0000","last_became_peered":"2023-10-08T16:11:46.784455+0000","last_unstale":"2023-10-08T16:11:47.807135+0000","last_undegraded":"2023-10-08T16:11:47.807135+0000","last_fullsized":"2023-10-08T16:11:47.807135+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.793477+0000","last_change":"2023-10-08T16:11:47.783605+0000","last_active":"2023-10-08T16:11:47.793477+0000","last_peered":"2023-10-08T16:11:47.793477+0000","last_clean":"2023-10-08T16:11:47.793477+0000","last_became_active":"2023-10-08T16:11:47.783335+0000","last_became_peered":"2023-10-08T16:11:47.783335+0000","last_unstale":"2023-10-08T16:11:47.793477+0000","last_undegraded":"2023-10-08T16:11:47.793477+0000","last_fullsized":"2023-10-08T16:11:47.793477+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:19.039859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.780019+0000","last_change":"2023-10-08T16:11:46.785071+0000","last_active":"2023-10-08T16:11:47.780019+0000","last_peered":"2023-10-08T16:11:47.780019+0000","last_clean":"2023-10-08T16:11:47.780019+0000","last_became_active":"2023-10-08T16:11:46.784793+0000","last_became_peered":"2023-10-08T16:11:46.784793+0000","last_unstale":"2023-10-08T16:11:47.780019+0000","last_undegraded":"2023-10-08T16:11:47.780019+0000","last_fullsized":"2023-10-08T16:11:47.780019+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:14.393450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.787137+0000","last_change":"2023-10-08T16:11:47.785463+0000","last_active":"2023-10-08T16:11:47.787137+0000","last_peered":"2023-10-08T16:11:47.787137+0000","last_clean":"2023-10-08T16:11:47.787137+0000","last_became_active":"2023-10-08T16:11:47.785026+0000","last_became_peered":"2023-10-08T16:11:47.785026+0000","last_unstale":"2023-10-08T16:11:47.787137+0000","last_undegraded":"2023-10-08T16:11:47.787137+0000","last_fullsized":"2023-10-08T16:11:47.787137+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:32:43.670275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796152+0000","last_change":"2023-10-08T16:11:46.782131+0000","last_active":"2023-10-08T16:11:47.796152+0000","last_peered":"2023-10-08T16:11:47.796152+0000","last_clean":"2023-10-08T16:11:47.796152+0000","last_became_active":"2023-10-08T16:11:46.781873+0000","last_became_peered":"2023-10-08T16:11:46.781873+0000","last_unstale":"2023-10-08T16:11:47.796152+0000","last_undegraded":"2023-10-08T16:11:47.796152+0000","last_fullsized":"2023-10-08T16:11:47.796152+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:39.290870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.798643+0000","last_change":"2023-10-08T16:11:47.792095+0000","last_active":"2023-10-08T16:11:47.798643+0000","last_peered":"2023-10-08T16:11:47.798643+0000","last_clean":"2023-10-08T16:11:47.798643+0000","last_became_active":"2023-10-08T16:11:47.791819+0000","last_became_peered":"2023-10-08T16:11:47.791819+0000","last_unstale":"2023-10-08T16:11:47.798643+0000","last_undegraded":"2023-10-08T16:11:47.798643+0000","last_fullsized":"2023-10-08T16:11:47.798643+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:02:30.356644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.776502+0000","last_change":"2023-10-08T16:11:46.787514+0000","last_active":"2023-10-08T16:11:47.776502+0000","last_peered":"2023-10-08T16:11:47.776502+0000","last_clean":"2023-10-08T16:11:47.776502+0000","last_became_active":"2023-10-08T16:11:46.787189+0000","last_became_peered":"2023-10-08T16:11:46.787189+0000","last_unstale":"2023-10-08T16:11:47.776502+0000","last_undegraded":"2023-10-08T16:11:47.776502+0000","last_fullsized":"2023-10-08T16:11:47.776502+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:07:02.465272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796899+0000","last_change":"2023-10-08T16:11:47.788734+0000","last_active":"2023-10-08T16:11:47.796899+0000","last_peered":"2023-10-08T16:11:47.796899+0000","last_clean":"2023-10-08T16:11:47.796899+0000","last_became_active":"2023-10-08T16:11:47.787445+0000","last_became_peered":"2023-10-08T16:11:47.787445+0000","last_unstale":"2023-10-08T16:11:47.796899+0000","last_undegraded":"2023-10-08T16:11:47.796899+0000","last_fullsized":"2023-10-08T16:11:47.796899+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:56:04.512790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.773197+0000","last_change":"2023-10-08T16:11:46.785212+0000","last_active":"2023-10-08T16:11:47.773197+0000","last_peered":"2023-10-08T16:11:47.773197+0000","last_clean":"2023-10-08T16:11:47.773197+0000","last_became_active":"2023-10-08T16:11:46.784928+0000","last_became_peered":"2023-10-08T16:11:46.784928+0000","last_unstale":"2023-10-08T16:11:47.773197+0000","last_undegraded":"2023-10-08T16:11:47.773197+0000","last_fullsized":"2023-10-08T16:11:47.773197+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:59:14.393450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782836+0000","last_change":"2023-10-08T16:11:47.782364+0000","last_active":"2023-10-08T16:11:47.782836+0000","last_peered":"2023-10-08T16:11:47.782836+0000","last_clean":"2023-10-08T16:11:47.782836+0000","last_became_active":"2023-10-08T16:11:47.782102+0000","last_became_peered":"2023-10-08T16:11:47.782102+0000","last_unstale":"2023-10-08T16:11:47.782836+0000","last_undegraded":"2023-10-08T16:11:47.782836+0000","last_fullsized":"2023-10-08T16:11:47.782836+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:24:33.907690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786474+0000","last_change":"2023-10-08T16:11:46.781294+0000","last_active":"2023-10-08T16:11:47.786474+0000","last_peered":"2023-10-08T16:11:47.786474+0000","last_clean":"2023-10-08T16:11:47.786474+0000","last_became_active":"2023-10-08T16:11:46.780870+0000","last_became_peered":"2023-10-08T16:11:46.780870+0000","last_unstale":"2023-10-08T16:11:47.786474+0000","last_undegraded":"2023-10-08T16:11:47.786474+0000","last_fullsized":"2023-10-08T16:11:47.786474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:51:34.295684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786163+0000","last_change":"2023-10-08T16:11:47.784309+0000","last_active":"2023-10-08T16:11:47.786163+0000","last_peered":"2023-10-08T16:11:47.786163+0000","last_clean":"2023-10-08T16:11:47.786163+0000","last_became_active":"2023-10-08T16:11:47.784018+0000","last_became_peered":"2023-10-08T16:11:47.784018+0000","last_unstale":"2023-10-08T16:11:47.786163+0000","last_undegraded":"2023-10-08T16:11:47.786163+0000","last_fullsized":"2023-10-08T16:11:47.786163+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:15:57.502642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"138'2","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806782+0000","last_change":"2023-10-08T16:11:46.789607+0000","last_active":"2023-10-08T16:11:47.806782+0000","last_peered":"2023-10-08T16:11:47.806782+0000","last_clean":"2023-10-08T16:11:47.806782+0000","last_became_active":"2023-10-08T16:11:46.789315+0000","last_became_peered":"2023-10-08T16:11:46.789315+0000","last_unstale":"2023-10-08T16:11:47.806782+0000","last_undegraded":"2023-10-08T16:11:47.806782+0000","last_fullsized":"2023-10-08T16:11:47.806782+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:25:03.088849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796474+0000","last_change":"2023-10-08T16:11:47.795194+0000","last_active":"2023-10-08T16:11:47.796474+0000","last_peered":"2023-10-08T16:11:47.796474+0000","last_clean":"2023-10-08T16:11:47.796474+0000","last_became_active":"2023-10-08T16:11:47.794957+0000","last_became_peered":"2023-10-08T16:11:47.794957+0000","last_unstale":"2023-10-08T16:11:47.796474+0000","last_undegraded":"2023-10-08T16:11:47.796474+0000","last_fullsized":"2023-10-08T16:11:47.796474+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.782342+0000","last_change":"2023-10-08T16:11:46.786702+0000","last_active":"2023-10-08T16:11:47.782342+0000","last_peered":"2023-10-08T16:11:47.782342+0000","last_clean":"2023-10-08T16:11:47.782342+0000","last_became_active":"2023-10-08T16:11:46.786459+0000","last_became_peered":"2023-10-08T16:11:46.786459+0000","last_unstale":"2023-10-08T16:11:47.782342+0000","last_undegraded":"2023-10-08T16:11:47.782342+0000","last_fullsized":"2023-10-08T16:11:47.782342+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:09.035001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.783002+0000","last_change":"2023-10-08T16:11:47.781663+0000","last_active":"2023-10-08T16:11:47.783002+0000","last_peered":"2023-10-08T16:11:47.783002+0000","last_clean":"2023-10-08T16:11:47.783002+0000","last_became_active":"2023-10-08T16:11:47.781422+0000","last_became_peered":"2023-10-08T16:11:47.781422+0000","last_unstale":"2023-10-08T16:11:47.783002+0000","last_undegraded":"2023-10-08T16:11:47.783002+0000","last_fullsized":"2023-10-08T16:11:47.783002+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:32:43.670275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.781976+0000","last_change":"2023-10-08T16:11:46.788969+0000","last_active":"2023-10-08T16:11:47.781976+0000","last_peered":"2023-10-08T16:11:47.781976+0000","last_clean":"2023-10-08T16:11:47.781976+0000","last_became_active":"2023-10-08T16:11:46.788162+0000","last_became_peered":"2023-10-08T16:11:46.788162+0000","last_unstale":"2023-10-08T16:11:47.781976+0000","last_undegraded":"2023-10-08T16:11:47.781976+0000","last_fullsized":"2023-10-08T16:11:47.781976+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:16:21.952273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796850+0000","last_change":"2023-10-08T16:11:47.794930+0000","last_active":"2023-10-08T16:11:47.796850+0000","last_peered":"2023-10-08T16:11:47.796850+0000","last_clean":"2023-10-08T16:11:47.796850+0000","last_became_active":"2023-10-08T16:11:47.794668+0000","last_became_peered":"2023-10-08T16:11:47.794668+0000","last_unstale":"2023-10-08T16:11:47.796850+0000","last_undegraded":"2023-10-08T16:11:47.796850+0000","last_fullsized":"2023-10-08T16:11:47.796850+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:00.078377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.775733+0000","last_change":"2023-10-08T16:11:46.784219+0000","last_active":"2023-10-08T16:11:47.775733+0000","last_peered":"2023-10-08T16:11:47.775733+0000","last_clean":"2023-10-08T16:11:47.775733+0000","last_became_active":"2023-10-08T16:11:46.783745+0000","last_became_peered":"2023-10-08T16:11:46.783745+0000","last_unstale":"2023-10-08T16:11:47.775733+0000","last_undegraded":"2023-10-08T16:11:47.775733+0000","last_fullsized":"2023-10-08T16:11:47.775733+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:51.554913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.806700+0000","last_change":"2023-10-08T16:11:47.792212+0000","last_active":"2023-10-08T16:11:47.806700+0000","last_peered":"2023-10-08T16:11:47.806700+0000","last_clean":"2023-10-08T16:11:47.806700+0000","last_became_active":"2023-10-08T16:11:47.792010+0000","last_became_peered":"2023-10-08T16:11:47.792010+0000","last_unstale":"2023-10-08T16:11:47.806700+0000","last_undegraded":"2023-10-08T16:11:47.806700+0000","last_fullsized":"2023-10-08T16:11:47.806700+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:13:57.987299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.785859+0000","last_change":"2023-10-08T16:11:46.784781+0000","last_active":"2023-10-08T16:11:47.785859+0000","last_peered":"2023-10-08T16:11:47.785859+0000","last_clean":"2023-10-08T16:11:47.785859+0000","last_became_active":"2023-10-08T16:11:46.784496+0000","last_became_peered":"2023-10-08T16:11:46.784496+0000","last_unstale":"2023-10-08T16:11:47.785859+0000","last_undegraded":"2023-10-08T16:11:47.785859+0000","last_fullsized":"2023-10-08T16:11:47.785859+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:14:14.105304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.807350+0000","last_change":"2023-10-08T16:11:47.801711+0000","last_active":"2023-10-08T16:11:47.807350+0000","last_peered":"2023-10-08T16:11:47.807350+0000","last_clean":"2023-10-08T16:11:47.807350+0000","last_became_active":"2023-10-08T16:11:47.798092+0000","last_became_peered":"2023-10-08T16:11:47.798092+0000","last_unstale":"2023-10-08T16:11:47.807350+0000","last_undegraded":"2023-10-08T16:11:47.807350+0000","last_fullsized":"2023-10-08T16:11:47.807350+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:27:35.254014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.785943+0000","last_change":"2023-10-08T16:11:46.782692+0000","last_active":"2023-10-08T16:11:47.785943+0000","last_peered":"2023-10-08T16:11:47.785943+0000","last_clean":"2023-10-08T16:11:47.785943+0000","last_became_active":"2023-10-08T16:11:46.782421+0000","last_became_peered":"2023-10-08T16:11:46.782421+0000","last_unstale":"2023-10-08T16:11:47.785943+0000","last_undegraded":"2023-10-08T16:11:47.785943+0000","last_fullsized":"2023-10-08T16:11:47.785943+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:40:34.527999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.796879+0000","last_change":"2023-10-08T16:11:47.788600+0000","last_active":"2023-10-08T16:11:47.796879+0000","last_peered":"2023-10-08T16:11:47.796879+0000","last_clean":"2023-10-08T16:11:47.796879+0000","last_became_active":"2023-10-08T16:11:47.787132+0000","last_became_peered":"2023-10-08T16:11:47.787132+0000","last_unstale":"2023-10-08T16:11:47.796879+0000","last_undegraded":"2023-10-08T16:11:47.796879+0000","last_fullsized":"2023-10-08T16:11:47.796879+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:12:58.472655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.780256+0000","last_change":"2023-10-08T16:11:46.784110+0000","last_active":"2023-10-08T16:11:47.780256+0000","last_peered":"2023-10-08T16:11:47.780256+0000","last_clean":"2023-10-08T16:11:47.780256+0000","last_became_active":"2023-10-08T16:11:46.783643+0000","last_became_peered":"2023-10-08T16:11:46.783643+0000","last_unstale":"2023-10-08T16:11:47.780256+0000","last_undegraded":"2023-10-08T16:11:47.780256+0000","last_fullsized":"2023-10-08T16:11:47.780256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:37:17.409105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.786362+0000","last_change":"2023-10-08T16:11:47.785590+0000","last_active":"2023-10-08T16:11:47.786362+0000","last_peered":"2023-10-08T16:11:47.786362+0000","last_clean":"2023-10-08T16:11:47.786362+0000","last_became_active":"2023-10-08T16:11:47.784857+0000","last_became_peered":"2023-10-08T16:11:47.784857+0000","last_unstale":"2023-10-08T16:11:47.786362+0000","last_undegraded":"2023-10-08T16:11:47.786362+0000","last_fullsized":"2023-10-08T16:11:47.786362+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:37:18.415815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.772646+0000","last_change":"2023-10-08T16:11:46.785049+0000","last_active":"2023-10-08T16:11:47.772646+0000","last_peered":"2023-10-08T16:11:47.772646+0000","last_clean":"2023-10-08T16:11:47.772646+0000","last_became_active":"2023-10-08T16:11:46.784787+0000","last_became_peered":"2023-10-08T16:11:46.784787+0000","last_unstale":"2023-10-08T16:11:47.772646+0000","last_undegraded":"2023-10-08T16:11:47.772646+0000","last_fullsized":"2023-10-08T16:11:47.772646+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:05:36.819458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.792467+0000","last_change":"2023-10-08T16:11:47.790578+0000","last_active":"2023-10-08T16:11:47.792467+0000","last_peered":"2023-10-08T16:11:47.792467+0000","last_clean":"2023-10-08T16:11:47.792467+0000","last_became_active":"2023-10-08T16:11:47.790347+0000","last_became_peered":"2023-10-08T16:11:47.790347+0000","last_unstale":"2023-10-08T16:11:47.792467+0000","last_undegraded":"2023-10-08T16:11:47.792467+0000","last_fullsized":"2023-10-08T16:11:47.792467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:46.764746+0000","last_clean_scrub_stamp":"2023-10-08T16:11:46.764746+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:50.588017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.779759+0000","last_change":"2023-10-08T16:11:46.784792+0000","last_active":"2023-10-08T16:11:47.779759+0000","last_peered":"2023-10-08T16:11:47.779759+0000","last_clean":"2023-10-08T16:11:47.779759+0000","last_became_active":"2023-10-08T16:11:46.784505+0000","last_became_peered":"2023-10-08T16:11:46.784505+0000","last_unstale":"2023-10-08T16:11:47.779759+0000","last_undegraded":"2023-10-08T16:11:47.779759+0000","last_fullsized":"2023-10-08T16:11:47.779759+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:11:45.758036+0000","last_clean_scrub_stamp":"2023-10-08T16:11:45.758036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:15:27.029115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":261,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-08T16:11:47.778985+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:11:47.778985+0000","last_peered":"2023-10-08T16:11:47.778985+0000","last_clean":"2023-10-08T16:11:47.778985+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:11:47.778985+0000","last_undegraded":"2023-10-08T16:11:47.778985+0000","last_fullsized":"2023-10-08T16:11:47.778985+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1370150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.86199999999999999},"last":0.504},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.754},"min":{"1min":0.38,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.55500000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":0.82199999999999995,"15min":1.083},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.78500000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.626,"5min":0.70299999999999996,"15min":1.175},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.80900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.442},{"interface":"front","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66100000000000003,"5min":0.71399999999999997,"15min":8.3409999999999993},"last":0.54100000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61599999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64400000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303088,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068752,"statfs":{"total":96636764160,"available":96326402048,"internally_reserved":0,"allocated":1744896,"data_stored":779678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.52100000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.626}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.54200000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.71099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.746},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":787502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.373},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.33800000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.57699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1370068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.249},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.28100000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.36699999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.51900000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.58699999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574939,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":780344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.30599999999999999},"min":{"1min":0.23400000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.36799999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.311},{"interface":"front","average":{"1min":0.315,"5min":0.31900000000000001,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.189,"15min":0.189},"max":{"1min":0.40899999999999997,"5min":0.49299999999999999,"15min":0.64300000000000002},"last":0.436}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.44,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.35799999999999998},"min":{"1min":0.28699999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46200000000000002,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.34399999999999997},"min":{"1min":0.25,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.38500000000000001},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46800000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.254,"15min":0.254},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.65900000000000003},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.53400000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.63},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.871},"last":0.59399999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82399999999999995,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574939,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303112,"kb_used_data":1728,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068728,"statfs":{"total":96636764160,"available":96326377472,"internally_reserved":0,"allocated":1769472,"data_stored":786160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.317},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.254}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.376}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.77800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.76300000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.66900000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.68000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303116,"kb_used_data":1732,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068724,"statfs":{"total":96636764160,"available":96326373376,"internally_reserved":0,"allocated":1773568,"data_stored":788056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39400000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.432},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.80700000000000005},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83099999999999996,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.503}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.30499999999999999},"max":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.77100000000000002},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.55900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56100000000000005,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.78700000000000003,"15min":0.83399999999999996},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.89200000000000002},"last":0.63300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":1716,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":1757184,"data_stored":785966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.61899999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.504},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.57299999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.59699999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:11:52.196 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:11:52.196 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:11:52.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.352+0000 7faeb89bf700 1 Processor -- start 2023-10-08T16:11:52.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.352+0000 7faeb89bf700 1 -- start start 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb89bf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeb4158d40 0x7faeb4140db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb89bf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faeb406c0a0 0x7faeb41412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb89bf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb415b790 con 0x7faeb406bcd0 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb89bf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb415ba70 con 0x7faeb4158d40 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb89bf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb415bd50 con 0x7faeb406c0a0 2023-10-08T16:11:52.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7faeb406bcd0 0x7faeb40c9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:49158/0 (socket says 172.21.15.170:49158) 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faea9d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeb4158d40 0x7faeb4140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb259c700 1 -- 172.21.15.170:0/1822587805 learned_addr learned my addr 172.21.15.170:0/1822587805 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.353+0000 7faeb2d9d700 1 --2- 172.21.15.170:0/1822587805 >> v2:172.21.15.191:3300/0 conn(0x7faeb406c0a0 0x7faeb41412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faea9d9b700 1 -- 172.21.15.170:0/1822587805 >> v2:172.21.15.191:3300/0 conn(0x7faeb406c0a0 msgr2=0x7faeb41412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faea9d9b700 1 --2- 172.21.15.170:0/1822587805 >> v2:172.21.15.191:3300/0 conn(0x7faeb406c0a0 0x7faeb41412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faea9d9b700 1 -- 172.21.15.170:0/1822587805 >> v1:172.21.15.170:6789/0 conn(0x7faeb406bcd0 legacy=0x7faeb40c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb1d9b700 1 -- 172.21.15.170:0/1822587805 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197155153 0 0) 0x7faeb415b790 con 0x7faeb406bcd0 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faea9d9b700 1 -- 172.21.15.170:0/1822587805 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb415c030 con 0x7faeb4158d40 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faea9d9b700 1 --2- 172.21.15.170:0/1822587805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb4158d40 0x7faeb4140db0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faeb40c9a60 tx=0x7fae9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6484ed4f154b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb1d9b700 1 -- 172.21.15.170:0/1822587805 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c011630 con 0x7faeb4158d40 2023-10-08T16:11:52.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb1d9b700 1 -- 172.21.15.170:0/1822587805 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c011790 con 0x7faeb4158d40 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb1d9b700 1 -- 172.21.15.170:0/1822587805 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c011b30 con 0x7faeb4158d40 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 -- 172.21.15.170:0/1822587805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb4158d40 msgr2=0x7faeb4140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 --2- 172.21.15.170:0/1822587805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb4158d40 0x7faeb4140db0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faeb40c9a60 tx=0x7fae9c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 -- 172.21.15.170:0/1822587805 shutdown_connections 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 --2- 172.21.15.170:0/1822587805 >> v2:172.21.15.191:3300/0 conn(0x7faeb406c0a0 0x7faeb41412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 --2- 172.21.15.170:0/1822587805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb4158d40 0x7faeb4140db0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.354+0000 7faeb89bf700 1 -- 172.21.15.170:0/1822587805 >> 172.21.15.170:0/1822587805 conn(0x7faeb40ce5c0 msgr2=0x7faeb40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.355+0000 7faeb89bf700 1 -- 172.21.15.170:0/1822587805 shutdown_connections 2023-10-08T16:11:52.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.355+0000 7faeb89bf700 1 -- 172.21.15.170:0/1822587805 wait complete. 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 Processor -- start 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 -- start start 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb406bcd0 0x7faeb41372b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 0x7faeb40b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40cdb00 con 0x7faeb406c0a0 2023-10-08T16:11:52.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb4159bd0 con 0x7faeb406bcd0 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.356+0000 7faeb89bf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb40cd540 con 0x7faeb4158d40 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43416/0 (socket says 172.21.15.170:43416) 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 -- 172.21.15.170:0/770833731 learned_addr learned my addr 172.21.15.170:0/770833731 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeb259c700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb406bcd0 0x7faeb41372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeb2d9d700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 0x7faeb40b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 -- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 msgr2=0x7faeb40b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 0x7faeb40b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 -- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb406bcd0 msgr2=0x7faeb41372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb406bcd0 0x7faeb41372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 -- 172.21.15.170:0/770833731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009420 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeb2d9d700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 0x7faeb40b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faea9d9b700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7faeb4069220 tx=0x7fae9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c011b10 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c0253f0 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb40cd820 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c025790 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.357+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb41377f0 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.358+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb40718b0 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.358+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae9c025b80 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.359+0000 7faeaaffd700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 0x7fae940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.359+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae9c0863e0 con 0x7faeb406c0a0 2023-10-08T16:11:52.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.359+0000 7faeb259c700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 0x7fae940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.359+0000 7faeb259c700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 0x7fae940697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faea4007fd0 tx=0x7faea4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.363+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae9c053070 con 0x7faeb406c0a0 2023-10-08T16:11:52.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.620+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faeb40cf1f0 con 0x7faeb406c0a0 2023-10-08T16:11:52.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.624+0000 7faeaaffd700 1 -- 172.21.15.170:0/770833731 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7fae9c056690 con 0x7faeb406c0a0 2023-10-08T16:11:52.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 msgr2=0x7fae940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 0x7fae940697d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7faea4007fd0 tx=0x7faea4007d30 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 msgr2=0x7faeb40b1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7faeb4069220 tx=0x7fae9c017000 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 shutdown_connections 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb4158d40 0x7faeb40b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb406bcd0 0x7faeb41372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae94067320 0x7fae940697d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 --2- 172.21.15.170:0/770833731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb406c0a0 0x7faeb40b1d00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 >> 172.21.15.170:0/770833731 conn(0x7faeb40ce5c0 msgr2=0x7faeb406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 shutdown_connections 2023-10-08T16:11:52.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.626+0000 7faeb89bf700 1 -- 172.21.15.170:0/770833731 wait complete. 2023-10-08T16:11:52.629 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:11:52.639 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:11:52.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.752+0000 7fd37484d700 1 Processor -- start 2023-10-08T16:11:52.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.752+0000 7fd37484d700 1 -- start start 2023-10-08T16:11:52.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd370159890 0x7fd37015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd370159c60 0x7fd37015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd37015db10 0x7fd370069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37015df50 con 0x7fd370159c60 2023-10-08T16:11:52.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37015a160 con 0x7fd370159890 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd37484d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700d13a0 con 0x7fd37015db10 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd365d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd370159c60 0x7fd37015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd365d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd370159c60 0x7fd37015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43420/0 (socket says 172.21.15.170:43420) 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd365d9b700 1 -- 172.21.15.170:0/3651955160 learned_addr learned my addr 172.21.15.170:0/3651955160 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.753+0000 7fd36e59c700 1 --2- 172.21.15.170:0/3651955160 >> v2:172.21.15.191:3300/0 conn(0x7fd370159890 0x7fd37015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd36ed9d700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd37015db10 0x7fd370069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 -- 172.21.15.170:0/3651955160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd37015db10 msgr2=0x7fd370069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd37015db10 0x7fd370069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 -- 172.21.15.170:0/3651955160 >> v2:172.21.15.191:3300/0 conn(0x7fd370159890 msgr2=0x7fd37015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:52.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 --2- 172.21.15.170:0/3651955160 >> v2:172.21.15.191:3300/0 conn(0x7fd370159890 0x7fd37015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 -- 172.21.15.170:0/3651955160 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37015e140 con 0x7fd370159c60 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd365d9b700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159c60 0x7fd37015d5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd3700d1090 tx=0x7fd358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c526ef55ec3901 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd36dd9b700 1 -- 172.21.15.170:0/3651955160 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800b860 con 0x7fd370159c60 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd36dd9b700 1 -- 172.21.15.170:0/3651955160 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800b9c0 con 0x7fd370159c60 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd36dd9b700 1 -- 172.21.15.170:0/3651955160 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800bd60 con 0x7fd370159c60 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 -- 172.21.15.170:0/3651955160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159c60 msgr2=0x7fd37015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159c60 0x7fd37015d5d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd3700d1090 tx=0x7fd358009250 comp rx=0 tx=0).stop 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 -- 172.21.15.170:0/3651955160 shutdown_connections 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd37015db10 0x7fd370069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 --2- 172.21.15.170:0/3651955160 >> v2:172.21.15.191:3300/0 conn(0x7fd370159890 0x7fd37015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 --2- 172.21.15.170:0/3651955160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159c60 0x7fd37015d5d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.754+0000 7fd37484d700 1 -- 172.21.15.170:0/3651955160 >> 172.21.15.170:0/3651955160 conn(0x7fd370068e20 msgr2=0x7fd3700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:52.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.755+0000 7fd37484d700 1 -- 172.21.15.170:0/3651955160 shutdown_connections 2023-10-08T16:11:52.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.755+0000 7fd37484d700 1 -- 172.21.15.170:0/3651955160 wait complete. 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 Processor -- start 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 -- start start 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd370159c60 0x7fd3700c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37015db10 0x7fd3700b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700ca6c0 con 0x7fd370159890 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700cda10 con 0x7fd37015db10 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.756+0000 7fd37484d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700ca3e0 con 0x7fd370159c60 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43422/0 (socket says 172.21.15.170:43422) 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 -- 172.21.15.170:0/2103533875 learned_addr learned my addr 172.21.15.170:0/2103533875 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 -- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd370159c60 msgr2=0x7fd3700c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd370159c60 0x7fd3700c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 -- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37015db10 msgr2=0x7fd3700b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37015db10 0x7fd3700b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 -- 172.21.15.170:0/2103533875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580094f0 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd36e59c700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd36000d860 tx=0x7fd360008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360009610 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360009770 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3700cd3c0 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360009b30 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.757+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd370148720 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.758+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37004c3c0 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.759+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd36001e460 con 0x7fd370159890 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.759+0000 7fd366ffd700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 0x7fd3500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:52.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.759+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd360085330 con 0x7fd370159890 2023-10-08T16:11:52.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.759+0000 7fd365d9b700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 0x7fd3500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:52.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.760+0000 7fd365d9b700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 0x7fd3500697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd3700d1090 tx=0x7fd358017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:52.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.763+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd360051fc0 con 0x7fd370159890 2023-10-08T16:11:52.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.992+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37006a5b0 con 0x7fd370159890 2023-10-08T16:11:52.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.992+0000 7fd366ffd700 1 -- 172.21.15.170:0/2103533875 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fd3600555e0 con 0x7fd370159890 2023-10-08T16:11:52.994 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:11:52.994 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":90,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:50.781439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 msgr2=0x7fd3500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 0x7fd3500697d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd3700d1090 tx=0x7fd358017000 comp rx=0 tx=0).stop 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 msgr2=0x7fd370148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd36000d860 tx=0x7fd360008040 comp rx=0 tx=0).stop 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 shutdown_connections 2023-10-08T16:11:52.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd37015db10 0x7fd3700b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd350067320 0x7fd3500697d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd370159890 0x7fd370148160 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 --2- 172.21.15.170:0/2103533875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd370159c60 0x7fd3700c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 >> 172.21.15.170:0/2103533875 conn(0x7fd370068e20 msgr2=0x7fd370158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 shutdown_connections 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:52.994+0000 7fd37484d700 1 -- 172.21.15.170:0/2103533875 wait complete. 2023-10-08T16:11:52.997 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 90 2023-10-08T16:11:53.085 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:11:53.085+0000 7f133a13c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:11:53.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:11:47.159522+0000', 'modified': '2023-10-08T16:11:50.781439+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7527': {'gid': 7527, 'name': 'd', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.191:6835/1124398845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.191:6833', 'nonce': 1124398845}, {'type': 'v1', 'addr': '172.21.15.191:6835', 'nonce': 1124398845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:11:53.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:11:53.410 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T16:11:53.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.527+0000 7f7593426700 1 Processor -- start 2023-10-08T16:11:53.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.527+0000 7f7593426700 1 -- start start 2023-10-08T16:11:53.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06bcd0 0x7f758c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f758c06c0a0 0x7f758c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f758c0c8e00 0x7f758c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c15c2a0 con 0x7f758c0c8e00 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c15b690 con 0x7f758c06c0a0 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f7593426700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c15b970 con 0x7f758c06bcd0 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f75919c3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f758c0c8e00 0x7f758c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f75919c3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f758c0c8e00 0x7f758c0c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43426/0 (socket says 172.21.15.170:43426) 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.528+0000 7f75919c3700 1 -- 172.21.15.170:0/2423872118 learned_addr learned my addr 172.21.15.170:0/2423872118 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75909c1700 1 --2- 172.21.15.170:0/2423872118 >> v2:172.21.15.191:3300/0 conn(0x7f758c06c0a0 0x7f758c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75911c2700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06bcd0 0x7f758c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 -- 172.21.15.170:0/2423872118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06bcd0 msgr2=0x7f758c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06bcd0 0x7f758c140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 -- 172.21.15.170:0/2423872118 >> v2:172.21.15.191:3300/0 conn(0x7f758c06c0a0 msgr2=0x7f758c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 --2- 172.21.15.170:0/2423872118 >> v2:172.21.15.191:3300/0 conn(0x7f758c06c0a0 0x7f758c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 -- 172.21.15.170:0/2423872118 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c15bc50 con 0x7f758c0c8e00 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f75919c3700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0c9260 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f758c0c5660 tx=0x7f7584009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ca2209a3dcc740 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f7583fff700 1 -- 172.21.15.170:0/2423872118 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400e070 con 0x7f758c0c8e00 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f7583fff700 1 -- 172.21.15.170:0/2423872118 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400ba50 con 0x7f758c0c8e00 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.529+0000 7f7583fff700 1 -- 172.21.15.170:0/2423872118 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840092e0 con 0x7f758c0c8e00 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 -- 172.21.15.170:0/2423872118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 msgr2=0x7f758c0c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0c9260 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f758c0c5660 tx=0x7f7584009840 comp rx=0 tx=0).stop 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 -- 172.21.15.170:0/2423872118 shutdown_connections 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0c9260 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 --2- 172.21.15.170:0/2423872118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06bcd0 0x7f758c140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 --2- 172.21.15.170:0/2423872118 >> v2:172.21.15.191:3300/0 conn(0x7f758c06c0a0 0x7f758c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 -- 172.21.15.170:0/2423872118 >> 172.21.15.170:0/2423872118 conn(0x7f758c0ce600 msgr2=0x7f758c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 -- 172.21.15.170:0/2423872118 shutdown_connections 2023-10-08T16:11:53.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.530+0000 7f7593426700 1 -- 172.21.15.170:0/2423872118 wait complete. 2023-10-08T16:11:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.531+0000 7f7593426700 1 Processor -- start 2023-10-08T16:11:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.531+0000 7f7593426700 1 -- start start 2023-10-08T16:11:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 0x7f758c13c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06c0a0 0x7f758c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c13c850 con 0x7f758c0c8e00 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0c6360 con 0x7f758c06bcd0 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f7593426700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c0c5750 con 0x7f758c06c0a0 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75911c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 0x7f758c13c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43442/0 (socket says 172.21.15.170:43442) 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75911c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 0x7f758c13c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:49448/0 (socket says 172.21.15.170:49448) 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 -- 172.21.15.170:0/220059292 learned_addr learned my addr 172.21.15.170:0/220059292 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75909c1700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06c0a0 0x7f758c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 -- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06c0a0 msgr2=0x7f758c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06c0a0 0x7f758c0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 -- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 msgr2=0x7f758c13c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 0x7f758c13c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 -- 172.21.15.170:0/220059292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009ae0 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.532+0000 7f75919c3700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f758c069220 tx=0x7f7584018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.533+0000 7f75817fa700 1 -- 172.21.15.170:0/220059292 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75840032f0 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.533+0000 7f75817fa700 1 -- 172.21.15.170:0/220059292 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584008d20 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.533+0000 7f75817fa700 1 -- 172.21.15.170:0/220059292 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758401c640 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.533+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c0c5a30 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.533+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c0c5ff0 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.534+0000 7f75817fa700 1 -- 172.21.15.170:0/220059292 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7584003040 con 0x7f758c0c8e00 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.535+0000 7f75817fa700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 0x7f75700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.535+0000 7f75911c2700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 0x7f75700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.535+0000 7f75817fa700 1 -- 172.21.15.170:0/220059292 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f75840865a0 con 0x7f758c0c8e00 2023-10-08T16:11:53.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.535+0000 7f75911c2700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 0x7f75700697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7588006fd0 tx=0x7f7588008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.536+0000 7f759c71eb80 1 Processor -- start 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 -- start start 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd58575a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd58548cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd584a3a70 con 0x55cd5854ca50 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd584a1270 con 0x55cd58596d00 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5849d060 con 0x55cd58575f60 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43456/0 (socket says 172.21.15.170:43456) 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.537+0000 7f75911c2700 1 -- 172.21.15.170:0/3402827703 learned_addr learned my addr 172.21.15.170:0/3402827703 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 -- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 msgr2=0x55cd58548cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd58548cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 -- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 msgr2=0x55cd58575a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75909c1700 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd58575a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd58575a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 -- 172.21.15.170:0/3402827703 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd58337270 con 0x55cd5854ca50 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75911c2700 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f758801b4c0 tx=0x7f758801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb3e111762bc833 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75767fc700 1 -- 172.21.15.170:0/3402827703 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588026400 con 0x55cd5854ca50 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75767fc700 1 -- 172.21.15.170:0/3402827703 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588015040 con 0x55cd5854ca50 2023-10-08T16:11:53.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f75767fc700 1 -- 172.21.15.170:0/3402827703 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588026770 con 0x55cd5854ca50 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f759c71eb80 1 -- 172.21.15.170:0/3402827703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 msgr2=0x55cd5854ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.538+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f758801b4c0 tx=0x7f758801b000 comp rx=0 tx=0).stop 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 -- 172.21.15.170:0/3402827703 shutdown_connections 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd58548cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5854ce20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3402827703 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd58575a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 -- 172.21.15.170:0/3402827703 >> 172.21.15.170:0/3402827703 conn(0x55cd5854e370 msgr2=0x55cd5854e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 -- 172.21.15.170:0/3402827703 shutdown_connections 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 -- 172.21.15.170:0/3402827703 wait complete. 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 -- start start 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.539+0000 7f759c71eb80 1 Processor -- start 2023-10-08T16:11:53.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd5858f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd58596660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd58448890 con 0x55cd5854ca50 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd58463ea0 con 0x55cd58575f60 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5831fe30 con 0x55cd58596d00 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f759c71eb80 10 client.0 _ll_register_callbacks cb 0x55cd58510110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43464/0 (socket says 172.21.15.170:43464) 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 -- 172.21.15.170:0/3458012563 learned_addr learned my addr 172.21.15.170:0/3458012563 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75919c3700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd58596660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75909c1700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd5858f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 -- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 msgr2=0x55cd58596660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd58596660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 -- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 msgr2=0x55cd5858f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd5858f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.540+0000 7f75911c2700 1 -- 172.21.15.170:0/3458012563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588014c30 con 0x55cd5854ca50 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f75911c2700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f758800f8a0 tx=0x7f758801d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9359fe09a65291 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588026820 con 0x55cd5854ca50 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7588031660 con 0x55cd5854ca50 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd58541b70 con 0x55cd5854ca50 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7588031a30 con 0x55cd5854ca50 2023-10-08T16:11:53.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 10 client.7659 fetch_fsmap learned FSMap version 90 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55cd582d9fe0 con 0x55cd5854ca50 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd58541b70 con 0x55cd5854ca50 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7588031b90 con 0x55cd5854ca50 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 10 client.7659 fetch_fsmap finished waiting for FSMap version 90 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 10 client.7659 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 20 client.7659 populate_metadata read hostname 'smithi170' 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd5857be80 0x55cd5857c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 4 client.7659 mds_command: new command op to 7527 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 --> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd5851eaa0 con 0x55cd5857be80 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.541+0000 7f75909c1700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd5857be80 0x55cd5857c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.542+0000 7f756b7fe700 10 client.7659 ms_handle_connect on v2:172.21.15.191:6833/1124398845 2023-10-08T16:11:53.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.542+0000 7f75909c1700 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd5857be80 0x55cd5857c250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40784d7e326f2926 server_cookie=a60182e8c26f882d in_seq=0 out_seq=0 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.542+0000 7f756b7fe700 1 -- 172.21.15.170:0/3458012563 <== mds.0 v2:172.21.15.191:6833/1124398845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55cd5851eaa0 con 0x55cd5857be80 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.542+0000 7f756b7fe700 10 client.7659 handle_command_reply: tid=0 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 client.7659 shutdown 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 msgr2=0x55cd5858ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f758800f8a0 tx=0x7f758801d030 comp rx=0 tx=0).stop 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 shutdown_connections 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd5857be80 0x55cd5857c250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd58596660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858ac20 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f756b7fe700 0 client.7659 ms_handle_reset on v2:172.21.15.191:6833/1124398845 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3458012563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd5858f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 >> 172.21.15.170:0/3458012563 conn(0x55cd5854e370 msgr2=0x55cd58549880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 shutdown_connections 2023-10-08T16:11:53.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 1 -- 172.21.15.170:0/3458012563 wait complete. 2023-10-08T16:11:53.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.543+0000 7f759c71eb80 20 client.7659 trim_cache size 0 max 16384 2023-10-08T16:11:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.559+0000 7f759c71eb80 1 Processor -- start 2023-10-08T16:11:53.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 -- start start 2023-10-08T16:11:53.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd585a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd585a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd585797b0 con 0x55cd5854ca50 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd58572f10 con 0x55cd58575f60 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd5854b910 con 0x55cd58596d00 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43468/0 (socket says 172.21.15.170:43468) 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.560+0000 7f75911c2700 1 -- 172.21.15.170:0/3287476165 learned_addr learned my addr 172.21.15.170:0/3287476165 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 -- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 msgr2=0x55cd585a2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd585a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 -- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 msgr2=0x55cd585a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75909c1700 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd585a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd585a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 -- 172.21.15.170:0/3287476165 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd5846b780 con 0x55cd5854ca50 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75909c1700 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd585a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f75911c2700 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f758800fd20 tx=0x7f758802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cb2f3c05e91c6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f7576ffd700 1 -- 172.21.15.170:0/3287476165 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588031660 con 0x55cd5854ca50 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f7576ffd700 1 -- 172.21.15.170:0/3287476165 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588026660 con 0x55cd5854ca50 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f7576ffd700 1 -- 172.21.15.170:0/3287476165 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588031c40 con 0x55cd5854ca50 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 -- 172.21.15.170:0/3287476165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 msgr2=0x55cd5858dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f758800fd20 tx=0x7f758802d000 comp rx=0 tx=0).stop 2023-10-08T16:11:53.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 -- 172.21.15.170:0/3287476165 shutdown_connections 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58596d00 0x55cd585a2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd5858dd30 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 --2- 172.21.15.170:0/3287476165 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58575f60 0x55cd585a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 -- 172.21.15.170:0/3287476165 >> 172.21.15.170:0/3287476165 conn(0x55cd5854e370 msgr2=0x55cd5857c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 -- 172.21.15.170:0/3287476165 shutdown_connections 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.561+0000 7f759c71eb80 1 -- 172.21.15.170:0/3287476165 wait complete. 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.562+0000 7f759c71eb80 1 -- start start 2023-10-08T16:11:53.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.562+0000 7f759c71eb80 1 Processor -- start 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd585b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd585b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd5854d430 con 0x55cd5854ca50 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd585b2b20 con 0x55cd58596d00 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd585754f0 con 0x55cd58575f60 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f759c71eb80 10 client.0 _ll_register_callbacks cb 0x55cd58510110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43474/0 (socket says 172.21.15.170:43474) 2023-10-08T16:11:53.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 -- 172.21.15.170:0/1232827332 learned_addr learned my addr 172.21.15.170:0/1232827332 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75909c1700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd585b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75919c3700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd585b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 -- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 msgr2=0x55cd585b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd585b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 -- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 msgr2=0x55cd585b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd585b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 -- 172.21.15.170:0/1232827332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd582d9fe0 con 0x55cd5854ca50 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75909c1700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd585b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f75911c2700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f75880141e0 tx=0x7f7588028070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1621dcf539dbd7d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f756b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:11:53.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.563+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588031c40 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7588005a00 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7588015040 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7588005dd0 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 10 client.7665 fetch_fsmap learned FSMap version 90 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55cd58452470 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588015040 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 90) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7588006040 con 0x55cd5854ca50 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 10 client.7665 fetch_fsmap finished waiting for FSMap version 90 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 10 client.7665 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 20 client.7665 populate_metadata read hostname 'smithi170' 2023-10-08T16:11:53.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd585f7ea0 0x55cd5858d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 4 client.7665 mds_command: new command op to 7527 tid=0[{"prefix": "status"}] 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 --> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cd5843fe00 con 0x55cd585f7ea0 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.564+0000 7f75909c1700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd585f7ea0 0x55cd5858d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f756b7fe700 10 client.7665 ms_handle_connect on v2:172.21.15.191:6833/1124398845 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f75909c1700 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd585f7ea0 0x55cd5858d6a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11426ead7bd71a02 server_cookie=ffbdb8f782823827 in_seq=0 out_seq=0 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f756b7fe700 1 -- 172.21.15.170:0/1232827332 <== mds.0 v2:172.21.15.191:6833/1124398845 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cd5843fe00 con 0x55cd585f7ea0 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f756b7fe700 10 client.7665 handle_command_reply: tid=0 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 client.7665 shutdown 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 msgr2=0x55cd58575980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f75880141e0 tx=0x7f7588028070 comp rx=0 tx=0).stop 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 shutdown_connections 2023-10-08T16:11:53.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:6833/1124398845,v1:172.21.15.191:6835/1124398845] conn(0x55cd585f7ea0 0x55cd5858d6a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55cd58596d00 0x55cd585b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd5854ca50 0x55cd58575980 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 --2- 172.21.15.170:0/1232827332 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55cd58575f60 0x55cd585b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f756b7fe700 0 client.7665 ms_handle_reset on v2:172.21.15.191:6833/1124398845 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 >> 172.21.15.170:0/1232827332 conn(0x55cd5854e370 msgr2=0x55cd5857c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 shutdown_connections 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 1 -- 172.21.15.170:0/1232827332 wait complete. 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.565+0000 7f759c71eb80 20 client.7665 trim_cache size 0 max 16384 2023-10-08T16:11:53.574 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7527, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.7876774290000004, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 90, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 139, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 139, 2023-10-08T16:11:53.575 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.895336216 2023-10-08T16:11:53.576 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:11:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 msgr2=0x7f75700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 0x7f75700697d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7588006fd0 tx=0x7f7588008040 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 msgr2=0x7f758c0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f758c069220 tx=0x7f7584018000 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 shutdown_connections 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7570067320 0x7f75700697d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f758c0c8e00 0x7f758c0cd960 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f758c06bcd0 0x7f758c13c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 --2- 172.21.15.170:0/220059292 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f758c06c0a0 0x7f758c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 >> 172.21.15.170:0/220059292 conn(0x7f758c0ce600 msgr2=0x7f758c0c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 shutdown_connections 2023-10-08T16:11:53.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.566+0000 7f7593426700 1 -- 172.21.15.170:0/220059292 wait complete. 2023-10-08T16:11:53.579 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:11:53.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.691+0000 7fb10a5c2700 1 Processor -- start 2023-10-08T16:11:53.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.691+0000 7fb10a5c2700 1 -- start start 2023-10-08T16:11:53.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb10a5c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb1040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb10a5c2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb10415d230 0x7fb10415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb10a5c2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10415dbd0 con 0x7fb104159420 2023-10-08T16:11:53.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb10a5c2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1040b11d0 con 0x7fb10415d230 2023-10-08T16:11:53.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb10a5c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1040b14b0 con 0x7fb1041597f0 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb109dc1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb10415d230 0x7fb10415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb108dbf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb1040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb109dc1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb10415d230 0x7fb10415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:49496/0 (socket says 172.21.15.170:49496) 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb109dc1700 1 -- 172.21.15.170:0/2859330583 learned_addr learned my addr 172.21.15.170:0/2859330583 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.692+0000 7fb1095c0700 1 --1- 172.21.15.170:0/2859330583 >> v1:172.21.15.191:6789/0 conn(0x7fb104159420 0x7fb1040b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:44432/0 (socket says 172.21.15.170:44432) 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb109dc1700 1 -- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 msgr2=0x7fb1040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb109dc1700 1 --2- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb1040b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb109dc1700 1 -- 172.21.15.170:0/2859330583 >> v1:172.21.15.191:6789/0 conn(0x7fb104159420 legacy=0x7fb1040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb109dc1700 1 -- 172.21.15.170:0/2859330583 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1040b1790 con 0x7fb10415d230 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb0fbfff700 1 -- 172.21.15.170:0/2859330583 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253023944 0 0) 0x7fb100001490 con 0x7fb104159420 2023-10-08T16:11:53.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb109dc1700 1 --2- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb10415d230 0x7fb10415d690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0099b0 tx=0x7fb0fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2d53e3810a069e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb0fbfff700 1 -- 172.21.15.170:0/2859330583 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc009ea0 con 0x7fb10415d230 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb0fbfff700 1 -- 172.21.15.170:0/2859330583 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00b860 con 0x7fb10415d230 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb0fbfff700 1 -- 172.21.15.170:0/2859330583 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00bc00 con 0x7fb10415d230 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb10a5c2700 1 -- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb10415d230 msgr2=0x7fb10415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.693+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb10415d230 0x7fb10415d690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0099b0 tx=0x7fb0fc009250 comp rx=0 tx=0).stop 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 -- 172.21.15.170:0/2859330583 shutdown_connections 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb10415d230 0x7fb10415d690 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/2859330583 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb1040b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 -- 172.21.15.170:0/2859330583 >> 172.21.15.170:0/2859330583 conn(0x7fb1040d0660 msgr2=0x7fb1040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 -- 172.21.15.170:0/2859330583 shutdown_connections 2023-10-08T16:11:53.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.694+0000 7fb10a5c2700 1 -- 172.21.15.170:0/2859330583 wait complete. 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.695+0000 7fb10a5c2700 1 Processor -- start 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.695+0000 7fb10a5c2700 1 -- start start 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb104159420 0x7fb1041550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb10415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104169a40 con 0x7fb10415d230 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10415c2b0 con 0x7fb104159420 2023-10-08T16:11:53.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10415c590 con 0x7fb1041597f0 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43476/0 (socket says 172.21.15.170:43476) 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 -- 172.21.15.170:0/205615905 learned_addr learned my addr 172.21.15.170:0/205615905 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb108dbf700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb10415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb1095c0700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb104159420 0x7fb1041550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 -- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 msgr2=0x7fb10415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb10415ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 -- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb104159420 msgr2=0x7fb1041550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb104159420 0x7fb1041550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 -- 172.21.15.170:0/205615905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0094f0 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb109dc1700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb104089340 tx=0x7fb0fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011630 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0117b0 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10415c870 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.696+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc011b70 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.697+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10415bf50 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.697+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040d01f0 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.698+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0fc0253f0 con 0x7fb10415d230 2023-10-08T16:11:53.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.698+0000 7fb0f97fa700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 0x7fb0e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:53.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.698+0000 7fb1095c0700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 0x7fb0e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:53.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.698+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb0fc086700 con 0x7fb10415d230 2023-10-08T16:11:53.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.699+0000 7fb1095c0700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 0x7fb0e80697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb100001ff0 tx=0x7fb100005cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:53.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.703+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0fc053410 con 0x7fb10415d230 2023-10-08T16:11:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.931+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb104169b80 con 0x7fb10415d230 2023-10-08T16:11:53.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.932+0000 7fb0f97fa700 1 -- 172.21.15.170:0/205615905 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0fc015060 con 0x7fb10415d230 2023-10-08T16:11:53.933 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:11:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 msgr2=0x7fb0e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 0x7fb0e80697d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb100001ff0 tx=0x7fb100005cf0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 msgr2=0x7fb10415f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb104089340 tx=0x7fb0fc017000 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 shutdown_connections 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e8067320 0x7fb0e80697d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb10415d230 0x7fb10415f340 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb104159420 0x7fb1041550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 --2- 172.21.15.170:0/205615905 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb1041597f0 0x7fb10415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 >> 172.21.15.170:0/205615905 conn(0x7fb1040d0660 msgr2=0x7fb1040d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:53.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 shutdown_connections 2023-10-08T16:11:53.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:53.934+0000 7fb10a5c2700 1 -- 172.21.15.170:0/205615905 wait complete. 2023-10-08T16:11:53.947 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:11:53.948 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:11:53.949 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:53.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:53.949 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:11:53.964 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:11:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:11:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:11:53.964 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:53.965 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42037sec preferred_lft 42037sec 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:11:53.966 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:11:53.967 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:53.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:53.968 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:11:54.027 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:11:54.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.028 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:11:54.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:11:54.086 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:11:54.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.185 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:11:54.199 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:11:54.199 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:11:54.200 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:54.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.200 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:11:54.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:54.265 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:54.319 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:11:54.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:54.320 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:54.374 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:11:54.374 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:11:54.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:54.453 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:54.453 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:54.472 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:54.472 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:54.472 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:54.472 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:54.473 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:54.473 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:54.473 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:54.473 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:54.496 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:11:54.497 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:11:54.497 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:11:54.498 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:11:54.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.502 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:54.563 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:11:54.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.716+0000 7f6215865700 1 Processor -- start 2023-10-08T16:11:54.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.716+0000 7f6215865700 1 -- start start 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f6215865700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159890 0x7f621015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f6215865700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6210159c60 0x7f621015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f6215865700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621015a160 con 0x7f621015db10 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f6215865700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100d13a0 con 0x7f6210159c60 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f6215865700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621015e140 con 0x7f6210159890 2023-10-08T16:11:54.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6210159c60 0x7f621015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6210159c60 0x7f621015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43490/0 (socket says 172.21.15.170:43490) 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 -- 172.21.15.170:0/1328983678 learned_addr learned my addr 172.21.15.170:0/1328983678 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f620effd700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159890 0x7f621015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 -- 172.21.15.170:0/1328983678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159890 msgr2=0x7f621015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159890 0x7f621015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 -- 172.21.15.170:0/1328983678 >> v1:172.21.15.191:6789/0 conn(0x7f621015db10 legacy=0x7f6210069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 -- 172.21.15.170:0/1328983678 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621015a3d0 con 0x7f6210159c60 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.717+0000 7f62067fc700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6210159c60 0x7f621015d5d0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f61f80099b0 tx=0x7f61f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4e28176d1e6ba3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f620e7fc700 1 -- 172.21.15.170:0/1328983678 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f800b820 con 0x7f6210159c60 2023-10-08T16:11:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f620e7fc700 1 -- 172.21.15.170:0/1328983678 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f800b980 con 0x7f6210159c60 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f620e7fc700 1 -- 172.21.15.170:0/1328983678 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f800bd20 con 0x7f6210159c60 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 -- 172.21.15.170:0/1328983678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6210159c60 msgr2=0x7f621015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6210159c60 0x7f621015d5d0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f61f80099b0 tx=0x7f61f8009250 comp rx=0 tx=0).stop 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 -- 172.21.15.170:0/1328983678 shutdown_connections 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159890 0x7f621015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 --2- 172.21.15.170:0/1328983678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6210159c60 0x7f621015d5d0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 -- 172.21.15.170:0/1328983678 >> 172.21.15.170:0/1328983678 conn(0x7f6210068e20 msgr2=0x7f62100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 -- 172.21.15.170:0/1328983678 shutdown_connections 2023-10-08T16:11:54.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.718+0000 7f6215865700 1 -- 172.21.15.170:0/1328983678 wait complete. 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.719+0000 7f6215865700 1 Processor -- start 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.719+0000 7f6215865700 1 -- start start 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6210159890 0x7f62100ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159c60 0x7f6210140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100c7f90 con 0x7f621015db10 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62100cfee0 con 0x7f6210159890 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f6215865700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62100cfc00 con 0x7f6210159c60 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f620f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f620f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43492/0 (socket says 172.21.15.170:43492) 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f620f7fe700 1 -- 172.21.15.170:0/2972678096 learned_addr learned my addr 172.21.15.170:0/2972678096 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f620effd700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6210159890 0x7f62100ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.720+0000 7f62067fc700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159c60 0x7f6210140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 -- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159c60 msgr2=0x7f6210140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159c60 0x7f6210140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 -- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6210159890 msgr2=0x7f62100ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6210159890 0x7f62100ca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 -- 172.21.15.170:0/2972678096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f80094f0 con 0x7f621015db10 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f620f7fe700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f620800bc30 tx=0x7f6208009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800c8d0 con 0x7f621015db10 2023-10-08T16:11:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620800ca70 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62100c7940 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620800ce30 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.721+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6210141300 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.722+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62100c8100 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.722+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6208014620 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.723+0000 7f62077fe700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 0x7f61f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.723+0000 7f620effd700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 0x7f61f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.723+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6208084c90 con 0x7f621015db10 2023-10-08T16:11:54.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.723+0000 7f620effd700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 0x7f61f00697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f620000acb0 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:54.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.727+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6208051920 con 0x7f621015db10 2023-10-08T16:11:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.957+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f621004c3c0 con 0x7f621015db10 2023-10-08T16:11:54.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.958+0000 7f62077fe700 1 -- 172.21.15.170:0/2972678096 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6208054f40 con 0x7f621015db10 2023-10-08T16:11:54.959 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:11:54.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 msgr2=0x7f61f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:54.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 0x7f61f00697d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f620000acb0 tx=0x7f6200008040 comp rx=0 tx=0).stop 2023-10-08T16:11:54.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 msgr2=0x7f62100cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f620800bc30 tx=0x7f6208009820 comp rx=0 tx=0).stop 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 shutdown_connections 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f61f0067320 0x7f61f00697d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f621015db10 0x7f62100cf3d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6210159890 0x7f62100ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 --2- 172.21.15.170:0/2972678096 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6210159c60 0x7f6210140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 >> 172.21.15.170:0/2972678096 conn(0x7f6210068e20 msgr2=0x7f6210069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 shutdown_connections 2023-10-08T16:11:54.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:54.960+0000 7f6215865700 1 -- 172.21.15.170:0/2972678096 wait complete. 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:11:54.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:11:54.975 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:54.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.975 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:11:54.993 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:54.994 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42036sec preferred_lft 42036sec 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:11:54.995 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:11:54.996 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:11:54.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:54.997 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:11:55.061 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:11:55.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:55.063 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:11:55.122 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:11:55.122 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:11:55.122 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:11:55.122 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:11:55.123 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:11:55.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:11:55.123 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:11:55.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:55.228 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:11:55.243 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:11:55.243 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:11:55.243 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:55.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:55.244 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:11:55.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:55.308 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:55.363 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:11:55.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:11:55.363 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:55.417 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:11:55.417 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:11:55.417 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:55.496 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:55.496 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:55.515 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:55.515 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:55.515 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:55.515 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:55.515 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:55.516 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:55.516 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:55.516 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:11:55.537 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:11:55.538 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:11:55.538 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:11:55.538 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:11:55.538 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:11:55.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:11:55.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:55.539 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:55.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:11:55.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.722+0000 7f25cd1c6700 1 Processor -- start 2023-10-08T16:11:55.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.722+0000 7f25cd1c6700 1 -- start start 2023-10-08T16:11:55.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25cd1c6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25c8157720 0x7f25c81589f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25cd1c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c806e7b0 0x7f25c806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25cd1c6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c8158560 con 0x7f25c80acf10 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25cd1c6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c806f140 con 0x7f25c8157720 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25cd1c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c806f320 con 0x7f25c806e7b0 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25c6d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25c8157720 0x7f25c81589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25c6d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25c8157720 0x7f25c81589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52932/0 (socket says 172.21.15.170:52932) 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.723+0000 7f25c6d9d700 1 -- 172.21.15.170:0/3522052997 learned_addr learned my addr 172.21.15.170:0/3522052997 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c6d9d700 1 -- 172.21.15.170:0/3522052997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c806e7b0 msgr2=0x7f25c806ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c6d9d700 1 --2- 172.21.15.170:0/3522052997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c806e7b0 0x7f25c806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c6d9d700 1 -- 172.21.15.170:0/3522052997 >> v1:172.21.15.191:6789/0 conn(0x7f25c80acf10 legacy=0x7f25c815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c6d9d700 1 -- 172.21.15.170:0/3522052997 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c806f600 con 0x7f25c8157720 2023-10-08T16:11:55.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c6d9d700 1 --2- 172.21.15.170:0/3522052997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c8157720 0x7f25c81589f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f25b80099b0 tx=0x7f25b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804a877b38e22eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c659c700 1 -- 172.21.15.170:0/3522052997 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800b820 con 0x7f25c8157720 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c659c700 1 -- 172.21.15.170:0/3522052997 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800b980 con 0x7f25c8157720 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25c659c700 1 -- 172.21.15.170:0/3522052997 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800bd20 con 0x7f25c8157720 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 -- 172.21.15.170:0/3522052997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c8157720 msgr2=0x7f25c81589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/3522052997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c8157720 0x7f25c81589f0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f25b80099b0 tx=0x7f25b8009250 comp rx=0 tx=0).stop 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 -- 172.21.15.170:0/3522052997 shutdown_connections 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/3522052997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c8157720 0x7f25c81589f0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/3522052997 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c806e7b0 0x7f25c806ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 -- 172.21.15.170:0/3522052997 >> 172.21.15.170:0/3522052997 conn(0x7f25c80cd640 msgr2=0x7f25c80cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 -- 172.21.15.170:0/3522052997 shutdown_connections 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.724+0000 7f25cd1c6700 1 -- 172.21.15.170:0/3522052997 wait complete. 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 Processor -- start 2023-10-08T16:11:55.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 -- start start 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c80acf10 0x7f25c80c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8157720 0x7f25c80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c1d80 con 0x7f25c806e7b0 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c11b0 con 0x7f25c8157720 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25cd1c6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c1450 con 0x7f25c80acf10 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25c6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25c6d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52946/0 (socket says 172.21.15.170:52946) 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.726+0000 7f25c6d9d700 1 -- 172.21.15.170:0/637652628 learned_addr learned my addr 172.21.15.170:0/637652628 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25be59c700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c80acf10 0x7f25c80c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c759e700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8157720 0x7f25c80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:55.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 -- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c80acf10 msgr2=0x7f25c80c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c80acf10 0x7f25c80c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 -- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8157720 msgr2=0x7f25c80c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8157720 0x7f25c80c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 -- 172.21.15.170:0/637652628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80094f0 con 0x7f25c806e7b0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25c6d9d700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009290 tx=0x7f25b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8011e40 con 0x7f25c806e7b0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b801f070 con 0x7f25c806e7b0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c80c1730 con 0x7f25c806e7b0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80266b0 con 0x7f25c806e7b0 2023-10-08T16:11:55.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.727+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c80c2210 con 0x7f25c806e7b0 2023-10-08T16:11:55.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.728+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c80c2060 con 0x7f25c806e7b0 2023-10-08T16:11:55.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.728+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25b801a440 con 0x7f25c806e7b0 2023-10-08T16:11:55.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.729+0000 7f25bf7fe700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 0x7f25a8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:55.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.729+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f25b8021070 con 0x7f25c806e7b0 2023-10-08T16:11:55.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.729+0000 7f25be59c700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 0x7f25a8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:55.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.729+0000 7f25be59c700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 0x7f25a8069800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f25c816b680 tx=0x7f25b00090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:55.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.733+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25b8053970 con 0x7f25c806e7b0 2023-10-08T16:11:55.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:55.964+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f25c815b1b0 con 0x7f25c806e7b0 2023-10-08T16:11:56.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.804+0000 7f25bf7fe700 1 -- 172.21.15.170:0/637652628 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v91) v1 ==== 136+0+0 (secure 0 0 0) 0x7f25b8056f90 con 0x7f25c806e7b0 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 msgr2=0x7f25a8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 0x7f25a8069800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f25c816b680 tx=0x7f25b00090f0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 msgr2=0x7f25c80c3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009290 tx=0x7f25b8016000 comp rx=0 tx=0).stop 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 shutdown_connections 2023-10-08T16:11:56.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c8157720 0x7f25c80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067350 0x7f25a8069800 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c806e7b0 0x7f25c80c3060 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 --2- 172.21.15.170:0/637652628 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c80acf10 0x7f25c80c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 >> 172.21.15.170:0/637652628 conn(0x7f25c80cd640 msgr2=0x7f25c8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 shutdown_connections 2023-10-08T16:11:56.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.806+0000 7f25cd1c6700 1 -- 172.21.15.170:0/637652628 wait complete. 2023-10-08T16:11:56.820 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:11:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.935+0000 7f0301bca700 1 Processor -- start 2023-10-08T16:11:56.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.935+0000 7f0301bca700 1 -- start start 2023-10-08T16:11:56.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f0301bca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02fc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f0301bca700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc0b2980 con 0x7f02fc15d230 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f0301bca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc0b11d0 con 0x7f02fc1597f0 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f0301bca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0b14b0 con 0x7f02fc159420 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f03013c9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f02fc15d230 0x7f02fc0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51920/0 (socket says 172.21.15.170:51920) 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f03013c9700 1 -- 172.21.15.170:0/4167908326 learned_addr learned my addr 172.21.15.170:0/4167908326 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307391628 0 0) 0x7f02fc0b2980 con 0x7f02fc15d230 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ec003570 con 0x7f02fc15d230 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f0300bc8700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02fc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3317004673 0 0) 0x7f02ec003570 con 0x7f02fc15d230 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 msgr2=0x7f02fc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02fc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 >> v1:172.21.15.191:6789/0 conn(0x7f02fc1597f0 legacy=0x7f02fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc0b1790 con 0x7f02fc15d230 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f02f40017f0 con 0x7f02fc15d230 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.936+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 >> v1:172.21.15.170:6789/0 conn(0x7f02fc15d230 legacy=0x7f02fc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec003a90 0x7f02ec003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec004810 0x7f02ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0b14b0 con 0x7f02ec003a90 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec00bb10 con 0x7f02ec004810 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec00bcd0 con 0x7f02fc159420 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec003a90 0x7f02ec003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fbfff700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec004810 0x7f02ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f03013c9700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 -- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 msgr2=0x7f02ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02fc159420 0x7f02ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 -- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec004810 msgr2=0x7f02ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec004810 0x7f02ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 -- 172.21.15.170:0/4167908326 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ec003570 con 0x7f02ec003a90 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fbfff700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02ec004810 0x7f02ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f0300bc8700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec003a90 0x7f02ec003e60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009c00 tx=0x7f02f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1c0c6ea67923fe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000e710 con 0x7f02ec003a90 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f000e870 con 0x7f02ec003a90 2023-10-08T16:11:56.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.937+0000 7f02fb7fe700 1 -- 172.21.15.170:0/4167908326 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f000ec30 con 0x7f02ec003a90 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 -- 172.21.15.170:0/4167908326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec003a90 msgr2=0x7f02ec003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 --2- 172.21.15.170:0/4167908326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02ec003a90 0x7f02ec003e60 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009c00 tx=0x7f02f00091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f03013c9700 1 -- 172.21.15.170:0/4167908326 reap_dead start 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 -- 172.21.15.170:0/4167908326 shutdown_connections 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 -- 172.21.15.170:0/4167908326 >> 172.21.15.170:0/4167908326 conn(0x7f02fc0d0660 msgr2=0x7f02fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 -- 172.21.15.170:0/4167908326 shutdown_connections 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.938+0000 7f0301bca700 1 -- 172.21.15.170:0/4167908326 wait complete. 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.939+0000 7f0301bca700 1 Processor -- start 2023-10-08T16:11:56.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.939+0000 7f0301bca700 1 -- start start 2023-10-08T16:11:56.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec003a90 0x7f02fc15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc15cc50 0x7f02fc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc169c40 con 0x7f02fc15c2a0 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc15b8d0 con 0x7f02fc15cc50 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc154cf0 con 0x7f02ec003a90 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52972/0 (socket says 172.21.15.170:52972) 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 -- 172.21.15.170:0/1125647146 learned_addr learned my addr 172.21.15.170:0/1125647146 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 -- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec003a90 msgr2=0x7f02fc15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec003a90 0x7f02fc15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 -- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc15cc50 msgr2=0x7f02fc1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc15cc50 0x7f02fc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 -- 172.21.15.170:0/1125647146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc158410 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02fbfff700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e860 tx=0x7f02e8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800c5a0 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800c700 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc15afa0 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800caa0 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.940+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc15b560 con 0x7f02fc15c2a0 2023-10-08T16:11:56.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.941+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc04c3c0 con 0x7f02fc15c2a0 2023-10-08T16:11:56.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.942+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f02e801d460 con 0x7f02fc15c2a0 2023-10-08T16:11:56.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.942+0000 7f02f8ff9700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 0x7f02e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:56.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.942+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f02e8084bb0 con 0x7f02fc15c2a0 2023-10-08T16:11:56.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.942+0000 7f0300bc8700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 0x7f02e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:56.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.943+0000 7f0300bc8700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 0x7f02e00697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02f00091e0 tx=0x7f02f001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:56.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:56.946+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02e8051840 con 0x7f02fc15c2a0 2023-10-08T16:11:57.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.154+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f02fc15ba40 con 0x7f02e0067320 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.156+0000 7f02f8ff9700 1 -- 172.21.15.170:0/1125647146 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f02fc15ba40 con 0x7f02e0067320 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:11:57.158 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:11:57.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 msgr2=0x7f02e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 0x7f02e00697d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f02f00091e0 tx=0x7f02f001a000 comp rx=0 tx=0).stop 2023-10-08T16:11:57.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 msgr2=0x7f02fc15c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f02e800e860 tx=0x7f02e8009040 comp rx=0 tx=0).stop 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 shutdown_connections 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f02fc15cc50 0x7f02fc1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f02ec003a90 0x7f02fc15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f02e0067320 0x7f02e00697d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 --2- 172.21.15.170:0/1125647146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02fc15c2a0 0x7f02fc15c710 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 >> 172.21.15.170:0/1125647146 conn(0x7f02fc0d0660 msgr2=0x7f02fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 shutdown_connections 2023-10-08T16:11:57.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.158+0000 7f0301bca700 1 -- 172.21.15.170:0/1125647146 wait complete. 2023-10-08T16:11:57.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:11:57.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.287+0000 7f6ac2fdd700 1 Processor -- start 2023-10-08T16:11:57.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.287+0000 7f6ac2fdd700 1 -- start start 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac2fdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6abc06aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac2fdd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc06bdb0 con 0x7f6abc0c83c0 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac2fdd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc0cb370 con 0x7f6abc069d60 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac2fdd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc159f60 con 0x7f6abc0aca80 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac157a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6abc0c83c0 0x7f6abc0cb800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51930/0 (socket says 172.21.15.170:51930) 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.288+0000 7f6ac157a700 1 -- 172.21.15.170:0/2554153141 learned_addr learned my addr 172.21.15.170:0/2554153141 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6abc06aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856796651 0 0) 0x7f6abc06bdb0 con 0x7f6abc0c83c0 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa4003570 con 0x7f6abc0c83c0 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625369026 0 0) 0x7f6aa4003570 con 0x7f6abc0c83c0 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 msgr2=0x7f6abc06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6abc06aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 >> v1:172.21.15.191:6789/0 conn(0x7f6abc069d60 legacy=0x7f6abc0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc06b510 con 0x7f6abc0c83c0 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6ab40017f0 con 0x7f6abc0c83c0 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 >> v1:172.21.15.170:6789/0 conn(0x7f6abc0c83c0 legacy=0x7f6abc0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6aa4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4004810 0x7f6aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.294 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6aa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc159f60 con 0x7f6aa4003a90 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa400bb10 con 0x7f6aa4004810 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa400bcd0 con 0x7f6abc0aca80 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6aa4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac157a700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6aa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ab3fff700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4004810 0x7f6aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 -- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 msgr2=0x7f6aa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0aca80 0x7f6aa40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 -- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4004810 msgr2=0x7f6aa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6aa4004810 0x7f6aa4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.289+0000 7f6ac0d79700 1 -- 172.21.15.170:0/2554153141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4003570 con 0x7f6aa4003a90 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6aa4003e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009bf0 tx=0x7f6ab8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d88258634aabe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8005a00 con 0x7f6aa4003a90 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8005b60 con 0x7f6aa4003a90 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ab37fe700 1 -- 172.21.15.170:0/2554153141 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab800e710 con 0x7f6aa4003a90 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/2554153141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 msgr2=0x7f6aa4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/2554153141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6aa4003e60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8009bf0 tx=0x7f6ab8009120 comp rx=0 tx=0).stop 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac157a700 1 -- 172.21.15.170:0/2554153141 reap_dead start 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/2554153141 shutdown_connections 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/2554153141 >> 172.21.15.170:0/2554153141 conn(0x7f6abc0cd670 msgr2=0x7f6abc0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/2554153141 shutdown_connections 2023-10-08T16:11:57.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.290+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/2554153141 wait complete. 2023-10-08T16:11:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.291+0000 7f6ac2fdd700 1 Processor -- start 2023-10-08T16:11:57.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 -- start start 2023-10-08T16:11:57.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6abc0b2d00 0x7f6abc14b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0cc6d0 0x7f6abc0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0cd080 con 0x7f6aa4003a90 2023-10-08T16:11:57.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc14ba70 con 0x7f6abc0b2d00 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac2fdd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0af6c0 con 0x7f6abc0cc6d0 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac0d79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac0d79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52982/0 (socket says 172.21.15.170:52982) 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ac0d79700 1 -- 172.21.15.170:0/4271563111 learned_addr learned my addr 172.21.15.170:0/4271563111 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.292+0000 7f6ab3fff700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6abc0b2d00 0x7f6abc14b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac157a700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0cc6d0 0x7f6abc0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 -- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0cc6d0 msgr2=0x7f6abc0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0cc6d0 0x7f6abc0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 -- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6abc0b2d00 msgr2=0x7f6abc14b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6abc0b2d00 0x7f6abc14b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 -- 172.21.15.170:0/4271563111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0af9a0 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac0d79700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e4b0 tx=0x7f6ab8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8005a00 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab8006240 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc0afc80 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab8006600 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.293+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc0b06f0 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.294+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc04c3c0 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.294+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6ab8013470 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.295+0000 7f6ab0ff9700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 0x7f6aa80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.295+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6ab8086120 con 0x7f6aa4003a90 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.295+0000 7f6ab3fff700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 0x7f6aa80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.295+0000 7f6ab3fff700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 0x7f6aa80697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6aac007fd0 tx=0x7f6aac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.299+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ab8052ec0 con 0x7f6aa4003a90 2023-10-08T16:11:57.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.481+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6abc06b7f0 con 0x7f6aa4003a90 2023-10-08T16:11:57.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.481+0000 7f6ab0ff9700 1 -- 172.21.15.170:0/4271563111 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ab80564e0 con 0x7f6aa4003a90 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 msgr2=0x7f6aa80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 0x7f6aa80697d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6aac007fd0 tx=0x7f6aac009040 comp rx=0 tx=0).stop 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 msgr2=0x7f6abc0aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800e4b0 tx=0x7f6ab8019000 comp rx=0 tx=0).stop 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 shutdown_connections 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6abc0cc6d0 0x7f6abc0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6aa8067320 0x7f6aa80697d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6aa4003a90 0x7f6abc0aefe0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 --2- 172.21.15.170:0/4271563111 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6abc0b2d00 0x7f6abc14b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 >> 172.21.15.170:0/4271563111 conn(0x7f6abc0cd670 msgr2=0x7f6abc159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:57.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 shutdown_connections 2023-10-08T16:11:57.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.483+0000 7f6ac2fdd700 1 -- 172.21.15.170:0/4271563111 wait complete. 2023-10-08T16:11:57.496 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:11:57.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:11:57.497 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:11:57.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:11:57.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:57.534 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:11:57.589 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:11:57.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:11:57.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:11:57.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:11:57.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:57.632 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:11:57.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:11:57.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:57.690 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T16:11:57.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:11:57.705 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T16:11:57.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:11:57.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:11:57.705 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T16:11:57.745 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T16:11:57.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:11:57.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000793388 --yes-i-really-mean-it 2023-10-08T16:11:57.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.864+0000 7f85e359e700 1 Processor -- start 2023-10-08T16:11:57.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.864+0000 7f85e359e700 1 -- start start 2023-10-08T16:11:57.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f85e40b03c0 0x7f85e40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85e4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e40cfa00 con 0x7f85e40b0790 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e40cb580 con 0x7f85e40b03c0 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40cf160 con 0x7f85e41591e0 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e1d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f85e40b0790 0x7f85e40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51932/0 (socket says 172.21.15.170:51932) 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e1d9b700 1 -- 172.21.15.170:0/3845450817 learned_addr learned my addr 172.21.15.170:0/3845450817 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e259c700 1 --2- 172.21.15.170:0/3845450817 >> v2:172.21.15.191:3300/0 conn(0x7f85e40b03c0 0x7f85e40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078008632 0 0) 0x7f85e40cfa00 con 0x7f85e40b0790 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85cc003570 con 0x7f85e40b0790 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.865+0000 7f85e2d9d700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85e4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 13832769 0 0) 0x7f85cc003570 con 0x7f85e40b0790 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 msgr2=0x7f85e4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85e4159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 >> v2:172.21.15.191:3300/0 conn(0x7f85e40b03c0 msgr2=0x7f85e40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 --2- 172.21.15.170:0/3845450817 >> v2:172.21.15.191:3300/0 conn(0x7f85e40b03c0 0x7f85e40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40cf440 con 0x7f85e40b0790 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f85d8001470 con 0x7f85e40b0790 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 >> v1:172.21.15.170:6789/0 conn(0x7f85e40b0790 legacy=0x7f85e40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85cc003a60 0x7f85cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40cb580 con 0x7f85cc003a60 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e40cf160 con 0x7f85cc004810 2023-10-08T16:11:57.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc00ba60 con 0x7f85e41591e0 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85cc003a60 0x7f85cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e1d9b700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e2d9d700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 -- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 msgr2=0x7f85cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e41591e0 0x7f85cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 -- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc004810 msgr2=0x7f85cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc004810 0x7f85cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 -- 172.21.15.170:0/3845450817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc003570 con 0x7f85cc003a60 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e259c700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85cc003a60 0x7f85cc003e70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009c30 tx=0x7f85d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbb5455c05182ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400e710 con 0x7f85cc003a60 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d400e870 con 0x7f85cc003a60 2023-10-08T16:11:57.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.866+0000 7f85e159a700 1 -- 172.21.15.170:0/3845450817 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d400ec30 con 0x7f85cc003a60 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 -- 172.21.15.170:0/3845450817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85cc003a60 msgr2=0x7f85cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 --2- 172.21.15.170:0/3845450817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85cc003a60 0x7f85cc003e70 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009c30 tx=0x7f85d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e2d9d700 1 -- 172.21.15.170:0/3845450817 reap_dead start 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 -- 172.21.15.170:0/3845450817 shutdown_connections 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 -- 172.21.15.170:0/3845450817 >> 172.21.15.170:0/3845450817 conn(0x7f85e4063230 msgr2=0x7f85e4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 -- 172.21.15.170:0/3845450817 shutdown_connections 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.867+0000 7f85e359e700 1 -- 172.21.15.170:0/3845450817 wait complete. 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.868+0000 7f85e359e700 1 Processor -- start 2023-10-08T16:11:57.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.868+0000 7f85e359e700 1 -- start start 2023-10-08T16:11:57.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc003a60 0x7f85e4157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e415da90 0x7f85e415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415cda0 con 0x7f85e40af610 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e4157eb0 con 0x7f85cc003a60 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e415b880 con 0x7f85e415da90 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52996/0 (socket says 172.21.15.170:52996) 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 -- 172.21.15.170:0/1134160876 learned_addr learned my addr 172.21.15.170:0/1134160876 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e1d9b700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e415da90 0x7f85e415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e259c700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc003a60 0x7f85e4157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 -- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e415da90 msgr2=0x7f85e415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e415da90 0x7f85e415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 -- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc003a60 msgr2=0x7f85e4157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc003a60 0x7f85e4157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 -- 172.21.15.170:0/1134160876 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e415d600 con 0x7f85e40af610 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.869+0000 7f85e2d9d700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009d70 tx=0x7f85dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00e040 con 0x7f85e40af610 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc00f040 con 0x7f85e40af610 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00fd40 con 0x7f85e40af610 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e415b2d0 con 0x7f85e40af610 2023-10-08T16:11:57.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e415cfb0 con 0x7f85e40af610 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.870+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e415d790 con 0x7f85e40af610 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.871+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f85dc015640 con 0x7f85e40af610 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.871+0000 7f85d27fc700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 0x7f85c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.871+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f85dc0854d0 con 0x7f85e40af610 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.871+0000 7f85e259c700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 0x7f85c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.872+0000 7f85e259c700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 0x7f85c40697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f85d40091e0 tx=0x7f85d401a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:11:57.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:57.875+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85dc052050 con 0x7f85e40af610 2023-10-08T16:11:58.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:11:58.085+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000793388", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f85e404c3c0 con 0x7f85c4067320 2023-10-08T16:12:00.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.816+0000 7f85d27fc700 1 -- 172.21.15.170:0/1134160876 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f85e404c3c0 con 0x7f85c4067320 2023-10-08T16:12:00.818 INFO:teuthology.orchestra.run.smithi170.stdout:FS volume 'cephfs' renamed to 'volume_0000000000793388' 2023-10-08T16:12:00.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 msgr2=0x7f85c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 0x7f85c40697d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f85d40091e0 tx=0x7f85d401a000 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 msgr2=0x7f85e4068df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009d70 tx=0x7f85dc00b040 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 shutdown_connections 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f85c4067320 0x7f85c40697d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85e40af610 0x7f85e4068df0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f85cc003a60 0x7f85e4157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 --2- 172.21.15.170:0/1134160876 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f85e415da90 0x7f85e415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 >> 172.21.15.170:0/1134160876 conn(0x7f85e4063230 msgr2=0x7f85e40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:00.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 shutdown_connections 2023-10-08T16:12:00.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.819+0000 7f85e359e700 1 -- 172.21.15.170:0/1134160876 wait complete. 2023-10-08T16:12:00.836 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:12:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.955+0000 7fad00577700 1 Processor -- start 2023-10-08T16:12:00.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.955+0000 7fad00577700 1 -- start start 2023-10-08T16:12:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.956+0000 7fad00577700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7facf8141500 0x7facf80c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.956+0000 7fad00577700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.956+0000 7fad00577700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf815c2a0 con 0x7facf806bcd0 2023-10-08T16:12:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.956+0000 7fad00577700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facf80c9880 con 0x7facf8141500 2023-10-08T16:12:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.956+0000 7fad00577700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf815b690 con 0x7facf806c0a0 2023-10-08T16:12:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7facf8141500 0x7facf80c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfeb14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfeb14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:49102/0 (socket says 172.21.15.170:49102) 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7facf8141500 0x7facf80c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:32944/0 (socket says 172.21.15.170:32944) 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfeb14700 1 -- 172.21.15.170:0/3495461943 learned_addr learned my addr 172.21.15.170:0/3495461943 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfe313700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7facf806bcd0 0x7facf8140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57884/0 (socket says 172.21.15.170:57884) 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfd311700 1 -- 172.21.15.170:0/3495461943 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252242722 0 0) 0x7facf815c2a0 con 0x7facf806bcd0 2023-10-08T16:12:00.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfd311700 1 -- 172.21.15.170:0/3495461943 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face4003570 con 0x7facf806bcd0 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 -- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 msgr2=0x7facf80c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 -- 172.21.15.170:0/3495461943 >> v1:172.21.15.191:6789/0 conn(0x7facf806bcd0 legacy=0x7facf8140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 -- 172.21.15.170:0/3495461943 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf815b970 con 0x7facf8141500 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfeb14700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfdb12700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c8e00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7face80098e0 tx=0x7face80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27f815a480ca1b59 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:00.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfd311700 1 -- 172.21.15.170:0/3495461943 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8011630 con 0x7facf8141500 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfd311700 1 -- 172.21.15.170:0/3495461943 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8011790 con 0x7facf8141500 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.957+0000 7facfd311700 1 -- 172.21.15.170:0/3495461943 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8011b30 con 0x7facf8141500 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 -- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 msgr2=0x7facf80c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c8e00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7face80098e0 tx=0x7face80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 -- 172.21.15.170:0/3495461943 shutdown_connections 2023-10-08T16:12:00.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806c0a0 0x7facf80c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 --2- 172.21.15.170:0/3495461943 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c8e00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 -- 172.21.15.170:0/3495461943 >> 172.21.15.170:0/3495461943 conn(0x7facf80ce600 msgr2=0x7facf80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 -- 172.21.15.170:0/3495461943 shutdown_connections 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.958+0000 7fad00577700 1 -- 172.21.15.170:0/3495461943 wait complete. 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.959+0000 7fad00577700 1 Processor -- start 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.959+0000 7fad00577700 1 -- start start 2023-10-08T16:12:00.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 0x7facf80b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b2d50 con 0x7facf806c0a0 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf813d6f0 con 0x7facf8141500 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7fad00577700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf80b24b0 con 0x7facf806bcd0 2023-10-08T16:12:00.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53000/0 (socket says 172.21.15.170:53000) 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 -- 172.21.15.170:0/2945820607 learned_addr learned my addr 172.21.15.170:0/2945820607 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfeb14700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfe313700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 0x7facf80b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 -- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 msgr2=0x7facf80b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 0x7facf80b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 -- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 msgr2=0x7facf80c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 -- 172.21.15.170:0/2945820607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0009ae0 con 0x7facf806c0a0 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfe313700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 0x7facf80b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfeb14700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:00.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.960+0000 7facfdb12700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7facf8069220 tx=0x7face8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.961+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8011900 con 0x7facf806c0a0 2023-10-08T16:12:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.961+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8011a80 con 0x7facf806c0a0 2023-10-08T16:12:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.961+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8025670 con 0x7facf806c0a0 2023-10-08T16:12:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.961+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face8009420 con 0x7facf806c0a0 2023-10-08T16:12:00.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.961+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf80b2a70 con 0x7facf806c0a0 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.962+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf804c3c0 con 0x7facf806c0a0 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.962+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7face8023070 con 0x7facf806c0a0 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.962+0000 7facee7fc700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.962+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7face8086120 con 0x7facf806c0a0 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.962+0000 7facfe313700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:00.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.963+0000 7facfe313700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 0x7facdc0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7facf4005f00 tx=0x7facf4007c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:00.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:00.967+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7face8052d60 con 0x7facf806c0a0 2023-10-08T16:12:01.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.180+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7facf806c9d0 con 0x7facdc067320 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.181+0000 7facee7fc700 1 -- 172.21.15.170:0/2945820607 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7facf806c9d0 con 0x7facdc067320 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000793388" 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:01.203 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 msgr2=0x7facdc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 0x7facdc0697d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7facf4005f00 tx=0x7facf4007c60 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 msgr2=0x7facf81572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7facf8069220 tx=0x7face8017000 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 shutdown_connections 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7facf8141500 0x7facf80c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7facf806bcd0 0x7facf80b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7facdc067320 0x7facdc0697d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 --2- 172.21.15.170:0/2945820607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7facf806c0a0 0x7facf81572c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 >> 172.21.15.170:0/2945820607 conn(0x7facf80ce600 msgr2=0x7facf80c33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 shutdown_connections 2023-10-08T16:12:01.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.183+0000 7fad00577700 1 -- 172.21.15.170:0/2945820607 wait complete. 2023-10-08T16:12:01.206 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:01.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.319+0000 7f4916334700 1 Processor -- start 2023-10-08T16:12:01.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.319+0000 7f4916334700 1 -- start start 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f4916334700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4910159890 0x7f491015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f4916334700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 0x7f491015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f4916334700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f491015a160 con 0x7f491015db10 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f4916334700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49100d13a0 con 0x7f4910159890 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f4916334700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491015e140 con 0x7f4910159c60 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f490ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4910159890 0x7f491015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f49148d1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f491015db10 0x7f4910069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51934/0 (socket says 172.21.15.170:51934) 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.320+0000 7f49148d1700 1 -- 172.21.15.170:0/3326823135 learned_addr learned my addr 172.21.15.170:0/3326823135 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490ffff700 1 -- 172.21.15.170:0/3326823135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 msgr2=0x7f491015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:01.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490f7fe700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 0x7f491015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490ffff700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 0x7f491015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490effd700 1 -- 172.21.15.170:0/3326823135 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767636610 0 0) 0x7f491015a160 con 0x7f491015db10 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490ffff700 1 -- 172.21.15.170:0/3326823135 >> v1:172.21.15.170:6789/0 conn(0x7f491015db10 legacy=0x7f4910069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490ffff700 1 -- 172.21.15.170:0/3326823135 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f491015a3d0 con 0x7f4910159890 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490f7fe700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 0x7f491015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490ffff700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015abd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f49000098e0 tx=0x7f49000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb656c851201ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490effd700 1 -- 172.21.15.170:0/3326823135 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900009e80 con 0x7f4910159890 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490effd700 1 -- 172.21.15.170:0/3326823135 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490000b860 con 0x7f4910159890 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f490effd700 1 -- 172.21.15.170:0/3326823135 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000bbe0 con 0x7f4910159890 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 -- 172.21.15.170:0/3326823135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 msgr2=0x7f491015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015abd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f49000098e0 tx=0x7f49000091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 -- 172.21.15.170:0/3326823135 shutdown_connections 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015abd0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 --2- 172.21.15.170:0/3326823135 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4910159c60 0x7f491015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 -- 172.21.15.170:0/3326823135 >> 172.21.15.170:0/3326823135 conn(0x7f4910068e20 msgr2=0x7f49100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 -- 172.21.15.170:0/3326823135 shutdown_connections 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.321+0000 7f4916334700 1 -- 172.21.15.170:0/3326823135 wait complete. 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.322+0000 7f4916334700 1 Processor -- start 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 -- start start 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4910159c60 0x7f49100b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f491015db10 0x7f49100ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910152390 con 0x7f4910159890 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910152670 con 0x7f4910159c60 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f4916334700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4910152950 con 0x7f491015db10 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f490ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.323+0000 7f490ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53018/0 (socket says 172.21.15.170:53018) 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 -- 172.21.15.170:0/1686084416 learned_addr learned my addr 172.21.15.170:0/1686084416 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490f7fe700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4910159c60 0x7f49100b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f49148d1700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f491015db10 0x7f49100ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 -- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f491015db10 msgr2=0x7f49100ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f491015db10 0x7f49100ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 -- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4910159c60 msgr2=0x7f49100b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4910159c60 0x7f49100b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 -- 172.21.15.170:0/1686084416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900009420 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f490ffff700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f49100899c0 tx=0x7f4900017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490000bbe0 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900011420 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4910152c30 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49000117e0 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.324+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49100b6410 con 0x7f4910159890 2023-10-08T16:12:01.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.325+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491004c3c0 con 0x7f4910159890 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.325+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4900011bd0 con 0x7f4910159890 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.326+0000 7f48f7fff700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 0x7f48fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.326+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f4900086850 con 0x7f4910159890 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.326+0000 7f490f7fe700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 0x7f48fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.326+0000 7f490f7fe700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 0x7f48fc0697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49100caa10 tx=0x7f4904006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.330+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4900053490 con 0x7f4910159890 2023-10-08T16:12:01.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.562+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f491006a5b0 con 0x7f4910159890 2023-10-08T16:12:01.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.562+0000 7f48f7fff700 1 -- 172.21.15.170:0/1686084416 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4624 (secure 0 0 0) 0x7f4900056ab0 con 0x7f4910159890 2023-10-08T16:12:01.597 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:01.597 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:58.806252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000793388","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 msgr2=0x7f48fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 0x7f48fc0697d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49100caa10 tx=0x7f4904006f90 comp rx=0 tx=0).stop 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 msgr2=0x7f491015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f49100899c0 tx=0x7f4900017000 comp rx=0 tx=0).stop 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 shutdown_connections 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f491015db10 0x7f49100ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f48fc067320 0x7f48fc0697d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4910159890 0x7f491015c870 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 --2- 172.21.15.170:0/1686084416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4910159c60 0x7f49100b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.564+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 >> 172.21.15.170:0/1686084416 conn(0x7f4910068e20 msgr2=0x7f491014d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.565+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 shutdown_connections 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.565+0000 7f4916334700 1 -- 172.21.15.170:0/1686084416 wait complete. 2023-10-08T16:12:01.600 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 92 2023-10-08T16:12:01.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:01.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.713+0000 7f2242318700 1 Processor -- start 2023-10-08T16:12:01.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.713+0000 7f2242318700 1 -- start start 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f2242318700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f223c069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f2242318700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f223c06a2b0 0x7f223c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f2242318700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223c0c96a0 con 0x7f223c1599e0 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f2242318700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223c0cb9e0 con 0x7f223c06a2b0 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f2242318700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cab60 con 0x7f223c159db0 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f223bfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f223c1599e0 0x7f223c069080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51948/0 (socket says 172.21.15.170:51948) 2023-10-08T16:12:01.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.714+0000 7f223bfff700 1 -- 172.21.15.170:0/3015009282 learned_addr learned my addr 172.21.15.170:0/3015009282 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f22408b5700 1 --2- 172.21.15.170:0/3015009282 >> v2:172.21.15.191:3300/0 conn(0x7f223c06a2b0 0x7f223c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223b7fe700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f223c069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924852524 0 0) 0x7f223c0c96a0 con 0x7f223c1599e0 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2224003570 con 0x7f223c1599e0 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818595009 0 0) 0x7f2224003570 con 0x7f223c1599e0 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 msgr2=0x7f223c069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:01.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f223c069ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 >> v2:172.21.15.191:3300/0 conn(0x7f223c06a2b0 msgr2=0x7f223c06a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 --2- 172.21.15.170:0/3015009282 >> v2:172.21.15.191:3300/0 conn(0x7f223c06a2b0 0x7f223c06a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0cb420 con 0x7f223c1599e0 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f222c001470 con 0x7f223c1599e0 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 >> v1:172.21.15.170:6789/0 conn(0x7f223c1599e0 legacy=0x7f223c069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224003a60 0x7f2224003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f22240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cb9e0 con 0x7f2224003a60 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0cab60 con 0x7f2224004810 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222400ba60 con 0x7f223c159db0 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224003a60 0x7f2224003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223b7fe700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f22408b5700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f22240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 -- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 msgr2=0x7f22240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f22240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 -- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2224004810 msgr2=0x7f2224004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:01.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223bfff700 1 -- 172.21.15.170:0/3015009282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2224003570 con 0x7f2224003a60 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f223b7fe700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2224004810 0x7f2224004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.715+0000 7f22408b5700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f223c159db0 0x7f22240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f223bfff700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224003a60 0x7f2224003e70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c3f0 tx=0x7f222c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e3b9d4d3ef2ec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cc80 con 0x7f2224003a60 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c00ce20 con 0x7f2224003a60 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f223affd700 1 -- 172.21.15.170:0/3015009282 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c005b70 con 0x7f2224003a60 2023-10-08T16:12:01.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 -- 172.21.15.170:0/3015009282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224003a60 msgr2=0x7f2224003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 --2- 172.21.15.170:0/3015009282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2224003a60 0x7f2224003e70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c3f0 tx=0x7f222c001e90 comp rx=0 tx=0).stop 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f22408b5700 1 -- 172.21.15.170:0/3015009282 reap_dead start 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 -- 172.21.15.170:0/3015009282 shutdown_connections 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 -- 172.21.15.170:0/3015009282 >> 172.21.15.170:0/3015009282 conn(0x7f223c0c9860 msgr2=0x7f223c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 -- 172.21.15.170:0/3015009282 shutdown_connections 2023-10-08T16:12:01.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.716+0000 7f2242318700 1 -- 172.21.15.170:0/3015009282 wait complete. 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.717+0000 7f2242318700 1 Processor -- start 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 -- start start 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2224003a60 0x7f223c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f223c0c2450 0x7f223c0c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c155fa0 con 0x7f223c156d90 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0c70e0 con 0x7f223c0c2450 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f2242318700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0c2de0 con 0x7f2224003a60 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f223b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f223b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53046/0 (socket says 172.21.15.170:53046) 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.718+0000 7f223b7fe700 1 -- 172.21.15.170:0/1411731152 learned_addr learned my addr 172.21.15.170:0/1411731152 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223bfff700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2224003a60 0x7f223c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f22408b5700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f223c0c2450 0x7f223c0c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 -- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2224003a60 msgr2=0x7f223c1431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2224003a60 0x7f223c1431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 -- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f223c0c2450 msgr2=0x7f223c0c28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f223c0c2450 0x7f223c0c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 -- 172.21.15.170:0/1411731152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223c0c64d0 con 0x7f223c156d90 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f223b7fe700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f22300118e0 tx=0x7f2230008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.729 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230011e80 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000f860 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c0c67b0 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000fc00 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.719+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c0c6d70 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.720+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c0cbd60 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.720+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2230015460 con 0x7f223c156d90 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.721+0000 7f2223fff700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 0x7f22280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:01.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.721+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f22300854b0 con 0x7f223c156d90 2023-10-08T16:12:01.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.721+0000 7f223bfff700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 0x7f22280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:01.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.721+0000 7f223bfff700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 0x7f22280697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c3f0 tx=0x7f222c015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:01.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.725+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22300520f0 con 0x7f223c156d90 2023-10-08T16:12:01.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.912+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f223c04c3c0 con 0x7f223c156d90 2023-10-08T16:12:01.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.913+0000 7f2223fff700 1 -- 172.21.15.170:0/1411731152 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f2230055710 con 0x7f223c156d90 2023-10-08T16:12:01.914 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:01.915 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:00.813675+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000793388.meta","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000793388"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000793388.data","create_time":"2023-10-08T16:11:46.115982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000793388"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:01.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 msgr2=0x7f22280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 0x7f22280697d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f222c00c3f0 tx=0x7f222c015000 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 msgr2=0x7f223c0c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f22300118e0 tx=0x7f2230008040 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 shutdown_connections 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f223c0c2450 0x7f223c0c28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2224003a60 0x7f223c1431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2228067320 0x7f22280697d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 --2- 172.21.15.170:0/1411731152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f223c156d90 0x7f223c0c5b80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 >> 172.21.15.170:0/1411731152 conn(0x7f223c0c9860 msgr2=0x7f223c1578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 shutdown_connections 2023-10-08T16:12:01.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:01.915+0000 7f2242318700 1 -- 172.21.15.170:0/1411731152 wait complete. 2023-10-08T16:12:01.928 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:12:01.929 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:01.971 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:02.016 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:12:02.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:02.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.129+0000 7f261359e700 1 Processor -- start 2023-10-08T16:12:02.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.129+0000 7f261359e700 1 -- start start 2023-10-08T16:12:02.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26140b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261359e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f261415d230 0x7f261415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261415dbd0 con 0x7f26141597f0 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261359e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26140b2980 con 0x7f261415d230 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140b11d0 con 0x7f2614159420 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f261415d230 0x7f261415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2611d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f26141597f0 0x7f26140b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51956/0 (socket says 172.21.15.170:51956) 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2611d9b700 1 -- 172.21.15.170:0/1381698368 learned_addr learned my addr 172.21.15.170:0/1381698368 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261259c700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26140b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 -- 172.21.15.170:0/1381698368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 msgr2=0x7f26140b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:02.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26140b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 -- 172.21.15.170:0/1381698368 >> v1:172.21.15.170:6789/0 conn(0x7f26141597f0 legacy=0x7f26140b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f261159a700 1 -- 172.21.15.170:0/1381698368 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 760090331 0 0) 0x7f25fc001110 con 0x7f26141597f0 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 -- 172.21.15.170:0/1381698368 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140b14b0 con 0x7f261415d230 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.130+0000 7f2612d9d700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261415d230 0x7f261415d690 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2604009e60 tx=0x7f2604009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687d1394352e1834 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261159a700 1 -- 172.21.15.170:0/1381698368 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260400e070 con 0x7f261415d230 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261159a700 1 -- 172.21.15.170:0/1381698368 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260400bb60 con 0x7f261415d230 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261159a700 1 -- 172.21.15.170:0/1381698368 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2604012430 con 0x7f261415d230 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 -- 172.21.15.170:0/1381698368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261415d230 msgr2=0x7f261415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261415d230 0x7f261415d690 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2604009e60 tx=0x7f2604009780 comp rx=0 tx=0).stop 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 -- 172.21.15.170:0/1381698368 shutdown_connections 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26140b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 --2- 172.21.15.170:0/1381698368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f261415d230 0x7f261415d690 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 -- 172.21.15.170:0/1381698368 >> 172.21.15.170:0/1381698368 conn(0x7f26140d0660 msgr2=0x7f26140d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:02.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 -- 172.21.15.170:0/1381698368 shutdown_connections 2023-10-08T16:12:02.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.131+0000 7f261359e700 1 -- 172.21.15.170:0/1381698368 wait complete. 2023-10-08T16:12:02.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.132+0000 7f261359e700 1 Processor -- start 2023-10-08T16:12:02.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.132+0000 7f261359e700 1 -- start start 2023-10-08T16:12:02.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26141697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261415d230 0x7f261415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614153cb0 con 0x7f26141597f0 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261415a3e0 con 0x7f261415d230 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261415c1f0 con 0x7f2614159420 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53062/0 (socket says 172.21.15.170:53062) 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 -- 172.21.15.170:0/1842457320 learned_addr learned my addr 172.21.15.170:0/1842457320 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f261259c700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26141697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2612d9d700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261415d230 0x7f261415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 -- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 msgr2=0x7f26141697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26141697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 -- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261415d230 msgr2=0x7f261415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261415d230 0x7f261415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.133+0000 7f2611d9b700 1 -- 172.21.15.170:0/1842457320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604009a20 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f2611d9b700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c810 tx=0x7f25fc007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc005b30 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc005c90 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261415bf10 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc007340 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.134+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261415a070 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.135+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261415a550 con 0x7f26141597f0 2023-10-08T16:12:02.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.135+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25fc0076f0 con 0x7f26141597f0 2023-10-08T16:12:02.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.135+0000 7f260a7fc700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 0x7f25f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.136+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f25fc0845c0 con 0x7f26141597f0 2023-10-08T16:12:02.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.136+0000 7f261259c700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 0x7f25f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.136+0000 7f261259c700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 0x7f25f40697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f260c009d40 tx=0x7f260c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.140+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25fc051200 con 0x7f26141597f0 2023-10-08T16:12:02.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.367+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f261415dbd0 con 0x7f26141597f0 2023-10-08T16:12:02.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.367+0000 7f260a7fc700 1 -- 172.21.15.170:0/1842457320 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4624 (secure 0 0 0) 0x7f261415dbd0 con 0x7f26141597f0 2023-10-08T16:12:02.369 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:02.369 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:58.806252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000793388","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:12:02.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 msgr2=0x7f25f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 0x7f25f40697d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f260c009d40 tx=0x7f260c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 msgr2=0x7f261415a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00c810 tx=0x7f25fc007c50 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 shutdown_connections 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f261415d230 0x7f261415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2614159420 0x7f26141697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25f4067320 0x7f25f40697d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 --2- 172.21.15.170:0/1842457320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26141597f0 0x7f261415a870 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 >> 172.21.15.170:0/1842457320 conn(0x7f26140d0660 msgr2=0x7f26140d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 shutdown_connections 2023-10-08T16:12:02.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.370+0000 7f261359e700 1 -- 172.21.15.170:0/1842457320 wait complete. 2023-10-08T16:12:02.373 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 92 2023-10-08T16:12:02.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.496+0000 7fab5e411700 1 Processor -- start 2023-10-08T16:12:02.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.496+0000 7fab5e411700 1 -- start start 2023-10-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5e411700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5e411700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fab58069560 0x7fab580699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5e411700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5815a160 con 0x7fab58159c60 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5e411700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab58069f20 con 0x7fab58069560 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5e411700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580d13a0 con 0x7fab58159890 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5c9ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fab58069560 0x7fab580699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab57fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab577fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fab58159c60 0x7fab5815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57892/0 (socket says 172.21.15.170:57892) 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab5c9ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fab58069560 0x7fab580699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:33002/0 (socket says 172.21.15.170:33002) 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab57fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:49202/0 (socket says 172.21.15.170:49202) 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.497+0000 7fab577fe700 1 -- 172.21.15.170:0/3044269712 learned_addr learned my addr 172.21.15.170:0/3044269712 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5c9ae700 1 -- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 msgr2=0x7fab5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:02.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5c9ae700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5c9ae700 1 -- 172.21.15.170:0/3044269712 >> v1:172.21.15.191:6789/0 conn(0x7fab58159c60 legacy=0x7fab5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab56ffd700 1 -- 172.21.15.170:0/3044269712 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513948935 0 0) 0x7fab5815a160 con 0x7fab58159c60 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5c9ae700 1 -- 172.21.15.170:0/3044269712 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5815e140 con 0x7fab58069560 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab57fff700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5c9ae700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58069560 0x7fab580699e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e60 tx=0x7fab50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74827dcc7419c25e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab56ffd700 1 -- 172.21.15.170:0/3044269712 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e070 con 0x7fab58069560 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab56ffd700 1 -- 172.21.15.170:0/3044269712 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5000bb60 con 0x7fab58069560 2023-10-08T16:12:02.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab56ffd700 1 -- 172.21.15.170:0/3044269712 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50012430 con 0x7fab58069560 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5e411700 1 -- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58069560 msgr2=0x7fab580699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.498+0000 7fab5e411700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58069560 0x7fab580699e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e60 tx=0x7fab50009780 comp rx=0 tx=0).stop 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 -- 172.21.15.170:0/3044269712 shutdown_connections 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58069560 0x7fab580699e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 --2- 172.21.15.170:0/3044269712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58159890 0x7fab5815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 -- 172.21.15.170:0/3044269712 >> 172.21.15.170:0/3044269712 conn(0x7fab58068e20 msgr2=0x7fab580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 -- 172.21.15.170:0/3044269712 shutdown_connections 2023-10-08T16:12:02.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.499+0000 7fab5e411700 1 -- 172.21.15.170:0/3044269712 wait complete. 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.500+0000 7fab5e411700 1 Processor -- start 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.500+0000 7fab5e411700 1 -- start start 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58069560 0x7fab58151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58159890 0x7fab580cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5815b4d0 con 0x7fab58159c60 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5815c330 con 0x7fab58159890 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580cd380 con 0x7fab58069560 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53068/0 (socket says 172.21.15.170:53068) 2023-10-08T16:12:02.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 -- 172.21.15.170:0/577052640 learned_addr learned my addr 172.21.15.170:0/577052640 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab577fe700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58159890 0x7fab580cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab57fff700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58069560 0x7fab58151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 -- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58069560 msgr2=0x7fab58151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58069560 0x7fab58151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 -- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58159890 msgr2=0x7fab580cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58159890 0x7fab580cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 -- 172.21.15.170:0/577052640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009420 con 0x7fab58159c60 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5c9ae700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b6c0 tx=0x7fab50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50012e90 con 0x7fab58159c60 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab50027470 con 0x7fab58159c60 2023-10-08T16:12:02.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab50009a20 con 0x7fab58159c60 2023-10-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50027830 con 0x7fab58159c60 2023-10-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.501+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5815ba00 con 0x7fab58159c60 2023-10-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.502+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5815c4a0 con 0x7fab58159c60 2023-10-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.503+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fab50024070 con 0x7fab58159c60 2023-10-08T16:12:02.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.503+0000 7fab3ffff700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 0x7fab440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.503+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fab500873f0 con 0x7fab58159c60 2023-10-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.503+0000 7fab57fff700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 0x7fab440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.504+0000 7fab57fff700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 0x7fab440697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fab48009f40 tx=0x7fab480091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.508+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab50054030 con 0x7fab58159c60 2023-10-08T16:12:02.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.692+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab5804c3c0 con 0x7fab58159c60 2023-10-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.693+0000 7fab3ffff700 1 -- 172.21.15.170:0/577052640 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15490 (secure 0 0 0) 0x7fab50057650 con 0x7fab58159c60 2023-10-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:02.694 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:00.813675+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000793388.meta","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000793388"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000793388.data","create_time":"2023-10-08T16:11:46.115982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000793388"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:02.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 msgr2=0x7fab440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 0x7fab440697d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fab48009f40 tx=0x7fab480091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 msgr2=0x7fab5815d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fab5000b6c0 tx=0x7fab50019000 comp rx=0 tx=0).stop 2023-10-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 shutdown_connections 2023-10-08T16:12:02.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fab44067320 0x7fab440697d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab58159c60 0x7fab5815d330 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fab58069560 0x7fab58151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 --2- 172.21.15.170:0/577052640 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fab58159890 0x7fab580cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 >> 172.21.15.170:0/577052640 conn(0x7fab58068e20 msgr2=0x7fab580d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 shutdown_connections 2023-10-08T16:12:02.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.695+0000 7fab5e411700 1 -- 172.21.15.170:0/577052640 wait complete. 2023-10-08T16:12:02.708 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 24, 'pool_name': 'cephfs.volume_0000000000793388.meta', 'create_time': '2023-10-08T16:11:45.493023+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000793388'}}}, {'pool': 25, 'pool_name': 'cephfs.volume_0000000000793388.data', 'create_time': '2023-10-08T16:11:46.115982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '140', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000793388'}}}] 2023-10-08T16:12:02.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.826+0000 7fcde395e700 1 Processor -- start 2023-10-08T16:12:02.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.826+0000 7fcde395e700 1 -- start start 2023-10-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde395e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcddc06c0a0 0x7fcddc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde395e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 0x7fcddc15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde395e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc15d7e0 con 0x7fcddc06bcd0 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde395e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc15dac0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde395e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc15aa30 con 0x7fcddc15df50 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.827+0000 7fcde0ef9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcddc06c0a0 0x7fcddc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcddc06c0a0 0x7fcddc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:33006/0 (socket says 172.21.15.170:33006) 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde1efb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 0x7fcddc15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 -- 172.21.15.170:0/3136385338 learned_addr learned my addr 172.21.15.170:0/3136385338 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde16fa700 1 --1- 172.21.15.170:0/3136385338 >> v1:172.21.15.191:6789/0 conn(0x7fcddc06bcd0 0x7fcddc15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57908/0 (socket says 172.21.15.170:57908) 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 -- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 msgr2=0x7fcddc15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:02.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 0x7fcddc15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 -- 172.21.15.170:0/3136385338 >> v1:172.21.15.191:6789/0 conn(0x7fcddc06bcd0 legacy=0x7fcddc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 -- 172.21.15.170:0/3136385338 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc15ad10 con 0x7fcddc06c0a0 2023-10-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde1efb700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 0x7fcddc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:02.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.828+0000 7fcde0ef9700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc06c0a0 0x7fcddc0caa90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099b0 tx=0x7fcdcc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3985188bbc31ff4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcdd3fff700 1 -- 172.21.15.170:0/3136385338 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc009ea0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcdd3fff700 1 -- 172.21.15.170:0/3136385338 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00b860 con 0x7fcddc06c0a0 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcdd3fff700 1 -- 172.21.15.170:0/3136385338 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bc20 con 0x7fcddc06c0a0 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 -- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc06c0a0 msgr2=0x7fcddc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc06c0a0 0x7fcddc0caa90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0099b0 tx=0x7fcdcc009250 comp rx=0 tx=0).stop 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 -- 172.21.15.170:0/3136385338 shutdown_connections 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc15df50 0x7fcddc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 --2- 172.21.15.170:0/3136385338 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc06c0a0 0x7fcddc0caa90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 -- 172.21.15.170:0/3136385338 >> 172.21.15.170:0/3136385338 conn(0x7fcddc0ce600 msgr2=0x7fcddc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.836+0000 7fcde395e700 1 -- 172.21.15.170:0/3136385338 shutdown_connections 2023-10-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.837+0000 7fcde395e700 1 -- 172.21.15.170:0/3136385338 wait complete. 2023-10-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 Processor -- start 2023-10-08T16:12:02.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 -- start start 2023-10-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc06bcd0 0x7fcddc148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc15df50 0x7fcddc155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1489b0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc158c80 con 0x7fcddc15df50 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.838+0000 7fcde395e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc16aca0 con 0x7fcddc06bcd0 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53078/0 (socket says 172.21.15.170:53078) 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 -- 172.21.15.170:0/1246511672 learned_addr learned my addr 172.21.15.170:0/1246511672 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde1efb700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc15df50 0x7fcddc155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde16fa700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc06bcd0 0x7fcddc148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 -- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc06bcd0 msgr2=0x7fcddc148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc06bcd0 0x7fcddc148470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 -- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc15df50 msgr2=0x7fcddc155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc15df50 0x7fcddc155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 -- 172.21.15.170:0/1246511672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8009420 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde0ef9700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00b3c0 tx=0x7fcdcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc01f070 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00bbe0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdcc0094f0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc011650 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.839+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc16a9c0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.840+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc04c3c0 con 0x7fcddc06c0a0 2023-10-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.841+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcdcc011a40 con 0x7fcddc06c0a0 2023-10-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.841+0000 7fcdd17fa700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 0x7fcdc00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.841+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fcdcc085f60 con 0x7fcddc06c0a0 2023-10-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.841+0000 7fcde16fa700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 0x7fcdc00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:02.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.841+0000 7fcde16fa700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 0x7fcdc00697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0ca9f0 tx=0x7fcdd4001a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:02.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:02.845+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdcc052ce0 con 0x7fcddc06c0a0 2023-10-08T16:12:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.030+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcddc15aff0 con 0x7fcddc06c0a0 2023-10-08T16:12:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.030+0000 7fcdd17fa700 1 -- 172.21.15.170:0/1246511672 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v142) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcdcc0280a0 con 0x7fcddc06c0a0 2023-10-08T16:12:03.032 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 msgr2=0x7fcdc00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 0x7fcdc00697d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0ca9f0 tx=0x7fcdd4001a30 comp rx=0 tx=0).stop 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 msgr2=0x7fcddc169ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00b3c0 tx=0x7fcdcc017000 comp rx=0 tx=0).stop 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 shutdown_connections 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcddc15df50 0x7fcddc155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcddc06bcd0 0x7fcddc148470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcdc0067320 0x7fcdc00697d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.032+0000 7fcde395e700 1 --2- 172.21.15.170:0/1246511672 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcddc06c0a0 0x7fcddc169ca0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.033+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 >> 172.21.15.170:0/1246511672 conn(0x7fcddc0ce600 msgr2=0x7fcddc15cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.033+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 shutdown_connections 2023-10-08T16:12:03.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.033+0000 7fcde395e700 1 -- 172.21.15.170:0/1246511672 wait complete. 2023-10-08T16:12:03.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000793388.meta pg_num 2023-10-08T16:12:03.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.162+0000 7f0363061700 1 Processor -- start 2023-10-08T16:12:03.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.162+0000 7f0363061700 1 -- start start 2023-10-08T16:12:03.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0363061700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0363061700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f035c0b16d0 0x7f035c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0363061700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0b2080 con 0x7f035c15d240 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0363061700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035c0cfb00 con 0x7f035c0b16d0 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0363061700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0b5c40 con 0x7f035c0b4e70 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f036185e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f035c0b16d0 0x7f035c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f036205f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f036185e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f035c0b16d0 0x7f035c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:33030/0 (socket says 172.21.15.170:33030) 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f036185e700 1 -- 172.21.15.170:0/1141529869 learned_addr learned my addr 172.21.15.170:0/1141529869 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.163+0000 7f0362860700 1 --1- 172.21.15.170:0/1141529869 >> v1:172.21.15.191:6789/0 conn(0x7f035c15d240 0x7f035c15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57924/0 (socket says 172.21.15.170:57924) 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036185e700 1 -- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 msgr2=0x7f035c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:03.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036185e700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036185e700 1 -- 172.21.15.170:0/1141529869 >> v1:172.21.15.191:6789/0 conn(0x7f035c15d240 legacy=0x7f035c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036185e700 1 -- 172.21.15.170:0/1141529869 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0b5f20 con 0x7f035c0b16d0 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036205f700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036105d700 1 -- 172.21.15.170:0/1141529869 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262267259 0 0) 0x7f035c0b2080 con 0x7f035c15d240 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036185e700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b16d0 0x7f035c0b1b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f034c0099b0 tx=0x7f034c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51a06992ae8fa2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036105d700 1 -- 172.21.15.170:0/1141529869 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c009ea0 con 0x7f035c0b16d0 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036105d700 1 -- 172.21.15.170:0/1141529869 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00b860 con 0x7f035c0b16d0 2023-10-08T16:12:03.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.164+0000 7f036105d700 1 -- 172.21.15.170:0/1141529869 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00bc00 con 0x7f035c0b16d0 2023-10-08T16:12:03.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 -- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b16d0 msgr2=0x7f035c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b16d0 0x7f035c0b1b40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f034c0099b0 tx=0x7f034c009250 comp rx=0 tx=0).stop 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 -- 172.21.15.170:0/1141529869 shutdown_connections 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 --2- 172.21.15.170:0/1141529869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c0b16d0 0x7f035c0b1b40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 -- 172.21.15.170:0/1141529869 >> 172.21.15.170:0/1141529869 conn(0x7f035c0d10c0 msgr2=0x7f035c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 -- 172.21.15.170:0/1141529869 shutdown_connections 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.165+0000 7f0363061700 1 -- 172.21.15.170:0/1141529869 wait complete. 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 Processor -- start 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 -- start start 2023-10-08T16:12:03.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c15d240 0x7f035c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15b550 con 0x7f035c0b16d0 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15b830 con 0x7f035c15d240 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.166+0000 7f0363061700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c15bb10 con 0x7f035c0b4e70 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53080/0 (socket says 172.21.15.170:53080) 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 -- 172.21.15.170:0/3012775802 learned_addr learned my addr 172.21.15.170:0/3012775802 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036185e700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f0362860700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c15d240 0x7f035c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 -- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 msgr2=0x7f035c0c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 -- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c15d240 msgr2=0x7f035c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c15d240 0x7f035c15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 -- 172.21.15.170:0/3012775802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358009420 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f036205f700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f035800ec80 tx=0x7f03580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035800c920 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035800ca80 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c0094f0 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035800ce00 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.167+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c15c0d0 con 0x7f035c0b16d0 2023-10-08T16:12:03.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.168+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c15c440 con 0x7f035c0b16d0 2023-10-08T16:12:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.169+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0358013680 con 0x7f035c0b16d0 2023-10-08T16:12:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.169+0000 7f03527fc700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 0x7f03400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.169+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f03580850a0 con 0x7f035c0b16d0 2023-10-08T16:12:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.169+0000 7f036185e700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 0x7f03400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.169+0000 7f036185e700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 0x7f03400697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f034c0099b0 tx=0x7f034c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.173+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0358051ce0 con 0x7f035c0b16d0 2023-10-08T16:12:03.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.353+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000793388.meta", "var": "pg_num"} v 0) v1 -- 0x7f035c0c5ea0 con 0x7f035c0b16d0 2023-10-08T16:12:03.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.354+0000 7f03527fc700 1 -- 172.21.15.170:0/3012775802 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000793388.meta", "var": "pg_num"}]=0 v142) v1 ==== 124+0+11 (secure 0 0 0) 0x7f0358055300 con 0x7f035c0b16d0 2023-10-08T16:12:03.355 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:12:03.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 msgr2=0x7f03400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 0x7f03400697d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f034c0099b0 tx=0x7f034c017000 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 msgr2=0x7f035c157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f035800ec80 tx=0x7f03580091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 shutdown_connections 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f035c15d240 0x7f035c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0340067320 0x7f03400697d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f035c0b16d0 0x7f035c157270 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 --2- 172.21.15.170:0/3012775802 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f035c0b4e70 0x7f035c0c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 >> 172.21.15.170:0/3012775802 conn(0x7f035c0d10c0 msgr2=0x7f035c069b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 shutdown_connections 2023-10-08T16:12:03.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.356+0000 7f0363061700 1 -- 172.21.15.170:0/3012775802 wait complete. 2023-10-08T16:12:03.369 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000793388.data pg_num 2023-10-08T16:12:03.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.485+0000 7f565b322700 1 Processor -- start 2023-10-08T16:12:03.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.485+0000 7f565b322700 1 -- start start 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565b322700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 0x7f56540ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565b322700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565b322700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56540cfb00 con 0x7f56540b16d0 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565b322700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56540b5c40 con 0x7f56540b4e70 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565b322700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56540b5f20 con 0x7f56540b1b40 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565a320700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 0x7f56540ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565ab21700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565a320700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 0x7f56540ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:33050/0 (socket says 172.21.15.170:33050) 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565ab21700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:49242/0 (socket says 172.21.15.170:49242) 2023-10-08T16:12:03.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f565a320700 1 -- 172.21.15.170:0/1885149126 learned_addr learned my addr 172.21.15.170:0/1885149126 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.486+0000 7f5659b1f700 1 --1- 172.21.15.170:0/1885149126 >> v1:172.21.15.191:6789/0 conn(0x7f56540b16d0 0x7f565415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57940/0 (socket says 172.21.15.170:57940) 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565ab21700 1 -- 172.21.15.170:0/1885149126 >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 msgr2=0x7f56540ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565ab21700 1 --2- 172.21.15.170:0/1885149126 >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 0x7f56540ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565931e700 1 -- 172.21.15.170:0/1885149126 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415994354 0 0) 0x7f56540cfb00 con 0x7f56540b16d0 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565ab21700 1 -- 172.21.15.170:0/1885149126 >> v1:172.21.15.191:6789/0 conn(0x7f56540b16d0 legacy=0x7f565415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565ab21700 1 -- 172.21.15.170:0/1885149126 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540b6200 con 0x7f56540b1b40 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565ab21700 1 --2- 172.21.15.170:0/1885149126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f564c009e60 tx=0x7f564c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49cb987a83e21534 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565931e700 1 -- 172.21.15.170:0/1885149126 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c00e070 con 0x7f56540b1b40 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565931e700 1 -- 172.21.15.170:0/1885149126 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564c00bb60 con 0x7f56540b1b40 2023-10-08T16:12:03.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565931e700 1 -- 172.21.15.170:0/1885149126 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f564c012430 con 0x7f56540b1b40 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 -- 172.21.15.170:0/1885149126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 msgr2=0x7f565415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 --2- 172.21.15.170:0/1885149126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f564c009e60 tx=0x7f564c009780 comp rx=0 tx=0).stop 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 -- 172.21.15.170:0/1885149126 shutdown_connections 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 --2- 172.21.15.170:0/1885149126 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b1b40 0x7f565415d950 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 --2- 172.21.15.170:0/1885149126 >> v2:172.21.15.191:3300/0 conn(0x7f56540b4e70 0x7f56540ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 -- 172.21.15.170:0/1885149126 >> 172.21.15.170:0/1885149126 conn(0x7f56540d10c0 msgr2=0x7f56540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.487+0000 7f565b322700 1 -- 172.21.15.170:0/1885149126 shutdown_connections 2023-10-08T16:12:03.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.488+0000 7f565b322700 1 -- 172.21.15.170:0/1885149126 wait complete. 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.488+0000 7f565b322700 1 Processor -- start 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 -- start start 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 0x7f5654169850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b4e70 0x7f565415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56540cc760 con 0x7f56540b1b40 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5654169d90 con 0x7f56540b16d0 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565b322700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56540cc1a0 con 0x7f56540b4e70 2023-10-08T16:12:03.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f5659b1f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f5659b1f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53090/0 (socket says 172.21.15.170:53090) 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f5659b1f700 1 -- 172.21.15.170:0/865082563 learned_addr learned my addr 172.21.15.170:0/865082563 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565a320700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 0x7f5654169850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.489+0000 7f565ab21700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b4e70 0x7f565415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 -- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b4e70 msgr2=0x7f565415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b4e70 0x7f565415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 -- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 msgr2=0x7f5654169850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 0x7f5654169850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 -- 172.21.15.170:0/865082563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c009a20 con 0x7f56540b1b40 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f565a320700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 0x7f5654169850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:03.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f5659b1f700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f564400c5b0 tx=0x7f5644001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5644005a70 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5644005bd0 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56540cc480 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56440070d0 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.490+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56540c7790 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.491+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56540c7310 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.491+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5644007480 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.492+0000 7f564a7fc700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 0x7f56380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.492+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f5644084110 con 0x7f56540b1b40 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.492+0000 7f565a320700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 0x7f56380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.492+0000 7f565a320700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 0x7f56380697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5650009fd0 tx=0x7f5650009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.496+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56540c7310 con 0x7f56540b1b40 2023-10-08T16:12:03.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.674+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000793388.data", "var": "pg_num"} v 0) v1 -- 0x7f56540c7310 con 0x7f56540b1b40 2023-10-08T16:12:03.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.675+0000 7f564a7fc700 1 -- 172.21.15.170:0/865082563 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000793388.data", "var": "pg_num"}]=0 v142) v1 ==== 124+0+11 (secure 0 0 0) 0x7f56540c7310 con 0x7f56540b1b40 2023-10-08T16:12:03.676 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:12:03.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 msgr2=0x7f56380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 0x7f56380697d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5650009fd0 tx=0x7f5650009250 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 msgr2=0x7f565414af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f564400c5b0 tx=0x7f5644001040 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 shutdown_connections 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f56540b4e70 0x7f565415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f56540b16d0 0x7f5654169850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5638067320 0x7f56380697d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 --2- 172.21.15.170:0/865082563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56540b1b40 0x7f565414af90 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 >> 172.21.15.170:0/865082563 conn(0x7f56540d10c0 msgr2=0x7f56540ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.679 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 shutdown_connections 2023-10-08T16:12:03.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.677+0000 7f565b322700 1 -- 172.21.15.170:0/865082563 wait complete. 2023-10-08T16:12:03.690 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:03.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 Processor -- start 2023-10-08T16:12:03.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 -- start start 2023-10-08T16:12:03.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5764159890 0x7f576415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576415a160 con 0x7f5764159c60 2023-10-08T16:12:03.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640d13a0 con 0x7f5764069560 2023-10-08T16:12:03.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.806+0000 7f57694fd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576415e140 con 0x7f5764159890 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5762ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5764159890 0x7f576415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f57627fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5764159c60 0x7f576415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51970/0 (socket says 172.21.15.170:51970) 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f57627fc700 1 -- 172.21.15.170:0/4135864002 learned_addr learned my addr 172.21.15.170:0/4135864002 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5762ffd700 1 -- 172.21.15.170:0/4135864002 >> v1:172.21.15.191:6789/0 conn(0x7f5764069560 legacy=0x7f57640ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5762ffd700 1 -- 172.21.15.170:0/4135864002 >> v1:172.21.15.170:6789/0 conn(0x7f5764159c60 legacy=0x7f576415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5762ffd700 1 -- 172.21.15.170:0/4135864002 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576415a3d0 con 0x7f5764159890 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5761ffb700 1 -- 172.21.15.170:0/4135864002 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414414165 0 0) 0x7f576415a160 con 0x7f5764159c60 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5762ffd700 1 --2- 172.21.15.170:0/4135864002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764159890 0x7f576415abd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f57540098e0 tx=0x7f57540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37916d327aeb2465 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5761ffb700 1 -- 172.21.15.170:0/4135864002 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754009e80 con 0x7f5764159890 2023-10-08T16:12:03.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5761ffb700 1 -- 172.21.15.170:0/4135864002 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575400b860 con 0x7f5764159890 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.807+0000 7f5761ffb700 1 -- 172.21.15.170:0/4135864002 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575400bbe0 con 0x7f5764159890 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 -- 172.21.15.170:0/4135864002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764159890 msgr2=0x7f576415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 --2- 172.21.15.170:0/4135864002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764159890 0x7f576415abd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f57540098e0 tx=0x7f57540091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 -- 172.21.15.170:0/4135864002 shutdown_connections 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 --2- 172.21.15.170:0/4135864002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764159890 0x7f576415abd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 -- 172.21.15.170:0/4135864002 >> 172.21.15.170:0/4135864002 conn(0x7f5764068e20 msgr2=0x7f57640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 -- 172.21.15.170:0/4135864002 shutdown_connections 2023-10-08T16:12:03.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.808+0000 7f57694fd700 1 -- 172.21.15.170:0/4135864002 wait complete. 2023-10-08T16:12:03.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.809+0000 7f57694fd700 1 Processor -- start 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.809+0000 7f57694fd700 1 -- start start 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764159890 0x7f576415d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5764159c60 0x7f576415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576414da20 con 0x7f5764069560 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640b3ec0 con 0x7f5764159890 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cef50 con 0x7f5764159c60 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53110/0 (socket says 172.21.15.170:53110) 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 -- 172.21.15.170:0/3262529277 learned_addr learned my addr 172.21.15.170:0/3262529277 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57627fc700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764159890 0x7f576415d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 -- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5764159c60 msgr2=0x7f576415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57637fe700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5764159c60 0x7f576415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5764159c60 0x7f576415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 -- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764159890 msgr2=0x7f576415d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764159890 0x7f576415d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 -- 172.21.15.170:0/3262529277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5754009420 con 0x7f5764069560 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f5762ffd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5764070e20 tx=0x7f5754017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575401f070 con 0x7f5764069560 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575400b9d0 con 0x7f5764069560 2023-10-08T16:12:03.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5754025670 con 0x7f5764069560 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.810+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57640b3590 con 0x7f5764069560 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.811+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57640b3b50 con 0x7f5764069560 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.811+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576404c3c0 con 0x7f5764069560 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.812+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5754023070 con 0x7f5764069560 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.812+0000 7f574affd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 0x7f57500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.812+0000 7f57627fc700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 0x7f57500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:03.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.812+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f5754086410 con 0x7f5764069560 2023-10-08T16:12:03.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.813+0000 7f57627fc700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 0x7f57500697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f57641590f0 tx=0x7f5758009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:03.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:03.817+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5754053050 con 0x7f5764069560 2023-10-08T16:12:04.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.042+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57640b1720 con 0x7f5764069560 2023-10-08T16:12:04.044 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.043+0000 7f574affd700 1 -- 172.21.15.170:0/3262529277 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4624 (secure 0 0 0) 0x7f5754056670 con 0x7f5764069560 2023-10-08T16:12:04.044 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:04.044 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:58.806252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000793388","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:12:04.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 msgr2=0x7f57500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 0x7f57500697d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f57641590f0 tx=0x7f5758009040 comp rx=0 tx=0).stop 2023-10-08T16:12:04.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 msgr2=0x7f576414d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5764070e20 tx=0x7f5754017000 comp rx=0 tx=0).stop 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 shutdown_connections 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5764159c60 0x7f576415d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5750067320 0x7f57500697d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5764069560 0x7f576414d4e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 --2- 172.21.15.170:0/3262529277 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5764159890 0x7f576415d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 >> 172.21.15.170:0/3262529277 conn(0x7f5764068e20 msgr2=0x7f57640b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 shutdown_connections 2023-10-08T16:12:04.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.045+0000 7f57694fd700 1 -- 172.21.15.170:0/3262529277 wait complete. 2023-10-08T16:12:04.048 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 92 2023-10-08T16:12:04.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:04.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.171+0000 7f8454afe700 1 Processor -- start 2023-10-08T16:12:04.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.171+0000 7f8454afe700 1 -- start start 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f8454afe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f8454afe700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845015a160 con 0x7f8450069560 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f8454afe700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500d13a0 con 0x7f8450159c60 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f8454afe700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845015e140 con 0x7f8450159890 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844e59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8450069560 0x7f84500ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51978/0 (socket says 172.21.15.170:51978) 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844ed9d700 1 -- 172.21.15.170:0/3187948492 learned_addr learned my addr 172.21.15.170:0/3187948492 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844e59c700 1 -- 172.21.15.170:0/3187948492 >> v1:172.21.15.191:6789/0 conn(0x7f8450159c60 legacy=0x7f845015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844e59c700 1 -- 172.21.15.170:0/3187948492 >> v1:172.21.15.170:6789/0 conn(0x7f8450069560 legacy=0x7f84500ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844e59c700 1 -- 172.21.15.170:0/3187948492 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845015a3d0 con 0x7f8450159890 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.172+0000 7f844dd9b700 1 -- 172.21.15.170:0/3187948492 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948860465 0 0) 0x7f845015a160 con 0x7f8450069560 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f844e59c700 1 --2- 172.21.15.170:0/3187948492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015abd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f84400098e0 tx=0x7f84400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5619061a315cb213 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f844dd9b700 1 -- 172.21.15.170:0/3187948492 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440009e80 con 0x7f8450159890 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f844dd9b700 1 -- 172.21.15.170:0/3187948492 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844000b860 con 0x7f8450159890 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f844dd9b700 1 -- 172.21.15.170:0/3187948492 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844000bbe0 con 0x7f8450159890 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 -- 172.21.15.170:0/3187948492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 msgr2=0x7f845015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 --2- 172.21.15.170:0/3187948492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015abd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f84400098e0 tx=0x7f84400091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 -- 172.21.15.170:0/3187948492 shutdown_connections 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 --2- 172.21.15.170:0/3187948492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015abd0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 -- 172.21.15.170:0/3187948492 >> 172.21.15.170:0/3187948492 conn(0x7f8450068e20 msgr2=0x7f84500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 -- 172.21.15.170:0/3187948492 shutdown_connections 2023-10-08T16:12:04.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.173+0000 7f8454afe700 1 -- 172.21.15.170:0/3187948492 wait complete. 2023-10-08T16:12:04.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 Processor -- start 2023-10-08T16:12:04.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 -- start start 2023-10-08T16:12:04.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8450069560 0x7f8450154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159c60 0x7f845015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845014cba0 con 0x7f8450159890 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845016b440 con 0x7f8450069560 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8454afe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845015ca30 con 0x7f8450159c60 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8445d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8445d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53118/0 (socket says 172.21.15.170:53118) 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.175+0000 7f8445d9b700 1 -- 172.21.15.170:0/1366636108 learned_addr learned my addr 172.21.15.170:0/1366636108 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f844e59c700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8450069560 0x7f8450154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f844ed9d700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159c60 0x7f845015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 -- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159c60 msgr2=0x7f845015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159c60 0x7f845015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 -- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8450069560 msgr2=0x7f8450154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8450069560 0x7f8450154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 -- 172.21.15.170:0/1366636108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8440009420 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8445d9b700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f843800c4e0 tx=0x7f843800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011590 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8438011710 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845016b160 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438011ad0 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.176+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845015cfb0 con 0x7f8450159890 2023-10-08T16:12:04.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.177+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845004c3c0 con 0x7f8450159890 2023-10-08T16:12:04.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.177+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f843801b3b0 con 0x7f8450159890 2023-10-08T16:12:04.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.178+0000 7f8446ffd700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 0x7f84300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.178+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f8438084050 con 0x7f8450159890 2023-10-08T16:12:04.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.178+0000 7f844e59c700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 0x7f84300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.178+0000 7f844e59c700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 0x7f84300697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84400093f0 tx=0x7f8440017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.182+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8438050c90 con 0x7f8450159890 2023-10-08T16:12:04.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.367+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8450158ef0 con 0x7f8450159890 2023-10-08T16:12:04.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.368+0000 7f8446ffd700 1 -- 172.21.15.170:0/1366636108 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f8438017070 con 0x7f8450159890 2023-10-08T16:12:04.370 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:04.370 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:00.813675+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000793388.meta","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000793388"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000793388.data","create_time":"2023-10-08T16:11:46.115982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000793388"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:04.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 msgr2=0x7f84300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 0x7f84300697d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f84400093f0 tx=0x7f8440017000 comp rx=0 tx=0).stop 2023-10-08T16:12:04.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 msgr2=0x7f845015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f843800c4e0 tx=0x7f843800c7f0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 shutdown_connections 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8450159c60 0x7f845015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8450069560 0x7f8450154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8430067320 0x7f84300697d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 --2- 172.21.15.170:0/1366636108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8450159890 0x7f845015d4d0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.370+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 >> 172.21.15.170:0/1366636108 conn(0x7f8450068e20 msgr2=0x7f8450152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.371+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 shutdown_connections 2023-10-08T16:12:04.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.371+0000 7f8454afe700 1 -- 172.21.15.170:0/1366636108 wait complete. 2023-10-08T16:12:04.385 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000793388 and related pools 2023-10-08T16:12:04.386 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:04.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.500+0000 7fbe19198700 1 Processor -- start 2023-10-08T16:12:04.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.500+0000 7fbe19198700 1 -- start start 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe19198700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe14159c60 0x7fbe1415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe19198700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1415a160 con 0x7fbe1415db10 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe19198700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe140d13a0 con 0x7fbe14159890 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe19198700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1415df50 con 0x7fbe14159c60 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe1259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbe14159c60 0x7fbe1415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe1359e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbe1415db10 0x7fbe14069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51994/0 (socket says 172.21.15.170:51994) 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe1359e700 1 -- 172.21.15.170:0/2661740434 learned_addr learned my addr 172.21.15.170:0/2661740434 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:04.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe1259c700 1 -- 172.21.15.170:0/2661740434 >> v1:172.21.15.191:6789/0 conn(0x7fbe14159890 legacy=0x7fbe1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe1259c700 1 -- 172.21.15.170:0/2661740434 >> v1:172.21.15.170:6789/0 conn(0x7fbe1415db10 legacy=0x7fbe14069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.501+0000 7fbe11d9b700 1 -- 172.21.15.170:0/2661740434 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407531206 0 0) 0x7fbe1415a160 con 0x7fbe1415db10 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe1259c700 1 -- 172.21.15.170:0/2661740434 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1415e140 con 0x7fbe14159c60 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe1259c700 1 --2- 172.21.15.170:0/2661740434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159c60 0x7fbe1415d5d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098e0 tx=0x7fbdfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1fde9da62297be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe11d9b700 1 -- 172.21.15.170:0/2661740434 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc009e80 con 0x7fbe14159c60 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe11d9b700 1 -- 172.21.15.170:0/2661740434 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc00b860 con 0x7fbe14159c60 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe11d9b700 1 -- 172.21.15.170:0/2661740434 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc00bbe0 con 0x7fbe14159c60 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 -- 172.21.15.170:0/2661740434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159c60 msgr2=0x7fbe1415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 --2- 172.21.15.170:0/2661740434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159c60 0x7fbe1415d5d0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0098e0 tx=0x7fbdfc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 -- 172.21.15.170:0/2661740434 shutdown_connections 2023-10-08T16:12:04.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 --2- 172.21.15.170:0/2661740434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159c60 0x7fbe1415d5d0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 -- 172.21.15.170:0/2661740434 >> 172.21.15.170:0/2661740434 conn(0x7fbe14068e20 msgr2=0x7fbe140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 -- 172.21.15.170:0/2661740434 shutdown_connections 2023-10-08T16:12:04.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.502+0000 7fbe19198700 1 -- 172.21.15.170:0/2661740434 wait complete. 2023-10-08T16:12:04.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 Processor -- start 2023-10-08T16:12:04.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 -- start start 2023-10-08T16:12:04.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe14159c60 0x7fbe140c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe1415db10 0x7fbe140c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140ca650 con 0x7fbe14159890 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1416d0e0 con 0x7fbe1415db10 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe19198700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1416a880 con 0x7fbe14159c60 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe12d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53146/0 (socket says 172.21.15.170:53146) 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.504+0000 7fbe12d9d700 1 -- 172.21.15.170:0/2024820070 learned_addr learned my addr 172.21.15.170:0/2024820070 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe1359e700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe1415db10 0x7fbe140c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe1259c700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe14159c60 0x7fbe140c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 -- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe14159c60 msgr2=0x7fbe140c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe14159c60 0x7fbe140c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 -- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe1415db10 msgr2=0x7fbe140c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe1415db10 0x7fbe140c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 -- 172.21.15.170:0/2024820070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc009420 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe12d9d700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00cb10 tx=0x7fbe0c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00ce20 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c00a7e0 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1416d330 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c00aba0 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.505+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1416d8f0 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.506+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1416db10 con 0x7fbe14159890 2023-10-08T16:12:04.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.506+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbe0c003030 con 0x7fbe14159890 2023-10-08T16:12:04.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.507+0000 7fbe0affd700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 0x7fbdf40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.507+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fbe0c084c90 con 0x7fbe14159890 2023-10-08T16:12:04.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.507+0000 7fbe1259c700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 0x7fbdf40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.507+0000 7fbe1259c700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 0x7fbdf40697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0093f0 tx=0x7fbdfc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.511+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe0c051890 con 0x7fbe14159890 2023-10-08T16:12:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.736+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1404c3c0 con 0x7fbe14159890 2023-10-08T16:12:04.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.736+0000 7fbe0affd700 1 -- 172.21.15.170:0/2024820070 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4624 (secure 0 0 0) 0x7fbe0c054eb0 con 0x7fbe14159890 2023-10-08T16:12:04.738 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:04.738 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:58.806252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000793388","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:12:04.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.738+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 msgr2=0x7fbdf40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.738+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 0x7fbdf40697d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0093f0 tx=0x7fbdfc017000 comp rx=0 tx=0).stop 2023-10-08T16:12:04.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.738+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 msgr2=0x7fbe140ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.738+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00cb10 tx=0x7fbe0c0017b0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.738+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 shutdown_connections 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbe1415db10 0x7fbe140c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbdf4067320 0x7fbdf40697d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe14159890 0x7fbe140ca110 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 --2- 172.21.15.170:0/2024820070 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbe14159c60 0x7fbe140c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 >> 172.21.15.170:0/2024820070 conn(0x7fbe14068e20 msgr2=0x7fbe14153da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 shutdown_connections 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.739+0000 7fbe19198700 1 -- 172.21.15.170:0/2024820070 wait complete. 2023-10-08T16:12:04.741 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 92 2023-10-08T16:12:04.752 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 Processor -- start 2023-10-08T16:12:04.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 -- start start 2023-10-08T16:12:04.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc74006bcd0 0x7fc740140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74015c2a0 con 0x7fc74006c0a0 2023-10-08T16:12:04.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc74015b690 con 0x7fc7400c9510 2023-10-08T16:12:04.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.869+0000 7fc746fd4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7400c9900 con 0x7fc74006bcd0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc744d70700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc74006bcd0 0x7fc740140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc74006c0a0 0x7fc7400c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51996/0 (socket says 172.21.15.170:51996) 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73ffff700 1 -- 172.21.15.170:0/707186716 learned_addr learned my addr 172.21.15.170:0/707186716 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc744d70700 1 -- 172.21.15.170:0/707186716 >> v1:172.21.15.191:6789/0 conn(0x7fc7400c9510 legacy=0x7fc7400cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc744d70700 1 -- 172.21.15.170:0/707186716 >> v1:172.21.15.170:6789/0 conn(0x7fc74006c0a0 legacy=0x7fc7400c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc744d70700 1 -- 172.21.15.170:0/707186716 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74015b970 con 0x7fc74006bcd0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73f7fe700 1 -- 172.21.15.170:0/707186716 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968228631 0 0) 0x7fc74015c2a0 con 0x7fc74006c0a0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc744d70700 1 --2- 172.21.15.170:0/707186716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006bcd0 0x7fc740140df0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc730009e60 tx=0x7fc730009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc4f9ed1cd73381 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73f7fe700 1 -- 172.21.15.170:0/707186716 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000e070 con 0x7fc74006bcd0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73f7fe700 1 -- 172.21.15.170:0/707186716 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000bc30 con 0x7fc74006bcd0 2023-10-08T16:12:04.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.870+0000 7fc73f7fe700 1 -- 172.21.15.170:0/707186716 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730012430 con 0x7fc74006bcd0 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 -- 172.21.15.170:0/707186716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006bcd0 msgr2=0x7fc740140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 --2- 172.21.15.170:0/707186716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006bcd0 0x7fc740140df0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fc730009e60 tx=0x7fc730009780 comp rx=0 tx=0).stop 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 -- 172.21.15.170:0/707186716 shutdown_connections 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 --2- 172.21.15.170:0/707186716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006bcd0 0x7fc740140df0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 -- 172.21.15.170:0/707186716 >> 172.21.15.170:0/707186716 conn(0x7fc7400ce600 msgr2=0x7fc7400cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 -- 172.21.15.170:0/707186716 shutdown_connections 2023-10-08T16:12:04.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.871+0000 7fc746fd4700 1 -- 172.21.15.170:0/707186716 wait complete. 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.872+0000 7fc746fd4700 1 Processor -- start 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.872+0000 7fc746fd4700 1 -- start start 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 0x7fc7400c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 0x7fc7400c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7400c7b50 con 0x7fc74006c0a0 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74016bcd0 con 0x7fc7400c9510 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74016c160 con 0x7fc74006bcd0 2023-10-08T16:12:04.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53154/0 (socket says 172.21.15.170:53154) 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 -- 172.21.15.170:0/2326946528 learned_addr learned my addr 172.21.15.170:0/2326946528 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc744d70700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 0x7fc7400c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc745571700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 0x7fc7400c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 -- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 msgr2=0x7fc7400c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 0x7fc7400c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 -- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 msgr2=0x7fc7400c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 0x7fc7400c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 -- 172.21.15.170:0/2326946528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc730009a20 con 0x7fc74006c0a0 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc744d70700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 0x7fc7400c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc745571700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 0x7fc7400c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:04.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73ffff700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c810 tx=0x7fc734007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734005b30 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc734005c90 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7400b41b0 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734007300 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.873+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74016b680 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.874+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74016be60 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.875+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc73401b400 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.875+0000 7fc73cff9700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 0x7fc72c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.875+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fc734084760 con 0x7fc74006c0a0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.875+0000 7fc744d70700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 0x7fc72c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:04.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.876+0000 7fc744d70700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 0x7fc72c0697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc730009e60 tx=0x7fc730018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:04.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:04.879+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7340514a0 con 0x7fc74006c0a0 2023-10-08T16:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.103+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc74004c3c0 con 0x7fc74006c0a0 2023-10-08T16:12:05.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.104+0000 7fc73cff9700 1 -- 172.21.15.170:0/2326946528 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4624 (secure 0 0 0) 0x7fc734054ac0 con 0x7fc74006c0a0 2023-10-08T16:12:05.104 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:05.104 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":92,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:11:47.159522+0000","modified":"2023-10-08T16:11:58.806252+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"d","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.191:6835/1124398845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":1124398845},{"type":"v1","addr":"172.21.15.191:6835","nonce":1124398845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"volume_0000000000793388","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T16:12:05.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 msgr2=0x7fc72c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 0x7fc72c0697d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc730009e60 tx=0x7fc730018000 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 msgr2=0x7fc7400c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fc73400c810 tx=0x7fc734007df0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 shutdown_connections 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc7400c9510 0x7fc7400c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc74006bcd0 0x7fc7400c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc72c067320 0x7fc72c0697d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 --2- 172.21.15.170:0/2326946528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc74006c0a0 0x7fc7400c7000 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 >> 172.21.15.170:0/2326946528 conn(0x7fc7400ce600 msgr2=0x7fc7400cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 shutdown_connections 2023-10-08T16:12:05.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.106+0000 7fc746fd4700 1 -- 172.21.15.170:0/2326946528 wait complete. 2023-10-08T16:12:05.108 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 92 2023-10-08T16:12:05.118 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:05.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.232+0000 7f29e5b95700 1 Processor -- start 2023-10-08T16:12:05.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.232+0000 7f29e5b95700 1 -- start start 2023-10-08T16:12:05.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29e5b95700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f29e00b1c20 0x7f29e01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29e5b95700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e0157c60 con 0x7f29e00b1850 2023-10-08T16:12:05.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29e5b95700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e00cb300 con 0x7f29e0157820 2023-10-08T16:12:05.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29e5b95700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e006fc30 con 0x7f29e00b1c20 2023-10-08T16:12:05.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29df7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f29e00b1850 0x7f29e00ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51772/0 (socket says 172.21.15.170:51772) 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29df7fe700 1 -- 172.21.15.170:0/2565621303 learned_addr learned my addr 172.21.15.170:0/2565621303 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.233+0000 7f29deffd700 1 --2- 172.21.15.170:0/2565621303 >> v2:172.21.15.170:3300/0 conn(0x7f29e00b1c20 0x7f29e01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29de7fc700 1 -- 172.21.15.170:0/2565621303 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923898606 0 0) 0x7f29e0157c60 con 0x7f29e00b1850 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29de7fc700 1 -- 172.21.15.170:0/2565621303 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c8003570 con 0x7f29e00b1850 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29deffd700 1 -- 172.21.15.170:0/2565621303 >> v1:172.21.15.191:6789/0 conn(0x7f29e0157820 legacy=0x7f29e00abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29deffd700 1 -- 172.21.15.170:0/2565621303 >> v1:172.21.15.170:6789/0 conn(0x7f29e00b1850 legacy=0x7f29e00ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29deffd700 1 -- 172.21.15.170:0/2565621303 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00cb5e0 con 0x7f29e00b1c20 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29deffd700 1 --2- 172.21.15.170:0/2565621303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00b1c20 0x7f29e01572e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098e0 tx=0x7f29d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3ea2fceb865aa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29de7fc700 1 -- 172.21.15.170:0/2565621303 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011630 con 0x7f29e00b1c20 2023-10-08T16:12:05.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29de7fc700 1 -- 172.21.15.170:0/2565621303 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4011790 con 0x7f29e00b1c20 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29de7fc700 1 -- 172.21.15.170:0/2565621303 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011b30 con 0x7f29e00b1c20 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 -- 172.21.15.170:0/2565621303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00b1c20 msgr2=0x7f29e01572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2565621303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00b1c20 0x7f29e01572e0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f29d40098e0 tx=0x7f29d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 -- 172.21.15.170:0/2565621303 shutdown_connections 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2565621303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e00b1c20 0x7f29e01572e0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 -- 172.21.15.170:0/2565621303 >> 172.21.15.170:0/2565621303 conn(0x7f29e006a320 msgr2=0x7f29e00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 -- 172.21.15.170:0/2565621303 shutdown_connections 2023-10-08T16:12:05.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.234+0000 7f29e5b95700 1 -- 172.21.15.170:0/2565621303 wait complete. 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 Processor -- start 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 -- start start 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29e00b1850 0x7f29e0151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29e00b1c20 0x7f29e0154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e00cc260 con 0x7f29e0157820 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e015b000 con 0x7f29e00b1850 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.236+0000 7f29e5b95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e015a3f0 con 0x7f29e00b1c20 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49608/0 (socket says 172.21.15.170:49608) 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 -- 172.21.15.170:0/2183862035 learned_addr learned my addr 172.21.15.170:0/2183862035 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29deffd700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29e00b1c20 0x7f29e0154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29df7fe700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29e00b1850 0x7f29e0151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 -- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29e00b1c20 msgr2=0x7f29e0154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29e00b1c20 0x7f29e0154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 -- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29e00b1850 msgr2=0x7f29e0151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29e00b1850 0x7f29e0151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 -- 172.21.15.170:0/2183862035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009420 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29dffff700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f29d800b4e0 tx=0x7f29d8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800f5e0 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d800f740 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e015a6d0 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800fb00 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.237+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e015ac90 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.238+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e004c3c0 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.239+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f29d801a400 con 0x7f29e0157820 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.239+0000 7f29c77fe700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 0x7f29cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.239+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7f29d8084830 con 0x7f29e0157820 2023-10-08T16:12:05.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.239+0000 7f29df7fe700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 0x7f29cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.240+0000 7f29df7fe700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 0x7f29cc0697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005c50 tx=0x7f29d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.243+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29d80514f0 con 0x7f29e0157820 2023-10-08T16:12:05.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.426+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29e00cb8c0 con 0x7f29e0157820 2023-10-08T16:12:05.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.427+0000 7f29c77fe700 1 -- 172.21.15.170:0/2183862035 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+15490 (secure 0 0 0) 0x7f29d807ce60 con 0x7f29e0157820 2023-10-08T16:12:05.428 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:05.428 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":142,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:00.813675+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs.volume_0000000000793388.meta","create_time":"2023-10-08T16:11:45.493023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000793388"}}},{"pool":25,"pool_name":"cephfs.volume_0000000000793388.data","create_time":"2023-10-08T16:11:46.115982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000793388"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 msgr2=0x7f29cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 0x7f29cc0697d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f29d0005c50 tx=0x7f29d0009040 comp rx=0 tx=0).stop 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 msgr2=0x7f29e0155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f29d800b4e0 tx=0x7f29d8006f90 comp rx=0 tx=0).stop 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 shutdown_connections 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f29cc067320 0x7f29cc0697d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29e0157820 0x7f29e0155070 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f29e00b1850 0x7f29e0151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 --2- 172.21.15.170:0/2183862035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f29e00b1c20 0x7f29e0154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 >> 172.21.15.170:0/2183862035 conn(0x7f29e006a320 msgr2=0x7f29e0152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 shutdown_connections 2023-10-08T16:12:05.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.429+0000 7f29e5b95700 1 -- 172.21.15.170:0/2183862035 wait complete. 2023-10-08T16:12:05.442 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000793388 2023-10-08T16:12:05.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.558+0000 7fa38d3d6700 1 Processor -- start 2023-10-08T16:12:05.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.558+0000 7fa38d3d6700 1 -- start start 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa38d3d6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa388159420 0x7fa3880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa38d3d6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880b2980 con 0x7fa3881597f0 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa38d3d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880b11d0 con 0x7fa38815d230 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa38d3d6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3880b14b0 con 0x7fa388159420 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa3877fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa3881597f0 0x7fa3880b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51774/0 (socket says 172.21.15.170:51774) 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa3877fe700 1 -- 172.21.15.170:0/4077002913 learned_addr learned my addr 172.21.15.170:0/4077002913 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa387fff700 1 --2- 172.21.15.170:0/4077002913 >> v2:172.21.15.191:3300/0 conn(0x7fa388159420 0x7fa3880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488395847 0 0) 0x7fa3880b2980 con 0x7fa3881597f0 2023-10-08T16:12:05.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.559+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa370003570 con 0x7fa3881597f0 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1685664251 0 0) 0x7fa370003570 con 0x7fa3881597f0 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa387fff700 1 -- 172.21.15.170:0/4077002913 >> v1:172.21.15.191:6789/0 conn(0x7fa38815d230 legacy=0x7fa38815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa387fff700 1 -- 172.21.15.170:0/4077002913 >> v1:172.21.15.170:6789/0 conn(0x7fa3881597f0 legacy=0x7fa3880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa387fff700 1 -- 172.21.15.170:0/4077002913 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880b1790 con 0x7fa388159420 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa387fff700 1 --2- 172.21.15.170:0/4077002913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa388159420 0x7fa3880b5f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa3780098e0 tx=0x7fa3780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e09b78e67bd25d02 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c001470 con 0x7fa388159420 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378011420 con 0x7fa388159420 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa386ffd700 1 -- 172.21.15.170:0/4077002913 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3780117c0 con 0x7fa388159420 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa38d3d6700 1 -- 172.21.15.170:0/4077002913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa388159420 msgr2=0x7fa3880b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/4077002913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa388159420 0x7fa3880b5f40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa3780098e0 tx=0x7fa3780091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.560+0000 7fa38d3d6700 1 -- 172.21.15.170:0/4077002913 shutdown_connections 2023-10-08T16:12:05.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.561+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/4077002913 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa388159420 0x7fa3880b5f40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.561+0000 7fa38d3d6700 1 -- 172.21.15.170:0/4077002913 >> 172.21.15.170:0/4077002913 conn(0x7fa3880d0660 msgr2=0x7fa3880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.561+0000 7fa38d3d6700 1 -- 172.21.15.170:0/4077002913 shutdown_connections 2023-10-08T16:12:05.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.561+0000 7fa38d3d6700 1 -- 172.21.15.170:0/4077002913 wait complete. 2023-10-08T16:12:05.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.562+0000 7fa38d3d6700 1 Processor -- start 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.562+0000 7fa38d3d6700 1 -- start start 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.562+0000 7fa38d3d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa388159420 0x7fa388155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38815d230 0x7fa3880b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388155570 con 0x7fa3881597f0 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38815ca20 con 0x7fa38815d230 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38815be10 con 0x7fa388159420 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49622/0 (socket says 172.21.15.170:49622) 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 -- 172.21.15.170:0/764064971 learned_addr learned my addr 172.21.15.170:0/764064971 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:05.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa387fff700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa388159420 0x7fa388155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38cbd5700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38815d230 0x7fa3880b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 -- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa388159420 msgr2=0x7fa388155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa388159420 0x7fa388155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 -- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38815d230 msgr2=0x7fa3880b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38815d230 0x7fa3880b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 -- 172.21.15.170:0/764064971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa378009420 con 0x7fa3881597f0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa3877fe700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c930 tx=0x7fa37c001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c005b30 con 0x7fa3881597f0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37c005cb0 con 0x7fa3881597f0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38815c0f0 con 0x7fa3881597f0 2023-10-08T16:12:05.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37c007340 con 0x7fa3881597f0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.563+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38815c6b0 con 0x7fa3881597f0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.564+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38815cb90 con 0x7fa3881597f0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.565+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa37c012400 con 0x7fa3881597f0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.565+0000 7fa36ffff700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 0x7fa3740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.565+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5783+0+0 (secure 0 0 0) 0x7fa37c0841f0 con 0x7fa3881597f0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.565+0000 7fa387fff700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 0x7fa3740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.566+0000 7fa387fff700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 0x7fa3740697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa3780091e0 tx=0x7fa378016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.569+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa37c04b5a0 con 0x7fa3881597f0 2023-10-08T16:12:05.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.794+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000793388"} v 0) v1 -- 0x7fa38804c3c0 con 0x7fa3881597f0 2023-10-08T16:12:05.795 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:12:05.795+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:12:05.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.799+0000 7fa36ffff700 1 -- 172.21.15.170:0/764064971 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000793388"}]=0 volume_0000000000793388 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) v1 ==== 195+0+0 (secure 0 0 0) 0x7fa37c050f60 con 0x7fa3881597f0 2023-10-08T16:12:05.800 INFO:tasks.ceph.mds.d.smithi191.stderr: -755> 2023-10-08T16:11:53.085+0000 7f133a13c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:12:05.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 msgr2=0x7fa3740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 0x7fa3740697d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa3780091e0 tx=0x7fa378016000 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 msgr2=0x7fa38815aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c930 tx=0x7fa37c001a10 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 shutdown_connections 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa38815d230 0x7fa3880b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa388159420 0x7fa388155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa374067320 0x7fa3740697d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 --2- 172.21.15.170:0/764064971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3881597f0 0x7fa38815aa00 secure :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c930 tx=0x7fa37c001a10 comp rx=0 tx=0).stop 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 >> 172.21.15.170:0/764064971 conn(0x7fa3880d0660 msgr2=0x7fa388156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 shutdown_connections 2023-10-08T16:12:05.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.802+0000 7fa38d3d6700 1 -- 172.21.15.170:0/764064971 wait complete. 2023-10-08T16:12:05.804 INFO:teuthology.orchestra.run.smithi170.stderr:volume_0000000000793388 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:12:05.815 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000793388 --yes-i-really-mean-it 2023-10-08T16:12:05.821 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:12:05.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.931+0000 7fa998eae700 1 Processor -- start 2023-10-08T16:12:05.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.931+0000 7fa998eae700 1 -- start start 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa998eae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa99415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa998eae700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99415a160 con 0x7fa994159c60 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa998eae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940d13a0 con 0x7fa994069560 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa998eae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99415e140 con 0x7fa994159890 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa991d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa994159c60 0x7fa99415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51776/0 (socket says 172.21.15.170:51776) 2023-10-08T16:12:05.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa991d9b700 1 -- 172.21.15.170:0/3467855178 learned_addr learned my addr 172.21.15.170:0/3467855178 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99259c700 1 --2- 172.21.15.170:0/3467855178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa99415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043556754 0 0) 0x7fa99415a160 con 0x7fa994159c60 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa97c003570 con 0x7fa994159c60 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374485038 0 0) 0x7fa9940d13a0 con 0x7fa994069560 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa99415a160 con 0x7fa994069560 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99259c700 1 -- 172.21.15.170:0/3467855178 >> v1:172.21.15.191:6789/0 conn(0x7fa994069560 legacy=0x7fa9940ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99259c700 1 -- 172.21.15.170:0/3467855178 >> v1:172.21.15.170:6789/0 conn(0x7fa994159c60 legacy=0x7fa99415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.932+0000 7fa99259c700 1 -- 172.21.15.170:0/3467855178 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99415a3d0 con 0x7fa994159890 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa99259c700 1 --2- 172.21.15.170:0/3467855178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa99415abd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa984009910 tx=0x7fa9840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ad6d9e033cf557 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011630 con 0x7fa994159890 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984011790 con 0x7fa994159890 2023-10-08T16:12:05.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa99159a700 1 -- 172.21.15.170:0/3467855178 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011b30 con 0x7fa994159890 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 -- 172.21.15.170:0/3467855178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 msgr2=0x7fa99415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 --2- 172.21.15.170:0/3467855178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa99415abd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa984009910 tx=0x7fa9840091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 -- 172.21.15.170:0/3467855178 shutdown_connections 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 --2- 172.21.15.170:0/3467855178 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa99415abd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 -- 172.21.15.170:0/3467855178 >> 172.21.15.170:0/3467855178 conn(0x7fa994068e20 msgr2=0x7fa9940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 -- 172.21.15.170:0/3467855178 shutdown_connections 2023-10-08T16:12:05.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.933+0000 7fa998eae700 1 -- 172.21.15.170:0/3467855178 wait complete. 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.934+0000 7fa998eae700 1 Processor -- start 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.934+0000 7fa998eae700 1 -- start start 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994069560 0x7fa99414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa994151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99416e290 con 0x7fa994159c60 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99416a220 con 0x7fa994069560 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa998eae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99416a500 con 0x7fa994159890 2023-10-08T16:12:05.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa992d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa992d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49624/0 (socket says 172.21.15.170:49624) 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa992d9d700 1 -- 172.21.15.170:0/3788857951 learned_addr learned my addr 172.21.15.170:0/3788857951 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa991d9b700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa994151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.935+0000 7fa99259c700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994069560 0x7fa99414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 -- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 msgr2=0x7fa994151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa994151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 -- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994069560 msgr2=0x7fa99414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994069560 0x7fa99414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 -- 172.21.15.170:0/3788857951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984009420 con 0x7fa994159c60 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa992d9d700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c910 tx=0x7fa98c0027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c012630 con 0x7fa994159c60 2023-10-08T16:12:05.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0127b0 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99416a7e0 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c012b70 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.936+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99416ad00 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.937+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99416aa00 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.937+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa98c01b400 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.937+0000 7fa9827fc700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 0x7fa9740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.938+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5869+0+0 (secure 0 0 0) 0x7fa98c085020 con 0x7fa994159c60 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.938+0000 7fa99259c700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 0x7fa9740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:05.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.938+0000 7fa99259c700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 0x7fa9740697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa984009860 tx=0x7fa984017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:05.942 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:05.942+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa98c051c10 con 0x7fa994159c60 2023-10-08T16:12:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.169+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000793388", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa99404c3c0 con 0x7fa994159c60 2023-10-08T16:12:06.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.800+0000 7fa9827fc700 1 -- 172.21.15.170:0/3788857951 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000793388", "yes_i_really_mean_it": true}]=0 v94) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa98c055230 con 0x7fa994159c60 2023-10-08T16:12:06.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 msgr2=0x7fa9740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 0x7fa9740697d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa984009860 tx=0x7fa984017000 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 msgr2=0x7fa994151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00c910 tx=0x7fa98c0027a0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 shutdown_connections 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa974067320 0x7fa9740697d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa994159c60 0x7fa994151f90 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa994069560 0x7fa99414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 --2- 172.21.15.170:0/3788857951 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa994159890 0x7fa994151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 >> 172.21.15.170:0/3788857951 conn(0x7fa994068e20 msgr2=0x7fa994150790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 shutdown_connections 2023-10-08T16:12:06.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.803+0000 7fa998eae700 1 -- 172.21.15.170:0/3788857951 wait complete. 2023-10-08T16:12:06.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000793388.meta cephfs.volume_0000000000793388.meta --yes-i-really-really-mean-it 2023-10-08T16:12:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.940+0000 7f7a5bcdf700 1 Processor -- start 2023-10-08T16:12:06.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.940+0000 7f7a5bcdf700 1 -- start start 2023-10-08T16:12:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5bcdf700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7a54155360 0x7f7a540b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5bcdf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a540b40f0 0x7f7a540ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5bcdf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5415c320 con 0x7f7a540ceb60 2023-10-08T16:12:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5bcdf700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a540b4590 con 0x7f7a540b40f0 2023-10-08T16:12:06.943 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5bcdf700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a540cefc0 con 0x7f7a54155360 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5927a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a540b40f0 0x7f7a540ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5a27c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7a540ceb60 0x7f7a5406d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51782/0 (socket says 172.21.15.170:51782) 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.941+0000 7f7a5a27c700 1 -- 172.21.15.170:0/4038981889 learned_addr learned my addr 172.21.15.170:0/4038981889 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a59a7b700 1 --2- 172.21.15.170:0/4038981889 >> v2:172.21.15.191:3300/0 conn(0x7f7a54155360 0x7f7a540b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5927a700 1 -- 172.21.15.170:0/4038981889 >> v2:172.21.15.191:3300/0 conn(0x7f7a54155360 msgr2=0x7f7a540b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5927a700 1 --2- 172.21.15.170:0/4038981889 >> v2:172.21.15.191:3300/0 conn(0x7f7a54155360 0x7f7a540b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5927a700 1 -- 172.21.15.170:0/4038981889 >> v1:172.21.15.170:6789/0 conn(0x7f7a540ceb60 legacy=0x7f7a5406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5927a700 1 -- 172.21.15.170:0/4038981889 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5415b710 con 0x7f7a540b40f0 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a58a79700 1 -- 172.21.15.170:0/4038981889 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612929593 0 0) 0x7f7a5415c320 con 0x7f7a540ceb60 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5927a700 1 --2- 172.21.15.170:0/4038981889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a540ce620 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7a54070e00 tx=0x7f7a440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec74bfcb97e85801 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:06.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a58a79700 1 -- 172.21.15.170:0/4038981889 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400b6b0 con 0x7f7a540b40f0 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a58a79700 1 -- 172.21.15.170:0/4038981889 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400b810 con 0x7f7a540b40f0 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a58a79700 1 -- 172.21.15.170:0/4038981889 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400bbb0 con 0x7f7a540b40f0 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.942+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/4038981889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 msgr2=0x7f7a540ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/4038981889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a540ce620 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7a54070e00 tx=0x7f7a440091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/4038981889 shutdown_connections 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/4038981889 >> v2:172.21.15.191:3300/0 conn(0x7f7a54155360 0x7f7a540b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/4038981889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a540ce620 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/4038981889 >> 172.21.15.170:0/4038981889 conn(0x7f7a5406baf0 msgr2=0x7f7a5406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/4038981889 shutdown_connections 2023-10-08T16:12:06.945 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.943+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/4038981889 wait complete. 2023-10-08T16:12:06.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 Processor -- start 2023-10-08T16:12:06.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 -- start start 2023-10-08T16:12:06.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a540ceb60 0x7f7a540b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a54155360 0x7f7a54156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c8950 con 0x7f7a540b40f0 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c7d40 con 0x7f7a540ceb60 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.944+0000 7f7a5bcdf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c8020 con 0x7f7a54155360 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49636/0 (socket says 172.21.15.170:49636) 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 -- 172.21.15.170:0/3243741990 learned_addr learned my addr 172.21.15.170:0/3243741990 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a5927a700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a540ceb60 0x7f7a540b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 -- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a54155360 msgr2=0x7f7a54156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a54155360 0x7f7a54156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 -- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a540ceb60 msgr2=0x7f7a540b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a540ceb60 0x7f7a540b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 -- 172.21.15.170:0/3243741990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44009420 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a59a7b700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009fa0 tx=0x7f7a50009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000ed20 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000ee80 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a540c8300 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50014670 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.945+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a54169ed0 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.946+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a540c8c30 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.947+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a500149e0 con 0x7f7a540b40f0 2023-10-08T16:12:06.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.947+0000 7f7a49ffb700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 0x7f7a380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:06.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.947+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5869+0+0 (secure 0 0 0) 0x7f7a500854a0 con 0x7f7a540b40f0 2023-10-08T16:12:06.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.947+0000 7f7a5927a700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 0x7f7a380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:06.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.947+0000 7f7a5927a700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 0x7f7a380697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7a540adc30 tx=0x7f7a44017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:06.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:06.951+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a50052090 con 0x7f7a540b40f0 2023-10-08T16:12:07.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.138+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000793388.meta", "pool2": "cephfs.volume_0000000000793388.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7a540718b0 con 0x7f7a540b40f0 2023-10-08T16:12:07.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.861+0000 7f7a49ffb700 1 -- 172.21.15.170:0/3243741990 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000793388.meta", "pool2": "cephfs.volume_0000000000793388.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000793388.meta' removed v144) v1 ==== 241+0+0 (secure 0 0 0) 0x7f7a500556b0 con 0x7f7a540b40f0 2023-10-08T16:12:07.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 msgr2=0x7f7a380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:07.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 0x7f7a380697d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7a540adc30 tx=0x7f7a44017000 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 msgr2=0x7f7a54169990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009fa0 tx=0x7f7a50009880 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 shutdown_connections 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a54155360 0x7f7a54156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a38067320 0x7f7a380697d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a540b40f0 0x7f7a54169990 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 --2- 172.21.15.170:0/3243741990 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a540ceb60 0x7f7a540b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 >> 172.21.15.170:0/3243741990 conn(0x7f7a5406baf0 msgr2=0x7f7a5414ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 shutdown_connections 2023-10-08T16:12:07.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:07.865+0000 7f7a5bcdf700 1 -- 172.21.15.170:0/3243741990 wait complete. 2023-10-08T16:12:07.867 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs.volume_0000000000793388.meta' removed 2023-10-08T16:12:07.880 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000793388.data cephfs.volume_0000000000793388.data --yes-i-really-really-mean-it 2023-10-08T16:12:08.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.000+0000 7f4d5dd5f700 1 Processor -- start 2023-10-08T16:12:08.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.000+0000 7f4d5dd5f700 1 -- start start 2023-10-08T16:12:08.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d5dd5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d580cd910 0x7f4d580ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d5dd5f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d580b5b00 con 0x7f4d580b55a0 2023-10-08T16:12:08.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d5dd5f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d580b5da0 con 0x7f4d58159ec0 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d5dd5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d580b6080 con 0x7f4d580cd910 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d57fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4d580b55a0 0x7f4d580ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51784/0 (socket says 172.21.15.170:51784) 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d57fff700 1 -- 172.21.15.170:0/941424649 learned_addr learned my addr 172.21.15.170:0/941424649 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d56ffd700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d580cd910 0x7f4d580ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071001885 0 0) 0x7f4d580b5b00 con 0x7f4d580b55a0 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d40003570 con 0x7f4d580b55a0 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2307844376 0 0) 0x7f4d40003570 con 0x7f4d580b55a0 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d580cd910 msgr2=0x7f4d580ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d580cd910 0x7f4d580ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 >> v1:172.21.15.191:6789/0 conn(0x7f4d58159ec0 legacy=0x7f4d580b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.001+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580cc100 con 0x7f4d580b55a0 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4d50001470 con 0x7f4d580b55a0 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 >> v1:172.21.15.170:6789/0 conn(0x7f4d580b55a0 legacy=0x7f4d580ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40003a90 0x7f4d40003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d40004810 0x7f4d40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40005180 0x7f4d4000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d580b6080 con 0x7f4d40005180 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4000bfa0 con 0x7f4d40003a90 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4000c0e0 con 0x7f4d40004810 2023-10-08T16:12:08.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40005180 0x7f4d4000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d577fe700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40003a90 0x7f4d40003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d56ffd700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d40004810 0x7f4d40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 -- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d40004810 msgr2=0x7f4d40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d40004810 0x7f4d40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 -- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40003a90 msgr2=0x7f4d40003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d40003a90 0x7f4d40003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 -- 172.21.15.170:0/941424649 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d40003570 con 0x7f4d40005180 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d57fff700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40005180 0x7f4d4000ba60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c3c0 tx=0x7f4d50001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0c068d82d50e4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5000cc90 con 0x7f4d40005180 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5000cdf0 con 0x7f4d40005180 2023-10-08T16:12:08.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.002+0000 7f4d567fc700 1 -- 172.21.15.170:0/941424649 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50005b70 con 0x7f4d40005180 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/941424649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40005180 msgr2=0x7f4d4000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/941424649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d40005180 0x7f4d4000ba60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c3c0 tx=0x7f4d50001e70 comp rx=0 tx=0).stop 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d57fff700 1 -- 172.21.15.170:0/941424649 reap_dead start 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/941424649 shutdown_connections 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/941424649 >> 172.21.15.170:0/941424649 conn(0x7f4d580999f0 msgr2=0x7f4d580b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/941424649 shutdown_connections 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.003+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/941424649 wait complete. 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.004+0000 7f4d5dd5f700 1 Processor -- start 2023-10-08T16:12:08.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.004+0000 7f4d5dd5f700 1 -- start start 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 0x7f4d580cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d5809c9d0 0x7f4d58169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5809b310 con 0x7f4d5809a930 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d58169f90 con 0x7f4d58156e90 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d5dd5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d580cc800 con 0x7f4d5809c9d0 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49656/0 (socket says 172.21.15.170:49656) 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 -- 172.21.15.170:0/3691052851 learned_addr learned my addr 172.21.15.170:0/3691052851 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 -- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d5809c9d0 msgr2=0x7f4d58169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d5809c9d0 0x7f4d58169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d577fe700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 0x7f4d580cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 -- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 msgr2=0x7f4d580cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 0x7f4d580cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 -- 172.21.15.170:0/3691052851 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d580cd0a0 con 0x7f4d5809a930 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d577fe700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 0x7f4d580cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d57fff700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4d50011000 tx=0x7f4d50015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:08.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.005+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50000c30 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.006+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50000dd0 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.006+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50007320 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.006+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5809ce70 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.006+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d580ccd30 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.006+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d58063c90 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.007+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d500076d0 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.007+0000 7f4d3f7fe700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 0x7f4d440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.007+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5394+0+0 (secure 0 0 0) 0x7f4d500854d0 con 0x7f4d5809a930 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.007+0000 7f4d577fe700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 0x7f4d440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:08.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.008+0000 7f4d577fe700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 0x7f4d440697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4d480042e0 tx=0x7f4d48001b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:08.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.012+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d50052340 con 0x7f4d5809a930 2023-10-08T16:12:08.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.193+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000793388.data", "pool2": "cephfs.volume_0000000000793388.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d580cd230 con 0x7f4d5809a930 2023-10-08T16:12:08.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.870+0000 7f4d3f7fe700 1 -- 172.21.15.170:0/3691052851 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000793388.data", "pool2": "cephfs.volume_0000000000793388.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000793388.data' removed v145) v1 ==== 241+0+0 (secure 0 0 0) 0x7f4d50055960 con 0x7f4d5809a930 2023-10-08T16:12:08.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 msgr2=0x7f4d440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 0x7f4d440697d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4d480042e0 tx=0x7f4d48001b10 comp rx=0 tx=0).stop 2023-10-08T16:12:08.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 msgr2=0x7f4d5809add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:08.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f4d50011000 tx=0x7f4d50015000 comp rx=0 tx=0).stop 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 shutdown_connections 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d44067320 0x7f4d440697d0 secure :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4d480042e0 tx=0x7f4d48001b10 comp rx=0 tx=0).stop 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d5809a930 0x7f4d5809add0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d58156e90 0x7f4d580cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 --2- 172.21.15.170:0/3691052851 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d5809c9d0 0x7f4d58169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 >> 172.21.15.170:0/3691052851 conn(0x7f4d580999f0 msgr2=0x7f4d580c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 shutdown_connections 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:08.876+0000 7f4d5dd5f700 1 -- 172.21.15.170:0/3691052851 wait complete. 2023-10-08T16:12:08.950 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs.volume_0000000000793388.data' removed 2023-10-08T16:12:08.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:12:09.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.071+0000 7f2fa64d0700 1 Processor -- start 2023-10-08T16:12:09.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.071+0000 7f2fa64d0700 1 -- start start 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa64d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 0x7f2fa0068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa64d0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2fa00688d0 0x7f2fa00c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa64d0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa00c7de0 con 0x7f2fa0140350 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa64d0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa0068d50 con 0x7f2fa00688d0 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa64d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa0140c10 con 0x7f2fa0140720 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2f9ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2fa0140350 0x7f2fa0069c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51786/0 (socket says 172.21.15.170:51786) 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2f9ffff700 1 -- 172.21.15.170:0/2535112319 learned_addr learned my addr 172.21.15.170:0/2535112319 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:09.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.072+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/2535112319 >> v2:172.21.15.170:3300/0 conn(0x7f2fa00688d0 0x7f2fa00c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9f7fe700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 0x7f2fa0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9effd700 1 -- 172.21.15.170:0/2535112319 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243438537 0 0) 0x7f2fa00c7de0 con 0x7f2fa0140350 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9effd700 1 -- 172.21.15.170:0/2535112319 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f88003570 con 0x7f2fa0140350 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/2535112319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 msgr2=0x7f2fa0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 0x7f2fa0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/2535112319 >> v1:172.21.15.170:6789/0 conn(0x7f2fa0140350 legacy=0x7f2fa0069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/2535112319 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa0141490 con 0x7f2fa00688d0 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9f7fe700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 0x7f2fa0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa00688d0 0x7f2fa00c8270 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009e60 tx=0x7f2f98009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5ea8f983d5d98e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9effd700 1 -- 172.21.15.170:0/2535112319 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800e070 con 0x7f2fa00688d0 2023-10-08T16:12:09.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9effd700 1 -- 172.21.15.170:0/2535112319 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9800bb60 con 0x7f2fa00688d0 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.073+0000 7f2f9effd700 1 -- 172.21.15.170:0/2535112319 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98012430 con 0x7f2fa00688d0 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 -- 172.21.15.170:0/2535112319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa00688d0 msgr2=0x7f2fa00c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa00688d0 0x7f2fa00c8270 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009e60 tx=0x7f2f98009780 comp rx=0 tx=0).stop 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 -- 172.21.15.170:0/2535112319 shutdown_connections 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa00688d0 0x7f2fa00c8270 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/2535112319 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa0140720 0x7f2fa0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 -- 172.21.15.170:0/2535112319 >> 172.21.15.170:0/2535112319 conn(0x7f2fa00cd670 msgr2=0x7f2fa00cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 -- 172.21.15.170:0/2535112319 shutdown_connections 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.074+0000 7f2fa64d0700 1 -- 172.21.15.170:0/2535112319 wait complete. 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.075+0000 7f2fa64d0700 1 Processor -- start 2023-10-08T16:12:09.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.075+0000 7f2fa64d0700 1 -- start start 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa00688d0 0x7f2fa00b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa0140350 0x7f2fa0157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c55a0 con 0x7f2fa0140720 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c5880 con 0x7f2fa0140350 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa64d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa00c5b60 con 0x7f2fa00688d0 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49680/0 (socket says 172.21.15.170:49680) 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/91353171 learned_addr learned my addr 172.21.15.170:0/91353171 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2f9ffff700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa00688d0 0x7f2fa00b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2f9f7fe700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa0140350 0x7f2fa0157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa00688d0 msgr2=0x7f2fa00b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa00688d0 0x7f2fa00b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa0140350 msgr2=0x7f2fa0157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa0140350 0x7f2fa0157100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 -- 172.21.15.170:0/91353171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f98009a20 con 0x7f2fa0140720 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2fa4a6d700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006c6a0 tx=0x7f2f98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.076+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98027470 con 0x7f2fa0140720 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.077+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f980275d0 con 0x7f2fa0140720 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.077+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa00c5e40 con 0x7f2fa0140720 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.077+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98027990 con 0x7f2fa0140720 2023-10-08T16:12:09.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.077+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa00b2c90 con 0x7f2fa0140720 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.077+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa004c3c0 con 0x7f2fa0140720 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.078+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f98024070 con 0x7f2fa0140720 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.078+0000 7f2f87fff700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 0x7f2f8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.078+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2f98087260 con 0x7f2fa0140720 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.078+0000 7f2f9ffff700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 0x7f2f8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.079+0000 7f2f9ffff700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 0x7f2f8c0697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001360 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.082+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f980541f0 con 0x7f2fa0140720 2023-10-08T16:12:09.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.269+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2fa0140ed0 con 0x7f2fa0140720 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.273+0000 7f2f87fff700 1 -- 172.21.15.170:0/91353171 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2f98057810 con 0x7f2fa0140720 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 msgr2=0x7f2f8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 0x7f2f8c0697d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2f90001360 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 msgr2=0x7f2fa013cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f2fa006c6a0 tx=0x7f2f98019000 comp rx=0 tx=0).stop 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 shutdown_connections 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f8c067320 0x7f2f8c0697d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2fa0140720 0x7f2fa013cdc0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2fa00688d0 0x7f2fa00b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 --2- 172.21.15.170:0/91353171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2fa0140350 0x7f2fa0157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 >> 172.21.15.170:0/91353171 conn(0x7f2fa00cd670 msgr2=0x7f2fa00c3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 shutdown_connections 2023-10-08T16:12:09.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.275+0000 7f2fa64d0700 1 -- 172.21.15.170:0/91353171 wait complete. 2023-10-08T16:12:09.312 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename' 2023-10-08T16:12:09.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.426+0000 7f71e8915700 1 Processor -- start 2023-10-08T16:12:09.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.426+0000 7f71e8915700 1 -- start start 2023-10-08T16:12:09.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e8915700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f71e41597f0 0x7f71e40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e8915700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 0x7f71e415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e8915700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e415db70 con 0x7f71e4159420 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e8915700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b11d0 con 0x7f71e41597f0 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e8915700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40b14b0 con 0x7f71e415d230 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f71e41597f0 0x7f71e40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f71e41597f0 0x7f71e40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49684/0 (socket says 172.21.15.170:49684) 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e2ffd700 1 -- 172.21.15.170:0/2647029911 learned_addr learned my addr 172.21.15.170:0/2647029911 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e3fff700 1 --2- 172.21.15.170:0/2647029911 >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 0x7f71e415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.427+0000 7f71e2ffd700 1 -- 172.21.15.170:0/2647029911 >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 msgr2=0x7f71e415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2647029911 >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 0x7f71e415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e2ffd700 1 -- 172.21.15.170:0/2647029911 >> v1:172.21.15.191:6789/0 conn(0x7f71e4159420 legacy=0x7f71e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e2ffd700 1 -- 172.21.15.170:0/2647029911 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40b1790 con 0x7f71e41597f0 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e3fff700 1 --2- 172.21.15.170:0/2647029911 >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 0x7f71e415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2647029911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e41597f0 0x7f71e40b2270 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f71e415d170 tx=0x7f71d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a5dba38f5770ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e27fc700 1 -- 172.21.15.170:0/2647029911 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800b820 con 0x7f71e41597f0 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e27fc700 1 -- 172.21.15.170:0/2647029911 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d800b980 con 0x7f71e41597f0 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e27fc700 1 -- 172.21.15.170:0/2647029911 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800bd20 con 0x7f71e41597f0 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 -- 172.21.15.170:0/2647029911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e41597f0 msgr2=0x7f71e40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 --2- 172.21.15.170:0/2647029911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e41597f0 0x7f71e40b2270 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f71e415d170 tx=0x7f71d8009250 comp rx=0 tx=0).stop 2023-10-08T16:12:09.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 -- 172.21.15.170:0/2647029911 shutdown_connections 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 --2- 172.21.15.170:0/2647029911 >> v2:172.21.15.191:3300/0 conn(0x7f71e415d230 0x7f71e415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 --2- 172.21.15.170:0/2647029911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e41597f0 0x7f71e40b2270 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 -- 172.21.15.170:0/2647029911 >> 172.21.15.170:0/2647029911 conn(0x7f71e40d0660 msgr2=0x7f71e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 -- 172.21.15.170:0/2647029911 shutdown_connections 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.428+0000 7f71e8915700 1 -- 172.21.15.170:0/2647029911 wait complete. 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 Processor -- start 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 -- start start 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 0x7f71e40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 0x7f71e40ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40cb940 con 0x7f71e4159420 2023-10-08T16:12:09.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40cb4b0 con 0x7f71e415d230 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e8915700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e415c350 con 0x7f71e41597f0 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.430+0000 7f71e37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49698/0 (socket says 172.21.15.170:49698) 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 -- 172.21.15.170:0/2587700416 learned_addr learned my addr 172.21.15.170:0/2587700416 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e3fff700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 0x7f71e40ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 0x7f71e40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 -- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 msgr2=0x7f71e40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 0x7f71e40c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 -- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 msgr2=0x7f71e40ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 0x7f71e40ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 -- 172.21.15.170:0/2587700416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80094f0 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 0x7f71e40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e3fff700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 0x7f71e40ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e37fe700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c5d0 tx=0x7f71d4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011590 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4011710 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e40ca8a0 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011ad0 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.431+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e40cae60 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.432+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e4088400 con 0x7f71e4159420 2023-10-08T16:12:09.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.432+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f71d401c460 con 0x7f71e4159420 2023-10-08T16:12:09.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.433+0000 7f71cb7fe700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:09.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.433+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f71d40846e0 con 0x7f71e4159420 2023-10-08T16:12:09.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.433+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:09.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.433+0000 7f71e2ffd700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 0x7f71d00697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f71e415d170 tx=0x7f71d8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:09.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.437+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71d4051670 con 0x7f71e4159420 2023-10-08T16:12:09.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.695+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]} v 0) v1 -- 0x7f71e404c3c0 con 0x7f71e4159420 2023-10-08T16:12:09.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.871+0000 7f71cb7fe700 1 -- 172.21.15.170:0/2587700416 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename"]}]=0 v376) v1 ==== 135+0+0 (secure 0 0 0) 0x7f71d4054c90 con 0x7f71e4159420 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 msgr2=0x7f71d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 0x7f71d00697d0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f71e415d170 tx=0x7f71d8016000 comp rx=0 tx=0).stop 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 msgr2=0x7f71e4154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f71d400c5d0 tx=0x7f71d4007f60 comp rx=0 tx=0).stop 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 shutdown_connections 2023-10-08T16:12:09.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f71e415d230 0x7f71e40ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f71d0067320 0x7f71d00697d0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71e4159420 0x7f71e4154360 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 --2- 172.21.15.170:0/2587700416 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f71e41597f0 0x7f71e40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 >> 172.21.15.170:0/2587700416 conn(0x7f71e40d0660 msgr2=0x7f71e4155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 shutdown_connections 2023-10-08T16:12:09.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:09.873+0000 7f71e8915700 1 -- 172.21.15.170:0/2587700416 wait complete. 2023-10-08T16:12:09.931 INFO:tasks.cephfs_test_runner:That volume, its file system and pools, can be renamed. ... ok 2023-10-08T16:12:09.931 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) That renaming volume fails without --yes-i-really-mean-it flag. 2023-10-08T16:12:09.932 INFO:tasks.cephfs_test_runner:test_volume_rename_fails_without_confirmation_flag (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:12:09.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-08T16:12:10.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.048+0000 7fb102d42700 1 Processor -- start 2023-10-08T16:12:10.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.048+0000 7fb102d42700 1 -- start start 2023-10-08T16:12:10.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb102d42700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb0fc159c60 0x7fb0fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb102d42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0fc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb102d42700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc15a160 con 0x7fb0fc159890 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb102d42700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0fc15df50 con 0x7fb0fc159c60 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb102d42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0d13a0 con 0x7fb0fc15db10 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb100ade700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb0fc159890 0x7fb0fc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51798/0 (socket says 172.21.15.170:51798) 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb100ade700 1 -- 172.21.15.170:0/1495261667 learned_addr learned my addr 172.21.15.170:0/1495261667 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb1012df700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0fc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> v2:172.21.15.191:3300/0 conn(0x7fb0fc159c60 0x7fb0fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648556827 0 0) 0x7fb0fc15a160 con 0x7fb0fc159890 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.049+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ec003570 con 0x7fb0fc159890 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409776012 0 0) 0x7fb0ec003570 con 0x7fb0fc159890 2023-10-08T16:12:10.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 msgr2=0x7fb0fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0fc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 >> v2:172.21.15.191:3300/0 conn(0x7fb0fc159c60 msgr2=0x7fb0fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/1495261667 >> v2:172.21.15.191:3300/0 conn(0x7fb0fc159c60 0x7fb0fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc15e140 con 0x7fb0fc159890 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb0f0001470 con 0x7fb0fc159890 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> v2:172.21.15.191:3300/0 conn(0x7fb0fc159c60 0x7fb0fc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 >> v1:172.21.15.170:6789/0 conn(0x7fb0fc159890 legacy=0x7fb0fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec003a60 0x7fb0ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0ec004810 0x7fb0ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.096 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0d13a0 con 0x7fb0ec004810 2023-10-08T16:12:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00ba60 con 0x7fb0ec003a60 2023-10-08T16:12:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00bba0 con 0x7fb0fc15db10 2023-10-08T16:12:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0ec004810 0x7fb0ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb100ade700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec003a60 0x7fb0ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb1012df700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 -- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 msgr2=0x7fb0ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc15db10 0x7fb0ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 -- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec003a60 msgr2=0x7fb0ec003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec003a60 0x7fb0ec003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 -- 172.21.15.170:0/1495261667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec003570 con 0x7fb0ec004810 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0ec004810 0x7fb0ec004c80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e20 tx=0x7fb0e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40472d3680b73dc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800e920 con 0x7fb0ec004810 2023-10-08T16:12:10.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800ea80 con 0x7fb0ec004810 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.050+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/1495261667 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800ee20 con 0x7fb0ec004810 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 -- 172.21.15.170:0/1495261667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0ec004810 msgr2=0x7fb0ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 --2- 172.21.15.170:0/1495261667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0ec004810 0x7fb0ec004c80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e20 tx=0x7fb0e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb1012df700 1 -- 172.21.15.170:0/1495261667 reap_dead start 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 -- 172.21.15.170:0/1495261667 shutdown_connections 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 -- 172.21.15.170:0/1495261667 >> 172.21.15.170:0/1495261667 conn(0x7fb0fc068e20 msgr2=0x7fb0fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 -- 172.21.15.170:0/1495261667 shutdown_connections 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.051+0000 7fb102d42700 1 -- 172.21.15.170:0/1495261667 wait complete. 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.052+0000 7fb102d42700 1 Processor -- start 2023-10-08T16:12:10.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.052+0000 7fb102d42700 1 -- start start 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 0x7fb0fc0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc16d810 0x7fb0fc14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc16d610 con 0x7fb0fc16cc60 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0b3590 con 0x7fb0ec004810 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb102d42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0b3870 con 0x7fb0fc16d810 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49712/0 (socket says 172.21.15.170:49712) 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 -- 172.21.15.170:0/270122415 learned_addr learned my addr 172.21.15.170:0/270122415 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb100ade700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 0x7fb0fc0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb0fbfff700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc16d810 0x7fb0fc14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 -- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc16d810 msgr2=0x7fb0fc14d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc16d810 0x7fb0fc14d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 -- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 msgr2=0x7fb0fc0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 0x7fb0fc0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 -- 172.21.15.170:0/270122415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0ce360 con 0x7fb0fc16cc60 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb100ade700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 0x7fb0fc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.053+0000 7fb1012df700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0d03b0 tx=0x7fb0f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400e070 con 0x7fb0fc16cc60 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4011d00 con 0x7fb0fc16cc60 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc0ce640 con 0x7fb0fc16cc60 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40166c0 con 0x7fb0fc16cc60 2023-10-08T16:12:10.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc0ce050 con 0x7fb0fc16cc60 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.054+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc04c3c0 con 0x7fb0fc16cc60 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.055+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0f4016ab0 con 0x7fb0fc16cc60 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.055+0000 7fb0f8ff9700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 0x7fb0e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.055+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb0f4085530 con 0x7fb0fc16cc60 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.055+0000 7fb100ade700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 0x7fb0e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.056+0000 7fb100ade700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 0x7fb0e00697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007f90 tx=0x7fb0f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:10.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.059+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0f40523b0 con 0x7fb0fc16cc60 2023-10-08T16:12:10.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.317+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7fb0fc06a5b0 con 0x7fb0fc16cc60 2023-10-08T16:12:10.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.875+0000 7fb0f8ff9700 1 -- 172.21.15.170:0/270122415 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v377) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb0f4085030 con 0x7fb0fc16cc60 2023-10-08T16:12:11.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 msgr2=0x7fb0e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 0x7fb0e00697d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007f90 tx=0x7fb0f0009040 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 msgr2=0x7fb0fc16d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0d03b0 tx=0x7fb0f400b040 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 shutdown_connections 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb0e0067320 0x7fb0e00697d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc16cc60 0x7fb0fc16d0d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0ec004810 0x7fb0fc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 --2- 172.21.15.170:0/270122415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc16d810 0x7fb0fc14d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 >> 172.21.15.170:0/270122415 conn(0x7fb0fc068e20 msgr2=0x7fb0fc0cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:11.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 shutdown_connections 2023-10-08T16:12:11.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:10.878+0000 7fb102d42700 1 -- 172.21.15.170:0/270122415 wait complete. 2023-10-08T16:12:11.647 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:12:11.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.776+0000 7fba87bc1700 1 Processor -- start 2023-10-08T16:12:12.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 -- start start 2023-10-08T16:12:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fba80155eb0 0x7fba8006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba800ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8006eb90 con 0x7fba800c70b0 2023-10-08T16:12:12.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8006ee70 con 0x7fba80155eb0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.777+0000 7fba87bc1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006f150 con 0x7fba80069e80 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8515c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fba800c70b0 0x7fba80069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51802/0 (socket says 172.21.15.170:51802) 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8515c700 1 -- 172.21.15.170:0/1063045225 learned_addr learned my addr 172.21.15.170:0/1063045225 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8595d700 1 --2- 172.21.15.170:0/1063045225 >> v2:172.21.15.191:3300/0 conn(0x7fba80155eb0 0x7fba8006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8615e700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba800ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695905092 0 0) 0x7fba8006eb90 con 0x7fba800c70b0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba6c003570 con 0x7fba800c70b0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549013196 0 0) 0x7fba6c003570 con 0x7fba800c70b0 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 msgr2=0x7fba800ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba800ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 >> v2:172.21.15.191:3300/0 conn(0x7fba80155eb0 msgr2=0x7fba8006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 --2- 172.21.15.170:0/1063045225 >> v2:172.21.15.191:3300/0 conn(0x7fba80155eb0 0x7fba8006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8006f430 con 0x7fba800c70b0 2023-10-08T16:12:12.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fba70001470 con 0x7fba800c70b0 2023-10-08T16:12:12.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 >> v1:172.21.15.170:6789/0 conn(0x7fba800c70b0 legacy=0x7fba80069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba6c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba6c003fe0 0x7fba6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba6c004d50 0x7fba6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006ee70 con 0x7fba6c003fe0 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8006f150 con 0x7fba6c004d50 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00ba90 con 0x7fba80069e80 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8515c700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba6c003fe0 0x7fba6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8595d700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba6c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.778+0000 7fba8615e700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba6c004d50 0x7fba6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 -- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 msgr2=0x7fba6c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba80069e80 0x7fba6c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 -- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba6c004d50 msgr2=0x7fba6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba6c004d50 0x7fba6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 -- 172.21.15.170:0/1063045225 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c003570 con 0x7fba6c003fe0 2023-10-08T16:12:12.275 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8515c700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba6c003fe0 0x7fba6c004810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c3c0 tx=0x7fba70001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea3b81fa28bfa55 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000cc30 con 0x7fba6c003fe0 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7000cdd0 con 0x7fba6c003fe0 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8495b700 1 -- 172.21.15.170:0/1063045225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70005b70 con 0x7fba6c003fe0 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba87bc1700 1 -- 172.21.15.170:0/1063045225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba6c003fe0 msgr2=0x7fba6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba87bc1700 1 --2- 172.21.15.170:0/1063045225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba6c003fe0 0x7fba6c004810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c3c0 tx=0x7fba70001e90 comp rx=0 tx=0).stop 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba8615e700 1 -- 172.21.15.170:0/1063045225 reap_dead start 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba87bc1700 1 -- 172.21.15.170:0/1063045225 shutdown_connections 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.779+0000 7fba87bc1700 1 -- 172.21.15.170:0/1063045225 >> 172.21.15.170:0/1063045225 conn(0x7fba80068e10 msgr2=0x7fba800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.780+0000 7fba87bc1700 1 -- 172.21.15.170:0/1063045225 shutdown_connections 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.780+0000 7fba87bc1700 1 -- 172.21.15.170:0/1063045225 wait complete. 2023-10-08T16:12:12.276 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 Processor -- start 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 -- start start 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6c003fe0 0x7fba80155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba80159690 0x7fba80159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800ae6e0 con 0x7fba800add30 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba801693a0 con 0x7fba80159690 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba87bc1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba801556c0 con 0x7fba6c003fe0 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.781+0000 7fba8615e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49734/0 (socket says 172.21.15.170:49734) 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 -- 172.21.15.170:0/2502982571 learned_addr learned my addr 172.21.15.170:0/2502982571 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:12.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8595d700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6c003fe0 0x7fba80155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8515c700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba80159690 0x7fba80159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 -- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6c003fe0 msgr2=0x7fba80155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6c003fe0 0x7fba80155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 -- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba80159690 msgr2=0x7fba80159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba80159690 0x7fba80159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 -- 172.21.15.170:0/2502982571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800ab340 con 0x7fba800add30 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba8615e700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fba78006f00 tx=0x7fba780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78013750 con 0x7fba800add30 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba780138b0 con 0x7fba800add30 2023-10-08T16:12:12.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8015a1e0 con 0x7fba800add30 2023-10-08T16:12:12.284 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78013c50 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.782+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba80168d50 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.783+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8015a700 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.784+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba78014440 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.784+0000 7fba75ffb700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 0x7fba640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.784+0000 7fba8595d700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 0x7fba640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.784+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fba78084d80 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.785+0000 7fba8595d700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 0x7fba640697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00a5e0 tx=0x7fba7c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.789+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba8015a700 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.973+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fba8004c430 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.976+0000 7fba75ffb700 1 -- 172.21.15.170:0/2502982571 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fba8004c430 con 0x7fba800add30 2023-10-08T16:12:12.285 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 msgr2=0x7fba640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 0x7fba640697d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00a5e0 tx=0x7fba7c00c040 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 msgr2=0x7fba800ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fba78006f00 tx=0x7fba780091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 shutdown_connections 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba64067320 0x7fba640697d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba800add30 0x7fba800ae1a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba6c003fe0 0x7fba80155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 --2- 172.21.15.170:0/2502982571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba80159690 0x7fba80159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.978+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 >> 172.21.15.170:0/2502982571 conn(0x7fba80068e10 msgr2=0x7fba800b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:12.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.979+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 shutdown_connections 2023-10-08T16:12:12.287 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:11.979+0000 7fba87bc1700 1 -- 172.21.15.170:0/2502982571 wait complete. 2023-10-08T16:12:12.287 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:12:12.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:12.324 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:12.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:12:12.526 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:12:12.526 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:12.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.641+0000 7f4a16266700 1 Processor -- start 2023-10-08T16:12:12.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.641+0000 7f4a16266700 1 -- start start 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a16266700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4a100c8830 0x7f4a10164790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a16266700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1015aa30 con 0x7f4a10153df0 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a16266700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1015a100 con 0x7f4a1006e8c0 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a16266700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1015a3e0 con 0x7f4a100c8830 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a15264700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4a10153df0 0x7f4a10063230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51814/0 (socket says 172.21.15.170:51814) 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.642+0000 7f4a15264700 1 -- 172.21.15.170:0/342145904 learned_addr learned my addr 172.21.15.170:0/342145904 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 --2- 172.21.15.170:0/342145904 >> v2:172.21.15.191:3300/0 conn(0x7f4a100c8830 0x7f4a10164790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116313661 0 0) 0x7f4a1015aa30 con 0x7f4a10153df0 2023-10-08T16:12:12.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49fc003570 con 0x7f4a10153df0 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978394354 0 0) 0x7f49fc003570 con 0x7f4a10153df0 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 >> v2:172.21.15.191:3300/0 conn(0x7f4a100c8830 msgr2=0x7f4a10164790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 --2- 172.21.15.170:0/342145904 >> v2:172.21.15.191:3300/0 conn(0x7f4a100c8830 0x7f4a10164790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 >> v1:172.21.15.191:6789/0 conn(0x7f4a1006e8c0 legacy=0x7f4a100c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1015a6c0 con 0x7f4a10153df0 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4a0c001470 con 0x7f4a10153df0 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 >> v1:172.21.15.170:6789/0 conn(0x7f4a10153df0 legacy=0x7f4a10063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc003a60 0x7f49fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49fc004810 0x7f49fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 0x7f49fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1015a3e0 con 0x7f49fc0051c0 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc00c160 con 0x7f49fc004810 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc00c320 con 0x7f49fc003a60 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 0x7f49fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a14a63700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49fc004810 0x7f49fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15264700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc003a60 0x7f49fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 -- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc003a60 msgr2=0x7f49fc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc003a60 0x7f49fc003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 -- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49fc004810 msgr2=0x7f49fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:12.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49fc004810 0x7f49fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 -- 172.21.15.170:0/342145904 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49fc003570 con 0x7f49fc0051c0 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15264700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc003a60 0x7f49fc003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a14a63700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f49fc004810 0x7f49fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.643+0000 7f4a15a65700 1 -- 172.21.15.170:0/342145904 reap_dead start 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a15a65700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 0x7f49fc00ba60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4a100c8c30 tx=0x7f4a0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4cbeaebfc3bcc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800ea90 con 0x7f49fc0051c0 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0800f040 con 0x7f49fc0051c0 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a07fff700 1 -- 172.21.15.170:0/342145904 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0800ee60 con 0x7f49fc0051c0 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 -- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 msgr2=0x7f49fc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 0x7f49fc00ba60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4a100c8c30 tx=0x7f4a0800b040 comp rx=0 tx=0).stop 2023-10-08T16:12:12.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 -- 172.21.15.170:0/342145904 shutdown_connections 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 --2- 172.21.15.170:0/342145904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49fc0051c0 0x7f49fc00ba60 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 -- 172.21.15.170:0/342145904 >> 172.21.15.170:0/342145904 conn(0x7f4a1006ddb0 msgr2=0x7f4a1006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 -- 172.21.15.170:0/342145904 shutdown_connections 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.644+0000 7f4a16266700 1 -- 172.21.15.170:0/342145904 wait complete. 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.645+0000 7f4a16266700 1 Processor -- start 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.645+0000 7f4a16266700 1 -- start start 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc0051c0 0x7f4a10169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1015e490 0x7f4a1015e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1016ad20 con 0x7f4a1016d9a0 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1015ee50 con 0x7f4a1015e490 2023-10-08T16:12:12.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a16266700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1016de10 con 0x7f49fc0051c0 2023-10-08T16:12:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a14a63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a14a63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49752/0 (socket says 172.21.15.170:49752) 2023-10-08T16:12:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a14a63700 1 -- 172.21.15.170:0/2448120992 learned_addr learned my addr 172.21.15.170:0/2448120992 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a15264700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc0051c0 0x7f4a10169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:12.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.646+0000 7f4a15a65700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1015e490 0x7f4a1015e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 -- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc0051c0 msgr2=0x7f4a10169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc0051c0 0x7f4a10169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 -- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1015e490 msgr2=0x7f4a1015e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1015e490 0x7f4a1015e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 -- 172.21.15.170:0/2448120992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1016a0b0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a14a63700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e5b0 tx=0x7f4a00009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00011460 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a00010040 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a000117d0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1016a390 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.647+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1016a950 con 0x7f4a1016d9a0 2023-10-08T16:12:13.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.648+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1016ae90 con 0x7f4a1016d9a0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.648+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4a00011bc0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.649+0000 7f4a057fa700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 0x7f49f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.649+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4a00083d00 con 0x7f4a1016d9a0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.649+0000 7f4a15264700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 0x7f49f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.649+0000 7f4a15264700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 0x7f49f40697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c000fd0 tx=0x7f4a0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.653+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a0004d5a0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.880+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a1016aae0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.881+0000 7f4a057fa700 1 -- 172.21.15.170:0/2448120992 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f4a00019de0 con 0x7f4a1016d9a0 2023-10-08T16:12:13.033 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:13.033 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":94,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/29427601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":29427601},{"type":"v1","addr":"172.21.15.191:6834","nonce":29427601}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/183668886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":183668886},{"type":"v1","addr":"172.21.15.191:6835","nonce":183668886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":94},{"gid":7512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3857128925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3857128925},{"type":"v1","addr":"172.21.15.170:6836","nonce":3857128925}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86},{"gid":7515,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2091555347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2091555347},{"type":"v1","addr":"172.21.15.170:6837","nonce":2091555347}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":86}],"filesystems":[]} 2023-10-08T16:12:13.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 msgr2=0x7f49f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 0x7f49f40697d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c000fd0 tx=0x7f4a0c009040 comp rx=0 tx=0).stop 2023-10-08T16:12:13.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 msgr2=0x7f4a10159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e5b0 tx=0x7f4a00009040 comp rx=0 tx=0).stop 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 shutdown_connections 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4a1015e490 0x7f4a1015e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49fc0051c0 0x7f4a10169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f49f4067320 0x7f49f40697d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 --2- 172.21.15.170:0/2448120992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4a1016d9a0 0x7f4a10159520 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 >> 172.21.15.170:0/2448120992 conn(0x7f4a1006ddb0 msgr2=0x7f4a1015bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 shutdown_connections 2023-10-08T16:12:13.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:12.883+0000 7f4a16266700 1 -- 172.21.15.170:0/2448120992 wait complete. 2023-10-08T16:12:13.036 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 94 2023-10-08T16:12:13.036 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:12:13.036 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:12:13.036 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:12:13.036 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:13.037 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:12:13.037 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:12:13.037 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:12:13.037 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:13.037 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:12:13.037 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:12:13.038 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:12:13.038 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:13.038 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:12:13.038 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:12:13.038 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:12:13.038 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:13.047 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:12:13.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:12:13.061 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:12:13.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:12:13.062 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:12:13.062 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:12:13.063 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:12:13.063 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:12:13.089 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:12:13.165 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:13.166 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:12:13.167 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:12:13.167 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:13.167 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:12:13.168 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:12:13.162+0000 7fdb98b23700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:12:13.168 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:12:13.169 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:12:13.169 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:13.169 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:13.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:12:13.249 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:12:13.250 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:12:13.250 INFO:tasks.ceph.mds.b.smithi191.stderr:2023-10-08T16:12:13.249+0000 7fa1f7c7e700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T16:12:13.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.285+0000 7f106d33b700 1 Processor -- start 2023-10-08T16:12:13.286 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.285+0000 7f106d33b700 1 -- start start 2023-10-08T16:12:13.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f106d33b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 0x7f10680caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.288 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f106d33b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f106815df50 0x7f106815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f106d33b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106815d7e0 con 0x7f106806c0a0 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f106d33b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106815dac0 con 0x7f106815df50 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f106d33b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f106815aa30 con 0x7f106806bcd0 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f10677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f106815df50 0x7f106815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f10677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f106815df50 0x7f106815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49794/0 (socket says 172.21.15.170:49794) 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f10677fe700 1 -- 172.21.15.170:0/3702170033 learned_addr learned my addr 172.21.15.170:0/3702170033 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f1066ffd700 1 --2- 172.21.15.170:0/3702170033 >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 0x7f10680caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f10677fe700 1 -- 172.21.15.170:0/3702170033 >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 msgr2=0x7f10680caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.286+0000 7f10677fe700 1 --2- 172.21.15.170:0/3702170033 >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 0x7f10680caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f10677fe700 1 -- 172.21.15.170:0/3702170033 >> v1:172.21.15.191:6789/0 conn(0x7f106806c0a0 legacy=0x7f106815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f10677fe700 1 -- 172.21.15.170:0/3702170033 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106815ad10 con 0x7f106815df50 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f1066ffd700 1 --2- 172.21.15.170:0/3702170033 >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 0x7f10680caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f10677fe700 1 --2- 172.21.15.170:0/3702170033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106815df50 0x7f106815e3b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f10680691a0 tx=0x7f1058009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde774c855d5f19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f1065ffb700 1 -- 172.21.15.170:0/3702170033 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105800e070 con 0x7f106815df50 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f1065ffb700 1 -- 172.21.15.170:0/3702170033 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105800ba10 con 0x7f106815df50 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f1065ffb700 1 -- 172.21.15.170:0/3702170033 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10580092e0 con 0x7f106815df50 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 -- 172.21.15.170:0/3702170033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106815df50 msgr2=0x7f106815e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 --2- 172.21.15.170:0/3702170033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106815df50 0x7f106815e3b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f10680691a0 tx=0x7f1058009840 comp rx=0 tx=0).stop 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 -- 172.21.15.170:0/3702170033 shutdown_connections 2023-10-08T16:12:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 --2- 172.21.15.170:0/3702170033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106815df50 0x7f106815e3b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 --2- 172.21.15.170:0/3702170033 >> v2:172.21.15.191:3300/0 conn(0x7f106806bcd0 0x7f10680caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 -- 172.21.15.170:0/3702170033 >> 172.21.15.170:0/3702170033 conn(0x7f10680ce600 msgr2=0x7f10680cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 -- 172.21.15.170:0/3702170033 shutdown_connections 2023-10-08T16:12:13.291 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.287+0000 7f106d33b700 1 -- 172.21.15.170:0/3702170033 wait complete. 2023-10-08T16:12:13.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 Processor -- start 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 -- start start 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f106806c0a0 0x7f106816af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106815df50 0x7f10680c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680b2da0 con 0x7f106806bcd0 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c6860 con 0x7f106815df50 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f106d33b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10680c5c50 con 0x7f106806c0a0 2023-10-08T16:12:13.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f1066ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f1066ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49796/0 (socket says 172.21.15.170:49796) 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.289+0000 7f1066ffd700 1 -- 172.21.15.170:0/2206082655 learned_addr learned my addr 172.21.15.170:0/2206082655 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f10667fc700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f106806c0a0 0x7f106816af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f10677fe700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106815df50 0x7f10680c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 -- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f106806c0a0 msgr2=0x7f106816af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f106806c0a0 0x7f106816af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 -- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106815df50 msgr2=0x7f10680c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106815df50 0x7f10680c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 -- 172.21.15.170:0/2206082655 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009ae0 con 0x7f106806bcd0 2023-10-08T16:12:13.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f1066ffd700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f106000c850 tx=0x7f1060008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011600 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060011760 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10680c5f30 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011b00 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.290+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10680c64f0 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.291+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10680706b0 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.291+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1060018350 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.292+0000 7f105effd700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 0x7f10480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.292+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1060084900 con 0x7f106806bcd0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.292+0000 7f10667fc700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 0x7f10480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:13.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.292+0000 7f10667fc700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 0x7f10480697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f10680b39c0 tx=0x7f1050009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:13.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.296+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1060051780 con 0x7f106806bcd0 2023-10-08T16:12:13.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:13.495+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f10680ca3a0 con 0x7f106806bcd0 2023-10-08T16:12:14.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:12:14.450 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:12:14.525 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.525+0000 7f105effd700 1 -- 172.21.15.170:0/2206082655 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) v1 ==== 97+0+0 (secure 0 0 0) 0x7f106001e080 con 0x7f106806bcd0 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 msgr2=0x7f10480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 0x7f10480697d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f10680b39c0 tx=0x7f1050009060 comp rx=0 tx=0).stop 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 msgr2=0x7f10680b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f106000c850 tx=0x7f1060008040 comp rx=0 tx=0).stop 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 shutdown_connections 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f106815df50 0x7f10680c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1048067320 0x7f10480697d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f106806bcd0 0x7f10680b2860 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 --2- 172.21.15.170:0/2206082655 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f106806c0a0 0x7f106816af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 >> 172.21.15.170:0/2206082655 conn(0x7f10680ce600 msgr2=0x7f10680c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:14.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 shutdown_connections 2023-10-08T16:12:14.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.528+0000 7f106d33b700 1 -- 172.21.15.170:0/2206082655 wait complete. 2023-10-08T16:12:14.605 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:12:14.605 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:12:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.758+0000 7fcdb0105700 1 Processor -- start 2023-10-08T16:12:14.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.758+0000 7fcdb0105700 1 -- start start 2023-10-08T16:12:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda815a160 con 0x7fcda8159890 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda80d13a0 con 0x7fcda815db10 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdb0105700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815e140 con 0x7fcda8159c60 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdadea1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdadea1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49804/0 (socket says 172.21.15.170:49804) 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.759+0000 7fcdadea1700 1 -- 172.21.15.170:0/3067122555 learned_addr learned my addr 172.21.15.170:0/3067122555 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdad6a0700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 -- 172.21.15.170:0/3067122555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159c60 msgr2=0x7fcda815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 -- 172.21.15.170:0/3067122555 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 msgr2=0x7fcda8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdae6a2700 1 --2- 172.21.15.170:0/3067122555 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 --2- 172.21.15.170:0/3067122555 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 -- 172.21.15.170:0/3067122555 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda815a3d0 con 0x7fcda8159890 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdadea1700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159890 0x7fcda815abd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcda8089b50 tx=0x7fcda4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab5ee3cf183d786 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdace9f700 1 -- 172.21.15.170:0/3067122555 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400b860 con 0x7fcda8159890 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdace9f700 1 -- 172.21.15.170:0/3067122555 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda400b9c0 con 0x7fcda8159890 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.760+0000 7fcdace9f700 1 -- 172.21.15.170:0/3067122555 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda400bd60 con 0x7fcda8159890 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 -- 172.21.15.170:0/3067122555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159890 msgr2=0x7fcda815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159890 0x7fcda815abd0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcda8089b50 tx=0x7fcda4009250 comp rx=0 tx=0).stop 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 -- 172.21.15.170:0/3067122555 shutdown_connections 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 --2- 172.21.15.170:0/3067122555 >> v2:172.21.15.191:3300/0 conn(0x7fcda815db10 0x7fcda8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda8159890 0x7fcda815abd0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 --2- 172.21.15.170:0/3067122555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159c60 0x7fcda815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 -- 172.21.15.170:0/3067122555 >> 172.21.15.170:0/3067122555 conn(0x7fcda8068e20 msgr2=0x7fcda80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:14.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 -- 172.21.15.170:0/3067122555 shutdown_connections 2023-10-08T16:12:14.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.761+0000 7fcdb0105700 1 -- 172.21.15.170:0/3067122555 wait complete. 2023-10-08T16:12:14.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.762+0000 7fcdb0105700 1 Processor -- start 2023-10-08T16:12:14.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.762+0000 7fcdb0105700 1 -- start start 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda8159c60 0x7fcda8153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cd5c0 con 0x7fcda815db10 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cd8a0 con 0x7fcda8159c60 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80cdb80 con 0x7fcda8159890 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49816/0 (socket says 172.21.15.170:49816) 2023-10-08T16:12:14.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 -- 172.21.15.170:0/200126730 learned_addr learned my addr 172.21.15.170:0/200126730 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdadea1700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdad6a0700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda8159c60 0x7fcda8153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 -- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 msgr2=0x7fcda80cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 -- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda8159c60 msgr2=0x7fcda8153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda8159c60 0x7fcda8153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 -- 172.21.15.170:0/200126730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40094f0 con 0x7fcda815db10 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdadea1700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdae6a2700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcda000cdb0 tx=0x7fcda00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:14.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000aa80 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000abe0 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda816cec0 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.763+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0017430 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.764+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda80cfdb0 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.764+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda816d0b0 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.765+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcda0003020 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.765+0000 7fcd9dffb700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 0x7fcd8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.765+0000 7fcdadea1700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 0x7fcd8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.765+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fcda0013070 con 0x7fcda815db10 2023-10-08T16:12:14.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.766+0000 7fcdadea1700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 0x7fcd8c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcda8089b50 tx=0x7fcda4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:14.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:14.770+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcda0051a80 con 0x7fcda815db10 2023-10-08T16:12:15.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.029+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcda804c3c0 con 0x7fcda815db10 2023-10-08T16:12:15.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.029+0000 7fcd9dffb700 1 -- 172.21.15.170:0/200126730 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcda00550a0 con 0x7fcda815db10 2023-10-08T16:12:15.063 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:15.063 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:12:15.063 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:12:15.063 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:12:15.064 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.065 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:15.066 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.067 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.068 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:12:15.069 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.070 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.071 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:12:15.072 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.073 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:15.074 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.075 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.076 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:12:15.077 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:12:15.078 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:15.079 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.079 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:15.079 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:12:15.079 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:12:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 msgr2=0x7fcd8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 0x7fcd8c0697d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcda8089b50 tx=0x7fcda4017000 comp rx=0 tx=0).stop 2023-10-08T16:12:15.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 msgr2=0x7fcda80c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fcda000cdb0 tx=0x7fcda00077d0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 shutdown_connections 2023-10-08T16:12:15.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd8c067320 0x7fcd8c0697d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcda815db10 0x7fcda80c8320 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcda8159890 0x7fcda80cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 --2- 172.21.15.170:0/200126730 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcda8159c60 0x7fcda8153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 >> 172.21.15.170:0/200126730 conn(0x7fcda8068e20 msgr2=0x7fcda80c7160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 shutdown_connections 2023-10-08T16:12:15.088 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.032+0000 7fcdb0105700 1 -- 172.21.15.170:0/200126730 wait complete. 2023-10-08T16:12:15.088 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:15.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.203+0000 7f3beffff700 1 Processor -- start 2023-10-08T16:12:15.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.203+0000 7f3beffff700 1 -- start start 2023-10-08T16:12:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3bf0159420 0x7f3bf00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00b2980 con 0x7f3bf01597f0 2023-10-08T16:12:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00b11d0 con 0x7f3bf015d230 2023-10-08T16:12:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beffff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf00b14b0 con 0x7f3bf0159420 2023-10-08T16:12:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beeffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3bf0159420 0x7f3bf00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3be67fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3bf01597f0 0x7f3bf00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35192/0 (socket says 172.21.15.170:35192) 2023-10-08T16:12:15.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beeffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3bf0159420 0x7f3bf00b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44336/0 (socket says 172.21.15.170:44336) 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3be67fc700 1 -- 172.21.15.170:0/3768071541 learned_addr learned my addr 172.21.15.170:0/3768071541 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beeffd700 1 -- 172.21.15.170:0/3768071541 >> v1:172.21.15.191:6789/0 conn(0x7f3bf015d230 legacy=0x7f3bf015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.204+0000 7f3beeffd700 1 -- 172.21.15.170:0/3768071541 >> v1:172.21.15.170:6789/0 conn(0x7f3bf01597f0 legacy=0x7f3bf00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beeffd700 1 -- 172.21.15.170:0/3768071541 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00b1790 con 0x7f3bf0159420 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3bee7fc700 1 -- 172.21.15.170:0/3768071541 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285694650 0 0) 0x7f3bf00b2980 con 0x7f3bf01597f0 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beeffd700 1 --2- 172.21.15.170:0/3768071541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf0159420 0x7f3bf00b5f40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3be00098e0 tx=0x7f3be00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128aac6bd2516fe4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3bee7fc700 1 -- 172.21.15.170:0/3768071541 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009e80 con 0x7f3bf0159420 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3bee7fc700 1 -- 172.21.15.170:0/3768071541 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000b860 con 0x7f3bf0159420 2023-10-08T16:12:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3bee7fc700 1 -- 172.21.15.170:0/3768071541 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000bbe0 con 0x7f3bf0159420 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 -- 172.21.15.170:0/3768071541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf0159420 msgr2=0x7f3bf00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 --2- 172.21.15.170:0/3768071541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf0159420 0x7f3bf00b5f40 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3be00098e0 tx=0x7f3be00091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 -- 172.21.15.170:0/3768071541 shutdown_connections 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 --2- 172.21.15.170:0/3768071541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf0159420 0x7f3bf00b5f40 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 -- 172.21.15.170:0/3768071541 >> 172.21.15.170:0/3768071541 conn(0x7f3bf00d0660 msgr2=0x7f3bf00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 -- 172.21.15.170:0/3768071541 shutdown_connections 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.205+0000 7f3beffff700 1 -- 172.21.15.170:0/3768071541 wait complete. 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 Processor -- start 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 -- start start 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bf0159420 0x7f3bf0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 0x7f3bf00ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00c7230 con 0x7f3bf01597f0 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00c6620 con 0x7f3bf015d230 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3beffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00c6900 con 0x7f3bf0159420 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3be67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3be67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44340/0 (socket says 172.21.15.170:44340) 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.207+0000 7f3be67fc700 1 -- 172.21.15.170:0/879492732 learned_addr learned my addr 172.21.15.170:0/879492732 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3beeffd700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bf0159420 0x7f3bf0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3bef7fe700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 0x7f3bf00ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 -- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bf0159420 msgr2=0x7f3bf0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bf0159420 0x7f3bf0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 -- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 msgr2=0x7f3bf00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 0x7f3bf00ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 -- 172.21.15.170:0/879492732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0009420 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3bef7fe700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 0x7f3bf00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be67fc700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c810 tx=0x7f3bd8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8005c00 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8005d60 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf00c6be0 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80072e0 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.208+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf00cadd0 con 0x7f3bf01597f0 2023-10-08T16:12:15.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.209+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf00c73e0 con 0x7f3bf01597f0 2023-10-08T16:12:15.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.209+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3bd8007690 con 0x7f3bf01597f0 2023-10-08T16:12:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.210+0000 7f3be77fe700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.210+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3bd8083d70 con 0x7f3bf01597f0 2023-10-08T16:12:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.210+0000 7f3beeffd700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.210+0000 7f3beeffd700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 0x7f3bd00697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3be00093f0 tx=0x7f3be0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.214+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bd804b5a0 con 0x7f3bf01597f0 2023-10-08T16:12:15.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.396+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf004c3c0 con 0x7f3bf01597f0 2023-10-08T16:12:15.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.397+0000 7f3be77fe700 1 -- 172.21.15.170:0/879492732 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f3bd8017020 con 0x7f3bf01597f0 2023-10-08T16:12:15.397 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:15.397 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":150,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:14.523044+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 msgr2=0x7f3bd00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 0x7f3bd00697d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3be00093f0 tx=0x7f3be0017000 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 msgr2=0x7f3bf00cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c810 tx=0x7f3bd8007db0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 shutdown_connections 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3bf015d230 0x7f3bf00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3bf0159420 0x7f3bf0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3bd0067320 0x7f3bd00697d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 --2- 172.21.15.170:0/879492732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01597f0 0x7f3bf00cc630 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.400 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 >> 172.21.15.170:0/879492732 conn(0x7f3bf00d0660 msgr2=0x7f3bf015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 shutdown_connections 2023-10-08T16:12:15.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.399+0000 7f3beffff700 1 -- 172.21.15.170:0/879492732 wait complete. 2023-10-08T16:12:15.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:12:15.520 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:12:15.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 Processor -- start 2023-10-08T16:12:15.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 -- start start 2023-10-08T16:12:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 0x7fd36015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd36015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36015a160 con 0x7fd36015db10 2023-10-08T16:12:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3600d13a0 con 0x7fd360159890 2023-10-08T16:12:15.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.637+0000 7fd3650c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36015e140 con 0x7fd360159c60 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35f59e700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fd36015db10 0x7fd360069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57274/0 (socket says 172.21.15.170:57274) 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35ed9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 0x7fd36015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35f59e700 1 -- 172.21.15.170:0/852517725 learned_addr learned my addr 172.21.15.170:0/852517725 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 --2- 172.21.15.170:0/852517725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd36015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35e59c700 1 -- 172.21.15.170:0/852517725 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584962649 0 0) 0x7fd36015a160 con 0x7fd36015db10 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35e59c700 1 -- 172.21.15.170:0/852517725 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd34c003570 con 0x7fd36015db10 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 -- 172.21.15.170:0/852517725 >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 msgr2=0x7fd36015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 --2- 172.21.15.170:0/852517725 >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 0x7fd36015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 -- 172.21.15.170:0/852517725 >> v1:172.21.15.191:6789/0 conn(0x7fd36015db10 legacy=0x7fd360069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 -- 172.21.15.170:0/852517725 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36015a3d0 con 0x7fd360159c60 2023-10-08T16:12:15.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35ed9d700 1 --2- 172.21.15.170:0/852517725 >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 0x7fd36015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35659c700 1 --2- 172.21.15.170:0/852517725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd36015d5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098e0 tx=0x7fd3480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd4a1c2da3ef029 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35e59c700 1 -- 172.21.15.170:0/852517725 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348011630 con 0x7fd360159c60 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35e59c700 1 -- 172.21.15.170:0/852517725 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348011790 con 0x7fd360159c60 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.638+0000 7fd35e59c700 1 -- 172.21.15.170:0/852517725 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348011b30 con 0x7fd360159c60 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 -- 172.21.15.170:0/852517725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 msgr2=0x7fd36015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 --2- 172.21.15.170:0/852517725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd36015d5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3480098e0 tx=0x7fd3480091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 -- 172.21.15.170:0/852517725 shutdown_connections 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 --2- 172.21.15.170:0/852517725 >> v2:172.21.15.191:3300/0 conn(0x7fd360159890 0x7fd36015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 --2- 172.21.15.170:0/852517725 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd36015d5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 -- 172.21.15.170:0/852517725 >> 172.21.15.170:0/852517725 conn(0x7fd360068e20 msgr2=0x7fd3600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 -- 172.21.15.170:0/852517725 shutdown_connections 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.639+0000 7fd3650c0700 1 -- 172.21.15.170:0/852517725 wait complete. 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.640+0000 7fd3650c0700 1 Processor -- start 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.640+0000 7fd3650c0700 1 -- start start 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd360159890 0x7fd36016a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd360150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd360154d10 con 0x7fd36015db10 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd360154ab0 con 0x7fd360159890 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd3650c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36015b930 con 0x7fd360159c60 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44346/0 (socket says 172.21.15.170:44346) 2023-10-08T16:12:15.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 -- 172.21.15.170:0/2047515599 learned_addr learned my addr 172.21.15.170:0/2047515599 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35ed9d700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd360159890 0x7fd36016a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35659c700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd360150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 -- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 msgr2=0x7fd360150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd360150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 -- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd360159890 msgr2=0x7fd36016a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd360159890 0x7fd36016a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.641+0000 7fd35f59e700 1 -- 172.21.15.170:0/2047515599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350009420 con 0x7fd36015db10 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd35f59e700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd358001460 tx=0x7fd358001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800a950 con 0x7fd36015db10 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800aab0 con 0x7fd36015db10 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd348009420 con 0x7fd36015db10 2023-10-08T16:12:15.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ae30 con 0x7fd36015db10 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.642+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36015bef0 con 0x7fd36015db10 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.643+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36004c3c0 con 0x7fd36015db10 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.643+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd358007300 con 0x7fd36015db10 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.644+0000 7fd3577fe700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 0x7fd3400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.644+0000 7fd35ed9d700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 0x7fd3400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.644+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd358084dc0 con 0x7fd36015db10 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.644+0000 7fd35ed9d700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 0x7fd3400697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd3500093f0 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.648+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd358051d90 con 0x7fd36015db10 2023-10-08T16:12:15.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.830+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd36006a5b0 con 0x7fd36015db10 2023-10-08T16:12:15.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.830+0000 7fd3577fe700 1 -- 172.21.15.170:0/2047515599 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3580553b0 con 0x7fd36015db10 2023-10-08T16:12:15.838 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:12:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 msgr2=0x7fd3400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 0x7fd3400697d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd3500093f0 tx=0x7fd350008040 comp rx=0 tx=0).stop 2023-10-08T16:12:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 msgr2=0x7fd360154570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fd358001460 tx=0x7fd358001a40 comp rx=0 tx=0).stop 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 shutdown_connections 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd340067320 0x7fd3400697d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd36015db10 0x7fd360154570 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.832+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd360159890 0x7fd36016a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.833+0000 7fd3650c0700 1 --2- 172.21.15.170:0/2047515599 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd360159c60 0x7fd360150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.833+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 >> 172.21.15.170:0/2047515599 conn(0x7fd360068e20 msgr2=0x7fd3600b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.833+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 shutdown_connections 2023-10-08T16:12:15.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.833+0000 7fd3650c0700 1 -- 172.21.15.170:0/2047515599 wait complete. 2023-10-08T16:12:15.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:12:15.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.959+0000 7fb41f1d2700 1 Processor -- start 2023-10-08T16:12:15.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.959+0000 7fb41f1d2700 1 -- start start 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41f1d2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb418069560 0x7fb4180699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41f1d2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb418069ee0 con 0x7fb418159890 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41f1d2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41815a160 con 0x7fb418159c60 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41f1d2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180d13a0 con 0x7fb418069560 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb418069560 0x7fb4180699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41cf6e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb418159890 0x7fb41815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35206/0 (socket says 172.21.15.170:35206) 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41cf6e700 1 -- 172.21.15.170:0/1620630968 learned_addr learned my addr 172.21.15.170:0/1620630968 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb418069560 0x7fb4180699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44352/0 (socket says 172.21.15.170:44352) 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 -- 172.21.15.170:0/1620630968 >> v1:172.21.15.191:6789/0 conn(0x7fb418159c60 legacy=0x7fb41815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 -- 172.21.15.170:0/1620630968 >> v1:172.21.15.170:6789/0 conn(0x7fb418159890 legacy=0x7fb41815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 -- 172.21.15.170:0/1620630968 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41815e140 con 0x7fb418069560 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb4177fe700 1 -- 172.21.15.170:0/1620630968 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067694726 0 0) 0x7fb418069ee0 con 0x7fb418159890 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.960+0000 7fb41d76f700 1 --2- 172.21.15.170:0/1620630968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418069560 0x7fb4180699a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb4000098e0 tx=0x7fb4000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f81acddd708333 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb4177fe700 1 -- 172.21.15.170:0/1620630968 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400009e80 con 0x7fb418069560 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb4177fe700 1 -- 172.21.15.170:0/1620630968 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40000b860 con 0x7fb418069560 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb4177fe700 1 -- 172.21.15.170:0/1620630968 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40000bbe0 con 0x7fb418069560 2023-10-08T16:12:15.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 -- 172.21.15.170:0/1620630968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418069560 msgr2=0x7fb4180699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/1620630968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418069560 0x7fb4180699a0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb4000098e0 tx=0x7fb4000091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 -- 172.21.15.170:0/1620630968 shutdown_connections 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/1620630968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418069560 0x7fb4180699a0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 -- 172.21.15.170:0/1620630968 >> 172.21.15.170:0/1620630968 conn(0x7fb418068e20 msgr2=0x7fb4180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 -- 172.21.15.170:0/1620630968 shutdown_connections 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.961+0000 7fb41f1d2700 1 -- 172.21.15.170:0/1620630968 wait complete. 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.962+0000 7fb41f1d2700 1 Processor -- start 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.962+0000 7fb41f1d2700 1 -- start start 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb418069560 0x7fb418149310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb418159890 0x7fb4180c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.967 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cde50 con 0x7fb418159c60 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb418149850 con 0x7fb418069560 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41f1d2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cd240 con 0x7fb418159890 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44354/0 (socket says 172.21.15.170:44354) 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 -- 172.21.15.170:0/3008213887 learned_addr learned my addr 172.21.15.170:0/3008213887 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41cf6e700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb418069560 0x7fb418149310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb417fff700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb418159890 0x7fb4180c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 -- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb418159890 msgr2=0x7fb4180c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb418159890 0x7fb4180c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 -- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb418069560 msgr2=0x7fb418149310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:15.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb418069560 0x7fb418149310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.963+0000 7fb41d76f700 1 -- 172.21.15.170:0/3008213887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009420 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb41d76f700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb41806a070 tx=0x7fb400017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40001f070 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40000b9d0 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180cd520 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400025650 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.964+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4180cdae0 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.965+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4180b2e40 con 0x7fb418159c60 2023-10-08T16:12:15.969 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.965+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb400023070 con 0x7fb418159c60 2023-10-08T16:12:15.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.965+0000 7fb414ff9700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 0x7fb3f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:15.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.966+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb400086150 con 0x7fb418159c60 2023-10-08T16:12:15.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.966+0000 7fb41cf6e700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 0x7fb3f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.966+0000 7fb41cf6e700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 0x7fb3f80697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb4100011e0 tx=0x7fb410009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:16.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:15.970+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb400053130 con 0x7fb418159c60 2023-10-08T16:12:16.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.155+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb41804c3c0 con 0x7fb418159c60 2023-10-08T16:12:16.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.156+0000 7fb414ff9700 1 -- 172.21.15.170:0/3008213887 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb400056750 con 0x7fb418159c60 2023-10-08T16:12:16.156 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:16.156 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:16.156 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:16.157 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:12:16.157 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:12:16.157 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:16.157 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 msgr2=0x7fb3f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 0x7fb3f80697d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb4100011e0 tx=0x7fb410009040 comp rx=0 tx=0).stop 2023-10-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 msgr2=0x7fb4180cadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fb41806a070 tx=0x7fb400017000 comp rx=0 tx=0).stop 2023-10-08T16:12:16.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 shutdown_connections 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb3f8067320 0x7fb3f80697d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb418159c60 0x7fb4180cadf0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb418069560 0x7fb418149310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.157+0000 7fb41f1d2700 1 --2- 172.21.15.170:0/3008213887 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb418159890 0x7fb4180c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.158+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 >> 172.21.15.170:0/3008213887 conn(0x7fb418068e20 msgr2=0x7fb418155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.158+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 shutdown_connections 2023-10-08T16:12:16.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.158+0000 7fb41f1d2700 1 -- 172.21.15.170:0/3008213887 wait complete. 2023-10-08T16:12:16.170 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:12:16.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:12:16.289 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.288+0000 7f4d813a2700 1 Processor -- start 2023-10-08T16:12:16.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.288+0000 7f4d813a2700 1 -- start start 2023-10-08T16:12:16.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d813a2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4d7c1597b0 0x7f4d7c0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d813a2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4d7c06a9c0 0x7f4d7c163650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d813a2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c069d80 con 0x7f4d7c159b80 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d813a2700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c068390 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d813a2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0ac530 con 0x7f4d7c1597b0 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4d7c06a9c0 0x7f4d7c163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d7b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4d7c06a9c0 0x7f4d7c163650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44356/0 (socket says 172.21.15.170:44356) 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/1957134094 learned_addr learned my addr 172.21.15.170:0/1957134094 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.289+0000 7f4d7affd700 1 --2- 172.21.15.170:0/1957134094 >> v2:172.21.15.191:3300/0 conn(0x7f4d7c1597b0 0x7f4d7c0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/1957134094 >> v2:172.21.15.191:3300/0 conn(0x7f4d7c1597b0 msgr2=0x7f4d7c0af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:16.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7b7fe700 1 --2- 172.21.15.170:0/1957134094 >> v2:172.21.15.191:3300/0 conn(0x7f4d7c1597b0 0x7f4d7c0af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/1957134094 >> v1:172.21.15.191:6789/0 conn(0x7f4d7c159b80 legacy=0x7f4d7c06a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/1957134094 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0aa680 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7b7fe700 1 --2- 172.21.15.170:0/1957134094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c163650 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009f20 tx=0x7f4d74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49799e8979498236 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7a7fc700 1 -- 172.21.15.170:0/1957134094 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400e070 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7a7fc700 1 -- 172.21.15.170:0/1957134094 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400b990 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.290+0000 7f4d7a7fc700 1 -- 172.21.15.170:0/1957134094 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740092e0 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 -- 172.21.15.170:0/1957134094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c06a9c0 msgr2=0x7f4d7c163650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 --2- 172.21.15.170:0/1957134094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c163650 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009f20 tx=0x7f4d74009840 comp rx=0 tx=0).stop 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 -- 172.21.15.170:0/1957134094 shutdown_connections 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 --2- 172.21.15.170:0/1957134094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c163650 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 --2- 172.21.15.170:0/1957134094 >> v2:172.21.15.191:3300/0 conn(0x7f4d7c1597b0 0x7f4d7c0af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 -- 172.21.15.170:0/1957134094 >> 172.21.15.170:0/1957134094 conn(0x7f4d7c0cd880 msgr2=0x7f4d7c0ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 -- 172.21.15.170:0/1957134094 shutdown_connections 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.291+0000 7f4d813a2700 1 -- 172.21.15.170:0/1957134094 wait complete. 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.292+0000 7f4d813a2700 1 Processor -- start 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.292+0000 7f4d813a2700 1 -- start start 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 0x7f4d7c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15b3e0 con 0x7f4d7c159b80 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15b6c0 con 0x7f4d7c06a9c0 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15b9a0 con 0x7f4d7c1597b0 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44358/0 (socket says 172.21.15.170:44358) 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/830039360 learned_addr learned my addr 172.21.15.170:0/830039360 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:16.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d727fc700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 0x7f4d7c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 msgr2=0x7f4d7c0b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7affd700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 0x7f4d7c0b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 msgr2=0x7f4d7c146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 -- 172.21.15.170:0/830039360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d74009ae0 con 0x7f4d7c159b80 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d727fc700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 0x7f4d7c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7affd700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d7b7fe700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009880 tx=0x7f4d74018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:16.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74008d00 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74008e60 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740266b0 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c15bc80 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.293+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c15c460 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.294+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c15c2d0 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.295+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4d74003040 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.295+0000 7f4d737fe700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 0x7f4d5c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.295+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f4d740869b0 con 0x7f4d7c159b80 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.295+0000 7f4d7affd700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 0x7f4d5c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.296+0000 7f4d7affd700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 0x7f4d5c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006f00 tx=0x7f4d6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:16.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.299+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d74053880 con 0x7f4d7c159b80 2023-10-08T16:12:16.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.486+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4d7c0718b0 con 0x7f4d7c159b80 2023-10-08T16:12:16.999 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:16.999+0000 7f4d737fe700 1 -- 172.21.15.170:0/830039360 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4d74056ea0 con 0x7f4d7c159b80 2023-10-08T16:12:17.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.002+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 msgr2=0x7f4d5c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.002+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 0x7f4d5c0697d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006f00 tx=0x7f4d6c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.002+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 msgr2=0x7f4d7c15ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.002+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009880 tx=0x7f4d74018000 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 shutdown_connections 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4d5c067320 0x7f4d5c0697d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d7c159b80 0x7f4d7c15ce80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4d7c06a9c0 0x7f4d7c146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 --2- 172.21.15.170:0/830039360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4d7c1597b0 0x7f4d7c0b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 >> 172.21.15.170:0/830039360 conn(0x7f4d7c0cd880 msgr2=0x7f4d7c0ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:17.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.003+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 shutdown_connections 2023-10-08T16:12:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.004+0000 7f4d813a2700 1 -- 172.21.15.170:0/830039360 wait complete. 2023-10-08T16:12:17.064 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:12:17.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:12:17.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.185+0000 7f9c0e558700 1 Processor -- start 2023-10-08T16:12:17.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.185+0000 7f9c0e558700 1 -- start start 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c0e558700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9c080cca50 0x7f9c080caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c0e558700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0815df40 0x7f9c0815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c0e558700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c08157c50 con 0x7f9c080cce20 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c0e558700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0815d7d0 con 0x7f9c080cca50 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c0e558700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0815aa20 con 0x7f9c0815df40 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c07fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9c080cca50 0x7f9c080caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c077fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9c080cce20 0x7f9c0815a1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35210/0 (socket says 172.21.15.170:35210) 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.186+0000 7f9c077fe700 1 -- 172.21.15.170:0/2520440304 learned_addr learned my addr 172.21.15.170:0/2520440304 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c0caf5700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0815df40 0x7f9c0815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c07fff700 1 -- 172.21.15.170:0/2520440304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0815df40 msgr2=0x7f9c0815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c07fff700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0815df40 0x7f9c0815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c07fff700 1 -- 172.21.15.170:0/2520440304 >> v1:172.21.15.170:6789/0 conn(0x7f9c080cce20 legacy=0x7f9c0815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c07fff700 1 -- 172.21.15.170:0/2520440304 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0815ad00 con 0x7f9c080cca50 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c06ffd700 1 -- 172.21.15.170:0/2520440304 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773561102 0 0) 0x7f9bfc001110 con 0x7f9c080cce20 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c07fff700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c080cca50 0x7f9c080caab0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098e0 tx=0x7f9bf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e455f81ffed5d59a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c06ffd700 1 -- 172.21.15.170:0/2520440304 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8009e80 con 0x7f9c080cca50 2023-10-08T16:12:17.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c06ffd700 1 -- 172.21.15.170:0/2520440304 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800b860 con 0x7f9c080cca50 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.187+0000 7f9c06ffd700 1 -- 172.21.15.170:0/2520440304 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800bbe0 con 0x7f9c080cca50 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 -- 172.21.15.170:0/2520440304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c080cca50 msgr2=0x7f9c080caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c080cca50 0x7f9c080caab0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80098e0 tx=0x7f9bf80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 -- 172.21.15.170:0/2520440304 shutdown_connections 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c0815df40 0x7f9c0815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 --2- 172.21.15.170:0/2520440304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c080cca50 0x7f9c080caab0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 -- 172.21.15.170:0/2520440304 >> 172.21.15.170:0/2520440304 conn(0x7f9c080ce5c0 msgr2=0x7f9c080ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 -- 172.21.15.170:0/2520440304 shutdown_connections 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.188+0000 7f9c0e558700 1 -- 172.21.15.170:0/2520440304 wait complete. 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.189+0000 7f9c0e558700 1 Processor -- start 2023-10-08T16:12:17.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.189+0000 7f9c0e558700 1 -- start start 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 0x7f9c080b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c080cce20 0x7f9c08159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080c6a30 con 0x7f9c0815df40 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080c6d10 con 0x7f9c080cca50 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c080c8440 con 0x7f9c080cce20 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44372/0 (socket says 172.21.15.170:44372) 2023-10-08T16:12:17.420 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 -- 172.21.15.170:0/3576867991 learned_addr learned my addr 172.21.15.170:0/3576867991 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c077fe700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c080cce20 0x7f9c08159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c07fff700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 0x7f9c080b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 -- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c080cce20 msgr2=0x7f9c08159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c080cce20 0x7f9c08159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 -- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 msgr2=0x7f9c080b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 0x7f9c080b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 -- 172.21.15.170:0/3576867991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf8009420 con 0x7f9c0815df40 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c07fff700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 0x7f9c080b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0caf5700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9c00007f30 tx=0x7f9c000098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000ecf0 con 0x7f9c0815df40 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0000ee50 con 0x7f9c0815df40 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c080c8720 con 0x7f9c0815df40 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00014670 con 0x7f9c0815df40 2023-10-08T16:12:17.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.190+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c08159550 con 0x7f9c0815df40 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.191+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c08159770 con 0x7f9c0815df40 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.192+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9c000149e0 con 0x7f9c0815df40 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.192+0000 7f9beffff700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 0x7f9bf40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.192+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f9c000852c0 con 0x7f9c0815df40 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.192+0000 7f9c07fff700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 0x7f9bf40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.192+0000 7f9c07fff700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 0x7f9bf40697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9c081577d0 tx=0x7f9bf8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.196+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c00055bc0 con 0x7f9c0815df40 2023-10-08T16:12:17.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:17.378+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9c080c82a0 con 0x7f9c0815df40 2023-10-08T16:12:18.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.002+0000 7f9beffff700 1 -- 172.21.15.170:0/3576867991 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9c080c82a0 con 0x7f9c0815df40 2023-10-08T16:12:18.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 msgr2=0x7f9bf40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 0x7f9bf40697d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9c081577d0 tx=0x7f9bf8017000 comp rx=0 tx=0).stop 2023-10-08T16:12:18.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 msgr2=0x7f9c08154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9c00007f30 tx=0x7f9c000098e0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 shutdown_connections 2023-10-08T16:12:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bf4067320 0x7f9bf40697d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c0815df40 0x7f9c08154180 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c080cca50 0x7f9c080b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 --2- 172.21.15.170:0/3576867991 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c080cce20 0x7f9c08159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.008+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 >> 172.21.15.170:0/3576867991 conn(0x7f9c080ce5c0 msgr2=0x7f9c08155640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.011+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 shutdown_connections 2023-10-08T16:12:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.011+0000 7f9c0e558700 1 -- 172.21.15.170:0/3576867991 wait complete. 2023-10-08T16:12:18.033 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:12:18.033 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:12:18.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.150+0000 7f2f2c598700 1 Processor -- start 2023-10-08T16:12:18.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.150+0000 7f2f2c598700 1 -- start start 2023-10-08T16:12:18.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2c598700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2f24159c60 0x7f2f2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2c598700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2f2415db10 0x7f2f24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2c598700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2415a160 con 0x7f2f24159890 2023-10-08T16:12:18.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2c598700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2415df50 con 0x7f2f24159c60 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2c598700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f240d13a0 con 0x7f2f2415db10 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f29b33700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2f24159c60 0x7f2f2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2a334700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2f24159890 0x7f2f2415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35226/0 (socket says 172.21.15.170:35226) 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f29b33700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2f24159c60 0x7f2f2415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44380/0 (socket says 172.21.15.170:44380) 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.151+0000 7f2f2a334700 1 -- 172.21.15.170:0/2077605694 learned_addr learned my addr 172.21.15.170:0/2077605694 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f2ab35700 1 --2- 172.21.15.170:0/2077605694 >> v2:172.21.15.191:3300/0 conn(0x7f2f2415db10 0x7f2f24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29b33700 1 -- 172.21.15.170:0/2077605694 >> v2:172.21.15.191:3300/0 conn(0x7f2f2415db10 msgr2=0x7f2f24069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29b33700 1 --2- 172.21.15.170:0/2077605694 >> v2:172.21.15.191:3300/0 conn(0x7f2f2415db10 0x7f2f24069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29b33700 1 -- 172.21.15.170:0/2077605694 >> v1:172.21.15.170:6789/0 conn(0x7f2f24159890 legacy=0x7f2f2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29332700 1 -- 172.21.15.170:0/2077605694 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658365645 0 0) 0x7f2f2415a160 con 0x7f2f24159890 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29b33700 1 -- 172.21.15.170:0/2077605694 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2415e140 con 0x7f2f24159c60 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29b33700 1 --2- 172.21.15.170:0/2077605694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24159c60 0x7f2f2415d5d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2f2406a890 tx=0x7f2f140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847fddb13b69fb9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29332700 1 -- 172.21.15.170:0/2077605694 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400b6b0 con 0x7f2f24159c60 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29332700 1 -- 172.21.15.170:0/2077605694 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1400b810 con 0x7f2f24159c60 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.152+0000 7f2f29332700 1 -- 172.21.15.170:0/2077605694 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400bbb0 con 0x7f2f24159c60 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 -- 172.21.15.170:0/2077605694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24159c60 msgr2=0x7f2f2415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 --2- 172.21.15.170:0/2077605694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24159c60 0x7f2f2415d5d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f2f2406a890 tx=0x7f2f140091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 -- 172.21.15.170:0/2077605694 shutdown_connections 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 --2- 172.21.15.170:0/2077605694 >> v2:172.21.15.191:3300/0 conn(0x7f2f2415db10 0x7f2f24069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 --2- 172.21.15.170:0/2077605694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24159c60 0x7f2f2415d5d0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 -- 172.21.15.170:0/2077605694 >> 172.21.15.170:0/2077605694 conn(0x7f2f24068e20 msgr2=0x7f2f240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:18.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 -- 172.21.15.170:0/2077605694 shutdown_connections 2023-10-08T16:12:18.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.153+0000 7f2f2c598700 1 -- 172.21.15.170:0/2077605694 wait complete. 2023-10-08T16:12:18.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.154+0000 7f2f2c598700 1 Processor -- start 2023-10-08T16:12:18.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.154+0000 7f2f2c598700 1 -- start start 2023-10-08T16:12:18.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 0x7f2f240b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f24159c60 0x7f2f2414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24152ac0 con 0x7f2f2415db10 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2414cc40 con 0x7f2f24159890 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240b2380 con 0x7f2f24159c60 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44392/0 (socket says 172.21.15.170:44392) 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 -- 172.21.15.170:0/1020110106 learned_addr learned my addr 172.21.15.170:0/1020110106 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2a334700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 0x7f2f240b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f29b33700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f24159c60 0x7f2f2414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 -- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f24159c60 msgr2=0x7f2f2414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f24159c60 0x7f2f2414c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 -- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 msgr2=0x7f2f240b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 0x7f2f240b1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 -- 172.21.15.170:0/1020110106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f14009420 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2a334700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 0x7f2f240b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2ab35700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00d860 tx=0x7f2f1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0125f0 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c012750 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c012b10 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f24152470 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.155+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f240b2660 con 0x7f2f2415db10 2023-10-08T16:12:18.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.156+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2404c3c0 con 0x7f2f2415db10 2023-10-08T16:12:18.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.157+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2f1c01e460 con 0x7f2f2415db10 2023-10-08T16:12:18.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.157+0000 7f2f1a7fc700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 0x7f2f080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:18.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.157+0000 7f2f2a334700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 0x7f2f080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:18.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.157+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f2f1c085270 con 0x7f2f2415db10 2023-10-08T16:12:18.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.158+0000 7f2f2a334700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 0x7f2f080697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d270 tx=0x7f2f20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:18.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.162+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f1c0559f0 con 0x7f2f2415db10 2023-10-08T16:12:18.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:18.388+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2f2406a5b0 con 0x7f2f2415db10 2023-10-08T16:12:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.024+0000 7f2f1a7fc700 1 -- 172.21.15.170:0/1020110106 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v96) v1 ==== 174+0+0 (secure 0 0 0) 0x7f2f1c055580 con 0x7f2f2415db10 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 msgr2=0x7f2f080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 0x7f2f080697d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d270 tx=0x7f2f20009040 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 msgr2=0x7f2f24158570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00d860 tx=0x7f2f1c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 shutdown_connections 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2f08067320 0x7f2f080697d0 secure :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f2000d270 tx=0x7f2f20009040 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f2415db10 0x7f2f24158570 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2f24159890 0x7f2f240b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 --2- 172.21.15.170:0/1020110106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2f24159c60 0x7f2f2414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.028+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 >> 172.21.15.170:0/1020110106 conn(0x7f2f24068e20 msgr2=0x7f2f2414d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.029+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 shutdown_connections 2023-10-08T16:12:19.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.029+0000 7f2f2c598700 1 -- 172.21.15.170:0/1020110106 wait complete. 2023-10-08T16:12:19.031 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 26 and data pool 27 2023-10-08T16:12:19.043 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:19.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.157+0000 7f090244b700 1 Processor -- start 2023-10-08T16:12:19.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.157+0000 7f090244b700 1 -- start start 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f090244b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f08fc159c60 0x7f08fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f090244b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08fc15db10 0x7f08fc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f090244b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc15a160 con 0x7f08fc159890 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f090244b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc15df50 con 0x7f08fc15db10 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f090244b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0d13a0 con 0x7f08fc159c60 2023-10-08T16:12:19.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08fc15db10 0x7f08fc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08fc15db10 0x7f08fc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44396/0 (socket says 172.21.15.170:44396) 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 -- 172.21.15.170:0/896237322 learned_addr learned my addr 172.21.15.170:0/896237322 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f08fb7fe700 1 --2- 172.21.15.170:0/896237322 >> v2:172.21.15.191:3300/0 conn(0x7f08fc159c60 0x7f08fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 -- 172.21.15.170:0/896237322 >> v2:172.21.15.191:3300/0 conn(0x7f08fc159c60 msgr2=0x7f08fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 --2- 172.21.15.170:0/896237322 >> v2:172.21.15.191:3300/0 conn(0x7f08fc159c60 0x7f08fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 -- 172.21.15.170:0/896237322 >> v1:172.21.15.191:6789/0 conn(0x7f08fc159890 legacy=0x7f08fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 -- 172.21.15.170:0/896237322 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc15e140 con 0x7f08fc15db10 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.158+0000 7f09009e8700 1 --2- 172.21.15.170:0/896237322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc15db10 0x7f08fc069560 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0b67f0 tx=0x7f08f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1d673fc0590cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f08faffd700 1 -- 172.21.15.170:0/896237322 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400e070 con 0x7f08fc15db10 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f08faffd700 1 -- 172.21.15.170:0/896237322 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400ba50 con 0x7f08fc15db10 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f08faffd700 1 -- 172.21.15.170:0/896237322 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40092e0 con 0x7f08fc15db10 2023-10-08T16:12:19.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 -- 172.21.15.170:0/896237322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc15db10 msgr2=0x7f08fc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 --2- 172.21.15.170:0/896237322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc15db10 0x7f08fc069560 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0b67f0 tx=0x7f08f4009840 comp rx=0 tx=0).stop 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 -- 172.21.15.170:0/896237322 shutdown_connections 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 --2- 172.21.15.170:0/896237322 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc15db10 0x7f08fc069560 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 --2- 172.21.15.170:0/896237322 >> v2:172.21.15.191:3300/0 conn(0x7f08fc159c60 0x7f08fc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 -- 172.21.15.170:0/896237322 >> 172.21.15.170:0/896237322 conn(0x7f08fc068e20 msgr2=0x7f08fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 -- 172.21.15.170:0/896237322 shutdown_connections 2023-10-08T16:12:19.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.159+0000 7f090244b700 1 -- 172.21.15.170:0/896237322 wait complete. 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.160+0000 7f090244b700 1 Processor -- start 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.160+0000 7f090244b700 1 -- start start 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08fc159c60 0x7f08fc0ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc15db10 0x7f08fc15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cf730 con 0x7f08fc159890 2023-10-08T16:12:19.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cd560 con 0x7f08fc15db10 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f090244b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cf170 con 0x7f08fc159c60 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f08fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f08fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44404/0 (socket says 172.21.15.170:44404) 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f08fbfff700 1 -- 172.21.15.170:0/1674926398 learned_addr learned my addr 172.21.15.170:0/1674926398 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f09009e8700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc15db10 0x7f08fc15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.161+0000 7f08fb7fe700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08fc159c60 0x7f08fc0ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 -- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08fc159c60 msgr2=0x7f08fc0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08fc159c60 0x7f08fc0ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 -- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc15db10 msgr2=0x7f08fc15c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc15db10 0x7f08fc15c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 -- 172.21.15.170:0/1674926398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009ae0 con 0x7f08fc159890 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08fbfff700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c590 tx=0x7f08ec00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec011590 con 0x7f08fc159890 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec011710 con 0x7f08fc159890 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec011ad0 con 0x7f08fc159890 2023-10-08T16:12:19.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc0cf450 con 0x7f08fc159890 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.162+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc0cd9f0 con 0x7f08fc159890 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.163+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc0cd3b0 con 0x7f08fc159890 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.164+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f08ec01b3b0 con 0x7f08fc159890 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.164+0000 7f08e3fff700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 0x7f08e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.164+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08ec084080 con 0x7f08fc159890 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.164+0000 7f08fb7fe700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 0x7f08e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.164+0000 7f08fb7fe700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 0x7f08e80697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0b5640 tx=0x7f08f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.168+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08ec050d10 con 0x7f08fc159890 2023-10-08T16:12:19.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.353+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08fc04c3c0 con 0x7f08fc159890 2023-10-08T16:12:19.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.354+0000 7f08e3fff700 1 -- 172.21.15.170:0/1674926398 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f08ec017070 con 0x7f08fc159890 2023-10-08T16:12:19.354 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:19.355 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":153,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:18.389760+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 msgr2=0x7f08e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 0x7f08e80697d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f08fc0b5640 tx=0x7f08f0008040 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 msgr2=0x7f08fc152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00c590 tx=0x7f08ec00c8a0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 shutdown_connections 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08fc15db10 0x7f08fc15c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08e8067320 0x7f08e80697d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08fc159890 0x7f08fc152560 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 --2- 172.21.15.170:0/1674926398 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08fc159c60 0x7f08fc0ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 >> 172.21.15.170:0/1674926398 conn(0x7f08fc068e20 msgr2=0x7f08fc1583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 shutdown_connections 2023-10-08T16:12:19.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.356+0000 7f090244b700 1 -- 172.21.15.170:0/1674926398 wait complete. 2023-10-08T16:12:19.369 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:19.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.483+0000 7f1282baf700 1 Processor -- start 2023-10-08T16:12:19.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.483+0000 7f1282baf700 1 -- start start 2023-10-08T16:12:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f1282baf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 0x7f127c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f1282baf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c0b2980 con 0x7f127c1597f0 2023-10-08T16:12:19.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f1282baf700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c0b11d0 con 0x7f127c15d230 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f1282baf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0b14b0 con 0x7f127c159420 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f12813ac700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f127c1597f0 0x7f127c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35232/0 (socket says 172.21.15.170:35232) 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f12813ac700 1 -- 172.21.15.170:0/1506733871 learned_addr learned my addr 172.21.15.170:0/1506733871 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.484+0000 7f1281bad700 1 --2- 172.21.15.170:0/1506733871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 0x7f127c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860425612 0 0) 0x7f127c0b2980 con 0x7f127c1597f0 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1268003570 con 0x7f127c1597f0 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150456095 0 0) 0x7f127c0b11d0 con 0x7f127c15d230 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1281bad700 1 -- 172.21.15.170:0/1506733871 >> v1:172.21.15.191:6789/0 conn(0x7f127c15d230 legacy=0x7f127c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1281bad700 1 -- 172.21.15.170:0/1506733871 >> v1:172.21.15.170:6789/0 conn(0x7f127c1597f0 legacy=0x7f127c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1281bad700 1 -- 172.21.15.170:0/1506733871 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0b1790 con 0x7f127c159420 2023-10-08T16:12:19.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1281bad700 1 --2- 172.21.15.170:0/1506733871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 0x7f127c0b5f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1278009910 tx=0x7f12780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=875f2c2ed520e757 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278011630 con 0x7f127c159420 2023-10-08T16:12:19.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278011790 con 0x7f127c159420 2023-10-08T16:12:19.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.485+0000 7f1280bab700 1 -- 172.21.15.170:0/1506733871 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278011b10 con 0x7f127c159420 2023-10-08T16:12:19.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 -- 172.21.15.170:0/1506733871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 msgr2=0x7f127c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 --2- 172.21.15.170:0/1506733871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 0x7f127c0b5f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1278009910 tx=0x7f12780091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 -- 172.21.15.170:0/1506733871 shutdown_connections 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 --2- 172.21.15.170:0/1506733871 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c159420 0x7f127c0b5f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 -- 172.21.15.170:0/1506733871 >> 172.21.15.170:0/1506733871 conn(0x7f127c0d0660 msgr2=0x7f127c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 -- 172.21.15.170:0/1506733871 shutdown_connections 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.486+0000 7f1282baf700 1 -- 172.21.15.170:0/1506733871 wait complete. 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.487+0000 7f1282baf700 1 Processor -- start 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.487+0000 7f1282baf700 1 -- start start 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f127c159420 0x7f127c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c1597f0 0x7f127c168d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c1518b0 con 0x7f127c15d230 2023-10-08T16:12:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c154a70 con 0x7f127c159420 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c15b5f0 con 0x7f127c1597f0 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44418/0 (socket says 172.21.15.170:44418) 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 -- 172.21.15.170:0/3445579146 learned_addr learned my addr 172.21.15.170:0/3445579146 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1281bad700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f127c159420 0x7f127c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12813ac700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c1597f0 0x7f127c168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 -- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c1597f0 msgr2=0x7f127c168d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c1597f0 0x7f127c168d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 -- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f127c159420 msgr2=0x7f127c14e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f127c159420 0x7f127c14e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 -- 172.21.15.170:0/3445579146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009420 con 0x7f127c15d230 2023-10-08T16:12:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f12823ae700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f127400c890 tx=0x7f1274001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274012630 con 0x7f127c15d230 2023-10-08T16:12:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12740127b0 con 0x7f127c15d230 2023-10-08T16:12:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c14ed50 con 0x7f127c15d230 2023-10-08T16:12:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.488+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274012b70 con 0x7f127c15d230 2023-10-08T16:12:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.489+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c151680 con 0x7f127c15d230 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.490+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c04c3c0 con 0x7f127c15d230 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.490+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1274003030 con 0x7f127c15d230 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.490+0000 7f1271ffb700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 0x7f1260069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.490+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1274084af0 con 0x7f127c15d230 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.490+0000 7f1281bad700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 0x7f1260069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.491+0000 7f1281bad700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 0x7f1260069800 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f12780091e0 tx=0x7f1278016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.495+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1274051780 con 0x7f127c15d230 2023-10-08T16:12:19.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.677+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f127c16d930 con 0x7f127c15d230 2023-10-08T16:12:19.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.678+0000 7f1271ffb700 1 -- 172.21.15.170:0/3445579146 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f127c16d930 con 0x7f127c15d230 2023-10-08T16:12:19.678 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:19.679 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":154,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:19.503764+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 msgr2=0x7f1260069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 0x7f1260069800 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f12780091e0 tx=0x7f1278016000 comp rx=0 tx=0).stop 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 msgr2=0x7f127c16cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f127400c890 tx=0x7f1274001630 comp rx=0 tx=0).stop 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 shutdown_connections 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1260067350 0x7f1260069800 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127c15d230 0x7f127c16cae0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f127c159420 0x7f127c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 --2- 172.21.15.170:0/3445579146 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127c1597f0 0x7f127c168d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 >> 172.21.15.170:0/3445579146 conn(0x7f127c0d0660 msgr2=0x7f127c157ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 shutdown_connections 2023-10-08T16:12:19.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.680+0000 7f1282baf700 1 -- 172.21.15.170:0/3445579146 wait complete. 2023-10-08T16:12:19.693 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:12:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.809+0000 7f33c23ae700 1 Processor -- start 2023-10-08T16:12:19.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.809+0000 7f33c23ae700 1 -- start start 2023-10-08T16:12:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.810+0000 7f33c23ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 0x7f33bc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.810+0000 7f33c23ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f33bc159c60 0x7f33bc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.810+0000 7f33c23ae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33bc15a160 con 0x7f33bc15db10 2023-10-08T16:12:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.810+0000 7f33c23ae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc0d13a0 con 0x7f33bc159c60 2023-10-08T16:12:19.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.810+0000 7f33c23ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc15e140 con 0x7f33bc159890 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bb7fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f33bc159c60 0x7f33bc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 0x7f33bc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33c094b700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f33bc15db10 0x7f33bc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57324/0 (socket says 172.21.15.170:57324) 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33c094b700 1 -- 172.21.15.170:0/581624708 learned_addr learned my addr 172.21.15.170:0/581624708 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 -- 172.21.15.170:0/581624708 >> v2:172.21.15.191:3300/0 conn(0x7f33bc159c60 msgr2=0x7f33bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 --2- 172.21.15.170:0/581624708 >> v2:172.21.15.191:3300/0 conn(0x7f33bc159c60 0x7f33bc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 -- 172.21.15.170:0/581624708 >> v1:172.21.15.191:6789/0 conn(0x7f33bc15db10 legacy=0x7f33bc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 -- 172.21.15.170:0/581624708 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc15a3d0 con 0x7f33bc159890 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33bbfff700 1 --2- 172.21.15.170:0/581624708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 0x7f33bc15abd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0098e0 tx=0x7f33ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1b98ef7f3f91df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33baffd700 1 -- 172.21.15.170:0/581624708 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00b640 con 0x7f33bc159890 2023-10-08T16:12:19.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33baffd700 1 -- 172.21.15.170:0/581624708 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac00b7a0 con 0x7f33bc159890 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.811+0000 7f33baffd700 1 -- 172.21.15.170:0/581624708 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00bb40 con 0x7f33bc159890 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 -- 172.21.15.170:0/581624708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 msgr2=0x7f33bc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 --2- 172.21.15.170:0/581624708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 0x7f33bc15abd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0098e0 tx=0x7f33ac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 -- 172.21.15.170:0/581624708 shutdown_connections 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 --2- 172.21.15.170:0/581624708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc159890 0x7f33bc15abd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 --2- 172.21.15.170:0/581624708 >> v2:172.21.15.191:3300/0 conn(0x7f33bc159c60 0x7f33bc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 -- 172.21.15.170:0/581624708 >> 172.21.15.170:0/581624708 conn(0x7f33bc068e20 msgr2=0x7f33bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 -- 172.21.15.170:0/581624708 shutdown_connections 2023-10-08T16:12:19.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.812+0000 7f33c23ae700 1 -- 172.21.15.170:0/581624708 wait complete. 2023-10-08T16:12:19.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.813+0000 7f33c23ae700 1 Processor -- start 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.813+0000 7f33c23ae700 1 -- start start 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 0x7f33bc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc15db10 0x7f33bc0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0b3950 con 0x7f33bc159c60 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc16ac80 con 0x7f33bc159890 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c23ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0cd840 con 0x7f33bc15db10 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44430/0 (socket says 172.21.15.170:44430) 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 -- 172.21.15.170:0/229577838 learned_addr learned my addr 172.21.15.170:0/229577838 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:19.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33c094b700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc15db10 0x7f33bc0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bbfff700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 0x7f33bc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 -- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc15db10 msgr2=0x7f33bc0c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc15db10 0x7f33bc0c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 -- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 msgr2=0x7f33bc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 0x7f33bc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bb7fe700 1 -- 172.21.15.170:0/229577838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac009420 con 0x7f33bc159c60 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.814+0000 7f33bbfff700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 0x7f33bc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33bb7fe700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f33b000ec80 tx=0x7f33b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b000c9d0 con 0x7f33bc159c60 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b000cb30 con 0x7f33bc159c60 2023-10-08T16:12:19.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc16a9a0 con 0x7f33bc159c60 2023-10-08T16:12:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0013470 con 0x7f33bc159c60 2023-10-08T16:12:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.815+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc0cd4d0 con 0x7f33bc159c60 2023-10-08T16:12:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.816+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc04c3c0 con 0x7f33bc159c60 2023-10-08T16:12:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.816+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f33b0013820 con 0x7f33bc159c60 2023-10-08T16:12:19.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.817+0000 7f33a3fff700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.817+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f33b0085020 con 0x7f33bc159c60 2023-10-08T16:12:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.817+0000 7f33bbfff700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:19.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.817+0000 7f33bbfff700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 0x7f33a80697d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f33ac003080 tx=0x7f33ac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:19.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:19.821+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33b0051c20 con 0x7f33bc159c60 2023-10-08T16:12:20.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:20.048+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f33bc0cd980 con 0x7f33bc159c60 2023-10-08T16:12:21.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.041+0000 7f33a3fff700 1 -- 172.21.15.170:0/229577838 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33b0055240 con 0x7f33bc159c60 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 msgr2=0x7f33a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 0x7f33a80697d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f33ac003080 tx=0x7f33ac017000 comp rx=0 tx=0).stop 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 msgr2=0x7f33bc0c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f33b000ec80 tx=0x7f33b00091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 shutdown_connections 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f33bc15db10 0x7f33bc0c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f33bc159890 0x7f33bc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f33a8067320 0x7f33a80697d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 --2- 172.21.15.170:0/229577838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f33bc159c60 0x7f33bc0c8cb0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 >> 172.21.15.170:0/229577838 conn(0x7f33bc068e20 msgr2=0x7f33bc153210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:21.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 shutdown_connections 2023-10-08T16:12:21.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.044+0000 7f33c23ae700 1 -- 172.21.15.170:0/229577838 wait complete. 2023-10-08T16:12:21.059 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:12:21.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.182+0000 7f8ee8b58700 1 Processor -- start 2023-10-08T16:12:21.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.182+0000 7f8ee8b58700 1 -- start start 2023-10-08T16:12:21.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee8b58700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8ee40b4e70 0x7f8ee40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:21.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee8b58700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee40cfb00 con 0x7f8ee40b16d0 2023-10-08T16:12:21.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee8b58700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee415e060 con 0x7f8ee40b1b40 2023-10-08T16:12:21.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee8b58700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee40b5c40 con 0x7f8ee40b4e70 2023-10-08T16:12:21.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8ee40b4e70 0x7f8ee40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee37fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8ee40b4e70 0x7f8ee40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44434/0 (socket says 172.21.15.170:44434) 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee37fe700 1 -- 172.21.15.170:0/2098671826 learned_addr learned my addr 172.21.15.170:0/2098671826 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.183+0000 7f8ee2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8ee40b16d0 0x7f8ee415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35246/0 (socket says 172.21.15.170:35246) 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee37fe700 1 -- 172.21.15.170:0/2098671826 >> v1:172.21.15.191:6789/0 conn(0x7f8ee40b1b40 legacy=0x7f8ee415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee37fe700 1 -- 172.21.15.170:0/2098671826 >> v1:172.21.15.170:6789/0 conn(0x7f8ee40b16d0 legacy=0x7f8ee415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee37fe700 1 -- 172.21.15.170:0/2098671826 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40b5f20 con 0x7f8ee40b4e70 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee27fc700 1 -- 172.21.15.170:0/2098671826 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131596889 0 0) 0x7f8ee40cfb00 con 0x7f8ee40b16d0 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee37fe700 1 --2- 172.21.15.170:0/2098671826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee40ccff0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009f20 tx=0x7f8ed4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6fa3da8d1fe62f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee27fc700 1 -- 172.21.15.170:0/2098671826 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400e070 con 0x7f8ee40b4e70 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee27fc700 1 -- 172.21.15.170:0/2098671826 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400ba40 con 0x7f8ee40b4e70 2023-10-08T16:12:21.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.184+0000 7f8ee27fc700 1 -- 172.21.15.170:0/2098671826 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed40092e0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098671826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 msgr2=0x7f8ee40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098671826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee40ccff0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009f20 tx=0x7f8ed4009840 comp rx=0 tx=0).stop 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098671826 shutdown_connections 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098671826 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee40ccff0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098671826 >> 172.21.15.170:0/2098671826 conn(0x7f8ee40d10c0 msgr2=0x7f8ee40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098671826 shutdown_connections 2023-10-08T16:12:21.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.185+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098671826 wait complete. 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.186+0000 7f8ee8b58700 1 Processor -- start 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.186+0000 7f8ee8b58700 1 -- start start 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee40b16d0 0x7f8ee416ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 0x7f8ee4146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee416b310 con 0x7f8ee40b4e70 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee416b5f0 con 0x7f8ee40b1b40 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee416b8d0 con 0x7f8ee40b16d0 2023-10-08T16:12:21.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44450/0 (socket says 172.21.15.170:44450) 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 -- 172.21.15.170:0/2098373824 learned_addr learned my addr 172.21.15.170:0/2098373824 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee2ffd700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 0x7f8ee4146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee37fe700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee40b16d0 0x7f8ee416ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 -- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee40b16d0 msgr2=0x7f8ee416ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee40b16d0 0x7f8ee416ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 -- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 msgr2=0x7f8ee4146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 0x7f8ee4146af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 -- 172.21.15.170:0/2098373824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4009ae0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee2ffd700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 0x7f8ee4146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee3fff700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00b5e0 tx=0x7f8edc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:21.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00f510 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc00f690 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee416bbb0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc00fa50 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.187+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee4147030 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.188+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee416c0d0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.189+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8edc01a400 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.189+0000 7f8ecb7fe700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 0x7f8ed00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.189+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8edc0845f0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.189+0000 7f8ee37fe700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 0x7f8ed00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:21.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.189+0000 7f8ee37fe700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 0x7f8ed00697d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009ab0 tx=0x7f8ed4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:21.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.193+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ee416c0d0 con 0x7f8ee40b4e70 2023-10-08T16:12:21.421 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:21.421+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8ee416c0d0 con 0x7f8ee40b4e70 2023-10-08T16:12:22.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.042+0000 7f8ecb7fe700 1 -- 172.21.15.170:0/2098373824 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ee416c0d0 con 0x7f8ee40b4e70 2023-10-08T16:12:22.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 msgr2=0x7f8ed00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 0x7f8ed00697d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8ed4009ab0 tx=0x7f8ed4017000 comp rx=0 tx=0).stop 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 msgr2=0x7f8ee416d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00b5e0 tx=0x7f8edc009120 comp rx=0 tx=0).stop 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 shutdown_connections 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8ed0067320 0x7f8ed00697d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ee40b4e70 0x7f8ee416d110 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8ee40b16d0 0x7f8ee416ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 --2- 172.21.15.170:0/2098373824 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8ee40b1b40 0x7f8ee4146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 >> 172.21.15.170:0/2098373824 conn(0x7f8ee40d10c0 msgr2=0x7f8ee40afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 shutdown_connections 2023-10-08T16:12:22.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.045+0000 7f8ee8b58700 1 -- 172.21.15.170:0/2098373824 wait complete. 2023-10-08T16:12:22.059 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:22.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.176+0000 7ff77b77a700 1 Processor -- start 2023-10-08T16:12:22.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.176+0000 7ff77b77a700 1 -- start start 2023-10-08T16:12:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff77b77a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff774159c10 0x7ff77415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff77b77a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff7740b5ea0 0x7ff774068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff77b77a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77415c480 con 0x7ff774068bb0 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff77b77a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740b2a40 con 0x7ff774159c10 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff77b77a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77415b5d0 con 0x7ff7740b5ea0 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff774159c10 0x7ff77415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff774159c10 0x7ff77415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44464/0 (socket says 172.21.15.170:44464) 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 -- 172.21.15.170:0/1517240244 learned_addr learned my addr 172.21.15.170:0/1517240244 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779d17700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff774068bb0 0x7ff77415c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35260/0 (socket says 172.21.15.170:35260) 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff778d15700 1 --2- 172.21.15.170:0/1517240244 >> v2:172.21.15.191:3300/0 conn(0x7ff7740b5ea0 0x7ff774068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 -- 172.21.15.170:0/1517240244 >> v2:172.21.15.191:3300/0 conn(0x7ff7740b5ea0 msgr2=0x7ff774068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 --2- 172.21.15.170:0/1517240244 >> v2:172.21.15.191:3300/0 conn(0x7ff7740b5ea0 0x7ff774068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 -- 172.21.15.170:0/1517240244 >> v1:172.21.15.170:6789/0 conn(0x7ff774068bb0 legacy=0x7ff77415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 -- 172.21.15.170:0/1517240244 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77415b870 con 0x7ff774159c10 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff773fff700 1 -- 172.21.15.170:0/1517240244 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701424475 0 0) 0x7ff77415c480 con 0x7ff774068bb0 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.177+0000 7ff779516700 1 --2- 172.21.15.170:0/1517240244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff77415b090 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff77409a150 tx=0x7ff76c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29bd6ce7ff7ce12d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff773fff700 1 -- 172.21.15.170:0/1517240244 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c00e070 con 0x7ff774159c10 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff773fff700 1 -- 172.21.15.170:0/1517240244 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c00ba50 con 0x7ff774159c10 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff773fff700 1 -- 172.21.15.170:0/1517240244 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0092e0 con 0x7ff774159c10 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 -- 172.21.15.170:0/1517240244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 msgr2=0x7ff77415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1517240244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff77415b090 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff77409a150 tx=0x7ff76c009840 comp rx=0 tx=0).stop 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 -- 172.21.15.170:0/1517240244 shutdown_connections 2023-10-08T16:12:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1517240244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff77415b090 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1517240244 >> v2:172.21.15.191:3300/0 conn(0x7ff7740b5ea0 0x7ff774068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 -- 172.21.15.170:0/1517240244 >> 172.21.15.170:0/1517240244 conn(0x7ff774099a20 msgr2=0x7ff7740af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 -- 172.21.15.170:0/1517240244 shutdown_connections 2023-10-08T16:12:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.178+0000 7ff77b77a700 1 -- 172.21.15.170:0/1517240244 wait complete. 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.179+0000 7ff77b77a700 1 Processor -- start 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.179+0000 7ff77b77a700 1 -- start start 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff774068bb0 0x7ff77414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7740b5ea0 0x7ff7741697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740c6520 con 0x7ff774159c10 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740c6800 con 0x7ff774068bb0 2023-10-08T16:12:22.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff77b77a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740c6ae0 con 0x7ff7740b5ea0 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff779d17700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff779d17700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44476/0 (socket says 172.21.15.170:44476) 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff779d17700 1 -- 172.21.15.170:0/1271356803 learned_addr learned my addr 172.21.15.170:0/1271356803 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff779516700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff774068bb0 0x7ff77414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.180+0000 7ff778d15700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7740b5ea0 0x7ff7741697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 -- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7740b5ea0 msgr2=0x7ff7741697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7740b5ea0 0x7ff7741697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 -- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff774068bb0 msgr2=0x7ff77414f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff774068bb0 0x7ff77414f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 -- 172.21.15.170:0/1271356803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c009ae0 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff779d17700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff764001eb0 tx=0x7ff764007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764005a70 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764005bd0 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640070d0 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7740c6dc0 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.181+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77414f690 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.182+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77404c3c0 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.182+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff764007480 con 0x7ff774159c10 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.183+0000 7ff7717fa700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.183+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff764084050 con 0x7ff774159c10 2023-10-08T16:12:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.183+0000 7ff779516700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.183+0000 7ff779516700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 0x7ff7600697d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009880 tx=0x7ff76c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.187+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff764050ce0 con 0x7ff774159c10 2023-10-08T16:12:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.413+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff77415bb50 con 0x7ff774159c10 2023-10-08T16:12:22.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.413+0000 7ff7717fa700 1 -- 172.21.15.170:0/1271356803 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4615 (secure 0 0 0) 0x7ff764017070 con 0x7ff774159c10 2023-10-08T16:12:22.414 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:22.414 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 msgr2=0x7ff7600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 0x7ff7600697d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009880 tx=0x7ff76c018000 comp rx=0 tx=0).stop 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 msgr2=0x7ff7740b05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff764001eb0 tx=0x7ff764007d10 comp rx=0 tx=0).stop 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 shutdown_connections 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff760067320 0x7ff7600697d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff774159c10 0x7ff7740b05a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff774068bb0 0x7ff77414f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 --2- 172.21.15.170:0/1271356803 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7740b5ea0 0x7ff7741697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 >> 172.21.15.170:0/1271356803 conn(0x7ff774099a20 msgr2=0x7ff7740690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 shutdown_connections 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.415+0000 7ff77b77a700 1 -- 172.21.15.170:0/1271356803 wait complete. 2023-10-08T16:12:22.417 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 100 2023-10-08T16:12:22.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:22.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.541+0000 7f6525a8e700 1 Processor -- start 2023-10-08T16:12:22.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.541+0000 7f6525a8e700 1 -- start start 2023-10-08T16:12:22.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6525a8e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f65200b3f20 0x7f6520159c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6525a8e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6520143420 con 0x7f6520143050 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6525a8e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652006a1e0 con 0x7f652006c0f0 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6525a8e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65200695d0 con 0x7f65200b3f20 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f652528d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f65200b3f20 0x7f6520159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6524a8c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6520143050 0x7f652006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35272/0 (socket says 172.21.15.170:35272) 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f6524a8c700 1 -- 172.21.15.170:0/1222803776 learned_addr learned my addr 172.21.15.170:0/1222803776 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f652528d700 1 -- 172.21.15.170:0/1222803776 >> v1:172.21.15.191:6789/0 conn(0x7f652006c0f0 legacy=0x7f65200b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f652528d700 1 -- 172.21.15.170:0/1222803776 >> v1:172.21.15.170:6789/0 conn(0x7f6520143050 legacy=0x7f652006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f652528d700 1 -- 172.21.15.170:0/1222803776 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200698b0 con 0x7f65200b3f20 2023-10-08T16:12:22.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.542+0000 7f65177fe700 1 -- 172.21.15.170:0/1222803776 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568136320 0 0) 0x7f6520143420 con 0x7f6520143050 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f652528d700 1 --2- 172.21.15.170:0/1222803776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65200b3f20 0x7f6520159c90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f65180098e0 tx=0x7f65180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837de004ec076138 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f65177fe700 1 -- 172.21.15.170:0/1222803776 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518009e80 con 0x7f65200b3f20 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f65177fe700 1 -- 172.21.15.170:0/1222803776 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800b820 con 0x7f65200b3f20 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f65177fe700 1 -- 172.21.15.170:0/1222803776 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800bba0 con 0x7f65200b3f20 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 -- 172.21.15.170:0/1222803776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65200b3f20 msgr2=0x7f6520159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1222803776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65200b3f20 0x7f6520159c90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f65180098e0 tx=0x7f65180091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 -- 172.21.15.170:0/1222803776 shutdown_connections 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1222803776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65200b3f20 0x7f6520159c90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 -- 172.21.15.170:0/1222803776 >> 172.21.15.170:0/1222803776 conn(0x7f65200ce430 msgr2=0x7f65200ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 -- 172.21.15.170:0/1222803776 shutdown_connections 2023-10-08T16:12:22.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.543+0000 7f6525a8e700 1 -- 172.21.15.170:0/1222803776 wait complete. 2023-10-08T16:12:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.544+0000 7f6525a8e700 1 Processor -- start 2023-10-08T16:12:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 -- start start 2023-10-08T16:12:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 0x7f652013bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6520143050 0x7f652016cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652013fcb0 con 0x7f652006c0f0 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652013b650 con 0x7f6520143050 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6525a8e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652013cdc0 con 0x7f65200b3f20 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6524a8c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6524a8c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44494/0 (socket says 172.21.15.170:44494) 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6524a8c700 1 -- 172.21.15.170:0/1266435578 learned_addr learned my addr 172.21.15.170:0/1266435578 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f652528d700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6520143050 0x7f652016cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.545+0000 7f6517fff700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 0x7f652013bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 -- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 msgr2=0x7f652013bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 0x7f652013bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 -- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6520143050 msgr2=0x7f652016cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6520143050 0x7f652016cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 -- 172.21.15.170:0/1266435578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518009420 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6517fff700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 0x7f652013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6524a8c700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f651c00cb60 tx=0x7f651c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c005d80 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c00a5d0 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f652013d0a0 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c00a990 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.546+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652013b000 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.547+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652004c3c0 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.547+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f651c00ad40 con 0x7f652006c0f0 2023-10-08T16:12:22.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.548+0000 7f6514ff9700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 0x7f650c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.548+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f651c084af0 con 0x7f652006c0f0 2023-10-08T16:12:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.548+0000 7f6517fff700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 0x7f650c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.548+0000 7f6517fff700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 0x7f650c0697d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6510006fd0 tx=0x7f6510008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.552+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f651c0517c0 con 0x7f652006c0f0 2023-10-08T16:12:22.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.735+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65200adad0 con 0x7f652006c0f0 2023-10-08T16:12:22.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.736+0000 7f6514ff9700 1 -- 172.21.15.170:0/1266435578 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f651c054de0 con 0x7f652006c0f0 2023-10-08T16:12:22.736 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:22.737 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":154,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:19.503764+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 msgr2=0x7f650c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 0x7f650c0697d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6510006fd0 tx=0x7f6510008040 comp rx=0 tx=0).stop 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 msgr2=0x7f652013f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f651c00cb60 tx=0x7f651c0017b0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 shutdown_connections 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6520143050 0x7f652016cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f650c067320 0x7f650c0697d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f652006c0f0 0x7f652013f770 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 --2- 172.21.15.170:0/1266435578 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65200b3f20 0x7f652013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 >> 172.21.15.170:0/1266435578 conn(0x7f65200ce430 msgr2=0x7f6520138bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 shutdown_connections 2023-10-08T16:12:22.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.738+0000 7f6525a8e700 1 -- 172.21.15.170:0/1266435578 wait complete. 2023-10-08T16:12:22.751 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:12:22.751 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:12:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.864+0000 7fe216934700 1 Processor -- start 2023-10-08T16:12:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 -- start start 2023-10-08T16:12:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2100aca80 0x7fe21006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe210069d60 0x7fe2100c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21006bdb0 con 0x7fe2100c81f0 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2100c8670 con 0x7fe210069d60 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe216934700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cb370 con 0x7fe2100aca80 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.865+0000 7fe20f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe210069d60 0x7fe2100c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe214ed1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe2100c81f0 0x7fe210068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35284/0 (socket says 172.21.15.170:35284) 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe214ed1700 1 -- 172.21.15.170:0/1739996142 learned_addr learned my addr 172.21.15.170:0/1739996142 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20ffff700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2100aca80 0x7fe21006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20f7fe700 1 -- 172.21.15.170:0/1739996142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2100aca80 msgr2=0x7fe21006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20f7fe700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2100aca80 0x7fe21006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20f7fe700 1 -- 172.21.15.170:0/1739996142 >> v1:172.21.15.170:6789/0 conn(0x7fe2100c81f0 legacy=0x7fe210068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20effd700 1 -- 172.21.15.170:0/1739996142 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3659966574 0 0) 0x7fe21006bdb0 con 0x7fe2100c81f0 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20f7fe700 1 -- 172.21.15.170:0/1739996142 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210159f60 con 0x7fe210069d60 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20f7fe700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe210069d60 0x7fe2100c7cb0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098e0 tx=0x7fe2040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fb9d0fc61db193 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20effd700 1 -- 172.21.15.170:0/1739996142 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204009e80 con 0x7fe210069d60 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20effd700 1 -- 172.21.15.170:0/1739996142 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20400b860 con 0x7fe210069d60 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.866+0000 7fe20effd700 1 -- 172.21.15.170:0/1739996142 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400bbe0 con 0x7fe210069d60 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 -- 172.21.15.170:0/1739996142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe210069d60 msgr2=0x7fe2100c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe210069d60 0x7fe2100c7cb0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fe2040098e0 tx=0x7fe2040091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 -- 172.21.15.170:0/1739996142 shutdown_connections 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe2100aca80 0x7fe21006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 --2- 172.21.15.170:0/1739996142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe210069d60 0x7fe2100c7cb0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 -- 172.21.15.170:0/1739996142 >> 172.21.15.170:0/1739996142 conn(0x7fe2100cd670 msgr2=0x7fe2100cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 -- 172.21.15.170:0/1739996142 shutdown_connections 2023-10-08T16:12:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.867+0000 7fe216934700 1 -- 172.21.15.170:0/1739996142 wait complete. 2023-10-08T16:12:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.868+0000 7fe216934700 1 Processor -- start 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.868+0000 7fe216934700 1 -- start start 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 0x7fe2100af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 0x7fe21016a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210169fd0 con 0x7fe2100aca80 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21016b740 con 0x7fe2100c81f0 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100b2d80 con 0x7fe210069d60 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44510/0 (socket says 172.21.15.170:44510) 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 -- 172.21.15.170:0/3076933496 learned_addr learned my addr 172.21.15.170:0/3076933496 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20ffff700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 0x7fe2100af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe214ed1700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 0x7fe21016a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 -- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 msgr2=0x7fe2100af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 0x7fe2100af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 -- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 msgr2=0x7fe21016a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 0x7fe21016a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 -- 172.21.15.170:0/3076933496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009420 con 0x7fe2100aca80 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20ffff700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 0x7fe2100af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe214ed1700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 0x7fe21016a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe20f7fe700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe204013000 tx=0x7fe204017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400bbe0 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204011420 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2100b3060 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2040117c0 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.869+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2101511f0 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.870+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21004c3c0 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.871+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe204011bb0 con 0x7fe2100aca80 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.872+0000 7fe1f7fff700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 0x7fe1fc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.872+0000 7fe20ffff700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 0x7fe1fc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.872+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe204086810 con 0x7fe2100aca80 2023-10-08T16:12:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.872+0000 7fe20ffff700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 0x7fe1fc0697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:22.876+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe204053520 con 0x7fe2100aca80 2023-10-08T16:12:23.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.057+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe21006b510 con 0x7fe1fc067320 2023-10-08T16:12:23.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.064+0000 7fe1f7fff700 1 -- 172.21.15.170:0/3076933496 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289633 (secure 0 0 0) 0x7fe21006b510 con 0x7fe1fc067320 2023-10-08T16:12:23.068 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 msgr2=0x7fe1fc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 0x7fe1fc0697d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe200006fd0 tx=0x7fe200008040 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 msgr2=0x7fe210150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe204013000 tx=0x7fe204017000 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 shutdown_connections 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe2100c81f0 0x7fe21016a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe210069d60 0x7fe2100af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe1fc067320 0x7fe1fc0697d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 --2- 172.21.15.170:0/3076933496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2100aca80 0x7fe210150cb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 >> 172.21.15.170:0/3076933496 conn(0x7fe2100cd670 msgr2=0x7fe21006a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:23.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 shutdown_connections 2023-10-08T16:12:23.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.068+0000 7fe216934700 1 -- 172.21.15.170:0/3076933496 wait complete. 2023-10-08T16:12:23.072 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:12:23.080 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2023-10-08T16:12:22.318755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427376,"kb_used_data":16304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547344,"statfs":{"total":773094113280,"available":770608480256,"internally_reserved":0,"allocated":16695296,"data_stored":8110475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319881"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525597+0000","last_change":"2023-10-08T16:12:18.525597+0000","last_active":"2023-10-08T16:12:18.525597+0000","last_peered":"2023-10-08T16:12:18.525597+0000","last_clean":"2023-10-08T16:12:18.525597+0000","last_became_active":"2023-10-08T16:12:18.525388+0000","last_became_peered":"2023-10-08T16:12:18.525388+0000","last_unstale":"2023-10-08T16:12:18.525597+0000","last_undegraded":"2023-10-08T16:12:18.525597+0000","last_fullsized":"2023-10-08T16:12:18.525597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:51.923781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518555+0000","last_change":"2023-10-08T16:12:18.518555+0000","last_active":"2023-10-08T16:12:18.518555+0000","last_peered":"2023-10-08T16:12:18.518555+0000","last_clean":"2023-10-08T16:12:18.518555+0000","last_became_active":"2023-10-08T16:12:18.517945+0000","last_became_peered":"2023-10-08T16:12:18.517945+0000","last_unstale":"2023-10-08T16:12:18.518555+0000","last_undegraded":"2023-10-08T16:12:18.518555+0000","last_fullsized":"2023-10-08T16:12:18.518555+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:49.927913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519251+0000","last_change":"2023-10-08T16:12:18.519251+0000","last_active":"2023-10-08T16:12:18.519251+0000","last_peered":"2023-10-08T16:12:18.519251+0000","last_clean":"2023-10-08T16:12:18.519251+0000","last_became_active":"2023-10-08T16:12:18.518809+0000","last_became_peered":"2023-10-08T16:12:18.518809+0000","last_unstale":"2023-10-08T16:12:18.519251+0000","last_undegraded":"2023-10-08T16:12:18.519251+0000","last_fullsized":"2023-10-08T16:12:18.519251+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:06.486109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518854+0000","last_change":"2023-10-08T16:12:18.518854+0000","last_active":"2023-10-08T16:12:18.518854+0000","last_peered":"2023-10-08T16:12:18.518854+0000","last_clean":"2023-10-08T16:12:18.518854+0000","last_became_active":"2023-10-08T16:12:18.518564+0000","last_became_peered":"2023-10-08T16:12:18.518564+0000","last_unstale":"2023-10-08T16:12:18.518854+0000","last_undegraded":"2023-10-08T16:12:18.518854+0000","last_fullsized":"2023-10-08T16:12:18.518854+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:51.892831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.509141+0000","last_change":"2023-10-08T16:12:18.509141+0000","last_active":"2023-10-08T16:12:18.509141+0000","last_peered":"2023-10-08T16:12:18.509141+0000","last_clean":"2023-10-08T16:12:18.509141+0000","last_became_active":"2023-10-08T16:12:18.508893+0000","last_became_peered":"2023-10-08T16:12:18.508893+0000","last_unstale":"2023-10-08T16:12:18.509141+0000","last_undegraded":"2023-10-08T16:12:18.509141+0000","last_fullsized":"2023-10-08T16:12:18.509141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:16.208663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515178+0000","last_change":"2023-10-08T16:12:18.515178+0000","last_active":"2023-10-08T16:12:18.515178+0000","last_peered":"2023-10-08T16:12:18.515178+0000","last_clean":"2023-10-08T16:12:18.515178+0000","last_became_active":"2023-10-08T16:12:18.514907+0000","last_became_peered":"2023-10-08T16:12:18.514907+0000","last_unstale":"2023-10-08T16:12:18.515178+0000","last_undegraded":"2023-10-08T16:12:18.515178+0000","last_fullsized":"2023-10-08T16:12:18.515178+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:08.879473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522445+0000","last_change":"2023-10-08T16:12:18.522445+0000","last_active":"2023-10-08T16:12:18.522445+0000","last_peered":"2023-10-08T16:12:18.522445+0000","last_clean":"2023-10-08T16:12:18.522445+0000","last_became_active":"2023-10-08T16:12:18.522211+0000","last_became_peered":"2023-10-08T16:12:18.522211+0000","last_unstale":"2023-10-08T16:12:18.522445+0000","last_undegraded":"2023-10-08T16:12:18.522445+0000","last_fullsized":"2023-10-08T16:12:18.522445+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:52.240033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512860+0000","last_change":"2023-10-08T16:12:18.512860+0000","last_active":"2023-10-08T16:12:18.512860+0000","last_peered":"2023-10-08T16:12:18.512860+0000","last_clean":"2023-10-08T16:12:18.512860+0000","last_became_active":"2023-10-08T16:12:18.512614+0000","last_became_peered":"2023-10-08T16:12:18.512614+0000","last_unstale":"2023-10-08T16:12:18.512860+0000","last_undegraded":"2023-10-08T16:12:18.512860+0000","last_fullsized":"2023-10-08T16:12:18.512860+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:38.175506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519882+0000","last_change":"2023-10-08T16:12:18.519882+0000","last_active":"2023-10-08T16:12:18.519882+0000","last_peered":"2023-10-08T16:12:18.519882+0000","last_clean":"2023-10-08T16:12:18.519882+0000","last_became_active":"2023-10-08T16:12:18.519633+0000","last_became_peered":"2023-10-08T16:12:18.519633+0000","last_unstale":"2023-10-08T16:12:18.519882+0000","last_undegraded":"2023-10-08T16:12:18.519882+0000","last_fullsized":"2023-10-08T16:12:18.519882+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:17.251764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515770+0000","last_change":"2023-10-08T16:12:18.515770+0000","last_active":"2023-10-08T16:12:18.515770+0000","last_peered":"2023-10-08T16:12:18.515770+0000","last_clean":"2023-10-08T16:12:18.515770+0000","last_became_active":"2023-10-08T16:12:18.515526+0000","last_became_peered":"2023-10-08T16:12:18.515526+0000","last_unstale":"2023-10-08T16:12:18.515770+0000","last_undegraded":"2023-10-08T16:12:18.515770+0000","last_fullsized":"2023-10-08T16:12:18.515770+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:20.163481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524852+0000","last_change":"2023-10-08T16:12:18.524852+0000","last_active":"2023-10-08T16:12:18.524852+0000","last_peered":"2023-10-08T16:12:18.524852+0000","last_clean":"2023-10-08T16:12:18.524852+0000","last_became_active":"2023-10-08T16:12:18.524596+0000","last_became_peered":"2023-10-08T16:12:18.524596+0000","last_unstale":"2023-10-08T16:12:18.524852+0000","last_undegraded":"2023-10-08T16:12:18.524852+0000","last_fullsized":"2023-10-08T16:12:18.524852+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:17.736981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521808+0000","last_change":"2023-10-08T16:12:18.521808+0000","last_active":"2023-10-08T16:12:18.521808+0000","last_peered":"2023-10-08T16:12:18.521808+0000","last_clean":"2023-10-08T16:12:18.521808+0000","last_became_active":"2023-10-08T16:12:18.521521+0000","last_became_peered":"2023-10-08T16:12:18.521521+0000","last_unstale":"2023-10-08T16:12:18.521808+0000","last_undegraded":"2023-10-08T16:12:18.521808+0000","last_fullsized":"2023-10-08T16:12:18.521808+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:24.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524386+0000","last_change":"2023-10-08T16:12:18.524386+0000","last_active":"2023-10-08T16:12:18.524386+0000","last_peered":"2023-10-08T16:12:18.524386+0000","last_clean":"2023-10-08T16:12:18.524386+0000","last_became_active":"2023-10-08T16:12:18.524121+0000","last_became_peered":"2023-10-08T16:12:18.524121+0000","last_unstale":"2023-10-08T16:12:18.524386+0000","last_undegraded":"2023-10-08T16:12:18.524386+0000","last_fullsized":"2023-10-08T16:12:18.524386+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:00:01.181336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524683+0000","last_change":"2023-10-08T16:12:18.524683+0000","last_active":"2023-10-08T16:12:18.524683+0000","last_peered":"2023-10-08T16:12:18.524683+0000","last_clean":"2023-10-08T16:12:18.524683+0000","last_became_active":"2023-10-08T16:12:18.524252+0000","last_became_peered":"2023-10-08T16:12:18.524252+0000","last_unstale":"2023-10-08T16:12:18.524683+0000","last_undegraded":"2023-10-08T16:12:18.524683+0000","last_fullsized":"2023-10-08T16:12:18.524683+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:28.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517069+0000","last_change":"2023-10-08T16:12:18.517069+0000","last_active":"2023-10-08T16:12:18.517069+0000","last_peered":"2023-10-08T16:12:18.517069+0000","last_clean":"2023-10-08T16:12:18.517069+0000","last_became_active":"2023-10-08T16:12:18.516727+0000","last_became_peered":"2023-10-08T16:12:18.516727+0000","last_unstale":"2023-10-08T16:12:18.517069+0000","last_undegraded":"2023-10-08T16:12:18.517069+0000","last_fullsized":"2023-10-08T16:12:18.517069+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:17.614646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522997+0000","last_change":"2023-10-08T16:12:18.522997+0000","last_active":"2023-10-08T16:12:18.522997+0000","last_peered":"2023-10-08T16:12:18.522997+0000","last_clean":"2023-10-08T16:12:18.522997+0000","last_became_active":"2023-10-08T16:12:18.522697+0000","last_became_peered":"2023-10-08T16:12:18.522697+0000","last_unstale":"2023-10-08T16:12:18.522997+0000","last_undegraded":"2023-10-08T16:12:18.522997+0000","last_fullsized":"2023-10-08T16:12:18.522997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:51.923781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525455+0000","last_change":"2023-10-08T16:12:18.525455+0000","last_active":"2023-10-08T16:12:18.525455+0000","last_peered":"2023-10-08T16:12:18.525455+0000","last_clean":"2023-10-08T16:12:18.525455+0000","last_became_active":"2023-10-08T16:12:18.525305+0000","last_became_peered":"2023-10-08T16:12:18.525305+0000","last_unstale":"2023-10-08T16:12:18.525455+0000","last_undegraded":"2023-10-08T16:12:18.525455+0000","last_fullsized":"2023-10-08T16:12:18.525455+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:54.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.513477+0000","last_change":"2023-10-08T16:12:18.513477+0000","last_active":"2023-10-08T16:12:18.513477+0000","last_peered":"2023-10-08T16:12:18.513477+0000","last_clean":"2023-10-08T16:12:18.513477+0000","last_became_active":"2023-10-08T16:12:18.513228+0000","last_became_peered":"2023-10-08T16:12:18.513228+0000","last_unstale":"2023-10-08T16:12:18.513477+0000","last_undegraded":"2023-10-08T16:12:18.513477+0000","last_fullsized":"2023-10-08T16:12:18.513477+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:54.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520597+0000","last_change":"2023-10-08T16:12:18.520597+0000","last_active":"2023-10-08T16:12:18.520597+0000","last_peered":"2023-10-08T16:12:18.520597+0000","last_clean":"2023-10-08T16:12:18.520597+0000","last_became_active":"2023-10-08T16:12:18.520336+0000","last_became_peered":"2023-10-08T16:12:18.520336+0000","last_unstale":"2023-10-08T16:12:18.520597+0000","last_undegraded":"2023-10-08T16:12:18.520597+0000","last_fullsized":"2023-10-08T16:12:18.520597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:48.604825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512972+0000","last_change":"2023-10-08T16:12:18.512972+0000","last_active":"2023-10-08T16:12:18.512972+0000","last_peered":"2023-10-08T16:12:18.512972+0000","last_clean":"2023-10-08T16:12:18.512972+0000","last_became_active":"2023-10-08T16:12:18.512722+0000","last_became_peered":"2023-10-08T16:12:18.512722+0000","last_unstale":"2023-10-08T16:12:18.512972+0000","last_undegraded":"2023-10-08T16:12:18.512972+0000","last_fullsized":"2023-10-08T16:12:18.512972+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:39:45.634438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518818+0000","last_change":"2023-10-08T16:12:18.518818+0000","last_active":"2023-10-08T16:12:18.518818+0000","last_peered":"2023-10-08T16:12:18.518818+0000","last_clean":"2023-10-08T16:12:18.518818+0000","last_became_active":"2023-10-08T16:12:18.518541+0000","last_became_peered":"2023-10-08T16:12:18.518541+0000","last_unstale":"2023-10-08T16:12:18.518818+0000","last_undegraded":"2023-10-08T16:12:18.518818+0000","last_fullsized":"2023-10-08T16:12:18.518818+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:03.100486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518007+0000","last_change":"2023-10-08T16:12:18.518007+0000","last_active":"2023-10-08T16:12:18.518007+0000","last_peered":"2023-10-08T16:12:18.518007+0000","last_clean":"2023-10-08T16:12:18.518007+0000","last_became_active":"2023-10-08T16:12:18.517753+0000","last_became_peered":"2023-10-08T16:12:18.517753+0000","last_unstale":"2023-10-08T16:12:18.518007+0000","last_undegraded":"2023-10-08T16:12:18.518007+0000","last_fullsized":"2023-10-08T16:12:18.518007+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:35:31.965365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520938+0000","last_change":"2023-10-08T16:12:18.520938+0000","last_active":"2023-10-08T16:12:18.520938+0000","last_peered":"2023-10-08T16:12:18.520938+0000","last_clean":"2023-10-08T16:12:18.520938+0000","last_became_active":"2023-10-08T16:12:18.520679+0000","last_became_peered":"2023-10-08T16:12:18.520679+0000","last_unstale":"2023-10-08T16:12:18.520938+0000","last_undegraded":"2023-10-08T16:12:18.520938+0000","last_fullsized":"2023-10-08T16:12:18.520938+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:35.788637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512779+0000","last_change":"2023-10-08T16:12:18.512779+0000","last_active":"2023-10-08T16:12:18.512779+0000","last_peered":"2023-10-08T16:12:18.512779+0000","last_clean":"2023-10-08T16:12:18.512779+0000","last_became_active":"2023-10-08T16:12:18.512486+0000","last_became_peered":"2023-10-08T16:12:18.512486+0000","last_unstale":"2023-10-08T16:12:18.512779+0000","last_undegraded":"2023-10-08T16:12:18.512779+0000","last_fullsized":"2023-10-08T16:12:18.512779+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:48.604825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515831+0000","last_change":"2023-10-08T16:12:18.515831+0000","last_active":"2023-10-08T16:12:18.515831+0000","last_peered":"2023-10-08T16:12:18.515831+0000","last_clean":"2023-10-08T16:12:18.515831+0000","last_became_active":"2023-10-08T16:12:18.515276+0000","last_became_peered":"2023-10-08T16:12:18.515276+0000","last_unstale":"2023-10-08T16:12:18.515831+0000","last_undegraded":"2023-10-08T16:12:18.515831+0000","last_fullsized":"2023-10-08T16:12:18.515831+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:19.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522500+0000","last_change":"2023-10-08T16:12:18.522500+0000","last_active":"2023-10-08T16:12:18.522500+0000","last_peered":"2023-10-08T16:12:18.522500+0000","last_clean":"2023-10-08T16:12:18.522500+0000","last_became_active":"2023-10-08T16:12:18.522154+0000","last_became_peered":"2023-10-08T16:12:18.522154+0000","last_unstale":"2023-10-08T16:12:18.522500+0000","last_undegraded":"2023-10-08T16:12:18.522500+0000","last_fullsized":"2023-10-08T16:12:18.522500+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:48.374644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523627+0000","last_change":"2023-10-08T16:12:18.523627+0000","last_active":"2023-10-08T16:12:18.523627+0000","last_peered":"2023-10-08T16:12:18.523627+0000","last_clean":"2023-10-08T16:12:18.523627+0000","last_became_active":"2023-10-08T16:12:18.522193+0000","last_became_peered":"2023-10-08T16:12:18.522193+0000","last_unstale":"2023-10-08T16:12:18.523627+0000","last_undegraded":"2023-10-08T16:12:18.523627+0000","last_fullsized":"2023-10-08T16:12:18.523627+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:20:48.700828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518977+0000","last_change":"2023-10-08T16:12:18.518977+0000","last_active":"2023-10-08T16:12:18.518977+0000","last_peered":"2023-10-08T16:12:18.518977+0000","last_clean":"2023-10-08T16:12:18.518977+0000","last_became_active":"2023-10-08T16:12:18.518723+0000","last_became_peered":"2023-10-08T16:12:18.518723+0000","last_unstale":"2023-10-08T16:12:18.518977+0000","last_undegraded":"2023-10-08T16:12:18.518977+0000","last_fullsized":"2023-10-08T16:12:18.518977+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:14:29.219394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523384+0000","last_change":"2023-10-08T16:12:18.523384+0000","last_active":"2023-10-08T16:12:18.523384+0000","last_peered":"2023-10-08T16:12:18.523384+0000","last_clean":"2023-10-08T16:12:18.523384+0000","last_became_active":"2023-10-08T16:12:18.522954+0000","last_became_peered":"2023-10-08T16:12:18.522954+0000","last_unstale":"2023-10-08T16:12:18.523384+0000","last_undegraded":"2023-10-08T16:12:18.523384+0000","last_fullsized":"2023-10-08T16:12:18.523384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:25:11.194066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.516144+0000","last_change":"2023-10-08T16:12:18.516144+0000","last_active":"2023-10-08T16:12:18.516144+0000","last_peered":"2023-10-08T16:12:18.516144+0000","last_clean":"2023-10-08T16:12:18.516144+0000","last_became_active":"2023-10-08T16:12:18.515896+0000","last_became_peered":"2023-10-08T16:12:18.515896+0000","last_unstale":"2023-10-08T16:12:18.516144+0000","last_undegraded":"2023-10-08T16:12:18.516144+0000","last_fullsized":"2023-10-08T16:12:18.516144+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:38.395269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512859+0000","last_change":"2023-10-08T16:12:18.512859+0000","last_active":"2023-10-08T16:12:18.512859+0000","last_peered":"2023-10-08T16:12:18.512859+0000","last_clean":"2023-10-08T16:12:18.512859+0000","last_became_active":"2023-10-08T16:12:18.512615+0000","last_became_peered":"2023-10-08T16:12:18.512615+0000","last_unstale":"2023-10-08T16:12:18.512859+0000","last_undegraded":"2023-10-08T16:12:18.512859+0000","last_fullsized":"2023-10-08T16:12:18.512859+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:34.131985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.516508+0000","last_change":"2023-10-08T16:12:18.516508+0000","last_active":"2023-10-08T16:12:18.516508+0000","last_peered":"2023-10-08T16:12:18.516508+0000","last_clean":"2023-10-08T16:12:18.516508+0000","last_became_active":"2023-10-08T16:12:18.516246+0000","last_became_peered":"2023-10-08T16:12:18.516246+0000","last_unstale":"2023-10-08T16:12:18.516508+0000","last_undegraded":"2023-10-08T16:12:18.516508+0000","last_fullsized":"2023-10-08T16:12:18.516508+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:18.630799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522024+0000","last_change":"2023-10-08T16:12:18.522024+0000","last_active":"2023-10-08T16:12:18.522024+0000","last_peered":"2023-10-08T16:12:18.522024+0000","last_clean":"2023-10-08T16:12:18.522024+0000","last_became_active":"2023-10-08T16:12:18.521733+0000","last_became_peered":"2023-10-08T16:12:18.521733+0000","last_unstale":"2023-10-08T16:12:18.522024+0000","last_undegraded":"2023-10-08T16:12:18.522024+0000","last_fullsized":"2023-10-08T16:12:18.522024+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:20:48.700828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.034210+0000","last_change":"2023-10-08T16:12:18.016594+0000","last_active":"2023-10-08T16:12:19.034210+0000","last_peered":"2023-10-08T16:12:19.034210+0000","last_clean":"2023-10-08T16:12:19.034210+0000","last_became_active":"2023-10-08T16:12:18.016301+0000","last_became_peered":"2023-10-08T16:12:18.016301+0000","last_unstale":"2023-10-08T16:12:19.034210+0000","last_undegraded":"2023-10-08T16:12:19.034210+0000","last_fullsized":"2023-10-08T16:12:19.034210+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:50.582504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522911+0000","last_change":"2023-10-08T16:12:18.522911+0000","last_active":"2023-10-08T16:12:18.522911+0000","last_peered":"2023-10-08T16:12:18.522911+0000","last_clean":"2023-10-08T16:12:18.522911+0000","last_became_active":"2023-10-08T16:12:18.522621+0000","last_became_peered":"2023-10-08T16:12:18.522621+0000","last_unstale":"2023-10-08T16:12:18.522911+0000","last_undegraded":"2023-10-08T16:12:18.522911+0000","last_fullsized":"2023-10-08T16:12:18.522911+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.469580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512590+0000","last_change":"2023-10-08T16:12:18.016313+0000","last_active":"2023-10-08T16:12:18.512590+0000","last_peered":"2023-10-08T16:12:18.512590+0000","last_clean":"2023-10-08T16:12:18.512590+0000","last_became_active":"2023-10-08T16:12:18.015979+0000","last_became_peered":"2023-10-08T16:12:18.015979+0000","last_unstale":"2023-10-08T16:12:18.512590+0000","last_undegraded":"2023-10-08T16:12:18.512590+0000","last_fullsized":"2023-10-08T16:12:18.512590+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:17.976665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514971+0000","last_change":"2023-10-08T16:12:18.514971+0000","last_active":"2023-10-08T16:12:18.514971+0000","last_peered":"2023-10-08T16:12:18.514971+0000","last_clean":"2023-10-08T16:12:18.514971+0000","last_became_active":"2023-10-08T16:12:18.514683+0000","last_became_peered":"2023-10-08T16:12:18.514683+0000","last_unstale":"2023-10-08T16:12:18.514971+0000","last_undegraded":"2023-10-08T16:12:18.514971+0000","last_fullsized":"2023-10-08T16:12:18.514971+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:45.252223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029425+0000","last_change":"2023-10-08T16:12:18.014292+0000","last_active":"2023-10-08T16:12:19.029425+0000","last_peered":"2023-10-08T16:12:19.029425+0000","last_clean":"2023-10-08T16:12:19.029425+0000","last_became_active":"2023-10-08T16:12:18.014121+0000","last_became_peered":"2023-10-08T16:12:18.014121+0000","last_unstale":"2023-10-08T16:12:19.029425+0000","last_undegraded":"2023-10-08T16:12:19.029425+0000","last_fullsized":"2023-10-08T16:12:19.029425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:22.789675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514458+0000","last_change":"2023-10-08T16:12:18.514458+0000","last_active":"2023-10-08T16:12:18.514458+0000","last_peered":"2023-10-08T16:12:18.514458+0000","last_clean":"2023-10-08T16:12:18.514458+0000","last_became_active":"2023-10-08T16:12:18.514207+0000","last_became_peered":"2023-10-08T16:12:18.514207+0000","last_unstale":"2023-10-08T16:12:18.514458+0000","last_undegraded":"2023-10-08T16:12:18.514458+0000","last_fullsized":"2023-10-08T16:12:18.514458+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:13.037734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.032644+0000","last_change":"2023-10-08T16:12:18.014038+0000","last_active":"2023-10-08T16:12:19.032644+0000","last_peered":"2023-10-08T16:12:19.032644+0000","last_clean":"2023-10-08T16:12:19.032644+0000","last_became_active":"2023-10-08T16:12:18.013776+0000","last_became_peered":"2023-10-08T16:12:18.013776+0000","last_unstale":"2023-10-08T16:12:19.032644+0000","last_undegraded":"2023-10-08T16:12:19.032644+0000","last_fullsized":"2023-10-08T16:12:19.032644+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:15.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525725+0000","last_change":"2023-10-08T16:12:18.525725+0000","last_active":"2023-10-08T16:12:18.525725+0000","last_peered":"2023-10-08T16:12:18.525725+0000","last_clean":"2023-10-08T16:12:18.525725+0000","last_became_active":"2023-10-08T16:12:18.525399+0000","last_became_peered":"2023-10-08T16:12:18.525399+0000","last_unstale":"2023-10-08T16:12:18.525725+0000","last_undegraded":"2023-10-08T16:12:18.525725+0000","last_fullsized":"2023-10-08T16:12:18.525725+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:24.731725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.505861+0000","last_change":"2023-10-08T16:12:18.015694+0000","last_active":"2023-10-08T16:12:18.505861+0000","last_peered":"2023-10-08T16:12:18.505861+0000","last_clean":"2023-10-08T16:12:18.505861+0000","last_became_active":"2023-10-08T16:12:18.015384+0000","last_became_peered":"2023-10-08T16:12:18.015384+0000","last_unstale":"2023-10-08T16:12:18.505861+0000","last_undegraded":"2023-10-08T16:12:18.505861+0000","last_fullsized":"2023-10-08T16:12:18.505861+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:34.055793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.511569+0000","last_change":"2023-10-08T16:12:18.511569+0000","last_active":"2023-10-08T16:12:18.511569+0000","last_peered":"2023-10-08T16:12:18.511569+0000","last_clean":"2023-10-08T16:12:18.511569+0000","last_became_active":"2023-10-08T16:12:18.511324+0000","last_became_peered":"2023-10-08T16:12:18.511324+0000","last_unstale":"2023-10-08T16:12:18.511569+0000","last_undegraded":"2023-10-08T16:12:18.511569+0000","last_fullsized":"2023-10-08T16:12:18.511569+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.747224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017915+0000","last_change":"2023-10-08T16:12:18.013211+0000","last_active":"2023-10-08T16:12:18.017915+0000","last_peered":"2023-10-08T16:12:18.017915+0000","last_clean":"2023-10-08T16:12:18.017915+0000","last_became_active":"2023-10-08T16:12:18.012976+0000","last_became_peered":"2023-10-08T16:12:18.012976+0000","last_unstale":"2023-10-08T16:12:18.017915+0000","last_undegraded":"2023-10-08T16:12:18.017915+0000","last_fullsized":"2023-10-08T16:12:18.017915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:30.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525275+0000","last_change":"2023-10-08T16:12:18.525275+0000","last_active":"2023-10-08T16:12:18.525275+0000","last_peered":"2023-10-08T16:12:18.525275+0000","last_clean":"2023-10-08T16:12:18.525275+0000","last_became_active":"2023-10-08T16:12:18.524987+0000","last_became_peered":"2023-10-08T16:12:18.524987+0000","last_unstale":"2023-10-08T16:12:18.525275+0000","last_undegraded":"2023-10-08T16:12:18.525275+0000","last_fullsized":"2023-10-08T16:12:18.525275+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:18.630799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018977+0000","last_change":"2023-10-08T16:12:18.014311+0000","last_active":"2023-10-08T16:12:18.018977+0000","last_peered":"2023-10-08T16:12:18.018977+0000","last_clean":"2023-10-08T16:12:18.018977+0000","last_became_active":"2023-10-08T16:12:18.014056+0000","last_became_peered":"2023-10-08T16:12:18.014056+0000","last_unstale":"2023-10-08T16:12:18.018977+0000","last_undegraded":"2023-10-08T16:12:18.018977+0000","last_fullsized":"2023-10-08T16:12:18.018977+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:19.758315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519888+0000","last_change":"2023-10-08T16:12:18.519888+0000","last_active":"2023-10-08T16:12:18.519888+0000","last_peered":"2023-10-08T16:12:18.519888+0000","last_clean":"2023-10-08T16:12:18.519888+0000","last_became_active":"2023-10-08T16:12:18.519365+0000","last_became_peered":"2023-10-08T16:12:18.519365+0000","last_unstale":"2023-10-08T16:12:18.519888+0000","last_undegraded":"2023-10-08T16:12:18.519888+0000","last_fullsized":"2023-10-08T16:12:18.519888+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:36.377416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018446+0000","last_change":"2023-10-08T16:12:18.018035+0000","last_active":"2023-10-08T16:12:18.018446+0000","last_peered":"2023-10-08T16:12:18.018446+0000","last_clean":"2023-10-08T16:12:18.018446+0000","last_became_active":"2023-10-08T16:12:18.015904+0000","last_became_peered":"2023-10-08T16:12:18.015904+0000","last_unstale":"2023-10-08T16:12:18.018446+0000","last_undegraded":"2023-10-08T16:12:18.018446+0000","last_fullsized":"2023-10-08T16:12:18.018446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:01.477383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521290+0000","last_change":"2023-10-08T16:12:18.521290+0000","last_active":"2023-10-08T16:12:18.521290+0000","last_peered":"2023-10-08T16:12:18.521290+0000","last_clean":"2023-10-08T16:12:18.521290+0000","last_became_active":"2023-10-08T16:12:18.521008+0000","last_became_peered":"2023-10-08T16:12:18.521008+0000","last_unstale":"2023-10-08T16:12:18.521290+0000","last_undegraded":"2023-10-08T16:12:18.521290+0000","last_fullsized":"2023-10-08T16:12:18.521290+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:20:14.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.020004+0000","last_change":"2023-10-08T16:12:18.017704+0000","last_active":"2023-10-08T16:12:18.020004+0000","last_peered":"2023-10-08T16:12:18.020004+0000","last_clean":"2023-10-08T16:12:18.020004+0000","last_became_active":"2023-10-08T16:12:18.017449+0000","last_became_peered":"2023-10-08T16:12:18.017449+0000","last_unstale":"2023-10-08T16:12:18.020004+0000","last_undegraded":"2023-10-08T16:12:18.020004+0000","last_fullsized":"2023-10-08T16:12:18.020004+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:50.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524905+0000","last_change":"2023-10-08T16:12:18.524905+0000","last_active":"2023-10-08T16:12:18.524905+0000","last_peered":"2023-10-08T16:12:18.524905+0000","last_clean":"2023-10-08T16:12:18.524905+0000","last_became_active":"2023-10-08T16:12:18.524595+0000","last_became_peered":"2023-10-08T16:12:18.524595+0000","last_unstale":"2023-10-08T16:12:18.524905+0000","last_undegraded":"2023-10-08T16:12:18.524905+0000","last_fullsized":"2023-10-08T16:12:18.524905+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:20.163481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.031141+0000","last_change":"2023-10-08T16:12:18.011918+0000","last_active":"2023-10-08T16:12:19.031141+0000","last_peered":"2023-10-08T16:12:19.031141+0000","last_clean":"2023-10-08T16:12:19.031141+0000","last_became_active":"2023-10-08T16:12:18.011652+0000","last_became_peered":"2023-10-08T16:12:18.011652+0000","last_unstale":"2023-10-08T16:12:19.031141+0000","last_undegraded":"2023-10-08T16:12:19.031141+0000","last_fullsized":"2023-10-08T16:12:19.031141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:37.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521746+0000","last_change":"2023-10-08T16:12:18.521746+0000","last_active":"2023-10-08T16:12:18.521746+0000","last_peered":"2023-10-08T16:12:18.521746+0000","last_clean":"2023-10-08T16:12:18.521746+0000","last_became_active":"2023-10-08T16:12:18.521133+0000","last_became_peered":"2023-10-08T16:12:18.521133+0000","last_unstale":"2023-10-08T16:12:18.521746+0000","last_undegraded":"2023-10-08T16:12:18.521746+0000","last_fullsized":"2023-10-08T16:12:18.521746+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:32.924426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"153'6","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.035768+0000","last_change":"2023-10-08T16:12:18.014603+0000","last_active":"2023-10-08T16:12:19.035768+0000","last_peered":"2023-10-08T16:12:19.035768+0000","last_clean":"2023-10-08T16:12:19.035768+0000","last_became_active":"2023-10-08T16:12:18.014348+0000","last_became_peered":"2023-10-08T16:12:18.014348+0000","last_unstale":"2023-10-08T16:12:19.035768+0000","last_undegraded":"2023-10-08T16:12:19.035768+0000","last_fullsized":"2023-10-08T16:12:19.035768+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:43.885298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.510432+0000","last_change":"2023-10-08T16:12:18.510432+0000","last_active":"2023-10-08T16:12:18.510432+0000","last_peered":"2023-10-08T16:12:18.510432+0000","last_clean":"2023-10-08T16:12:18.510432+0000","last_became_active":"2023-10-08T16:12:18.510178+0000","last_became_peered":"2023-10-08T16:12:18.510178+0000","last_unstale":"2023-10-08T16:12:18.510432+0000","last_undegraded":"2023-10-08T16:12:18.510432+0000","last_fullsized":"2023-10-08T16:12:18.510432+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:44.889341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.031441+0000","last_change":"2023-10-08T16:12:18.010833+0000","last_active":"2023-10-08T16:12:19.031441+0000","last_peered":"2023-10-08T16:12:19.031441+0000","last_clean":"2023-10-08T16:12:19.031441+0000","last_became_active":"2023-10-08T16:12:18.010578+0000","last_became_peered":"2023-10-08T16:12:18.010578+0000","last_unstale":"2023-10-08T16:12:19.031441+0000","last_undegraded":"2023-10-08T16:12:19.031441+0000","last_fullsized":"2023-10-08T16:12:19.031441+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:04.998863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515204+0000","last_change":"2023-10-08T16:12:18.515204+0000","last_active":"2023-10-08T16:12:18.515204+0000","last_peered":"2023-10-08T16:12:18.515204+0000","last_clean":"2023-10-08T16:12:18.515204+0000","last_became_active":"2023-10-08T16:12:18.514945+0000","last_became_peered":"2023-10-08T16:12:18.514945+0000","last_unstale":"2023-10-08T16:12:18.515204+0000","last_undegraded":"2023-10-08T16:12:18.515204+0000","last_fullsized":"2023-10-08T16:12:18.515204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:36.377416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019975+0000","last_change":"2023-10-08T16:12:18.017651+0000","last_active":"2023-10-08T16:12:18.019975+0000","last_peered":"2023-10-08T16:12:18.019975+0000","last_clean":"2023-10-08T16:12:18.019975+0000","last_became_active":"2023-10-08T16:12:18.017110+0000","last_became_peered":"2023-10-08T16:12:18.017110+0000","last_unstale":"2023-10-08T16:12:18.019975+0000","last_undegraded":"2023-10-08T16:12:18.019975+0000","last_fullsized":"2023-10-08T16:12:18.019975+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:13.487062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524525+0000","last_change":"2023-10-08T16:12:18.524525+0000","last_active":"2023-10-08T16:12:18.524525+0000","last_peered":"2023-10-08T16:12:18.524525+0000","last_clean":"2023-10-08T16:12:18.524525+0000","last_became_active":"2023-10-08T16:12:18.524290+0000","last_became_peered":"2023-10-08T16:12:18.524290+0000","last_unstale":"2023-10-08T16:12:18.524525+0000","last_undegraded":"2023-10-08T16:12:18.524525+0000","last_fullsized":"2023-10-08T16:12:18.524525+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:51.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018283+0000","last_change":"2023-10-08T16:12:18.017151+0000","last_active":"2023-10-08T16:12:18.018283+0000","last_peered":"2023-10-08T16:12:18.018283+0000","last_clean":"2023-10-08T16:12:18.018283+0000","last_became_active":"2023-10-08T16:12:18.016886+0000","last_became_peered":"2023-10-08T16:12:18.016886+0000","last_unstale":"2023-10-08T16:12:18.018283+0000","last_undegraded":"2023-10-08T16:12:18.018283+0000","last_fullsized":"2023-10-08T16:12:18.018283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:07.875431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517470+0000","last_change":"2023-10-08T16:12:18.517470+0000","last_active":"2023-10-08T16:12:18.517470+0000","last_peered":"2023-10-08T16:12:18.517470+0000","last_clean":"2023-10-08T16:12:18.517470+0000","last_became_active":"2023-10-08T16:12:18.517040+0000","last_became_peered":"2023-10-08T16:12:18.517040+0000","last_unstale":"2023-10-08T16:12:18.517470+0000","last_undegraded":"2023-10-08T16:12:18.517470+0000","last_fullsized":"2023-10-08T16:12:18.517470+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:13.037734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"153'8","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.033905+0000","last_change":"2023-10-08T16:12:18.014980+0000","last_active":"2023-10-08T16:12:19.033905+0000","last_peered":"2023-10-08T16:12:19.033905+0000","last_clean":"2023-10-08T16:12:19.033905+0000","last_became_active":"2023-10-08T16:12:18.014740+0000","last_became_peered":"2023-10-08T16:12:18.014740+0000","last_unstale":"2023-10-08T16:12:19.033905+0000","last_undegraded":"2023-10-08T16:12:19.033905+0000","last_fullsized":"2023-10-08T16:12:19.033905+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:40.269763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518786+0000","last_change":"2023-10-08T16:12:18.518786+0000","last_active":"2023-10-08T16:12:18.518786+0000","last_peered":"2023-10-08T16:12:18.518786+0000","last_clean":"2023-10-08T16:12:18.518786+0000","last_became_active":"2023-10-08T16:12:18.518307+0000","last_became_peered":"2023-10-08T16:12:18.518307+0000","last_unstale":"2023-10-08T16:12:18.518786+0000","last_undegraded":"2023-10-08T16:12:18.518786+0000","last_fullsized":"2023-10-08T16:12:18.518786+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:24.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.506103+0000","last_change":"2023-10-08T16:12:18.011202+0000","last_active":"2023-10-08T16:12:18.506103+0000","last_peered":"2023-10-08T16:12:18.506103+0000","last_clean":"2023-10-08T16:12:18.506103+0000","last_became_active":"2023-10-08T16:12:18.010942+0000","last_became_peered":"2023-10-08T16:12:18.010942+0000","last_unstale":"2023-10-08T16:12:18.506103+0000","last_undegraded":"2023-10-08T16:12:18.506103+0000","last_fullsized":"2023-10-08T16:12:18.506103+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:20.742182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515647+0000","last_change":"2023-10-08T16:12:18.515647+0000","last_active":"2023-10-08T16:12:18.515647+0000","last_peered":"2023-10-08T16:12:18.515647+0000","last_clean":"2023-10-08T16:12:18.515647+0000","last_became_active":"2023-10-08T16:12:18.514987+0000","last_became_peered":"2023-10-08T16:12:18.514987+0000","last_unstale":"2023-10-08T16:12:18.515647+0000","last_undegraded":"2023-10-08T16:12:18.515647+0000","last_fullsized":"2023-10-08T16:12:18.515647+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:00:01.181336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.033079+0000","last_change":"2023-10-08T16:12:18.015622+0000","last_active":"2023-10-08T16:12:19.033079+0000","last_peered":"2023-10-08T16:12:19.033079+0000","last_clean":"2023-10-08T16:12:19.033079+0000","last_became_active":"2023-10-08T16:12:18.014228+0000","last_became_peered":"2023-10-08T16:12:18.014228+0000","last_unstale":"2023-10-08T16:12:19.033079+0000","last_undegraded":"2023-10-08T16:12:19.033079+0000","last_fullsized":"2023-10-08T16:12:19.033079+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:04.370565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518228+0000","last_change":"2023-10-08T16:12:18.518228+0000","last_active":"2023-10-08T16:12:18.518228+0000","last_peered":"2023-10-08T16:12:18.518228+0000","last_clean":"2023-10-08T16:12:18.518228+0000","last_became_active":"2023-10-08T16:12:18.517983+0000","last_became_peered":"2023-10-08T16:12:18.517983+0000","last_unstale":"2023-10-08T16:12:18.518228+0000","last_undegraded":"2023-10-08T16:12:18.518228+0000","last_fullsized":"2023-10-08T16:12:18.518228+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:31.310472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.508296+0000","last_change":"2023-10-08T16:12:18.014402+0000","last_active":"2023-10-08T16:12:18.508296+0000","last_peered":"2023-10-08T16:12:18.508296+0000","last_clean":"2023-10-08T16:12:18.508296+0000","last_became_active":"2023-10-08T16:12:18.014148+0000","last_became_peered":"2023-10-08T16:12:18.014148+0000","last_unstale":"2023-10-08T16:12:18.508296+0000","last_undegraded":"2023-10-08T16:12:18.508296+0000","last_fullsized":"2023-10-08T16:12:18.508296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:43.885298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514344+0000","last_change":"2023-10-08T16:12:18.514344+0000","last_active":"2023-10-08T16:12:18.514344+0000","last_peered":"2023-10-08T16:12:18.514344+0000","last_clean":"2023-10-08T16:12:18.514344+0000","last_became_active":"2023-10-08T16:12:18.514093+0000","last_became_peered":"2023-10-08T16:12:18.514093+0000","last_unstale":"2023-10-08T16:12:18.514344+0000","last_undegraded":"2023-10-08T16:12:18.514344+0000","last_fullsized":"2023-10-08T16:12:18.514344+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:53:02.337480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.013915+0000","last_change":"2023-10-08T16:12:18.013031+0000","last_active":"2023-10-08T16:12:18.013915+0000","last_peered":"2023-10-08T16:12:18.013915+0000","last_clean":"2023-10-08T16:12:18.013915+0000","last_became_active":"2023-10-08T16:12:18.012775+0000","last_became_peered":"2023-10-08T16:12:18.012775+0000","last_unstale":"2023-10-08T16:12:18.013915+0000","last_undegraded":"2023-10-08T16:12:18.013915+0000","last_fullsized":"2023-10-08T16:12:18.013915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:45:48.230836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.513383+0000","last_change":"2023-10-08T16:12:18.513383+0000","last_active":"2023-10-08T16:12:18.513383+0000","last_peered":"2023-10-08T16:12:18.513383+0000","last_clean":"2023-10-08T16:12:18.513383+0000","last_became_active":"2023-10-08T16:12:18.513135+0000","last_became_peered":"2023-10-08T16:12:18.513135+0000","last_unstale":"2023-10-08T16:12:18.513383+0000","last_undegraded":"2023-10-08T16:12:18.513383+0000","last_fullsized":"2023-10-08T16:12:18.513383+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:01.341868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029504+0000","last_change":"2023-10-08T16:12:18.014104+0000","last_active":"2023-10-08T16:12:19.029504+0000","last_peered":"2023-10-08T16:12:19.029504+0000","last_clean":"2023-10-08T16:12:19.029504+0000","last_became_active":"2023-10-08T16:12:18.013740+0000","last_became_peered":"2023-10-08T16:12:18.013740+0000","last_unstale":"2023-10-08T16:12:19.029504+0000","last_undegraded":"2023-10-08T16:12:19.029504+0000","last_fullsized":"2023-10-08T16:12:19.029504+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:37.082523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517409+0000","last_change":"2023-10-08T16:12:18.517409+0000","last_active":"2023-10-08T16:12:18.517409+0000","last_peered":"2023-10-08T16:12:18.517409+0000","last_clean":"2023-10-08T16:12:18.517409+0000","last_became_active":"2023-10-08T16:12:18.517156+0000","last_became_peered":"2023-10-08T16:12:18.517156+0000","last_unstale":"2023-10-08T16:12:18.517409+0000","last_undegraded":"2023-10-08T16:12:18.517409+0000","last_fullsized":"2023-10-08T16:12:18.517409+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:07.856415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017678+0000","last_change":"2023-10-08T16:12:18.014802+0000","last_active":"2023-10-08T16:12:18.017678+0000","last_peered":"2023-10-08T16:12:18.017678+0000","last_clean":"2023-10-08T16:12:18.017678+0000","last_became_active":"2023-10-08T16:12:18.014559+0000","last_became_peered":"2023-10-08T16:12:18.014559+0000","last_unstale":"2023-10-08T16:12:18.017678+0000","last_undegraded":"2023-10-08T16:12:18.017678+0000","last_fullsized":"2023-10-08T16:12:18.017678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:09.817436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517878+0000","last_change":"2023-10-08T16:12:18.517878+0000","last_active":"2023-10-08T16:12:18.517878+0000","last_peered":"2023-10-08T16:12:18.517878+0000","last_clean":"2023-10-08T16:12:18.517878+0000","last_became_active":"2023-10-08T16:12:18.517598+0000","last_became_peered":"2023-10-08T16:12:18.517598+0000","last_unstale":"2023-10-08T16:12:18.517878+0000","last_undegraded":"2023-10-08T16:12:18.517878+0000","last_fullsized":"2023-10-08T16:12:18.517878+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:25:11.194066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.013692+0000","last_change":"2023-10-08T16:12:18.012498+0000","last_active":"2023-10-08T16:12:18.013692+0000","last_peered":"2023-10-08T16:12:18.013692+0000","last_clean":"2023-10-08T16:12:18.013692+0000","last_became_active":"2023-10-08T16:12:18.012239+0000","last_became_peered":"2023-10-08T16:12:18.012239+0000","last_unstale":"2023-10-08T16:12:18.013692+0000","last_undegraded":"2023-10-08T16:12:18.013692+0000","last_fullsized":"2023-10-08T16:12:18.013692+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:47.610560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523242+0000","last_change":"2023-10-08T16:12:18.523242+0000","last_active":"2023-10-08T16:12:18.523242+0000","last_peered":"2023-10-08T16:12:18.523242+0000","last_clean":"2023-10-08T16:12:18.523242+0000","last_became_active":"2023-10-08T16:12:18.523002+0000","last_became_peered":"2023-10-08T16:12:18.523002+0000","last_unstale":"2023-10-08T16:12:18.523242+0000","last_undegraded":"2023-10-08T16:12:18.523242+0000","last_fullsized":"2023-10-08T16:12:18.523242+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:28.697116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.015115+0000","last_change":"2023-10-08T16:12:18.012304+0000","last_active":"2023-10-08T16:12:18.015115+0000","last_peered":"2023-10-08T16:12:18.015115+0000","last_clean":"2023-10-08T16:12:18.015115+0000","last_became_active":"2023-10-08T16:12:18.012056+0000","last_became_peered":"2023-10-08T16:12:18.012056+0000","last_unstale":"2023-10-08T16:12:18.015115+0000","last_undegraded":"2023-10-08T16:12:18.015115+0000","last_fullsized":"2023-10-08T16:12:18.015115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.376610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520778+0000","last_change":"2023-10-08T16:12:18.520778+0000","last_active":"2023-10-08T16:12:18.520778+0000","last_peered":"2023-10-08T16:12:18.520778+0000","last_clean":"2023-10-08T16:12:18.520778+0000","last_became_active":"2023-10-08T16:12:18.520505+0000","last_became_peered":"2023-10-08T16:12:18.520505+0000","last_unstale":"2023-10-08T16:12:18.520778+0000","last_undegraded":"2023-10-08T16:12:18.520778+0000","last_fullsized":"2023-10-08T16:12:18.520778+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:17.251764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029652+0000","last_change":"2023-10-08T16:12:18.015469+0000","last_active":"2023-10-08T16:12:19.029652+0000","last_peered":"2023-10-08T16:12:19.029652+0000","last_clean":"2023-10-08T16:12:19.029652+0000","last_became_active":"2023-10-08T16:12:18.015222+0000","last_became_peered":"2023-10-08T16:12:18.015222+0000","last_unstale":"2023-10-08T16:12:19.029652+0000","last_undegraded":"2023-10-08T16:12:19.029652+0000","last_fullsized":"2023-10-08T16:12:19.029652+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:42.462432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520974+0000","last_change":"2023-10-08T16:12:18.520974+0000","last_active":"2023-10-08T16:12:18.520974+0000","last_peered":"2023-10-08T16:12:18.520974+0000","last_clean":"2023-10-08T16:12:18.520974+0000","last_became_active":"2023-10-08T16:12:18.520711+0000","last_became_peered":"2023-10-08T16:12:18.520711+0000","last_unstale":"2023-10-08T16:12:18.520974+0000","last_undegraded":"2023-10-08T16:12:18.520974+0000","last_fullsized":"2023-10-08T16:12:18.520974+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:12.144723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.020817+0000","last_change":"2023-10-08T16:12:18.019461+0000","last_active":"2023-10-08T16:12:18.020817+0000","last_peered":"2023-10-08T16:12:18.020817+0000","last_clean":"2023-10-08T16:12:18.020817+0000","last_became_active":"2023-10-08T16:12:18.019100+0000","last_became_peered":"2023-10-08T16:12:18.019100+0000","last_unstale":"2023-10-08T16:12:18.020817+0000","last_undegraded":"2023-10-08T16:12:18.020817+0000","last_fullsized":"2023-10-08T16:12:18.020817+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:37.082523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517461+0000","last_change":"2023-10-08T16:12:18.517461+0000","last_active":"2023-10-08T16:12:18.517461+0000","last_peered":"2023-10-08T16:12:18.517461+0000","last_clean":"2023-10-08T16:12:18.517461+0000","last_became_active":"2023-10-08T16:12:18.517174+0000","last_became_peered":"2023-10-08T16:12:18.517174+0000","last_unstale":"2023-10-08T16:12:18.517461+0000","last_undegraded":"2023-10-08T16:12:18.517461+0000","last_fullsized":"2023-10-08T16:12:18.517461+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:11.873458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018318+0000","last_change":"2023-10-08T16:12:18.017586+0000","last_active":"2023-10-08T16:12:18.018318+0000","last_peered":"2023-10-08T16:12:18.018318+0000","last_clean":"2023-10-08T16:12:18.018318+0000","last_became_active":"2023-10-08T16:12:18.017355+0000","last_became_peered":"2023-10-08T16:12:18.017355+0000","last_unstale":"2023-10-08T16:12:18.018318+0000","last_undegraded":"2023-10-08T16:12:18.018318+0000","last_fullsized":"2023-10-08T16:12:18.018318+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:35:30.961323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520910+0000","last_change":"2023-10-08T16:12:18.520910+0000","last_active":"2023-10-08T16:12:18.520910+0000","last_peered":"2023-10-08T16:12:18.520910+0000","last_clean":"2023-10-08T16:12:18.520910+0000","last_became_active":"2023-10-08T16:12:18.520601+0000","last_became_peered":"2023-10-08T16:12:18.520601+0000","last_unstale":"2023-10-08T16:12:18.520910+0000","last_undegraded":"2023-10-08T16:12:18.520910+0000","last_fullsized":"2023-10-08T16:12:18.520910+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:36.071132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017161+0000","last_change":"2023-10-08T16:12:18.015935+0000","last_active":"2023-10-08T16:12:18.017161+0000","last_peered":"2023-10-08T16:12:18.017161+0000","last_clean":"2023-10-08T16:12:18.017161+0000","last_became_active":"2023-10-08T16:12:18.015671+0000","last_became_peered":"2023-10-08T16:12:18.015671+0000","last_unstale":"2023-10-08T16:12:18.017161+0000","last_undegraded":"2023-10-08T16:12:18.017161+0000","last_fullsized":"2023-10-08T16:12:18.017161+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:10.935080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522331+0000","last_change":"2023-10-08T16:12:18.522331+0000","last_active":"2023-10-08T16:12:18.522331+0000","last_peered":"2023-10-08T16:12:18.522331+0000","last_clean":"2023-10-08T16:12:18.522331+0000","last_became_active":"2023-10-08T16:12:18.522091+0000","last_became_peered":"2023-10-08T16:12:18.522091+0000","last_unstale":"2023-10-08T16:12:18.522331+0000","last_undegraded":"2023-10-08T16:12:18.522331+0000","last_fullsized":"2023-10-08T16:12:18.522331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:50.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.014065+0000","last_change":"2023-10-08T16:12:18.012006+0000","last_active":"2023-10-08T16:12:18.014065+0000","last_peered":"2023-10-08T16:12:18.014065+0000","last_clean":"2023-10-08T16:12:18.014065+0000","last_became_active":"2023-10-08T16:12:18.011469+0000","last_became_peered":"2023-10-08T16:12:18.011469+0000","last_unstale":"2023-10-08T16:12:18.014065+0000","last_undegraded":"2023-10-08T16:12:18.014065+0000","last_fullsized":"2023-10-08T16:12:18.014065+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:07.875431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523214+0000","last_change":"2023-10-08T16:12:18.523214+0000","last_active":"2023-10-08T16:12:18.523214+0000","last_peered":"2023-10-08T16:12:18.523214+0000","last_clean":"2023-10-08T16:12:18.523214+0000","last_became_active":"2023-10-08T16:12:18.522633+0000","last_became_peered":"2023-10-08T16:12:18.522633+0000","last_unstale":"2023-10-08T16:12:18.523214+0000","last_undegraded":"2023-10-08T16:12:18.523214+0000","last_fullsized":"2023-10-08T16:12:18.523214+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.903952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.015386+0000","last_change":"2023-10-08T16:12:18.011394+0000","last_active":"2023-10-08T16:12:18.015386+0000","last_peered":"2023-10-08T16:12:18.015386+0000","last_clean":"2023-10-08T16:12:18.015386+0000","last_became_active":"2023-10-08T16:12:18.011140+0000","last_became_peered":"2023-10-08T16:12:18.011140+0000","last_unstale":"2023-10-08T16:12:18.015386+0000","last_undegraded":"2023-10-08T16:12:18.015386+0000","last_fullsized":"2023-10-08T16:12:18.015386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:50.254850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524498+0000","last_change":"2023-10-08T16:12:18.524498+0000","last_active":"2023-10-08T16:12:18.524498+0000","last_peered":"2023-10-08T16:12:18.524498+0000","last_clean":"2023-10-08T16:12:18.524498+0000","last_became_active":"2023-10-08T16:12:18.524263+0000","last_became_peered":"2023-10-08T16:12:18.524263+0000","last_unstale":"2023-10-08T16:12:18.524498+0000","last_undegraded":"2023-10-08T16:12:18.524498+0000","last_fullsized":"2023-10-08T16:12:18.524498+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:33:14.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019912+0000","last_change":"2023-10-08T16:12:18.017394+0000","last_active":"2023-10-08T16:12:18.019912+0000","last_peered":"2023-10-08T16:12:18.019912+0000","last_clean":"2023-10-08T16:12:18.019912+0000","last_became_active":"2023-10-08T16:12:18.017142+0000","last_became_peered":"2023-10-08T16:12:18.017142+0000","last_unstale":"2023-10-08T16:12:18.019912+0000","last_undegraded":"2023-10-08T16:12:18.019912+0000","last_fullsized":"2023-10-08T16:12:18.019912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.376610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521419+0000","last_change":"2023-10-08T16:12:18.521419+0000","last_active":"2023-10-08T16:12:18.521419+0000","last_peered":"2023-10-08T16:12:18.521419+0000","last_clean":"2023-10-08T16:12:18.521419+0000","last_became_active":"2023-10-08T16:12:18.521085+0000","last_became_peered":"2023-10-08T16:12:18.521085+0000","last_unstale":"2023-10-08T16:12:18.521419+0000","last_undegraded":"2023-10-08T16:12:18.521419+0000","last_fullsized":"2023-10-08T16:12:18.521419+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.903952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019383+0000","last_change":"2023-10-08T16:12:18.016264+0000","last_active":"2023-10-08T16:12:18.019383+0000","last_peered":"2023-10-08T16:12:18.019383+0000","last_clean":"2023-10-08T16:12:18.019383+0000","last_became_active":"2023-10-08T16:12:18.016003+0000","last_became_peered":"2023-10-08T16:12:18.016003+0000","last_unstale":"2023-10-08T16:12:18.019383+0000","last_undegraded":"2023-10-08T16:12:18.019383+0000","last_fullsized":"2023-10-08T16:12:18.019383+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:06.852372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520937+0000","last_change":"2023-10-08T16:12:18.520937+0000","last_active":"2023-10-08T16:12:18.520937+0000","last_peered":"2023-10-08T16:12:18.520937+0000","last_clean":"2023-10-08T16:12:18.520937+0000","last_became_active":"2023-10-08T16:12:18.520676+0000","last_became_peered":"2023-10-08T16:12:18.520676+0000","last_unstale":"2023-10-08T16:12:18.520937+0000","last_undegraded":"2023-10-08T16:12:18.520937+0000","last_fullsized":"2023-10-08T16:12:18.520937+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:00.489669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519087+0000","last_change":"2023-10-08T16:12:18.015971+0000","last_active":"2023-10-08T16:12:18.519087+0000","last_peered":"2023-10-08T16:12:18.519087+0000","last_clean":"2023-10-08T16:12:18.519087+0000","last_became_active":"2023-10-08T16:12:18.015449+0000","last_became_peered":"2023-10-08T16:12:18.015449+0000","last_unstale":"2023-10-08T16:12:18.519087+0000","last_undegraded":"2023-10-08T16:12:18.519087+0000","last_fullsized":"2023-10-08T16:12:18.519087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:33:13.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.007454+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:12:18.007454+0000","last_peered":"2023-10-08T16:12:18.007454+0000","last_clean":"2023-10-08T16:12:18.007454+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:12:18.007454+0000","last_undegraded":"2023-10-08T16:12:18.007454+0000","last_fullsized":"2023-10-08T16:12:18.007454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.86199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.754},"min":{"1min":0.38,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":0.82199999999999995,"15min":1.083},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.78500000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.42599999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.626,"5min":0.70299999999999996,"15min":1.175},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.80900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.53100000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66100000000000003,"5min":0.71399999999999997,"15min":8.3409999999999993},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.502},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.63},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":867089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.36899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.503}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.442},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.314}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.628}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.189},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.23000000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.308},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.32400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.437},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.51200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.42499999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.53200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.30599999999999999},"min":{"1min":0.23400000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.36799999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.315,"5min":0.31900000000000001,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.189,"15min":0.189},"max":{"1min":0.40899999999999997,"5min":0.49299999999999999,"15min":0.64300000000000002},"last":0.52500000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.44,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.35799999999999998},"min":{"1min":0.28699999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46200000000000002,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.47399999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.34399999999999997},"min":{"1min":0.25,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.38500000000000001},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46800000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.254,"15min":0.254},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.65900000000000003},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.871},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82399999999999995,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.98999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":867215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.34999999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.375}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.80700000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83099999999999996,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.30499999999999999},"max":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.77100000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.55900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56100000000000005,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.78700000000000003,"15min":0.83399999999999996},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.89200000000000002},"last":0.71699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":1925120,"data_stored":860270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.436}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:12:23.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:12:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.235+0000 7fc2a0ae5700 1 Processor -- start 2023-10-08T16:12:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.235+0000 7fc2a0ae5700 1 -- start start 2023-10-08T16:12:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc2a0ae5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc29c0ccf90 0x7fc29c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc2a0ae5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c0cc300 con 0x7fc29c0cbf30 2023-10-08T16:12:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc2a0ae5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c0ad370 con 0x7fc29c0b1420 2023-10-08T16:12:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc2a0ae5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c156bf0 con 0x7fc29c0ccf90 2023-10-08T16:12:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc29a59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc29c0cbf30 0x7fc29c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35290/0 (socket says 172.21.15.170:35290) 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc29a59c700 1 -- 172.21.15.170:0/1582754604 learned_addr learned my addr 172.21.15.170:0/1582754604 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc29ad9d700 1 --2- 172.21.15.170:0/1582754604 >> v2:172.21.15.191:3300/0 conn(0x7fc29c0ccf90 0x7fc29c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569277400 0 0) 0x7fc29c0cc300 con 0x7fc29c0cbf30 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.236+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc288003570 con 0x7fc29c0cbf30 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1168517243 0 0) 0x7fc29c0ad370 con 0x7fc29c0b1420 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc29c0cc300 con 0x7fc29c0b1420 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128323578 0 0) 0x7fc288003570 con 0x7fc29c0cbf30 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc29ad9d700 1 -- 172.21.15.170:0/1582754604 >> v1:172.21.15.191:6789/0 conn(0x7fc29c0b1420 legacy=0x7fc29c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc29ad9d700 1 -- 172.21.15.170:0/1582754604 >> v1:172.21.15.170:6789/0 conn(0x7fc29c0cbf30 legacy=0x7fc29c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc29ad9d700 1 -- 172.21.15.170:0/1582754604 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0ad650 con 0x7fc29c0ccf90 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc29ad9d700 1 --2- 172.21.15.170:0/1582754604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c0ae410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e60 tx=0x7fc294009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573f1a8802d2a367 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29400e070 con 0x7fc29c0ccf90 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29400b7a0 con 0x7fc29c0ccf90 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc299d9b700 1 -- 172.21.15.170:0/1582754604 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c001470 con 0x7fc29c0ccf90 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/1582754604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 msgr2=0x7fc29c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.237+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/1582754604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c0ae410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc294009e60 tx=0x7fc294009780 comp rx=0 tx=0).stop 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.238+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/1582754604 shutdown_connections 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.238+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/1582754604 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c0ae410 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.238+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/1582754604 >> 172.21.15.170:0/1582754604 conn(0x7fc29c06a320 msgr2=0x7fc29c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.238+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/1582754604 shutdown_connections 2023-10-08T16:12:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.238+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/1582754604 wait complete. 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 Processor -- start 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 -- start start 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc29c0b1420 0x7fc29c15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0c2d60 con 0x7fc29c0cbf30 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0a6490 con 0x7fc29c0ccf90 2023-10-08T16:12:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.239+0000 7fc2a0ae5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0c21e0 con 0x7fc29c0b1420 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44526/0 (socket says 172.21.15.170:44526) 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 -- 172.21.15.170:0/2623913571 learned_addr learned my addr 172.21.15.170:0/2623913571 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc29ad9d700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc29a59c700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc29c0b1420 0x7fc29c15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 -- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc29c0b1420 msgr2=0x7fc29c15a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc29c0b1420 0x7fc29c15a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 -- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 msgr2=0x7fc29c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 -- 172.21.15.170:0/2623913571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc294009a20 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc29ad9d700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc291d9b700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c7c0 tx=0x7fc284007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284010510 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284010690 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284010a50 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c0c2520 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.240+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c15a8f0 con 0x7fc29c0cbf30 2023-10-08T16:12:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.241+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c0a6600 con 0x7fc29c0cbf30 2023-10-08T16:12:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.241+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc28401b400 con 0x7fc29c0cbf30 2023-10-08T16:12:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.242+0000 7fc292ffd700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 0x7fc27c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.242+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc2840847f0 con 0x7fc29c0cbf30 2023-10-08T16:12:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.242+0000 7fc29a59c700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 0x7fc27c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.242+0000 7fc29a59c700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 0x7fc27c0697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc28c005c20 tx=0x7fc28c000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.246+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc29c0a6600 con 0x7fc29c0cbf30 2023-10-08T16:12:23.425 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.424+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc29c0a6600 con 0x7fc27c067320 2023-10-08T16:12:23.432 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.432+0000 7fc292ffd700 1 -- 172.21.15.170:0/2623913571 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289633 (secure 0 0 0) 0x7fc29c0a6600 con 0x7fc27c067320 2023-10-08T16:12:23.436 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:23.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 msgr2=0x7fc27c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 0x7fc27c0697d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc28c005c20 tx=0x7fc28c000bc0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 msgr2=0x7fc29c0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc28400c7c0 tx=0x7fc284007db0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 shutdown_connections 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc29c0ccf90 0x7fc29c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc29c0b1420 0x7fc29c15a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc27c067320 0x7fc27c0697d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 --2- 172.21.15.170:0/2623913571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc29c0cbf30 0x7fc29c0c9f30 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.435+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 >> 172.21.15.170:0/2623913571 conn(0x7fc29c06a320 msgr2=0x7fc29c0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.436+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 shutdown_connections 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.436+0000 7fc2a0ae5700 1 -- 172.21.15.170:0/2623913571 wait complete. 2023-10-08T16:12:23.439 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:12:23.448 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2023-10-08T16:12:22.318755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":69,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427376,"kb_used_data":16304,"kb_used_omap":0,"kb_used_meta":2411008,"kb_avail":752547344,"statfs":{"total":773094113280,"available":770608480256,"internally_reserved":0,"allocated":16695296,"data_stored":8110475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2468872192},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.319881"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525597+0000","last_change":"2023-10-08T16:12:18.525597+0000","last_active":"2023-10-08T16:12:18.525597+0000","last_peered":"2023-10-08T16:12:18.525597+0000","last_clean":"2023-10-08T16:12:18.525597+0000","last_became_active":"2023-10-08T16:12:18.525388+0000","last_became_peered":"2023-10-08T16:12:18.525388+0000","last_unstale":"2023-10-08T16:12:18.525597+0000","last_undegraded":"2023-10-08T16:12:18.525597+0000","last_fullsized":"2023-10-08T16:12:18.525597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:51.923781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518555+0000","last_change":"2023-10-08T16:12:18.518555+0000","last_active":"2023-10-08T16:12:18.518555+0000","last_peered":"2023-10-08T16:12:18.518555+0000","last_clean":"2023-10-08T16:12:18.518555+0000","last_became_active":"2023-10-08T16:12:18.517945+0000","last_became_peered":"2023-10-08T16:12:18.517945+0000","last_unstale":"2023-10-08T16:12:18.518555+0000","last_undegraded":"2023-10-08T16:12:18.518555+0000","last_fullsized":"2023-10-08T16:12:18.518555+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:40:49.927913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519251+0000","last_change":"2023-10-08T16:12:18.519251+0000","last_active":"2023-10-08T16:12:18.519251+0000","last_peered":"2023-10-08T16:12:18.519251+0000","last_clean":"2023-10-08T16:12:18.519251+0000","last_became_active":"2023-10-08T16:12:18.518809+0000","last_became_peered":"2023-10-08T16:12:18.518809+0000","last_unstale":"2023-10-08T16:12:18.519251+0000","last_undegraded":"2023-10-08T16:12:18.519251+0000","last_fullsized":"2023-10-08T16:12:18.519251+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:06.486109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518854+0000","last_change":"2023-10-08T16:12:18.518854+0000","last_active":"2023-10-08T16:12:18.518854+0000","last_peered":"2023-10-08T16:12:18.518854+0000","last_clean":"2023-10-08T16:12:18.518854+0000","last_became_active":"2023-10-08T16:12:18.518564+0000","last_became_peered":"2023-10-08T16:12:18.518564+0000","last_unstale":"2023-10-08T16:12:18.518854+0000","last_undegraded":"2023-10-08T16:12:18.518854+0000","last_fullsized":"2023-10-08T16:12:18.518854+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:51.892831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.509141+0000","last_change":"2023-10-08T16:12:18.509141+0000","last_active":"2023-10-08T16:12:18.509141+0000","last_peered":"2023-10-08T16:12:18.509141+0000","last_clean":"2023-10-08T16:12:18.509141+0000","last_became_active":"2023-10-08T16:12:18.508893+0000","last_became_peered":"2023-10-08T16:12:18.508893+0000","last_unstale":"2023-10-08T16:12:18.509141+0000","last_undegraded":"2023-10-08T16:12:18.509141+0000","last_fullsized":"2023-10-08T16:12:18.509141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:16.208663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515178+0000","last_change":"2023-10-08T16:12:18.515178+0000","last_active":"2023-10-08T16:12:18.515178+0000","last_peered":"2023-10-08T16:12:18.515178+0000","last_clean":"2023-10-08T16:12:18.515178+0000","last_became_active":"2023-10-08T16:12:18.514907+0000","last_became_peered":"2023-10-08T16:12:18.514907+0000","last_unstale":"2023-10-08T16:12:18.515178+0000","last_undegraded":"2023-10-08T16:12:18.515178+0000","last_fullsized":"2023-10-08T16:12:18.515178+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:08.879473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522445+0000","last_change":"2023-10-08T16:12:18.522445+0000","last_active":"2023-10-08T16:12:18.522445+0000","last_peered":"2023-10-08T16:12:18.522445+0000","last_clean":"2023-10-08T16:12:18.522445+0000","last_became_active":"2023-10-08T16:12:18.522211+0000","last_became_peered":"2023-10-08T16:12:18.522211+0000","last_unstale":"2023-10-08T16:12:18.522445+0000","last_undegraded":"2023-10-08T16:12:18.522445+0000","last_fullsized":"2023-10-08T16:12:18.522445+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:52.240033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512860+0000","last_change":"2023-10-08T16:12:18.512860+0000","last_active":"2023-10-08T16:12:18.512860+0000","last_peered":"2023-10-08T16:12:18.512860+0000","last_clean":"2023-10-08T16:12:18.512860+0000","last_became_active":"2023-10-08T16:12:18.512614+0000","last_became_peered":"2023-10-08T16:12:18.512614+0000","last_unstale":"2023-10-08T16:12:18.512860+0000","last_undegraded":"2023-10-08T16:12:18.512860+0000","last_fullsized":"2023-10-08T16:12:18.512860+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:38.175506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519882+0000","last_change":"2023-10-08T16:12:18.519882+0000","last_active":"2023-10-08T16:12:18.519882+0000","last_peered":"2023-10-08T16:12:18.519882+0000","last_clean":"2023-10-08T16:12:18.519882+0000","last_became_active":"2023-10-08T16:12:18.519633+0000","last_became_peered":"2023-10-08T16:12:18.519633+0000","last_unstale":"2023-10-08T16:12:18.519882+0000","last_undegraded":"2023-10-08T16:12:18.519882+0000","last_fullsized":"2023-10-08T16:12:18.519882+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:17.251764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515770+0000","last_change":"2023-10-08T16:12:18.515770+0000","last_active":"2023-10-08T16:12:18.515770+0000","last_peered":"2023-10-08T16:12:18.515770+0000","last_clean":"2023-10-08T16:12:18.515770+0000","last_became_active":"2023-10-08T16:12:18.515526+0000","last_became_peered":"2023-10-08T16:12:18.515526+0000","last_unstale":"2023-10-08T16:12:18.515770+0000","last_undegraded":"2023-10-08T16:12:18.515770+0000","last_fullsized":"2023-10-08T16:12:18.515770+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:20.163481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524852+0000","last_change":"2023-10-08T16:12:18.524852+0000","last_active":"2023-10-08T16:12:18.524852+0000","last_peered":"2023-10-08T16:12:18.524852+0000","last_clean":"2023-10-08T16:12:18.524852+0000","last_became_active":"2023-10-08T16:12:18.524596+0000","last_became_peered":"2023-10-08T16:12:18.524596+0000","last_unstale":"2023-10-08T16:12:18.524852+0000","last_undegraded":"2023-10-08T16:12:18.524852+0000","last_fullsized":"2023-10-08T16:12:18.524852+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:26:17.736981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521808+0000","last_change":"2023-10-08T16:12:18.521808+0000","last_active":"2023-10-08T16:12:18.521808+0000","last_peered":"2023-10-08T16:12:18.521808+0000","last_clean":"2023-10-08T16:12:18.521808+0000","last_became_active":"2023-10-08T16:12:18.521521+0000","last_became_peered":"2023-10-08T16:12:18.521521+0000","last_unstale":"2023-10-08T16:12:18.521808+0000","last_undegraded":"2023-10-08T16:12:18.521808+0000","last_fullsized":"2023-10-08T16:12:18.521808+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:24.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524386+0000","last_change":"2023-10-08T16:12:18.524386+0000","last_active":"2023-10-08T16:12:18.524386+0000","last_peered":"2023-10-08T16:12:18.524386+0000","last_clean":"2023-10-08T16:12:18.524386+0000","last_became_active":"2023-10-08T16:12:18.524121+0000","last_became_peered":"2023-10-08T16:12:18.524121+0000","last_unstale":"2023-10-08T16:12:18.524386+0000","last_undegraded":"2023-10-08T16:12:18.524386+0000","last_fullsized":"2023-10-08T16:12:18.524386+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:00:01.181336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524683+0000","last_change":"2023-10-08T16:12:18.524683+0000","last_active":"2023-10-08T16:12:18.524683+0000","last_peered":"2023-10-08T16:12:18.524683+0000","last_clean":"2023-10-08T16:12:18.524683+0000","last_became_active":"2023-10-08T16:12:18.524252+0000","last_became_peered":"2023-10-08T16:12:18.524252+0000","last_unstale":"2023-10-08T16:12:18.524683+0000","last_undegraded":"2023-10-08T16:12:18.524683+0000","last_fullsized":"2023-10-08T16:12:18.524683+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:28.195979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517069+0000","last_change":"2023-10-08T16:12:18.517069+0000","last_active":"2023-10-08T16:12:18.517069+0000","last_peered":"2023-10-08T16:12:18.517069+0000","last_clean":"2023-10-08T16:12:18.517069+0000","last_became_active":"2023-10-08T16:12:18.516727+0000","last_became_peered":"2023-10-08T16:12:18.516727+0000","last_unstale":"2023-10-08T16:12:18.517069+0000","last_undegraded":"2023-10-08T16:12:18.517069+0000","last_fullsized":"2023-10-08T16:12:18.517069+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:14:17.614646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522997+0000","last_change":"2023-10-08T16:12:18.522997+0000","last_active":"2023-10-08T16:12:18.522997+0000","last_peered":"2023-10-08T16:12:18.522997+0000","last_clean":"2023-10-08T16:12:18.522997+0000","last_became_active":"2023-10-08T16:12:18.522697+0000","last_became_peered":"2023-10-08T16:12:18.522697+0000","last_unstale":"2023-10-08T16:12:18.522997+0000","last_undegraded":"2023-10-08T16:12:18.522997+0000","last_fullsized":"2023-10-08T16:12:18.522997+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:30:51.923781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525455+0000","last_change":"2023-10-08T16:12:18.525455+0000","last_active":"2023-10-08T16:12:18.525455+0000","last_peered":"2023-10-08T16:12:18.525455+0000","last_clean":"2023-10-08T16:12:18.525455+0000","last_became_active":"2023-10-08T16:12:18.525305+0000","last_became_peered":"2023-10-08T16:12:18.525305+0000","last_unstale":"2023-10-08T16:12:18.525455+0000","last_undegraded":"2023-10-08T16:12:18.525455+0000","last_fullsized":"2023-10-08T16:12:18.525455+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:54.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.513477+0000","last_change":"2023-10-08T16:12:18.513477+0000","last_active":"2023-10-08T16:12:18.513477+0000","last_peered":"2023-10-08T16:12:18.513477+0000","last_clean":"2023-10-08T16:12:18.513477+0000","last_became_active":"2023-10-08T16:12:18.513228+0000","last_became_peered":"2023-10-08T16:12:18.513228+0000","last_unstale":"2023-10-08T16:12:18.513477+0000","last_undegraded":"2023-10-08T16:12:18.513477+0000","last_fullsized":"2023-10-08T16:12:18.513477+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:58:54.451707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520597+0000","last_change":"2023-10-08T16:12:18.520597+0000","last_active":"2023-10-08T16:12:18.520597+0000","last_peered":"2023-10-08T16:12:18.520597+0000","last_clean":"2023-10-08T16:12:18.520597+0000","last_became_active":"2023-10-08T16:12:18.520336+0000","last_became_peered":"2023-10-08T16:12:18.520336+0000","last_unstale":"2023-10-08T16:12:18.520597+0000","last_undegraded":"2023-10-08T16:12:18.520597+0000","last_fullsized":"2023-10-08T16:12:18.520597+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:48.604825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512972+0000","last_change":"2023-10-08T16:12:18.512972+0000","last_active":"2023-10-08T16:12:18.512972+0000","last_peered":"2023-10-08T16:12:18.512972+0000","last_clean":"2023-10-08T16:12:18.512972+0000","last_became_active":"2023-10-08T16:12:18.512722+0000","last_became_peered":"2023-10-08T16:12:18.512722+0000","last_unstale":"2023-10-08T16:12:18.512972+0000","last_undegraded":"2023-10-08T16:12:18.512972+0000","last_fullsized":"2023-10-08T16:12:18.512972+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:39:45.634438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518818+0000","last_change":"2023-10-08T16:12:18.518818+0000","last_active":"2023-10-08T16:12:18.518818+0000","last_peered":"2023-10-08T16:12:18.518818+0000","last_clean":"2023-10-08T16:12:18.518818+0000","last_became_active":"2023-10-08T16:12:18.518541+0000","last_became_peered":"2023-10-08T16:12:18.518541+0000","last_unstale":"2023-10-08T16:12:18.518818+0000","last_undegraded":"2023-10-08T16:12:18.518818+0000","last_fullsized":"2023-10-08T16:12:18.518818+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:03.100486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518007+0000","last_change":"2023-10-08T16:12:18.518007+0000","last_active":"2023-10-08T16:12:18.518007+0000","last_peered":"2023-10-08T16:12:18.518007+0000","last_clean":"2023-10-08T16:12:18.518007+0000","last_became_active":"2023-10-08T16:12:18.517753+0000","last_became_peered":"2023-10-08T16:12:18.517753+0000","last_unstale":"2023-10-08T16:12:18.518007+0000","last_undegraded":"2023-10-08T16:12:18.518007+0000","last_fullsized":"2023-10-08T16:12:18.518007+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:35:31.965365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520938+0000","last_change":"2023-10-08T16:12:18.520938+0000","last_active":"2023-10-08T16:12:18.520938+0000","last_peered":"2023-10-08T16:12:18.520938+0000","last_clean":"2023-10-08T16:12:18.520938+0000","last_became_active":"2023-10-08T16:12:18.520679+0000","last_became_peered":"2023-10-08T16:12:18.520679+0000","last_unstale":"2023-10-08T16:12:18.520938+0000","last_undegraded":"2023-10-08T16:12:18.520938+0000","last_fullsized":"2023-10-08T16:12:18.520938+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:35.788637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512779+0000","last_change":"2023-10-08T16:12:18.512779+0000","last_active":"2023-10-08T16:12:18.512779+0000","last_peered":"2023-10-08T16:12:18.512779+0000","last_clean":"2023-10-08T16:12:18.512779+0000","last_became_active":"2023-10-08T16:12:18.512486+0000","last_became_peered":"2023-10-08T16:12:18.512486+0000","last_unstale":"2023-10-08T16:12:18.512779+0000","last_undegraded":"2023-10-08T16:12:18.512779+0000","last_fullsized":"2023-10-08T16:12:18.512779+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:48.604825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515831+0000","last_change":"2023-10-08T16:12:18.515831+0000","last_active":"2023-10-08T16:12:18.515831+0000","last_peered":"2023-10-08T16:12:18.515831+0000","last_clean":"2023-10-08T16:12:18.515831+0000","last_became_active":"2023-10-08T16:12:18.515276+0000","last_became_peered":"2023-10-08T16:12:18.515276+0000","last_unstale":"2023-10-08T16:12:18.515831+0000","last_undegraded":"2023-10-08T16:12:18.515831+0000","last_fullsized":"2023-10-08T16:12:18.515831+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:54:19.938846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522500+0000","last_change":"2023-10-08T16:12:18.522500+0000","last_active":"2023-10-08T16:12:18.522500+0000","last_peered":"2023-10-08T16:12:18.522500+0000","last_clean":"2023-10-08T16:12:18.522500+0000","last_became_active":"2023-10-08T16:12:18.522154+0000","last_became_peered":"2023-10-08T16:12:18.522154+0000","last_unstale":"2023-10-08T16:12:18.522500+0000","last_undegraded":"2023-10-08T16:12:18.522500+0000","last_fullsized":"2023-10-08T16:12:18.522500+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:48.374644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523627+0000","last_change":"2023-10-08T16:12:18.523627+0000","last_active":"2023-10-08T16:12:18.523627+0000","last_peered":"2023-10-08T16:12:18.523627+0000","last_clean":"2023-10-08T16:12:18.523627+0000","last_became_active":"2023-10-08T16:12:18.522193+0000","last_became_peered":"2023-10-08T16:12:18.522193+0000","last_unstale":"2023-10-08T16:12:18.523627+0000","last_undegraded":"2023-10-08T16:12:18.523627+0000","last_fullsized":"2023-10-08T16:12:18.523627+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:20:48.700828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518977+0000","last_change":"2023-10-08T16:12:18.518977+0000","last_active":"2023-10-08T16:12:18.518977+0000","last_peered":"2023-10-08T16:12:18.518977+0000","last_clean":"2023-10-08T16:12:18.518977+0000","last_became_active":"2023-10-08T16:12:18.518723+0000","last_became_peered":"2023-10-08T16:12:18.518723+0000","last_unstale":"2023-10-08T16:12:18.518977+0000","last_undegraded":"2023-10-08T16:12:18.518977+0000","last_fullsized":"2023-10-08T16:12:18.518977+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:14:29.219394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523384+0000","last_change":"2023-10-08T16:12:18.523384+0000","last_active":"2023-10-08T16:12:18.523384+0000","last_peered":"2023-10-08T16:12:18.523384+0000","last_clean":"2023-10-08T16:12:18.523384+0000","last_became_active":"2023-10-08T16:12:18.522954+0000","last_became_peered":"2023-10-08T16:12:18.522954+0000","last_unstale":"2023-10-08T16:12:18.523384+0000","last_undegraded":"2023-10-08T16:12:18.523384+0000","last_fullsized":"2023-10-08T16:12:18.523384+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:25:11.194066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.516144+0000","last_change":"2023-10-08T16:12:18.516144+0000","last_active":"2023-10-08T16:12:18.516144+0000","last_peered":"2023-10-08T16:12:18.516144+0000","last_clean":"2023-10-08T16:12:18.516144+0000","last_became_active":"2023-10-08T16:12:18.515896+0000","last_became_peered":"2023-10-08T16:12:18.515896+0000","last_unstale":"2023-10-08T16:12:18.516144+0000","last_undegraded":"2023-10-08T16:12:18.516144+0000","last_fullsized":"2023-10-08T16:12:18.516144+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:38.395269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512859+0000","last_change":"2023-10-08T16:12:18.512859+0000","last_active":"2023-10-08T16:12:18.512859+0000","last_peered":"2023-10-08T16:12:18.512859+0000","last_clean":"2023-10-08T16:12:18.512859+0000","last_became_active":"2023-10-08T16:12:18.512615+0000","last_became_peered":"2023-10-08T16:12:18.512615+0000","last_unstale":"2023-10-08T16:12:18.512859+0000","last_undegraded":"2023-10-08T16:12:18.512859+0000","last_fullsized":"2023-10-08T16:12:18.512859+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:34.131985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.516508+0000","last_change":"2023-10-08T16:12:18.516508+0000","last_active":"2023-10-08T16:12:18.516508+0000","last_peered":"2023-10-08T16:12:18.516508+0000","last_clean":"2023-10-08T16:12:18.516508+0000","last_became_active":"2023-10-08T16:12:18.516246+0000","last_became_peered":"2023-10-08T16:12:18.516246+0000","last_unstale":"2023-10-08T16:12:18.516508+0000","last_undegraded":"2023-10-08T16:12:18.516508+0000","last_fullsized":"2023-10-08T16:12:18.516508+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:18.630799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522024+0000","last_change":"2023-10-08T16:12:18.522024+0000","last_active":"2023-10-08T16:12:18.522024+0000","last_peered":"2023-10-08T16:12:18.522024+0000","last_clean":"2023-10-08T16:12:18.522024+0000","last_became_active":"2023-10-08T16:12:18.521733+0000","last_became_peered":"2023-10-08T16:12:18.521733+0000","last_unstale":"2023-10-08T16:12:18.522024+0000","last_undegraded":"2023-10-08T16:12:18.522024+0000","last_fullsized":"2023-10-08T16:12:18.522024+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:20:48.700828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"153'4","reported_seq":13,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.034210+0000","last_change":"2023-10-08T16:12:18.016594+0000","last_active":"2023-10-08T16:12:19.034210+0000","last_peered":"2023-10-08T16:12:19.034210+0000","last_clean":"2023-10-08T16:12:19.034210+0000","last_became_active":"2023-10-08T16:12:18.016301+0000","last_became_peered":"2023-10-08T16:12:18.016301+0000","last_unstale":"2023-10-08T16:12:19.034210+0000","last_undegraded":"2023-10-08T16:12:19.034210+0000","last_fullsized":"2023-10-08T16:12:19.034210+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:50.582504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522911+0000","last_change":"2023-10-08T16:12:18.522911+0000","last_active":"2023-10-08T16:12:18.522911+0000","last_peered":"2023-10-08T16:12:18.522911+0000","last_clean":"2023-10-08T16:12:18.522911+0000","last_became_active":"2023-10-08T16:12:18.522621+0000","last_became_peered":"2023-10-08T16:12:18.522621+0000","last_unstale":"2023-10-08T16:12:18.522911+0000","last_undegraded":"2023-10-08T16:12:18.522911+0000","last_fullsized":"2023-10-08T16:12:18.522911+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:41:56.469580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.512590+0000","last_change":"2023-10-08T16:12:18.016313+0000","last_active":"2023-10-08T16:12:18.512590+0000","last_peered":"2023-10-08T16:12:18.512590+0000","last_clean":"2023-10-08T16:12:18.512590+0000","last_became_active":"2023-10-08T16:12:18.015979+0000","last_became_peered":"2023-10-08T16:12:18.015979+0000","last_unstale":"2023-10-08T16:12:18.512590+0000","last_undegraded":"2023-10-08T16:12:18.512590+0000","last_fullsized":"2023-10-08T16:12:18.512590+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:10:17.976665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514971+0000","last_change":"2023-10-08T16:12:18.514971+0000","last_active":"2023-10-08T16:12:18.514971+0000","last_peered":"2023-10-08T16:12:18.514971+0000","last_clean":"2023-10-08T16:12:18.514971+0000","last_became_active":"2023-10-08T16:12:18.514683+0000","last_became_peered":"2023-10-08T16:12:18.514683+0000","last_unstale":"2023-10-08T16:12:18.514971+0000","last_undegraded":"2023-10-08T16:12:18.514971+0000","last_fullsized":"2023-10-08T16:12:18.514971+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:41:45.252223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029425+0000","last_change":"2023-10-08T16:12:18.014292+0000","last_active":"2023-10-08T16:12:19.029425+0000","last_peered":"2023-10-08T16:12:19.029425+0000","last_clean":"2023-10-08T16:12:19.029425+0000","last_became_active":"2023-10-08T16:12:18.014121+0000","last_became_peered":"2023-10-08T16:12:18.014121+0000","last_unstale":"2023-10-08T16:12:19.029425+0000","last_undegraded":"2023-10-08T16:12:19.029425+0000","last_fullsized":"2023-10-08T16:12:19.029425+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:22.789675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514458+0000","last_change":"2023-10-08T16:12:18.514458+0000","last_active":"2023-10-08T16:12:18.514458+0000","last_peered":"2023-10-08T16:12:18.514458+0000","last_clean":"2023-10-08T16:12:18.514458+0000","last_became_active":"2023-10-08T16:12:18.514207+0000","last_became_peered":"2023-10-08T16:12:18.514207+0000","last_unstale":"2023-10-08T16:12:18.514458+0000","last_undegraded":"2023-10-08T16:12:18.514458+0000","last_fullsized":"2023-10-08T16:12:18.514458+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:13.037734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.032644+0000","last_change":"2023-10-08T16:12:18.014038+0000","last_active":"2023-10-08T16:12:19.032644+0000","last_peered":"2023-10-08T16:12:19.032644+0000","last_clean":"2023-10-08T16:12:19.032644+0000","last_became_active":"2023-10-08T16:12:18.013776+0000","last_became_peered":"2023-10-08T16:12:18.013776+0000","last_unstale":"2023-10-08T16:12:19.032644+0000","last_undegraded":"2023-10-08T16:12:19.032644+0000","last_fullsized":"2023-10-08T16:12:19.032644+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:15.204621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525725+0000","last_change":"2023-10-08T16:12:18.525725+0000","last_active":"2023-10-08T16:12:18.525725+0000","last_peered":"2023-10-08T16:12:18.525725+0000","last_clean":"2023-10-08T16:12:18.525725+0000","last_became_active":"2023-10-08T16:12:18.525399+0000","last_became_peered":"2023-10-08T16:12:18.525399+0000","last_unstale":"2023-10-08T16:12:18.525725+0000","last_undegraded":"2023-10-08T16:12:18.525725+0000","last_fullsized":"2023-10-08T16:12:18.525725+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:24.731725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.505861+0000","last_change":"2023-10-08T16:12:18.015694+0000","last_active":"2023-10-08T16:12:18.505861+0000","last_peered":"2023-10-08T16:12:18.505861+0000","last_clean":"2023-10-08T16:12:18.505861+0000","last_became_active":"2023-10-08T16:12:18.015384+0000","last_became_peered":"2023-10-08T16:12:18.015384+0000","last_unstale":"2023-10-08T16:12:18.505861+0000","last_undegraded":"2023-10-08T16:12:18.505861+0000","last_fullsized":"2023-10-08T16:12:18.505861+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:14:34.055793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.511569+0000","last_change":"2023-10-08T16:12:18.511569+0000","last_active":"2023-10-08T16:12:18.511569+0000","last_peered":"2023-10-08T16:12:18.511569+0000","last_clean":"2023-10-08T16:12:18.511569+0000","last_became_active":"2023-10-08T16:12:18.511324+0000","last_became_peered":"2023-10-08T16:12:18.511324+0000","last_unstale":"2023-10-08T16:12:18.511569+0000","last_undegraded":"2023-10-08T16:12:18.511569+0000","last_fullsized":"2023-10-08T16:12:18.511569+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:54:09.747224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017915+0000","last_change":"2023-10-08T16:12:18.013211+0000","last_active":"2023-10-08T16:12:18.017915+0000","last_peered":"2023-10-08T16:12:18.017915+0000","last_clean":"2023-10-08T16:12:18.017915+0000","last_became_active":"2023-10-08T16:12:18.012976+0000","last_became_peered":"2023-10-08T16:12:18.012976+0000","last_unstale":"2023-10-08T16:12:18.017915+0000","last_undegraded":"2023-10-08T16:12:18.017915+0000","last_fullsized":"2023-10-08T16:12:18.017915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:17:30.725453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.525275+0000","last_change":"2023-10-08T16:12:18.525275+0000","last_active":"2023-10-08T16:12:18.525275+0000","last_peered":"2023-10-08T16:12:18.525275+0000","last_clean":"2023-10-08T16:12:18.525275+0000","last_became_active":"2023-10-08T16:12:18.524987+0000","last_became_peered":"2023-10-08T16:12:18.524987+0000","last_unstale":"2023-10-08T16:12:18.525275+0000","last_undegraded":"2023-10-08T16:12:18.525275+0000","last_fullsized":"2023-10-08T16:12:18.525275+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:18.630799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018977+0000","last_change":"2023-10-08T16:12:18.014311+0000","last_active":"2023-10-08T16:12:18.018977+0000","last_peered":"2023-10-08T16:12:18.018977+0000","last_clean":"2023-10-08T16:12:18.018977+0000","last_became_active":"2023-10-08T16:12:18.014056+0000","last_became_peered":"2023-10-08T16:12:18.014056+0000","last_unstale":"2023-10-08T16:12:18.018977+0000","last_undegraded":"2023-10-08T16:12:18.018977+0000","last_fullsized":"2023-10-08T16:12:18.018977+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:46:19.758315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519888+0000","last_change":"2023-10-08T16:12:18.519888+0000","last_active":"2023-10-08T16:12:18.519888+0000","last_peered":"2023-10-08T16:12:18.519888+0000","last_clean":"2023-10-08T16:12:18.519888+0000","last_became_active":"2023-10-08T16:12:18.519365+0000","last_became_peered":"2023-10-08T16:12:18.519365+0000","last_unstale":"2023-10-08T16:12:18.519888+0000","last_undegraded":"2023-10-08T16:12:18.519888+0000","last_fullsized":"2023-10-08T16:12:18.519888+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:36.377416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018446+0000","last_change":"2023-10-08T16:12:18.018035+0000","last_active":"2023-10-08T16:12:18.018446+0000","last_peered":"2023-10-08T16:12:18.018446+0000","last_clean":"2023-10-08T16:12:18.018446+0000","last_became_active":"2023-10-08T16:12:18.015904+0000","last_became_peered":"2023-10-08T16:12:18.015904+0000","last_unstale":"2023-10-08T16:12:18.018446+0000","last_undegraded":"2023-10-08T16:12:18.018446+0000","last_fullsized":"2023-10-08T16:12:18.018446+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:50:01.477383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521290+0000","last_change":"2023-10-08T16:12:18.521290+0000","last_active":"2023-10-08T16:12:18.521290+0000","last_peered":"2023-10-08T16:12:18.521290+0000","last_clean":"2023-10-08T16:12:18.521290+0000","last_became_active":"2023-10-08T16:12:18.521008+0000","last_became_peered":"2023-10-08T16:12:18.521008+0000","last_unstale":"2023-10-08T16:12:18.521290+0000","last_undegraded":"2023-10-08T16:12:18.521290+0000","last_fullsized":"2023-10-08T16:12:18.521290+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:20:14.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.020004+0000","last_change":"2023-10-08T16:12:18.017704+0000","last_active":"2023-10-08T16:12:18.020004+0000","last_peered":"2023-10-08T16:12:18.020004+0000","last_clean":"2023-10-08T16:12:18.020004+0000","last_became_active":"2023-10-08T16:12:18.017449+0000","last_became_peered":"2023-10-08T16:12:18.017449+0000","last_unstale":"2023-10-08T16:12:18.020004+0000","last_undegraded":"2023-10-08T16:12:18.020004+0000","last_fullsized":"2023-10-08T16:12:18.020004+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:50.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524905+0000","last_change":"2023-10-08T16:12:18.524905+0000","last_active":"2023-10-08T16:12:18.524905+0000","last_peered":"2023-10-08T16:12:18.524905+0000","last_clean":"2023-10-08T16:12:18.524905+0000","last_became_active":"2023-10-08T16:12:18.524595+0000","last_became_peered":"2023-10-08T16:12:18.524595+0000","last_unstale":"2023-10-08T16:12:18.524905+0000","last_undegraded":"2023-10-08T16:12:18.524905+0000","last_fullsized":"2023-10-08T16:12:18.524905+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:12:20.163481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"153'3","reported_seq":12,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.031141+0000","last_change":"2023-10-08T16:12:18.011918+0000","last_active":"2023-10-08T16:12:19.031141+0000","last_peered":"2023-10-08T16:12:19.031141+0000","last_clean":"2023-10-08T16:12:19.031141+0000","last_became_active":"2023-10-08T16:12:18.011652+0000","last_became_peered":"2023-10-08T16:12:18.011652+0000","last_unstale":"2023-10-08T16:12:19.031141+0000","last_undegraded":"2023-10-08T16:12:19.031141+0000","last_fullsized":"2023-10-08T16:12:19.031141+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:37.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521746+0000","last_change":"2023-10-08T16:12:18.521746+0000","last_active":"2023-10-08T16:12:18.521746+0000","last_peered":"2023-10-08T16:12:18.521746+0000","last_clean":"2023-10-08T16:12:18.521746+0000","last_became_active":"2023-10-08T16:12:18.521133+0000","last_became_peered":"2023-10-08T16:12:18.521133+0000","last_unstale":"2023-10-08T16:12:18.521746+0000","last_undegraded":"2023-10-08T16:12:18.521746+0000","last_fullsized":"2023-10-08T16:12:18.521746+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:32.924426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"153'6","reported_seq":15,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.035768+0000","last_change":"2023-10-08T16:12:18.014603+0000","last_active":"2023-10-08T16:12:19.035768+0000","last_peered":"2023-10-08T16:12:19.035768+0000","last_clean":"2023-10-08T16:12:19.035768+0000","last_became_active":"2023-10-08T16:12:18.014348+0000","last_became_peered":"2023-10-08T16:12:18.014348+0000","last_unstale":"2023-10-08T16:12:19.035768+0000","last_undegraded":"2023-10-08T16:12:19.035768+0000","last_fullsized":"2023-10-08T16:12:19.035768+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:43.885298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.510432+0000","last_change":"2023-10-08T16:12:18.510432+0000","last_active":"2023-10-08T16:12:18.510432+0000","last_peered":"2023-10-08T16:12:18.510432+0000","last_clean":"2023-10-08T16:12:18.510432+0000","last_became_active":"2023-10-08T16:12:18.510178+0000","last_became_peered":"2023-10-08T16:12:18.510178+0000","last_unstale":"2023-10-08T16:12:18.510432+0000","last_undegraded":"2023-10-08T16:12:18.510432+0000","last_fullsized":"2023-10-08T16:12:18.510432+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:44.889341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.031441+0000","last_change":"2023-10-08T16:12:18.010833+0000","last_active":"2023-10-08T16:12:19.031441+0000","last_peered":"2023-10-08T16:12:19.031441+0000","last_clean":"2023-10-08T16:12:19.031441+0000","last_became_active":"2023-10-08T16:12:18.010578+0000","last_became_peered":"2023-10-08T16:12:18.010578+0000","last_unstale":"2023-10-08T16:12:19.031441+0000","last_undegraded":"2023-10-08T16:12:19.031441+0000","last_fullsized":"2023-10-08T16:12:19.031441+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:04.998863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515204+0000","last_change":"2023-10-08T16:12:18.515204+0000","last_active":"2023-10-08T16:12:18.515204+0000","last_peered":"2023-10-08T16:12:18.515204+0000","last_clean":"2023-10-08T16:12:18.515204+0000","last_became_active":"2023-10-08T16:12:18.514945+0000","last_became_peered":"2023-10-08T16:12:18.514945+0000","last_unstale":"2023-10-08T16:12:18.515204+0000","last_undegraded":"2023-10-08T16:12:18.515204+0000","last_fullsized":"2023-10-08T16:12:18.515204+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:44:36.377416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019975+0000","last_change":"2023-10-08T16:12:18.017651+0000","last_active":"2023-10-08T16:12:18.019975+0000","last_peered":"2023-10-08T16:12:18.019975+0000","last_clean":"2023-10-08T16:12:18.019975+0000","last_became_active":"2023-10-08T16:12:18.017110+0000","last_became_peered":"2023-10-08T16:12:18.017110+0000","last_unstale":"2023-10-08T16:12:18.019975+0000","last_undegraded":"2023-10-08T16:12:18.019975+0000","last_fullsized":"2023-10-08T16:12:18.019975+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:39:13.487062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524525+0000","last_change":"2023-10-08T16:12:18.524525+0000","last_active":"2023-10-08T16:12:18.524525+0000","last_peered":"2023-10-08T16:12:18.524525+0000","last_clean":"2023-10-08T16:12:18.524525+0000","last_became_active":"2023-10-08T16:12:18.524290+0000","last_became_peered":"2023-10-08T16:12:18.524290+0000","last_unstale":"2023-10-08T16:12:18.524525+0000","last_undegraded":"2023-10-08T16:12:18.524525+0000","last_fullsized":"2023-10-08T16:12:18.524525+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:48:51.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018283+0000","last_change":"2023-10-08T16:12:18.017151+0000","last_active":"2023-10-08T16:12:18.018283+0000","last_peered":"2023-10-08T16:12:18.018283+0000","last_clean":"2023-10-08T16:12:18.018283+0000","last_became_active":"2023-10-08T16:12:18.016886+0000","last_became_peered":"2023-10-08T16:12:18.016886+0000","last_unstale":"2023-10-08T16:12:18.018283+0000","last_undegraded":"2023-10-08T16:12:18.018283+0000","last_fullsized":"2023-10-08T16:12:18.018283+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:07.875431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517470+0000","last_change":"2023-10-08T16:12:18.517470+0000","last_active":"2023-10-08T16:12:18.517470+0000","last_peered":"2023-10-08T16:12:18.517470+0000","last_clean":"2023-10-08T16:12:18.517470+0000","last_became_active":"2023-10-08T16:12:18.517040+0000","last_became_peered":"2023-10-08T16:12:18.517040+0000","last_unstale":"2023-10-08T16:12:18.517470+0000","last_undegraded":"2023-10-08T16:12:18.517470+0000","last_fullsized":"2023-10-08T16:12:18.517470+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:06:13.037734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"153'8","reported_seq":17,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.033905+0000","last_change":"2023-10-08T16:12:18.014980+0000","last_active":"2023-10-08T16:12:19.033905+0000","last_peered":"2023-10-08T16:12:19.033905+0000","last_clean":"2023-10-08T16:12:19.033905+0000","last_became_active":"2023-10-08T16:12:18.014740+0000","last_became_peered":"2023-10-08T16:12:18.014740+0000","last_unstale":"2023-10-08T16:12:19.033905+0000","last_undegraded":"2023-10-08T16:12:19.033905+0000","last_fullsized":"2023-10-08T16:12:19.033905+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:40.269763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518786+0000","last_change":"2023-10-08T16:12:18.518786+0000","last_active":"2023-10-08T16:12:18.518786+0000","last_peered":"2023-10-08T16:12:18.518786+0000","last_clean":"2023-10-08T16:12:18.518786+0000","last_became_active":"2023-10-08T16:12:18.518307+0000","last_became_peered":"2023-10-08T16:12:18.518307+0000","last_unstale":"2023-10-08T16:12:18.518786+0000","last_undegraded":"2023-10-08T16:12:18.518786+0000","last_fullsized":"2023-10-08T16:12:18.518786+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:42:24.458300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.506103+0000","last_change":"2023-10-08T16:12:18.011202+0000","last_active":"2023-10-08T16:12:18.506103+0000","last_peered":"2023-10-08T16:12:18.506103+0000","last_clean":"2023-10-08T16:12:18.506103+0000","last_became_active":"2023-10-08T16:12:18.010942+0000","last_became_peered":"2023-10-08T16:12:18.010942+0000","last_unstale":"2023-10-08T16:12:18.506103+0000","last_undegraded":"2023-10-08T16:12:18.506103+0000","last_fullsized":"2023-10-08T16:12:18.506103+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:44:20.742182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.515647+0000","last_change":"2023-10-08T16:12:18.515647+0000","last_active":"2023-10-08T16:12:18.515647+0000","last_peered":"2023-10-08T16:12:18.515647+0000","last_clean":"2023-10-08T16:12:18.515647+0000","last_became_active":"2023-10-08T16:12:18.514987+0000","last_became_peered":"2023-10-08T16:12:18.514987+0000","last_unstale":"2023-10-08T16:12:18.515647+0000","last_undegraded":"2023-10-08T16:12:18.515647+0000","last_fullsized":"2023-10-08T16:12:18.515647+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:00:01.181336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.033079+0000","last_change":"2023-10-08T16:12:18.015622+0000","last_active":"2023-10-08T16:12:19.033079+0000","last_peered":"2023-10-08T16:12:19.033079+0000","last_clean":"2023-10-08T16:12:19.033079+0000","last_became_active":"2023-10-08T16:12:18.014228+0000","last_became_peered":"2023-10-08T16:12:18.014228+0000","last_unstale":"2023-10-08T16:12:19.033079+0000","last_undegraded":"2023-10-08T16:12:19.033079+0000","last_fullsized":"2023-10-08T16:12:19.033079+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:04.370565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.518228+0000","last_change":"2023-10-08T16:12:18.518228+0000","last_active":"2023-10-08T16:12:18.518228+0000","last_peered":"2023-10-08T16:12:18.518228+0000","last_clean":"2023-10-08T16:12:18.518228+0000","last_became_active":"2023-10-08T16:12:18.517983+0000","last_became_peered":"2023-10-08T16:12:18.517983+0000","last_unstale":"2023-10-08T16:12:18.518228+0000","last_undegraded":"2023-10-08T16:12:18.518228+0000","last_fullsized":"2023-10-08T16:12:18.518228+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:31.310472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.508296+0000","last_change":"2023-10-08T16:12:18.014402+0000","last_active":"2023-10-08T16:12:18.508296+0000","last_peered":"2023-10-08T16:12:18.508296+0000","last_clean":"2023-10-08T16:12:18.508296+0000","last_became_active":"2023-10-08T16:12:18.014148+0000","last_became_peered":"2023-10-08T16:12:18.014148+0000","last_unstale":"2023-10-08T16:12:18.508296+0000","last_undegraded":"2023-10-08T16:12:18.508296+0000","last_fullsized":"2023-10-08T16:12:18.508296+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:03:43.885298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.514344+0000","last_change":"2023-10-08T16:12:18.514344+0000","last_active":"2023-10-08T16:12:18.514344+0000","last_peered":"2023-10-08T16:12:18.514344+0000","last_clean":"2023-10-08T16:12:18.514344+0000","last_became_active":"2023-10-08T16:12:18.514093+0000","last_became_peered":"2023-10-08T16:12:18.514093+0000","last_unstale":"2023-10-08T16:12:18.514344+0000","last_undegraded":"2023-10-08T16:12:18.514344+0000","last_fullsized":"2023-10-08T16:12:18.514344+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:53:02.337480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.013915+0000","last_change":"2023-10-08T16:12:18.013031+0000","last_active":"2023-10-08T16:12:18.013915+0000","last_peered":"2023-10-08T16:12:18.013915+0000","last_clean":"2023-10-08T16:12:18.013915+0000","last_became_active":"2023-10-08T16:12:18.012775+0000","last_became_peered":"2023-10-08T16:12:18.012775+0000","last_unstale":"2023-10-08T16:12:18.013915+0000","last_undegraded":"2023-10-08T16:12:18.013915+0000","last_fullsized":"2023-10-08T16:12:18.013915+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:45:48.230836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.513383+0000","last_change":"2023-10-08T16:12:18.513383+0000","last_active":"2023-10-08T16:12:18.513383+0000","last_peered":"2023-10-08T16:12:18.513383+0000","last_clean":"2023-10-08T16:12:18.513383+0000","last_became_active":"2023-10-08T16:12:18.513135+0000","last_became_peered":"2023-10-08T16:12:18.513135+0000","last_unstale":"2023-10-08T16:12:18.513383+0000","last_undegraded":"2023-10-08T16:12:18.513383+0000","last_fullsized":"2023-10-08T16:12:18.513383+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:01.341868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029504+0000","last_change":"2023-10-08T16:12:18.014104+0000","last_active":"2023-10-08T16:12:19.029504+0000","last_peered":"2023-10-08T16:12:19.029504+0000","last_clean":"2023-10-08T16:12:19.029504+0000","last_became_active":"2023-10-08T16:12:18.013740+0000","last_became_peered":"2023-10-08T16:12:18.013740+0000","last_unstale":"2023-10-08T16:12:19.029504+0000","last_undegraded":"2023-10-08T16:12:19.029504+0000","last_fullsized":"2023-10-08T16:12:19.029504+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:37.082523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517409+0000","last_change":"2023-10-08T16:12:18.517409+0000","last_active":"2023-10-08T16:12:18.517409+0000","last_peered":"2023-10-08T16:12:18.517409+0000","last_clean":"2023-10-08T16:12:18.517409+0000","last_became_active":"2023-10-08T16:12:18.517156+0000","last_became_peered":"2023-10-08T16:12:18.517156+0000","last_unstale":"2023-10-08T16:12:18.517409+0000","last_undegraded":"2023-10-08T16:12:18.517409+0000","last_fullsized":"2023-10-08T16:12:18.517409+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:07.856415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017678+0000","last_change":"2023-10-08T16:12:18.014802+0000","last_active":"2023-10-08T16:12:18.017678+0000","last_peered":"2023-10-08T16:12:18.017678+0000","last_clean":"2023-10-08T16:12:18.017678+0000","last_became_active":"2023-10-08T16:12:18.014559+0000","last_became_peered":"2023-10-08T16:12:18.014559+0000","last_unstale":"2023-10-08T16:12:18.017678+0000","last_undegraded":"2023-10-08T16:12:18.017678+0000","last_fullsized":"2023-10-08T16:12:18.017678+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:09.817436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517878+0000","last_change":"2023-10-08T16:12:18.517878+0000","last_active":"2023-10-08T16:12:18.517878+0000","last_peered":"2023-10-08T16:12:18.517878+0000","last_clean":"2023-10-08T16:12:18.517878+0000","last_became_active":"2023-10-08T16:12:18.517598+0000","last_became_peered":"2023-10-08T16:12:18.517598+0000","last_unstale":"2023-10-08T16:12:18.517878+0000","last_undegraded":"2023-10-08T16:12:18.517878+0000","last_fullsized":"2023-10-08T16:12:18.517878+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:25:11.194066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.013692+0000","last_change":"2023-10-08T16:12:18.012498+0000","last_active":"2023-10-08T16:12:18.013692+0000","last_peered":"2023-10-08T16:12:18.013692+0000","last_clean":"2023-10-08T16:12:18.013692+0000","last_became_active":"2023-10-08T16:12:18.012239+0000","last_became_peered":"2023-10-08T16:12:18.012239+0000","last_unstale":"2023-10-08T16:12:18.013692+0000","last_undegraded":"2023-10-08T16:12:18.013692+0000","last_fullsized":"2023-10-08T16:12:18.013692+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:47.610560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523242+0000","last_change":"2023-10-08T16:12:18.523242+0000","last_active":"2023-10-08T16:12:18.523242+0000","last_peered":"2023-10-08T16:12:18.523242+0000","last_clean":"2023-10-08T16:12:18.523242+0000","last_became_active":"2023-10-08T16:12:18.523002+0000","last_became_peered":"2023-10-08T16:12:18.523002+0000","last_unstale":"2023-10-08T16:12:18.523242+0000","last_undegraded":"2023-10-08T16:12:18.523242+0000","last_fullsized":"2023-10-08T16:12:18.523242+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:25:28.697116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.015115+0000","last_change":"2023-10-08T16:12:18.012304+0000","last_active":"2023-10-08T16:12:18.015115+0000","last_peered":"2023-10-08T16:12:18.015115+0000","last_clean":"2023-10-08T16:12:18.015115+0000","last_became_active":"2023-10-08T16:12:18.012056+0000","last_became_peered":"2023-10-08T16:12:18.012056+0000","last_unstale":"2023-10-08T16:12:18.015115+0000","last_undegraded":"2023-10-08T16:12:18.015115+0000","last_fullsized":"2023-10-08T16:12:18.015115+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.376610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520778+0000","last_change":"2023-10-08T16:12:18.520778+0000","last_active":"2023-10-08T16:12:18.520778+0000","last_peered":"2023-10-08T16:12:18.520778+0000","last_clean":"2023-10-08T16:12:18.520778+0000","last_became_active":"2023-10-08T16:12:18.520505+0000","last_became_peered":"2023-10-08T16:12:18.520505+0000","last_unstale":"2023-10-08T16:12:18.520778+0000","last_undegraded":"2023-10-08T16:12:18.520778+0000","last_fullsized":"2023-10-08T16:12:18.520778+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:36:17.251764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"153'2","reported_seq":11,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:19.029652+0000","last_change":"2023-10-08T16:12:18.015469+0000","last_active":"2023-10-08T16:12:19.029652+0000","last_peered":"2023-10-08T16:12:19.029652+0000","last_clean":"2023-10-08T16:12:19.029652+0000","last_became_active":"2023-10-08T16:12:18.015222+0000","last_became_peered":"2023-10-08T16:12:18.015222+0000","last_unstale":"2023-10-08T16:12:19.029652+0000","last_undegraded":"2023-10-08T16:12:19.029652+0000","last_fullsized":"2023-10-08T16:12:19.029652+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:10:42.462432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520974+0000","last_change":"2023-10-08T16:12:18.520974+0000","last_active":"2023-10-08T16:12:18.520974+0000","last_peered":"2023-10-08T16:12:18.520974+0000","last_clean":"2023-10-08T16:12:18.520974+0000","last_became_active":"2023-10-08T16:12:18.520711+0000","last_became_peered":"2023-10-08T16:12:18.520711+0000","last_unstale":"2023-10-08T16:12:18.520974+0000","last_undegraded":"2023-10-08T16:12:18.520974+0000","last_fullsized":"2023-10-08T16:12:18.520974+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:12.144723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.020817+0000","last_change":"2023-10-08T16:12:18.019461+0000","last_active":"2023-10-08T16:12:18.020817+0000","last_peered":"2023-10-08T16:12:18.020817+0000","last_clean":"2023-10-08T16:12:18.020817+0000","last_became_active":"2023-10-08T16:12:18.019100+0000","last_became_peered":"2023-10-08T16:12:18.019100+0000","last_unstale":"2023-10-08T16:12:18.020817+0000","last_undegraded":"2023-10-08T16:12:18.020817+0000","last_fullsized":"2023-10-08T16:12:18.020817+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:16:37.082523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.517461+0000","last_change":"2023-10-08T16:12:18.517461+0000","last_active":"2023-10-08T16:12:18.517461+0000","last_peered":"2023-10-08T16:12:18.517461+0000","last_clean":"2023-10-08T16:12:18.517461+0000","last_became_active":"2023-10-08T16:12:18.517174+0000","last_became_peered":"2023-10-08T16:12:18.517174+0000","last_unstale":"2023-10-08T16:12:18.517461+0000","last_undegraded":"2023-10-08T16:12:18.517461+0000","last_fullsized":"2023-10-08T16:12:18.517461+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:11.873458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.018318+0000","last_change":"2023-10-08T16:12:18.017586+0000","last_active":"2023-10-08T16:12:18.018318+0000","last_peered":"2023-10-08T16:12:18.018318+0000","last_clean":"2023-10-08T16:12:18.018318+0000","last_became_active":"2023-10-08T16:12:18.017355+0000","last_became_peered":"2023-10-08T16:12:18.017355+0000","last_unstale":"2023-10-08T16:12:18.018318+0000","last_undegraded":"2023-10-08T16:12:18.018318+0000","last_fullsized":"2023-10-08T16:12:18.018318+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:35:30.961323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520910+0000","last_change":"2023-10-08T16:12:18.520910+0000","last_active":"2023-10-08T16:12:18.520910+0000","last_peered":"2023-10-08T16:12:18.520910+0000","last_clean":"2023-10-08T16:12:18.520910+0000","last_became_active":"2023-10-08T16:12:18.520601+0000","last_became_peered":"2023-10-08T16:12:18.520601+0000","last_unstale":"2023-10-08T16:12:18.520910+0000","last_undegraded":"2023-10-08T16:12:18.520910+0000","last_fullsized":"2023-10-08T16:12:18.520910+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:26:36.071132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.017161+0000","last_change":"2023-10-08T16:12:18.015935+0000","last_active":"2023-10-08T16:12:18.017161+0000","last_peered":"2023-10-08T16:12:18.017161+0000","last_clean":"2023-10-08T16:12:18.017161+0000","last_became_active":"2023-10-08T16:12:18.015671+0000","last_became_peered":"2023-10-08T16:12:18.015671+0000","last_unstale":"2023-10-08T16:12:18.017161+0000","last_undegraded":"2023-10-08T16:12:18.017161+0000","last_fullsized":"2023-10-08T16:12:18.017161+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:01:10.935080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.522331+0000","last_change":"2023-10-08T16:12:18.522331+0000","last_active":"2023-10-08T16:12:18.522331+0000","last_peered":"2023-10-08T16:12:18.522331+0000","last_clean":"2023-10-08T16:12:18.522331+0000","last_became_active":"2023-10-08T16:12:18.522091+0000","last_became_peered":"2023-10-08T16:12:18.522091+0000","last_unstale":"2023-10-08T16:12:18.522331+0000","last_undegraded":"2023-10-08T16:12:18.522331+0000","last_fullsized":"2023-10-08T16:12:18.522331+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:18:50.451855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.014065+0000","last_change":"2023-10-08T16:12:18.012006+0000","last_active":"2023-10-08T16:12:18.014065+0000","last_peered":"2023-10-08T16:12:18.014065+0000","last_clean":"2023-10-08T16:12:18.014065+0000","last_became_active":"2023-10-08T16:12:18.011469+0000","last_became_peered":"2023-10-08T16:12:18.011469+0000","last_unstale":"2023-10-08T16:12:18.014065+0000","last_undegraded":"2023-10-08T16:12:18.014065+0000","last_fullsized":"2023-10-08T16:12:18.014065+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:16:07.875431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.523214+0000","last_change":"2023-10-08T16:12:18.523214+0000","last_active":"2023-10-08T16:12:18.523214+0000","last_peered":"2023-10-08T16:12:18.523214+0000","last_clean":"2023-10-08T16:12:18.523214+0000","last_became_active":"2023-10-08T16:12:18.522633+0000","last_became_peered":"2023-10-08T16:12:18.522633+0000","last_unstale":"2023-10-08T16:12:18.523214+0000","last_undegraded":"2023-10-08T16:12:18.523214+0000","last_fullsized":"2023-10-08T16:12:18.523214+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.903952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.015386+0000","last_change":"2023-10-08T16:12:18.011394+0000","last_active":"2023-10-08T16:12:18.015386+0000","last_peered":"2023-10-08T16:12:18.015386+0000","last_clean":"2023-10-08T16:12:18.015386+0000","last_became_active":"2023-10-08T16:12:18.011140+0000","last_became_peered":"2023-10-08T16:12:18.011140+0000","last_unstale":"2023-10-08T16:12:18.015386+0000","last_undegraded":"2023-10-08T16:12:18.015386+0000","last_fullsized":"2023-10-08T16:12:18.015386+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:47:50.254850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.524498+0000","last_change":"2023-10-08T16:12:18.524498+0000","last_active":"2023-10-08T16:12:18.524498+0000","last_peered":"2023-10-08T16:12:18.524498+0000","last_clean":"2023-10-08T16:12:18.524498+0000","last_became_active":"2023-10-08T16:12:18.524263+0000","last_became_peered":"2023-10-08T16:12:18.524263+0000","last_unstale":"2023-10-08T16:12:18.524498+0000","last_undegraded":"2023-10-08T16:12:18.524498+0000","last_fullsized":"2023-10-08T16:12:18.524498+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:33:14.902370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019912+0000","last_change":"2023-10-08T16:12:18.017394+0000","last_active":"2023-10-08T16:12:18.019912+0000","last_peered":"2023-10-08T16:12:18.019912+0000","last_clean":"2023-10-08T16:12:18.019912+0000","last_became_active":"2023-10-08T16:12:18.017142+0000","last_became_peered":"2023-10-08T16:12:18.017142+0000","last_unstale":"2023-10-08T16:12:18.019912+0000","last_undegraded":"2023-10-08T16:12:18.019912+0000","last_fullsized":"2023-10-08T16:12:18.019912+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.376610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.521419+0000","last_change":"2023-10-08T16:12:18.521419+0000","last_active":"2023-10-08T16:12:18.521419+0000","last_peered":"2023-10-08T16:12:18.521419+0000","last_clean":"2023-10-08T16:12:18.521419+0000","last_became_active":"2023-10-08T16:12:18.521085+0000","last_became_peered":"2023-10-08T16:12:18.521085+0000","last_unstale":"2023-10-08T16:12:18.521419+0000","last_undegraded":"2023-10-08T16:12:18.521419+0000","last_fullsized":"2023-10-08T16:12:18.521419+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:42.903952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.019383+0000","last_change":"2023-10-08T16:12:18.016264+0000","last_active":"2023-10-08T16:12:18.019383+0000","last_peered":"2023-10-08T16:12:18.019383+0000","last_clean":"2023-10-08T16:12:18.019383+0000","last_became_active":"2023-10-08T16:12:18.016003+0000","last_became_peered":"2023-10-08T16:12:18.016003+0000","last_unstale":"2023-10-08T16:12:18.019383+0000","last_undegraded":"2023-10-08T16:12:18.019383+0000","last_fullsized":"2023-10-08T16:12:18.019383+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:36:06.852372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":8,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.520937+0000","last_change":"2023-10-08T16:12:18.520937+0000","last_active":"2023-10-08T16:12:18.520937+0000","last_peered":"2023-10-08T16:12:18.520937+0000","last_clean":"2023-10-08T16:12:18.520937+0000","last_became_active":"2023-10-08T16:12:18.520676+0000","last_became_peered":"2023-10-08T16:12:18.520676+0000","last_unstale":"2023-10-08T16:12:18.520937+0000","last_undegraded":"2023-10-08T16:12:18.520937+0000","last_fullsized":"2023-10-08T16:12:18.520937+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:17.996841+0000","last_clean_scrub_stamp":"2023-10-08T16:12:17.996841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:00.489669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"153'1","reported_seq":10,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.519087+0000","last_change":"2023-10-08T16:12:18.015971+0000","last_active":"2023-10-08T16:12:18.519087+0000","last_peered":"2023-10-08T16:12:18.519087+0000","last_clean":"2023-10-08T16:12:18.519087+0000","last_became_active":"2023-10-08T16:12:18.015449+0000","last_became_peered":"2023-10-08T16:12:18.015449+0000","last_unstale":"2023-10-08T16:12:18.519087+0000","last_undegraded":"2023-10-08T16:12:18.519087+0000","last_fullsized":"2023-10-08T16:12:18.519087+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:16.992798+0000","last_clean_scrub_stamp":"2023-10-08T16:12:16.992798+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:33:13.898328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":273,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-10-08T16:12:18.007454+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:12:18.007454+0000","last_peered":"2023-10-08T16:12:18.007454+0000","last_clean":"2023-10-08T16:12:18.007454+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:12:18.007454+0000","last_undegraded":"2023-10-08T16:12:18.007454+0000","last_fullsized":"2023-10-08T16:12:18.007454+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52300000000000002,"15min":0.51600000000000001},"min":{"1min":0.39500000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.77600000000000002,"15min":0.86199999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51200000000000001,"15min":0.754},"min":{"1min":0.38,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.72899999999999998,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.58099999999999996}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.33600000000000002,"15min":0.29199999999999998},"max":{"1min":0.71399999999999997,"5min":0.82199999999999995,"15min":1.083},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.78500000000000003},"min":{"1min":0.38,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.42599999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.49399999999999999,"15min":0.53600000000000003},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.626,"5min":0.70299999999999996,"15min":1.175},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.52800000000000002,"15min":0.80900000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.53100000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53300000000000003,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.34499999999999997},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.53200000000000003,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.66100000000000003,"5min":0.71399999999999997,"15min":8.3409999999999993},"last":0.46999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.48599999999999999},"min":{"1min":0.314,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54800000000000004},"min":{"1min":0.41499999999999998,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.54200000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71199999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53000000000000003,"15min":0.55400000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57199999999999995,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56299999999999994,"15min":0.58299999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71499999999999997,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496}]}]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.56799999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.502},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.52000000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.53900000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.59899999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.63},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.67400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303284,"kb_used_data":1900,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068556,"statfs":{"total":96636764160,"available":96326201344,"internally_reserved":0,"allocated":1945600,"data_stored":867089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.36899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.503}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.442},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.314}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.628}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2531328,"data_stored":1457401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.189},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.23000000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.29299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.308},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.32400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.437},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.51200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.42499999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39700000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.53200000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29899999999999999,"15min":0.30599999999999999},"min":{"1min":0.23400000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.36799999999999999,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.315,"5min":0.31900000000000001,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.189,"15min":0.189},"max":{"1min":0.40899999999999997,"5min":0.49299999999999999,"15min":0.64300000000000002},"last":0.52500000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.34799999999999998},"min":{"1min":0.23000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.44,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.34399999999999997,"15min":0.35799999999999998},"min":{"1min":0.28699999999999998,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.46200000000000002,"5min":0.54300000000000004,"15min":0.68300000000000005},"last":0.47399999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.32300000000000001,"15min":0.34399999999999997},"min":{"1min":0.25,"5min":0.221,"15min":0.221},"max":{"1min":0.432,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.371,"15min":0.38500000000000001},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.46800000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.38400000000000001,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.313,"5min":0.254,"15min":0.254},"max":{"1min":0.71099999999999997,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.38500000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.71499999999999997,"5min":0.82199999999999995,"15min":0.85899999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55500000000000005,"15min":0.65900000000000003},"min":{"1min":0.375,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.69499999999999995,"5min":0.90700000000000003,"15min":4.1029999999999998},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78900000000000003,"5min":0.91500000000000004,"15min":0.92900000000000005},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82099999999999995,"5min":0.85199999999999998,"15min":0.871},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.59299999999999997},"min":{"1min":0.378,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82399999999999995,"5min":0.93999999999999995,"15min":0.96899999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80100000000000005,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.98999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303288,"kb_used_data":1904,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068552,"statfs":{"total":96636764160,"available":96326197248,"internally_reserved":0,"allocated":1949696,"data_stored":867215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.36099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.34999999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.375}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.66400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.68600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.69699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574946,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303276,"kb_used_data":1892,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068564,"statfs":{"total":96636764160,"available":96326209536,"internally_reserved":0,"allocated":1937408,"data_stored":867033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.52900000000000003},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.65200000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.748,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.50700000000000001,"15min":0.49199999999999999},"min":{"1min":0.33000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.41399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.80700000000000005},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.58299999999999996},"min":{"1min":0.34999999999999998,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.83099999999999996,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.624},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.46700000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56100000000000005,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.498,"15min":0.52300000000000002},"min":{"1min":0.33600000000000002,"5min":0.31,"15min":0.30499999999999999},"max":{"1min":0.67200000000000004,"5min":0.73099999999999998,"15min":0.77100000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.55900000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.70599999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.53500000000000003,"15min":0.55900000000000005},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56100000000000005,"15min":0.58499999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.34300000000000003},"max":{"1min":0.752,"5min":0.78700000000000003,"15min":0.83399999999999996},"last":0.65600000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:11:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57399999999999995,"15min":0.60399999999999998},"min":{"1min":0.35699999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59699999999999998,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.89200000000000002},"last":0.71699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303264,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068576,"statfs":{"total":96636764160,"available":96326221824,"internally_reserved":0,"allocated":1925120,"data_stored":860270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.63600000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.67300000000000004}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.436}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.503}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:12:23.454 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:12:23.454 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:12:23.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.610+0000 7ff2db25f700 1 Processor -- start 2023-10-08T16:12:23.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.610+0000 7ff2db25f700 1 -- start start 2023-10-08T16:12:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2db25f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff2d4159950 0x7ff2d40cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2db25f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2db25f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d40ce380 con 0x7ff2d4069aa0 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2db25f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d4069f20 con 0x7ff2d4159950 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2db25f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40cec20 con 0x7ff2d40af320 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:48504/0 (socket says 172.21.15.170:48504) 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2d8ffb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff2d4159950 0x7ff2d40cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.611+0000 7ff2cbfff700 1 -- 172.21.15.170:0/3660050260 learned_addr learned my addr 172.21.15.170:0/3660050260 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cb7fe700 1 -- 172.21.15.170:0/3660050260 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554152491 0 0) 0x7ff2d40ce380 con 0x7ff2d4069aa0 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cb7fe700 1 -- 172.21.15.170:0/3660050260 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2bc003570 con 0x7ff2d4069aa0 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2d8ffb700 1 -- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 msgr2=0x7ff2d4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2d8ffb700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2d8ffb700 1 -- 172.21.15.170:0/3660050260 >> v1:172.21.15.191:6789/0 conn(0x7ff2d4069aa0 legacy=0x7ff2d415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2d8ffb700 1 -- 172.21.15.170:0/3660050260 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40afa90 con 0x7ff2d4159950 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cbfff700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2d8ffb700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40cbb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009e60 tx=0x7ff2d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c616980ad26f80f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cb7fe700 1 -- 172.21.15.170:0/3660050260 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d000e070 con 0x7ff2d4159950 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cb7fe700 1 -- 172.21.15.170:0/3660050260 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d000bab0 con 0x7ff2d4159950 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.612+0000 7ff2cb7fe700 1 -- 172.21.15.170:0/3660050260 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d000be30 con 0x7ff2d4159950 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 -- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 msgr2=0x7ff2d40cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40cbb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009e60 tx=0x7ff2d0009780 comp rx=0 tx=0).stop 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 -- 172.21.15.170:0/3660050260 shutdown_connections 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40cbb90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3660050260 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d40af320 0x7ff2d4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 -- 172.21.15.170:0/3660050260 >> 172.21.15.170:0/3660050260 conn(0x7ff2d4068e00 msgr2=0x7ff2d40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 -- 172.21.15.170:0/3660050260 shutdown_connections 2023-10-08T16:12:23.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.613+0000 7ff2db25f700 1 -- 172.21.15.170:0/3660050260 wait complete. 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.614+0000 7ff2db25f700 1 Processor -- start 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.614+0000 7ff2db25f700 1 -- start start 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d4069aa0 0x7ff2d40ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b56d0 con 0x7ff2d40af320 2023-10-08T16:12:23.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40b5b60 con 0x7ff2d4159950 2023-10-08T16:12:23.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d40ca760 con 0x7ff2d4069aa0 2023-10-08T16:12:23.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44528/0 (socket says 172.21.15.170:44528) 2023-10-08T16:12:23.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 -- 172.21.15.170:0/3009187056 learned_addr learned my addr 172.21.15.170:0/3009187056 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2d8ffb700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d4069aa0 0x7ff2d40ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2d97fc700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 -- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d4069aa0 msgr2=0x7ff2d40ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d4069aa0 0x7ff2d40ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 -- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 msgr2=0x7ff2d40c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 -- 172.21.15.170:0/3009187056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c40094f0 con 0x7ff2d40af320 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2cbfff700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ec50 tx=0x7ff2c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400c850 con 0x7ff2d40af320 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c400c9b0 con 0x7ff2d40af320 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c400cd50 con 0x7ff2d40af320 2023-10-08T16:12:23.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d0009a20 con 0x7ff2d40af320 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.615+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d40b4e20 con 0x7ff2d40af320 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.616+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d40cabc0 con 0x7ff2d40af320 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.617+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff2c4013680 con 0x7ff2d40af320 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.617+0000 7ff2c8ff9700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 0x7ff2c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.617+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff2c4084fa0 con 0x7ff2d40af320 2023-10-08T16:12:23.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.617+0000 7ff2d8ffb700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 0x7ff2c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.618+0000 7ff2d8ffb700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 0x7ff2c00697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00097c0 tx=0x7ff2d0019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:23.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.622+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2c4051c30 con 0x7ff2d40af320 2023-10-08T16:12:23.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.880+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff2d40718b0 con 0x7ff2d40af320 2023-10-08T16:12:23.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.884+0000 7ff2c8ff9700 1 -- 172.21.15.170:0/3009187056 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff2c4055250 con 0x7ff2d40af320 2023-10-08T16:12:23.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 msgr2=0x7ff2c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 0x7ff2c00697d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00097c0 tx=0x7ff2d0019000 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 msgr2=0x7ff2d414d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff2c400ec50 tx=0x7ff2c4009250 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 shutdown_connections 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff2d4159950 0x7ff2d40c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff2d4069aa0 0x7ff2d40ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff2c0067320 0x7ff2c00697d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 --2- 172.21.15.170:0/3009187056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2d40af320 0x7ff2d414d550 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 >> 172.21.15.170:0/3009187056 conn(0x7ff2d4068e00 msgr2=0x7ff2d4153810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 shutdown_connections 2023-10-08T16:12:23.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:23.886+0000 7ff2db25f700 1 -- 172.21.15.170:0/3009187056 wait complete. 2023-10-08T16:12:23.888 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:12:23.898 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:23.992 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:12:23.992+0000 7fdb92b17700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:12:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.011+0000 7f830529c700 1 Processor -- start 2023-10-08T16:12:24.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.011+0000 7f830529c700 1 -- start start 2023-10-08T16:12:24.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f830529c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8300069d60 0x7f83000c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f830529c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83000c81f0 0x7f83000cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f830529c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000c8650 con 0x7f83000aca80 2023-10-08T16:12:24.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f830529c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830006bdb0 con 0x7f8300069d60 2023-10-08T16:12:24.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f830529c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000cb370 con 0x7f83000c81f0 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f82f67fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8300069d60 0x7f83000c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f82f67fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8300069d60 0x7f83000c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44532/0 (socket says 172.21.15.170:44532) 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f82f67fc700 1 -- 172.21.15.170:0/2076696695 learned_addr learned my addr 172.21.15.170:0/2076696695 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.012+0000 7f82feffd700 1 --1- 172.21.15.170:0/2076696695 >> v1:172.21.15.170:6789/0 conn(0x7f83000aca80 0x7f830006aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35302/0 (socket says 172.21.15.170:35302) 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82ff7fe700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83000c81f0 0x7f83000cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82f67fc700 1 -- 172.21.15.170:0/2076696695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83000c81f0 msgr2=0x7f83000cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82f67fc700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83000c81f0 0x7f83000cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82f67fc700 1 -- 172.21.15.170:0/2076696695 >> v1:172.21.15.170:6789/0 conn(0x7f83000aca80 legacy=0x7f830006aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82fe7fc700 1 -- 172.21.15.170:0/2076696695 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921840751 0 0) 0x7f82f00011d0 con 0x7f83000aca80 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82f67fc700 1 -- 172.21.15.170:0/2076696695 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300159f60 con 0x7f8300069d60 2023-10-08T16:12:24.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82f67fc700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8300069d60 0x7f83000c7cb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099b0 tx=0x7f82e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26a0926b5580c6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82fe7fc700 1 -- 172.21.15.170:0/2076696695 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800b860 con 0x7f8300069d60 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82fe7fc700 1 -- 172.21.15.170:0/2076696695 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800b9c0 con 0x7f8300069d60 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f82fe7fc700 1 -- 172.21.15.170:0/2076696695 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800bd60 con 0x7f8300069d60 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f830529c700 1 -- 172.21.15.170:0/2076696695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8300069d60 msgr2=0x7f83000c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.013+0000 7f830529c700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8300069d60 0x7f83000c7cb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f82e80099b0 tx=0x7f82e8009250 comp rx=0 tx=0).stop 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 -- 172.21.15.170:0/2076696695 shutdown_connections 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f83000c81f0 0x7f83000cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 --2- 172.21.15.170:0/2076696695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8300069d60 0x7f83000c7cb0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 -- 172.21.15.170:0/2076696695 >> 172.21.15.170:0/2076696695 conn(0x7f83000cd670 msgr2=0x7f83000cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 -- 172.21.15.170:0/2076696695 shutdown_connections 2023-10-08T16:12:24.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.014+0000 7f830529c700 1 -- 172.21.15.170:0/2076696695 wait complete. 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.015+0000 7f830529c700 1 Processor -- start 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.015+0000 7f830529c700 1 -- start start 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8300069d60 0x7f83000af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83000aca80 0x7f830016b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000ad780 con 0x7f83000c81f0 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830016b090 con 0x7f83000aca80 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000ace50 con 0x7f8300069d60 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44538/0 (socket says 172.21.15.170:44538) 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 -- 172.21.15.170:0/4279410541 learned_addr learned my addr 172.21.15.170:0/4279410541 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82feffd700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8300069d60 0x7f83000af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82f67fc700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83000aca80 0x7f830016b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 -- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8300069d60 msgr2=0x7f83000af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8300069d60 0x7f83000af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 -- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83000aca80 msgr2=0x7f830016b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83000aca80 0x7f830016b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 -- 172.21.15.170:0/4279410541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80094f0 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82ff7fe700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007f20 tx=0x7f82f8009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f800eb90 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f800ed10 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000ad130 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f8014670 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.016+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83000b2290 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.017+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830004c3c0 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.018+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f82f80149e0 con 0x7f83000c81f0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.018+0000 7f82f77fe700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.018+0000 7f82feffd700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.018+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f82f80851f0 con 0x7f83000c81f0 2023-10-08T16:12:24.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.019+0000 7f82feffd700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 0x7f82e00697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f830006e3d0 tx=0x7f82f0007ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.023+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82f8051e80 con 0x7f83000c81f0 2023-10-08T16:12:24.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.249+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83000ad8c0 con 0x7f83000c81f0 2023-10-08T16:12:24.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.250+0000 7f82f77fe700 1 -- 172.21.15.170:0/4279410541 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f82f80554a0 con 0x7f83000c81f0 2023-10-08T16:12:24.250 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:24.250 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":100,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:24.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 msgr2=0x7f82e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 0x7f82e00697d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f830006e3d0 tx=0x7f82f0007ac0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 msgr2=0x7f83000b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f82f8007f20 tx=0x7f82f8009860 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 shutdown_connections 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f82e0067320 0x7f82e00697d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83000c81f0 0x7f83000b1d50 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8300069d60 0x7f83000af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 --2- 172.21.15.170:0/4279410541 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f83000aca80 0x7f830016b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 >> 172.21.15.170:0/4279410541 conn(0x7f83000cd670 msgr2=0x7f8300157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 shutdown_connections 2023-10-08T16:12:24.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.252+0000 7f830529c700 1 -- 172.21.15.170:0/4279410541 wait complete. 2023-10-08T16:12:24.254 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 100 2023-10-08T16:12:24.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:12:18.390075+0000', 'modified': '2023-10-08T16:12:22.040533+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7857}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7857': {'gid': 7857, 'name': 'c', 'rank': 0, 'incarnation': 96, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/1646734631', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 1646734631}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1646734631}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:12:24.667 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:12:24.667 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T16:12:24.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.783+0000 7fca8e8d8700 1 Processor -- start 2023-10-08T16:12:24.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.783+0000 7fca8e8d8700 1 -- start start 2023-10-08T16:12:24.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8e8d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca880ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8e8d8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca880cefe0 con 0x7fca880b1e50 2023-10-08T16:12:24.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8e8d8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca880cfe20 con 0x7fca881593b0 2023-10-08T16:12:24.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8e8d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880cf210 con 0x7fca88159780 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8d0d5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fca880b1e50 0x7fca880b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35310/0 (socket says 172.21.15.170:35310) 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8d0d5700 1 -- 172.21.15.170:0/1008121786 learned_addr learned my addr 172.21.15.170:0/1008121786 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca880ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411361276 0 0) 0x7fca880cefe0 con 0x7fca880b1e50 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.784+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca74003570 con 0x7fca880b1e50 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416012711 0 0) 0x7fca74003570 con 0x7fca880b1e50 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 msgr2=0x7fca880ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca880ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 >> v1:172.21.15.191:6789/0 conn(0x7fca881593b0 legacy=0x7fca880b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880cf4f0 con 0x7fca880b1e50 2023-10-08T16:12:24.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fca78001470 con 0x7fca880b1e50 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 >> v1:172.21.15.170:6789/0 conn(0x7fca880b1e50 legacy=0x7fca880b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca74003a90 0x7fca74003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca74004d50 0x7fca740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca880cf210 con 0x7fca74003a90 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7400bb40 con 0x7fca74004d50 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7400bd00 con 0x7fca88159780 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca74003a90 0x7fca74003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca74004d50 0x7fca740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 -- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 msgr2=0x7fca74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca88159780 0x7fca74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 -- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca74004d50 msgr2=0x7fca740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca74004d50 0x7fca740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 -- 172.21.15.170:0/1008121786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca74003570 con 0x7fca74003a90 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca74004d50 0x7fca740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca74003a90 0x7fca74003e60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca840020a0 tx=0x7fca840016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c51430f05000d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840019e0 con 0x7fca74003a90 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8400c8b0 con 0x7fca74003a90 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.785+0000 7fca8c8d4700 1 -- 172.21.15.170:0/1008121786 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400cc50 con 0x7fca74003a90 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1008121786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca74003a90 msgr2=0x7fca74003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1008121786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca74003a90 0x7fca74003e60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fca840020a0 tx=0x7fca840016d0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1008121786 reap_dead start 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1008121786 shutdown_connections 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1008121786 >> 172.21.15.170:0/1008121786 conn(0x7fca880d1130 msgr2=0x7fca880b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1008121786 shutdown_connections 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.786+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1008121786 wait complete. 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.787+0000 7fca8e8d8700 1 Processor -- start 2023-10-08T16:12:24.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.787+0000 7fca8e8d8700 1 -- start start 2023-10-08T16:12:24.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca74003a90 0x7fca880c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 0x7fca88156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8815f230 con 0x7fca8815e870 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca880c96d0 con 0x7fca8815a7a0 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e8d8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880c7670 con 0x7fca74003a90 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44570/0 (socket says 172.21.15.170:44570) 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1263327072 learned_addr learned my addr 172.21.15.170:0/1263327072 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 0x7fca88156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca74003a90 0x7fca880c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca74003a90 msgr2=0x7fca880c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca74003a90 0x7fca880c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 msgr2=0x7fca88156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 0x7fca88156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1263327072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca880c7950 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 0x7fca88156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.788+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fca8806b660 tx=0x7fca80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.789+0000 7fca7dffb700 1 -- 172.21.15.170:0/1263327072 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80015740 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.789+0000 7fca7dffb700 1 -- 172.21.15.170:0/1263327072 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca800158a0 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.789+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca880c7c30 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.789+0000 7fca7dffb700 1 -- 172.21.15.170:0/1263327072 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80015c60 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.789+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880c7360 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.790+0000 7fca7dffb700 1 -- 172.21.15.170:0/1263327072 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fca80020460 con 0x7fca8815e870 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.791+0000 7fca7dffb700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 0x7fca6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.791+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 0x7fca6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.791+0000 7fca7dffb700 1 -- 172.21.15.170:0/1263327072 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca80084dd0 con 0x7fca8815e870 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.791+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 0x7fca6c0697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca840020a0 tx=0x7fca84016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 Processor -- start 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 -- start start 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079ce87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cbeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079c15a70 con 0x561079cbc5f0 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079c13270 con 0x561079cbeb80 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561079c0f060 con 0x561079d09480 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca8d0d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca8d0d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44582/0 (socket says 172.21.15.170:44582) 2023-10-08T16:12:24.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.793+0000 7fca8d0d5700 1 -- 172.21.15.170:0/3899721387 learned_addr learned my addr 172.21.15.170:0/3899721387 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079ce87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cbeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 -- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 msgr2=0x561079ce87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079ce87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 -- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 msgr2=0x561079cbeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cbeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 -- 172.21.15.170:0/3899721387 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561079aa9270 con 0x561079cbc5f0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fca78007ce0 tx=0x7fca78009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd702ec92a88ae8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca72ffd700 1 -- 172.21.15.170:0/3899721387 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7801a070 con 0x561079cbc5f0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca72ffd700 1 -- 172.21.15.170:0/3899721387 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca78007390 con 0x561079cbc5f0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.794+0000 7fca72ffd700 1 -- 172.21.15.170:0/3899721387 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78007750 con 0x561079cbc5f0 2023-10-08T16:12:24.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- 172.21.15.170:0/3899721387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 msgr2=0x561079d09850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fca78007ce0 tx=0x7fca78009040 comp rx=0 tx=0).stop 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- 172.21.15.170:0/3899721387 shutdown_connections 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cbeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079ce87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 --2- 172.21.15.170:0/3899721387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079d09850 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- 172.21.15.170:0/3899721387 >> 172.21.15.170:0/3899721387 conn(0x561079cc00c0 msgr2=0x561079cc08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- 172.21.15.170:0/3899721387 shutdown_connections 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- 172.21.15.170:0/3899721387 wait complete. 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 -- start start 2023-10-08T16:12:24.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.795+0000 7fca98e32b80 1 Processor -- start 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079d08a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079d18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079bba890 con 0x561079cbc5f0 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079bd5ea0 con 0x561079cbeb80 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561079a91e30 con 0x561079d09480 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca98e32b80 10 client.0 _ll_register_callbacks cb 0x561079c822c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44592/0 (socket says 172.21.15.170:44592) 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 -- 172.21.15.170:0/2207039807 learned_addr learned my addr 172.21.15.170:0/2207039807 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 -- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 msgr2=0x561079d18fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079d18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 -- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 msgr2=0x561079d08a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079d08a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.796+0000 7fca8d8d6700 1 -- 172.21.15.170:0/2207039807 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880cf4f0 con 0x561079cbc5f0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fca8401e7b0 tx=0x7fca84020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bf11250502a3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84034470 con 0x561079cbc5f0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca840345d0 con 0x561079cbc5f0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561079cb3cd0 con 0x561079cbc5f0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca840349a0 con 0x561079cbc5f0 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 10 client.8010 fetch_fsmap learned FSMap version 100 2023-10-08T16:12:24.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x561079a4bfe0 con 0x561079cbc5f0 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561079cb3cd0 con 0x561079cbc5f0 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca84034b00 con 0x561079cbc5f0 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 10 client.8010 fetch_fsmap finished waiting for FSMap version 100 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 10 client.8010 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 20 client.8010 populate_metadata read hostname 'smithi170' 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079cd7290 0x561079cd7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 4 client.8010 mds_command: new command op to 7857 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 --> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561079c90aa0 con 0x561079cd7290 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.797+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079cd7290 0x561079cd7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca67fff700 10 client.8010 ms_handle_connect on v2:172.21.15.170:6835/1646734631 2023-10-08T16:12:24.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079cd7290 0x561079cd7660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba133a59206d766e server_cookie=4743f8156199c4c4 in_seq=0 out_seq=0 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca67fff700 1 -- 172.21.15.170:0/2207039807 <== mds.0 v2:172.21.15.170:6835/1646734631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561079c90aa0 con 0x561079cd7290 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca67fff700 10 client.8010 handle_command_reply: tid=0 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 client.8010 shutdown 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 msgr2=0x561079ced8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fca8401e7b0 tx=0x7fca84020040 comp rx=0 tx=0).stop 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 shutdown_connections 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079d09480 0x561079d18fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079cbc5f0 0x561079ced8a0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079cd7290 0x561079cd7660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2207039807 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079d08a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca67fff700 0 client.8010 ms_handle_reset on v2:172.21.15.170:6835/1646734631 2023-10-08T16:12:24.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 >> 172.21.15.170:0/2207039807 conn(0x561079cc00c0 msgr2=0x561079cbae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 shutdown_connections 2023-10-08T16:12:24.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.798+0000 7fca98e32b80 1 -- 172.21.15.170:0/2207039807 wait complete. 2023-10-08T16:12:24.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.799+0000 7fca98e32b80 20 client.8010 trim_cache size 0 max 16384 2023-10-08T16:12:24.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.815+0000 7fca98e32b80 1 Processor -- start 2023-10-08T16:12:24.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.815+0000 7fca98e32b80 1 -- start start 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbc5f0 0x561079ce97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079d07600 con 0x561079d09480 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079d27f20 con 0x561079cbeb80 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561079cee8b0 con 0x561079cbc5f0 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44602/0 (socket says 172.21.15.170:44602) 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 -- 172.21.15.170:0/2526002757 learned_addr learned my addr 172.21.15.170:0/2526002757 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 -- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbc5f0 msgr2=0x561079ce97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbc5f0 0x561079ce97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 -- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 msgr2=0x561079cc1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.816+0000 7fca8e0d7700 1 -- 172.21.15.170:0/2526002757 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561079bdd780 con 0x561079d09480 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fca80001eb0 tx=0x7fca80006cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b87791ea824f263 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca737fe700 1 -- 172.21.15.170:0/2526002757 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80055de0 con 0x561079d09480 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca737fe700 1 -- 172.21.15.170:0/2526002757 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca800556e0 con 0x561079d09480 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca737fe700 1 -- 172.21.15.170:0/2526002757 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80058930 con 0x561079d09480 2023-10-08T16:12:24.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 -- 172.21.15.170:0/2526002757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 msgr2=0x561079d126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fca80001eb0 tx=0x7fca80006cb0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 -- 172.21.15.170:0/2526002757 shutdown_connections 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079d126a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbc5f0 0x561079ce97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 --2- 172.21.15.170:0/2526002757 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 -- 172.21.15.170:0/2526002757 >> 172.21.15.170:0/2526002757 conn(0x561079cc00c0 msgr2=0x561079ce8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 -- 172.21.15.170:0/2526002757 shutdown_connections 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.817+0000 7fca98e32b80 1 -- 172.21.15.170:0/2526002757 wait complete. 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.818+0000 7fca98e32b80 1 -- start start 2023-10-08T16:12:24.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.818+0000 7fca98e32b80 1 Processor -- start 2023-10-08T16:12:24.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 0x561079d6be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079d1bf40 con 0x561079d09480 2023-10-08T16:12:24.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561079d1b2c0 con 0x561079cbc5f0 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561079cb4fd0 con 0x561079cbeb80 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 10 client.0 _ll_register_callbacks cb 0x561079c822c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44608/0 (socket says 172.21.15.170:44608) 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1721661821 learned_addr learned my addr 172.21.15.170:0/1721661821 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 0x561079d6be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 msgr2=0x561079cc1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 msgr2=0x561079d6be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 0x561079d6be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 -- 172.21.15.170:0/1721661821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880c7360 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 0x561079d6be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8d0d5700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca8e0d7700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fca800553e0 tx=0x7fca80068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d448fc4eb638c614 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca67fff700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80058930 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fca80071200 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.819+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561079d155a0 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8005c400 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 10 client.8016 fetch_fsmap learned FSMap version 100 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x561079bc4470 con 0x561079d09480 2023-10-08T16:12:24.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561079d155a0 con 0x561079d09480 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 100) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fca800514d0 con 0x561079d09480 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 10 client.8016 fetch_fsmap finished waiting for FSMap version 100 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 10 client.8016 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 20 client.8016 populate_metadata read hostname 'smithi170' 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079d6f570 0x561079d6f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 4 client.8016 mds_command: new command op to 7857 tid=0[{"prefix": "status"}] 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 --> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fca78007390 con 0x561079d6f570 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079d6f570 0x561079d6f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 10 client.8016 ms_handle_connect on v2:172.21.15.170:6835/1646734631 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca8d8d6700 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079d6f570 0x561079d6f940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4fde876df90bbb server_cookie=8196fcf53918ce4a in_seq=0 out_seq=0 2023-10-08T16:12:24.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 1 -- 172.21.15.170:0/1721661821 <== mds.0 v2:172.21.15.170:6835/1646734631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x7fca78007390 con 0x561079d6f570 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca67fff700 10 client.8016 handle_command_reply: tid=0 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 client.8016 shutdown 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 msgr2=0x561079cc2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fca800553e0 tx=0x7fca80068000 comp rx=0 tx=0).stop 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 shutdown_connections 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561079d09480 0x561079cc2360 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.820+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.170:6835/1646734631,v1:172.21.15.170:6837/1646734631] conn(0x561079d6f570 0x561079d6f940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561079cbc5f0 0x561079d6be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca67fff700 0 client.8016 ms_handle_reset on v2:172.21.15.170:6835/1646734631 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 1 --2- 172.21.15.170:0/1721661821 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561079cbeb80 0x561079cc1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 >> 172.21.15.170:0/1721661821 conn(0x561079cc00c0 msgr2=0x561079ce8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 shutdown_connections 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 1 -- 172.21.15.170:0/1721661821 wait complete. 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca98e32b80 20 client.8016 trim_cache size 0 max 16384 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:12:24.830 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 7857, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.797789012, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 100, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 154, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 154, 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 11.656973228 2023-10-08T16:12:24.831 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:12:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 msgr2=0x7fca6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 0x7fca6c0697d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca840020a0 tx=0x7fca84016000 comp rx=0 tx=0).stop 2023-10-08T16:12:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 msgr2=0x7fca8815ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fca8806b660 tx=0x7fca80008040 comp rx=0 tx=0).stop 2023-10-08T16:12:24.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 shutdown_connections 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fca6c067320 0x7fca6c0697d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fca8815e870 0x7fca8815ecf0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fca74003a90 0x7fca880c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 --2- 172.21.15.170:0/1263327072 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fca8815a7a0 0x7fca88156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 >> 172.21.15.170:0/1263327072 conn(0x7fca880d1130 msgr2=0x7fca880cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 shutdown_connections 2023-10-08T16:12:24.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.821+0000 7fca8e8d8700 1 -- 172.21.15.170:0/1263327072 wait complete. 2023-10-08T16:12:24.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:12:24.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.949+0000 7f1f0f566700 1 Processor -- start 2023-10-08T16:12:24.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.949+0000 7f1f0f566700 1 -- start start 2023-10-08T16:12:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0f566700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0f566700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1f080c81f0 0x7f1f080cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0f566700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0806bdb0 con 0x7f1f080aca80 2023-10-08T16:12:24.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0f566700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080c8670 con 0x7f1f080c81f0 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0f566700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080cb370 con 0x7f1f08069d60 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0d302700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1f080aca80 0x7f1f0806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35318/0 (socket says 172.21.15.170:35318) 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.950+0000 7f1f0d302700 1 -- 172.21.15.170:0/2263175749 learned_addr learned my addr 172.21.15.170:0/2263175749 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 --2- 172.21.15.170:0/2263175749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0db03700 1 --2- 172.21.15.170:0/2263175749 >> v2:172.21.15.191:3300/0 conn(0x7f1f080c81f0 0x7f1f080cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475423527 0 0) 0x7f1f0806bdb0 con 0x7f1f080aca80 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef0003570 con 0x7f1f080aca80 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 -- 172.21.15.170:0/2263175749 >> v2:172.21.15.191:3300/0 conn(0x7f1f080c81f0 msgr2=0x7f1f080cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 --2- 172.21.15.170:0/2263175749 >> v2:172.21.15.191:3300/0 conn(0x7f1f080c81f0 0x7f1f080cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 -- 172.21.15.170:0/2263175749 >> v1:172.21.15.170:6789/0 conn(0x7f1f080aca80 legacy=0x7f1f0806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650818574 0 0) 0x7f1ef0003570 con 0x7f1f080aca80 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 -- 172.21.15.170:0/2263175749 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08159f60 con 0x7f1f08069d60 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f0cb01700 1 --2- 172.21.15.170:0/2263175749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080c7cb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0098e0 tx=0x7f1efc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bad0cb14d8e7bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc011630 con 0x7f1f08069d60 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc011790 con 0x7f1f08069d60 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.951+0000 7f1f07fff700 1 -- 172.21.15.170:0/2263175749 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc011b10 con 0x7f1f08069d60 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 -- 172.21.15.170:0/2263175749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 msgr2=0x7f1f080c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2263175749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080c7cb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1efc0098e0 tx=0x7f1efc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 -- 172.21.15.170:0/2263175749 shutdown_connections 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2263175749 >> v2:172.21.15.191:3300/0 conn(0x7f1f080c81f0 0x7f1f080cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2263175749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080c7cb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 -- 172.21.15.170:0/2263175749 >> 172.21.15.170:0/2263175749 conn(0x7f1f080cd670 msgr2=0x7f1f080cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:24.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 -- 172.21.15.170:0/2263175749 shutdown_connections 2023-10-08T16:12:24.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.952+0000 7f1f0f566700 1 -- 172.21.15.170:0/2263175749 wait complete. 2023-10-08T16:12:24.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.953+0000 7f1f0f566700 1 Processor -- start 2023-10-08T16:12:24.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.953+0000 7f1f0f566700 1 -- start start 2023-10-08T16:12:24.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f080aca80 0x7f1f08154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080ac360 con 0x7f1f080c81f0 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080ae2b0 con 0x7f1f080aca80 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0f566700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080b0ec0 con 0x7f1f08069d60 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:44622/0 (socket says 172.21.15.170:44622) 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 -- 172.21.15.170:0/2392738259 learned_addr learned my addr 172.21.15.170:0/2392738259 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0cb01700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f080aca80 0x7f1f08154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0d302700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 -- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 msgr2=0x7f1f080abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080abe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 -- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f080aca80 msgr2=0x7f1f08154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f080aca80 0x7f1f08154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.954+0000 7f1f0db03700 1 -- 172.21.15.170:0/2392738259 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009420 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f0db03700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e8d0 tx=0x7f1f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000ee70 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0000c850 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080b11a0 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000cc50 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.955+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080b1760 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.956+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0804c3c0 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.956+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1f00013680 con 0x7f1f080c81f0 2023-10-08T16:12:24.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.956+0000 7f1f057fa700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 0x7f1ef40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:24.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.956+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1f00088120 con 0x7f1f080c81f0 2023-10-08T16:12:24.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.957+0000 7f1f0d302700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 0x7f1ef40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:24.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.957+0000 7f1f0d302700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 0x7f1ef40697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80016e0 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:24.960 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:24.960+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f00051ac0 con 0x7f1f080c81f0 2023-10-08T16:12:25.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.190+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1f080678d0 con 0x7f1f080c81f0 2023-10-08T16:12:25.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.190+0000 7f1f057fa700 1 -- 172.21.15.170:0/2392738259 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1f000550e0 con 0x7f1f080c81f0 2023-10-08T16:12:25.191 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:12:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 msgr2=0x7f1ef40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:25.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 0x7f1ef40697d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80016e0 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 msgr2=0x7f1f08169a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000e8d0 tx=0x7f1f000091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 shutdown_connections 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ef4067320 0x7f1ef40697d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1f080c81f0 0x7f1f08169a20 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1f08069d60 0x7f1f080abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.192+0000 7f1f0f566700 1 --2- 172.21.15.170:0/2392738259 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1f080aca80 0x7f1f08154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.193+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 >> 172.21.15.170:0/2392738259 conn(0x7f1f080cd670 msgr2=0x7f1f080cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:25.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.193+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 shutdown_connections 2023-10-08T16:12:25.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.193+0000 7f1f0f566700 1 -- 172.21.15.170:0/2392738259 wait complete. 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:12:25.205 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:25.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.206 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.221 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:25.222 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 42006sec preferred_lft 42006sec 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:25.223 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:25.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.224 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:12:25.284 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:12:25.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.286 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:12:25.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:12:25.343 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:12:25.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.442 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:12:25.457 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:12:25.457 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:12:25.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:25.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.458 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:12:25.521 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:25.522 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:25.577 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:25.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:25.577 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:25.631 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:12:25.631 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:12:25.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:25.711 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:25.711 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:25.730 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:25.731 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:25.731 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:25.752 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:25.753 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:25.753 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:25.753 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:12:25.753 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:12:25.753 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:12:25.754 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:12:25.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:25.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:25.819 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:12:25.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.973+0000 7f32b0544700 1 Processor -- start 2023-10-08T16:12:25.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.973+0000 7f32b0544700 1 -- start start 2023-10-08T16:12:25.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32b0544700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f32a8159c60 0x7f32a815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32b0544700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a815db10 0x7f32a8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32b0544700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a815a160 con 0x7f32a8159890 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32b0544700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a815df50 con 0x7f32a8159c60 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32b0544700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80d13a0 con 0x7f32a815db10 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32adadf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f32a8159c60 0x7f32a815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32adadf700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f32a8159c60 0x7f32a815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39106/0 (socket says 172.21.15.170:39106) 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.974+0000 7f32adadf700 1 -- 172.21.15.170:0/1555825351 learned_addr learned my addr 172.21.15.170:0/1555825351 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32aeae1700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a815db10 0x7f32a8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32adadf700 1 -- 172.21.15.170:0/1555825351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a815db10 msgr2=0x7f32a8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32adadf700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a815db10 0x7f32a8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32adadf700 1 -- 172.21.15.170:0/1555825351 >> v1:172.21.15.191:6789/0 conn(0x7f32a8159890 legacy=0x7f32a815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32adadf700 1 -- 172.21.15.170:0/1555825351 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a815e140 con 0x7f32a8159c60 2023-10-08T16:12:25.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32adadf700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a8159c60 0x7f32a815d5d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f32980099b0 tx=0x7f3298009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6acbf5a93feefd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:25.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32ad2de700 1 -- 172.21.15.170:0/1555825351 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800b820 con 0x7f32a8159c60 2023-10-08T16:12:25.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32ad2de700 1 -- 172.21.15.170:0/1555825351 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800b980 con 0x7f32a8159c60 2023-10-08T16:12:25.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32ad2de700 1 -- 172.21.15.170:0/1555825351 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800bd20 con 0x7f32a8159c60 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32b0544700 1 -- 172.21.15.170:0/1555825351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a8159c60 msgr2=0x7f32a815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.975+0000 7f32b0544700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a8159c60 0x7f32a815d5d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f32980099b0 tx=0x7f3298009250 comp rx=0 tx=0).stop 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 -- 172.21.15.170:0/1555825351 shutdown_connections 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a815db10 0x7f32a8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 --2- 172.21.15.170:0/1555825351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a8159c60 0x7f32a815d5d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 -- 172.21.15.170:0/1555825351 >> 172.21.15.170:0/1555825351 conn(0x7f32a8068e20 msgr2=0x7f32a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:25.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 -- 172.21.15.170:0/1555825351 shutdown_connections 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.976+0000 7f32b0544700 1 -- 172.21.15.170:0/1555825351 wait complete. 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 Processor -- start 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 -- start start 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32a8159890 0x7f32a80b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a8159c60 0x7f32a80cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80ce140 con 0x7f32a815db10 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80cd770 con 0x7f32a8159890 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32b0544700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a80cd160 con 0x7f32a8159c60 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32aeae1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32aeae1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39116/0 (socket says 172.21.15.170:39116) 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.977+0000 7f32aeae1700 1 -- 172.21.15.170:0/3627845470 learned_addr learned my addr 172.21.15.170:0/3627845470 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:25.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32adadf700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a8159c60 0x7f32a80cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32ae2e0700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32a8159890 0x7f32a80b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 -- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a8159c60 msgr2=0x7f32a80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a8159c60 0x7f32a80cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 -- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32a8159890 msgr2=0x7f32a80b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32a8159890 0x7f32a80b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 -- 172.21.15.170:0/3627845470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980094f0 con 0x7f32a815db10 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32aeae1700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f32a0007f20 tx=0x7f32a0009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000eb90 con 0x7f32a815db10 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a000ed10 con 0x7f32a815db10 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a80cfbc0 con 0x7f32a815db10 2023-10-08T16:12:25.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0014670 con 0x7f32a815db10 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.978+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a80cfea0 con 0x7f32a815db10 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.979+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a80cd8e0 con 0x7f32a815db10 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.979+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f32a00149e0 con 0x7f32a815db10 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.980+0000 7f329e7fc700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 0x7f328c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.980+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f32a0085220 con 0x7f32a815db10 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.980+0000 7f32ae2e0700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 0x7f328c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:25.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.980+0000 7f32ae2e0700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 0x7f328c0697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f32a4003a30 tx=0x7f32a4007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:25.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:25.984+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32a0051eb0 con 0x7f32a815db10 2023-10-08T16:12:26.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.213+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f32a804c3c0 con 0x7f32a815db10 2023-10-08T16:12:26.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.214+0000 7f329e7fc700 1 -- 172.21.15.170:0/3627845470 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32a00554d0 con 0x7f32a815db10 2023-10-08T16:12:26.214 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:12:26.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.215+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 msgr2=0x7f328c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:26.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.215+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 0x7f328c0697d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f32a4003a30 tx=0x7f32a4007ea0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.215+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 msgr2=0x7f32a8153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:26.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.215+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f32a0007f20 tx=0x7f32a0009860 comp rx=0 tx=0).stop 2023-10-08T16:12:26.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.215+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 shutdown_connections 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f328c067320 0x7f328c0697d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32a815db10 0x7f32a8153980 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f32a8159890 0x7f32a80b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 --2- 172.21.15.170:0/3627845470 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f32a8159c60 0x7f32a80cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 >> 172.21.15.170:0/3627845470 conn(0x7f32a8068e20 msgr2=0x7f32a8158630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 shutdown_connections 2023-10-08T16:12:26.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.216+0000 7f32b0544700 1 -- 172.21.15.170:0/3627845470 wait complete. 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:12:26.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:26.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.229 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:12:26.247 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:12:26.247 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:26.248 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 42005sec preferred_lft 42005sec 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:12:26.249 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:26.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.251 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:12:26.315 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:12:26.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.317 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:12:26.375 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:12:26.375 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:12:26.375 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:12:26.375 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:12:26.375 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:12:26.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:12:26.376 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:12:26.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.482 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:12:26.497 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:12:26.497 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:12:26.497 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:26.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.497 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:12:26.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:26.562 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:26.618 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:26.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:26.618 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:26.672 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:12:26.672 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:12:26.672 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:26.751 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:26.751 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:26.770 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:26.792 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:26.793 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:12:26.793 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:12:26.793 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:12:26.794 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:12:26.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:26.794 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:26.857 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:12:26.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.975+0000 7f792c4f6700 1 Processor -- start 2023-10-08T16:12:26.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.975+0000 7f792c4f6700 1 -- start start 2023-10-08T16:12:26.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7924159ec0 0x7f79240b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79240b53d0 0x7f79240cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240cce40 0x7f79240cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240b5b00 con 0x7f79240b53d0 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79240b5850 con 0x7f7924159ec0 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792c4f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79240b5da0 con 0x7f79240cce40 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79240b53d0 0x7f79240cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f79240b53d0 0x7f79240cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39126/0 (socket says 172.21.15.170:39126) 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 -- 172.21.15.170:0/3246687088 learned_addr learned my addr 172.21.15.170:0/3246687088 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:26.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792a292700 1 --2- 172.21.15.170:0/3246687088 >> v2:172.21.15.191:3300/0 conn(0x7f7924159ec0 0x7f79240b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f792aa93700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240cce40 0x7f79240cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 -- 172.21.15.170:0/3246687088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240cce40 msgr2=0x7f79240cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240cce40 0x7f79240cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 -- 172.21.15.170:0/3246687088 >> v2:172.21.15.191:3300/0 conn(0x7f7924159ec0 msgr2=0x7f79240b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 --2- 172.21.15.170:0/3246687088 >> v2:172.21.15.191:3300/0 conn(0x7f7924159ec0 0x7f79240b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 -- 172.21.15.170:0/3246687088 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240b6080 con 0x7f79240b53d0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.976+0000 7f7929a91700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240b53d0 0x7f79240cc900 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7924089120 tx=0x7f7914009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe6388f477e8cd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f7929290700 1 -- 172.21.15.170:0/3246687088 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400b640 con 0x7f79240b53d0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f7929290700 1 -- 172.21.15.170:0/3246687088 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791400b7a0 con 0x7f79240b53d0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f7929290700 1 -- 172.21.15.170:0/3246687088 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400bb60 con 0x7f79240b53d0 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 -- 172.21.15.170:0/3246687088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240b53d0 msgr2=0x7f79240cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:26.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240b53d0 0x7f79240cc900 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7924089120 tx=0x7f7914009250 comp rx=0 tx=0).stop 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 -- 172.21.15.170:0/3246687088 shutdown_connections 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240cce40 0x7f79240cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 --2- 172.21.15.170:0/3246687088 >> v2:172.21.15.191:3300/0 conn(0x7f7924159ec0 0x7f79240b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 --2- 172.21.15.170:0/3246687088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240b53d0 0x7f79240cc900 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 -- 172.21.15.170:0/3246687088 >> 172.21.15.170:0/3246687088 conn(0x7f79240999f0 msgr2=0x7f79240b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 -- 172.21.15.170:0/3246687088 shutdown_connections 2023-10-08T16:12:26.980 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.977+0000 7f792c4f6700 1 -- 172.21.15.170:0/3246687088 wait complete. 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.978+0000 7f792c4f6700 1 Processor -- start 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.978+0000 7f792c4f6700 1 -- start start 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 0x7f7924155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7924159ec0 0x7f792409c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7924068810 con 0x7f79240cce40 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792416e040 con 0x7f7924159ec0 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792c4f6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792416da80 con 0x7f79240b53d0 2023-10-08T16:12:26.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39128/0 (socket says 172.21.15.170:39128) 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 -- 172.21.15.170:0/1371471060 learned_addr learned my addr 172.21.15.170:0/1371471060 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792a292700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 0x7f7924155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f792aa93700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7924159ec0 0x7f792409c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 -- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 msgr2=0x7f7924155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 0x7f7924155f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 -- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7924159ec0 msgr2=0x7f792409c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7924159ec0 0x7f792409c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.979+0000 7f7929a91700 1 -- 172.21.15.170:0/1371471060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79140094f0 con 0x7f79240cce40 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f792a292700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 0x7f7924155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f7929a91700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:26.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791400b910 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f791400ba70 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792416dd60 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7914011650 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.980+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7924156460 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.981+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7924063c90 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.981+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f79140119c0 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.981+0000 7f791a7fc700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 0x7f79080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.981+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f79140865e0 con 0x7f79240cce40 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.982+0000 7f792a292700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 0x7f79080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:26.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.982+0000 7f792a292700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 0x7f79080697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f792000e900 tx=0x7f792000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:26.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:26.986+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7914053270 con 0x7f79240cce40 2023-10-08T16:12:27.211 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.211+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f79240718b0 con 0x7f79240cce40 2023-10-08T16:12:27.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.522+0000 7f791a7fc700 1 -- 172.21.15.170:0/1371471060 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7914056890 con 0x7f79240cce40 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 msgr2=0x7f79080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 0x7f79080697d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f792000e900 tx=0x7f792000b040 comp rx=0 tx=0).stop 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 msgr2=0x7f792416d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7914006fd0 tx=0x7f7914017000 comp rx=0 tx=0).stop 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 shutdown_connections 2023-10-08T16:12:27.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7924159ec0 0x7f792409c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f79240b53d0 0x7f7924155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7908067320 0x7f79080697d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 --2- 172.21.15.170:0/1371471060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79240cce40 0x7f792416d110 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 >> 172.21.15.170:0/1371471060 conn(0x7f79240999f0 msgr2=0x7f79240d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 shutdown_connections 2023-10-08T16:12:27.527 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.525+0000 7f792c4f6700 1 -- 172.21.15.170:0/1371471060 wait complete. 2023-10-08T16:12:27.538 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:12:27.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.654+0000 7f93eeae6700 1 Processor -- start 2023-10-08T16:12:27.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.654+0000 7f93eeae6700 1 -- start start 2023-10-08T16:12:27.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93eeae6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f93e8159420 0x7f93e80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:27.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93eeae6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e80b2980 con 0x7f93e81597f0 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93eeae6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e80b11d0 con 0x7f93e815d230 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93eeae6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80b14b0 con 0x7f93e8159420 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93edae4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f93e8159420 0x7f93e80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93ed2e3700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f93e81597f0 0x7f93e80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39400/0 (socket says 172.21.15.170:39400) 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93ed2e3700 1 -- 172.21.15.170:0/2588322713 learned_addr learned my addr 172.21.15.170:0/2588322713 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93edae4700 1 -- 172.21.15.170:0/2588322713 >> v1:172.21.15.191:6789/0 conn(0x7f93e815d230 legacy=0x7f93e815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93edae4700 1 -- 172.21.15.170:0/2588322713 >> v1:172.21.15.170:6789/0 conn(0x7f93e81597f0 legacy=0x7f93e80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.655+0000 7f93edae4700 1 -- 172.21.15.170:0/2588322713 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80b1790 con 0x7f93e8159420 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93edae4700 1 --2- 172.21.15.170:0/2588322713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e8159420 0x7f93e80b5f40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009e60 tx=0x7f93d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c272fe930e2733 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93ecae2700 1 -- 172.21.15.170:0/2588322713 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d800e070 con 0x7f93e8159420 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93ecae2700 1 -- 172.21.15.170:0/2588322713 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d800bc30 con 0x7f93e8159420 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93ecae2700 1 -- 172.21.15.170:0/2588322713 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8012430 con 0x7f93e8159420 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 -- 172.21.15.170:0/2588322713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e8159420 msgr2=0x7f93e80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2588322713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e8159420 0x7f93e80b5f40 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009e60 tx=0x7f93d8009780 comp rx=0 tx=0).stop 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 -- 172.21.15.170:0/2588322713 shutdown_connections 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2588322713 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e8159420 0x7f93e80b5f40 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 -- 172.21.15.170:0/2588322713 >> 172.21.15.170:0/2588322713 conn(0x7f93e80d0660 msgr2=0x7f93e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 -- 172.21.15.170:0/2588322713 shutdown_connections 2023-10-08T16:12:27.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.656+0000 7f93eeae6700 1 -- 172.21.15.170:0/2588322713 wait complete. 2023-10-08T16:12:27.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.657+0000 7f93eeae6700 1 Processor -- start 2023-10-08T16:12:27.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.657+0000 7f93eeae6700 1 -- start start 2023-10-08T16:12:27.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e8159420 0x7f93e816ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:27.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e815d230 0x7f93e81551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e816e330 con 0x7f93e81597f0 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c69b0 con 0x7f93e8159420 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93eeae6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80c7cb0 con 0x7f93e815d230 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39146/0 (socket says 172.21.15.170:39146) 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 -- 172.21.15.170:0/2125744405 learned_addr learned my addr 172.21.15.170:0/2125744405 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93edae4700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e8159420 0x7f93e816ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:27.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ee2e5700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e815d230 0x7f93e81551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 -- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e815d230 msgr2=0x7f93e81551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e815d230 0x7f93e81551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 -- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e8159420 msgr2=0x7f93e816ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e8159420 0x7f93e816ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.658+0000 7f93ed2e3700 1 -- 172.21.15.170:0/2125744405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8009a20 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93ed2e3700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00c790 tx=0x7f93dc007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc005b30 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc005cb0 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e80c6be0 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc004a80 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.659+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e80c71a0 con 0x7f93e81597f0 2023-10-08T16:12:27.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.660+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e80c7a80 con 0x7f93e81597f0 2023-10-08T16:12:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.660+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f93dc0070d0 con 0x7f93e81597f0 2023-10-08T16:12:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.660+0000 7f93e5ffb700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 0x7f93d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.661+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93dc083f30 con 0x7f93e81597f0 2023-10-08T16:12:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.661+0000 7f93edae4700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 0x7f93d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:27.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.661+0000 7f93edae4700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 0x7f93d40697d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009e60 tx=0x7f93d8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:27.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.665+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93dc04b5a0 con 0x7f93e81597f0 2023-10-08T16:12:27.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.871+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f93e8088400 con 0x7f93d4067320 2023-10-08T16:12:27.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.872+0000 7f93e5ffb700 1 -- 172.21.15.170:0/2125744405 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f93e8088400 con 0x7f93d4067320 2023-10-08T16:12:27.873 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:27.873 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:27.873 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:12:27.873 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:27.873 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 msgr2=0x7f93d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 0x7f93d40697d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f93d8009e60 tx=0x7f93d8019000 comp rx=0 tx=0).stop 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 msgr2=0x7f93e80c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00c790 tx=0x7f93dc007f40 comp rx=0 tx=0).stop 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 shutdown_connections 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f93e815d230 0x7f93e81551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f93e8159420 0x7f93e816ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f93d4067320 0x7f93d40697d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 --2- 172.21.15.170:0/2125744405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93e81597f0 0x7f93e80c6470 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:27.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 >> 172.21.15.170:0/2125744405 conn(0x7f93e80d0660 msgr2=0x7f93e8158700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:27.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 shutdown_connections 2023-10-08T16:12:27.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:27.874+0000 7f93eeae6700 1 -- 172.21.15.170:0/2125744405 wait complete. 2023-10-08T16:12:27.887 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:12:28.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.003+0000 7f67e0564700 1 Processor -- start 2023-10-08T16:12:28.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.003+0000 7f67e0564700 1 -- start start 2023-10-08T16:12:28.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67e0564700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f67d8159890 0x7f67d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67e0564700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f67d8069560 0x7f67d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67e0564700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d8069ee0 con 0x7f67d8159c60 2023-10-08T16:12:28.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67e0564700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d815a160 con 0x7f67d8159890 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67e0564700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d80d13a0 con 0x7f67d8069560 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67de300700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f67d8159890 0x7f67d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67ddaff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f67d8159c60 0x7f67d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39404/0 (socket says 172.21.15.170:39404) 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.004+0000 7f67ddaff700 1 -- 172.21.15.170:0/2266634851 learned_addr learned my addr 172.21.15.170:0/2266634851 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67deb01700 1 --2- 172.21.15.170:0/2266634851 >> v2:172.21.15.191:3300/0 conn(0x7f67d8069560 0x7f67d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67de300700 1 -- 172.21.15.170:0/2266634851 >> v2:172.21.15.191:3300/0 conn(0x7f67d8069560 msgr2=0x7f67d80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67de300700 1 --2- 172.21.15.170:0/2266634851 >> v2:172.21.15.191:3300/0 conn(0x7f67d8069560 0x7f67d80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67de300700 1 -- 172.21.15.170:0/2266634851 >> v1:172.21.15.170:6789/0 conn(0x7f67d8159c60 legacy=0x7f67d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67de300700 1 -- 172.21.15.170:0/2266634851 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d815e140 con 0x7f67d8159890 2023-10-08T16:12:28.006 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67dd2fe700 1 -- 172.21.15.170:0/2266634851 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082038839 0 0) 0x7f67d8069ee0 con 0x7f67d8159c60 2023-10-08T16:12:28.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67de300700 1 --2- 172.21.15.170:0/2266634851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8159890 0x7f67d815abd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f67d806a890 tx=0x7f67d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc450c69c16a3aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67dd2fe700 1 -- 172.21.15.170:0/2266634851 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400b6b0 con 0x7f67d8159890 2023-10-08T16:12:28.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67dd2fe700 1 -- 172.21.15.170:0/2266634851 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d400b810 con 0x7f67d8159890 2023-10-08T16:12:28.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67dd2fe700 1 -- 172.21.15.170:0/2266634851 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400bbb0 con 0x7f67d8159890 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67e0564700 1 -- 172.21.15.170:0/2266634851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8159890 msgr2=0x7f67d815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.005+0000 7f67e0564700 1 --2- 172.21.15.170:0/2266634851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8159890 0x7f67d815abd0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f67d806a890 tx=0x7f67d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 -- 172.21.15.170:0/2266634851 shutdown_connections 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 --2- 172.21.15.170:0/2266634851 >> v2:172.21.15.191:3300/0 conn(0x7f67d8069560 0x7f67d80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 --2- 172.21.15.170:0/2266634851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8159890 0x7f67d815abd0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 -- 172.21.15.170:0/2266634851 >> 172.21.15.170:0/2266634851 conn(0x7f67d8068e20 msgr2=0x7f67d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:28.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 -- 172.21.15.170:0/2266634851 shutdown_connections 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.006+0000 7f67e0564700 1 -- 172.21.15.170:0/2266634851 wait complete. 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 Processor -- start 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 -- start start 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 0x7f67d8149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67d8159c60 0x7f67d815ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8153180 con 0x7f67d8069560 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80cd380 con 0x7f67d8159c60 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67e0564700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d814a160 con 0x7f67d8159890 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67de300700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67de300700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39154/0 (socket says 172.21.15.170:39154) 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.007+0000 7f67de300700 1 -- 172.21.15.170:0/3385910817 learned_addr learned my addr 172.21.15.170:0/3385910817 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67deb01700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67d8159c60 0x7f67d815ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67ddaff700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 0x7f67d8149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 -- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 msgr2=0x7f67d8149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 0x7f67d8149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 -- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67d8159c60 msgr2=0x7f67d815ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67d8159c60 0x7f67d815ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 -- 172.21.15.170:0/3385910817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4009420 con 0x7f67d8069560 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67ddaff700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 0x7f67d8149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67de300700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f67d8070e20 tx=0x7f67d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d400bb90 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d4011420 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d40117c0 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d815cfb0 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.008+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d815d570 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.009+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d804c3c0 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.009+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f67d4011bf0 con 0x7f67d8069560 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.010+0000 7f67ce7fc700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 0x7f67bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.010+0000 7f67ddaff700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 0x7f67bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.010+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f67d4086640 con 0x7f67d8069560 2023-10-08T16:12:28.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.010+0000 7f67ddaff700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 0x7f67bc0697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f67d814abb0 tx=0x7f67c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.014+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67d40532d0 con 0x7f67d8069560 2023-10-08T16:12:28.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.195+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f67d806a5b0 con 0x7f67d8069560 2023-10-08T16:12:28.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.196+0000 7f67ce7fc700 1 -- 172.21.15.170:0/3385910817 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67d40568f0 con 0x7f67d8069560 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 msgr2=0x7f67bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 0x7f67bc0697d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f67d814abb0 tx=0x7f67c8009040 comp rx=0 tx=0).stop 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 msgr2=0x7f67d815c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f67d8070e20 tx=0x7f67d4017000 comp rx=0 tx=0).stop 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 shutdown_connections 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67d8159c60 0x7f67d815ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67bc067320 0x7f67bc0697d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67d8069560 0x7f67d815c630 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 --2- 172.21.15.170:0/3385910817 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67d8159890 0x7f67d8149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 >> 172.21.15.170:0/3385910817 conn(0x7f67d8068e20 msgr2=0x7f67d80b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:28.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 shutdown_connections 2023-10-08T16:12:28.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.198+0000 7f67e0564700 1 -- 172.21.15.170:0/3385910817 wait complete. 2023-10-08T16:12:28.210 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000002921 2023-10-08T16:12:28.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.328+0000 7f1556b75700 1 Processor -- start 2023-10-08T16:12:28.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.328+0000 7f1556b75700 1 -- start start 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1550159890 0x7f155015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159c60 0x7f155015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 0x7f1550069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155015a160 con 0x7f1550159890 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15500d13a0 con 0x7f155015db10 2023-10-08T16:12:28.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1556b75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155015e140 con 0x7f1550159c60 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1550159890 0x7f155015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1550159890 0x7f155015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39160/0 (socket says 172.21.15.170:39160) 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 -- 172.21.15.170:0/1295376008 learned_addr learned my addr 172.21.15.170:0/1295376008 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 -- 172.21.15.170:0/1295376008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159c60 msgr2=0x7f155015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 --2- 172.21.15.170:0/1295376008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159c60 0x7f155015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 -- 172.21.15.170:0/1295376008 >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 msgr2=0x7f1550069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1555112700 1 --2- 172.21.15.170:0/1295376008 >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 0x7f1550069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 --2- 172.21.15.170:0/1295376008 >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 0x7f1550069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 -- 172.21.15.170:0/1295376008 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155015a3d0 con 0x7f1550159890 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1555112700 1 --2- 172.21.15.170:0/1295376008 >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 0x7f1550069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:28.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f1554911700 1 --2- 172.21.15.170:0/1295376008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1550159890 0x7f155015abd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1550089b50 tx=0x7f154c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1eb43ec51c77226 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.329+0000 7f15477fe700 1 -- 172.21.15.170:0/1295376008 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c00e070 con 0x7f1550159890 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f15477fe700 1 -- 172.21.15.170:0/1295376008 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c00ba50 con 0x7f1550159890 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f15477fe700 1 -- 172.21.15.170:0/1295376008 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0092e0 con 0x7f1550159890 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 -- 172.21.15.170:0/1295376008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1550159890 msgr2=0x7f155015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 --2- 172.21.15.170:0/1295376008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1550159890 0x7f155015abd0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f1550089b50 tx=0x7f154c009840 comp rx=0 tx=0).stop 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 -- 172.21.15.170:0/1295376008 shutdown_connections 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 --2- 172.21.15.170:0/1295376008 >> v2:172.21.15.191:3300/0 conn(0x7f155015db10 0x7f1550069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 --2- 172.21.15.170:0/1295376008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1550159890 0x7f155015abd0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 --2- 172.21.15.170:0/1295376008 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159c60 0x7f155015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 -- 172.21.15.170:0/1295376008 >> 172.21.15.170:0/1295376008 conn(0x7f1550068e20 msgr2=0x7f15500d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:28.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 -- 172.21.15.170:0/1295376008 shutdown_connections 2023-10-08T16:12:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.330+0000 7f1556b75700 1 -- 172.21.15.170:0/1295376008 wait complete. 2023-10-08T16:12:28.334 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.331+0000 7f1556b75700 1 Processor -- start 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.331+0000 7f1556b75700 1 -- start start 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159890 0x7f15500cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1550159c60 0x7f1550169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f155015db10 0x7f15500c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155016ae30 con 0x7f155015db10 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15500cff70 con 0x7f1550159c60 2023-10-08T16:12:28.336 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1556b75700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155016a870 con 0x7f1550159890 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f155015db10 0x7f15500c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f155015db10 0x7f15500c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39174/0 (socket says 172.21.15.170:39174) 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 -- 172.21.15.170:0/2957826508 learned_addr learned my addr 172.21.15.170:0/2957826508 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1547fff700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1550159c60 0x7f1550169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1554911700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159890 0x7f15500cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 -- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159890 msgr2=0x7f15500cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1550159890 0x7f15500cfa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 -- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1550159c60 msgr2=0x7f1550169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1550159c60 0x7f1550169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1555112700 1 -- 172.21.15.170:0/2957826508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c009ae0 con 0x7f155015db10 2023-10-08T16:12:28.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.332+0000 7f1547fff700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1550159c60 0x7f1550169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1555112700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f155015db10 0x7f15500c8cc0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f154800c850 tx=0x7f1548008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548011600 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15480117c0 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1556b75700 1 -- 172.21.15.170:0/2957826508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155016ab50 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548011b80 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.333+0000 7f1556b75700 1 -- 172.21.15.170:0/2957826508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155016d910 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.334+0000 7f1556b75700 1 -- 172.21.15.170:0/2957826508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15500caae0 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.334+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1548018350 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.334+0000 7f1544ff9700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f153c067320 0x7f153c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.335+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1548084cb0 con 0x7f155015db10 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.335+0000 7f1554911700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f153c067320 0x7f153c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.335+0000 7f1554911700 1 --2- 172.21.15.170:0/2957826508 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f153c067320 0x7f153c0697d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f154c009ab0 tx=0x7f154c018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.339 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.339+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1548051830 con 0x7f155015db10 2023-10-08T16:12:28.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.550+0000 7f1556b75700 1 -- 172.21.15.170:0/2957826508 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000002921", "target": ["mon-mgr", ""]}) v1 -- 0x7f1550159120 con 0x7f153c067320 2023-10-08T16:12:28.551 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:12:28.551+0000 7f60a202f700 -1 mgr.server reply reply (1) Operation not permitted WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-08T16:12:28.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.551+0000 7f1544ff9700 1 -- 172.21.15.170:0/2957826508 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: -1 WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue.) v1 ==== 283+0+0 (secure 0 0 0) 0x7f1550159120 con 0x7f153c067320 2023-10-08T16:12:28.553 INFO:teuthology.orchestra.run.smithi170.stderr:Error EPERM: WARNING: This will rename the filesystem and possibly its pools. It is a potentially disruptive operation, clients' cephx credentials need reauthorized to access the file system and its pools with the new name. Add --yes-i-really-mean-it if you are sure you wish to continue. 2023-10-08T16:12:28.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:12:28.556 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:12:28.557 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:28.598 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:28.640 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:12:28.641 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:12:28.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:12:28.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:28.679 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:12:28.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:28.679 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:12:28.734 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:12:28.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:12:28.734 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:28.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:12:28.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:28.775 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:12:28.832 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:28.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.946+0000 7fb966d5f700 1 Processor -- start 2023-10-08T16:12:28.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.946+0000 7fb966d5f700 1 -- start start 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb966d5f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb960159c60 0x7fb96015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb966d5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 0x7fb960069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb966d5f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96015a160 con 0x7fb960159890 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb966d5f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96015df50 con 0x7fb960159c60 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb966d5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600d13a0 con 0x7fb96015db10 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb964afb700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb960159890 0x7fb96015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39418/0 (socket says 172.21.15.170:39418) 2023-10-08T16:12:28.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb957fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb960159c60 0x7fb96015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb964afb700 1 -- 172.21.15.170:0/4130183520 learned_addr learned my addr 172.21.15.170:0/4130183520 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb9652fc700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 0x7fb960069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb957fff700 1 -- 172.21.15.170:0/4130183520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 msgr2=0x7fb960069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb957fff700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 0x7fb960069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb957fff700 1 -- 172.21.15.170:0/4130183520 >> v1:172.21.15.170:6789/0 conn(0x7fb960159890 legacy=0x7fb96015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb9577fe700 1 -- 172.21.15.170:0/4130183520 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624179197 0 0) 0x7fb96015a160 con 0x7fb960159890 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb957fff700 1 -- 172.21.15.170:0/4130183520 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96015e140 con 0x7fb960159c60 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.947+0000 7fb9652fc700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 0x7fb960069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb957fff700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb96015d5d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098e0 tx=0x7fb9500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9f6fa4063a94d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb9577fe700 1 -- 172.21.15.170:0/4130183520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950009e80 con 0x7fb960159c60 2023-10-08T16:12:28.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb9577fe700 1 -- 172.21.15.170:0/4130183520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000b860 con 0x7fb960159c60 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb9577fe700 1 -- 172.21.15.170:0/4130183520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000bbe0 con 0x7fb960159c60 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 -- 172.21.15.170:0/4130183520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 msgr2=0x7fb96015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb96015d5d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb9500098e0 tx=0x7fb9500091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 -- 172.21.15.170:0/4130183520 shutdown_connections 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb96015db10 0x7fb960069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 --2- 172.21.15.170:0/4130183520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb96015d5d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 -- 172.21.15.170:0/4130183520 >> 172.21.15.170:0/4130183520 conn(0x7fb960068e20 msgr2=0x7fb9600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 -- 172.21.15.170:0/4130183520 shutdown_connections 2023-10-08T16:12:28.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.948+0000 7fb966d5f700 1 -- 172.21.15.170:0/4130183520 wait complete. 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.949+0000 7fb966d5f700 1 Processor -- start 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 -- start start 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb960159890 0x7fb9600b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb96015db10 0x7fb9600cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b36c0 con 0x7fb960159c60 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b39a0 con 0x7fb96015db10 2023-10-08T16:12:28.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb966d5f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600b0ff0 con 0x7fb960159890 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb957fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.950+0000 7fb957fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39184/0 (socket says 172.21.15.170:39184) 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 -- 172.21.15.170:0/2287025435 learned_addr learned my addr 172.21.15.170:0/2287025435 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb964afb700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb960159890 0x7fb9600b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb9652fc700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb96015db10 0x7fb9600cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 -- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb960159890 msgr2=0x7fb9600b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb960159890 0x7fb9600b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 -- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb96015db10 msgr2=0x7fb9600cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb96015db10 0x7fb9600cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 -- 172.21.15.170:0/2287025435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009420 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb957fff700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb9600899c0 tx=0x7fb950017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950000cc0 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000b9d0 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9600b12d0 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950025670 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.951+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9600cfad0 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.952+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96004c3c0 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.952+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb950023070 con 0x7fb960159c60 2023-10-08T16:12:28.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.953+0000 7fb954ff9700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 0x7fb94c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:28.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.953+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb9500862b0 con 0x7fb960159c60 2023-10-08T16:12:28.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.953+0000 7fb964afb700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 0x7fb94c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:28.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.953+0000 7fb964afb700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 0x7fb94c0697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb95c002250 tx=0x7fb95c001a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:28.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:28.957+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb950052fc0 con 0x7fb960159c60 2023-10-08T16:12:29.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.186+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb96006a5b0 con 0x7fb960159c60 2023-10-08T16:12:29.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.186+0000 7fb954ff9700 1 -- 172.21.15.170:0/2287025435 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4615 (secure 0 0 0) 0x7fb9500565e0 con 0x7fb960159c60 2023-10-08T16:12:29.187 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:29.187 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:29.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 msgr2=0x7fb94c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 0x7fb94c0697d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb95c002250 tx=0x7fb95c001a20 comp rx=0 tx=0).stop 2023-10-08T16:12:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 msgr2=0x7fb9600cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb9600899c0 tx=0x7fb950017000 comp rx=0 tx=0).stop 2023-10-08T16:12:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 shutdown_connections 2023-10-08T16:12:29.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb96015db10 0x7fb9600cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb960159890 0x7fb9600b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb94c067320 0x7fb94c0697d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 --2- 172.21.15.170:0/2287025435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb960159c60 0x7fb9600cf590 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 >> 172.21.15.170:0/2287025435 conn(0x7fb960068e20 msgr2=0x7fb960155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.191 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 shutdown_connections 2023-10-08T16:12:29.192 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.188+0000 7fb966d5f700 1 -- 172.21.15.170:0/2287025435 wait complete. 2023-10-08T16:12:29.192 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2023-10-08T16:12:29.201 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:29.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.316+0000 7f5db359e700 1 Processor -- start 2023-10-08T16:12:29.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.316+0000 7f5db359e700 1 -- start start 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5db415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40b2980 con 0x7f5db4159420 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db359e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db415dbb0 con 0x7f5db41597f0 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b11d0 con 0x7f5db415d230 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db259c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5db4159420 0x7f5db40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39432/0 (socket says 172.21.15.170:39432) 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db259c700 1 -- 172.21.15.170:0/1626903855 learned_addr learned my addr 172.21.15.170:0/1626903855 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5db415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179933786 0 0) 0x7f5db40b2980 con 0x7f5db4159420 2023-10-08T16:12:29.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5da0003570 con 0x7f5db4159420 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192164281 0 0) 0x7f5da0003570 con 0x7f5db4159420 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 msgr2=0x7f5db415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5db415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 >> v1:172.21.15.191:6789/0 conn(0x7f5db41597f0 legacy=0x7f5db40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.317+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40b14b0 con 0x7f5db4159420 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f5da4001470 con 0x7f5db4159420 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 >> v1:172.21.15.170:6789/0 conn(0x7f5db4159420 legacy=0x7f5db40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5da0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da0004010 0x7f5da0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5da0004d50 0x7f5da00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40b11d0 con 0x7f5da0004d50 2023-10-08T16:12:29.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000bbf0 con 0x7f5da0004010 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da000bdb0 con 0x7f5db415d230 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5da0004d50 0x7f5da00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db259c700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5da0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5da9d9b700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da0004010 0x7f5da0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 -- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 msgr2=0x7f5da0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5da0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 -- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da0004010 msgr2=0x7f5da0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5da0004010 0x7f5da0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 -- 172.21.15.170:0/1626903855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da0003570 con 0x7f5da0004d50 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db259c700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db415d230 0x7f5da0003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5da0004d50 0x7f5da00051c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0094d0 tx=0x7f5dac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7eb53dff5f7ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00ea90 con 0x7f5da0004d50 2023-10-08T16:12:29.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dac00f040 con 0x7f5da0004d50 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.318+0000 7f5db1d9b700 1 -- 172.21.15.170:0/1626903855 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00ee20 con 0x7f5da0004d50 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 -- 172.21.15.170:0/1626903855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5da0004d50 msgr2=0x7f5da00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 --2- 172.21.15.170:0/1626903855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5da0004d50 0x7f5da00051c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0094d0 tx=0x7f5dac00b040 comp rx=0 tx=0).stop 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db2d9d700 1 -- 172.21.15.170:0/1626903855 reap_dead start 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 -- 172.21.15.170:0/1626903855 shutdown_connections 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 -- 172.21.15.170:0/1626903855 >> 172.21.15.170:0/1626903855 conn(0x7f5db40d0660 msgr2=0x7f5db40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 -- 172.21.15.170:0/1626903855 shutdown_connections 2023-10-08T16:12:29.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.319+0000 7f5db359e700 1 -- 172.21.15.170:0/1626903855 wait complete. 2023-10-08T16:12:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.320+0000 7f5db359e700 1 Processor -- start 2023-10-08T16:12:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.320+0000 7f5db359e700 1 -- start start 2023-10-08T16:12:29.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db4167320 0x7f5db415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db416cce0 0x7f5db416d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db416d6a0 con 0x7f5db415c860 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db416a8b0 con 0x7f5db416cce0 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db416ab00 con 0x7f5db4167320 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39208/0 (socket says 172.21.15.170:39208) 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 -- 172.21.15.170:0/4255375002 learned_addr learned my addr 172.21.15.170:0/4255375002 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5da9d9b700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db4167320 0x7f5db415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db2d9d700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db416cce0 0x7f5db416d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.330 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 -- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db4167320 msgr2=0x7f5db415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db4167320 0x7f5db415f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 -- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db416cce0 msgr2=0x7f5db416d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db416cce0 0x7f5db416d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 -- 172.21.15.170:0/4255375002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db416ade0 con 0x7f5db415c860 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db259c700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5da4007710 tx=0x7f5da4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4018030 con 0x7f5db415c860 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da400fc60 con 0x7f5db415c860 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da40133f0 con 0x7f5db415c860 2023-10-08T16:12:29.331 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.321+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db416b0c0 con 0x7f5db415c860 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.322+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db416b680 con 0x7f5db415c860 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.323+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db404c3c0 con 0x7f5db415c860 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.323+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5da401b070 con 0x7f5db415c860 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.323+0000 7f5daaffd700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 0x7f5d940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.323+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5da4019070 con 0x7f5db415c860 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.323+0000 7f5da9d9b700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 0x7f5d940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.324+0000 7f5da9d9b700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 0x7f5d940697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f60 tx=0x7f5d9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.327+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5da4051ab0 con 0x7f5db415c860 2023-10-08T16:12:29.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.510+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5db40b1790 con 0x7f5db415c860 2023-10-08T16:12:29.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.511+0000 7f5daaffd700 1 -- 172.21.15.170:0/4255375002 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f5da40550d0 con 0x7f5db415c860 2023-10-08T16:12:29.512 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:29.512 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":154,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:19.503764+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 msgr2=0x7f5d940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 0x7f5d940697d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c000f60 tx=0x7f5d9c009040 comp rx=0 tx=0).stop 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 msgr2=0x7f5db415f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f5da4007710 tx=0x7f5da4001e50 comp rx=0 tx=0).stop 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 shutdown_connections 2023-10-08T16:12:29.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5db416cce0 0x7f5db416d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5d94067320 0x7f5d940697d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5db415c860 0x7f5db415f490 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 --2- 172.21.15.170:0/4255375002 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5db4167320 0x7f5db415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 >> 172.21.15.170:0/4255375002 conn(0x7f5db40d0660 msgr2=0x7f5db4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 shutdown_connections 2023-10-08T16:12:29.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.513+0000 7f5db359e700 1 -- 172.21.15.170:0/4255375002 wait complete. 2023-10-08T16:12:29.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 26, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:12:16.504693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 27, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:12:17.396600+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:12:29.527 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:12:29.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.644+0000 7fed54289700 1 Processor -- start 2023-10-08T16:12:29.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.644+0000 7fed54289700 1 -- start start 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed54289700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fed4c159c60 0x7fed4c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed54289700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fed4c15db10 0x7fed4c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed54289700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4c15a160 con 0x7fed4c159890 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed54289700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c15df50 con 0x7fed4c15db10 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed54289700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4c0d13a0 con 0x7fed4c159c60 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed52826700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fed4c15db10 0x7fed4c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed52826700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fed4c15db10 0x7fed4c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39216/0 (socket says 172.21.15.170:39216) 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed52025700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fed4c159890 0x7fed4c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39438/0 (socket says 172.21.15.170:39438) 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.645+0000 7fed52826700 1 -- 172.21.15.170:0/188112646 learned_addr learned my addr 172.21.15.170:0/188112646 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed51824700 1 --2- 172.21.15.170:0/188112646 >> v2:172.21.15.191:3300/0 conn(0x7fed4c159c60 0x7fed4c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed52826700 1 -- 172.21.15.170:0/188112646 >> v2:172.21.15.191:3300/0 conn(0x7fed4c159c60 msgr2=0x7fed4c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed52826700 1 --2- 172.21.15.170:0/188112646 >> v2:172.21.15.191:3300/0 conn(0x7fed4c159c60 0x7fed4c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed52826700 1 -- 172.21.15.170:0/188112646 >> v1:172.21.15.170:6789/0 conn(0x7fed4c159890 legacy=0x7fed4c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed52826700 1 -- 172.21.15.170:0/188112646 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c15e140 con 0x7fed4c15db10 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed51023700 1 -- 172.21.15.170:0/188112646 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280006731 0 0) 0x7fed4c15a160 con 0x7fed4c159890 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed52826700 1 --2- 172.21.15.170:0/188112646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c15db10 0x7fed4c069560 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0b67f0 tx=0x7fed44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6802bf4d78b1a7d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed51023700 1 -- 172.21.15.170:0/188112646 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400b860 con 0x7fed4c15db10 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed51023700 1 -- 172.21.15.170:0/188112646 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4400b9c0 con 0x7fed4c15db10 2023-10-08T16:12:29.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed51023700 1 -- 172.21.15.170:0/188112646 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400bd60 con 0x7fed4c15db10 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed54289700 1 -- 172.21.15.170:0/188112646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c15db10 msgr2=0x7fed4c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.646+0000 7fed54289700 1 --2- 172.21.15.170:0/188112646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c15db10 0x7fed4c069560 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0b67f0 tx=0x7fed44009250 comp rx=0 tx=0).stop 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 -- 172.21.15.170:0/188112646 shutdown_connections 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 --2- 172.21.15.170:0/188112646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c15db10 0x7fed4c069560 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 --2- 172.21.15.170:0/188112646 >> v2:172.21.15.191:3300/0 conn(0x7fed4c159c60 0x7fed4c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 -- 172.21.15.170:0/188112646 >> 172.21.15.170:0/188112646 conn(0x7fed4c068e20 msgr2=0x7fed4c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 -- 172.21.15.170:0/188112646 shutdown_connections 2023-10-08T16:12:29.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.647+0000 7fed54289700 1 -- 172.21.15.170:0/188112646 wait complete. 2023-10-08T16:12:29.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 Processor -- start 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 -- start start 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed4c159c60 0x7fed4c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed4c15db10 0x7fed4c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0b11f0 con 0x7fed4c159890 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0b05e0 con 0x7fed4c15db10 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed54289700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c0b08c0 con 0x7fed4c159c60 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed52025700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed52025700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39228/0 (socket says 172.21.15.170:39228) 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.648+0000 7fed52025700 1 -- 172.21.15.170:0/1528500742 learned_addr learned my addr 172.21.15.170:0/1528500742 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:29.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed51824700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed4c159c60 0x7fed4c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52826700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed4c15db10 0x7fed4c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 -- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed4c159c60 msgr2=0x7fed4c0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed4c159c60 0x7fed4c0cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 -- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed4c15db10 msgr2=0x7fed4c0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed4c15db10 0x7fed4c0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 -- 172.21.15.170:0/1528500742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440094f0 con 0x7fed4c159890 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed52025700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fed4800cdd0 tx=0x7fed48001bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed48005df0 con 0x7fed4c159890 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4800a960 con 0x7fed4c159890 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4c0b0ba0 con 0x7fed4c159890 2023-10-08T16:12:29.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4800ad20 con 0x7fed4c159890 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.649+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4c0b6380 con 0x7fed4c159890 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.650+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4c04c3c0 con 0x7fed4c159890 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.651+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fed48007300 con 0x7fed4c159890 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.651+0000 7fed427fc700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 0x7fed300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.651+0000 7fed51824700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 0x7fed300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.651+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed48084f00 con 0x7fed4c159890 2023-10-08T16:12:29.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.651+0000 7fed51824700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 0x7fed300697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006fd0 tx=0x7fed3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.655+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed48051bd0 con 0x7fed4c159890 2023-10-08T16:12:29.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.836+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed4c06a5b0 con 0x7fed4c159890 2023-10-08T16:12:29.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.836+0000 7fed427fc700 1 -- 172.21.15.170:0/1528500742 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed480551f0 con 0x7fed4c159890 2023-10-08T16:12:29.837 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:12:29.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 msgr2=0x7fed300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 0x7fed300697d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fed3c006fd0 tx=0x7fed3c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 msgr2=0x7fed4c0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fed4800cdd0 tx=0x7fed48001bc0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 shutdown_connections 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed4c15db10 0x7fed4c0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed30067320 0x7fed300697d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed4c159890 0x7fed4c0b5e40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 --2- 172.21.15.170:0/1528500742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed4c159c60 0x7fed4c0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 >> 172.21.15.170:0/1528500742 conn(0x7fed4c068e20 msgr2=0x7fed4c0c5590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 shutdown_connections 2023-10-08T16:12:29.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.838+0000 7fed54289700 1 -- 172.21.15.170:0/1528500742 wait complete. 2023-10-08T16:12:29.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:12:29.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.967+0000 7f60d1346700 1 Processor -- start 2023-10-08T16:12:29.968 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.967+0000 7f60d1346700 1 -- start start 2023-10-08T16:12:29.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159890 0x7f60cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60cc159c60 0x7f60cc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f60cc15db10 0x7f60cc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc15a160 con 0x7f60cc159c60 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0d13a0 con 0x7f60cc15db10 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60d1346700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15e140 con 0x7f60cc159890 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60cc159c60 0x7f60cc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f60cc159c60 0x7f60cc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39234/0 (socket says 172.21.15.170:39234) 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 -- 172.21.15.170:0/879755066 learned_addr learned my addr 172.21.15.170:0/879755066 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60cb7fe700 1 --2- 172.21.15.170:0/879755066 >> v2:172.21.15.191:3300/0 conn(0x7f60cc15db10 0x7f60cc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60caffd700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159890 0x7f60cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 -- 172.21.15.170:0/879755066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159890 msgr2=0x7f60cc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159890 0x7f60cc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 -- 172.21.15.170:0/879755066 >> v2:172.21.15.191:3300/0 conn(0x7f60cc15db10 msgr2=0x7f60cc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 --2- 172.21.15.170:0/879755066 >> v2:172.21.15.191:3300/0 conn(0x7f60cc15db10 0x7f60cc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.968+0000 7f60c27fc700 1 -- 172.21.15.170:0/879755066 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc15a3d0 con 0x7f60cc159c60 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60c27fc700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159c60 0x7f60cc15d5d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f60cc089b50 tx=0x7f60b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e771c89b208f0d58 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60ca7fc700 1 -- 172.21.15.170:0/879755066 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400b860 con 0x7f60cc159c60 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60ca7fc700 1 -- 172.21.15.170:0/879755066 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b400b9c0 con 0x7f60cc159c60 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60ca7fc700 1 -- 172.21.15.170:0/879755066 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400bd60 con 0x7f60cc159c60 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 -- 172.21.15.170:0/879755066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159c60 msgr2=0x7f60cc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159c60 0x7f60cc15d5d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f60cc089b50 tx=0x7f60b4009250 comp rx=0 tx=0).stop 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 -- 172.21.15.170:0/879755066 shutdown_connections 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 --2- 172.21.15.170:0/879755066 >> v2:172.21.15.191:3300/0 conn(0x7f60cc15db10 0x7f60cc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159890 0x7f60cc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 --2- 172.21.15.170:0/879755066 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159c60 0x7f60cc15d5d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 -- 172.21.15.170:0/879755066 >> 172.21.15.170:0/879755066 conn(0x7f60cc068e20 msgr2=0x7f60cc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:29.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 -- 172.21.15.170:0/879755066 shutdown_connections 2023-10-08T16:12:29.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.969+0000 7f60d1346700 1 -- 172.21.15.170:0/879755066 wait complete. 2023-10-08T16:12:29.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.970+0000 7f60d1346700 1 Processor -- start 2023-10-08T16:12:29.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 -- start start 2023-10-08T16:12:29.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159c60 0x7f60cc155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60cc15db10 0x7f60cc0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc152ef0 con 0x7f60cc159890 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc1531d0 con 0x7f60cc15db10 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60d1346700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc1534b0 con 0x7f60cc159c60 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60caffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60caffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39240/0 (socket says 172.21.15.170:39240) 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60caffd700 1 -- 172.21.15.170:0/100171801 learned_addr learned my addr 172.21.15.170:0/100171801 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60c27fc700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159c60 0x7f60cc155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.971+0000 7f60cb7fe700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60cc15db10 0x7f60cc0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 -- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159c60 msgr2=0x7f60cc155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159c60 0x7f60cc155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 -- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60cc15db10 msgr2=0x7f60cc0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:29.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60cc15db10 0x7f60cc0b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:29.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 -- 172.21.15.170:0/100171801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40094f0 con 0x7f60cc159890 2023-10-08T16:12:29.977 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60caffd700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c7d0 tx=0x7f60bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011600 con 0x7f60cc159890 2023-10-08T16:12:29.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60bc011760 con 0x7f60cc159890 2023-10-08T16:12:29.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60bc011b20 con 0x7f60cc159890 2023-10-08T16:12:29.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc0cd0c0 con 0x7f60cc159890 2023-10-08T16:12:29.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.972+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc152b30 con 0x7f60cc159890 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.973+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc04c3c0 con 0x7f60cc159890 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.973+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f60bc0193f0 con 0x7f60cc159890 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.974+0000 7f60c37fe700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 0x7f60ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.974+0000 7f60c27fc700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 0x7f60ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.974+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f60bc085020 con 0x7f60cc159890 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.974+0000 7f60c27fc700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 0x7f60ac0697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f60cc089b50 tx=0x7f60b4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:29.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:29.978+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60bc051ba0 con 0x7f60cc159890 2023-10-08T16:12:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.162+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60cc0d09c0 con 0x7f60cc159890 2023-10-08T16:12:30.163 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.163+0000 7f60c37fe700 1 -- 172.21.15.170:0/100171801 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60bc0551c0 con 0x7f60cc159890 2023-10-08T16:12:30.163 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 msgr2=0x7f60ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 0x7f60ac0697d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f60cc089b50 tx=0x7f60b4017000 comp rx=0 tx=0).stop 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 msgr2=0x7f60cc1525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c7d0 tx=0x7f60bc008040 comp rx=0 tx=0).stop 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 shutdown_connections 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f60cc15db10 0x7f60cc0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f60ac067320 0x7f60ac0697d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f60cc159890 0x7f60cc1525f0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 --2- 172.21.15.170:0/100171801 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f60cc159c60 0x7f60cc155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 >> 172.21.15.170:0/100171801 conn(0x7f60cc068e20 msgr2=0x7f60cc153850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 shutdown_connections 2023-10-08T16:12:30.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.165+0000 7f60d1346700 1 -- 172.21.15.170:0/100171801 wait complete. 2023-10-08T16:12:30.178 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:12:30.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.295+0000 7f5e270dc700 1 Processor -- start 2023-10-08T16:12:30.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.295+0000 7f5e270dc700 1 -- start start 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e270dc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e200b4e70 0x7f5e200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e270dc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b1b40 0x7f5e2015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e270dc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e200cfb00 con 0x7f5e200b16d0 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e270dc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e200b5c40 con 0x7f5e200b4e70 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e270dc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200b5f20 con 0x7f5e200b1b40 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e200b4e70 0x7f5e200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e258d9700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5e200b16d0 0x7f5e2015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39446/0 (socket says 172.21.15.170:39446) 2023-10-08T16:12:30.298 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e258d9700 1 -- 172.21.15.170:0/1671481089 learned_addr learned my addr 172.21.15.170:0/1671481089 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e268db700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b1b40 0x7f5e2015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 -- 172.21.15.170:0/1671481089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b1b40 msgr2=0x7f5e2015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b1b40 0x7f5e2015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 -- 172.21.15.170:0/1671481089 >> v1:172.21.15.170:6789/0 conn(0x7f5e200b16d0 legacy=0x7f5e2015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 -- 172.21.15.170:0/1671481089 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200b6200 con 0x7f5e200b4e70 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e250d8700 1 -- 172.21.15.170:0/1671481089 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491450194 0 0) 0x7f5e200cfb00 con 0x7f5e200b16d0 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.296+0000 7f5e260da700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b4e70 0x7f5e200ccff0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5e180098e0 tx=0x7f5e180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a96f853f0b72135 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e250d8700 1 -- 172.21.15.170:0/1671481089 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18009e80 con 0x7f5e200b4e70 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e250d8700 1 -- 172.21.15.170:0/1671481089 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1800b860 con 0x7f5e200b4e70 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e250d8700 1 -- 172.21.15.170:0/1671481089 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800bbe0 con 0x7f5e200b4e70 2023-10-08T16:12:30.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 -- 172.21.15.170:0/1671481089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b4e70 msgr2=0x7f5e200ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b4e70 0x7f5e200ccff0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5e180098e0 tx=0x7f5e180091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 -- 172.21.15.170:0/1671481089 shutdown_connections 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b1b40 0x7f5e2015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 --2- 172.21.15.170:0/1671481089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b4e70 0x7f5e200ccff0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 -- 172.21.15.170:0/1671481089 >> 172.21.15.170:0/1671481089 conn(0x7f5e200d10c0 msgr2=0x7f5e200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 -- 172.21.15.170:0/1671481089 shutdown_connections 2023-10-08T16:12:30.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.297+0000 7f5e270dc700 1 -- 172.21.15.170:0/1671481089 wait complete. 2023-10-08T16:12:30.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.298+0000 7f5e270dc700 1 Processor -- start 2023-10-08T16:12:30.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.298+0000 7f5e270dc700 1 -- start start 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e200b16d0 0x7f5e2016dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b4e70 0x7f5e20146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200ca450 con 0x7f5e200b1b40 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2016e290 con 0x7f5e200b16d0 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e270dc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200c9560 con 0x7f5e200b4e70 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39258/0 (socket says 172.21.15.170:39258) 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 -- 172.21.15.170:0/613191118 learned_addr learned my addr 172.21.15.170:0/613191118 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:30.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e268db700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b4e70 0x7f5e20146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e260da700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e200b16d0 0x7f5e2016dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 -- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b4e70 msgr2=0x7f5e20146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b4e70 0x7f5e20146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 -- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e200b16d0 msgr2=0x7f5e2016dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e200b16d0 0x7f5e2016dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.299+0000 7f5e258d9700 1 -- 172.21.15.170:0/613191118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18009420 con 0x7f5e200b1b40 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e258d9700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400cc70 tx=0x7f5e14001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1400ce80 con 0x7f5e200b1b40 2023-10-08T16:12:30.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1400ab70 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e140070f0 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200c9840 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.300+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e200c9e00 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.301+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e200ca600 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.301+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e140074e0 con 0x7f5e200b1b40 2023-10-08T16:12:30.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.302+0000 7f5e1e7fc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 0x7f5e0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.302+0000 7f5e260da700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 0x7f5e0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.302+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5e14085090 con 0x7f5e200b1b40 2023-10-08T16:12:30.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.302+0000 7f5e260da700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 0x7f5e0c0697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e180091e0 tx=0x7f5e18017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.306+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e200ca600 con 0x7f5e200b1b40 2023-10-08T16:12:30.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.486+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e200ca590 con 0x7f5e200b1b40 2023-10-08T16:12:30.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.486+0000 7f5e1e7fc700 1 -- 172.21.15.170:0/613191118 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e14051da0 con 0x7f5e200b1b40 2023-10-08T16:12:30.487 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:12:30.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 msgr2=0x7f5e0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 0x7f5e0c0697d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e180091e0 tx=0x7f5e18017000 comp rx=0 tx=0).stop 2023-10-08T16:12:30.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 msgr2=0x7f5e200b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5e1400cc70 tx=0x7f5e14001a30 comp rx=0 tx=0).stop 2023-10-08T16:12:30.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 shutdown_connections 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e200b4e70 0x7f5e20146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e200b16d0 0x7f5e2016dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e0c067320 0x7f5e0c0697d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 --2- 172.21.15.170:0/613191118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e200b1b40 0x7f5e200b3610 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.488+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 >> 172.21.15.170:0/613191118 conn(0x7f5e200d10c0 msgr2=0x7f5e20069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.489+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 shutdown_connections 2023-10-08T16:12:30.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.489+0000 7f5e270dc700 1 -- 172.21.15.170:0/613191118 wait complete. 2023-10-08T16:12:30.501 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.615+0000 7f65ced3a700 1 Processor -- start 2023-10-08T16:12:30.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 -- start start 2023-10-08T16:12:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c815a160 con 0x7f65c8159890 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80d13a0 con 0x7f65c815db10 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.616+0000 7f65ced3a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c815e140 con 0x7f65c8159c60 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65cd2d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65ccad6700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f65c8159890 0x7f65c815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:57326/0 (socket says 172.21.15.170:57326) 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:47584/0 (socket says 172.21.15.170:47584) 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65cd2d7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:54002/0 (socket says 172.21.15.170:54002) 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65ccad6700 1 -- 172.21.15.170:0/1710720542 learned_addr learned my addr 172.21.15.170:0/1710720542 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:30.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 -- 172.21.15.170:0/1710720542 >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 msgr2=0x7f65c8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 --2- 172.21.15.170:0/1710720542 >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 -- 172.21.15.170:0/1710720542 >> v1:172.21.15.191:6789/0 conn(0x7f65c8159890 legacy=0x7f65c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 -- 172.21.15.170:0/1710720542 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c815a3d0 con 0x7f65c8159c60 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65cd2d7700 1 --2- 172.21.15.170:0/1710720542 >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bffff700 1 --2- 172.21.15.170:0/1710720542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65b80098e0 tx=0x7f65b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f143f2f802fced4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.617+0000 7f65bf7fe700 1 -- 172.21.15.170:0/1710720542 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800b640 con 0x7f65c8159c60 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65bf7fe700 1 -- 172.21.15.170:0/1710720542 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800b7a0 con 0x7f65c8159c60 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65bf7fe700 1 -- 172.21.15.170:0/1710720542 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800bb40 con 0x7f65c8159c60 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 -- 172.21.15.170:0/1710720542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 msgr2=0x7f65c815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 --2- 172.21.15.170:0/1710720542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65b80098e0 tx=0x7f65b80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.620 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 -- 172.21.15.170:0/1710720542 shutdown_connections 2023-10-08T16:12:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 --2- 172.21.15.170:0/1710720542 >> v2:172.21.15.191:3300/0 conn(0x7f65c815db10 0x7f65c8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 --2- 172.21.15.170:0/1710720542 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c8159c60 0x7f65c815d5d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 -- 172.21.15.170:0/1710720542 >> 172.21.15.170:0/1710720542 conn(0x7f65c8068e20 msgr2=0x7f65c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 -- 172.21.15.170:0/1710720542 shutdown_connections 2023-10-08T16:12:30.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.618+0000 7f65ced3a700 1 -- 172.21.15.170:0/1710720542 wait complete. 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.619+0000 7f65ced3a700 1 Processor -- start 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.619+0000 7f65ced3a700 1 -- start start 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 0x7f65c8155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c815db10 0x7f65c80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8157480 con 0x7f65c8159890 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b36c0 con 0x7f65c8159c60 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ced3a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8157670 con 0x7f65c815db10 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ccad6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ccad6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39264/0 (socket says 172.21.15.170:39264) 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ccad6700 1 -- 172.21.15.170:0/3747568054 learned_addr learned my addr 172.21.15.170:0/3747568054 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65bffff700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 0x7f65c8155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65cd2d7700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c815db10 0x7f65c80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.620+0000 7f65ccad6700 1 -- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c815db10 msgr2=0x7f65c80b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ccad6700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c815db10 0x7f65c80b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ccad6700 1 -- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 msgr2=0x7f65c8155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ccad6700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 0x7f65c8155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ccad6700 1 -- 172.21.15.170:0/3747568054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009a20 con 0x7f65c8159890 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65bffff700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 0x7f65c8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ccad6700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c7c0 tx=0x7f65c4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4005b30 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c4005cb0 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b8009420 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c4004a80 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.621+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c8157f10 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.622+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c804c3c0 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.622+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f65c40070f0 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.623+0000 7f65bcff9700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 0x7f65b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.623+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f65c4084070 con 0x7f65c8159890 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.623+0000 7f65bffff700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 0x7f65b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.623+0000 7f65bffff700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 0x7f65b40697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f65c815a090 tx=0x7f65b8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.627+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65c4050e00 con 0x7f65c8159890 2023-10-08T16:12:30.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.851+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65c806a5b0 con 0x7f65c8159890 2023-10-08T16:12:30.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.851+0000 7f65bcff9700 1 -- 172.21.15.170:0/3747568054 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f65c4018070 con 0x7f65c8159890 2023-10-08T16:12:30.852 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:30.852 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:30.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.853+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 msgr2=0x7f65b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 0x7f65b40697d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f65c815a090 tx=0x7f65b8017000 comp rx=0 tx=0).stop 2023-10-08T16:12:30.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 msgr2=0x7f65c8156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f65c400c7c0 tx=0x7f65c4007f70 comp rx=0 tx=0).stop 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 shutdown_connections 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65c815db10 0x7f65c80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65b4067320 0x7f65b40697d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65c8159890 0x7f65c8156f40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 --2- 172.21.15.170:0/3747568054 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65c8159c60 0x7f65c8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 >> 172.21.15.170:0/3747568054 conn(0x7f65c8068e20 msgr2=0x7f65c815cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 shutdown_connections 2023-10-08T16:12:30.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.854+0000 7f65ced3a700 1 -- 172.21.15.170:0/3747568054 wait complete. 2023-10-08T16:12:30.856 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2023-10-08T16:12:30.866 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:30.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.980+0000 7ff02d081700 1 Processor -- start 2023-10-08T16:12:30.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.980+0000 7ff02d081700 1 -- start start 2023-10-08T16:12:30.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02d081700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 0x7ff02815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02d081700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff02815db10 0x7ff028069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02d081700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02815a160 con 0x7ff028159890 2023-10-08T16:12:30.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02d081700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0280d13a0 con 0x7ff02815db10 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02d081700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02815e140 con 0x7ff028159c60 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff026d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff028159890 0x7ff02815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39448/0 (socket says 172.21.15.170:39448) 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff026d9d700 1 -- 172.21.15.170:0/1351064082 learned_addr learned my addr 172.21.15.170:0/1351064082 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff01e59c700 1 --2- 172.21.15.170:0/1351064082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 0x7ff02815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02759e700 1 --2- 172.21.15.170:0/1351064082 >> v2:172.21.15.191:3300/0 conn(0x7ff02815db10 0x7ff028069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177517676 0 0) 0x7ff02815a160 con 0x7ff028159890 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.981+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff014003570 con 0x7ff028159890 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff01e59c700 1 -- 172.21.15.170:0/1351064082 >> v2:172.21.15.191:3300/0 conn(0x7ff02815db10 msgr2=0x7ff028069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff01e59c700 1 --2- 172.21.15.170:0/1351064082 >> v2:172.21.15.191:3300/0 conn(0x7ff02815db10 0x7ff028069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff01e59c700 1 -- 172.21.15.170:0/1351064082 >> v1:172.21.15.170:6789/0 conn(0x7ff028159890 legacy=0x7ff02815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107889061 0 0) 0x7ff014003570 con 0x7ff028159890 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff01e59c700 1 -- 172.21.15.170:0/1351064082 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02815a3d0 con 0x7ff028159c60 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff01e59c700 1 --2- 172.21.15.170:0/1351064082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 0x7ff02815d5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0100098e0 tx=0x7ff0100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e272f1b24988bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff018001470 con 0x7ff028159c60 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff010011420 con 0x7ff028159c60 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.982+0000 7ff02659c700 1 -- 172.21.15.170:0/1351064082 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0100117c0 con 0x7ff028159c60 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 -- 172.21.15.170:0/1351064082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 msgr2=0x7ff02815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 --2- 172.21.15.170:0/1351064082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 0x7ff02815d5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0100098e0 tx=0x7ff0100091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 -- 172.21.15.170:0/1351064082 shutdown_connections 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 --2- 172.21.15.170:0/1351064082 >> v2:172.21.15.191:3300/0 conn(0x7ff02815db10 0x7ff028069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 --2- 172.21.15.170:0/1351064082 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff028159c60 0x7ff02815d5d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 -- 172.21.15.170:0/1351064082 >> 172.21.15.170:0/1351064082 conn(0x7ff028068e20 msgr2=0x7ff0280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 -- 172.21.15.170:0/1351064082 shutdown_connections 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.983+0000 7ff02d081700 1 -- 172.21.15.170:0/1351064082 wait complete. 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.984+0000 7ff02d081700 1 Processor -- start 2023-10-08T16:12:30.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.984+0000 7ff02d081700 1 -- start start 2023-10-08T16:12:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 0x7ff0281493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff02815db10 0x7ff028155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280cea50 con 0x7ff028159c60 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280cde40 con 0x7ff028159890 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0280ce120 con 0x7ff02815db10 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39276/0 (socket says 172.21.15.170:39276) 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 -- 172.21.15.170:0/3593832897 learned_addr learned my addr 172.21.15.170:0/3593832897 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02759e700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff02815db10 0x7ff028155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff026d9d700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 0x7ff0281493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 -- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff02815db10 msgr2=0x7ff028155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff02815db10 0x7ff028155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 -- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 msgr2=0x7ff0281493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 0x7ff0281493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 -- 172.21.15.170:0/3593832897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010009420 con 0x7ff028159c60 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff026d9d700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 0x7ff0281493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01e59c700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff0100030c0 tx=0x7ff010016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01001cbe0 con 0x7ff028159c60 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01001cd40 con 0x7ff028159c60 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0280ce400 con 0x7ff028159c60 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01001a6a0 con 0x7ff028159c60 2023-10-08T16:12:30.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.985+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0280cdad0 con 0x7ff028159c60 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.986+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0280cebc0 con 0x7ff028159c60 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.987+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff01001aad0 con 0x7ff028159c60 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.987+0000 7ff01f7fe700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 0x7ff0080697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.987+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff0100864f0 con 0x7ff028159c60 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.987+0000 7ff026d9d700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 0x7ff0080697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.987+0000 7ff026d9d700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 0x7ff0080697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff0180011e0 tx=0x7ff0180015e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:30.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:30.991+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff010053180 con 0x7ff028159c60 2023-10-08T16:12:31.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.177+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff02804c3c0 con 0x7ff028159c60 2023-10-08T16:12:31.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.177+0000 7ff01f7fe700 1 -- 172.21.15.170:0/3593832897 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15412 (secure 0 0 0) 0x7ff0100567a0 con 0x7ff028159c60 2023-10-08T16:12:31.178 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:31.178 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":154,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:19.503764+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:31.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 msgr2=0x7ff0080697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 0x7ff0080697d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff0180011e0 tx=0x7ff0180015e0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 msgr2=0x7ff028152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff0100030c0 tx=0x7ff010016000 comp rx=0 tx=0).stop 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 shutdown_connections 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff02815db10 0x7ff028155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff028159890 0x7ff0281493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff008067320 0x7ff0080697d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 --2- 172.21.15.170:0/3593832897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff028159c60 0x7ff028152270 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 >> 172.21.15.170:0/3593832897 conn(0x7ff028068e20 msgr2=0x7ff028153270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 shutdown_connections 2023-10-08T16:12:31.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.180+0000 7ff02d081700 1 -- 172.21.15.170:0/3593832897 wait complete. 2023-10-08T16:12:31.192 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:12:31.192 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:31.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.306+0000 7f9124e76700 1 Processor -- start 2023-10-08T16:12:31.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.306+0000 7f9124e76700 1 -- start start 2023-10-08T16:12:31.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f912015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 0x7f912015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f912015db10 0x7f9120069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912015a160 con 0x7f912015db10 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91200d13a0 con 0x7f9120159c60 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9124e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912015e140 con 0x7f9120159890 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f911ed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f912015db10 0x7f9120069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f911ed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f912015db10 0x7f9120069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39278/0 (socket says 172.21.15.170:39278) 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f911ed9d700 1 -- 172.21.15.170:0/659384426 learned_addr learned my addr 172.21.15.170:0/659384426 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f911e59c700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f912015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.307+0000 7f9115d9b700 1 --2- 172.21.15.170:0/659384426 >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 0x7f912015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 -- 172.21.15.170:0/659384426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 msgr2=0x7f912015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f912015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 -- 172.21.15.170:0/659384426 >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 msgr2=0x7f912015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 --2- 172.21.15.170:0/659384426 >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 0x7f912015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 -- 172.21.15.170:0/659384426 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912015a3d0 con 0x7f912015db10 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9115d9b700 1 --2- 172.21.15.170:0/659384426 >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 0x7f912015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911ed9d700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f912015db10 0x7f9120069560 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f91200650f0 tx=0x7f9118009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a3ac0d645e6f6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911dd9b700 1 -- 172.21.15.170:0/659384426 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911800e070 con 0x7f912015db10 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911dd9b700 1 -- 172.21.15.170:0/659384426 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800ba20 con 0x7f912015db10 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f911dd9b700 1 -- 172.21.15.170:0/659384426 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180092e0 con 0x7f912015db10 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 -- 172.21.15.170:0/659384426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f912015db10 msgr2=0x7f9120069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f912015db10 0x7f9120069560 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f91200650f0 tx=0x7f9118009840 comp rx=0 tx=0).stop 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 -- 172.21.15.170:0/659384426 shutdown_connections 2023-10-08T16:12:31.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f912015db10 0x7f9120069560 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 --2- 172.21.15.170:0/659384426 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f912015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 --2- 172.21.15.170:0/659384426 >> v2:172.21.15.191:3300/0 conn(0x7f9120159c60 0x7f912015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 -- 172.21.15.170:0/659384426 >> 172.21.15.170:0/659384426 conn(0x7f9120068e20 msgr2=0x7f91200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:31.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 -- 172.21.15.170:0/659384426 shutdown_connections 2023-10-08T16:12:31.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.308+0000 7f9124e76700 1 -- 172.21.15.170:0/659384426 wait complete. 2023-10-08T16:12:31.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 Processor -- start 2023-10-08T16:12:31.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 -- start start 2023-10-08T16:12:31.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f91201491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f912015db10 0x7f91200b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91200b4f70 con 0x7f9120159c60 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9120155bb0 con 0x7f912015db10 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9124e76700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91200b46d0 con 0x7f9120159890 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9115d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9115d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39288/0 (socket says 172.21.15.170:39288) 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.310+0000 7f9115d9b700 1 -- 172.21.15.170:0/2366664043 learned_addr learned my addr 172.21.15.170:0/2366664043 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f911ed9d700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f912015db10 0x7f91200b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f911e59c700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f91201491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 -- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 msgr2=0x7f91201491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f91201491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 -- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f912015db10 msgr2=0x7f91200b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f912015db10 0x7f91200b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 -- 172.21.15.170:0/2366664043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9118009ae0 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9115d9b700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f910800e8a0 tx=0x7f9108008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108013410 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108013570 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91200b49b0 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108013930 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.311+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91200ce710 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.312+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91200ce570 con 0x7f9120159c60 2023-10-08T16:12:31.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.312+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9108013ca0 con 0x7f9120159c60 2023-10-08T16:12:31.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.313+0000 7f9116ffd700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 0x7f91000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.313+0000 7f911e59c700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 0x7f91000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.313+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9108084a30 con 0x7f9120159c60 2023-10-08T16:12:31.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.313+0000 7f911e59c700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 0x7f91000697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9110009db0 tx=0x7f91100091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.317+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91200ce570 con 0x7f9120159c60 2023-10-08T16:12:31.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.541+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91200ce570 con 0x7f9120159c60 2023-10-08T16:12:31.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.542+0000 7f9116ffd700 1 -- 172.21.15.170:0/2366664043 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f91200ce570 con 0x7f9120159c60 2023-10-08T16:12:31.542 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:31.542 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.543+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 msgr2=0x7f91000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.543+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 0x7f91000697d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9110009db0 tx=0x7f91100091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 msgr2=0x7f91200b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f910800e8a0 tx=0x7f9108008040 comp rx=0 tx=0).stop 2023-10-08T16:12:31.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 shutdown_connections 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f912015db10 0x7f91200b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9120159890 0x7f91201491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9100067320 0x7f91000697d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 --2- 172.21.15.170:0/2366664043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9120159c60 0x7f91200b1a20 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 >> 172.21.15.170:0/2366664043 conn(0x7f9120068e20 msgr2=0x7f912015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 shutdown_connections 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.544+0000 7f9124e76700 1 -- 172.21.15.170:0/2366664043 wait complete. 2023-10-08T16:12:31.545 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2023-10-08T16:12:31.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:31.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.670+0000 7f81f2863700 1 Processor -- start 2023-10-08T16:12:31.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.670+0000 7f81f2863700 1 -- start start 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2863700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81ec0ad870 0x7f81ec06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2863700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec0b40f0 con 0x7f81ec06bf20 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2863700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ec069640 con 0x7f81ec142ea0 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2863700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ec06c3a0 con 0x7f81ec0ad870 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2062700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f81ec06bf20 0x7f81ec06a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39450/0 (socket says 172.21.15.170:39450) 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f2062700 1 -- 172.21.15.170:0/937671101 learned_addr learned my addr 172.21.15.170:0/937671101 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941244074 0 0) 0x7f81ec0b40f0 con 0x7f81ec06bf20 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f1060700 1 --2- 172.21.15.170:0/937671101 >> v2:172.21.15.191:3300/0 conn(0x7f81ec0ad870 0x7f81ec06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.671+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d8003570 con 0x7f81ec06bf20 2023-10-08T16:12:31.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3780559299 0 0) 0x7f81d8003570 con 0x7f81ec06bf20 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 >> v2:172.21.15.191:3300/0 conn(0x7f81ec0ad870 msgr2=0x7f81ec06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 --2- 172.21.15.170:0/937671101 >> v2:172.21.15.191:3300/0 conn(0x7f81ec0ad870 0x7f81ec06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 >> v1:172.21.15.191:6789/0 conn(0x7f81ec142ea0 legacy=0x7f81ec0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec069920 con 0x7f81ec06bf20 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f81e4001470 con 0x7f81ec06bf20 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 >> v1:172.21.15.170:6789/0 conn(0x7f81ec06bf20 legacy=0x7f81ec06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81d8003a60 0x7f81d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 0x7f81d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d80051c0 0x7f81d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec06c3a0 con 0x7f81d8004810 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d800c050 con 0x7f81d80051c0 2023-10-08T16:12:31.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d800c210 con 0x7f81d8003a60 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 0x7f81d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f2062700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d80051c0 0x7f81d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1861700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81d8003a60 0x7f81d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 -- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81d8003a60 msgr2=0x7f81d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81d8003a60 0x7f81d8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 -- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d80051c0 msgr2=0x7f81d800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d80051c0 0x7f81d800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 -- 172.21.15.170:0/937671101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d8003570 con 0x7f81d8004810 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f2062700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d80051c0 0x7f81d800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f2062700 1 -- 172.21.15.170:0/937671101 reap_dead start 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f1060700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 0x7f81d8004c80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0b12a0 tx=0x7f81dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804383271fa32d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00e710 con 0x7f81d8004810 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc00e870 con 0x7f81d8004810 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.672+0000 7f81f085f700 1 -- 172.21.15.170:0/937671101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00ec30 con 0x7f81d8004810 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 -- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 msgr2=0x7f81d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 0x7f81d8004c80 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0b12a0 tx=0x7f81dc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 -- 172.21.15.170:0/937671101 shutdown_connections 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 --2- 172.21.15.170:0/937671101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81d8004810 0x7f81d8004c80 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 -- 172.21.15.170:0/937671101 >> 172.21.15.170:0/937671101 conn(0x7f81ec0ce430 msgr2=0x7f81ec0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 -- 172.21.15.170:0/937671101 shutdown_connections 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.673+0000 7f81f2863700 1 -- 172.21.15.170:0/937671101 wait complete. 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.674+0000 7f81f2863700 1 Processor -- start 2023-10-08T16:12:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.674+0000 7f81f2863700 1 -- start start 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d8004810 0x7f81ec13fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ec15a400 0x7f81ec0c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec15cbd0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec15c3c0 con 0x7f81d8004810 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2863700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ec1699a0 con 0x7f81ec15a400 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39310/0 (socket says 172.21.15.170:39310) 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 -- 172.21.15.170:0/3925650199 learned_addr learned my addr 172.21.15.170:0/3925650199 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f1861700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d8004810 0x7f81ec13fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f1060700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ec15a400 0x7f81ec0c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 -- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ec15a400 msgr2=0x7f81ec0c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ec15a400 0x7f81ec0c85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 -- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d8004810 msgr2=0x7f81ec13fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d8004810 0x7f81ec13fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.675+0000 7f81f2062700 1 -- 172.21.15.170:0/3925650199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec169c80 con 0x7f81ec0c8af0 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81f2062700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f81ec06f390 tx=0x7f81e4001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400c540 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e400c6c0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e400ca80 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ec15c610 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.676+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ec15a870 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.677+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ec169e10 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.677+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f81e40133e0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.677+0000 7f81e1ffb700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 0x7f81d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.677+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f81e4084640 con 0x7f81ec0c8af0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.677+0000 7f81f1861700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 0x7f81d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:31.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.678+0000 7f81f1861700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 0x7f81d00697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f81e8004290 tx=0x7f81e80016d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:31.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.682+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81e40512d0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.908+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81ec04c3c0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.908+0000 7f81e1ffb700 1 -- 172.21.15.170:0/3925650199 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4615 (secure 0 0 0) 0x7f81e40548f0 con 0x7f81ec0c8af0 2023-10-08T16:12:31.909 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:31.909 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":101,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:18.390075+0000","modified":"2023-10-08T16:12:22.040533+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7857},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7857":{"gid":7857,"name":"c","rank":0,"incarnation":96,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1646734631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1646734631},{"type":"v1","addr":"172.21.15.170:6837","nonce":1646734631}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 msgr2=0x7f81d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 0x7f81d00697d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f81e8004290 tx=0x7f81e80016d0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 msgr2=0x7f81ec16a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f81ec06f390 tx=0x7f81e4001cc0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 shutdown_connections 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81d0067320 0x7f81d00697d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81ec0c8af0 0x7f81ec16a110 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81d8004810 0x7f81ec13fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 --2- 172.21.15.170:0/3925650199 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81ec15a400 0x7f81ec0c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 >> 172.21.15.170:0/3925650199 conn(0x7f81ec0ce430 msgr2=0x7f81ec0afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 shutdown_connections 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:31.910+0000 7f81f2863700 1 -- 172.21.15.170:0/3925650199 wait complete. 2023-10-08T16:12:31.912 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 101 2023-10-08T16:12:31.923 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:32.037 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.037+0000 7f89ed9b1700 1 Processor -- start 2023-10-08T16:12:32.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.037+0000 7f89ed9b1700 1 -- start start 2023-10-08T16:12:32.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed9b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89e8159420 0x7f89e80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed9b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e81597f0 0x7f89e80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed9b1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e80b11d0 con 0x7f89e815d230 2023-10-08T16:12:32.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed9b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e80b14b0 con 0x7f89e8159420 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed9b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80b1790 con 0x7f89e81597f0 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed1b0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89e815d230 0x7f89e80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39458/0 (socket says 172.21.15.170:39458) 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ed1b0700 1 -- 172.21.15.170:0/1314314304 learned_addr learned my addr 172.21.15.170:0/1314314304 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ec9af700 1 --2- 172.21.15.170:0/1314314304 >> v2:172.21.15.191:3300/0 conn(0x7f89e8159420 0x7f89e80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89e7fff700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e81597f0 0x7f89e80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89e77fe700 1 -- 172.21.15.170:0/1314314304 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834434688 0 0) 0x7f89e80b11d0 con 0x7f89e815d230 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89e77fe700 1 -- 172.21.15.170:0/1314314304 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d8003570 con 0x7f89e815d230 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ec9af700 1 -- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e81597f0 msgr2=0x7f89e80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ec9af700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e81597f0 0x7f89e80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ec9af700 1 -- 172.21.15.170:0/1314314304 >> v1:172.21.15.170:6789/0 conn(0x7f89e815d230 legacy=0x7f89e80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.038+0000 7f89ec9af700 1 -- 172.21.15.170:0/1314314304 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80b1a70 con 0x7f89e8159420 2023-10-08T16:12:32.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ec9af700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e80b5f40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098e0 tx=0x7f89dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9946d40b8d2f567 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89e77fe700 1 -- 172.21.15.170:0/1314314304 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc011630 con 0x7f89e8159420 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89e77fe700 1 -- 172.21.15.170:0/1314314304 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc011790 con 0x7f89e8159420 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89e77fe700 1 -- 172.21.15.170:0/1314314304 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc011b30 con 0x7f89e8159420 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 -- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 msgr2=0x7f89e80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e80b5f40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0098e0 tx=0x7f89dc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 -- 172.21.15.170:0/1314314304 shutdown_connections 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e80b5f40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/1314314304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e81597f0 0x7f89e80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 -- 172.21.15.170:0/1314314304 >> 172.21.15.170:0/1314314304 conn(0x7f89e80d0660 msgr2=0x7f89e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 -- 172.21.15.170:0/1314314304 shutdown_connections 2023-10-08T16:12:32.042 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.039+0000 7f89ed9b1700 1 -- 172.21.15.170:0/1314314304 wait complete. 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.040+0000 7f89ed9b1700 1 Processor -- start 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 -- start start 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e816d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 0x7f89e80c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c6e00 con 0x7f89e81597f0 2023-10-08T16:12:32.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e816dc20 con 0x7f89e8159420 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89ed9b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80c7cb0 con 0x7f89e815d230 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89e7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39314/0 (socket says 172.21.15.170:39314) 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.041+0000 7f89e7fff700 1 -- 172.21.15.170:0/4021474270 learned_addr learned my addr 172.21.15.170:0/4021474270 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89ec9af700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e816d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89ed1b0700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 0x7f89e80c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 -- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 msgr2=0x7f89e80c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 0x7f89e80c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 -- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 msgr2=0x7f89e816d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e816d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 -- 172.21.15.170:0/4021474270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc009420 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89ed1b0700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 0x7f89e80c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e7fff700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f89d400ec80 tx=0x7f89d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400c9d0 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d400cb30 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e80c61f0 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4013470 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.042+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e80c67b0 con 0x7f89e81597f0 2023-10-08T16:12:32.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.043+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e815ac30 con 0x7f89e81597f0 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.044+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89d4013820 con 0x7f89e81597f0 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.044+0000 7f89e4ff9700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 0x7f89cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.044+0000 7f89ec9af700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 0x7f89cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.044+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f89d4085030 con 0x7f89e81597f0 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.045+0000 7f89ec9af700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 0x7f89cc0697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0091e0 tx=0x7f89dc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.048+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e815ac30 con 0x7f89e81597f0 2023-10-08T16:12:32.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.232+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89e815ac30 con 0x7f89e81597f0 2023-10-08T16:12:32.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.233+0000 7f89e4ff9700 1 -- 172.21.15.170:0/4021474270 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f89e815ac30 con 0x7f89e81597f0 2023-10-08T16:12:32.233 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:32.234 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":154,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:19.503764+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:16.504693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":27,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:17.396600+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 msgr2=0x7f89cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 0x7f89cc0697d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0091e0 tx=0x7f89dc017000 comp rx=0 tx=0).stop 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 msgr2=0x7f89e8157160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f89d400ec80 tx=0x7f89d40091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 shutdown_connections 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e815d230 0x7f89e80c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89e8159420 0x7f89e816d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89cc067320 0x7f89cc0697d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 --2- 172.21.15.170:0/4021474270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e81597f0 0x7f89e8157160 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 >> 172.21.15.170:0/4021474270 conn(0x7f89e80d0660 msgr2=0x7f89e80699f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:32.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 shutdown_connections 2023-10-08T16:12:32.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.235+0000 7f89ed9b1700 1 -- 172.21.15.170:0/4021474270 wait complete. 2023-10-08T16:12:32.248 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:12:32.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.363+0000 7fc15a4ff700 1 Processor -- start 2023-10-08T16:12:32.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.363+0000 7fc15a4ff700 1 -- start start 2023-10-08T16:12:32.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc15a4ff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 0x7fc15415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc15a4ff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc154159c60 0x7fc15415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc15a4ff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc15415a160 con 0x7fc15415db10 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc15a4ff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1540d13a0 con 0x7fc154159c60 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc15a4ff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15415e140 con 0x7fc154159890 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc158a9c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc15415db10 0x7fc154069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39468/0 (socket says 172.21.15.170:39468) 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc158a9c700 1 -- 172.21.15.170:0/2520616233 learned_addr learned my addr 172.21.15.170:0/2520616233 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc1537fe700 1 --2- 172.21.15.170:0/2520616233 >> v2:172.21.15.191:3300/0 conn(0x7fc154159c60 0x7fc15415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.364+0000 7fc153fff700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 0x7fc15415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097806836 0 0) 0x7fc15415a160 con 0x7fc15415db10 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc13c003570 con 0x7fc15415db10 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326743780 0 0) 0x7fc13c003570 con 0x7fc15415db10 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc1537fe700 1 -- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 msgr2=0x7fc15415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.367 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc1537fe700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 0x7fc15415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc1537fe700 1 -- 172.21.15.170:0/2520616233 >> v1:172.21.15.170:6789/0 conn(0x7fc15415db10 legacy=0x7fc154069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc1537fe700 1 -- 172.21.15.170:0/2520616233 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15415a3d0 con 0x7fc154159c60 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc153fff700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 0x7fc15415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc1537fe700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159c60 0x7fc15415d5d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1480098e0 tx=0x7fc1480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d598e84b7d1533e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0017f0 con 0x7fc154159c60 2023-10-08T16:12:32.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc148011420 con 0x7fc154159c60 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.365+0000 7fc152ffd700 1 -- 172.21.15.170:0/2520616233 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480117c0 con 0x7fc154159c60 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 -- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159c60 msgr2=0x7fc15415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159c60 0x7fc15415d5d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc1480098e0 tx=0x7fc1480091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 -- 172.21.15.170:0/2520616233 shutdown_connections 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc154159890 0x7fc15415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/2520616233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159c60 0x7fc15415d5d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 -- 172.21.15.170:0/2520616233 >> 172.21.15.170:0/2520616233 conn(0x7fc154068e20 msgr2=0x7fc1540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:32.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 -- 172.21.15.170:0/2520616233 shutdown_connections 2023-10-08T16:12:32.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.366+0000 7fc15a4ff700 1 -- 172.21.15.170:0/2520616233 wait complete. 2023-10-08T16:12:32.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.367+0000 7fc15a4ff700 1 Processor -- start 2023-10-08T16:12:32.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.367+0000 7fc15a4ff700 1 -- start start 2023-10-08T16:12:32.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 0x7fc15416d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 0x7fc1540cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15416de90 con 0x7fc154159c60 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc154140d10 con 0x7fc154159890 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1540b58d0 con 0x7fc15415db10 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39330/0 (socket says 172.21.15.170:39330) 2023-10-08T16:12:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 -- 172.21.15.170:0/713746285 learned_addr learned my addr 172.21.15.170:0/713746285 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc158a9c700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 0x7fc1540cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 -- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 msgr2=0x7fc1540cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc153fff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 0x7fc15416d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 0x7fc1540cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 -- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 msgr2=0x7fc15416d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 0x7fc15416d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 -- 172.21.15.170:0/713746285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc144009420 con 0x7fc154159c60 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc158a9c700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 0x7fc1540cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc153fff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 0x7fc15416d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc1537fe700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc1480030c0 tx=0x7fc148016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801cc90 con 0x7fc154159c60 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14801cdf0 con 0x7fc154159c60 2023-10-08T16:12:32.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc148009420 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801a6a0 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.368+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc154152430 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.369+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1540b5a60 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.370+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc14801aad0 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.370+0000 7fc13bfff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 0x7fc1400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.370+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc1480862d0 con 0x7fc154159c60 2023-10-08T16:12:32.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.370+0000 7fc153fff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 0x7fc1400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.371+0000 7fc153fff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 0x7fc1400697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1440093f0 tx=0x7fc144008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.380 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.374+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc148052fe0 con 0x7fc154159c60 2023-10-08T16:12:32.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.601+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc15406a5b0 con 0x7fc154159c60 2023-10-08T16:12:32.603 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:12:32.602+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:12:32.604 INFO:tasks.ceph.mds.c.smithi170.stderr: -1796> 2023-10-08T16:12:13.162+0000 7fdb98b23700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-10-08T16:12:32.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.605+0000 7fc13bfff700 1 -- 172.21.15.170:0/713746285 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc148056580 con 0x7fc154159c60 2023-10-08T16:12:32.606 INFO:tasks.ceph.mds.c.smithi170.stderr: -620> 2023-10-08T16:12:23.992+0000 7fdb92b17700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:12:32.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 msgr2=0x7fc1400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 0x7fc1400697d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc1440093f0 tx=0x7fc144008040 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 msgr2=0x7fc1540cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc1480030c0 tx=0x7fc148016000 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 shutdown_connections 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc15415db10 0x7fc1540cfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc154159890 0x7fc15416d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc140067320 0x7fc1400697d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 --2- 172.21.15.170:0/713746285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc154159c60 0x7fc1540cd0c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 >> 172.21.15.170:0/713746285 conn(0x7fc154068e20 msgr2=0x7fc154153070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 shutdown_connections 2023-10-08T16:12:32.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.608+0000 7fc15a4ff700 1 -- 172.21.15.170:0/713746285 wait complete. 2023-10-08T16:12:32.610 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:12:32.621 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:12:32.626 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:12:32.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.737+0000 7fdd5eb2a700 1 Processor -- start 2023-10-08T16:12:32.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.737+0000 7fdd5eb2a700 1 -- start start 2023-10-08T16:12:32.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5eb2a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd58159c60 0x7fdd5815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5eb2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd58069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5eb2a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5815a160 con 0x7fdd58159890 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5eb2a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5815df50 con 0x7fdd58159c60 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5eb2a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580d13a0 con 0x7fdd5815db10 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5c8c6700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdd58159890 0x7fdd5815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39470/0 (socket says 172.21.15.170:39470) 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5c8c6700 1 -- 172.21.15.170:0/1550913841 learned_addr learned my addr 172.21.15.170:0/1550913841 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> v2:172.21.15.191:3300/0 conn(0x7fdd58159c60 0x7fdd5815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd5d0c7700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd58069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408870679 0 0) 0x7fdd5815a160 con 0x7fdd58159890 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.738+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40003570 con 0x7fdd58159890 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767069586 0 0) 0x7fdd40003570 con 0x7fdd58159890 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 msgr2=0x7fdd58069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd58069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 >> v2:172.21.15.191:3300/0 conn(0x7fdd58159c60 msgr2=0x7fdd5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 --2- 172.21.15.170:0/1550913841 >> v2:172.21.15.191:3300/0 conn(0x7fdd58159c60 0x7fdd5815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5815e140 con 0x7fdd58159890 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fdd540017f0 con 0x7fdd58159890 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd5d0c7700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd58069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> v2:172.21.15.191:3300/0 conn(0x7fdd58159c60 0x7fdd5815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:32.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 >> v1:172.21.15.170:6789/0 conn(0x7fdd58159890 legacy=0x7fdd5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd40003a60 0x7fdd40003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd580d13a0 con 0x7fdd40004810 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4000ba60 con 0x7fdd40003a60 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4000bba0 con 0x7fdd5815db10 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd5c8c6700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd40003a60 0x7fdd40003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd5d0c7700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 -- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 msgr2=0x7fdd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd5815db10 0x7fdd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 -- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd40003a60 msgr2=0x7fdd40003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd40003a60 0x7fdd40003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 -- 172.21.15.170:0/1550913841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50009420 con 0x7fdd40004810 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd40004c80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009df0 tx=0x7fdd480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91af601edae406f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800e920 con 0x7fdd40004810 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800ea80 con 0x7fdd40004810 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.739+0000 7fdd4f7fe700 1 -- 172.21.15.170:0/1550913841 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800ee20 con 0x7fdd40004810 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/1550913841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 msgr2=0x7fdd40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/1550913841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd40004c80 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009df0 tx=0x7fdd480091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5d0c7700 1 -- 172.21.15.170:0/1550913841 reap_dead start 2023-10-08T16:12:32.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/1550913841 shutdown_connections 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/1550913841 >> 172.21.15.170:0/1550913841 conn(0x7fdd58068e20 msgr2=0x7fdd580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/1550913841 shutdown_connections 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.740+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/1550913841 wait complete. 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.741+0000 7fdd5eb2a700 1 Processor -- start 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.741+0000 7fdd5eb2a700 1 -- start start 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd580b3590 0x7fdd5814d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 0x7fdd5816dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815d2f0 con 0x7fdd40004810 2023-10-08T16:12:32.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58156350 con 0x7fdd5816d760 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5eb2a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5815d010 con 0x7fdd580b3590 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39364/0 (socket says 172.21.15.170:39364) 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 -- 172.21.15.170:0/459310224 learned_addr learned my addr 172.21.15.170:0/459310224 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5d0c7700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 0x7fdd5816dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd580b3590 0x7fdd5814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 -- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd580b3590 msgr2=0x7fdd5814d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd580b3590 0x7fdd5814d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 -- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 msgr2=0x7fdd5816dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 0x7fdd5816dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 -- 172.21.15.170:0/459310224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58155d00 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5d0c7700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 0x7fdd5816dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.742+0000 7fdd5c8c6700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdd540019a0 tx=0x7fdd54002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.743+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5400ae80 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.743+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540133f0 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.743+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd58155fe0 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.743+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54013790 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.743+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5816e2e0 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.744+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5804c3c0 con 0x7fdd40004810 2023-10-08T16:12:32.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.744+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd54013b80 con 0x7fdd40004810 2023-10-08T16:12:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.744+0000 7fdd4cff9700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 0x7fdd440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.744+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fdd54085c20 con 0x7fdd40004810 2023-10-08T16:12:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.744+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 0x7fdd440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.745+0000 7fdd4ffff700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 0x7fdd440697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd480091e0 tx=0x7fdd4801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:32.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.748+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd54052860 con 0x7fdd40004810 2023-10-08T16:12:32.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:32.973+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd580b0140 con 0x7fdd40004810 2023-10-08T16:12:33.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.607+0000 7fdd4cff9700 1 -- 172.21.15.170:0/459310224 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd54055e80 con 0x7fdd40004810 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 msgr2=0x7fdd440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 0x7fdd440697d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd480091e0 tx=0x7fdd4801a000 comp rx=0 tx=0).stop 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 msgr2=0x7fdd5814ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdd540019a0 tx=0x7fdd54002210 comp rx=0 tx=0).stop 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 shutdown_connections 2023-10-08T16:12:33.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd5816d760 0x7fdd5816dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd44067320 0x7fdd440697d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd40004810 0x7fdd5814ced0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 --2- 172.21.15.170:0/459310224 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd580b3590 0x7fdd5814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.609+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 >> 172.21.15.170:0/459310224 conn(0x7fdd58068e20 msgr2=0x7fdd580c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.610+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 shutdown_connections 2023-10-08T16:12:33.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.610+0000 7fdd5eb2a700 1 -- 172.21.15.170:0/459310224 wait complete. 2023-10-08T16:12:33.624 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:12:33.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 Processor -- start 2023-10-08T16:12:33.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 -- start start 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01d00b55a0 0x7f01d0164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d00cfa60 con 0x7f01d0159ec0 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d00b5b00 con 0x7f01d00cd910 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01d5eeb700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d00b5da0 con 0x7f01d00b55a0 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.785+0000 7f01cffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01d00b55a0 0x7f01d0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:33.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cf7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f01d0159ec0 0x7f01d00b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39484/0 (socket says 172.21.15.170:39484) 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f01d00b55a0 0x7f01d0164400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39376/0 (socket says 172.21.15.170:39376) 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cf7fe700 1 -- 172.21.15.170:0/3271618277 learned_addr learned my addr 172.21.15.170:0/3271618277 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cffff700 1 -- 172.21.15.170:0/3271618277 >> v1:172.21.15.191:6789/0 conn(0x7f01d00cd910 legacy=0x7f01d00ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cffff700 1 -- 172.21.15.170:0/3271618277 >> v1:172.21.15.170:6789/0 conn(0x7f01d0159ec0 legacy=0x7f01d00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cffff700 1 -- 172.21.15.170:0/3271618277 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00b6080 con 0x7f01d00b55a0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01ce7fc700 1 -- 172.21.15.170:0/3271618277 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261572327 0 0) 0x7f01d00cfa60 con 0x7f01d0159ec0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01cffff700 1 --2- 172.21.15.170:0/3271618277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d00b55a0 0x7f01d0164400 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009910 tx=0x7f01c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb036c04cf479c0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01ce7fc700 1 -- 172.21.15.170:0/3271618277 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c800b7e0 con 0x7f01d00b55a0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01ce7fc700 1 -- 172.21.15.170:0/3271618277 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c800b940 con 0x7f01d00b55a0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.786+0000 7f01ce7fc700 1 -- 172.21.15.170:0/3271618277 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c800bce0 con 0x7f01d00b55a0 2023-10-08T16:12:33.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 -- 172.21.15.170:0/3271618277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d00b55a0 msgr2=0x7f01d0164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/3271618277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d00b55a0 0x7f01d0164400 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009910 tx=0x7f01c80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 -- 172.21.15.170:0/3271618277 shutdown_connections 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/3271618277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d00b55a0 0x7f01d0164400 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 -- 172.21.15.170:0/3271618277 >> 172.21.15.170:0/3271618277 conn(0x7f01d00999f0 msgr2=0x7f01d00b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 -- 172.21.15.170:0/3271618277 shutdown_connections 2023-10-08T16:12:33.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.787+0000 7f01d5eeb700 1 -- 172.21.15.170:0/3271618277 wait complete. 2023-10-08T16:12:33.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.788+0000 7f01d5eeb700 1 Processor -- start 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.788+0000 7f01d5eeb700 1 -- start start 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d00b55a0 0x7f01d0151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d0159ec0 0x7f01d0144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d00684c0 con 0x7f01d006adf0 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0153840 con 0x7f01d0159ec0 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d0153b20 con 0x7f01d00b55a0 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39392/0 (socket says 172.21.15.170:39392) 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 -- 172.21.15.170:0/5384196 learned_addr learned my addr 172.21.15.170:0/5384196 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:33.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cf7fe700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d00b55a0 0x7f01d0151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01ceffd700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d0159ec0 0x7f01d0144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 -- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d00b55a0 msgr2=0x7f01d0151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d00b55a0 0x7f01d0151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 -- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d0159ec0 msgr2=0x7f01d0144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d0159ec0 0x7f01d0144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 -- 172.21.15.170:0/5384196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8009420 con 0x7f01d006adf0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01cffff700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098e0 tx=0x7f01c8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8011ea0 con 0x7f01d006adf0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8020070 con 0x7f01d006adf0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d0153e00 con 0x7f01d006adf0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c801b680 con 0x7f01d006adf0 2023-10-08T16:12:33.793 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.789+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d006b260 con 0x7f01d006adf0 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.790+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d0063c90 con 0x7f01d006adf0 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.791+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f01c8022030 con 0x7f01d006adf0 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.791+0000 7f01b77fe700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 0x7f01bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.791+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f01c8086c10 con 0x7f01d006adf0 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.791+0000 7f01cf7fe700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 0x7f01bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:33.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.792+0000 7f01cf7fe700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 0x7f01bc0697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f01d00cad60 tx=0x7f01c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:33.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.796+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01c8053960 con 0x7f01d006adf0 2023-10-08T16:12:33.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:33.982+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01d0063c90 con 0x7f01d006adf0 2023-10-08T16:12:34.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.612+0000 7f01b77fe700 1 -- 172.21.15.170:0/5384196 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v156) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01c8006cb0 con 0x7f01d006adf0 2023-10-08T16:12:34.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 msgr2=0x7f01bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 0x7f01bc0697d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f01d00cad60 tx=0x7f01c0009040 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 msgr2=0x7f01d0154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098e0 tx=0x7f01c8016000 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 shutdown_connections 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f01bc067320 0x7f01bc0697d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01d006adf0 0x7f01d0154970 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f01d00b55a0 0x7f01d0151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 --2- 172.21.15.170:0/5384196 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f01d0159ec0 0x7f01d0144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 >> 172.21.15.170:0/5384196 conn(0x7f01d00999f0 msgr2=0x7f01d00c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 shutdown_connections 2023-10-08T16:12:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.618+0000 7f01d5eeb700 1 -- 172.21.15.170:0/5384196 wait complete. 2023-10-08T16:12:34.620 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:12:34.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:12:34.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.752+0000 7f7016b14700 1 Processor -- start 2023-10-08T16:12:34.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.752+0000 7f7016b14700 1 -- start start 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f7016b14700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7010159ec0 0x7f70100b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f7016b14700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100b5b00 con 0x7f70100cf350 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f7016b14700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100b5850 con 0x7f70100b53d0 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f7016b14700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100b5da0 con 0x7f7010159ec0 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7010159ec0 0x7f70100b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7010159ec0 0x7f70100b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39404/0 (socket says 172.21.15.170:39404) 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70150b1700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f70100cf350 0x7f70100cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39486/0 (socket says 172.21.15.170:39486) 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 -- 172.21.15.170:0/2425739225 learned_addr learned my addr 172.21.15.170:0/2425739225 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 -- 172.21.15.170:0/2425739225 >> v1:172.21.15.191:6789/0 conn(0x7f70100b53d0 legacy=0x7f70100cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 -- 172.21.15.170:0/2425739225 >> v1:172.21.15.170:6789/0 conn(0x7f70100cf350 legacy=0x7f70100cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 -- 172.21.15.170:0/2425739225 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100b6080 con 0x7f7010159ec0 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f700f7fe700 1 -- 172.21.15.170:0/2425739225 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678095413 0 0) 0x7f70100b5b00 con 0x7f70100cf350 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f70148b0700 1 --2- 172.21.15.170:0/2425739225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f70100b4e90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7008009f70 tx=0x7f7008009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2984fcc98dccdca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.753+0000 7f700f7fe700 1 -- 172.21.15.170:0/2425739225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700800e070 con 0x7f7010159ec0 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f700f7fe700 1 -- 172.21.15.170:0/2425739225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700800ba90 con 0x7f7010159ec0 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f700f7fe700 1 -- 172.21.15.170:0/2425739225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080092e0 con 0x7f7010159ec0 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 -- 172.21.15.170:0/2425739225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 msgr2=0x7f70100b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 --2- 172.21.15.170:0/2425739225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f70100b4e90 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7008009f70 tx=0x7f7008009840 comp rx=0 tx=0).stop 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 -- 172.21.15.170:0/2425739225 shutdown_connections 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 --2- 172.21.15.170:0/2425739225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f70100b4e90 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 -- 172.21.15.170:0/2425739225 >> 172.21.15.170:0/2425739225 conn(0x7f70100999f0 msgr2=0x7f70100b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:34.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 -- 172.21.15.170:0/2425739225 shutdown_connections 2023-10-08T16:12:34.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.754+0000 7f7016b14700 1 -- 172.21.15.170:0/2425739225 wait complete. 2023-10-08T16:12:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.755+0000 7f7016b14700 1 Processor -- start 2023-10-08T16:12:34.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.755+0000 7f7016b14700 1 -- start start 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70100b53d0 0x7f70101573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 0x7f7010069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100690d0 con 0x7f7010159ec0 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100684c0 con 0x7f70100b53d0 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f7016b14700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70100687a0 con 0x7f70100cf350 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70150b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70150b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:39414/0 (socket says 172.21.15.170:39414) 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70150b1700 1 -- 172.21.15.170:0/2817529944 learned_addr learned my addr 172.21.15.170:0/2817529944 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70148b0700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70100b53d0 0x7f70101573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:34.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f700ffff700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 0x7f7010069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70150b1700 1 -- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 msgr2=0x7f7010069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.756+0000 7f70150b1700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 0x7f7010069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f70150b1700 1 -- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70100b53d0 msgr2=0x7f70101573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f70150b1700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70100b53d0 0x7f70101573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f70150b1700 1 -- 172.21.15.170:0/2817529944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008009ae0 con 0x7f7010159ec0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f700ffff700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 0x7f7010069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f70150b1700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f700000c5e0 tx=0x7f7000007d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000005a70 con 0x7f7010159ec0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000005bd0 con 0x7f7010159ec0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7010068a80 con 0x7f7010159ec0 2023-10-08T16:12:34.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000070d0 con 0x7f7010159ec0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.757+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7010157910 con 0x7f7010159ec0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.758+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7010069280 con 0x7f7010159ec0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.758+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7000007440 con 0x7f7010159ec0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.759+0000 7f700cff9700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 0x7f6ffc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.759+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5353+0+0 (secure 0 0 0) 0x7f7000084130 con 0x7f7010159ec0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.759+0000 7f70148b0700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 0x7f6ffc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:34.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.759+0000 7f70148b0700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 0x7f6ffc0697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7008009880 tx=0x7f7008017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:34.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.763+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7010069280 con 0x7f7010159ec0 2023-10-08T16:12:34.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:34.946+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7010063c90 con 0x7f7010159ec0 2023-10-08T16:12:35.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.617+0000 7f700cff9700 1 -- 172.21.15.170:0/2817529944 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v157) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7000050f20 con 0x7f7010159ec0 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 msgr2=0x7f6ffc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 0x7f6ffc0697d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7008009880 tx=0x7f7008017000 comp rx=0 tx=0).stop 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 msgr2=0x7f701009b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f700000c5e0 tx=0x7f7000007d60 comp rx=0 tx=0).stop 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 shutdown_connections 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6ffc067320 0x7f6ffc0697d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7010159ec0 0x7f701009b2f0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f70100b53d0 0x7f70101573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 --2- 172.21.15.170:0/2817529944 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f70100cf350 0x7f7010069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 >> 172.21.15.170:0/2817529944 conn(0x7f70100999f0 msgr2=0x7f7010159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.621+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 shutdown_connections 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.622+0000 7f7016b14700 1 -- 172.21.15.170:0/2817529944 wait complete. 2023-10-08T16:12:35.623 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:12:35.635 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:12:35.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.750+0000 7ff5fa799700 1 Processor -- start 2023-10-08T16:12:35.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.750+0000 7ff5fa799700 1 -- start start 2023-10-08T16:12:35.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.751+0000 7ff5fa799700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff5f406b160 0x7ff5f406b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.751+0000 7ff5fa799700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f406a2b0 0x7ff5f406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.751+0000 7ff5fa799700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40ac450 con 0x7ff5f40637d0 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.751+0000 7ff5fa799700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f40693d0 con 0x7ff5f406b160 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.751+0000 7ff5fa799700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4069d00 con 0x7ff5f406a2b0 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff5f406b160 0x7ff5f406b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f8d36700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f406a2b0 0x7ff5f406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f37fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff5f40637d0 0x7ff5f4068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:59472/0 (socket says 172.21.15.170:59472) 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f37fe700 1 -- 172.21.15.170:0/1838946068 learned_addr learned my addr 172.21.15.170:0/1838946068 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 -- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f406a2b0 msgr2=0x7ff5f406a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 --2- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f406a2b0 0x7ff5f406a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 -- 172.21.15.170:0/1838946068 >> v1:172.21.15.191:6789/0 conn(0x7ff5f40637d0 legacy=0x7ff5f4068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 -- 172.21.15.170:0/1838946068 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4068830 con 0x7ff5f406b160 2023-10-08T16:12:35.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f3fff700 1 --2- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406b160 0x7ff5f406b530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098e0 tx=0x7ff5e40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c39f94b336da56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:35.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f2ffd700 1 -- 172.21.15.170:0/1838946068 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400b640 con 0x7ff5f406b160 2023-10-08T16:12:35.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f2ffd700 1 -- 172.21.15.170:0/1838946068 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400b7a0 con 0x7ff5f406b160 2023-10-08T16:12:35.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.752+0000 7ff5f2ffd700 1 -- 172.21.15.170:0/1838946068 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400bb40 con 0x7ff5f406b160 2023-10-08T16:12:35.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 -- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406b160 msgr2=0x7ff5f406b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406b160 0x7ff5f406b530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098e0 tx=0x7ff5e40091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 -- 172.21.15.170:0/1838946068 shutdown_connections 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f406a2b0 0x7ff5f406a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1838946068 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406b160 0x7ff5f406b530 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 -- 172.21.15.170:0/1838946068 >> 172.21.15.170:0/1838946068 conn(0x7ff5f406ded0 msgr2=0x7ff5f406e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 -- 172.21.15.170:0/1838946068 shutdown_connections 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.753+0000 7ff5fa799700 1 -- 172.21.15.170:0/1838946068 wait complete. 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.754+0000 7ff5fa799700 1 Processor -- start 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.754+0000 7ff5fa799700 1 -- start start 2023-10-08T16:12:35.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f40637d0 0x7ff5f4156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406a2b0 0x7ff5f416c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f416d050 con 0x7ff5f406b160 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f41596f0 con 0x7ff5f406a2b0 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5fa799700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4158ae0 con 0x7ff5f40637d0 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35642/0 (socket says 172.21.15.170:35642) 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 -- 172.21.15.170:0/1436581573 learned_addr learned my addr 172.21.15.170:0/1436581573 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f37fe700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406a2b0 0x7ff5f416c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f3fff700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f40637d0 0x7ff5f4156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 -- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f40637d0 msgr2=0x7ff5f4156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f40637d0 0x7ff5f4156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 -- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406a2b0 msgr2=0x7ff5f416c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406a2b0 0x7ff5f416c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 -- 172.21.15.170:0/1436581573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e4009420 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.755+0000 7ff5f8d36700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e90 tx=0x7ff5ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00cc20 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00f040 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f4158dc0 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec014480 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f4159380 con 0x7ff5f406b160 2023-10-08T16:12:35.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.756+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f404c3c0 con 0x7ff5f406b160 2023-10-08T16:12:35.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.757+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff5ec014830 con 0x7ff5f406b160 2023-10-08T16:12:35.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.757+0000 7ff5dbfff700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 0x7ff5e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:35.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.757+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff5ec0855c0 con 0x7ff5f406b160 2023-10-08T16:12:35.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.757+0000 7ff5f3fff700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 0x7ff5e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:35.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.758+0000 7ff5f3fff700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 0x7ff5e00697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4003080 tx=0x7ff5e4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:35.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.761+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5ec055e90 con 0x7ff5f406b160 2023-10-08T16:12:35.944 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.943+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff5f4159830 con 0x7ff5f406b160 2023-10-08T16:12:35.947 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.946+0000 7ff5dbfff700 1 -- 172.21.15.170:0/1436581573 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5ec055a20 con 0x7ff5f406b160 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 msgr2=0x7ff5e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 0x7ff5e00697d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4003080 tx=0x7ff5e4017000 comp rx=0 tx=0).stop 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 msgr2=0x7ff5f416cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009e90 tx=0x7ff5ec00b040 comp rx=0 tx=0).stop 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 shutdown_connections 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff5e0067320 0x7ff5e00697d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5f406b160 0x7ff5f416cb10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff5f40637d0 0x7ff5f4156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 --2- 172.21.15.170:0/1436581573 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff5f406a2b0 0x7ff5f416c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:35.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.948+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 >> 172.21.15.170:0/1436581573 conn(0x7ff5f406ded0 msgr2=0x7ff5f4153750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:35.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.949+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 shutdown_connections 2023-10-08T16:12:35.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:35.949+0000 7ff5fa799700 1 -- 172.21.15.170:0/1436581573 wait complete. 2023-10-08T16:12:35.962 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag' 2023-10-08T16:12:36.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.074+0000 7f87e3331700 1 Processor -- start 2023-10-08T16:12:36.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 -- start start 2023-10-08T16:12:36.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87dc068550 0x7f87dc068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f87dc15e010 0x7f87dc06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc06ec70 con 0x7f87dc06f640 2023-10-08T16:12:36.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0b3430 con 0x7f87dc068550 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.075+0000 7f87e3331700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc0b2e70 con 0x7f87dc15e010 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87dc068550 0x7f87dc068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f87dc068550 0x7f87dc068920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35644/0 (socket says 172.21.15.170:35644) 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 -- 172.21.15.170:0/514971722 learned_addr learned my addr 172.21.15.170:0/514971722 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e08cc700 1 --2- 172.21.15.170:0/514971722 >> v2:172.21.15.191:3300/0 conn(0x7f87dc15e010 0x7f87dc06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 -- 172.21.15.170:0/514971722 >> v2:172.21.15.191:3300/0 conn(0x7f87dc15e010 msgr2=0x7f87dc06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 --2- 172.21.15.170:0/514971722 >> v2:172.21.15.191:3300/0 conn(0x7f87dc15e010 0x7f87dc06f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 -- 172.21.15.170:0/514971722 >> v1:172.21.15.191:6789/0 conn(0x7f87dc06f640 legacy=0x7f87dc06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 -- 172.21.15.170:0/514971722 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0b3150 con 0x7f87dc068550 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87e10cd700 1 --2- 172.21.15.170:0/514971722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc068920 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f87dc06d310 tx=0x7f87d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851fb308b3c96e0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87d3fff700 1 -- 172.21.15.170:0/514971722 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b820 con 0x7f87dc068550 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87d3fff700 1 -- 172.21.15.170:0/514971722 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800b980 con 0x7f87dc068550 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.076+0000 7f87d3fff700 1 -- 172.21.15.170:0/514971722 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800bd20 con 0x7f87dc068550 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 -- 172.21.15.170:0/514971722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 msgr2=0x7f87dc068920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 --2- 172.21.15.170:0/514971722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc068920 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f87dc06d310 tx=0x7f87d8009250 comp rx=0 tx=0).stop 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 -- 172.21.15.170:0/514971722 shutdown_connections 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 --2- 172.21.15.170:0/514971722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc068920 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 --2- 172.21.15.170:0/514971722 >> v2:172.21.15.191:3300/0 conn(0x7f87dc15e010 0x7f87dc06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 -- 172.21.15.170:0/514971722 >> 172.21.15.170:0/514971722 conn(0x7f87dc0cd480 msgr2=0x7f87dc0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 -- 172.21.15.170:0/514971722 shutdown_connections 2023-10-08T16:12:36.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.077+0000 7f87e3331700 1 -- 172.21.15.170:0/514971722 wait complete. 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.078+0000 7f87e3331700 1 Processor -- start 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.078+0000 7f87e3331700 1 -- start start 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87dc06f640 0x7f87dc1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87dc15e010 0x7f87dc143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0aee30 con 0x7f87dc068550 2023-10-08T16:12:36.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc144430 con 0x7f87dc15e010 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0ae1b0 con 0x7f87dc06f640 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35650/0 (socket says 172.21.15.170:35650) 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 -- 172.21.15.170:0/2362815907 learned_addr learned my addr 172.21.15.170:0/2362815907 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 -- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87dc06f640 msgr2=0x7f87dc1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87dc06f640 0x7f87dc1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 -- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87dc15e010 msgr2=0x7f87dc143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87dc15e010 0x7f87dc143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 -- 172.21.15.170:0/2362815907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80094f0 con 0x7f87dc068550 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e10cd700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f87d80030c0 tx=0x7f87d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011e60 con 0x7f87dc068550 2023-10-08T16:12:36.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8020070 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d801b6a0 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0ae490 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.079+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0aea50 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.080+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc04c3c0 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.081+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f87d8022030 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.081+0000 7f87d17fa700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 0x7f87c00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.081+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f87d8086830 con 0x7f87dc068550 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.081+0000 7f87e08cc700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 0x7f87c00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.082+0000 7f87e08cc700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 0x7f87c00697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00a900 tx=0x7f87cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.086+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87d80538d0 con 0x7f87dc068550 2023-10-08T16:12:36.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.352+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]} v 0) v1 -- 0x7f87dc04c3c0 con 0x7f87dc068550 2023-10-08T16:12:36.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.618+0000 7f87d17fa700 1 -- 172.21.15.170:0/2362815907 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_fails_without_confirmation_flag"]}]=0 v402) v1 ==== 167+0+0 (secure 0 0 0) 0x7f87d8056ef0 con 0x7f87dc068550 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 msgr2=0x7f87c00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 0x7f87c00697d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00a900 tx=0x7f87cc008040 comp rx=0 tx=0).stop 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 msgr2=0x7f87dc15c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f87d80030c0 tx=0x7f87d8016000 comp rx=0 tx=0).stop 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 shutdown_connections 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f87dc15e010 0x7f87dc143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f87c0067320 0x7f87c00697d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87dc068550 0x7f87dc15c010 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 --2- 172.21.15.170:0/2362815907 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f87dc06f640 0x7f87dc1439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 >> 172.21.15.170:0/2362815907 conn(0x7f87dc0cd480 msgr2=0x7f87dc158870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:36.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 shutdown_connections 2023-10-08T16:12:36.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.620+0000 7f87e3331700 1 -- 172.21.15.170:0/2362815907 wait complete. 2023-10-08T16:12:36.634 INFO:tasks.cephfs_test_runner:That renaming volume fails without --yes-i-really-mean-it flag. ... ok 2023-10-08T16:12:36.634 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) That renaming a volume with more than one data pool does not change 2023-10-08T16:12:36.634 INFO:tasks.cephfs_test_runner:test_volume_rename_for_more_than_one_data_pool (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:12:36.634 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-08T16:12:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.748+0000 7f73bd591700 1 Processor -- start 2023-10-08T16:12:36.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.748+0000 7f73bd591700 1 -- start start 2023-10-08T16:12:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73bd591700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f73b8069560 0x7f73b80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73bd591700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b8069ee0 con 0x7f73b8159c60 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73bd591700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b815a160 con 0x7f73b8159890 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73bd591700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b80d13a0 con 0x7f73b8069560 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b77fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f73b8069560 0x7f73b80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73affff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f73b8159c60 0x7f73b815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46198/0 (socket says 172.21.15.170:46198) 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73affff700 1 -- 172.21.15.170:0/3445979207 learned_addr learned my addr 172.21.15.170:0/3445979207 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b77fe700 1 -- 172.21.15.170:0/3445979207 >> v1:172.21.15.191:6789/0 conn(0x7f73b8159890 legacy=0x7f73b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b67fc700 1 -- 172.21.15.170:0/3445979207 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747522355 0 0) 0x7f73b8069ee0 con 0x7f73b8159c60 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b77fe700 1 -- 172.21.15.170:0/3445979207 >> v1:172.21.15.170:6789/0 conn(0x7f73b8159c60 legacy=0x7f73b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b77fe700 1 -- 172.21.15.170:0/3445979207 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b815e140 con 0x7f73b8069560 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b77fe700 1 --2- 172.21.15.170:0/3445979207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8069560 0x7f73b80699a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e60 tx=0x7f73b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656833c5229e7635 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b67fc700 1 -- 172.21.15.170:0/3445979207 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b000e070 con 0x7f73b8069560 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b67fc700 1 -- 172.21.15.170:0/3445979207 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000bbf0 con 0x7f73b8069560 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.749+0000 7f73b67fc700 1 -- 172.21.15.170:0/3445979207 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0012430 con 0x7f73b8069560 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 -- 172.21.15.170:0/3445979207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8069560 msgr2=0x7f73b80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 --2- 172.21.15.170:0/3445979207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8069560 0x7f73b80699a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009e60 tx=0x7f73b0009780 comp rx=0 tx=0).stop 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 -- 172.21.15.170:0/3445979207 shutdown_connections 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 --2- 172.21.15.170:0/3445979207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8069560 0x7f73b80699a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 -- 172.21.15.170:0/3445979207 >> 172.21.15.170:0/3445979207 conn(0x7f73b8068e20 msgr2=0x7f73b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 -- 172.21.15.170:0/3445979207 shutdown_connections 2023-10-08T16:12:36.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.750+0000 7f73bd591700 1 -- 172.21.15.170:0/3445979207 wait complete. 2023-10-08T16:12:36.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.751+0000 7f73bd591700 1 Processor -- start 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.751+0000 7f73bd591700 1 -- start start 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73b8069560 0x7f73b814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73b8159890 0x7f73b8153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b815c4e0 con 0x7f73b8159c60 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80ce170 con 0x7f73b8159890 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73bd591700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b80ce450 con 0x7f73b8069560 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35668/0 (socket says 172.21.15.170:35668) 2023-10-08T16:12:36.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 -- 172.21.15.170:0/3798666270 learned_addr learned my addr 172.21.15.170:0/3798666270 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b6ffd700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73b8069560 0x7f73b814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73affff700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73b8159890 0x7f73b8153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 -- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73b8069560 msgr2=0x7f73b814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73b8069560 0x7f73b814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 -- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73b8159890 msgr2=0x7f73b8153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73b8159890 0x7f73b8153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 -- 172.21.15.170:0/3798666270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b0009a20 con 0x7f73b8159c60 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.752+0000 7f73b77fe700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f73b806a070 tx=0x7f73b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.753+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b001e640 con 0x7f73b8159c60 2023-10-08T16:12:36.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.753+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b001e7a0 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.753+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b80ce730 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.753+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b001eb40 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.753+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b80cd5c0 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.754+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b815c650 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.754+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f73b0024070 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.754+0000 7f73aeffd700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 0x7f73980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.754+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f73b0087320 con 0x7f73b8159c60 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.754+0000 7f73b6ffd700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 0x7f73980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:36.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.755+0000 7f73b6ffd700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 0x7f73980697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003870 tx=0x7f73a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:36.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:36.758+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73b00543b0 con 0x7f73b8159c60 2023-10-08T16:12:37.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.015+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f73b804c3c0 con 0x7f73b8159c60 2023-10-08T16:12:37.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.619+0000 7f73aeffd700 1 -- 172.21.15.170:0/3798666270 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v403) v1 ==== 166+0+0 (secure 0 0 0) 0x7f73b00579d0 con 0x7f73b8159c60 2023-10-08T16:12:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.621+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 msgr2=0x7f73980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.621+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 0x7f73980697d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f73a8003870 tx=0x7f73a8009040 comp rx=0 tx=0).stop 2023-10-08T16:12:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.621+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 msgr2=0x7f73b8153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.621+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f73b806a070 tx=0x7f73b0018000 comp rx=0 tx=0).stop 2023-10-08T16:12:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 shutdown_connections 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7398067320 0x7f73980697d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f73b8159c60 0x7f73b8153d70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f73b8069560 0x7f73b814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 --2- 172.21.15.170:0/3798666270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f73b8159890 0x7f73b8153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 >> 172.21.15.170:0/3798666270 conn(0x7f73b8068e20 msgr2=0x7f73b8157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 shutdown_connections 2023-10-08T16:12:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.622+0000 7f73bd591700 1 -- 172.21.15.170:0/3798666270 wait complete. 2023-10-08T16:12:37.636 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:12:37.755 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.754+0000 7f7db8b47700 1 Processor -- start 2023-10-08T16:12:37.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.754+0000 7f7db8b47700 1 -- start start 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db8b47700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 0x7f7db415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db8b47700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7db415db10 0x7f7db4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db8b47700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db415a160 con 0x7f7db4159890 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db8b47700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db40d13a0 con 0x7f7db415db10 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db8b47700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db415e140 con 0x7f7db4159c60 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7db415db10 0x7f7db4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7db415db10 0x7f7db4069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35678/0 (socket says 172.21.15.170:35678) 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 -- 172.21.15.170:0/1638923085 learned_addr learned my addr 172.21.15.170:0/1638923085 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7da9d9b700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 0x7f7db415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 -- 172.21.15.170:0/1638923085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 msgr2=0x7f7db415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 0x7f7db415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 -- 172.21.15.170:0/1638923085 >> v1:172.21.15.191:6789/0 conn(0x7f7db4159890 legacy=0x7f7db415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7db2d9d700 1 -- 172.21.15.170:0/1638923085 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db415a3d0 con 0x7f7db415db10 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.755+0000 7f7da9d9b700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 0x7f7db415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db2d9d700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db415db10 0x7f7db4069560 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009f20 tx=0x7f7dac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15f10ebeb2771f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db1d9b700 1 -- 172.21.15.170:0/1638923085 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00e070 con 0x7f7db415db10 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db1d9b700 1 -- 172.21.15.170:0/1638923085 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dac00ba40 con 0x7f7db415db10 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db1d9b700 1 -- 172.21.15.170:0/1638923085 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0092e0 con 0x7f7db415db10 2023-10-08T16:12:37.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 -- 172.21.15.170:0/1638923085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db415db10 msgr2=0x7f7db4069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db415db10 0x7f7db4069560 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009f20 tx=0x7f7dac009840 comp rx=0 tx=0).stop 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 -- 172.21.15.170:0/1638923085 shutdown_connections 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db415db10 0x7f7db4069560 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 --2- 172.21.15.170:0/1638923085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db4159c60 0x7f7db415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 -- 172.21.15.170:0/1638923085 >> 172.21.15.170:0/1638923085 conn(0x7f7db4068e20 msgr2=0x7f7db40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 -- 172.21.15.170:0/1638923085 shutdown_connections 2023-10-08T16:12:37.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.756+0000 7f7db8b47700 1 -- 172.21.15.170:0/1638923085 wait complete. 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 Processor -- start 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 -- start start 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db4159890 0x7f7db40b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db415db10 0x7f7db40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41519b0 con 0x7f7db4159c60 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db4146bd0 con 0x7f7db4159890 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db8b47700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db41527d0 con 0x7f7db415db10 2023-10-08T16:12:37.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7da9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7da9d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35684/0 (socket says 172.21.15.170:35684) 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7da9d9b700 1 -- 172.21.15.170:0/278907389 learned_addr learned my addr 172.21.15.170:0/278907389 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db2d9d700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db415db10 0x7f7db40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.758+0000 7f7db259c700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db4159890 0x7f7db40b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 -- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db415db10 msgr2=0x7f7db40cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db415db10 0x7f7db40cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 -- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db4159890 msgr2=0x7f7db40b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db4159890 0x7f7db40b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 -- 172.21.15.170:0/278907389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dac009ae0 con 0x7f7db4159c60 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7da9d9b700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c880 tx=0x7f7d9c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:37.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011600 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c011760 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db4151bc0 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c011ae0 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.759+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db4152180 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.760+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db404c3c0 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.760+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7d9c0193b0 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.761+0000 7f7daaffd700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 0x7f7d940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.761+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7d9c084a70 con 0x7f7db4159c60 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.761+0000 7f7db259c700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 0x7f7d940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:37.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.761+0000 7f7db259c700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 0x7f7d940697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7da4003a30 tx=0x7f7da4007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:37.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.765+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7d9c0518f0 con 0x7f7db4159c60 2023-10-08T16:12:37.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.948+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7db4156e90 con 0x7f7db4159c60 2023-10-08T16:12:37.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.952+0000 7f7daaffd700 1 -- 172.21.15.170:0/278907389 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7d9c0150c0 con 0x7f7db4159c60 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 msgr2=0x7f7d940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 0x7f7d940697d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7da4003a30 tx=0x7f7da4007ea0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 msgr2=0x7f7db4151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c880 tx=0x7f7d9c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 shutdown_connections 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7db415db10 0x7f7db40cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7db4159890 0x7f7db40b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7d94067320 0x7f7d940697d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 --2- 172.21.15.170:0/278907389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7db4159c60 0x7f7db4151470 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 >> 172.21.15.170:0/278907389 conn(0x7f7db4068e20 msgr2=0x7f7db414d760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 shutdown_connections 2023-10-08T16:12:37.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:37.954+0000 7f7db8b47700 1 -- 172.21.15.170:0/278907389 wait complete. 2023-10-08T16:12:37.968 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:12:37.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:38.006 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:38.050 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:12:38.050 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:12:38.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:38.164 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.163+0000 7f082310e700 1 Processor -- start 2023-10-08T16:12:38.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.163+0000 7f082310e700 1 -- start start 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f082310e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f081c069d60 0x7f081c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f082310e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f081c0c81f0 0x7f081c0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f082310e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c06bdb0 con 0x7f081c0aca80 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f082310e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0cb370 con 0x7f081c0c81f0 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f082310e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c159f60 con 0x7f081c069d60 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f081c0c81f0 0x7f081c0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f0820eaa700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f081c0aca80 0x7f081c06aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46212/0 (socket says 172.21.15.170:46212) 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f0820eaa700 1 -- 172.21.15.170:0/2292430702 learned_addr learned my addr 172.21.15.170:0/2292430702 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:38.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f081bfff700 1 --2- 172.21.15.170:0/2292430702 >> v2:172.21.15.191:3300/0 conn(0x7f081c069d60 0x7f081c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 -- 172.21.15.170:0/2292430702 >> v2:172.21.15.191:3300/0 conn(0x7f081c069d60 msgr2=0x7f081c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 --2- 172.21.15.170:0/2292430702 >> v2:172.21.15.191:3300/0 conn(0x7f081c069d60 0x7f081c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 -- 172.21.15.170:0/2292430702 >> v1:172.21.15.170:6789/0 conn(0x7f081c0aca80 legacy=0x7f081c06aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f081b7fe700 1 -- 172.21.15.170:0/2292430702 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694678681 0 0) 0x7f081c06bdb0 con 0x7f081c0aca80 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 -- 172.21.15.170:0/2292430702 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c06b2b0 con 0x7f081c0c81f0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.164+0000 7f08216ab700 1 --2- 172.21.15.170:0/2292430702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0c81f0 0x7f081c0cb800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f081c06e540 tx=0x7f0814009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d693c5f3a5063f7a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f081b7fe700 1 -- 172.21.15.170:0/2292430702 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400e070 con 0x7f081c0c81f0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f081b7fe700 1 -- 172.21.15.170:0/2292430702 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400bc30 con 0x7f081c0c81f0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f081b7fe700 1 -- 172.21.15.170:0/2292430702 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814012430 con 0x7f081c0c81f0 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 -- 172.21.15.170:0/2292430702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0c81f0 msgr2=0x7f081c0cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 --2- 172.21.15.170:0/2292430702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0c81f0 0x7f081c0cb800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f081c06e540 tx=0x7f0814009780 comp rx=0 tx=0).stop 2023-10-08T16:12:38.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 -- 172.21.15.170:0/2292430702 shutdown_connections 2023-10-08T16:12:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 --2- 172.21.15.170:0/2292430702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0c81f0 0x7f081c0cb800 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 --2- 172.21.15.170:0/2292430702 >> v2:172.21.15.191:3300/0 conn(0x7f081c069d60 0x7f081c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 -- 172.21.15.170:0/2292430702 >> 172.21.15.170:0/2292430702 conn(0x7f081c0cd670 msgr2=0x7f081c0cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 -- 172.21.15.170:0/2292430702 shutdown_connections 2023-10-08T16:12:38.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.165+0000 7f082310e700 1 -- 172.21.15.170:0/2292430702 wait complete. 2023-10-08T16:12:38.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.166+0000 7f082310e700 1 Processor -- start 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 -- start start 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f081c069d60 0x7f081c1593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f081c0c81f0 0x7f081c0b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c15fef0 con 0x7f081c0aca80 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c16d270 con 0x7f081c0c81f0 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f082310e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c16cc60 con 0x7f081c069d60 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f081bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f081bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35706/0 (socket says 172.21.15.170:35706) 2023-10-08T16:12:38.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f081bfff700 1 -- 172.21.15.170:0/1944028786 learned_addr learned my addr 172.21.15.170:0/1944028786 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f0820eaa700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f081c069d60 0x7f081c1593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.167+0000 7f08216ab700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f081c0c81f0 0x7f081c0b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 -- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f081c069d60 msgr2=0x7f081c1593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f081c069d60 0x7f081c1593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 -- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f081c0c81f0 msgr2=0x7f081c0b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f081c0c81f0 0x7f081c0b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 -- 172.21.15.170:0/1944028786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814009a20 con 0x7f081c0aca80 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f081bfff700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f080800d890 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08080125f0 con 0x7f081c0aca80 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0808012770 con 0x7f081c0aca80 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c16a170 con 0x7f081c0aca80 2023-10-08T16:12:38.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808012b30 con 0x7f081c0aca80 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.168+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c16cf00 con 0x7f081c0aca80 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.169+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c16d3e0 con 0x7f081c0aca80 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.169+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f080801e460 con 0x7f081c0aca80 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.170+0000 7f0818ff9700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 0x7f08000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.170+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0808084f10 con 0x7f081c0aca80 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.170+0000 7f0820eaa700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 0x7f08000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.170+0000 7f0820eaa700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 0x7f08000697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f081c06c120 tx=0x7f0810009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.174+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0808055930 con 0x7f081c0aca80 2023-10-08T16:12:38.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.398+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f081c159980 con 0x7f081c0aca80 2023-10-08T16:12:38.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.398+0000 7f0818ff9700 1 -- 172.21.15.170:0/1944028786 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+3543 (secure 0 0 0) 0x7f08080554c0 con 0x7f081c0aca80 2023-10-08T16:12:38.400 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:38.400 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6485,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/4129171933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":4129171933},{"type":"v1","addr":"172.21.15.191:6835","nonce":4129171933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7159,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/4000595139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":4000595139},{"type":"v1","addr":"172.21.15.191:6834","nonce":4000595139}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":7854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1645183383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1645183383},{"type":"v1","addr":"172.21.15.170:6836","nonce":1645183383}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":95},{"gid":8139,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/2809489427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":2809489427},{"type":"v1","addr":"172.21.15.170:6837","nonce":2809489427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":103}],"filesystems":[]} 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 msgr2=0x7f08000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 0x7f08000697d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f081c06c120 tx=0x7f0810009040 comp rx=0 tx=0).stop 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 msgr2=0x7f081c169660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f080800d890 tx=0x7f0808008040 comp rx=0 tx=0).stop 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 shutdown_connections 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f081c0c81f0 0x7f081c0b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f081c069d60 0x7f081c1593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0800067320 0x7f08000697d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 --2- 172.21.15.170:0/1944028786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f081c0aca80 0x7f081c169660 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 >> 172.21.15.170:0/1944028786 conn(0x7f081c0cd670 msgr2=0x7f081c0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:38.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 shutdown_connections 2023-10-08T16:12:38.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.400+0000 7f082310e700 1 -- 172.21.15.170:0/1944028786 wait complete. 2023-10-08T16:12:38.403 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 103 2023-10-08T16:12:38.414 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:12:38.414 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:12:38.414 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:12:38.414 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:38.415 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:12:38.415 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:12:38.415 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:12:38.415 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:38.415 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:12:38.415 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:12:38.416 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:12:38.416 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:38.416 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:12:38.416 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:12:38.416 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:12:38.416 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:12:38.422 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:12:38.423 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:12:38.423 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:12:38.424 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:12:38.427 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:12:38.427 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:12:38.428 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:12:38.428 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:12:38.465 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:12:38.465 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:38.466 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:12:38.466 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:38.469 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:12:38.469 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:38.470 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:12:38.470 DEBUG:teuthology.parallel:result is None 2023-10-08T16:12:38.470 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:12:38.539 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:12:38.539 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:12:38.548 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:12:38.548 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:12:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.590+0000 7f3f6e192700 1 Processor -- start 2023-10-08T16:12:38.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.590+0000 7f3f6e192700 1 -- start start 2023-10-08T16:12:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f6e192700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3f68159890 0x7f3f6815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f6e192700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3f68159c60 0x7f3f6815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f6e192700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6815a160 con 0x7f3f6815db10 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f6e192700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f6815df50 con 0x7f3f68159890 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f6e192700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f680d13a0 con 0x7f3f68159c60 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3f68159890 0x7f3f6815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f677fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3f68159890 0x7f3f6815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35734/0 (socket says 172.21.15.170:35734) 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f677fe700 1 -- 172.21.15.170:0/477037437 learned_addr learned my addr 172.21.15.170:0/477037437 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f66ffd700 1 --2- 172.21.15.170:0/477037437 >> v2:172.21.15.191:3300/0 conn(0x7f3f68159c60 0x7f3f6815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f677fe700 1 -- 172.21.15.170:0/477037437 >> v2:172.21.15.191:3300/0 conn(0x7f3f68159c60 msgr2=0x7f3f6815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:38.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.591+0000 7f3f677fe700 1 --2- 172.21.15.170:0/477037437 >> v2:172.21.15.191:3300/0 conn(0x7f3f68159c60 0x7f3f6815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f677fe700 1 -- 172.21.15.170:0/477037437 >> v1:172.21.15.191:6789/0 conn(0x7f3f6815db10 legacy=0x7f3f68069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f677fe700 1 -- 172.21.15.170:0/477037437 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6815e140 con 0x7f3f68159890 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f677fe700 1 --2- 172.21.15.170:0/477037437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159890 0x7f3f6815abd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3f680b67f0 tx=0x7f3f58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e7140ce2102ed0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f667fc700 1 -- 172.21.15.170:0/477037437 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800e070 con 0x7f3f68159890 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f667fc700 1 -- 172.21.15.170:0/477037437 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5800ba20 con 0x7f3f68159890 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f667fc700 1 -- 172.21.15.170:0/477037437 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f580092e0 con 0x7f3f68159890 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 -- 172.21.15.170:0/477037437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159890 msgr2=0x7f3f6815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 --2- 172.21.15.170:0/477037437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159890 0x7f3f6815abd0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3f680b67f0 tx=0x7f3f58009840 comp rx=0 tx=0).stop 2023-10-08T16:12:38.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 -- 172.21.15.170:0/477037437 shutdown_connections 2023-10-08T16:12:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 --2- 172.21.15.170:0/477037437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159890 0x7f3f6815abd0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 --2- 172.21.15.170:0/477037437 >> v2:172.21.15.191:3300/0 conn(0x7f3f68159c60 0x7f3f6815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 -- 172.21.15.170:0/477037437 >> 172.21.15.170:0/477037437 conn(0x7f3f68068e20 msgr2=0x7f3f680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 -- 172.21.15.170:0/477037437 shutdown_connections 2023-10-08T16:12:38.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.592+0000 7f3f6e192700 1 -- 172.21.15.170:0/477037437 wait complete. 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 Processor -- start 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 -- start start 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 0x7f3f680ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f6815db10 0x7f3f681587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680cfeb0 con 0x7f3f68159c60 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680c7bf0 con 0x7f3f68159890 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f6e192700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f680cf3b0 con 0x7f3f6815db10 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.594+0000 7f3f66ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f677fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 0x7f3f680ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35740/0 (socket says 172.21.15.170:35740) 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 -- 172.21.15.170:0/1610281886 learned_addr learned my addr 172.21.15.170:0/1610281886 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f67fff700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f6815db10 0x7f3f681587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 -- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f6815db10 msgr2=0x7f3f681587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f6815db10 0x7f3f681587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 -- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 msgr2=0x7f3f680ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 0x7f3f680ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 -- 172.21.15.170:0/1610281886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f58009ae0 con 0x7f3f68159c60 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f677fe700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 0x7f3f680ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f66ffd700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000ec80 tx=0x7f3f500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000c850 con 0x7f3f68159c60 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5000c9d0 con 0x7f3f68159c60 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f680cf610 con 0x7f3f68159c60 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5000cd90 con 0x7f3f68159c60 2023-10-08T16:12:38.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.595+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f680cfc50 con 0x7f3f68159c60 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.596+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f680b4550 con 0x7f3f68159c60 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.596+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3f50013680 con 0x7f3f68159c60 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.597+0000 7f3f5f7fe700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 0x7f3f48069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.597+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3f50084c40 con 0x7f3f68159c60 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.597+0000 7f3f677fe700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 0x7f3f48069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:38.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.597+0000 7f3f677fe700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 0x7f3f48069850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009880 tx=0x7f3f58018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:38.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.601+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f680b4550 con 0x7f3f68159c60 2023-10-08T16:12:38.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:38.805+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3f680b4550 con 0x7f3f68159c60 2023-10-08T16:12:39.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.639+0000 7f3f5f7fe700 1 -- 172.21.15.170:0/1610281886 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v162) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3f680b4550 con 0x7f3f68159c60 2023-10-08T16:12:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 msgr2=0x7f3f48069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 0x7f3f48069850 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009880 tx=0x7f3f58018000 comp rx=0 tx=0).stop 2023-10-08T16:12:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 msgr2=0x7f3f680cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000ec80 tx=0x7f3f500091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 shutdown_connections 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f6815db10 0x7f3f681587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3f68159890 0x7f3f680ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3f480673a0 0x7f3f48069850 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 --2- 172.21.15.170:0/1610281886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f68159c60 0x7f3f680cee70 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 >> 172.21.15.170:0/1610281886 conn(0x7f3f68068e20 msgr2=0x7f3f68153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 shutdown_connections 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.641+0000 7f3f6e192700 1 -- 172.21.15.170:0/1610281886 wait complete. 2023-10-08T16:12:39.738 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:12:39.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:12:39.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.854+0000 7f25cb409700 1 Processor -- start 2023-10-08T16:12:39.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.854+0000 7f25cb409700 1 -- start start 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25cb409700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 0x7f25c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25cb409700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25c4069560 0x7f25c40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25cb409700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c415a160 con 0x7f25c4159c60 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25cb409700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c4069f20 con 0x7f25c4069560 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25cb409700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c40d13a0 con 0x7f25c4159890 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c99a6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f25c4069560 0x7f25c40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c89a4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f25c4159c60 0x7f25c415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46226/0 (socket says 172.21.15.170:46226) 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c89a4700 1 -- 172.21.15.170:0/806741229 learned_addr learned my addr 172.21.15.170:0/806741229 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c91a5700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 0x7f25c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c99a6700 1 -- 172.21.15.170:0/806741229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 msgr2=0x7f25c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c99a6700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 0x7f25c415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c99a6700 1 -- 172.21.15.170:0/806741229 >> v1:172.21.15.170:6789/0 conn(0x7f25c4159c60 legacy=0x7f25c415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c99a6700 1 -- 172.21.15.170:0/806741229 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c415e140 con 0x7f25c4069560 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25bbfff700 1 -- 172.21.15.170:0/806741229 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669287271 0 0) 0x7f25c415a160 con 0x7f25c4159c60 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.855+0000 7f25c91a5700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 0x7f25c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25c99a6700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4069560 0x7f25c40699e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0098e0 tx=0x7f25bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe20046357840611 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25bbfff700 1 -- 172.21.15.170:0/806741229 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc009e80 con 0x7f25c4069560 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25bbfff700 1 -- 172.21.15.170:0/806741229 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc00b860 con 0x7f25c4069560 2023-10-08T16:12:39.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25bbfff700 1 -- 172.21.15.170:0/806741229 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc00bbe0 con 0x7f25c4069560 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 -- 172.21.15.170:0/806741229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4069560 msgr2=0x7f25c40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4069560 0x7f25c40699e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f25bc0098e0 tx=0x7f25bc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 -- 172.21.15.170:0/806741229 shutdown_connections 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4159890 0x7f25c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 --2- 172.21.15.170:0/806741229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4069560 0x7f25c40699e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 -- 172.21.15.170:0/806741229 >> 172.21.15.170:0/806741229 conn(0x7f25c4068e20 msgr2=0x7f25c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 -- 172.21.15.170:0/806741229 shutdown_connections 2023-10-08T16:12:39.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.856+0000 7f25cb409700 1 -- 172.21.15.170:0/806741229 wait complete. 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.857+0000 7f25cb409700 1 Processor -- start 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.857+0000 7f25cb409700 1 -- start start 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4069560 0x7f25c4148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c4159890 0x7f25c4169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415ce20 con 0x7f25c4159c60 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415c210 con 0x7f25c4159890 2023-10-08T16:12:39.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25cb409700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c415c4f0 con 0x7f25c4069560 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25c99a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25c99a6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35760/0 (socket says 172.21.15.170:35760) 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.858+0000 7f25c99a6700 1 -- 172.21.15.170:0/3244754950 learned_addr learned my addr 172.21.15.170:0/3244754950 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c91a5700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4069560 0x7f25c4148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c89a4700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c4159890 0x7f25c4169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 -- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4069560 msgr2=0x7f25c4148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4069560 0x7f25c4148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 -- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c4159890 msgr2=0x7f25c4169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c4159890 0x7f25c4169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 -- 172.21.15.170:0/3244754950 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc009420 con 0x7f25c4159c60 2023-10-08T16:12:39.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25c99a6700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3c0 tx=0x7f25bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011630 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc0117b0 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c415c7d0 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011b70 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.859+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c41486a0 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.860+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c415cf90 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.860+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f25bc0253f0 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.861+0000 7f25b97fa700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 0x7f25a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.861+0000 7f25c91a5700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 0x7f25a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.861+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f25bc086560 con 0x7f25c4159c60 2023-10-08T16:12:39.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.861+0000 7f25c91a5700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 0x7f25a80697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e650 tx=0x7f25c0007710 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:39.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:39.865+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25bc053500 con 0x7f25c4159c60 2023-10-08T16:12:40.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.122+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f25c404c3c0 con 0x7f25c4159c60 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.122+0000 7f25b97fa700 1 -- 172.21.15.170:0/3244754950 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f25bc056b20 con 0x7f25c4159c60 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:12:40.124 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.125 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.126 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.127 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.129 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.130 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.131 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.132 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.133 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:12:40.134 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:12:40.135 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.136 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.137 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.138 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:12:40.139 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:12:40.140 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:12:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:40.141 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:12:40.141 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:12:40.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.124+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 msgr2=0x7f25a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.124+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 0x7f25a80697d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e650 tx=0x7f25c0007710 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.124+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 msgr2=0x7f25c40b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.124+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b3c0 tx=0x7f25bc017000 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 shutdown_connections 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f25a8067320 0x7f25a80697d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f25c4159c60 0x7f25c40b5d30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f25c4069560 0x7f25c4148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 --2- 172.21.15.170:0/3244754950 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f25c4159890 0x7f25c4169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 >> 172.21.15.170:0/3244754950 conn(0x7f25c4068e20 msgr2=0x7f25c4157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 shutdown_connections 2023-10-08T16:12:40.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.125+0000 7f25cb409700 1 -- 172.21.15.170:0/3244754950 wait complete. 2023-10-08T16:12:40.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:40.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.257+0000 7f087b976700 1 Processor -- start 2023-10-08T16:12:40.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.257+0000 7f087b976700 1 -- start start 2023-10-08T16:12:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0874159890 0x7f087415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0874159c60 0x7f087415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 0x7f0874069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087415a160 con 0x7f0874159890 2023-10-08T16:12:40.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08740d13a0 con 0x7f087415db10 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f087b976700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087415e140 con 0x7f0874159c60 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f0879712700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0874159890 0x7f087415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f0879712700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0874159890 0x7f087415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35766/0 (socket says 172.21.15.170:35766) 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f0879712700 1 -- 172.21.15.170:0/549690798 learned_addr learned my addr 172.21.15.170:0/549690798 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f0878f11700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0874159c60 0x7f087415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.258+0000 7f0879f13700 1 --2- 172.21.15.170:0/549690798 >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 0x7f0874069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 -- 172.21.15.170:0/549690798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0874159c60 msgr2=0x7f087415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0874159c60 0x7f087415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 -- 172.21.15.170:0/549690798 >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 msgr2=0x7f0874069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 --2- 172.21.15.170:0/549690798 >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 0x7f0874069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 -- 172.21.15.170:0/549690798 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087415a3d0 con 0x7f0874159890 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879f13700 1 --2- 172.21.15.170:0/549690798 >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 0x7f0874069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f0879712700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159890 0x7f087415abd0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0874089b50 tx=0x7f0870009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7a3c6e28441d6f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f086bfff700 1 -- 172.21.15.170:0/549690798 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870009e70 con 0x7f0874159890 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f086bfff700 1 -- 172.21.15.170:0/549690798 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000b860 con 0x7f0874159890 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f086bfff700 1 -- 172.21.15.170:0/549690798 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000bc00 con 0x7f0874159890 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 -- 172.21.15.170:0/549690798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159890 msgr2=0x7f087415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159890 0x7f087415abd0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f0874089b50 tx=0x7f0870009250 comp rx=0 tx=0).stop 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 -- 172.21.15.170:0/549690798 shutdown_connections 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 --2- 172.21.15.170:0/549690798 >> v2:172.21.15.191:3300/0 conn(0x7f087415db10 0x7f0874069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159890 0x7f087415abd0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 --2- 172.21.15.170:0/549690798 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0874159c60 0x7f087415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 -- 172.21.15.170:0/549690798 >> 172.21.15.170:0/549690798 conn(0x7f0874068e20 msgr2=0x7f08740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.259+0000 7f087b976700 1 -- 172.21.15.170:0/549690798 shutdown_connections 2023-10-08T16:12:40.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.260+0000 7f087b976700 1 -- 172.21.15.170:0/549690798 wait complete. 2023-10-08T16:12:40.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 Processor -- start 2023-10-08T16:12:40.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 -- start start 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0874159890 0x7f0874147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 0x7f0874169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b5570 con 0x7f0874159c60 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0874152730 con 0x7f0874159890 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f087b976700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08740b4960 con 0x7f087415db10 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.261+0000 7f0878f11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35774/0 (socket says 172.21.15.170:35774) 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 -- 172.21.15.170:0/2617302360 learned_addr learned my addr 172.21.15.170:0/2617302360 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0879712700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0874159890 0x7f0874147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.267 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0879f13700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 0x7f0874169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 -- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 msgr2=0x7f0874169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 0x7f0874169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 -- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0874159890 msgr2=0x7f0874147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0874159890 0x7f0874147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 -- 172.21.15.170:0/2617302360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700094f0 con 0x7f0874159c60 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0879f13700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 0x7f0874169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f0878f11700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f086400ec50 tx=0x7f08640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086400c850 con 0x7f0874159c60 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086400c9b0 con 0x7f0874159c60 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08740b4c40 con 0x7f0874159c60 2023-10-08T16:12:40.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086400cd70 con 0x7f0874159c60 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.262+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08740b5200 con 0x7f0874159c60 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.263+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08740b5720 con 0x7f0874159c60 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.263+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0864013680 con 0x7f0874159c60 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.264+0000 7f08697fa700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 0x7f08580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.264+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0864084be0 con 0x7f0874159c60 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.264+0000 7f0879712700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 0x7f08580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.264+0000 7f0879712700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 0x7f08580697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0874089b50 tx=0x7f0870017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.268+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08740b5720 con 0x7f0874159c60 2023-10-08T16:12:40.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.453+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08740cbfc0 con 0x7f0874159c60 2023-10-08T16:12:40.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.454+0000 7f08697fa700 1 -- 172.21.15.170:0/2617302360 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f0864051bc0 con 0x7f0874159c60 2023-10-08T16:12:40.455 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:40.456 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":162,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:39.637809+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 msgr2=0x7f08580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 0x7f08580697d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0874089b50 tx=0x7f0870017000 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 msgr2=0x7f08740ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f086400ec50 tx=0x7f08640091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 shutdown_connections 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f087415db10 0x7f0874169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0874159890 0x7f0874147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0858067320 0x7f08580697d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 --2- 172.21.15.170:0/2617302360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0874159c60 0x7f08740ca1f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 >> 172.21.15.170:0/2617302360 conn(0x7f0874068e20 msgr2=0x7f08740b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 shutdown_connections 2023-10-08T16:12:40.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.456+0000 7f087b976700 1 -- 172.21.15.170:0/2617302360 wait complete. 2023-10-08T16:12:40.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:12:40.470 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:12:40.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.586+0000 7fc17136e700 1 Processor -- start 2023-10-08T16:12:40.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.586+0000 7fc17136e700 1 -- start start 2023-10-08T16:12:40.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc17136e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c159890 0x7fc16c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc17136e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc16c069560 0x7fc16c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc17136e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c15a160 con 0x7fc16c159c60 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc17136e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c069f20 con 0x7fc16c069560 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc17136e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0d13a0 con 0x7fc16c159890 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc16b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc16c069560 0x7fc16c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc1627fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc16c159c60 0x7fc16c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46232/0 (socket says 172.21.15.170:46232) 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.587+0000 7fc1627fc700 1 -- 172.21.15.170:0/3814351035 learned_addr learned my addr 172.21.15.170:0/3814351035 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16affd700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c159890 0x7fc16c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3814351035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c159890 msgr2=0x7fc16c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16b7fe700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c159890 0x7fc16c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3814351035 >> v1:172.21.15.170:6789/0 conn(0x7fc16c159c60 legacy=0x7fc16c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3814351035 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c15e140 con 0x7fc16c069560 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16a7fc700 1 -- 172.21.15.170:0/3814351035 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902367163 0 0) 0x7fc16c15a160 con 0x7fc16c159c60 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16b7fe700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c069560 0x7fc16c0699e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fc164009e60 tx=0x7fc164009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1789264afc3dcc79 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16a7fc700 1 -- 172.21.15.170:0/3814351035 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400e070 con 0x7fc16c069560 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16a7fc700 1 -- 172.21.15.170:0/3814351035 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16400bc30 con 0x7fc16c069560 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc16a7fc700 1 -- 172.21.15.170:0/3814351035 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164012430 con 0x7fc16c069560 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.588+0000 7fc17136e700 1 -- 172.21.15.170:0/3814351035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c069560 msgr2=0x7fc16c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c069560 0x7fc16c0699e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fc164009e60 tx=0x7fc164009780 comp rx=0 tx=0).stop 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 -- 172.21.15.170:0/3814351035 shutdown_connections 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c159890 0x7fc16c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 --2- 172.21.15.170:0/3814351035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c069560 0x7fc16c0699e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 -- 172.21.15.170:0/3814351035 >> 172.21.15.170:0/3814351035 conn(0x7fc16c068e20 msgr2=0x7fc16c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 -- 172.21.15.170:0/3814351035 shutdown_connections 2023-10-08T16:12:40.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.589+0000 7fc17136e700 1 -- 172.21.15.170:0/3814351035 wait complete. 2023-10-08T16:12:40.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.590+0000 7fc17136e700 1 Processor -- start 2023-10-08T16:12:40.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.590+0000 7fc17136e700 1 -- start start 2023-10-08T16:12:40.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 0x7fc16c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c159890 0x7fc16c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0cf550 con 0x7fc16c159c60 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0cf830 con 0x7fc16c159890 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0cfb10 con 0x7fc16c069560 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35790/0 (socket says 172.21.15.170:35790) 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3123461108 learned_addr learned my addr 172.21.15.170:0/3123461108 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16affd700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 0x7fc16c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 msgr2=0x7fc16c140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 0x7fc16c140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c159890 msgr2=0x7fc16c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c159890 0x7fc16c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 -- 172.21.15.170:0/3123461108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009a20 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16affd700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 0x7fc16c140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc16b7fe700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc164013000 tx=0x7fc164019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164012ea0 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16401e630 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c0cfdf0 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16401e9d0 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.591+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c140e80 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.592+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c04c3c0 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.593+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc164024070 con 0x7fc16c159c60 2023-10-08T16:12:40.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.593+0000 7fc1637fe700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 0x7fc14c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.593+0000 7fc16affd700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 0x7fc14c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.593+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fc164087220 con 0x7fc16c159c60 2023-10-08T16:12:40.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.593+0000 7fc16affd700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 0x7fc14c0697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.597+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc164054200 con 0x7fc16c159c60 2023-10-08T16:12:40.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.780+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc16c06a5b0 con 0x7fc16c159c60 2023-10-08T16:12:40.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.780+0000 7fc1637fe700 1 -- 172.21.15.170:0/3123461108 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc164057820 con 0x7fc16c159c60 2023-10-08T16:12:40.782 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:12:40.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 msgr2=0x7fc14c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 0x7fc14c0697d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc15c006fd0 tx=0x7fc15c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 msgr2=0x7fc16c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc164013000 tx=0x7fc164019000 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 shutdown_connections 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc14c067320 0x7fc14c0697d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc16c159c60 0x7fc16c0b0540 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc16c069560 0x7fc16c140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 --2- 172.21.15.170:0/3123461108 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc16c159890 0x7fc16c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 >> 172.21.15.170:0/3123461108 conn(0x7fc16c068e20 msgr2=0x7fc16c152b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 shutdown_connections 2023-10-08T16:12:40.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.783+0000 7fc17136e700 1 -- 172.21.15.170:0/3123461108 wait complete. 2023-10-08T16:12:40.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:12:40.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.909+0000 7fa4a33a4700 1 Processor -- start 2023-10-08T16:12:40.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.909+0000 7fa4a33a4700 1 -- start start 2023-10-08T16:12:40.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa49c159420 0x7fa49c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 0x7fa49c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c15dbd0 con 0x7fa49c159420 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0b11d0 con 0x7fa49c15d230 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a33a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0b14b0 con 0x7fa49c1597f0 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa49c159420 0x7fa49c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa49c159420 0x7fa49c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35806/0 (socket says 172.21.15.170:35806) 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3062227692 learned_addr learned my addr 172.21.15.170:0/3062227692 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a2ba3700 1 --2- 172.21.15.170:0/3062227692 >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 0x7fa49c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a1ba1700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3062227692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 msgr2=0x7fa49c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3062227692 >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 msgr2=0x7fa49c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3062227692 >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 0x7fa49c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3062227692 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c0b1790 con 0x7fa49c159420 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a1ba1700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a2ba3700 1 --2- 172.21.15.170:0/3062227692 >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 0x7fa49c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.910+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b5f40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d1720 tx=0x7fa498009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aafecdc0d0f3b7bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a13a0700 1 -- 172.21.15.170:0/3062227692 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800e070 con 0x7fa49c159420 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a13a0700 1 -- 172.21.15.170:0/3062227692 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49800ba50 con 0x7fa49c159420 2023-10-08T16:12:40.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a13a0700 1 -- 172.21.15.170:0/3062227692 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4980092e0 con 0x7fa49c159420 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3062227692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 msgr2=0x7fa49c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b5f40 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa49c0d1720 tx=0x7fa498009840 comp rx=0 tx=0).stop 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3062227692 shutdown_connections 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3062227692 >> v2:172.21.15.191:3300/0 conn(0x7fa49c15d230 0x7fa49c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b5f40 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3062227692 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3062227692 >> 172.21.15.170:0/3062227692 conn(0x7fa49c0d0660 msgr2=0x7fa49c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3062227692 shutdown_connections 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.911+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3062227692 wait complete. 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.912+0000 7fa4a33a4700 1 Processor -- start 2023-10-08T16:12:40.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.912+0000 7fa4a33a4700 1 -- start start 2023-10-08T16:12:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa49c15d230 0x7fa49c155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c169890 con 0x7fa49c159420 2023-10-08T16:12:40.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c169b30 con 0x7fa49c15d230 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a33a4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c15c2c0 con 0x7fa49c1597f0 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35822/0 (socket says 172.21.15.170:35822) 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3396024226 learned_addr learned my addr 172.21.15.170:0/3396024226 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a1ba1700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a2ba3700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa49c15d230 0x7fa49c155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 msgr2=0x7fa49c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa49c15d230 msgr2=0x7fa49c155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa49c15d230 0x7fa49c155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:40.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.913+0000 7fa4a23a2700 1 -- 172.21.15.170:0/3396024226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa498009ae0 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4a23a2700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa498009880 tx=0x7fa498018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498008a80 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498008be0 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c15c5a0 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498026460 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.914+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c0c5fa0 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.915+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c15cac0 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.915+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa498003040 con 0x7fa49c159420 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.915+0000 7fa4927fc700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 0x7fa4800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:40.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.915+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa4980868d0 con 0x7fa49c159420 2023-10-08T16:12:40.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.916+0000 7fa4a1ba1700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 0x7fa4800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:40.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.916+0000 7fa4a1ba1700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 0x7fa4800697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa49c088560 tx=0x7fa48c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:40.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:40.920+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa49c15cac0 con 0x7fa49c159420 2023-10-08T16:12:41.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.102+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa49c15cac0 con 0x7fa49c159420 2023-10-08T16:12:41.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.103+0000 7fa4927fc700 1 -- 172.21.15.170:0/3396024226 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v162) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa498021060 con 0x7fa49c159420 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:41.105 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:41.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 msgr2=0x7fa4800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 0x7fa4800697d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa49c088560 tx=0x7fa48c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 msgr2=0x7fa49c0b59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7fa498009880 tx=0x7fa498018000 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 shutdown_connections 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa49c15d230 0x7fa49c155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa480067320 0x7fa4800697d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa49c159420 0x7fa49c0b59a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 --2- 172.21.15.170:0/3396024226 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa49c1597f0 0x7fa49c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 >> 172.21.15.170:0/3396024226 conn(0x7fa49c0d0660 msgr2=0x7fa49c158600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:41.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 shutdown_connections 2023-10-08T16:12:41.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.104+0000 7fa4a33a4700 1 -- 172.21.15.170:0/3396024226 wait complete. 2023-10-08T16:12:41.118 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:12:41.118 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:12:41.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.235+0000 7fe7f0a42700 1 Processor -- start 2023-10-08T16:12:41.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.235+0000 7fe7f0a42700 1 -- start start 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7f0a42700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe7ec159ec0 0x7fe7ec0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7f0a42700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0b5b00 con 0x7fe7ec0cf350 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7f0a42700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0b5850 con 0x7fe7ec0b53d0 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7f0a42700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b5da0 con 0x7fe7ec159ec0 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7ead9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe7ec0cf350 0x7fe7ec0cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46236/0 (socket says 172.21.15.170:46236) 2023-10-08T16:12:41.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7ead9d700 1 -- 172.21.15.170:0/2661358192 learned_addr learned my addr 172.21.15.170:0/2661358192 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/2661358192 >> v2:172.21.15.191:3300/0 conn(0x7fe7ec159ec0 0x7fe7ec0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705061666 0 0) 0x7fe7ec0b5b00 con 0x7fe7ec0cf350 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.236+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7d8003570 con 0x7fe7ec0cf350 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337240421 0 0) 0x7fe7d8003570 con 0x7fe7ec0cf350 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 >> v2:172.21.15.191:3300/0 conn(0x7fe7ec159ec0 msgr2=0x7fe7ec0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 --2- 172.21.15.170:0/2661358192 >> v2:172.21.15.191:3300/0 conn(0x7fe7ec159ec0 0x7fe7ec0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 >> v1:172.21.15.191:6789/0 conn(0x7fe7ec0b53d0 legacy=0x7fe7ec0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0b6080 con 0x7fe7ec0cf350 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fe7e40017f0 con 0x7fe7ec0cf350 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 >> v1:172.21.15.170:6789/0 conn(0x7fe7ec0cf350 legacy=0x7fe7ec0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d8003a60 0x7fe7d8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 0x7fe7d800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b5da0 con 0x7fe7d80051c0 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d800c0d0 con 0x7fe7d8004810 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d800c290 con 0x7fe7d8003a60 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 0x7fe7d800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d8003a60 0x7fe7d8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e1d9b700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 -- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d8003a60 msgr2=0x7fe7d8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d8003a60 0x7fe7d8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 -- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7d8004810 msgr2=0x7fe7d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:41.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 -- 172.21.15.170:0/2661358192 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d8003570 con 0x7fe7d80051c0 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e1d9b700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 -- 172.21.15.170:0/2661358192 reap_dead start 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7ead9d700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 0x7fe7d800ba60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a930 tx=0x7fe7e40021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf4fa6b10f5d451 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40070d0 con 0x7fe7d80051c0 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4007230 con 0x7fe7d80051c0 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.237+0000 7fe7e9d9b700 1 -- 172.21.15.170:0/2661358192 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40075d0 con 0x7fe7d80051c0 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 -- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 msgr2=0x7fe7d800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 0x7fe7d800ba60 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a930 tx=0x7fe7e40021d0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 -- 172.21.15.170:0/2661358192 shutdown_connections 2023-10-08T16:12:41.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/2661358192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7d80051c0 0x7fe7d800ba60 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 -- 172.21.15.170:0/2661358192 >> 172.21.15.170:0/2661358192 conn(0x7fe7ec0999f0 msgr2=0x7fe7ec0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:41.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 -- 172.21.15.170:0/2661358192 shutdown_connections 2023-10-08T16:12:41.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.238+0000 7fe7f0a42700 1 -- 172.21.15.170:0/2661358192 wait complete. 2023-10-08T16:12:41.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.239+0000 7fe7f0a42700 1 Processor -- start 2023-10-08T16:12:41.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.239+0000 7fe7f0a42700 1 -- start start 2023-10-08T16:12:41.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 0x7fe7ec150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec06a800 0x7fe7ec169760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec06aca0 con 0x7fe7ec155130 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec152c80 con 0x7fe7ec06a800 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7f0a42700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec069690 con 0x7fe7d80051c0 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35850/0 (socket says 172.21.15.170:35850) 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 -- 172.21.15.170:0/1784328181 learned_addr learned my addr 172.21.15.170:0/1784328181 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 0x7fe7ec150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7ead9d700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec06a800 0x7fe7ec169760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 -- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 msgr2=0x7fe7ec150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 0x7fe7ec150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 -- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec06a800 msgr2=0x7fe7ec169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec06a800 0x7fe7ec169760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 -- 172.21.15.170:0/1784328181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0690d0 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 0x7fe7ec150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e1d9b700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e580 tx=0x7fe7d4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.240+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4011460 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.241+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d4010040 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.241+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40117d0 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.241+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec0693b0 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.241+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec151740 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.241+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec152ad0 con 0x7fe7ec155130 2023-10-08T16:12:41.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.242+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe7d4011b40 con 0x7fe7ec155130 2023-10-08T16:12:41.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.242+0000 7fe7e2ffd700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.242+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe7d4083c30 con 0x7fe7ec155130 2023-10-08T16:12:41.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.242+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.243+0000 7fe7ea59c700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 0x7fe7cc0697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0090f0 tx=0x7fe7dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.246+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7d404d5a0 con 0x7fe7ec155130 2023-10-08T16:12:41.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.430+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7ec063c90 con 0x7fe7ec155130 2023-10-08T16:12:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.646+0000 7fe7e2ffd700 1 -- 172.21.15.170:0/1784328181 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v163) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe7d40190b0 con 0x7fe7ec155130 2023-10-08T16:12:41.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.649+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 msgr2=0x7fe7cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.649+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 0x7fe7cc0697d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0090f0 tx=0x7fe7dc008040 comp rx=0 tx=0).stop 2023-10-08T16:12:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.649+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 msgr2=0x7fe7ec06a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.649+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e580 tx=0x7fe7d4009040 comp rx=0 tx=0).stop 2023-10-08T16:12:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 shutdown_connections 2023-10-08T16:12:41.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe7ec06a800 0x7fe7ec169760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe7d80051c0 0x7fe7ec150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe7cc067320 0x7fe7cc0697d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 --2- 172.21.15.170:0/1784328181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec155130 0x7fe7ec06a2c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 >> 172.21.15.170:0/1784328181 conn(0x7fe7ec0999f0 msgr2=0x7fe7ec151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 shutdown_connections 2023-10-08T16:12:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.650+0000 7fe7f0a42700 1 -- 172.21.15.170:0/1784328181 wait complete. 2023-10-08T16:12:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:12:41.665 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:12:41.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.785+0000 7fe844d1e700 1 Processor -- start 2023-10-08T16:12:41.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.785+0000 7fe844d1e700 1 -- start start 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe844d1e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe840159950 0x7fe8400cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe844d1e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe8400af320 0x7fe840069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe844d1e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400ce380 con 0x7fe840069aa0 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe844d1e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400cec20 con 0x7fe840159950 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe844d1e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe840069ee0 con 0x7fe8400af320 2023-10-08T16:12:41.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe840159950 0x7fe8400cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe840159950 0x7fe8400cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35858/0 (socket says 172.21.15.170:35858) 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 -- 172.21.15.170:0/1519577773 learned_addr learned my addr 172.21.15.170:0/1519577773 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 -- 172.21.15.170:0/1519577773 >> v2:172.21.15.191:3300/0 conn(0x7fe8400af320 msgr2=0x7fe840069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 --2- 172.21.15.170:0/1519577773 >> v2:172.21.15.191:3300/0 conn(0x7fe8400af320 0x7fe840069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 -- 172.21.15.170:0/1519577773 >> v1:172.21.15.191:6789/0 conn(0x7fe840069aa0 legacy=0x7fe84015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 -- 172.21.15.170:0/1519577773 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400afa90 con 0x7fe840159950 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.786+0000 7fe83e59c700 1 --2- 172.21.15.170:0/1519577773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe8400cbb90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe84015d450 tx=0x7fe830009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f9717bad2fb29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe83dd9b700 1 -- 172.21.15.170:0/1519577773 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000b820 con 0x7fe840159950 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe83dd9b700 1 -- 172.21.15.170:0/1519577773 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000b980 con 0x7fe840159950 2023-10-08T16:12:41.790 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe83dd9b700 1 -- 172.21.15.170:0/1519577773 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83000bd20 con 0x7fe840159950 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 -- 172.21.15.170:0/1519577773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 msgr2=0x7fe8400cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 --2- 172.21.15.170:0/1519577773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe8400cbb90 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fe84015d450 tx=0x7fe830009250 comp rx=0 tx=0).stop 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 -- 172.21.15.170:0/1519577773 shutdown_connections 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 --2- 172.21.15.170:0/1519577773 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe8400cbb90 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 --2- 172.21.15.170:0/1519577773 >> v2:172.21.15.191:3300/0 conn(0x7fe8400af320 0x7fe840069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 -- 172.21.15.170:0/1519577773 >> 172.21.15.170:0/1519577773 conn(0x7fe840068e00 msgr2=0x7fe8400d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 -- 172.21.15.170:0/1519577773 shutdown_connections 2023-10-08T16:12:41.791 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.787+0000 7fe844d1e700 1 -- 172.21.15.170:0/1519577773 wait complete. 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 Processor -- start 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 -- start start 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe840069aa0 0x7fe840150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8400af320 0x7fe840169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84016b750 con 0x7fe840159950 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b5850 con 0x7fe840069aa0 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe844d1e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84016ab10 con 0x7fe8400af320 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe83ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe83ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35872/0 (socket says 172.21.15.170:35872) 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe83ed9d700 1 -- 172.21.15.170:0/980149171 learned_addr learned my addr 172.21.15.170:0/980149171 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.789+0000 7fe83e59c700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe840069aa0 0x7fe840150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe835d9b700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8400af320 0x7fe840169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 -- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8400af320 msgr2=0x7fe840169f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8400af320 0x7fe840169f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 -- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe840069aa0 msgr2=0x7fe840150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:41.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe840069aa0 0x7fe840150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 -- 172.21.15.170:0/980149171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300094f0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe83ed9d700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe83800c930 tx=0x7fe8380026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838012630 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8380127d0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84016adf0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838012b90 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.790+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84016b3e0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.791+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8400718b0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.792+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe838006ba0 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.792+0000 7fe836ffd700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 0x7fe8200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.792+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fe838084e90 con 0x7fe840159950 2023-10-08T16:12:41.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.792+0000 7fe83e59c700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 0x7fe8200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:41.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.793+0000 7fe83e59c700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 0x7fe8200697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe84015d450 tx=0x7fe830016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:41.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.796+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe838051d80 con 0x7fe840159950 2023-10-08T16:12:41.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:41.976+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe84016b890 con 0x7fe840159950 2023-10-08T16:12:42.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.648+0000 7fe836ffd700 1 -- 172.21.15.170:0/980149171 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v164) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe8380553a0 con 0x7fe840159950 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 msgr2=0x7fe8200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 0x7fe8200697d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe84015d450 tx=0x7fe830016000 comp rx=0 tx=0).stop 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 msgr2=0x7fe84016a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fe83800c930 tx=0x7fe8380026e0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 shutdown_connections 2023-10-08T16:12:42.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe820067320 0x7fe8200697d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe840159950 0x7fe84016a470 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe840069aa0 0x7fe840150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 --2- 172.21.15.170:0/980149171 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe8400af320 0x7fe840169f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.656+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 >> 172.21.15.170:0/980149171 conn(0x7fe840068e00 msgr2=0x7fe840150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.657+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 shutdown_connections 2023-10-08T16:12:42.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.657+0000 7fe844d1e700 1 -- 172.21.15.170:0/980149171 wait complete. 2023-10-08T16:12:42.662 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:12:42.678 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:12:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.794+0000 7f9c01655700 1 Processor -- start 2023-10-08T16:12:42.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.794+0000 7f9c01655700 1 -- start start 2023-10-08T16:12:42.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9c01655700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9bfc1575a0 0x7f9bfc0cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9c01655700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0b1100 con 0x7f9bfc063580 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9c01655700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bfc0c3ae0 con 0x7f9bfc06eb80 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9c01655700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bfc0cbe60 con 0x7f9bfc1575a0 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9bfb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9bfc063580 0x7f9bfc06e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46252/0 (socket says 172.21.15.170:46252) 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.795+0000 7f9bfb7fe700 1 -- 172.21.15.170:0/2898198500 learned_addr learned my addr 172.21.15.170:0/2898198500 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfbfff700 1 --2- 172.21.15.170:0/2898198500 >> v2:172.21.15.191:3300/0 conn(0x7f9bfc1575a0 0x7f9bfc0cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1101062758 0 0) 0x7f9bfc0b1100 con 0x7f9bfc063580 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be4003570 con 0x7f9bfc063580 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453965256 0 0) 0x7f9bfc0c3ae0 con 0x7f9bfc06eb80 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bfc0b1100 con 0x7f9bfc06eb80 2023-10-08T16:12:42.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2898198500 >> v1:172.21.15.191:6789/0 conn(0x7f9bfc06eb80 legacy=0x7f9bfc0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432983581 0 0) 0x7f9be4003570 con 0x7f9bfc063580 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2898198500 >> v1:172.21.15.170:6789/0 conn(0x7f9bfc063580 legacy=0x7f9bfc06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2898198500 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0cc140 con 0x7f9bfc1575a0 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfbfff700 1 --2- 172.21.15.170:0/2898198500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc0cb610 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0098e0 tx=0x7f9bec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421a37d086251e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0001470 con 0x7f9bfc1575a0 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec011420 con 0x7f9bfc1575a0 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.796+0000 7f9bfaffd700 1 -- 172.21.15.170:0/2898198500 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0117a0 con 0x7f9bfc1575a0 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 -- 172.21.15.170:0/2898198500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 msgr2=0x7f9bfc0cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 --2- 172.21.15.170:0/2898198500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc0cb610 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0098e0 tx=0x7f9bec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 -- 172.21.15.170:0/2898198500 shutdown_connections 2023-10-08T16:12:42.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 --2- 172.21.15.170:0/2898198500 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc0cb610 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 -- 172.21.15.170:0/2898198500 >> 172.21.15.170:0/2898198500 conn(0x7f9bfc0c9f70 msgr2=0x7f9bfc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 -- 172.21.15.170:0/2898198500 shutdown_connections 2023-10-08T16:12:42.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.797+0000 7f9c01655700 1 -- 172.21.15.170:0/2898198500 wait complete. 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.798+0000 7f9c01655700 1 Processor -- start 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.798+0000 7f9c01655700 1 -- start start 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bfc06eb80 0x7f9bfc154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc167b50 con 0x7f9bfc063580 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc168950 con 0x7f9bfc1575a0 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c01655700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc167d40 con 0x7f9bfc06eb80 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:42.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35888/0 (socket says 172.21.15.170:35888) 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2289225006 learned_addr learned my addr 172.21.15.170:0/2289225006 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfb7fe700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bfc06eb80 0x7f9bfc154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c00e54700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bfc06eb80 msgr2=0x7f9bfc154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bfc06eb80 0x7f9bfc154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 msgr2=0x7f9bfc1553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc1553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 -- 172.21.15.170:0/2289225006 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009420 con 0x7f9bfc063580 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9c00e54700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9bfbfff700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0093f0 tx=0x7f9bec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:42.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01cc90 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.799+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec01ce30 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.800+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bfc168020 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.800+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec01a6a0 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.800+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bfc1685e0 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.800+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bfc06cfa0 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.801+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9bec01aa10 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.801+0000 7f9be3fff700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 0x7f9be8069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.802+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f9bec086d60 con 0x7f9bfc063580 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.802+0000 7f9bfb7fe700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 0x7f9be8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:42.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.802+0000 7f9bfb7fe700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 0x7f9be8069800 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc15a860 tx=0x7f9bf0000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:42.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:42.805+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bec053b00 con 0x7f9bfc063580 2023-10-08T16:12:43.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.030+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9bfc0cc420 con 0x7f9bfc063580 2023-10-08T16:12:43.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.672+0000 7f9be3fff700 1 -- 172.21.15.170:0/2289225006 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v105) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9bec0330c0 con 0x7f9bfc063580 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.675+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 msgr2=0x7f9be8069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.675+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 0x7f9be8069800 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc15a860 tx=0x7f9bf0000bc0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.675+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 msgr2=0x7f9bfc1595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.675+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0093f0 tx=0x7f9bec016000 comp rx=0 tx=0).stop 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 shutdown_connections 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9bfc1575a0 0x7f9bfc1553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9be8067350 0x7f9be8069800 secure :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc15a860 tx=0x7f9bf0000bc0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bfc063580 0x7f9bfc1595f0 secure :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0093f0 tx=0x7f9bec016000 comp rx=0 tx=0).stop 2023-10-08T16:12:43.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 --2- 172.21.15.170:0/2289225006 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9bfc06eb80 0x7f9bfc154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 >> 172.21.15.170:0/2289225006 conn(0x7f9bfc0c9f70 msgr2=0x7f9bfc0c32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:43.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 shutdown_connections 2023-10-08T16:12:43.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.676+0000 7f9c01655700 1 -- 172.21.15.170:0/2289225006 wait complete. 2023-10-08T16:12:43.684 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 28 and data pool 29 2023-10-08T16:12:43.693 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:43.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.806+0000 7f68cd7ae700 1 Processor -- start 2023-10-08T16:12:43.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.806+0000 7f68cd7ae700 1 -- start start 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68cd7ae700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f68c8159db0 0x7f68c8069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68cd7ae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f68c806a2b0 0x7f68c806a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68cd7ae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c80c96a0 con 0x7f68c81599e0 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68cd7ae700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c80cb9e0 con 0x7f68c8159db0 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68cd7ae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c80cab60 con 0x7f68c806a2b0 2023-10-08T16:12:43.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68be7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f68c8159db0 0x7f68c8069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68be7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f68c8159db0 0x7f68c8069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35904/0 (socket says 172.21.15.170:35904) 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.807+0000 7f68be7fc700 1 -- 172.21.15.170:0/2343034655 learned_addr learned my addr 172.21.15.170:0/2343034655 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68be7fc700 1 -- 172.21.15.170:0/2343034655 >> v2:172.21.15.191:3300/0 conn(0x7f68c806a2b0 msgr2=0x7f68c806a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68c77fe700 1 --2- 172.21.15.170:0/2343034655 >> v2:172.21.15.191:3300/0 conn(0x7f68c806a2b0 0x7f68c806a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68be7fc700 1 --2- 172.21.15.170:0/2343034655 >> v2:172.21.15.191:3300/0 conn(0x7f68c806a2b0 0x7f68c806a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68be7fc700 1 -- 172.21.15.170:0/2343034655 >> v1:172.21.15.191:6789/0 conn(0x7f68c81599e0 legacy=0x7f68c8069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68be7fc700 1 -- 172.21.15.170:0/2343034655 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c80cb420 con 0x7f68c8159db0 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68be7fc700 1 --2- 172.21.15.170:0/2343034655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c8159db0 0x7f68c8069ac0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f68c806cbe0 tx=0x7f68b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca9cde06d7339b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68c67fc700 1 -- 172.21.15.170:0/2343034655 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000b820 con 0x7f68c8159db0 2023-10-08T16:12:43.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68c67fc700 1 -- 172.21.15.170:0/2343034655 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b000b980 con 0x7f68c8159db0 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68c67fc700 1 -- 172.21.15.170:0/2343034655 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b000bd20 con 0x7f68c8159db0 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2343034655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c8159db0 msgr2=0x7f68c8069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2343034655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c8159db0 0x7f68c8069ac0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f68c806cbe0 tx=0x7f68b0009250 comp rx=0 tx=0).stop 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2343034655 shutdown_connections 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2343034655 >> v2:172.21.15.191:3300/0 conn(0x7f68c806a2b0 0x7f68c806a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2343034655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c8159db0 0x7f68c8069ac0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2343034655 >> 172.21.15.170:0/2343034655 conn(0x7f68c80c9860 msgr2=0x7f68c80c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.808+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2343034655 shutdown_connections 2023-10-08T16:12:43.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.809+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2343034655 wait complete. 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 Processor -- start 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 -- start start 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68c806a2b0 0x7f68c815ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 0x7f68c814c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80c3f30 con 0x7f68c81599e0 2023-10-08T16:12:43.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80b2550 con 0x7f68c8159db0 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.810+0000 7f68cd7ae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c814cdc0 con 0x7f68c806a2b0 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35918/0 (socket says 172.21.15.170:35918) 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 -- 172.21.15.170:0/2662429898 learned_addr learned my addr 172.21.15.170:0/2662429898 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68c77fe700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 0x7f68c814c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68c6ffd700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68c806a2b0 0x7f68c815ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 -- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68c806a2b0 msgr2=0x7f68c815ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68c806a2b0 0x7f68c815ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 -- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 msgr2=0x7f68c814c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 0x7f68c814c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:43.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 -- 172.21.15.170:0/2662429898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00094f0 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68c77fe700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 0x7f68c814c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68be7fc700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009290 tx=0x7f68b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b0011e20 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b0020070 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b001b6a0 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c816a6c0 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.811+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c816ac80 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.812+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c80c40a0 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.812+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f68b0022030 con 0x7f68c81599e0 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.813+0000 7f68bf7fe700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 0x7f68a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:43.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.813+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f68b0086a60 con 0x7f68c81599e0 2023-10-08T16:12:43.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.813+0000 7f68c6ffd700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 0x7f68a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:43.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.813+0000 7f68c6ffd700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 0x7f68a80697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f68b800a8c0 tx=0x7f68b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:43.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:43.817+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68c80c40a0 con 0x7f68c81599e0 2023-10-08T16:12:44.003 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.002+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68c80c40a0 con 0x7f68c81599e0 2023-10-08T16:12:44.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.003+0000 7f68bf7fe700 1 -- 172.21.15.170:0/2662429898 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f68c80c40a0 con 0x7f68c81599e0 2023-10-08T16:12:44.004 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:44.004 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":165,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:43.032109+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 msgr2=0x7f68a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 0x7f68a80697d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f68b800a8c0 tx=0x7f68b8008040 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 msgr2=0x7f68c816b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009290 tx=0x7f68b0016000 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 shutdown_connections 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f68c8159db0 0x7f68c814c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f68c806a2b0 0x7f68c815ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f68a8067320 0x7f68a80697d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 --2- 172.21.15.170:0/2662429898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68c81599e0 0x7f68c816b240 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 >> 172.21.15.170:0/2662429898 conn(0x7f68c80c9860 msgr2=0x7f68c814ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:44.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 shutdown_connections 2023-10-08T16:12:44.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.005+0000 7f68cd7ae700 1 -- 172.21.15.170:0/2662429898 wait complete. 2023-10-08T16:12:44.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:44.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.132+0000 7fe4a0dd1700 1 Processor -- start 2023-10-08T16:12:44.133 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.132+0000 7fe4a0dd1700 1 -- start start 2023-10-08T16:12:44.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe4a0dd1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe49c159c60 0x7fe49c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe4a0dd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe4a0dd1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c15a160 con 0x7fe49c159890 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe4a0dd1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c15df50 con 0x7fe49c159c60 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe4a0dd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0d13a0 con 0x7fe49c15db10 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe49c159c60 0x7fe49c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49a59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe49c159890 0x7fe49c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46256/0 (socket says 172.21.15.170:46256) 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49a59c700 1 -- 172.21.15.170:0/3543591680 learned_addr learned my addr 172.21.15.170:0/3543591680 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49ad9d700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 -- 172.21.15.170:0/3543591680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 msgr2=0x7fe49c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:44.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 -- 172.21.15.170:0/3543591680 >> v1:172.21.15.170:6789/0 conn(0x7fe49c159890 legacy=0x7fe49c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 -- 172.21.15.170:0/3543591680 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c15e140 con 0x7fe49c159c60 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49959a700 1 -- 172.21.15.170:0/3543591680 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748595774 0 0) 0x7fe49c15a160 con 0x7fe49c159890 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe499d9b700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159c60 0x7fe49c15d5d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098e0 tx=0x7fe4900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1ac3be05e98eb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49959a700 1 -- 172.21.15.170:0/3543591680 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe490009e80 con 0x7fe49c159c60 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49959a700 1 -- 172.21.15.170:0/3543591680 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49000b860 con 0x7fe49c159c60 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.133+0000 7fe49959a700 1 -- 172.21.15.170:0/3543591680 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000bbe0 con 0x7fe49c159c60 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/3543591680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159c60 msgr2=0x7fe49c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159c60 0x7fe49c15d5d0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe4900098e0 tx=0x7fe4900091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/3543591680 shutdown_connections 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/3543591680 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159c60 0x7fe49c15d5d0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/3543591680 >> 172.21.15.170:0/3543591680 conn(0x7fe49c068e20 msgr2=0x7fe49c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/3543591680 shutdown_connections 2023-10-08T16:12:44.138 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.134+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/3543591680 wait complete. 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.135+0000 7fe4a0dd1700 1 Processor -- start 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.135+0000 7fe4a0dd1700 1 -- start start 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 0x7fe49c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0b2770 con 0x7fe49c159890 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0b2490 con 0x7fe49c159c60 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe4a0dd1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0b5dd0 con 0x7fe49c15db10 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35928/0 (socket says 172.21.15.170:35928) 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 -- 172.21.15.170:0/999250520 learned_addr learned my addr 172.21.15.170:0/999250520 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe499d9b700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 0x7fe49c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49ad9d700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 -- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 msgr2=0x7fe49c158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 -- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 msgr2=0x7fe49c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 0x7fe49c0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 -- 172.21.15.170:0/999250520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe490009420 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe499d9b700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 0x7fe49c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49ad9d700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.136+0000 7fe49a59c700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c7e0 tx=0x7fe48c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.137+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c005b30 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.137+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c005c90 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.137+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c0b60b0 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.137+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c007300 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.137+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c144660 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.138+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c0cd100 con 0x7fe49c159890 2023-10-08T16:12:44.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.138+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe48c01b400 con 0x7fe49c159890 2023-10-08T16:12:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.138+0000 7fe48a7fc700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.139+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe48c084670 con 0x7fe49c159890 2023-10-08T16:12:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.139+0000 7fe499d9b700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.139+0000 7fe499d9b700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 0x7fe47c0697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0ca9f0 tx=0x7fe490017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.143+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe48c051480 con 0x7fe49c159890 2023-10-08T16:12:44.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.327+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe49c04c3c0 con 0x7fe49c159890 2023-10-08T16:12:44.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.328+0000 7fe48a7fc700 1 -- 172.21.15.170:0/999250520 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+15425 (secure 0 0 0) 0x7fe48c054aa0 con 0x7fe49c159890 2023-10-08T16:12:44.329 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:44.330 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":165,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:43.032109+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 msgr2=0x7fe47c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 0x7fe47c0697d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0ca9f0 tx=0x7fe490017000 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 msgr2=0x7fe49c144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c7e0 tx=0x7fe48c007df0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 shutdown_connections 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe49c15db10 0x7fe49c158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe47c067320 0x7fe47c0697d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe49c159890 0x7fe49c144120 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 --2- 172.21.15.170:0/999250520 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe49c159c60 0x7fe49c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.332 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 >> 172.21.15.170:0/999250520 conn(0x7fe49c068e20 msgr2=0x7fe49c153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:44.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 shutdown_connections 2023-10-08T16:12:44.333 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.330+0000 7fe4a0dd1700 1 -- 172.21.15.170:0/999250520 wait complete. 2023-10-08T16:12:44.344 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:12:44.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.459+0000 7f800d099700 1 Processor -- start 2023-10-08T16:12:44.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.459+0000 7f800d099700 1 -- start start 2023-10-08T16:12:44.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800d099700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8008159c60 0x7f800815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800d099700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800815a160 con 0x7f800815db10 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800d099700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080d13a0 con 0x7f8008159890 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800d099700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800815df50 con 0x7f8008159c60 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800759e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f800815db10 0x7f8008069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46270/0 (socket says 172.21.15.170:46270) 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.460+0000 7f800759e700 1 -- 172.21.15.170:0/1801374388 learned_addr learned my addr 172.21.15.170:0/1801374388 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800659c700 1 --2- 172.21.15.170:0/1801374388 >> v2:172.21.15.191:3300/0 conn(0x7f8008159c60 0x7f800815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011656998 0 0) 0x7f800815a160 con 0x7f800815db10 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff0003570 con 0x7f800815db10 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2389048262 0 0) 0x7f7ff0003570 con 0x7f800815db10 2023-10-08T16:12:44.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 >> v2:172.21.15.191:3300/0 conn(0x7f8008159c60 msgr2=0x7f800815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 --2- 172.21.15.170:0/1801374388 >> v2:172.21.15.191:3300/0 conn(0x7f8008159c60 0x7f800815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 >> v1:172.21.15.191:6789/0 conn(0x7f8008159890 legacy=0x7f800815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800815e140 con 0x7f800815db10 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f80000017f0 con 0x7f800815db10 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 >> v1:172.21.15.170:6789/0 conn(0x7f800815db10 legacy=0x7f8008069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ff0003a60 0x7f7ff0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff0004810 0x7f7ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 0x7f7ff000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000c0a0 con 0x7f7ff00051c0 2023-10-08T16:12:44.466 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000c210 con 0x7f7ff0004810 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000c3d0 con 0x7f7ff0003a60 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 0x7f7ff000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800659c700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff0004810 0x7f7ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8006d9d700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ff0003a60 0x7f7ff0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 -- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ff0003a60 msgr2=0x7f7ff0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ff0003a60 0x7f7ff0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 -- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff0004810 msgr2=0x7f7ff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff0004810 0x7f7ff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 -- 172.21.15.170:0/1801374388 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0003570 con 0x7f7ff00051c0 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f8006d9d700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ff0003a60 0x7f7ff0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 -- 172.21.15.170:0/1801374388 reap_dead start 2023-10-08T16:12:44.467 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.461+0000 7f800759e700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 0x7f7ff000ba60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f800000a930 tx=0x7f80000021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4404a0c3079b8dbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000070d0 con 0x7f7ff00051c0 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000007230 con 0x7f7ff00051c0 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f8005d9b700 1 -- 172.21.15.170:0/1801374388 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000075d0 con 0x7f7ff00051c0 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 -- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 msgr2=0x7f7ff000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 0x7f7ff000ba60 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f800000a930 tx=0x7f80000021d0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 -- 172.21.15.170:0/1801374388 shutdown_connections 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 --2- 172.21.15.170:0/1801374388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ff00051c0 0x7f7ff000ba60 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 -- 172.21.15.170:0/1801374388 >> 172.21.15.170:0/1801374388 conn(0x7f8008068e20 msgr2=0x7f80080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 -- 172.21.15.170:0/1801374388 shutdown_connections 2023-10-08T16:12:44.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.462+0000 7f800d099700 1 -- 172.21.15.170:0/1801374388 wait complete. 2023-10-08T16:12:44.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.463+0000 7f800d099700 1 Processor -- start 2023-10-08T16:12:44.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.463+0000 7f800d099700 1 -- start start 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff00051c0 0x7f80080cf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80080cfa90 0x7f800815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80080cff30 con 0x7f8008158630 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008149520 con 0x7f7ff00051c0 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800d099700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8008156840 con 0x7f80080cfa90 2023-10-08T16:12:44.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:35952/0 (socket says 172.21.15.170:35952) 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800759e700 1 -- 172.21.15.170:0/2706143555 learned_addr learned my addr 172.21.15.170:0/2706143555 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f8006d9d700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff00051c0 0x7f80080cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800659c700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80080cfa90 0x7f800815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.464+0000 7f800759e700 1 -- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80080cfa90 msgr2=0x7f800815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800759e700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80080cfa90 0x7f800815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800759e700 1 -- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff00051c0 msgr2=0x7f80080cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800759e700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff00051c0 0x7f80080cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800759e700 1 -- 172.21.15.170:0/2706143555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008156b20 con 0x7f8008158630 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800759e700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f800000afd0 tx=0x7f8000016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000075b0 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000017b0 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8008156e00 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800001d420 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.465+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800815bdb0 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.466+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008157320 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.466+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f800001d7f0 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.466+0000 7f7feeffd700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.467+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8000086c50 con 0x7f8008158630 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.467+0000 7f8006d9d700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:44.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.467+0000 7f8006d9d700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8000c00 tx=0x7f7ff800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:44.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.471+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8008157320 con 0x7f8008158630 2023-10-08T16:12:44.512 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:12:44.515 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:12:44.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:44.717+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8008157320 con 0x7f8008158630 2023-10-08T16:12:45.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.690+0000 7f7feeffd700 1 -- 172.21.15.170:0/2706143555 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v108) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8008157320 con 0x7f8008158630 2023-10-08T16:12:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 msgr2=0x7f7ff40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8000c00 tx=0x7f7ff800c040 comp rx=0 tx=0).stop 2023-10-08T16:12:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 msgr2=0x7f8008157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f800000afd0 tx=0x7f8000016000 comp rx=0 tx=0).stop 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 shutdown_connections 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7ff4067320 0x7f7ff40697d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008158630 0x7f8008157600 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ff00051c0 0x7f80080cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 --2- 172.21.15.170:0/2706143555 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f80080cfa90 0x7f800815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 >> 172.21.15.170:0/2706143555 conn(0x7f8008068e20 msgr2=0x7f8008149c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 shutdown_connections 2023-10-08T16:12:45.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.693+0000 7f800d099700 1 -- 172.21.15.170:0/2706143555 wait complete. 2023-10-08T16:12:45.707 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:12:45.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.824+0000 7f13031c5700 1 Processor -- start 2023-10-08T16:12:45.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.824+0000 7f13031c5700 1 -- start start 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f13031c5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f12fc159c60 0x7f12fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f13031c5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc15a160 con 0x7f12fc159890 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f13031c5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12fc0d13a0 con 0x7f12fc15db10 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f13031c5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc15df50 con 0x7f12fc159c60 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f1300f61700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f12fc159890 0x7f12fc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39402/0 (socket says 172.21.15.170:39402) 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f1300f61700 1 -- 172.21.15.170:0/1587325842 learned_addr learned my addr 172.21.15.170:0/1587325842 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f12f3fff700 1 --2- 172.21.15.170:0/1587325842 >> v2:172.21.15.191:3300/0 conn(0x7f12fc159c60 0x7f12fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791506005 0 0) 0x7f12fc15a160 con 0x7f12fc159890 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.825+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12e4003570 con 0x7f12fc159890 2023-10-08T16:12:45.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88816489 0 0) 0x7f12e4003570 con 0x7f12fc159890 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 >> v2:172.21.15.191:3300/0 conn(0x7f12fc159c60 msgr2=0x7f12fc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 --2- 172.21.15.170:0/1587325842 >> v2:172.21.15.191:3300/0 conn(0x7f12fc159c60 0x7f12fc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 >> v1:172.21.15.191:6789/0 conn(0x7f12fc15db10 legacy=0x7f12fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc15e140 con 0x7f12fc159890 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f12f8001470 con 0x7f12fc159890 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 >> v1:172.21.15.170:6789/0 conn(0x7f12fc159890 legacy=0x7f12fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4003a60 0x7f12e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 0x7f12e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e40051c0 0x7f12e400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e400c0a0 con 0x7f12e4004810 2023-10-08T16:12:45.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e400c210 con 0x7f12e4003a60 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e400c3d0 con 0x7f12e40051c0 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 0x7f12e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f1300f61700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4003a60 0x7f12e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f1301762700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e40051c0 0x7f12e400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 -- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e40051c0 msgr2=0x7f12e400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12e40051c0 0x7f12e400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 -- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4003a60 msgr2=0x7f12e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4003a60 0x7f12e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 -- 172.21.15.170:0/1587325842 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4003570 con 0x7f12e4004810 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f1301762700 1 -- 172.21.15.170:0/1587325842 reap_dead start 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f3fff700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 0x7f12e4004c80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009c00 tx=0x7f12ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54494b71deffcbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00e710 con 0x7f12e4004810 2023-10-08T16:12:45.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00e870 con 0x7f12e4004810 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.826+0000 7f12f37fe700 1 -- 172.21.15.170:0/1587325842 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00ec30 con 0x7f12e4004810 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 -- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 msgr2=0x7f12e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 0x7f12e4004c80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009c00 tx=0x7f12ec0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 -- 172.21.15.170:0/1587325842 shutdown_connections 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 --2- 172.21.15.170:0/1587325842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12e4004810 0x7f12e4004c80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 -- 172.21.15.170:0/1587325842 >> 172.21.15.170:0/1587325842 conn(0x7f12fc068e20 msgr2=0x7f12fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 -- 172.21.15.170:0/1587325842 shutdown_connections 2023-10-08T16:12:45.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.827+0000 7f13031c5700 1 -- 172.21.15.170:0/1587325842 wait complete. 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.828+0000 7f13031c5700 1 Processor -- start 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.828+0000 7f13031c5700 1 -- start start 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 0x7f12fc150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12fc0c9870 0x7f12fc0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc16ace0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0c8250 con 0x7f12e4004810 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f13031c5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0caaa0 con 0x7f12fc0c9870 2023-10-08T16:12:45.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60988/0 (socket says 172.21.15.170:60988) 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 -- 172.21.15.170:0/3261344742 learned_addr learned my addr 172.21.15.170:0/3261344742 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f1300f61700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 0x7f12fc150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f1301762700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12fc0c9870 0x7f12fc0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 -- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12fc0c9870 msgr2=0x7f12fc0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12fc0c9870 0x7f12fc0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 -- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 msgr2=0x7f12fc150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 0x7f12fc150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 -- 172.21.15.170:0/3261344742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc0ce530 con 0x7f12fc0c78a0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f1300f61700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 0x7f12fc150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.829+0000 7f12f3fff700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f12e400c1e0 tx=0x7f12ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.830+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec022070 con 0x7f12fc0c78a0 2023-10-08T16:12:45.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.830+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00e9e0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.830+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc169fd0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.830+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec01c450 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.830+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc0ca4e0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.831+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc0ce6c0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.831+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f12ec01c880 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.831+0000 7f12f0ff9700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 0x7f12e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.831+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12ec0863f0 con 0x7f12fc0c78a0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.832+0000 7f1300f61700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 0x7f12e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.832+0000 7f1300f61700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 0x7f12e80697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f12f8001050 tx=0x7f12f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:45.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:45.836+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12ec053130 con 0x7f12fc0c78a0 2023-10-08T16:12:46.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.061+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f12fc16acc0 con 0x7f12fc0c78a0 2023-10-08T16:12:46.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.695+0000 7f12f0ff9700 1 -- 172.21.15.170:0/3261344742 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v109) v1 ==== 122+0+0 (secure 0 0 0) 0x7f12ec056750 con 0x7f12fc0c78a0 2023-10-08T16:12:46.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 msgr2=0x7f12e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 0x7f12e80697d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f12f8001050 tx=0x7f12f8009040 comp rx=0 tx=0).stop 2023-10-08T16:12:46.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 msgr2=0x7f12fc0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f12e400c1e0 tx=0x7f12ec01a000 comp rx=0 tx=0).stop 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 shutdown_connections 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12fc0c9870 0x7f12fc0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12e4004810 0x7f12fc150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12e8067320 0x7f12e80697d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 --2- 172.21.15.170:0/3261344742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12fc0c78a0 0x7f12fc0c7d10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 >> 172.21.15.170:0/3261344742 conn(0x7f12fc068e20 msgr2=0x7f12fc0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 shutdown_connections 2023-10-08T16:12:46.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.698+0000 7f13031c5700 1 -- 172.21.15.170:0/3261344742 wait complete. 2023-10-08T16:12:46.711 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:46.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.824+0000 7f8e7287f700 1 Processor -- start 2023-10-08T16:12:46.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.824+0000 7f8e7287f700 1 -- start start 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7287f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8e6c159420 0x7f8e6c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7287f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0b2980 con 0x7f8e6c15d230 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7287f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0b11d0 con 0x7f8e6c1597f0 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7287f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b14b0 con 0x7f8e6c159420 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7207e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8e6c15d230 0x7f8e6c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39412/0 (socket says 172.21.15.170:39412) 2023-10-08T16:12:46.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7207e700 1 -- 172.21.15.170:0/1616968325 learned_addr learned my addr 172.21.15.170:0/1616968325 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342780244 0 0) 0x7f8e6c0b2980 con 0x7f8e6c15d230 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e58003570 con 0x7f8e6c15d230 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7187d700 1 --2- 172.21.15.170:0/1616968325 >> v2:172.21.15.191:3300/0 conn(0x7f8e6c159420 0x7f8e6c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 440203076 0 0) 0x7f8e58003570 con 0x7f8e6c15d230 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 >> v2:172.21.15.191:3300/0 conn(0x7f8e6c159420 msgr2=0x7f8e6c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 --2- 172.21.15.170:0/1616968325 >> v2:172.21.15.191:3300/0 conn(0x7f8e6c159420 0x7f8e6c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 >> v1:172.21.15.191:6789/0 conn(0x7f8e6c1597f0 legacy=0x7f8e6c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c0b1790 con 0x7f8e6c15d230 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f8e680017f0 con 0x7f8e6c15d230 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 >> v1:172.21.15.170:6789/0 conn(0x7f8e6c15d230 legacy=0x7f8e6c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.825+0000 7f8e7087b700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 0x7f8e58003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e58004810 0x7f8e58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58005230 0x7f8e5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b14b0 con 0x7f8e58003a60 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5800c160 con 0x7f8e58004810 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5800c320 con 0x7f8e58005230 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 0x7f8e58003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7107c700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e58004810 0x7f8e58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7207e700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58005230 0x7f8e5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 -- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58005230 msgr2=0x7f8e5800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58005230 0x7f8e5800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 -- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e58004810 msgr2=0x7f8e58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:46.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e58004810 0x7f8e58004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 -- 172.21.15.170:0/1616968325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58003570 con 0x7f8e58003a60 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7107c700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e58004810 0x7f8e58004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7207e700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58005230 0x7f8e5800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7207e700 1 -- 172.21.15.170:0/1616968325 reap_dead start 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7187d700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 0x7f8e58003e50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400bb90 tx=0x7f8e64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23deeca171b1800 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400d5d0 con 0x7f8e58003a60 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6400d730 con 0x7f8e58003a60 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.826+0000 7f8e7087b700 1 -- 172.21.15.170:0/1616968325 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400daf0 con 0x7f8e58003a60 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 -- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 msgr2=0x7f8e58003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 0x7f8e58003e50 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400bb90 tx=0x7f8e64008040 comp rx=0 tx=0).stop 2023-10-08T16:12:46.834 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 -- 172.21.15.170:0/1616968325 shutdown_connections 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 --2- 172.21.15.170:0/1616968325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e58003a60 0x7f8e58003e50 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 -- 172.21.15.170:0/1616968325 >> 172.21.15.170:0/1616968325 conn(0x7f8e6c0d0660 msgr2=0x7f8e6c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 -- 172.21.15.170:0/1616968325 shutdown_connections 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.827+0000 7f8e7287f700 1 -- 172.21.15.170:0/1616968325 wait complete. 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.828+0000 7f8e7287f700 1 Processor -- start 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.828+0000 7f8e7287f700 1 -- start start 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58003a60 0x7f8e6c150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 0x7f8e6c16cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c15cb80 con 0x7f8e6c15bee0 2023-10-08T16:12:46.835 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c152c80 con 0x7f8e6c152810 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c15a0b0 con 0x7f8e58003a60 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32784/0 (socket says 172.21.15.170:32784) 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 -- 172.21.15.170:0/3558715988 learned_addr learned my addr 172.21.15.170:0/3558715988 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7207e700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 0x7f8e6c16cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7187d700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58003a60 0x7f8e6c150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 -- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58003a60 msgr2=0x7f8e6c150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58003a60 0x7f8e6c150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 -- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 msgr2=0x7f8e6c16cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 0x7f8e6c16cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 -- 172.21.15.170:0/3558715988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c150740 con 0x7f8e6c15bee0 2023-10-08T16:12:46.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7207e700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 0x7f8e6c16cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7107c700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e890 tx=0x7f8e5c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c019070 con 0x7f8e6c15bee0 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c006820 con 0x7f8e6c15bee0 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c006bc0 con 0x7f8e6c15bee0 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c151810 con 0x7f8e6c15bee0 2023-10-08T16:12:46.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.829+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c151dd0 con 0x7f8e6c15bee0 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.830+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c15a260 con 0x7f8e6c15bee0 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.831+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8e5c007020 con 0x7f8e6c15bee0 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.831+0000 7f8e61ffb700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 0x7f8e500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.831+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8e5c084e70 con 0x7f8e6c15bee0 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.831+0000 7f8e7187d700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 0x7f8e500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:46.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.831+0000 7f8e7187d700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 0x7f8e500697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400bb90 tx=0x7f8e6401b000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:46.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:46.835+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e6c15a260 con 0x7f8e6c15bee0 2023-10-08T16:12:47.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.058+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e6c0cd9d0 con 0x7f8e6c15bee0 2023-10-08T16:12:47.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.058+0000 7f8e61ffb700 1 -- 172.21.15.170:0/3558715988 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f8e5c051a00 con 0x7f8e6c15bee0 2023-10-08T16:12:47.060 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:47.060 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":109,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:46.691353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:12:47.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 msgr2=0x7f8e500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 0x7f8e500697d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400bb90 tx=0x7f8e6401b000 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 msgr2=0x7f8e6c1522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00e890 tx=0x7f8e5c009040 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 shutdown_connections 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8e6c152810 0x7f8e6c16cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8e58003a60 0x7f8e6c150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8e50067320 0x7f8e500697d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 --2- 172.21.15.170:0/3558715988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e6c15bee0 0x7f8e6c1522d0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 >> 172.21.15.170:0/3558715988 conn(0x7f8e6c0d0660 msgr2=0x7f8e6c0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 shutdown_connections 2023-10-08T16:12:47.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.061+0000 7f8e7287f700 1 -- 172.21.15.170:0/3558715988 wait complete. 2023-10-08T16:12:47.064 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 109 2023-10-08T16:12:47.074 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:47.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.188+0000 7f7163bd5700 1 Processor -- start 2023-10-08T16:12:47.189 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.188+0000 7f7163bd5700 1 -- start start 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7163bd5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f715c140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7163bd5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f715c0c9510 0x7f715c141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7163bd5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715c15c2a0 con 0x7f715c06c0a0 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7163bd5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c15b690 con 0x7f715c0c9510 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7163bd5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c15b970 con 0x7f715c06bcd0 2023-10-08T16:12:47.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7161170700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f715c06c0a0 0x7f715c0c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39422/0 (socket says 172.21.15.170:39422) 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7161170700 1 -- 172.21.15.170:0/1916155840 learned_addr learned my addr 172.21.15.170:0/1916155840 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7161971700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f715c140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245324932 0 0) 0x7f715c15c2a0 con 0x7f715c06c0a0 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7148003570 con 0x7f715c06c0a0 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7162172700 1 --2- 172.21.15.170:0/1916155840 >> v2:172.21.15.191:3300/0 conn(0x7f715c0c9510 0x7f715c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795182933 0 0) 0x7f7148003570 con 0x7f715c06c0a0 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 msgr2=0x7f715c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f715c140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 >> v2:172.21.15.191:3300/0 conn(0x7f715c0c9510 msgr2=0x7f715c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 --2- 172.21.15.170:0/1916155840 >> v2:172.21.15.191:3300/0 conn(0x7f715c0c9510 0x7f715c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c15bc50 con 0x7f715c06c0a0 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f714c001470 con 0x7f715c06c0a0 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 >> v1:172.21.15.170:6789/0 conn(0x7f715c06c0a0 legacy=0x7f715c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7148003a60 0x7f7148003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f7148004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7148004d50 0x7f71480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c15b690 con 0x7f7148004d50 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c15b970 con 0x7f7148003a60 2023-10-08T16:12:47.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714800ba60 con 0x7f715c06bcd0 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7162172700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7148004d50 0x7f71480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7161170700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f7148004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.189+0000 7f7161971700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7148003a60 0x7f7148003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 -- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 msgr2=0x7f7148004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c06bcd0 0x7f7148004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 -- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7148003a60 msgr2=0x7f7148003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7148003a60 0x7f7148003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 -- 172.21.15.170:0/1916155840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148003570 con 0x7f7148004d50 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7148004d50 0x7f71480051c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7154009c00 tx=0x7f71540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9e4fe8f2a8aa91 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e6b0 con 0x7f7148004d50 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400e810 con 0x7f7148004d50 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f716096f700 1 -- 172.21.15.170:0/1916155840 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400ebd0 con 0x7f7148004d50 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7163bd5700 1 -- 172.21.15.170:0/1916155840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7148004d50 msgr2=0x7f71480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7163bd5700 1 --2- 172.21.15.170:0/1916155840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7148004d50 0x7f71480051c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7154009c00 tx=0x7f71540091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.190+0000 7f7162172700 1 -- 172.21.15.170:0/1916155840 reap_dead start 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.191+0000 7f7163bd5700 1 -- 172.21.15.170:0/1916155840 shutdown_connections 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.191+0000 7f7163bd5700 1 -- 172.21.15.170:0/1916155840 >> 172.21.15.170:0/1916155840 conn(0x7f715c0ce600 msgr2=0x7f715c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.191+0000 7f7163bd5700 1 -- 172.21.15.170:0/1916155840 shutdown_connections 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.191+0000 7f7163bd5700 1 -- 172.21.15.170:0/1916155840 wait complete. 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 Processor -- start 2023-10-08T16:12:47.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 -- start start 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f715c13b0d0 0x7f715c0c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c0b3d20 0x7f715c0c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c1403e0 con 0x7f715c13b4a0 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0c6c90 con 0x7f715c13b0d0 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.192+0000 7f7163bd5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c0c6f70 con 0x7f715c0b3d20 2023-10-08T16:12:47.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32798/0 (socket says 172.21.15.170:32798) 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 -- 172.21.15.170:0/3274944141 learned_addr learned my addr 172.21.15.170:0/3274944141 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7161971700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f715c13b0d0 0x7f715c0c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7161170700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c0b3d20 0x7f715c0c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 -- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c0b3d20 msgr2=0x7f715c0c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c0b3d20 0x7f715c0c7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 -- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f715c13b0d0 msgr2=0x7f715c0c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f715c13b0d0 0x7f715c0c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 -- 172.21.15.170:0/3274944141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c0b13a0 con 0x7f715c13b4a0 2023-10-08T16:12:47.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7162172700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7148004730 tx=0x7f715401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154022070 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715400eb90 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715400e6b0 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c0b1680 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.193+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c167e70 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.194+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c04c3c0 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.195+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f71540279a0 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.195+0000 7f7151ffb700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 0x7f71400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.195+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7154086700 con 0x7f715c13b4a0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.195+0000 7f7161971700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 0x7f71400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.196+0000 7f7161971700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 0x7f71400697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f715c158e50 tx=0x7f715800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.199+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7154053470 con 0x7f715c13b4a0 2023-10-08T16:12:47.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.385+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f715c06c9d0 con 0x7f715c13b4a0 2023-10-08T16:12:47.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.386+0000 7f7151ffb700 1 -- 172.21.15.170:0/3274944141 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f7154056a90 con 0x7f715c13b4a0 2023-10-08T16:12:47.410 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:47.410 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":166,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:44.508366+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 msgr2=0x7f71400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 0x7f71400697d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f715c158e50 tx=0x7f715800b040 comp rx=0 tx=0).stop 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 msgr2=0x7f715c0c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f7148004730 tx=0x7f715401a000 comp rx=0 tx=0).stop 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 shutdown_connections 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7140067320 0x7f71400697d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f715c13b4a0 0x7f715c0c7d60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.412 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f715c13b0d0 0x7f715c0c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 --2- 172.21.15.170:0/3274944141 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f715c0b3d20 0x7f715c0c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 >> 172.21.15.170:0/3274944141 conn(0x7f715c0ce600 msgr2=0x7f715c140bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 shutdown_connections 2023-10-08T16:12:47.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.388+0000 7f7163bd5700 1 -- 172.21.15.170:0/3274944141 wait complete. 2023-10-08T16:12:47.413 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:12:47.414 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:12:47.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 Processor -- start 2023-10-08T16:12:47.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 -- start start 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6338159890 0x7f633815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f6338069560 0x7f63380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6338069ee0 con 0x7f6338159c60 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633815a160 con 0x7f6338159890 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f633d8e7700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380d13a0 con 0x7f6338069560 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6338159890 0x7f633815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.531 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6338159890 0x7f633815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32804/0 (socket says 172.21.15.170:32804) 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 -- 172.21.15.170:0/3428381635 learned_addr learned my addr 172.21.15.170:0/3428381635 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f63377fe700 1 --2- 172.21.15.170:0/3428381635 >> v2:172.21.15.191:3300/0 conn(0x7f6338069560 0x7f63380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 -- 172.21.15.170:0/3428381635 >> v2:172.21.15.191:3300/0 conn(0x7f6338069560 msgr2=0x7f63380699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 --2- 172.21.15.170:0/3428381635 >> v2:172.21.15.191:3300/0 conn(0x7f6338069560 0x7f63380699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 -- 172.21.15.170:0/3428381635 >> v1:172.21.15.191:6789/0 conn(0x7f6338159c60 legacy=0x7f633815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 -- 172.21.15.170:0/3428381635 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633815e140 con 0x7f6338159890 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6336ffd700 1 --2- 172.21.15.170:0/3428381635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338159890 0x7f633815abd0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f633806a890 tx=0x7f6328009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7410318bd60ab5c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.532 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6335ffb700 1 -- 172.21.15.170:0/3428381635 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328009e70 con 0x7f6338159890 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6335ffb700 1 -- 172.21.15.170:0/3428381635 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632800b860 con 0x7f6338159890 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.528+0000 7f6335ffb700 1 -- 172.21.15.170:0/3428381635 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800bc00 con 0x7f6338159890 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 -- 172.21.15.170:0/3428381635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338159890 msgr2=0x7f633815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 --2- 172.21.15.170:0/3428381635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338159890 0x7f633815abd0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f633806a890 tx=0x7f6328009250 comp rx=0 tx=0).stop 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 -- 172.21.15.170:0/3428381635 shutdown_connections 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 --2- 172.21.15.170:0/3428381635 >> v2:172.21.15.191:3300/0 conn(0x7f6338069560 0x7f63380699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 --2- 172.21.15.170:0/3428381635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338159890 0x7f633815abd0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 -- 172.21.15.170:0/3428381635 >> 172.21.15.170:0/3428381635 conn(0x7f6338068e20 msgr2=0x7f63380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 -- 172.21.15.170:0/3428381635 shutdown_connections 2023-10-08T16:12:47.534 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.529+0000 7f633d8e7700 1 -- 172.21.15.170:0/3428381635 wait complete. 2023-10-08T16:12:47.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.530+0000 7f633d8e7700 1 Processor -- start 2023-10-08T16:12:47.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.530+0000 7f633d8e7700 1 -- start start 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6338159890 0x7f63380b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6338159c60 0x7f633816d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b64c0 con 0x7f6338069560 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b4430 con 0x7f6338159c60 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f633d8e7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63380b4710 con 0x7f6338159890 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32820/0 (socket says 172.21.15.170:32820) 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 -- 172.21.15.170:0/1883058044 learned_addr learned my addr 172.21.15.170:0/1883058044 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:47.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 -- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6338159890 msgr2=0x7f63380b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f63377fe700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6338159c60 0x7f633816d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6338159890 0x7f63380b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 -- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6338159c60 msgr2=0x7f633816d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6338159c60 0x7f633816d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.531+0000 7f6336ffd700 1 -- 172.21.15.170:0/1883058044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280094f0 con 0x7f6338069560 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f6336ffd700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6338070e20 tx=0x7f6328017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011420 con 0x7f6338069560 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328011580 con 0x7f6338069560 2023-10-08T16:12:47.538 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63380b49f0 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011940 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.532+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63380b61b0 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.533+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633804c3c0 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.533+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f632802c3b0 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.533+0000 7f631effd700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 0x7f63240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.533+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6328086320 con 0x7f6338069560 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.534+0000 7f63367fc700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 0x7f63240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.534+0000 7f63367fc700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 0x7f63240697d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f63380cf150 tx=0x7f632c007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.538+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6328052fb0 con 0x7f6338069560 2023-10-08T16:12:47.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.716+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f633806a5b0 con 0x7f6324067320 2023-10-08T16:12:47.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.724+0000 7f631effd700 1 -- 172.21.15.170:0/1883058044 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289829 (secure 0 0 0) 0x7f633806a5b0 con 0x7f6324067320 2023-10-08T16:12:47.728 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 msgr2=0x7f63240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 0x7f63240697d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f63380cf150 tx=0x7f632c007b40 comp rx=0 tx=0).stop 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 msgr2=0x7f63380c7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f6338070e20 tx=0x7f6328017000 comp rx=0 tx=0).stop 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 shutdown_connections 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6338159c60 0x7f633816d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6324067320 0x7f63240697d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6338069560 0x7f63380c7c60 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 --2- 172.21.15.170:0/1883058044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6338159890 0x7f63380b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 >> 172.21.15.170:0/1883058044 conn(0x7f6338068e20 msgr2=0x7f63381534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 shutdown_connections 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.727+0000 7f633d8e7700 1 -- 172.21.15.170:0/1883058044 wait complete. 2023-10-08T16:12:47.732 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:12:47.741 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2023-10-08T16:12:46.323193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432744,"kb_used_data":17576,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541976,"statfs":{"total":773094113280,"available":770602983424,"internally_reserved":0,"allocated":17997824,"data_stored":8668580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.677654"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526827+0000","last_change":"2023-10-08T16:12:43.526827+0000","last_active":"2023-10-08T16:12:43.526827+0000","last_peered":"2023-10-08T16:12:43.526827+0000","last_clean":"2023-10-08T16:12:43.526827+0000","last_became_active":"2023-10-08T16:12:43.526346+0000","last_became_peered":"2023-10-08T16:12:43.526346+0000","last_unstale":"2023-10-08T16:12:43.526827+0000","last_undegraded":"2023-10-08T16:12:43.526827+0000","last_fullsized":"2023-10-08T16:12:43.526827+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:01.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531582+0000","last_change":"2023-10-08T16:12:43.531582+0000","last_active":"2023-10-08T16:12:43.531582+0000","last_peered":"2023-10-08T16:12:43.531582+0000","last_clean":"2023-10-08T16:12:43.531582+0000","last_became_active":"2023-10-08T16:12:43.531148+0000","last_became_peered":"2023-10-08T16:12:43.531148+0000","last_unstale":"2023-10-08T16:12:43.531582+0000","last_undegraded":"2023-10-08T16:12:43.531582+0000","last_fullsized":"2023-10-08T16:12:43.531582+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:24:55.246457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530593+0000","last_change":"2023-10-08T16:12:43.530593+0000","last_active":"2023-10-08T16:12:43.530593+0000","last_peered":"2023-10-08T16:12:43.530593+0000","last_clean":"2023-10-08T16:12:43.530593+0000","last_became_active":"2023-10-08T16:12:43.529979+0000","last_became_peered":"2023-10-08T16:12:43.529979+0000","last_unstale":"2023-10-08T16:12:43.530593+0000","last_undegraded":"2023-10-08T16:12:43.530593+0000","last_fullsized":"2023-10-08T16:12:43.530593+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:57.572729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522358+0000","last_change":"2023-10-08T16:12:43.522358+0000","last_active":"2023-10-08T16:12:43.522358+0000","last_peered":"2023-10-08T16:12:43.522358+0000","last_clean":"2023-10-08T16:12:43.522358+0000","last_became_active":"2023-10-08T16:12:43.522034+0000","last_became_peered":"2023-10-08T16:12:43.522034+0000","last_unstale":"2023-10-08T16:12:43.522358+0000","last_undegraded":"2023-10-08T16:12:43.522358+0000","last_fullsized":"2023-10-08T16:12:43.522358+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:48:19.482375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523128+0000","last_change":"2023-10-08T16:12:43.523128+0000","last_active":"2023-10-08T16:12:43.523128+0000","last_peered":"2023-10-08T16:12:43.523128+0000","last_clean":"2023-10-08T16:12:43.523128+0000","last_became_active":"2023-10-08T16:12:43.522828+0000","last_became_peered":"2023-10-08T16:12:43.522828+0000","last_unstale":"2023-10-08T16:12:43.523128+0000","last_undegraded":"2023-10-08T16:12:43.523128+0000","last_fullsized":"2023-10-08T16:12:43.523128+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:10.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526103+0000","last_change":"2023-10-08T16:12:43.526103+0000","last_active":"2023-10-08T16:12:43.526103+0000","last_peered":"2023-10-08T16:12:43.526103+0000","last_clean":"2023-10-08T16:12:43.526103+0000","last_became_active":"2023-10-08T16:12:43.525823+0000","last_became_peered":"2023-10-08T16:12:43.525823+0000","last_unstale":"2023-10-08T16:12:43.526103+0000","last_undegraded":"2023-10-08T16:12:43.526103+0000","last_fullsized":"2023-10-08T16:12:43.526103+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:48.744850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530164+0000","last_change":"2023-10-08T16:12:43.530164+0000","last_active":"2023-10-08T16:12:43.530164+0000","last_peered":"2023-10-08T16:12:43.530164+0000","last_clean":"2023-10-08T16:12:43.530164+0000","last_became_active":"2023-10-08T16:12:43.529842+0000","last_became_peered":"2023-10-08T16:12:43.529842+0000","last_unstale":"2023-10-08T16:12:43.530164+0000","last_undegraded":"2023-10-08T16:12:43.530164+0000","last_fullsized":"2023-10-08T16:12:43.530164+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:09.900526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.524584+0000","last_change":"2023-10-08T16:12:43.524584+0000","last_active":"2023-10-08T16:12:43.524584+0000","last_peered":"2023-10-08T16:12:43.524584+0000","last_clean":"2023-10-08T16:12:43.524584+0000","last_became_active":"2023-10-08T16:12:43.524332+0000","last_became_peered":"2023-10-08T16:12:43.524332+0000","last_unstale":"2023-10-08T16:12:43.524584+0000","last_undegraded":"2023-10-08T16:12:43.524584+0000","last_fullsized":"2023-10-08T16:12:43.524584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.990171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532670+0000","last_change":"2023-10-08T16:12:43.532670+0000","last_active":"2023-10-08T16:12:43.532670+0000","last_peered":"2023-10-08T16:12:43.532670+0000","last_clean":"2023-10-08T16:12:43.532670+0000","last_became_active":"2023-10-08T16:12:43.532400+0000","last_became_peered":"2023-10-08T16:12:43.532400+0000","last_unstale":"2023-10-08T16:12:43.532670+0000","last_undegraded":"2023-10-08T16:12:43.532670+0000","last_fullsized":"2023-10-08T16:12:43.532670+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:54.612519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525247+0000","last_change":"2023-10-08T16:12:43.525247+0000","last_active":"2023-10-08T16:12:43.525247+0000","last_peered":"2023-10-08T16:12:43.525247+0000","last_clean":"2023-10-08T16:12:43.525247+0000","last_became_active":"2023-10-08T16:12:43.524905+0000","last_became_peered":"2023-10-08T16:12:43.524905+0000","last_unstale":"2023-10-08T16:12:43.525247+0000","last_undegraded":"2023-10-08T16:12:43.525247+0000","last_fullsized":"2023-10-08T16:12:43.525247+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:13.060175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531008+0000","last_change":"2023-10-08T16:12:43.531008+0000","last_active":"2023-10-08T16:12:43.531008+0000","last_peered":"2023-10-08T16:12:43.531008+0000","last_clean":"2023-10-08T16:12:43.531008+0000","last_became_active":"2023-10-08T16:12:43.530738+0000","last_became_peered":"2023-10-08T16:12:43.530738+0000","last_unstale":"2023-10-08T16:12:43.531008+0000","last_undegraded":"2023-10-08T16:12:43.531008+0000","last_fullsized":"2023-10-08T16:12:43.531008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:40.661491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520729+0000","last_change":"2023-10-08T16:12:43.520729+0000","last_active":"2023-10-08T16:12:43.520729+0000","last_peered":"2023-10-08T16:12:43.520729+0000","last_clean":"2023-10-08T16:12:43.520729+0000","last_became_active":"2023-10-08T16:12:43.520367+0000","last_became_peered":"2023-10-08T16:12:43.520367+0000","last_unstale":"2023-10-08T16:12:43.520729+0000","last_undegraded":"2023-10-08T16:12:43.520729+0000","last_fullsized":"2023-10-08T16:12:43.520729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:59.939112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526071+0000","last_change":"2023-10-08T16:12:43.526071+0000","last_active":"2023-10-08T16:12:43.526071+0000","last_peered":"2023-10-08T16:12:43.526071+0000","last_clean":"2023-10-08T16:12:43.526071+0000","last_became_active":"2023-10-08T16:12:43.525778+0000","last_became_peered":"2023-10-08T16:12:43.525778+0000","last_unstale":"2023-10-08T16:12:43.526071+0000","last_undegraded":"2023-10-08T16:12:43.526071+0000","last_fullsized":"2023-10-08T16:12:43.526071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:29.142447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525129+0000","last_change":"2023-10-08T16:12:43.525129+0000","last_active":"2023-10-08T16:12:43.525129+0000","last_peered":"2023-10-08T16:12:43.525129+0000","last_clean":"2023-10-08T16:12:43.525129+0000","last_became_active":"2023-10-08T16:12:43.524775+0000","last_became_peered":"2023-10-08T16:12:43.524775+0000","last_unstale":"2023-10-08T16:12:43.525129+0000","last_undegraded":"2023-10-08T16:12:43.525129+0000","last_fullsized":"2023-10-08T16:12:43.525129+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:16.888336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530335+0000","last_change":"2023-10-08T16:12:43.530335+0000","last_active":"2023-10-08T16:12:43.530335+0000","last_peered":"2023-10-08T16:12:43.530335+0000","last_clean":"2023-10-08T16:12:43.530335+0000","last_became_active":"2023-10-08T16:12:43.530140+0000","last_became_peered":"2023-10-08T16:12:43.530140+0000","last_unstale":"2023-10-08T16:12:43.530335+0000","last_undegraded":"2023-10-08T16:12:43.530335+0000","last_fullsized":"2023-10-08T16:12:43.530335+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:52.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517213+0000","last_change":"2023-10-08T16:12:43.517213+0000","last_active":"2023-10-08T16:12:43.517213+0000","last_peered":"2023-10-08T16:12:43.517213+0000","last_clean":"2023-10-08T16:12:43.517213+0000","last_became_active":"2023-10-08T16:12:43.516887+0000","last_became_peered":"2023-10-08T16:12:43.516887+0000","last_unstale":"2023-10-08T16:12:43.517213+0000","last_undegraded":"2023-10-08T16:12:43.517213+0000","last_fullsized":"2023-10-08T16:12:43.517213+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:54.612519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528542+0000","last_change":"2023-10-08T16:12:43.528542+0000","last_active":"2023-10-08T16:12:43.528542+0000","last_peered":"2023-10-08T16:12:43.528542+0000","last_clean":"2023-10-08T16:12:43.528542+0000","last_became_active":"2023-10-08T16:12:43.525304+0000","last_became_peered":"2023-10-08T16:12:43.525304+0000","last_unstale":"2023-10-08T16:12:43.528542+0000","last_undegraded":"2023-10-08T16:12:43.528542+0000","last_fullsized":"2023-10-08T16:12:43.528542+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:09.900526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534209+0000","last_change":"2023-10-08T16:12:43.534209+0000","last_active":"2023-10-08T16:12:43.534209+0000","last_peered":"2023-10-08T16:12:43.534209+0000","last_clean":"2023-10-08T16:12:43.534209+0000","last_became_active":"2023-10-08T16:12:43.533980+0000","last_became_peered":"2023-10-08T16:12:43.533980+0000","last_unstale":"2023-10-08T16:12:43.534209+0000","last_undegraded":"2023-10-08T16:12:43.534209+0000","last_fullsized":"2023-10-08T16:12:43.534209+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:49.031323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527750+0000","last_change":"2023-10-08T16:12:43.527750+0000","last_active":"2023-10-08T16:12:43.527750+0000","last_peered":"2023-10-08T16:12:43.527750+0000","last_clean":"2023-10-08T16:12:43.527750+0000","last_became_active":"2023-10-08T16:12:43.526754+0000","last_became_peered":"2023-10-08T16:12:43.526754+0000","last_unstale":"2023-10-08T16:12:43.527750+0000","last_undegraded":"2023-10-08T16:12:43.527750+0000","last_fullsized":"2023-10-08T16:12:43.527750+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:40.742811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521700+0000","last_change":"2023-10-08T16:12:43.521700+0000","last_active":"2023-10-08T16:12:43.521700+0000","last_peered":"2023-10-08T16:12:43.521700+0000","last_clean":"2023-10-08T16:12:43.521700+0000","last_became_active":"2023-10-08T16:12:43.520410+0000","last_became_peered":"2023-10-08T16:12:43.520410+0000","last_unstale":"2023-10-08T16:12:43.521700+0000","last_undegraded":"2023-10-08T16:12:43.521700+0000","last_fullsized":"2023-10-08T16:12:43.521700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:21.529765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525895+0000","last_change":"2023-10-08T16:12:43.525895+0000","last_active":"2023-10-08T16:12:43.525895+0000","last_peered":"2023-10-08T16:12:43.525895+0000","last_clean":"2023-10-08T16:12:43.525895+0000","last_became_active":"2023-10-08T16:12:43.525245+0000","last_became_peered":"2023-10-08T16:12:43.525245+0000","last_unstale":"2023-10-08T16:12:43.525895+0000","last_undegraded":"2023-10-08T16:12:43.525895+0000","last_fullsized":"2023-10-08T16:12:43.525895+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:43.196609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522534+0000","last_change":"2023-10-08T16:12:43.522534+0000","last_active":"2023-10-08T16:12:43.522534+0000","last_peered":"2023-10-08T16:12:43.522534+0000","last_clean":"2023-10-08T16:12:43.522534+0000","last_became_active":"2023-10-08T16:12:43.522269+0000","last_became_peered":"2023-10-08T16:12:43.522269+0000","last_unstale":"2023-10-08T16:12:43.522534+0000","last_undegraded":"2023-10-08T16:12:43.522534+0000","last_fullsized":"2023-10-08T16:12:43.522534+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:07.697011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.524887+0000","last_change":"2023-10-08T16:12:43.524887+0000","last_active":"2023-10-08T16:12:43.524887+0000","last_peered":"2023-10-08T16:12:43.524887+0000","last_clean":"2023-10-08T16:12:43.524887+0000","last_became_active":"2023-10-08T16:12:43.523971+0000","last_became_peered":"2023-10-08T16:12:43.523971+0000","last_unstale":"2023-10-08T16:12:43.524887+0000","last_undegraded":"2023-10-08T16:12:43.524887+0000","last_fullsized":"2023-10-08T16:12:43.524887+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:01.959857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521231+0000","last_change":"2023-10-08T16:12:43.521231+0000","last_active":"2023-10-08T16:12:43.521231+0000","last_peered":"2023-10-08T16:12:43.521231+0000","last_clean":"2023-10-08T16:12:43.521231+0000","last_became_active":"2023-10-08T16:12:43.520996+0000","last_became_peered":"2023-10-08T16:12:43.520996+0000","last_unstale":"2023-10-08T16:12:43.521231+0000","last_undegraded":"2023-10-08T16:12:43.521231+0000","last_fullsized":"2023-10-08T16:12:43.521231+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:54.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526888+0000","last_change":"2023-10-08T16:12:43.526888+0000","last_active":"2023-10-08T16:12:43.526888+0000","last_peered":"2023-10-08T16:12:43.526888+0000","last_clean":"2023-10-08T16:12:43.526888+0000","last_became_active":"2023-10-08T16:12:43.526562+0000","last_became_peered":"2023-10-08T16:12:43.526562+0000","last_unstale":"2023-10-08T16:12:43.526888+0000","last_undegraded":"2023-10-08T16:12:43.526888+0000","last_fullsized":"2023-10-08T16:12:43.526888+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:49.031323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527829+0000","last_change":"2023-10-08T16:12:43.527829+0000","last_active":"2023-10-08T16:12:43.527829+0000","last_peered":"2023-10-08T16:12:43.527829+0000","last_clean":"2023-10-08T16:12:43.527829+0000","last_became_active":"2023-10-08T16:12:43.527589+0000","last_became_peered":"2023-10-08T16:12:43.527589+0000","last_unstale":"2023-10-08T16:12:43.527829+0000","last_undegraded":"2023-10-08T16:12:43.527829+0000","last_fullsized":"2023-10-08T16:12:43.527829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:40.856967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530215+0000","last_change":"2023-10-08T16:12:43.530215+0000","last_active":"2023-10-08T16:12:43.530215+0000","last_peered":"2023-10-08T16:12:43.530215+0000","last_clean":"2023-10-08T16:12:43.530215+0000","last_became_active":"2023-10-08T16:12:43.529814+0000","last_became_peered":"2023-10-08T16:12:43.529814+0000","last_unstale":"2023-10-08T16:12:43.530215+0000","last_undegraded":"2023-10-08T16:12:43.530215+0000","last_fullsized":"2023-10-08T16:12:43.530215+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:13.060175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.519554+0000","last_change":"2023-10-08T16:12:43.519554+0000","last_active":"2023-10-08T16:12:43.519554+0000","last_peered":"2023-10-08T16:12:43.519554+0000","last_clean":"2023-10-08T16:12:43.519554+0000","last_became_active":"2023-10-08T16:12:43.519288+0000","last_became_peered":"2023-10-08T16:12:43.519288+0000","last_unstale":"2023-10-08T16:12:43.519554+0000","last_undegraded":"2023-10-08T16:12:43.519554+0000","last_fullsized":"2023-10-08T16:12:43.519554+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:31.737355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531956+0000","last_change":"2023-10-08T16:12:43.531956+0000","last_active":"2023-10-08T16:12:43.531956+0000","last_peered":"2023-10-08T16:12:43.531956+0000","last_clean":"2023-10-08T16:12:43.531956+0000","last_became_active":"2023-10-08T16:12:43.531694+0000","last_became_peered":"2023-10-08T16:12:43.531694+0000","last_unstale":"2023-10-08T16:12:43.531956+0000","last_undegraded":"2023-10-08T16:12:43.531956+0000","last_fullsized":"2023-10-08T16:12:43.531956+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:57.572729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531981+0000","last_change":"2023-10-08T16:12:43.531981+0000","last_active":"2023-10-08T16:12:43.531981+0000","last_peered":"2023-10-08T16:12:43.531981+0000","last_clean":"2023-10-08T16:12:43.531981+0000","last_became_active":"2023-10-08T16:12:43.531716+0000","last_became_peered":"2023-10-08T16:12:43.531716+0000","last_unstale":"2023-10-08T16:12:43.531981+0000","last_undegraded":"2023-10-08T16:12:43.531981+0000","last_fullsized":"2023-10-08T16:12:43.531981+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:07.697011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.519027+0000","last_change":"2023-10-08T16:12:43.519027+0000","last_active":"2023-10-08T16:12:43.519027+0000","last_peered":"2023-10-08T16:12:43.519027+0000","last_clean":"2023-10-08T16:12:43.519027+0000","last_became_active":"2023-10-08T16:12:43.518809+0000","last_became_peered":"2023-10-08T16:12:43.518809+0000","last_unstale":"2023-10-08T16:12:43.519027+0000","last_undegraded":"2023-10-08T16:12:43.519027+0000","last_fullsized":"2023-10-08T16:12:43.519027+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:38:23.977868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531682+0000","last_change":"2023-10-08T16:12:43.531682+0000","last_active":"2023-10-08T16:12:43.531682+0000","last_peered":"2023-10-08T16:12:43.531682+0000","last_clean":"2023-10-08T16:12:43.531682+0000","last_became_active":"2023-10-08T16:12:43.531415+0000","last_became_peered":"2023-10-08T16:12:43.531415+0000","last_unstale":"2023-10-08T16:12:43.531682+0000","last_undegraded":"2023-10-08T16:12:43.531682+0000","last_fullsized":"2023-10-08T16:12:43.531682+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:01.959857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532478+0000","last_change":"2023-10-08T16:12:43.532478+0000","last_active":"2023-10-08T16:12:43.532478+0000","last_peered":"2023-10-08T16:12:43.532478+0000","last_clean":"2023-10-08T16:12:43.532478+0000","last_became_active":"2023-10-08T16:12:43.531927+0000","last_became_peered":"2023-10-08T16:12:43.531927+0000","last_unstale":"2023-10-08T16:12:43.532478+0000","last_undegraded":"2023-10-08T16:12:43.532478+0000","last_fullsized":"2023-10-08T16:12:43.532478+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:27.500513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"165'4","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683093+0000","last_change":"2023-10-08T16:12:42.669847+0000","last_active":"2023-10-08T16:12:43.683093+0000","last_peered":"2023-10-08T16:12:43.683093+0000","last_clean":"2023-10-08T16:12:43.683093+0000","last_became_active":"2023-10-08T16:12:42.669571+0000","last_became_peered":"2023-10-08T16:12:42.669571+0000","last_unstale":"2023-10-08T16:12:43.683093+0000","last_undegraded":"2023-10-08T16:12:43.683093+0000","last_fullsized":"2023-10-08T16:12:43.683093+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:48.029681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527753+0000","last_change":"2023-10-08T16:12:43.527753+0000","last_active":"2023-10-08T16:12:43.527753+0000","last_peered":"2023-10-08T16:12:43.527753+0000","last_clean":"2023-10-08T16:12:43.527753+0000","last_became_active":"2023-10-08T16:12:43.527181+0000","last_became_peered":"2023-10-08T16:12:43.527181+0000","last_unstale":"2023-10-08T16:12:43.527753+0000","last_undegraded":"2023-10-08T16:12:43.527753+0000","last_fullsized":"2023-10-08T16:12:43.527753+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:13.262906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517083+0000","last_change":"2023-10-08T16:12:42.663668+0000","last_active":"2023-10-08T16:12:43.517083+0000","last_peered":"2023-10-08T16:12:43.517083+0000","last_clean":"2023-10-08T16:12:43.517083+0000","last_became_active":"2023-10-08T16:12:42.663363+0000","last_became_peered":"2023-10-08T16:12:42.663363+0000","last_unstale":"2023-10-08T16:12:43.517083+0000","last_undegraded":"2023-10-08T16:12:43.517083+0000","last_fullsized":"2023-10-08T16:12:43.517083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:08.898885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534188+0000","last_change":"2023-10-08T16:12:43.534188+0000","last_active":"2023-10-08T16:12:43.534188+0000","last_peered":"2023-10-08T16:12:43.534188+0000","last_clean":"2023-10-08T16:12:43.534188+0000","last_became_active":"2023-10-08T16:12:43.532957+0000","last_became_peered":"2023-10-08T16:12:43.532957+0000","last_unstale":"2023-10-08T16:12:43.534188+0000","last_undegraded":"2023-10-08T16:12:43.534188+0000","last_fullsized":"2023-10-08T16:12:43.534188+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:52.844283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.678842+0000","last_change":"2023-10-08T16:12:42.656853+0000","last_active":"2023-10-08T16:12:43.678842+0000","last_peered":"2023-10-08T16:12:43.678842+0000","last_clean":"2023-10-08T16:12:43.678842+0000","last_became_active":"2023-10-08T16:12:42.656571+0000","last_became_peered":"2023-10-08T16:12:42.656571+0000","last_unstale":"2023-10-08T16:12:43.678842+0000","last_undegraded":"2023-10-08T16:12:43.678842+0000","last_fullsized":"2023-10-08T16:12:43.678842+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:15.886695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.535194+0000","last_change":"2023-10-08T16:12:43.535194+0000","last_active":"2023-10-08T16:12:43.535194+0000","last_peered":"2023-10-08T16:12:43.535194+0000","last_clean":"2023-10-08T16:12:43.535194+0000","last_became_active":"2023-10-08T16:12:43.534875+0000","last_became_peered":"2023-10-08T16:12:43.534875+0000","last_unstale":"2023-10-08T16:12:43.535194+0000","last_undegraded":"2023-10-08T16:12:43.535194+0000","last_fullsized":"2023-10-08T16:12:43.535194+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:10:47.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517363+0000","last_change":"2023-10-08T16:12:42.665562+0000","last_active":"2023-10-08T16:12:43.517363+0000","last_peered":"2023-10-08T16:12:43.517363+0000","last_clean":"2023-10-08T16:12:43.517363+0000","last_became_active":"2023-10-08T16:12:42.665310+0000","last_became_peered":"2023-10-08T16:12:42.665310+0000","last_unstale":"2023-10-08T16:12:43.517363+0000","last_undegraded":"2023-10-08T16:12:43.517363+0000","last_fullsized":"2023-10-08T16:12:43.517363+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:06:50.209391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528891+0000","last_change":"2023-10-08T16:12:43.528891+0000","last_active":"2023-10-08T16:12:43.528891+0000","last_peered":"2023-10-08T16:12:43.528891+0000","last_clean":"2023-10-08T16:12:43.528891+0000","last_became_active":"2023-10-08T16:12:43.528650+0000","last_became_peered":"2023-10-08T16:12:43.528650+0000","last_unstale":"2023-10-08T16:12:43.528891+0000","last_undegraded":"2023-10-08T16:12:43.528891+0000","last_fullsized":"2023-10-08T16:12:43.528891+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:37.684541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527588+0000","last_change":"2023-10-08T16:12:42.666801+0000","last_active":"2023-10-08T16:12:43.527588+0000","last_peered":"2023-10-08T16:12:43.527588+0000","last_clean":"2023-10-08T16:12:43.527588+0000","last_became_active":"2023-10-08T16:12:42.666522+0000","last_became_peered":"2023-10-08T16:12:42.666522+0000","last_unstale":"2023-10-08T16:12:43.527588+0000","last_undegraded":"2023-10-08T16:12:43.527588+0000","last_fullsized":"2023-10-08T16:12:43.527588+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:20.232268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521135+0000","last_change":"2023-10-08T16:12:43.521135+0000","last_active":"2023-10-08T16:12:43.521135+0000","last_peered":"2023-10-08T16:12:43.521135+0000","last_clean":"2023-10-08T16:12:43.521135+0000","last_became_active":"2023-10-08T16:12:43.520879+0000","last_became_peered":"2023-10-08T16:12:43.520879+0000","last_unstale":"2023-10-08T16:12:43.521135+0000","last_undegraded":"2023-10-08T16:12:43.521135+0000","last_fullsized":"2023-10-08T16:12:43.521135+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:36.777496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670317+0000","last_change":"2023-10-08T16:12:42.668295+0000","last_active":"2023-10-08T16:12:42.670317+0000","last_peered":"2023-10-08T16:12:42.670317+0000","last_clean":"2023-10-08T16:12:42.670317+0000","last_became_active":"2023-10-08T16:12:42.667998+0000","last_became_peered":"2023-10-08T16:12:42.667998+0000","last_unstale":"2023-10-08T16:12:42.670317+0000","last_undegraded":"2023-10-08T16:12:42.670317+0000","last_fullsized":"2023-10-08T16:12:42.670317+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530179+0000","last_change":"2023-10-08T16:12:43.530179+0000","last_active":"2023-10-08T16:12:43.530179+0000","last_peered":"2023-10-08T16:12:43.530179+0000","last_clean":"2023-10-08T16:12:43.530179+0000","last_became_active":"2023-10-08T16:12:43.529897+0000","last_became_peered":"2023-10-08T16:12:43.529897+0000","last_unstale":"2023-10-08T16:12:43.530179+0000","last_undegraded":"2023-10-08T16:12:43.530179+0000","last_fullsized":"2023-10-08T16:12:43.530179+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:01.776870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664547+0000","last_change":"2023-10-08T16:12:42.663795+0000","last_active":"2023-10-08T16:12:42.664547+0000","last_peered":"2023-10-08T16:12:42.664547+0000","last_clean":"2023-10-08T16:12:42.664547+0000","last_became_active":"2023-10-08T16:12:42.663529+0000","last_became_peered":"2023-10-08T16:12:42.663529+0000","last_unstale":"2023-10-08T16:12:42.664547+0000","last_undegraded":"2023-10-08T16:12:42.664547+0000","last_fullsized":"2023-10-08T16:12:42.664547+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:09.739973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531393+0000","last_change":"2023-10-08T16:12:43.531393+0000","last_active":"2023-10-08T16:12:43.531393+0000","last_peered":"2023-10-08T16:12:43.531393+0000","last_clean":"2023-10-08T16:12:43.531393+0000","last_became_active":"2023-10-08T16:12:43.530826+0000","last_became_peered":"2023-10-08T16:12:43.530826+0000","last_unstale":"2023-10-08T16:12:43.531393+0000","last_undegraded":"2023-10-08T16:12:43.531393+0000","last_fullsized":"2023-10-08T16:12:43.531393+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:14:16.673225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672549+0000","last_change":"2023-10-08T16:12:42.668796+0000","last_active":"2023-10-08T16:12:42.672549+0000","last_peered":"2023-10-08T16:12:42.672549+0000","last_clean":"2023-10-08T16:12:42.672549+0000","last_became_active":"2023-10-08T16:12:42.668432+0000","last_became_peered":"2023-10-08T16:12:42.668432+0000","last_unstale":"2023-10-08T16:12:42.672549+0000","last_undegraded":"2023-10-08T16:12:42.672549+0000","last_fullsized":"2023-10-08T16:12:42.672549+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:54:37.903831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526946+0000","last_change":"2023-10-08T16:12:43.526946+0000","last_active":"2023-10-08T16:12:43.526946+0000","last_peered":"2023-10-08T16:12:43.526946+0000","last_clean":"2023-10-08T16:12:43.526946+0000","last_became_active":"2023-10-08T16:12:43.526588+0000","last_became_peered":"2023-10-08T16:12:43.526588+0000","last_unstale":"2023-10-08T16:12:43.526946+0000","last_undegraded":"2023-10-08T16:12:43.526946+0000","last_fullsized":"2023-10-08T16:12:43.526946+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:48.744850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664070+0000","last_change":"2023-10-08T16:12:42.663411+0000","last_active":"2023-10-08T16:12:42.664070+0000","last_peered":"2023-10-08T16:12:42.664070+0000","last_clean":"2023-10-08T16:12:42.664070+0000","last_became_active":"2023-10-08T16:12:42.663160+0000","last_became_peered":"2023-10-08T16:12:42.663160+0000","last_unstale":"2023-10-08T16:12:42.664070+0000","last_undegraded":"2023-10-08T16:12:42.664070+0000","last_fullsized":"2023-10-08T16:12:42.664070+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:30.735713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532327+0000","last_change":"2023-10-08T16:12:43.532327+0000","last_active":"2023-10-08T16:12:43.532327+0000","last_peered":"2023-10-08T16:12:43.532327+0000","last_clean":"2023-10-08T16:12:43.532327+0000","last_became_active":"2023-10-08T16:12:43.531889+0000","last_became_peered":"2023-10-08T16:12:43.531889+0000","last_unstale":"2023-10-08T16:12:43.532327+0000","last_undegraded":"2023-10-08T16:12:43.532327+0000","last_fullsized":"2023-10-08T16:12:43.532327+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.137973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"165'3","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.682795+0000","last_change":"2023-10-08T16:12:42.661354+0000","last_active":"2023-10-08T16:12:43.682795+0000","last_peered":"2023-10-08T16:12:43.682795+0000","last_clean":"2023-10-08T16:12:43.682795+0000","last_became_active":"2023-10-08T16:12:42.661094+0000","last_became_peered":"2023-10-08T16:12:42.661094+0000","last_unstale":"2023-10-08T16:12:43.682795+0000","last_undegraded":"2023-10-08T16:12:43.682795+0000","last_fullsized":"2023-10-08T16:12:43.682795+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:27.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534289+0000","last_change":"2023-10-08T16:12:43.534289+0000","last_active":"2023-10-08T16:12:43.534289+0000","last_peered":"2023-10-08T16:12:43.534289+0000","last_clean":"2023-10-08T16:12:43.534289+0000","last_became_active":"2023-10-08T16:12:43.534037+0000","last_became_peered":"2023-10-08T16:12:43.534037+0000","last_unstale":"2023-10-08T16:12:43.534289+0000","last_undegraded":"2023-10-08T16:12:43.534289+0000","last_fullsized":"2023-10-08T16:12:43.534289+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:31.737355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"165'6","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.679488+0000","last_change":"2023-10-08T16:12:42.662964+0000","last_active":"2023-10-08T16:12:43.679488+0000","last_peered":"2023-10-08T16:12:43.679488+0000","last_clean":"2023-10-08T16:12:43.679488+0000","last_became_active":"2023-10-08T16:12:42.662725+0000","last_became_peered":"2023-10-08T16:12:42.662725+0000","last_unstale":"2023-10-08T16:12:43.679488+0000","last_undegraded":"2023-10-08T16:12:43.679488+0000","last_fullsized":"2023-10-08T16:12:43.679488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:16:42.535018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525485+0000","last_change":"2023-10-08T16:12:43.525485+0000","last_active":"2023-10-08T16:12:43.525485+0000","last_peered":"2023-10-08T16:12:43.525485+0000","last_clean":"2023-10-08T16:12:43.525485+0000","last_became_active":"2023-10-08T16:12:43.525220+0000","last_became_peered":"2023-10-08T16:12:43.525220+0000","last_unstale":"2023-10-08T16:12:43.525485+0000","last_undegraded":"2023-10-08T16:12:43.525485+0000","last_fullsized":"2023-10-08T16:12:43.525485+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:46.674446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.680970+0000","last_change":"2023-10-08T16:12:42.668813+0000","last_active":"2023-10-08T16:12:43.680970+0000","last_peered":"2023-10-08T16:12:43.680970+0000","last_clean":"2023-10-08T16:12:43.680970+0000","last_became_active":"2023-10-08T16:12:42.668439+0000","last_became_peered":"2023-10-08T16:12:42.668439+0000","last_unstale":"2023-10-08T16:12:43.680970+0000","last_undegraded":"2023-10-08T16:12:43.680970+0000","last_fullsized":"2023-10-08T16:12:43.680970+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:36.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528823+0000","last_change":"2023-10-08T16:12:43.528823+0000","last_active":"2023-10-08T16:12:43.528823+0000","last_peered":"2023-10-08T16:12:43.528823+0000","last_clean":"2023-10-08T16:12:43.528823+0000","last_became_active":"2023-10-08T16:12:43.528532+0000","last_became_peered":"2023-10-08T16:12:43.528532+0000","last_unstale":"2023-10-08T16:12:43.528823+0000","last_undegraded":"2023-10-08T16:12:43.528823+0000","last_fullsized":"2023-10-08T16:12:43.528823+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:56.730218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664094+0000","last_change":"2023-10-08T16:12:42.663427+0000","last_active":"2023-10-08T16:12:42.664094+0000","last_peered":"2023-10-08T16:12:42.664094+0000","last_clean":"2023-10-08T16:12:42.664094+0000","last_became_active":"2023-10-08T16:12:42.663195+0000","last_became_peered":"2023-10-08T16:12:42.663195+0000","last_unstale":"2023-10-08T16:12:42.664094+0000","last_undegraded":"2023-10-08T16:12:42.664094+0000","last_fullsized":"2023-10-08T16:12:42.664094+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:47.743209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.529082+0000","last_change":"2023-10-08T16:12:43.529082+0000","last_active":"2023-10-08T16:12:43.529082+0000","last_peered":"2023-10-08T16:12:43.529082+0000","last_clean":"2023-10-08T16:12:43.529082+0000","last_became_active":"2023-10-08T16:12:43.528805+0000","last_became_peered":"2023-10-08T16:12:43.528805+0000","last_unstale":"2023-10-08T16:12:43.529082+0000","last_undegraded":"2023-10-08T16:12:43.529082+0000","last_fullsized":"2023-10-08T16:12:43.529082+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:13.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664020+0000","last_change":"2023-10-08T16:12:42.662469+0000","last_active":"2023-10-08T16:12:42.664020+0000","last_peered":"2023-10-08T16:12:42.664020+0000","last_clean":"2023-10-08T16:12:42.664020+0000","last_became_active":"2023-10-08T16:12:42.662212+0000","last_became_peered":"2023-10-08T16:12:42.662212+0000","last_unstale":"2023-10-08T16:12:42.664020+0000","last_undegraded":"2023-10-08T16:12:42.664020+0000","last_fullsized":"2023-10-08T16:12:42.664020+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:26.498871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527406+0000","last_change":"2023-10-08T16:12:43.527406+0000","last_active":"2023-10-08T16:12:43.527406+0000","last_peered":"2023-10-08T16:12:43.527406+0000","last_clean":"2023-10-08T16:12:43.527406+0000","last_became_active":"2023-10-08T16:12:43.525012+0000","last_became_peered":"2023-10-08T16:12:43.525012+0000","last_unstale":"2023-10-08T16:12:43.527406+0000","last_undegraded":"2023-10-08T16:12:43.527406+0000","last_fullsized":"2023-10-08T16:12:43.527406+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:16.572084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"165'8","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683113+0000","last_change":"2023-10-08T16:12:42.664293+0000","last_active":"2023-10-08T16:12:43.683113+0000","last_peered":"2023-10-08T16:12:43.683113+0000","last_clean":"2023-10-08T16:12:43.683113+0000","last_became_active":"2023-10-08T16:12:42.662935+0000","last_became_peered":"2023-10-08T16:12:42.662935+0000","last_unstale":"2023-10-08T16:12:43.683113+0000","last_undegraded":"2023-10-08T16:12:43.683113+0000","last_fullsized":"2023-10-08T16:12:43.683113+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:12.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523804+0000","last_change":"2023-10-08T16:12:43.523804+0000","last_active":"2023-10-08T16:12:43.523804+0000","last_peered":"2023-10-08T16:12:43.523804+0000","last_clean":"2023-10-08T16:12:43.523804+0000","last_became_active":"2023-10-08T16:12:43.523538+0000","last_became_peered":"2023-10-08T16:12:43.523538+0000","last_unstale":"2023-10-08T16:12:43.523804+0000","last_undegraded":"2023-10-08T16:12:43.523804+0000","last_fullsized":"2023-10-08T16:12:43.523804+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:12.788019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527791+0000","last_change":"2023-10-08T16:12:42.669802+0000","last_active":"2023-10-08T16:12:43.527791+0000","last_peered":"2023-10-08T16:12:43.527791+0000","last_clean":"2023-10-08T16:12:43.527791+0000","last_became_active":"2023-10-08T16:12:42.669512+0000","last_became_peered":"2023-10-08T16:12:42.669512+0000","last_unstale":"2023-10-08T16:12:43.527791+0000","last_undegraded":"2023-10-08T16:12:43.527791+0000","last_fullsized":"2023-10-08T16:12:43.527791+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:10:08.755528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534206+0000","last_change":"2023-10-08T16:12:43.534206+0000","last_active":"2023-10-08T16:12:43.534206+0000","last_peered":"2023-10-08T16:12:43.534206+0000","last_clean":"2023-10-08T16:12:43.534206+0000","last_became_active":"2023-10-08T16:12:43.533960+0000","last_became_peered":"2023-10-08T16:12:43.533960+0000","last_unstale":"2023-10-08T16:12:43.534206+0000","last_undegraded":"2023-10-08T16:12:43.534206+0000","last_fullsized":"2023-10-08T16:12:43.534206+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:01.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.680397+0000","last_change":"2023-10-08T16:12:42.666968+0000","last_active":"2023-10-08T16:12:43.680397+0000","last_peered":"2023-10-08T16:12:43.680397+0000","last_clean":"2023-10-08T16:12:43.680397+0000","last_became_active":"2023-10-08T16:12:42.666717+0000","last_became_peered":"2023-10-08T16:12:42.666717+0000","last_unstale":"2023-10-08T16:12:43.680397+0000","last_undegraded":"2023-10-08T16:12:43.680397+0000","last_fullsized":"2023-10-08T16:12:43.680397+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:46.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531359+0000","last_change":"2023-10-08T16:12:43.531359+0000","last_active":"2023-10-08T16:12:43.531359+0000","last_peered":"2023-10-08T16:12:43.531359+0000","last_clean":"2023-10-08T16:12:43.531359+0000","last_became_active":"2023-10-08T16:12:43.530989+0000","last_became_peered":"2023-10-08T16:12:43.530989+0000","last_unstale":"2023-10-08T16:12:43.531359+0000","last_undegraded":"2023-10-08T16:12:43.531359+0000","last_fullsized":"2023-10-08T16:12:43.531359+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:11:03.905560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520838+0000","last_change":"2023-10-08T16:12:42.669007+0000","last_active":"2023-10-08T16:12:43.520838+0000","last_peered":"2023-10-08T16:12:43.520838+0000","last_clean":"2023-10-08T16:12:43.520838+0000","last_became_active":"2023-10-08T16:12:42.668333+0000","last_became_peered":"2023-10-08T16:12:42.668333+0000","last_unstale":"2023-10-08T16:12:43.520838+0000","last_undegraded":"2023-10-08T16:12:43.520838+0000","last_fullsized":"2023-10-08T16:12:43.520838+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:15.661851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530650+0000","last_change":"2023-10-08T16:12:43.530650+0000","last_active":"2023-10-08T16:12:43.530650+0000","last_peered":"2023-10-08T16:12:43.530650+0000","last_clean":"2023-10-08T16:12:43.530650+0000","last_became_active":"2023-10-08T16:12:43.530260+0000","last_became_peered":"2023-10-08T16:12:43.530260+0000","last_unstale":"2023-10-08T16:12:43.530650+0000","last_undegraded":"2023-10-08T16:12:43.530650+0000","last_fullsized":"2023-10-08T16:12:43.530650+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:16.888336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.668176+0000","last_change":"2023-10-08T16:12:42.667690+0000","last_active":"2023-10-08T16:12:42.668176+0000","last_peered":"2023-10-08T16:12:42.668176+0000","last_clean":"2023-10-08T16:12:42.668176+0000","last_became_active":"2023-10-08T16:12:42.667372+0000","last_became_peered":"2023-10-08T16:12:42.667372+0000","last_unstale":"2023-10-08T16:12:42.668176+0000","last_undegraded":"2023-10-08T16:12:42.668176+0000","last_fullsized":"2023-10-08T16:12:42.668176+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:53:48.102950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526334+0000","last_change":"2023-10-08T16:12:43.526334+0000","last_active":"2023-10-08T16:12:43.526334+0000","last_peered":"2023-10-08T16:12:43.526334+0000","last_clean":"2023-10-08T16:12:43.526334+0000","last_became_active":"2023-10-08T16:12:43.526025+0000","last_became_peered":"2023-10-08T16:12:43.526025+0000","last_unstale":"2023-10-08T16:12:43.526334+0000","last_undegraded":"2023-10-08T16:12:43.526334+0000","last_fullsized":"2023-10-08T16:12:43.526334+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:16.663493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683029+0000","last_change":"2023-10-08T16:12:42.669361+0000","last_active":"2023-10-08T16:12:43.683029+0000","last_peered":"2023-10-08T16:12:43.683029+0000","last_clean":"2023-10-08T16:12:43.683029+0000","last_became_active":"2023-10-08T16:12:42.669018+0000","last_became_peered":"2023-10-08T16:12:42.669018+0000","last_unstale":"2023-10-08T16:12:43.683029+0000","last_undegraded":"2023-10-08T16:12:43.683029+0000","last_fullsized":"2023-10-08T16:12:43.683029+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:38:22.997785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.533008+0000","last_change":"2023-10-08T16:12:43.533008+0000","last_active":"2023-10-08T16:12:43.533008+0000","last_peered":"2023-10-08T16:12:43.533008+0000","last_clean":"2023-10-08T16:12:43.533008+0000","last_became_active":"2023-10-08T16:12:43.532361+0000","last_became_peered":"2023-10-08T16:12:43.532361+0000","last_unstale":"2023-10-08T16:12:43.533008+0000","last_undegraded":"2023-10-08T16:12:43.533008+0000","last_fullsized":"2023-10-08T16:12:43.533008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:35.318963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.662422+0000","last_change":"2023-10-08T16:12:42.657003+0000","last_active":"2023-10-08T16:12:42.662422+0000","last_peered":"2023-10-08T16:12:42.662422+0000","last_clean":"2023-10-08T16:12:42.662422+0000","last_became_active":"2023-10-08T16:12:42.656774+0000","last_became_peered":"2023-10-08T16:12:42.656774+0000","last_unstale":"2023-10-08T16:12:42.662422+0000","last_undegraded":"2023-10-08T16:12:42.662422+0000","last_fullsized":"2023-10-08T16:12:42.662422+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:42.277461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523830+0000","last_change":"2023-10-08T16:12:43.523830+0000","last_active":"2023-10-08T16:12:43.523830+0000","last_peered":"2023-10-08T16:12:43.523830+0000","last_clean":"2023-10-08T16:12:43.523830+0000","last_became_active":"2023-10-08T16:12:43.523547+0000","last_became_peered":"2023-10-08T16:12:43.523547+0000","last_unstale":"2023-10-08T16:12:43.523830+0000","last_undegraded":"2023-10-08T16:12:43.523830+0000","last_fullsized":"2023-10-08T16:12:43.523830+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:35.469782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672611+0000","last_change":"2023-10-08T16:12:42.669923+0000","last_active":"2023-10-08T16:12:42.672611+0000","last_peered":"2023-10-08T16:12:42.672611+0000","last_clean":"2023-10-08T16:12:42.672611+0000","last_became_active":"2023-10-08T16:12:42.669645+0000","last_became_peered":"2023-10-08T16:12:42.669645+0000","last_unstale":"2023-10-08T16:12:42.672611+0000","last_undegraded":"2023-10-08T16:12:42.672611+0000","last_fullsized":"2023-10-08T16:12:42.672611+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:15.539493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523118+0000","last_change":"2023-10-08T16:12:43.523118+0000","last_active":"2023-10-08T16:12:43.523118+0000","last_peered":"2023-10-08T16:12:43.523118+0000","last_clean":"2023-10-08T16:12:43.523118+0000","last_became_active":"2023-10-08T16:12:43.522852+0000","last_became_peered":"2023-10-08T16:12:43.522852+0000","last_unstale":"2023-10-08T16:12:43.523118+0000","last_undegraded":"2023-10-08T16:12:43.523118+0000","last_fullsized":"2023-10-08T16:12:43.523118+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:21.622599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670015+0000","last_change":"2023-10-08T16:12:42.665851+0000","last_active":"2023-10-08T16:12:42.670015+0000","last_peered":"2023-10-08T16:12:42.670015+0000","last_clean":"2023-10-08T16:12:42.670015+0000","last_became_active":"2023-10-08T16:12:42.665359+0000","last_became_peered":"2023-10-08T16:12:42.665359+0000","last_unstale":"2023-10-08T16:12:42.670015+0000","last_undegraded":"2023-10-08T16:12:42.670015+0000","last_fullsized":"2023-10-08T16:12:42.670015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:48.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521891+0000","last_change":"2023-10-08T16:12:43.521891+0000","last_active":"2023-10-08T16:12:43.521891+0000","last_peered":"2023-10-08T16:12:43.521891+0000","last_clean":"2023-10-08T16:12:43.521891+0000","last_became_active":"2023-10-08T16:12:43.521631+0000","last_became_peered":"2023-10-08T16:12:43.521631+0000","last_unstale":"2023-10-08T16:12:43.521891+0000","last_undegraded":"2023-10-08T16:12:43.521891+0000","last_fullsized":"2023-10-08T16:12:43.521891+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:20.078674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683447+0000","last_change":"2023-10-08T16:12:42.670922+0000","last_active":"2023-10-08T16:12:43.683447+0000","last_peered":"2023-10-08T16:12:43.683447+0000","last_clean":"2023-10-08T16:12:43.683447+0000","last_became_active":"2023-10-08T16:12:42.670664+0000","last_became_peered":"2023-10-08T16:12:42.670664+0000","last_unstale":"2023-10-08T16:12:43.683447+0000","last_undegraded":"2023-10-08T16:12:43.683447+0000","last_fullsized":"2023-10-08T16:12:43.683447+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:53.610877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.537001+0000","last_change":"2023-10-08T16:12:43.537001+0000","last_active":"2023-10-08T16:12:43.537001+0000","last_peered":"2023-10-08T16:12:43.537001+0000","last_clean":"2023-10-08T16:12:43.537001+0000","last_became_active":"2023-10-08T16:12:43.536760+0000","last_became_peered":"2023-10-08T16:12:43.536760+0000","last_unstale":"2023-10-08T16:12:43.537001+0000","last_undegraded":"2023-10-08T16:12:43.537001+0000","last_fullsized":"2023-10-08T16:12:43.537001+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:56.882751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672558+0000","last_change":"2023-10-08T16:12:42.668813+0000","last_active":"2023-10-08T16:12:42.672558+0000","last_peered":"2023-10-08T16:12:42.672558+0000","last_clean":"2023-10-08T16:12:42.672558+0000","last_became_active":"2023-10-08T16:12:42.668488+0000","last_became_peered":"2023-10-08T16:12:42.668488+0000","last_unstale":"2023-10-08T16:12:42.672558+0000","last_undegraded":"2023-10-08T16:12:42.672558+0000","last_fullsized":"2023-10-08T16:12:42.672558+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:53.003952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.533385+0000","last_change":"2023-10-08T16:12:43.533385+0000","last_active":"2023-10-08T16:12:43.533385+0000","last_peered":"2023-10-08T16:12:43.533385+0000","last_clean":"2023-10-08T16:12:43.533385+0000","last_became_active":"2023-10-08T16:12:43.533023+0000","last_became_peered":"2023-10-08T16:12:43.533023+0000","last_unstale":"2023-10-08T16:12:43.533385+0000","last_undegraded":"2023-10-08T16:12:43.533385+0000","last_fullsized":"2023-10-08T16:12:43.533385+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:28.588041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672697+0000","last_change":"2023-10-08T16:12:42.669067+0000","last_active":"2023-10-08T16:12:42.672697+0000","last_peered":"2023-10-08T16:12:42.672697+0000","last_clean":"2023-10-08T16:12:42.672697+0000","last_became_active":"2023-10-08T16:12:42.668708+0000","last_became_peered":"2023-10-08T16:12:42.668708+0000","last_unstale":"2023-10-08T16:12:42.672697+0000","last_undegraded":"2023-10-08T16:12:42.672697+0000","last_fullsized":"2023-10-08T16:12:42.672697+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:47:15.279636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520816+0000","last_change":"2023-10-08T16:12:43.520816+0000","last_active":"2023-10-08T16:12:43.520816+0000","last_peered":"2023-10-08T16:12:43.520816+0000","last_clean":"2023-10-08T16:12:43.520816+0000","last_became_active":"2023-10-08T16:12:43.520575+0000","last_became_peered":"2023-10-08T16:12:43.520575+0000","last_unstale":"2023-10-08T16:12:43.520816+0000","last_undegraded":"2023-10-08T16:12:43.520816+0000","last_fullsized":"2023-10-08T16:12:43.520816+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:16:43.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.669965+0000","last_change":"2023-10-08T16:12:42.665234+0000","last_active":"2023-10-08T16:12:42.669965+0000","last_peered":"2023-10-08T16:12:42.669965+0000","last_clean":"2023-10-08T16:12:42.669965+0000","last_became_active":"2023-10-08T16:12:42.664918+0000","last_became_peered":"2023-10-08T16:12:42.664918+0000","last_unstale":"2023-10-08T16:12:42.669965+0000","last_undegraded":"2023-10-08T16:12:42.669965+0000","last_fullsized":"2023-10-08T16:12:42.669965+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:12.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523243+0000","last_change":"2023-10-08T16:12:43.523243+0000","last_active":"2023-10-08T16:12:43.523243+0000","last_peered":"2023-10-08T16:12:43.523243+0000","last_clean":"2023-10-08T16:12:43.523243+0000","last_became_active":"2023-10-08T16:12:43.522703+0000","last_became_peered":"2023-10-08T16:12:43.522703+0000","last_unstale":"2023-10-08T16:12:43.523243+0000","last_undegraded":"2023-10-08T16:12:43.523243+0000","last_fullsized":"2023-10-08T16:12:43.523243+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:06.674016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.668156+0000","last_change":"2023-10-08T16:12:42.667341+0000","last_active":"2023-10-08T16:12:42.668156+0000","last_peered":"2023-10-08T16:12:42.668156+0000","last_clean":"2023-10-08T16:12:42.668156+0000","last_became_active":"2023-10-08T16:12:42.667077+0000","last_became_peered":"2023-10-08T16:12:42.667077+0000","last_unstale":"2023-10-08T16:12:42.668156+0000","last_undegraded":"2023-10-08T16:12:42.668156+0000","last_fullsized":"2023-10-08T16:12:42.668156+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:48.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528784+0000","last_change":"2023-10-08T16:12:43.528784+0000","last_active":"2023-10-08T16:12:43.528784+0000","last_peered":"2023-10-08T16:12:43.528784+0000","last_clean":"2023-10-08T16:12:43.528784+0000","last_became_active":"2023-10-08T16:12:43.528497+0000","last_became_peered":"2023-10-08T16:12:43.528497+0000","last_unstale":"2023-10-08T16:12:43.528784+0000","last_undegraded":"2023-10-08T16:12:43.528784+0000","last_fullsized":"2023-10-08T16:12:43.528784+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:59:45.243014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.667245+0000","last_change":"2023-10-08T16:12:42.666338+0000","last_active":"2023-10-08T16:12:42.667245+0000","last_peered":"2023-10-08T16:12:42.667245+0000","last_clean":"2023-10-08T16:12:42.667245+0000","last_became_active":"2023-10-08T16:12:42.666081+0000","last_became_peered":"2023-10-08T16:12:42.666081+0000","last_unstale":"2023-10-08T16:12:42.667245+0000","last_undegraded":"2023-10-08T16:12:42.667245+0000","last_fullsized":"2023-10-08T16:12:42.667245+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:35.775855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526794+0000","last_change":"2023-10-08T16:12:43.526794+0000","last_active":"2023-10-08T16:12:43.526794+0000","last_peered":"2023-10-08T16:12:43.526794+0000","last_clean":"2023-10-08T16:12:43.526794+0000","last_became_active":"2023-10-08T16:12:43.524473+0000","last_became_peered":"2023-10-08T16:12:43.524473+0000","last_unstale":"2023-10-08T16:12:43.526794+0000","last_undegraded":"2023-10-08T16:12:43.526794+0000","last_fullsized":"2023-10-08T16:12:43.526794+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.137973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670042+0000","last_change":"2023-10-08T16:12:42.669549+0000","last_active":"2023-10-08T16:12:42.670042+0000","last_peered":"2023-10-08T16:12:42.670042+0000","last_clean":"2023-10-08T16:12:42.670042+0000","last_became_active":"2023-10-08T16:12:42.669240+0000","last_became_peered":"2023-10-08T16:12:42.669240+0000","last_unstale":"2023-10-08T16:12:42.670042+0000","last_undegraded":"2023-10-08T16:12:42.670042+0000","last_fullsized":"2023-10-08T16:12:42.670042+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:34.317322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525048+0000","last_change":"2023-10-08T16:12:43.525048+0000","last_active":"2023-10-08T16:12:43.525048+0000","last_peered":"2023-10-08T16:12:43.525048+0000","last_clean":"2023-10-08T16:12:43.525048+0000","last_became_active":"2023-10-08T16:12:43.524557+0000","last_became_peered":"2023-10-08T16:12:43.524557+0000","last_unstale":"2023-10-08T16:12:43.525048+0000","last_undegraded":"2023-10-08T16:12:43.525048+0000","last_fullsized":"2023-10-08T16:12:43.525048+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:47.501664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.669990+0000","last_change":"2023-10-08T16:12:42.665648+0000","last_active":"2023-10-08T16:12:42.669990+0000","last_peered":"2023-10-08T16:12:42.669990+0000","last_clean":"2023-10-08T16:12:42.669990+0000","last_became_active":"2023-10-08T16:12:42.664969+0000","last_became_peered":"2023-10-08T16:12:42.664969+0000","last_unstale":"2023-10-08T16:12:42.669990+0000","last_undegraded":"2023-10-08T16:12:42.669990+0000","last_fullsized":"2023-10-08T16:12:42.669990+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:42.277461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522736+0000","last_change":"2023-10-08T16:12:43.522736+0000","last_active":"2023-10-08T16:12:43.522736+0000","last_peered":"2023-10-08T16:12:43.522736+0000","last_clean":"2023-10-08T16:12:43.522736+0000","last_became_active":"2023-10-08T16:12:43.522486+0000","last_became_peered":"2023-10-08T16:12:43.522486+0000","last_unstale":"2023-10-08T16:12:43.522736+0000","last_undegraded":"2023-10-08T16:12:43.522736+0000","last_fullsized":"2023-10-08T16:12:43.522736+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:21.529765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.682888+0000","last_change":"2023-10-08T16:12:42.669871+0000","last_active":"2023-10-08T16:12:43.682888+0000","last_peered":"2023-10-08T16:12:43.682888+0000","last_clean":"2023-10-08T16:12:43.682888+0000","last_became_active":"2023-10-08T16:12:42.669535+0000","last_became_peered":"2023-10-08T16:12:42.669535+0000","last_unstale":"2023-10-08T16:12:43.682888+0000","last_undegraded":"2023-10-08T16:12:43.682888+0000","last_fullsized":"2023-10-08T16:12:43.682888+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:26.747148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.663951+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:12:42.663951+0000","last_peered":"2023-10-08T16:12:42.663951+0000","last_clean":"2023-10-08T16:12:42.663951+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:12:42.663951+0000","last_undegraded":"2023-10-08T16:12:42.663951+0000","last_fullsized":"2023-10-08T16:12:42.663951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.86199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72799999999999998},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.82199999999999995,"15min":1.083},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.64900000000000002,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.70299999999999996,"15min":1.175},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.77900000000000003},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.374,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":1.677,"15min":1.677},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.78100000000000003},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63600000000000001,"5min":0.69899999999999995,"15min":8.3409999999999993},"last":0.71799999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44400000000000001,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.753,"15min":0.753},"last":0.70399999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":935460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.42099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.61499999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.504},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.44}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307528,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064312,"statfs":{"total":96636764160,"available":96321855488,"internally_reserved":0,"allocated":2097152,"data_stored":935494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.249},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.26600000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.23499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.32000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.46200000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.38600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304016,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067824,"statfs":{"total":96636764160,"available":96325451776,"internally_reserved":0,"allocated":2695168,"data_stored":1526416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.375},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.40799999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.47599999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.49099999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.45700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303448,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068392,"statfs":{"total":96636764160,"available":96326033408,"internally_reserved":0,"allocated":2113536,"data_stored":936996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.307},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.46400000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.34699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.44400000000000001,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.35999999999999999},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.34200000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48999999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.313}]},{"osd":4,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.48999999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.503},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81499999999999995,"15min":4.1029999999999998},"last":0.38500000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74399999999999999,"5min":0.85199999999999998,"15min":0.871},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.47299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":937010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.246},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.47999999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.46800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.434}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":935958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.52600000000000002},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.40000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":0.83399999999999996},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.60099999999999998},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.34899999999999998},"max":{"1min":0.80400000000000005,"5min":0.875,"15min":0.875},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.89200000000000002},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303428,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068412,"statfs":{"total":96636764160,"available":96326053888,"internally_reserved":0,"allocated":2093056,"data_stored":935404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.63}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.38},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:12:47.748 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:12:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.894+0000 7f7325388700 1 Processor -- start 2023-10-08T16:12:47.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.894+0000 7f7325388700 1 -- start start 2023-10-08T16:12:47.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f7325388700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7320159c60 0x7f732015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f7325388700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f732015db10 0x7f7320069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f7325388700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732015a160 con 0x7f7320159890 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f7325388700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732015df50 con 0x7f7320159c60 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f7325388700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200d13a0 con 0x7f732015db10 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f731e7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7320159c60 0x7f732015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f731e7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7320159c60 0x7f732015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32824/0 (socket says 172.21.15.170:32824) 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f731e7fc700 1 -- 172.21.15.170:0/1763263642 learned_addr learned my addr 172.21.15.170:0/1763263642 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.895+0000 7f731f7fe700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f732015db10 0x7f7320069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731e7fc700 1 -- 172.21.15.170:0/1763263642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f732015db10 msgr2=0x7f7320069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731e7fc700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f732015db10 0x7f7320069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731e7fc700 1 -- 172.21.15.170:0/1763263642 >> v1:172.21.15.191:6789/0 conn(0x7f7320159890 legacy=0x7f732015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731e7fc700 1 -- 172.21.15.170:0/1763263642 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732015e140 con 0x7f7320159c60 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731e7fc700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f732015d5d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f73080099b0 tx=0x7f7308009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a2bf4a46ef7a7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731dffb700 1 -- 172.21.15.170:0/1763263642 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800b860 con 0x7f7320159c60 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731dffb700 1 -- 172.21.15.170:0/1763263642 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730800b9c0 con 0x7f7320159c60 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f731dffb700 1 -- 172.21.15.170:0/1763263642 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730800bd60 con 0x7f7320159c60 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 -- 172.21.15.170:0/1763263642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 msgr2=0x7f732015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f732015d5d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f73080099b0 tx=0x7f7308009250 comp rx=0 tx=0).stop 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 -- 172.21.15.170:0/1763263642 shutdown_connections 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f732015db10 0x7f7320069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 --2- 172.21.15.170:0/1763263642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f732015d5d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 -- 172.21.15.170:0/1763263642 >> 172.21.15.170:0/1763263642 conn(0x7f7320068e20 msgr2=0x7f73200d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:47.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 -- 172.21.15.170:0/1763263642 shutdown_connections 2023-10-08T16:12:47.901 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.896+0000 7f7325388700 1 -- 172.21.15.170:0/1763263642 wait complete. 2023-10-08T16:12:47.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 Processor -- start 2023-10-08T16:12:47.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 -- start start 2023-10-08T16:12:47.903 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7320159890 0x7f73200cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f732015db10 0x7f73200b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7320158e40 con 0x7f7320159c60 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200c8dd0 con 0x7f732015db10 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f7325388700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73200b5b60 con 0x7f7320159890 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.898+0000 7f731e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32826/0 (socket says 172.21.15.170:32826) 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 -- 172.21.15.170:0/2231150616 learned_addr learned my addr 172.21.15.170:0/2231150616 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:47.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731effd700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7320159890 0x7f73200cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731f7fe700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f732015db10 0x7f73200b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 -- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7320159890 msgr2=0x7f73200cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7320159890 0x7f73200cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 -- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f732015db10 msgr2=0x7f73200b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f732015db10 0x7f73200b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 -- 172.21.15.170:0/2231150616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080094f0 con 0x7f7320159c60 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f731e7fc700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f73200899c0 tx=0x7f7308017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308011e40 con 0x7f7320159c60 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308024440 con 0x7f7320159c60 2023-10-08T16:12:47.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73200b5e40 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308024800 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.899+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73200c86c0 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.900+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73200b6030 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.901+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f730801a420 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.901+0000 7f7316ffd700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 0x7f73000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.901+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7308086b30 con 0x7f7320159c60 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.901+0000 7f731effd700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 0x7f73000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:47.906 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.901+0000 7f731effd700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 0x7f73000697d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7318007fd0 tx=0x7f73180016f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:47.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:47.905+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73080537c0 con 0x7f7320159c60 2023-10-08T16:12:48.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.082+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73200c88e0 con 0x7f7300067320 2023-10-08T16:12:48.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.091+0000 7f7316ffd700 1 -- 172.21.15.170:0/2231150616 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289829 (secure 0 0 0) 0x7f731800e970 con 0x7f7300067320 2023-10-08T16:12:48.095 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:48.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 msgr2=0x7f73000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 0x7f73000697d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7318007fd0 tx=0x7f73180016f0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 msgr2=0x7f7320158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f73200899c0 tx=0x7f7308017000 comp rx=0 tx=0).stop 2023-10-08T16:12:48.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 shutdown_connections 2023-10-08T16:12:48.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f732015db10 0x7f73200b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7320159890 0x7f73200cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7300067320 0x7f73000697d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 --2- 172.21.15.170:0/2231150616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7320159c60 0x7f7320158900 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 >> 172.21.15.170:0/2231150616 conn(0x7f7320068e20 msgr2=0x7f73200c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:48.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 shutdown_connections 2023-10-08T16:12:48.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.095+0000 7f7325388700 1 -- 172.21.15.170:0/2231150616 wait complete. 2023-10-08T16:12:48.101 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:12:48.107 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2023-10-08T16:12:46.323193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2432744,"kb_used_data":17576,"kb_used_omap":0,"kb_used_meta":2415104,"kb_avail":752541976,"statfs":{"total":773094113280,"available":770602983424,"internally_reserved":0,"allocated":17997824,"data_stored":8668580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2473066496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.677654"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526827+0000","last_change":"2023-10-08T16:12:43.526827+0000","last_active":"2023-10-08T16:12:43.526827+0000","last_peered":"2023-10-08T16:12:43.526827+0000","last_clean":"2023-10-08T16:12:43.526827+0000","last_became_active":"2023-10-08T16:12:43.526346+0000","last_became_peered":"2023-10-08T16:12:43.526346+0000","last_unstale":"2023-10-08T16:12:43.526827+0000","last_undegraded":"2023-10-08T16:12:43.526827+0000","last_fullsized":"2023-10-08T16:12:43.526827+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:01.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531582+0000","last_change":"2023-10-08T16:12:43.531582+0000","last_active":"2023-10-08T16:12:43.531582+0000","last_peered":"2023-10-08T16:12:43.531582+0000","last_clean":"2023-10-08T16:12:43.531582+0000","last_became_active":"2023-10-08T16:12:43.531148+0000","last_became_peered":"2023-10-08T16:12:43.531148+0000","last_unstale":"2023-10-08T16:12:43.531582+0000","last_undegraded":"2023-10-08T16:12:43.531582+0000","last_fullsized":"2023-10-08T16:12:43.531582+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:24:55.246457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530593+0000","last_change":"2023-10-08T16:12:43.530593+0000","last_active":"2023-10-08T16:12:43.530593+0000","last_peered":"2023-10-08T16:12:43.530593+0000","last_clean":"2023-10-08T16:12:43.530593+0000","last_became_active":"2023-10-08T16:12:43.529979+0000","last_became_peered":"2023-10-08T16:12:43.529979+0000","last_unstale":"2023-10-08T16:12:43.530593+0000","last_undegraded":"2023-10-08T16:12:43.530593+0000","last_fullsized":"2023-10-08T16:12:43.530593+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:57.572729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522358+0000","last_change":"2023-10-08T16:12:43.522358+0000","last_active":"2023-10-08T16:12:43.522358+0000","last_peered":"2023-10-08T16:12:43.522358+0000","last_clean":"2023-10-08T16:12:43.522358+0000","last_became_active":"2023-10-08T16:12:43.522034+0000","last_became_peered":"2023-10-08T16:12:43.522034+0000","last_unstale":"2023-10-08T16:12:43.522358+0000","last_undegraded":"2023-10-08T16:12:43.522358+0000","last_fullsized":"2023-10-08T16:12:43.522358+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:48:19.482375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523128+0000","last_change":"2023-10-08T16:12:43.523128+0000","last_active":"2023-10-08T16:12:43.523128+0000","last_peered":"2023-10-08T16:12:43.523128+0000","last_clean":"2023-10-08T16:12:43.523128+0000","last_became_active":"2023-10-08T16:12:43.522828+0000","last_became_peered":"2023-10-08T16:12:43.522828+0000","last_unstale":"2023-10-08T16:12:43.523128+0000","last_undegraded":"2023-10-08T16:12:43.523128+0000","last_fullsized":"2023-10-08T16:12:43.523128+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:10.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526103+0000","last_change":"2023-10-08T16:12:43.526103+0000","last_active":"2023-10-08T16:12:43.526103+0000","last_peered":"2023-10-08T16:12:43.526103+0000","last_clean":"2023-10-08T16:12:43.526103+0000","last_became_active":"2023-10-08T16:12:43.525823+0000","last_became_peered":"2023-10-08T16:12:43.525823+0000","last_unstale":"2023-10-08T16:12:43.526103+0000","last_undegraded":"2023-10-08T16:12:43.526103+0000","last_fullsized":"2023-10-08T16:12:43.526103+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:48.744850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530164+0000","last_change":"2023-10-08T16:12:43.530164+0000","last_active":"2023-10-08T16:12:43.530164+0000","last_peered":"2023-10-08T16:12:43.530164+0000","last_clean":"2023-10-08T16:12:43.530164+0000","last_became_active":"2023-10-08T16:12:43.529842+0000","last_became_peered":"2023-10-08T16:12:43.529842+0000","last_unstale":"2023-10-08T16:12:43.530164+0000","last_undegraded":"2023-10-08T16:12:43.530164+0000","last_fullsized":"2023-10-08T16:12:43.530164+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:09.900526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.524584+0000","last_change":"2023-10-08T16:12:43.524584+0000","last_active":"2023-10-08T16:12:43.524584+0000","last_peered":"2023-10-08T16:12:43.524584+0000","last_clean":"2023-10-08T16:12:43.524584+0000","last_became_active":"2023-10-08T16:12:43.524332+0000","last_became_peered":"2023-10-08T16:12:43.524332+0000","last_unstale":"2023-10-08T16:12:43.524584+0000","last_undegraded":"2023-10-08T16:12:43.524584+0000","last_fullsized":"2023-10-08T16:12:43.524584+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.990171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532670+0000","last_change":"2023-10-08T16:12:43.532670+0000","last_active":"2023-10-08T16:12:43.532670+0000","last_peered":"2023-10-08T16:12:43.532670+0000","last_clean":"2023-10-08T16:12:43.532670+0000","last_became_active":"2023-10-08T16:12:43.532400+0000","last_became_peered":"2023-10-08T16:12:43.532400+0000","last_unstale":"2023-10-08T16:12:43.532670+0000","last_undegraded":"2023-10-08T16:12:43.532670+0000","last_fullsized":"2023-10-08T16:12:43.532670+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:54.612519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525247+0000","last_change":"2023-10-08T16:12:43.525247+0000","last_active":"2023-10-08T16:12:43.525247+0000","last_peered":"2023-10-08T16:12:43.525247+0000","last_clean":"2023-10-08T16:12:43.525247+0000","last_became_active":"2023-10-08T16:12:43.524905+0000","last_became_peered":"2023-10-08T16:12:43.524905+0000","last_unstale":"2023-10-08T16:12:43.525247+0000","last_undegraded":"2023-10-08T16:12:43.525247+0000","last_fullsized":"2023-10-08T16:12:43.525247+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:13.060175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531008+0000","last_change":"2023-10-08T16:12:43.531008+0000","last_active":"2023-10-08T16:12:43.531008+0000","last_peered":"2023-10-08T16:12:43.531008+0000","last_clean":"2023-10-08T16:12:43.531008+0000","last_became_active":"2023-10-08T16:12:43.530738+0000","last_became_peered":"2023-10-08T16:12:43.530738+0000","last_unstale":"2023-10-08T16:12:43.531008+0000","last_undegraded":"2023-10-08T16:12:43.531008+0000","last_fullsized":"2023-10-08T16:12:43.531008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:07:40.661491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520729+0000","last_change":"2023-10-08T16:12:43.520729+0000","last_active":"2023-10-08T16:12:43.520729+0000","last_peered":"2023-10-08T16:12:43.520729+0000","last_clean":"2023-10-08T16:12:43.520729+0000","last_became_active":"2023-10-08T16:12:43.520367+0000","last_became_peered":"2023-10-08T16:12:43.520367+0000","last_unstale":"2023-10-08T16:12:43.520729+0000","last_undegraded":"2023-10-08T16:12:43.520729+0000","last_fullsized":"2023-10-08T16:12:43.520729+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:40:59.939112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526071+0000","last_change":"2023-10-08T16:12:43.526071+0000","last_active":"2023-10-08T16:12:43.526071+0000","last_peered":"2023-10-08T16:12:43.526071+0000","last_clean":"2023-10-08T16:12:43.526071+0000","last_became_active":"2023-10-08T16:12:43.525778+0000","last_became_peered":"2023-10-08T16:12:43.525778+0000","last_unstale":"2023-10-08T16:12:43.526071+0000","last_undegraded":"2023-10-08T16:12:43.526071+0000","last_fullsized":"2023-10-08T16:12:43.526071+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:19:29.142447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525129+0000","last_change":"2023-10-08T16:12:43.525129+0000","last_active":"2023-10-08T16:12:43.525129+0000","last_peered":"2023-10-08T16:12:43.525129+0000","last_clean":"2023-10-08T16:12:43.525129+0000","last_became_active":"2023-10-08T16:12:43.524775+0000","last_became_peered":"2023-10-08T16:12:43.524775+0000","last_unstale":"2023-10-08T16:12:43.525129+0000","last_undegraded":"2023-10-08T16:12:43.525129+0000","last_fullsized":"2023-10-08T16:12:43.525129+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:16.888336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530335+0000","last_change":"2023-10-08T16:12:43.530335+0000","last_active":"2023-10-08T16:12:43.530335+0000","last_peered":"2023-10-08T16:12:43.530335+0000","last_clean":"2023-10-08T16:12:43.530335+0000","last_became_active":"2023-10-08T16:12:43.530140+0000","last_became_peered":"2023-10-08T16:12:43.530140+0000","last_unstale":"2023-10-08T16:12:43.530335+0000","last_undegraded":"2023-10-08T16:12:43.530335+0000","last_fullsized":"2023-10-08T16:12:43.530335+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:49:52.721323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517213+0000","last_change":"2023-10-08T16:12:43.517213+0000","last_active":"2023-10-08T16:12:43.517213+0000","last_peered":"2023-10-08T16:12:43.517213+0000","last_clean":"2023-10-08T16:12:43.517213+0000","last_became_active":"2023-10-08T16:12:43.516887+0000","last_became_peered":"2023-10-08T16:12:43.516887+0000","last_unstale":"2023-10-08T16:12:43.517213+0000","last_undegraded":"2023-10-08T16:12:43.517213+0000","last_fullsized":"2023-10-08T16:12:43.517213+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:54.612519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528542+0000","last_change":"2023-10-08T16:12:43.528542+0000","last_active":"2023-10-08T16:12:43.528542+0000","last_peered":"2023-10-08T16:12:43.528542+0000","last_clean":"2023-10-08T16:12:43.528542+0000","last_became_active":"2023-10-08T16:12:43.525304+0000","last_became_peered":"2023-10-08T16:12:43.525304+0000","last_unstale":"2023-10-08T16:12:43.528542+0000","last_undegraded":"2023-10-08T16:12:43.528542+0000","last_fullsized":"2023-10-08T16:12:43.528542+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:09.900526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534209+0000","last_change":"2023-10-08T16:12:43.534209+0000","last_active":"2023-10-08T16:12:43.534209+0000","last_peered":"2023-10-08T16:12:43.534209+0000","last_clean":"2023-10-08T16:12:43.534209+0000","last_became_active":"2023-10-08T16:12:43.533980+0000","last_became_peered":"2023-10-08T16:12:43.533980+0000","last_unstale":"2023-10-08T16:12:43.534209+0000","last_undegraded":"2023-10-08T16:12:43.534209+0000","last_fullsized":"2023-10-08T16:12:43.534209+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:49.031323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527750+0000","last_change":"2023-10-08T16:12:43.527750+0000","last_active":"2023-10-08T16:12:43.527750+0000","last_peered":"2023-10-08T16:12:43.527750+0000","last_clean":"2023-10-08T16:12:43.527750+0000","last_became_active":"2023-10-08T16:12:43.526754+0000","last_became_peered":"2023-10-08T16:12:43.526754+0000","last_unstale":"2023-10-08T16:12:43.527750+0000","last_undegraded":"2023-10-08T16:12:43.527750+0000","last_fullsized":"2023-10-08T16:12:43.527750+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:46:40.742811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521700+0000","last_change":"2023-10-08T16:12:43.521700+0000","last_active":"2023-10-08T16:12:43.521700+0000","last_peered":"2023-10-08T16:12:43.521700+0000","last_clean":"2023-10-08T16:12:43.521700+0000","last_became_active":"2023-10-08T16:12:43.520410+0000","last_became_peered":"2023-10-08T16:12:43.520410+0000","last_unstale":"2023-10-08T16:12:43.521700+0000","last_undegraded":"2023-10-08T16:12:43.521700+0000","last_fullsized":"2023-10-08T16:12:43.521700+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:21.529765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525895+0000","last_change":"2023-10-08T16:12:43.525895+0000","last_active":"2023-10-08T16:12:43.525895+0000","last_peered":"2023-10-08T16:12:43.525895+0000","last_clean":"2023-10-08T16:12:43.525895+0000","last_became_active":"2023-10-08T16:12:43.525245+0000","last_became_peered":"2023-10-08T16:12:43.525245+0000","last_unstale":"2023-10-08T16:12:43.525895+0000","last_undegraded":"2023-10-08T16:12:43.525895+0000","last_fullsized":"2023-10-08T16:12:43.525895+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:43.196609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522534+0000","last_change":"2023-10-08T16:12:43.522534+0000","last_active":"2023-10-08T16:12:43.522534+0000","last_peered":"2023-10-08T16:12:43.522534+0000","last_clean":"2023-10-08T16:12:43.522534+0000","last_became_active":"2023-10-08T16:12:43.522269+0000","last_became_peered":"2023-10-08T16:12:43.522269+0000","last_unstale":"2023-10-08T16:12:43.522534+0000","last_undegraded":"2023-10-08T16:12:43.522534+0000","last_fullsized":"2023-10-08T16:12:43.522534+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:07.697011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.524887+0000","last_change":"2023-10-08T16:12:43.524887+0000","last_active":"2023-10-08T16:12:43.524887+0000","last_peered":"2023-10-08T16:12:43.524887+0000","last_clean":"2023-10-08T16:12:43.524887+0000","last_became_active":"2023-10-08T16:12:43.523971+0000","last_became_peered":"2023-10-08T16:12:43.523971+0000","last_unstale":"2023-10-08T16:12:43.524887+0000","last_undegraded":"2023-10-08T16:12:43.524887+0000","last_fullsized":"2023-10-08T16:12:43.524887+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:01.959857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521231+0000","last_change":"2023-10-08T16:12:43.521231+0000","last_active":"2023-10-08T16:12:43.521231+0000","last_peered":"2023-10-08T16:12:43.521231+0000","last_clean":"2023-10-08T16:12:43.521231+0000","last_became_active":"2023-10-08T16:12:43.520996+0000","last_became_peered":"2023-10-08T16:12:43.520996+0000","last_unstale":"2023-10-08T16:12:43.521231+0000","last_undegraded":"2023-10-08T16:12:43.521231+0000","last_fullsized":"2023-10-08T16:12:43.521231+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:54.005594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526888+0000","last_change":"2023-10-08T16:12:43.526888+0000","last_active":"2023-10-08T16:12:43.526888+0000","last_peered":"2023-10-08T16:12:43.526888+0000","last_clean":"2023-10-08T16:12:43.526888+0000","last_became_active":"2023-10-08T16:12:43.526562+0000","last_became_peered":"2023-10-08T16:12:43.526562+0000","last_unstale":"2023-10-08T16:12:43.526888+0000","last_undegraded":"2023-10-08T16:12:43.526888+0000","last_fullsized":"2023-10-08T16:12:43.526888+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:49.031323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527829+0000","last_change":"2023-10-08T16:12:43.527829+0000","last_active":"2023-10-08T16:12:43.527829+0000","last_peered":"2023-10-08T16:12:43.527829+0000","last_clean":"2023-10-08T16:12:43.527829+0000","last_became_active":"2023-10-08T16:12:43.527589+0000","last_became_peered":"2023-10-08T16:12:43.527589+0000","last_unstale":"2023-10-08T16:12:43.527829+0000","last_undegraded":"2023-10-08T16:12:43.527829+0000","last_fullsized":"2023-10-08T16:12:43.527829+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:24:40.856967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530215+0000","last_change":"2023-10-08T16:12:43.530215+0000","last_active":"2023-10-08T16:12:43.530215+0000","last_peered":"2023-10-08T16:12:43.530215+0000","last_clean":"2023-10-08T16:12:43.530215+0000","last_became_active":"2023-10-08T16:12:43.529814+0000","last_became_peered":"2023-10-08T16:12:43.529814+0000","last_unstale":"2023-10-08T16:12:43.530215+0000","last_undegraded":"2023-10-08T16:12:43.530215+0000","last_fullsized":"2023-10-08T16:12:43.530215+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:13.060175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.519554+0000","last_change":"2023-10-08T16:12:43.519554+0000","last_active":"2023-10-08T16:12:43.519554+0000","last_peered":"2023-10-08T16:12:43.519554+0000","last_clean":"2023-10-08T16:12:43.519554+0000","last_became_active":"2023-10-08T16:12:43.519288+0000","last_became_peered":"2023-10-08T16:12:43.519288+0000","last_unstale":"2023-10-08T16:12:43.519554+0000","last_undegraded":"2023-10-08T16:12:43.519554+0000","last_fullsized":"2023-10-08T16:12:43.519554+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:31.737355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531956+0000","last_change":"2023-10-08T16:12:43.531956+0000","last_active":"2023-10-08T16:12:43.531956+0000","last_peered":"2023-10-08T16:12:43.531956+0000","last_clean":"2023-10-08T16:12:43.531956+0000","last_became_active":"2023-10-08T16:12:43.531694+0000","last_became_peered":"2023-10-08T16:12:43.531694+0000","last_unstale":"2023-10-08T16:12:43.531956+0000","last_undegraded":"2023-10-08T16:12:43.531956+0000","last_fullsized":"2023-10-08T16:12:43.531956+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:28:57.572729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531981+0000","last_change":"2023-10-08T16:12:43.531981+0000","last_active":"2023-10-08T16:12:43.531981+0000","last_peered":"2023-10-08T16:12:43.531981+0000","last_clean":"2023-10-08T16:12:43.531981+0000","last_became_active":"2023-10-08T16:12:43.531716+0000","last_became_peered":"2023-10-08T16:12:43.531716+0000","last_unstale":"2023-10-08T16:12:43.531981+0000","last_undegraded":"2023-10-08T16:12:43.531981+0000","last_fullsized":"2023-10-08T16:12:43.531981+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:07.697011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.519027+0000","last_change":"2023-10-08T16:12:43.519027+0000","last_active":"2023-10-08T16:12:43.519027+0000","last_peered":"2023-10-08T16:12:43.519027+0000","last_clean":"2023-10-08T16:12:43.519027+0000","last_became_active":"2023-10-08T16:12:43.518809+0000","last_became_peered":"2023-10-08T16:12:43.518809+0000","last_unstale":"2023-10-08T16:12:43.519027+0000","last_undegraded":"2023-10-08T16:12:43.519027+0000","last_fullsized":"2023-10-08T16:12:43.519027+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:38:23.977868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531682+0000","last_change":"2023-10-08T16:12:43.531682+0000","last_active":"2023-10-08T16:12:43.531682+0000","last_peered":"2023-10-08T16:12:43.531682+0000","last_clean":"2023-10-08T16:12:43.531682+0000","last_became_active":"2023-10-08T16:12:43.531415+0000","last_became_peered":"2023-10-08T16:12:43.531415+0000","last_unstale":"2023-10-08T16:12:43.531682+0000","last_undegraded":"2023-10-08T16:12:43.531682+0000","last_fullsized":"2023-10-08T16:12:43.531682+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:01.959857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532478+0000","last_change":"2023-10-08T16:12:43.532478+0000","last_active":"2023-10-08T16:12:43.532478+0000","last_peered":"2023-10-08T16:12:43.532478+0000","last_clean":"2023-10-08T16:12:43.532478+0000","last_became_active":"2023-10-08T16:12:43.531927+0000","last_became_peered":"2023-10-08T16:12:43.531927+0000","last_unstale":"2023-10-08T16:12:43.532478+0000","last_undegraded":"2023-10-08T16:12:43.532478+0000","last_fullsized":"2023-10-08T16:12:43.532478+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:27.500513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"165'4","reported_seq":13,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683093+0000","last_change":"2023-10-08T16:12:42.669847+0000","last_active":"2023-10-08T16:12:43.683093+0000","last_peered":"2023-10-08T16:12:43.683093+0000","last_clean":"2023-10-08T16:12:43.683093+0000","last_became_active":"2023-10-08T16:12:42.669571+0000","last_became_peered":"2023-10-08T16:12:42.669571+0000","last_unstale":"2023-10-08T16:12:43.683093+0000","last_undegraded":"2023-10-08T16:12:43.683093+0000","last_fullsized":"2023-10-08T16:12:43.683093+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:22:48.029681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527753+0000","last_change":"2023-10-08T16:12:43.527753+0000","last_active":"2023-10-08T16:12:43.527753+0000","last_peered":"2023-10-08T16:12:43.527753+0000","last_clean":"2023-10-08T16:12:43.527753+0000","last_became_active":"2023-10-08T16:12:43.527181+0000","last_became_peered":"2023-10-08T16:12:43.527181+0000","last_unstale":"2023-10-08T16:12:43.527753+0000","last_undegraded":"2023-10-08T16:12:43.527753+0000","last_fullsized":"2023-10-08T16:12:43.527753+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:43:13.262906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517083+0000","last_change":"2023-10-08T16:12:42.663668+0000","last_active":"2023-10-08T16:12:43.517083+0000","last_peered":"2023-10-08T16:12:43.517083+0000","last_clean":"2023-10-08T16:12:43.517083+0000","last_became_active":"2023-10-08T16:12:42.663363+0000","last_became_peered":"2023-10-08T16:12:42.663363+0000","last_unstale":"2023-10-08T16:12:43.517083+0000","last_undegraded":"2023-10-08T16:12:43.517083+0000","last_fullsized":"2023-10-08T16:12:43.517083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:42:08.898885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534188+0000","last_change":"2023-10-08T16:12:43.534188+0000","last_active":"2023-10-08T16:12:43.534188+0000","last_peered":"2023-10-08T16:12:43.534188+0000","last_clean":"2023-10-08T16:12:43.534188+0000","last_became_active":"2023-10-08T16:12:43.532957+0000","last_became_peered":"2023-10-08T16:12:43.532957+0000","last_unstale":"2023-10-08T16:12:43.534188+0000","last_undegraded":"2023-10-08T16:12:43.534188+0000","last_fullsized":"2023-10-08T16:12:43.534188+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:11:52.844283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.678842+0000","last_change":"2023-10-08T16:12:42.656853+0000","last_active":"2023-10-08T16:12:43.678842+0000","last_peered":"2023-10-08T16:12:43.678842+0000","last_clean":"2023-10-08T16:12:43.678842+0000","last_became_active":"2023-10-08T16:12:42.656571+0000","last_became_peered":"2023-10-08T16:12:42.656571+0000","last_unstale":"2023-10-08T16:12:43.678842+0000","last_undegraded":"2023-10-08T16:12:43.678842+0000","last_fullsized":"2023-10-08T16:12:43.678842+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:15.886695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.535194+0000","last_change":"2023-10-08T16:12:43.535194+0000","last_active":"2023-10-08T16:12:43.535194+0000","last_peered":"2023-10-08T16:12:43.535194+0000","last_clean":"2023-10-08T16:12:43.535194+0000","last_became_active":"2023-10-08T16:12:43.534875+0000","last_became_peered":"2023-10-08T16:12:43.534875+0000","last_unstale":"2023-10-08T16:12:43.535194+0000","last_undegraded":"2023-10-08T16:12:43.535194+0000","last_fullsized":"2023-10-08T16:12:43.535194+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:10:47.843465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.517363+0000","last_change":"2023-10-08T16:12:42.665562+0000","last_active":"2023-10-08T16:12:43.517363+0000","last_peered":"2023-10-08T16:12:43.517363+0000","last_clean":"2023-10-08T16:12:43.517363+0000","last_became_active":"2023-10-08T16:12:42.665310+0000","last_became_peered":"2023-10-08T16:12:42.665310+0000","last_unstale":"2023-10-08T16:12:43.517363+0000","last_undegraded":"2023-10-08T16:12:43.517363+0000","last_fullsized":"2023-10-08T16:12:43.517363+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:06:50.209391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528891+0000","last_change":"2023-10-08T16:12:43.528891+0000","last_active":"2023-10-08T16:12:43.528891+0000","last_peered":"2023-10-08T16:12:43.528891+0000","last_clean":"2023-10-08T16:12:43.528891+0000","last_became_active":"2023-10-08T16:12:43.528650+0000","last_became_peered":"2023-10-08T16:12:43.528650+0000","last_unstale":"2023-10-08T16:12:43.528891+0000","last_undegraded":"2023-10-08T16:12:43.528891+0000","last_fullsized":"2023-10-08T16:12:43.528891+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:37:37.684541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527588+0000","last_change":"2023-10-08T16:12:42.666801+0000","last_active":"2023-10-08T16:12:43.527588+0000","last_peered":"2023-10-08T16:12:43.527588+0000","last_clean":"2023-10-08T16:12:43.527588+0000","last_became_active":"2023-10-08T16:12:42.666522+0000","last_became_peered":"2023-10-08T16:12:42.666522+0000","last_unstale":"2023-10-08T16:12:43.527588+0000","last_undegraded":"2023-10-08T16:12:43.527588+0000","last_fullsized":"2023-10-08T16:12:43.527588+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:04:20.232268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521135+0000","last_change":"2023-10-08T16:12:43.521135+0000","last_active":"2023-10-08T16:12:43.521135+0000","last_peered":"2023-10-08T16:12:43.521135+0000","last_clean":"2023-10-08T16:12:43.521135+0000","last_became_active":"2023-10-08T16:12:43.520879+0000","last_became_peered":"2023-10-08T16:12:43.520879+0000","last_unstale":"2023-10-08T16:12:43.521135+0000","last_undegraded":"2023-10-08T16:12:43.521135+0000","last_fullsized":"2023-10-08T16:12:43.521135+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:36.777496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670317+0000","last_change":"2023-10-08T16:12:42.668295+0000","last_active":"2023-10-08T16:12:42.670317+0000","last_peered":"2023-10-08T16:12:42.670317+0000","last_clean":"2023-10-08T16:12:42.670317+0000","last_became_active":"2023-10-08T16:12:42.667998+0000","last_became_peered":"2023-10-08T16:12:42.667998+0000","last_unstale":"2023-10-08T16:12:42.670317+0000","last_undegraded":"2023-10-08T16:12:42.670317+0000","last_fullsized":"2023-10-08T16:12:42.670317+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:12.251487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530179+0000","last_change":"2023-10-08T16:12:43.530179+0000","last_active":"2023-10-08T16:12:43.530179+0000","last_peered":"2023-10-08T16:12:43.530179+0000","last_clean":"2023-10-08T16:12:43.530179+0000","last_became_active":"2023-10-08T16:12:43.529897+0000","last_became_peered":"2023-10-08T16:12:43.529897+0000","last_unstale":"2023-10-08T16:12:43.530179+0000","last_undegraded":"2023-10-08T16:12:43.530179+0000","last_fullsized":"2023-10-08T16:12:43.530179+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:01.776870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664547+0000","last_change":"2023-10-08T16:12:42.663795+0000","last_active":"2023-10-08T16:12:42.664547+0000","last_peered":"2023-10-08T16:12:42.664547+0000","last_clean":"2023-10-08T16:12:42.664547+0000","last_became_active":"2023-10-08T16:12:42.663529+0000","last_became_peered":"2023-10-08T16:12:42.663529+0000","last_unstale":"2023-10-08T16:12:42.664547+0000","last_undegraded":"2023-10-08T16:12:42.664547+0000","last_fullsized":"2023-10-08T16:12:42.664547+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:09.739973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531393+0000","last_change":"2023-10-08T16:12:43.531393+0000","last_active":"2023-10-08T16:12:43.531393+0000","last_peered":"2023-10-08T16:12:43.531393+0000","last_clean":"2023-10-08T16:12:43.531393+0000","last_became_active":"2023-10-08T16:12:43.530826+0000","last_became_peered":"2023-10-08T16:12:43.530826+0000","last_unstale":"2023-10-08T16:12:43.531393+0000","last_undegraded":"2023-10-08T16:12:43.531393+0000","last_fullsized":"2023-10-08T16:12:43.531393+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:14:16.673225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672549+0000","last_change":"2023-10-08T16:12:42.668796+0000","last_active":"2023-10-08T16:12:42.672549+0000","last_peered":"2023-10-08T16:12:42.672549+0000","last_clean":"2023-10-08T16:12:42.672549+0000","last_became_active":"2023-10-08T16:12:42.668432+0000","last_became_peered":"2023-10-08T16:12:42.668432+0000","last_unstale":"2023-10-08T16:12:42.672549+0000","last_undegraded":"2023-10-08T16:12:42.672549+0000","last_fullsized":"2023-10-08T16:12:42.672549+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:54:37.903831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526946+0000","last_change":"2023-10-08T16:12:43.526946+0000","last_active":"2023-10-08T16:12:43.526946+0000","last_peered":"2023-10-08T16:12:43.526946+0000","last_clean":"2023-10-08T16:12:43.526946+0000","last_became_active":"2023-10-08T16:12:43.526588+0000","last_became_peered":"2023-10-08T16:12:43.526588+0000","last_unstale":"2023-10-08T16:12:43.526946+0000","last_undegraded":"2023-10-08T16:12:43.526946+0000","last_fullsized":"2023-10-08T16:12:43.526946+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:48.744850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664070+0000","last_change":"2023-10-08T16:12:42.663411+0000","last_active":"2023-10-08T16:12:42.664070+0000","last_peered":"2023-10-08T16:12:42.664070+0000","last_clean":"2023-10-08T16:12:42.664070+0000","last_became_active":"2023-10-08T16:12:42.663160+0000","last_became_peered":"2023-10-08T16:12:42.663160+0000","last_unstale":"2023-10-08T16:12:42.664070+0000","last_undegraded":"2023-10-08T16:12:42.664070+0000","last_fullsized":"2023-10-08T16:12:42.664070+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:30.735713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.532327+0000","last_change":"2023-10-08T16:12:43.532327+0000","last_active":"2023-10-08T16:12:43.532327+0000","last_peered":"2023-10-08T16:12:43.532327+0000","last_clean":"2023-10-08T16:12:43.532327+0000","last_became_active":"2023-10-08T16:12:43.531889+0000","last_became_peered":"2023-10-08T16:12:43.531889+0000","last_unstale":"2023-10-08T16:12:43.532327+0000","last_undegraded":"2023-10-08T16:12:43.532327+0000","last_fullsized":"2023-10-08T16:12:43.532327+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.137973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"165'3","reported_seq":12,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.682795+0000","last_change":"2023-10-08T16:12:42.661354+0000","last_active":"2023-10-08T16:12:43.682795+0000","last_peered":"2023-10-08T16:12:43.682795+0000","last_clean":"2023-10-08T16:12:43.682795+0000","last_became_active":"2023-10-08T16:12:42.661094+0000","last_became_peered":"2023-10-08T16:12:42.661094+0000","last_unstale":"2023-10-08T16:12:43.682795+0000","last_undegraded":"2023-10-08T16:12:43.682795+0000","last_fullsized":"2023-10-08T16:12:43.682795+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:27.586399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534289+0000","last_change":"2023-10-08T16:12:43.534289+0000","last_active":"2023-10-08T16:12:43.534289+0000","last_peered":"2023-10-08T16:12:43.534289+0000","last_clean":"2023-10-08T16:12:43.534289+0000","last_became_active":"2023-10-08T16:12:43.534037+0000","last_became_peered":"2023-10-08T16:12:43.534037+0000","last_unstale":"2023-10-08T16:12:43.534289+0000","last_undegraded":"2023-10-08T16:12:43.534289+0000","last_fullsized":"2023-10-08T16:12:43.534289+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:54:31.737355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"165'6","reported_seq":15,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.679488+0000","last_change":"2023-10-08T16:12:42.662964+0000","last_active":"2023-10-08T16:12:43.679488+0000","last_peered":"2023-10-08T16:12:43.679488+0000","last_clean":"2023-10-08T16:12:43.679488+0000","last_became_active":"2023-10-08T16:12:42.662725+0000","last_became_peered":"2023-10-08T16:12:42.662725+0000","last_unstale":"2023-10-08T16:12:43.679488+0000","last_undegraded":"2023-10-08T16:12:43.679488+0000","last_fullsized":"2023-10-08T16:12:43.679488+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:16:42.535018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525485+0000","last_change":"2023-10-08T16:12:43.525485+0000","last_active":"2023-10-08T16:12:43.525485+0000","last_peered":"2023-10-08T16:12:43.525485+0000","last_clean":"2023-10-08T16:12:43.525485+0000","last_became_active":"2023-10-08T16:12:43.525220+0000","last_became_peered":"2023-10-08T16:12:43.525220+0000","last_unstale":"2023-10-08T16:12:43.525485+0000","last_undegraded":"2023-10-08T16:12:43.525485+0000","last_fullsized":"2023-10-08T16:12:43.525485+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:21:46.674446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.680970+0000","last_change":"2023-10-08T16:12:42.668813+0000","last_active":"2023-10-08T16:12:43.680970+0000","last_peered":"2023-10-08T16:12:43.680970+0000","last_clean":"2023-10-08T16:12:43.680970+0000","last_became_active":"2023-10-08T16:12:42.668439+0000","last_became_peered":"2023-10-08T16:12:42.668439+0000","last_unstale":"2023-10-08T16:12:43.680970+0000","last_undegraded":"2023-10-08T16:12:43.680970+0000","last_fullsized":"2023-10-08T16:12:43.680970+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:55:36.915056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528823+0000","last_change":"2023-10-08T16:12:43.528823+0000","last_active":"2023-10-08T16:12:43.528823+0000","last_peered":"2023-10-08T16:12:43.528823+0000","last_clean":"2023-10-08T16:12:43.528823+0000","last_became_active":"2023-10-08T16:12:43.528532+0000","last_became_peered":"2023-10-08T16:12:43.528532+0000","last_unstale":"2023-10-08T16:12:43.528823+0000","last_undegraded":"2023-10-08T16:12:43.528823+0000","last_fullsized":"2023-10-08T16:12:43.528823+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:56:56.730218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664094+0000","last_change":"2023-10-08T16:12:42.663427+0000","last_active":"2023-10-08T16:12:42.664094+0000","last_peered":"2023-10-08T16:12:42.664094+0000","last_clean":"2023-10-08T16:12:42.664094+0000","last_became_active":"2023-10-08T16:12:42.663195+0000","last_became_peered":"2023-10-08T16:12:42.663195+0000","last_unstale":"2023-10-08T16:12:42.664094+0000","last_undegraded":"2023-10-08T16:12:42.664094+0000","last_fullsized":"2023-10-08T16:12:42.664094+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:51:47.743209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.529082+0000","last_change":"2023-10-08T16:12:43.529082+0000","last_active":"2023-10-08T16:12:43.529082+0000","last_peered":"2023-10-08T16:12:43.529082+0000","last_clean":"2023-10-08T16:12:43.529082+0000","last_became_active":"2023-10-08T16:12:43.528805+0000","last_became_peered":"2023-10-08T16:12:43.528805+0000","last_unstale":"2023-10-08T16:12:43.529082+0000","last_undegraded":"2023-10-08T16:12:43.529082+0000","last_fullsized":"2023-10-08T16:12:43.529082+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:13.022948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.664020+0000","last_change":"2023-10-08T16:12:42.662469+0000","last_active":"2023-10-08T16:12:42.664020+0000","last_peered":"2023-10-08T16:12:42.664020+0000","last_clean":"2023-10-08T16:12:42.664020+0000","last_became_active":"2023-10-08T16:12:42.662212+0000","last_became_peered":"2023-10-08T16:12:42.662212+0000","last_unstale":"2023-10-08T16:12:42.664020+0000","last_undegraded":"2023-10-08T16:12:42.664020+0000","last_fullsized":"2023-10-08T16:12:42.664020+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:08:26.498871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527406+0000","last_change":"2023-10-08T16:12:43.527406+0000","last_active":"2023-10-08T16:12:43.527406+0000","last_peered":"2023-10-08T16:12:43.527406+0000","last_clean":"2023-10-08T16:12:43.527406+0000","last_became_active":"2023-10-08T16:12:43.525012+0000","last_became_peered":"2023-10-08T16:12:43.525012+0000","last_unstale":"2023-10-08T16:12:43.527406+0000","last_undegraded":"2023-10-08T16:12:43.527406+0000","last_fullsized":"2023-10-08T16:12:43.527406+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:31:16.572084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"165'8","reported_seq":17,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683113+0000","last_change":"2023-10-08T16:12:42.664293+0000","last_active":"2023-10-08T16:12:43.683113+0000","last_peered":"2023-10-08T16:12:43.683113+0000","last_clean":"2023-10-08T16:12:43.683113+0000","last_became_active":"2023-10-08T16:12:42.662935+0000","last_became_peered":"2023-10-08T16:12:42.662935+0000","last_unstale":"2023-10-08T16:12:43.683113+0000","last_undegraded":"2023-10-08T16:12:43.683113+0000","last_fullsized":"2023-10-08T16:12:43.683113+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:12.021306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523804+0000","last_change":"2023-10-08T16:12:43.523804+0000","last_active":"2023-10-08T16:12:43.523804+0000","last_peered":"2023-10-08T16:12:43.523804+0000","last_clean":"2023-10-08T16:12:43.523804+0000","last_became_active":"2023-10-08T16:12:43.523538+0000","last_became_peered":"2023-10-08T16:12:43.523538+0000","last_unstale":"2023-10-08T16:12:43.523804+0000","last_undegraded":"2023-10-08T16:12:43.523804+0000","last_fullsized":"2023-10-08T16:12:43.523804+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:53:12.788019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.527791+0000","last_change":"2023-10-08T16:12:42.669802+0000","last_active":"2023-10-08T16:12:43.527791+0000","last_peered":"2023-10-08T16:12:43.527791+0000","last_clean":"2023-10-08T16:12:43.527791+0000","last_became_active":"2023-10-08T16:12:42.669512+0000","last_became_peered":"2023-10-08T16:12:42.669512+0000","last_unstale":"2023-10-08T16:12:43.527791+0000","last_undegraded":"2023-10-08T16:12:43.527791+0000","last_fullsized":"2023-10-08T16:12:43.527791+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:10:08.755528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.534206+0000","last_change":"2023-10-08T16:12:43.534206+0000","last_active":"2023-10-08T16:12:43.534206+0000","last_peered":"2023-10-08T16:12:43.534206+0000","last_clean":"2023-10-08T16:12:43.534206+0000","last_became_active":"2023-10-08T16:12:43.533960+0000","last_became_peered":"2023-10-08T16:12:43.533960+0000","last_unstale":"2023-10-08T16:12:43.534206+0000","last_undegraded":"2023-10-08T16:12:43.534206+0000","last_fullsized":"2023-10-08T16:12:43.534206+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:01.346163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.680397+0000","last_change":"2023-10-08T16:12:42.666968+0000","last_active":"2023-10-08T16:12:43.680397+0000","last_peered":"2023-10-08T16:12:43.680397+0000","last_clean":"2023-10-08T16:12:43.680397+0000","last_became_active":"2023-10-08T16:12:42.666717+0000","last_became_peered":"2023-10-08T16:12:42.666717+0000","last_unstale":"2023-10-08T16:12:43.680397+0000","last_undegraded":"2023-10-08T16:12:43.680397+0000","last_fullsized":"2023-10-08T16:12:43.680397+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:46.500022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.531359+0000","last_change":"2023-10-08T16:12:43.531359+0000","last_active":"2023-10-08T16:12:43.531359+0000","last_peered":"2023-10-08T16:12:43.531359+0000","last_clean":"2023-10-08T16:12:43.531359+0000","last_became_active":"2023-10-08T16:12:43.530989+0000","last_became_peered":"2023-10-08T16:12:43.530989+0000","last_unstale":"2023-10-08T16:12:43.531359+0000","last_undegraded":"2023-10-08T16:12:43.531359+0000","last_fullsized":"2023-10-08T16:12:43.531359+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:11:03.905560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"165'1","reported_seq":10,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520838+0000","last_change":"2023-10-08T16:12:42.669007+0000","last_active":"2023-10-08T16:12:43.520838+0000","last_peered":"2023-10-08T16:12:43.520838+0000","last_clean":"2023-10-08T16:12:43.520838+0000","last_became_active":"2023-10-08T16:12:42.668333+0000","last_became_peered":"2023-10-08T16:12:42.668333+0000","last_unstale":"2023-10-08T16:12:43.520838+0000","last_undegraded":"2023-10-08T16:12:43.520838+0000","last_fullsized":"2023-10-08T16:12:43.520838+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:15.661851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.530650+0000","last_change":"2023-10-08T16:12:43.530650+0000","last_active":"2023-10-08T16:12:43.530650+0000","last_peered":"2023-10-08T16:12:43.530650+0000","last_clean":"2023-10-08T16:12:43.530650+0000","last_became_active":"2023-10-08T16:12:43.530260+0000","last_became_peered":"2023-10-08T16:12:43.530260+0000","last_unstale":"2023-10-08T16:12:43.530650+0000","last_undegraded":"2023-10-08T16:12:43.530650+0000","last_fullsized":"2023-10-08T16:12:43.530650+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:07:16.888336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.668176+0000","last_change":"2023-10-08T16:12:42.667690+0000","last_active":"2023-10-08T16:12:42.668176+0000","last_peered":"2023-10-08T16:12:42.668176+0000","last_clean":"2023-10-08T16:12:42.668176+0000","last_became_active":"2023-10-08T16:12:42.667372+0000","last_became_peered":"2023-10-08T16:12:42.667372+0000","last_unstale":"2023-10-08T16:12:42.668176+0000","last_undegraded":"2023-10-08T16:12:42.668176+0000","last_fullsized":"2023-10-08T16:12:42.668176+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:53:48.102950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526334+0000","last_change":"2023-10-08T16:12:43.526334+0000","last_active":"2023-10-08T16:12:43.526334+0000","last_peered":"2023-10-08T16:12:43.526334+0000","last_clean":"2023-10-08T16:12:43.526334+0000","last_became_active":"2023-10-08T16:12:43.526025+0000","last_became_peered":"2023-10-08T16:12:43.526025+0000","last_unstale":"2023-10-08T16:12:43.526334+0000","last_undegraded":"2023-10-08T16:12:43.526334+0000","last_fullsized":"2023-10-08T16:12:43.526334+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:45:16.663493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683029+0000","last_change":"2023-10-08T16:12:42.669361+0000","last_active":"2023-10-08T16:12:43.683029+0000","last_peered":"2023-10-08T16:12:43.683029+0000","last_clean":"2023-10-08T16:12:43.683029+0000","last_became_active":"2023-10-08T16:12:42.669018+0000","last_became_peered":"2023-10-08T16:12:42.669018+0000","last_unstale":"2023-10-08T16:12:43.683029+0000","last_undegraded":"2023-10-08T16:12:43.683029+0000","last_fullsized":"2023-10-08T16:12:43.683029+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:38:22.997785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.533008+0000","last_change":"2023-10-08T16:12:43.533008+0000","last_active":"2023-10-08T16:12:43.533008+0000","last_peered":"2023-10-08T16:12:43.533008+0000","last_clean":"2023-10-08T16:12:43.533008+0000","last_became_active":"2023-10-08T16:12:43.532361+0000","last_became_peered":"2023-10-08T16:12:43.532361+0000","last_unstale":"2023-10-08T16:12:43.533008+0000","last_undegraded":"2023-10-08T16:12:43.533008+0000","last_fullsized":"2023-10-08T16:12:43.533008+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:35.318963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.662422+0000","last_change":"2023-10-08T16:12:42.657003+0000","last_active":"2023-10-08T16:12:42.662422+0000","last_peered":"2023-10-08T16:12:42.662422+0000","last_clean":"2023-10-08T16:12:42.662422+0000","last_became_active":"2023-10-08T16:12:42.656774+0000","last_became_peered":"2023-10-08T16:12:42.656774+0000","last_unstale":"2023-10-08T16:12:42.662422+0000","last_undegraded":"2023-10-08T16:12:42.662422+0000","last_fullsized":"2023-10-08T16:12:42.662422+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:42.277461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523830+0000","last_change":"2023-10-08T16:12:43.523830+0000","last_active":"2023-10-08T16:12:43.523830+0000","last_peered":"2023-10-08T16:12:43.523830+0000","last_clean":"2023-10-08T16:12:43.523830+0000","last_became_active":"2023-10-08T16:12:43.523547+0000","last_became_peered":"2023-10-08T16:12:43.523547+0000","last_unstale":"2023-10-08T16:12:43.523830+0000","last_undegraded":"2023-10-08T16:12:43.523830+0000","last_fullsized":"2023-10-08T16:12:43.523830+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:58:35.469782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672611+0000","last_change":"2023-10-08T16:12:42.669923+0000","last_active":"2023-10-08T16:12:42.672611+0000","last_peered":"2023-10-08T16:12:42.672611+0000","last_clean":"2023-10-08T16:12:42.672611+0000","last_became_active":"2023-10-08T16:12:42.669645+0000","last_became_peered":"2023-10-08T16:12:42.669645+0000","last_unstale":"2023-10-08T16:12:42.672611+0000","last_undegraded":"2023-10-08T16:12:42.672611+0000","last_fullsized":"2023-10-08T16:12:42.672611+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:15.539493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523118+0000","last_change":"2023-10-08T16:12:43.523118+0000","last_active":"2023-10-08T16:12:43.523118+0000","last_peered":"2023-10-08T16:12:43.523118+0000","last_clean":"2023-10-08T16:12:43.523118+0000","last_became_active":"2023-10-08T16:12:43.522852+0000","last_became_peered":"2023-10-08T16:12:43.522852+0000","last_unstale":"2023-10-08T16:12:43.523118+0000","last_undegraded":"2023-10-08T16:12:43.523118+0000","last_fullsized":"2023-10-08T16:12:43.523118+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:51:21.622599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670015+0000","last_change":"2023-10-08T16:12:42.665851+0000","last_active":"2023-10-08T16:12:42.670015+0000","last_peered":"2023-10-08T16:12:42.670015+0000","last_clean":"2023-10-08T16:12:42.670015+0000","last_became_active":"2023-10-08T16:12:42.665359+0000","last_became_peered":"2023-10-08T16:12:42.665359+0000","last_unstale":"2023-10-08T16:12:42.670015+0000","last_undegraded":"2023-10-08T16:12:42.670015+0000","last_fullsized":"2023-10-08T16:12:42.670015+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:48.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.521891+0000","last_change":"2023-10-08T16:12:43.521891+0000","last_active":"2023-10-08T16:12:43.521891+0000","last_peered":"2023-10-08T16:12:43.521891+0000","last_clean":"2023-10-08T16:12:43.521891+0000","last_became_active":"2023-10-08T16:12:43.521631+0000","last_became_peered":"2023-10-08T16:12:43.521631+0000","last_unstale":"2023-10-08T16:12:43.521891+0000","last_undegraded":"2023-10-08T16:12:43.521891+0000","last_fullsized":"2023-10-08T16:12:43.521891+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:35:20.078674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.683447+0000","last_change":"2023-10-08T16:12:42.670922+0000","last_active":"2023-10-08T16:12:43.683447+0000","last_peered":"2023-10-08T16:12:43.683447+0000","last_clean":"2023-10-08T16:12:43.683447+0000","last_became_active":"2023-10-08T16:12:42.670664+0000","last_became_peered":"2023-10-08T16:12:42.670664+0000","last_unstale":"2023-10-08T16:12:43.683447+0000","last_undegraded":"2023-10-08T16:12:43.683447+0000","last_fullsized":"2023-10-08T16:12:43.683447+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:05:53.610877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.537001+0000","last_change":"2023-10-08T16:12:43.537001+0000","last_active":"2023-10-08T16:12:43.537001+0000","last_peered":"2023-10-08T16:12:43.537001+0000","last_clean":"2023-10-08T16:12:43.537001+0000","last_became_active":"2023-10-08T16:12:43.536760+0000","last_became_peered":"2023-10-08T16:12:43.536760+0000","last_unstale":"2023-10-08T16:12:43.537001+0000","last_undegraded":"2023-10-08T16:12:43.537001+0000","last_fullsized":"2023-10-08T16:12:43.537001+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:37:56.882751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672558+0000","last_change":"2023-10-08T16:12:42.668813+0000","last_active":"2023-10-08T16:12:42.672558+0000","last_peered":"2023-10-08T16:12:42.672558+0000","last_clean":"2023-10-08T16:12:42.672558+0000","last_became_active":"2023-10-08T16:12:42.668488+0000","last_became_peered":"2023-10-08T16:12:42.668488+0000","last_unstale":"2023-10-08T16:12:42.672558+0000","last_undegraded":"2023-10-08T16:12:42.672558+0000","last_fullsized":"2023-10-08T16:12:42.672558+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:53.003952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.533385+0000","last_change":"2023-10-08T16:12:43.533385+0000","last_active":"2023-10-08T16:12:43.533385+0000","last_peered":"2023-10-08T16:12:43.533385+0000","last_clean":"2023-10-08T16:12:43.533385+0000","last_became_active":"2023-10-08T16:12:43.533023+0000","last_became_peered":"2023-10-08T16:12:43.533023+0000","last_unstale":"2023-10-08T16:12:43.533385+0000","last_undegraded":"2023-10-08T16:12:43.533385+0000","last_fullsized":"2023-10-08T16:12:43.533385+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:28.588041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.672697+0000","last_change":"2023-10-08T16:12:42.669067+0000","last_active":"2023-10-08T16:12:42.672697+0000","last_peered":"2023-10-08T16:12:42.672697+0000","last_clean":"2023-10-08T16:12:42.672697+0000","last_became_active":"2023-10-08T16:12:42.668708+0000","last_became_peered":"2023-10-08T16:12:42.668708+0000","last_unstale":"2023-10-08T16:12:42.672697+0000","last_undegraded":"2023-10-08T16:12:42.672697+0000","last_fullsized":"2023-10-08T16:12:42.672697+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:47:15.279636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.520816+0000","last_change":"2023-10-08T16:12:43.520816+0000","last_active":"2023-10-08T16:12:43.520816+0000","last_peered":"2023-10-08T16:12:43.520816+0000","last_clean":"2023-10-08T16:12:43.520816+0000","last_became_active":"2023-10-08T16:12:43.520575+0000","last_became_peered":"2023-10-08T16:12:43.520575+0000","last_unstale":"2023-10-08T16:12:43.520816+0000","last_undegraded":"2023-10-08T16:12:43.520816+0000","last_fullsized":"2023-10-08T16:12:43.520816+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:16:43.536660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.669965+0000","last_change":"2023-10-08T16:12:42.665234+0000","last_active":"2023-10-08T16:12:42.669965+0000","last_peered":"2023-10-08T16:12:42.669965+0000","last_clean":"2023-10-08T16:12:42.669965+0000","last_became_active":"2023-10-08T16:12:42.664918+0000","last_became_peered":"2023-10-08T16:12:42.664918+0000","last_unstale":"2023-10-08T16:12:42.669965+0000","last_undegraded":"2023-10-08T16:12:42.669965+0000","last_fullsized":"2023-10-08T16:12:42.669965+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:41:12.994594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.523243+0000","last_change":"2023-10-08T16:12:43.523243+0000","last_active":"2023-10-08T16:12:43.523243+0000","last_peered":"2023-10-08T16:12:43.523243+0000","last_clean":"2023-10-08T16:12:43.523243+0000","last_became_active":"2023-10-08T16:12:43.522703+0000","last_became_peered":"2023-10-08T16:12:43.522703+0000","last_unstale":"2023-10-08T16:12:43.523243+0000","last_undegraded":"2023-10-08T16:12:43.523243+0000","last_fullsized":"2023-10-08T16:12:43.523243+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:35:06.674016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.668156+0000","last_change":"2023-10-08T16:12:42.667341+0000","last_active":"2023-10-08T16:12:42.668156+0000","last_peered":"2023-10-08T16:12:42.668156+0000","last_clean":"2023-10-08T16:12:42.668156+0000","last_became_active":"2023-10-08T16:12:42.667077+0000","last_became_peered":"2023-10-08T16:12:42.667077+0000","last_unstale":"2023-10-08T16:12:42.668156+0000","last_undegraded":"2023-10-08T16:12:42.668156+0000","last_fullsized":"2023-10-08T16:12:42.668156+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:48.378386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.528784+0000","last_change":"2023-10-08T16:12:43.528784+0000","last_active":"2023-10-08T16:12:43.528784+0000","last_peered":"2023-10-08T16:12:43.528784+0000","last_clean":"2023-10-08T16:12:43.528784+0000","last_became_active":"2023-10-08T16:12:43.528497+0000","last_became_peered":"2023-10-08T16:12:43.528497+0000","last_unstale":"2023-10-08T16:12:43.528784+0000","last_undegraded":"2023-10-08T16:12:43.528784+0000","last_fullsized":"2023-10-08T16:12:43.528784+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:59:45.243014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.667245+0000","last_change":"2023-10-08T16:12:42.666338+0000","last_active":"2023-10-08T16:12:42.667245+0000","last_peered":"2023-10-08T16:12:42.667245+0000","last_clean":"2023-10-08T16:12:42.667245+0000","last_became_active":"2023-10-08T16:12:42.666081+0000","last_became_peered":"2023-10-08T16:12:42.666081+0000","last_unstale":"2023-10-08T16:12:42.667245+0000","last_undegraded":"2023-10-08T16:12:42.667245+0000","last_fullsized":"2023-10-08T16:12:42.667245+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:58:35.775855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.526794+0000","last_change":"2023-10-08T16:12:43.526794+0000","last_active":"2023-10-08T16:12:43.526794+0000","last_peered":"2023-10-08T16:12:43.526794+0000","last_clean":"2023-10-08T16:12:43.526794+0000","last_became_active":"2023-10-08T16:12:43.524473+0000","last_became_peered":"2023-10-08T16:12:43.524473+0000","last_unstale":"2023-10-08T16:12:43.526794+0000","last_undegraded":"2023-10-08T16:12:43.526794+0000","last_fullsized":"2023-10-08T16:12:43.526794+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:25:25.137973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.670042+0000","last_change":"2023-10-08T16:12:42.669549+0000","last_active":"2023-10-08T16:12:42.670042+0000","last_peered":"2023-10-08T16:12:42.670042+0000","last_clean":"2023-10-08T16:12:42.670042+0000","last_became_active":"2023-10-08T16:12:42.669240+0000","last_became_peered":"2023-10-08T16:12:42.669240+0000","last_unstale":"2023-10-08T16:12:42.670042+0000","last_undegraded":"2023-10-08T16:12:42.670042+0000","last_fullsized":"2023-10-08T16:12:42.670042+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:48:34.317322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.525048+0000","last_change":"2023-10-08T16:12:43.525048+0000","last_active":"2023-10-08T16:12:43.525048+0000","last_peered":"2023-10-08T16:12:43.525048+0000","last_clean":"2023-10-08T16:12:43.525048+0000","last_became_active":"2023-10-08T16:12:43.524557+0000","last_became_peered":"2023-10-08T16:12:43.524557+0000","last_unstale":"2023-10-08T16:12:43.525048+0000","last_undegraded":"2023-10-08T16:12:43.525048+0000","last_fullsized":"2023-10-08T16:12:43.525048+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:06:47.501664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.669990+0000","last_change":"2023-10-08T16:12:42.665648+0000","last_active":"2023-10-08T16:12:42.669990+0000","last_peered":"2023-10-08T16:12:42.669990+0000","last_clean":"2023-10-08T16:12:42.669990+0000","last_became_active":"2023-10-08T16:12:42.664969+0000","last_became_peered":"2023-10-08T16:12:42.664969+0000","last_unstale":"2023-10-08T16:12:42.669990+0000","last_undegraded":"2023-10-08T16:12:42.669990+0000","last_fullsized":"2023-10-08T16:12:42.669990+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:31:42.277461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":8,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.522736+0000","last_change":"2023-10-08T16:12:43.522736+0000","last_active":"2023-10-08T16:12:43.522736+0000","last_peered":"2023-10-08T16:12:43.522736+0000","last_clean":"2023-10-08T16:12:43.522736+0000","last_became_active":"2023-10-08T16:12:43.522486+0000","last_became_peered":"2023-10-08T16:12:43.522486+0000","last_unstale":"2023-10-08T16:12:43.522736+0000","last_undegraded":"2023-10-08T16:12:43.522736+0000","last_fullsized":"2023-10-08T16:12:43.522736+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":164,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:42.645144+0000","last_clean_scrub_stamp":"2023-10-08T16:12:42.645144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:21.529765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"165'2","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-10-08T16:12:43.682888+0000","last_change":"2023-10-08T16:12:42.669871+0000","last_active":"2023-10-08T16:12:43.682888+0000","last_peered":"2023-10-08T16:12:43.682888+0000","last_clean":"2023-10-08T16:12:43.682888+0000","last_became_active":"2023-10-08T16:12:42.669535+0000","last_became_peered":"2023-10-08T16:12:42.669535+0000","last_unstale":"2023-10-08T16:12:43.682888+0000","last_undegraded":"2023-10-08T16:12:43.682888+0000","last_fullsized":"2023-10-08T16:12:43.682888+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:12:41.643503+0000","last_clean_scrub_stamp":"2023-10-08T16:12:41.643503+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:10:26.747148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-10-08T16:12:42.663951+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:12:42.663951+0000","last_peered":"2023-10-08T16:12:42.663951+0000","last_clean":"2023-10-08T16:12:42.663951+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:12:42.663951+0000","last_undegraded":"2023-10-08T16:12:42.663951+0000","last_fullsized":"2023-10-08T16:12:42.663951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304012,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067828,"statfs":{"total":96636764160,"available":96325455872,"internally_reserved":0,"allocated":2691072,"data_stored":1525842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.86199999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72799999999999998},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.82199999999999995,"15min":1.083},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.64900000000000002,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.70299999999999996,"15min":1.175},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.77900000000000003},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.68300000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.374,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":1.677,"15min":1.677},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.78100000000000003},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63600000000000001,"5min":0.69899999999999995,"15min":8.3409999999999993},"last":0.71799999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44400000000000001,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.753,"15min":0.753},"last":0.70399999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.67000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303436,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068404,"statfs":{"total":96636764160,"available":96326045696,"internally_reserved":0,"allocated":2101248,"data_stored":935460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50700000000000001,"15min":0.49099999999999999},"min":{"1min":0.36699999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.64900000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.29499999999999998,"15min":0.29299999999999998},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.90600000000000003},"last":0.42099999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51500000000000001,"15min":0.495},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52600000000000002,"15min":0.53500000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.747,"5min":0.76300000000000001,"15min":0.76700000000000002},"last":0.61499999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.53800000000000003},"min":{"1min":0.35099999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.504},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71199999999999997,"5min":0.75,"15min":0.96599999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68600000000000005,"5min":0.745,"15min":0.94199999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.69599999999999995,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48399999999999999,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.53200000000000003},"min":{"1min":0.314,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.73099999999999998,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.60399999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49399999999999999,"15min":0.52100000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72299999999999998,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56200000000000006,"15min":0.58199999999999996},"min":{"1min":0.377,"5min":0.311,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.874},"last":0.44}]},{"osd":7,"last update":"Sun Oct 8 16:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.59299999999999997},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77200000000000002,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58599999999999997,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307528,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064312,"statfs":{"total":96636764160,"available":96321855488,"internally_reserved":0,"allocated":2097152,"data_stored":935494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.30299999999999999,"15min":0.311},"min":{"1min":0.20599999999999999,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.249},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"min":{"1min":0.23699999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.77500000000000002},"last":0.26600000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.33300000000000002,"15min":0.35699999999999998},"min":{"1min":0.23499999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.373},"min":{"1min":0.24199999999999999,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.72399999999999998},"last":0.23499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.33700000000000002,"15min":0.35899999999999999},"min":{"1min":0.245,"5min":0.215,"15min":0.215},"max":{"1min":0.52700000000000002,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.32000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80500000000000005,"5min":0.84099999999999997,"15min":1.077},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.375,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.90000000000000002,"5min":1.696,"15min":1.696},"last":0.46200000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85299999999999998,"5min":1.657,"15min":1.657},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.38900000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72099999999999997,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.38600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.45400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88600000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:11:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.377,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.59599999999999997},"min":{"1min":0.379,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.875,"5min":0.875,"15min":1.1839999999999999},"last":0.41899999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304016,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067824,"statfs":{"total":96636764160,"available":96325451776,"internally_reserved":0,"allocated":2695168,"data_stored":1526416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.31,"15min":0.51100000000000001},"min":{"1min":0.216,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":0.372,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.317,"5min":0.32200000000000001,"15min":0.45400000000000001},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.38900000000000001,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.35899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.54400000000000004},"min":{"1min":0.25,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.46400000000000002,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.375},"min":{"1min":0.27900000000000003,"5min":0.219,"15min":0.219},"max":{"1min":0.432,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.41899999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.33600000000000002,"15min":0.46400000000000002},"min":{"1min":0.25700000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.41299999999999998,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.375},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.377,"15min":0.58099999999999996},"min":{"1min":0.28199999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.44700000000000001,"5min":1.498,"15min":5.5970000000000004},"last":0.40799999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53200000000000003,"15min":0.54800000000000004},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.81699999999999995,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54500000000000004,"15min":0.65000000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.75800000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.47599999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.847,"15min":3.2890000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55500000000000005,"15min":0.75600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.49099999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58899999999999997,"15min":0.77900000000000003},"min":{"1min":0.39300000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78300000000000003,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.44400000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.749,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59799999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.57999999999999996,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.78800000000000003,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59299999999999997,"15min":0.71899999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.73299999999999998,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.45700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303448,"kb_used_data":2064,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068392,"statfs":{"total":96636764160,"available":96326033408,"internally_reserved":0,"allocated":2113536,"data_stored":936996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.307},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.46400000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.32400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.34699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.44400000000000001,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.35999999999999999},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.34200000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48999999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.313}]},{"osd":4,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.48999999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.503},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81499999999999995,"15min":4.1029999999999998},"last":0.38500000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74399999999999999,"5min":0.85199999999999998,"15min":0.871},"last":0.52600000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.47299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303444,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068396,"statfs":{"total":96636764160,"available":96326037504,"internally_reserved":0,"allocated":2109440,"data_stored":937010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.314,"5min":0.312,"15min":0.34799999999999998},"min":{"1min":0.20100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.48699999999999999,"5min":0.53600000000000003,"15min":1.238},"last":0.246},{"interface":"front","average":{"1min":0.309,"5min":0.32900000000000001,"15min":0.38100000000000001},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.40799999999999997,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.27300000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.35499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45500000000000002,"5min":0.53300000000000003,"15min":1.0269999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35799999999999998,"15min":0.40200000000000002},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.29599999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.39400000000000002},"min":{"1min":0.34599999999999997,"5min":0.254,"15min":0.248},"max":{"1min":0.53900000000000003,"5min":0.60299999999999998,"15min":1.1399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.38500000000000001,"15min":0.41699999999999998},"min":{"1min":0.29699999999999999,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":1.327},"last":0.34000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.59699999999999998},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.71299999999999997,"5min":0.95599999999999996,"15min":1.9119999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39500000000000002,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.77800000000000002,"5min":1.099,"15min":1.099},"last":0.47999999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.41499999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.70599999999999996,"5min":0.77900000000000003,"15min":1.27},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.61199999999999999},"min":{"1min":0.46700000000000003,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.76300000000000001,"5min":0.91900000000000004,"15min":1.879},"last":0.46800000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.60499999999999998},"min":{"1min":0.42899999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.81399999999999995,"5min":0.94599999999999995,"15min":1.0620000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60899999999999999,"15min":0.622},"min":{"1min":0.50700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.60999999999999999,"15min":0.64100000000000001},"min":{"1min":0.48499999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.97599999999999998,"15min":1.825},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60499999999999998,"15min":0.627},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82899999999999996,"5min":1.0009999999999999,"15min":1.115},"last":0.434}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303432,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068408,"statfs":{"total":96636764160,"available":96326049792,"internally_reserved":0,"allocated":2097152,"data_stored":935958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.52600000000000002},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61499999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.40000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58499999999999996}]},{"osd":2,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.66000000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":0.83399999999999996},"last":0.63200000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.60099999999999998},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.34899999999999998},"max":{"1min":0.80400000000000005,"5min":0.875,"15min":0.875},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.89200000000000002},"last":0.64800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303428,"kb_used_data":2044,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068412,"statfs":{"total":96636764160,"available":96326053888,"internally_reserved":0,"allocated":2093056,"data_stored":935404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53100000000000003,"15min":0.63600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.52000000000000002,"15min":0.63},"min":{"1min":0.33900000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.80100000000000005,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51300000000000001,"15min":0.96199999999999997},"min":{"1min":0.34699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.84299999999999997,"5min":0.84399999999999997,"15min":12.227},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.871,"5min":0.871,"15min":1.252},"last":0.66600000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.65100000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.74299999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.376,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.63}]},{"osd":3,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.92300000000000004},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":9.3170000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.69099999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.879,"5min":0.879,"15min":3.8719999999999999},"last":0.65000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.93799999999999994},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52300000000000002,"15min":0.52200000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.38},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:12:48.114 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:12:48.114 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:12:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.269+0000 7f857bfff700 1 Processor -- start 2023-10-08T16:12:48.270 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.269+0000 7f857bfff700 1 -- start start 2023-10-08T16:12:48.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f857c1595d0 0x7f857c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857bfff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c0b10f0 con 0x7f857c15d930 2023-10-08T16:12:48.272 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857bfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857c0cd530 con 0x7f857c0cf480 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857bfff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857c15e000 con 0x7f857c1595d0 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f857c15d930 0x7f857c0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39426/0 (socket says 172.21.15.170:39426) 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857b7fe700 1 -- 172.21.15.170:0/2209711976 learned_addr learned my addr 172.21.15.170:0/2209711976 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f857affd700 1 --2- 172.21.15.170:0/2209711976 >> v2:172.21.15.191:3300/0 conn(0x7f857c1595d0 0x7f857c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.270+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105004217 0 0) 0x7f857c0b10f0 con 0x7f857c15d930 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8564003570 con 0x7f857c15d930 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439079318 0 0) 0x7f857c0cd530 con 0x7f857c0cf480 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f857c0b10f0 con 0x7f857c0cf480 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857affd700 1 -- 172.21.15.170:0/2209711976 >> v1:172.21.15.191:6789/0 conn(0x7f857c0cf480 legacy=0x7f857c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857affd700 1 -- 172.21.15.170:0/2209711976 >> v1:172.21.15.170:6789/0 conn(0x7f857c15d930 legacy=0x7f857c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.273 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857affd700 1 -- 172.21.15.170:0/2209711976 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0cf990 con 0x7f857c1595d0 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857affd700 1 --2- 172.21.15.170:0/2209711976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c1595d0 0x7f857c0ccff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f856c0098d0 tx=0x7f856c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=434211b3c02943f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c005c10 con 0x7f857c1595d0 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c005d70 con 0x7f857c1595d0 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f8579ffb700 1 -- 172.21.15.170:0/2209711976 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c006260 con 0x7f857c1595d0 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857bfff700 1 -- 172.21.15.170:0/2209711976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c1595d0 msgr2=0x7f857c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857bfff700 1 --2- 172.21.15.170:0/2209711976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c1595d0 0x7f857c0ccff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f856c0098d0 tx=0x7f856c009120 comp rx=0 tx=0).stop 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857bfff700 1 -- 172.21.15.170:0/2209711976 shutdown_connections 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857bfff700 1 --2- 172.21.15.170:0/2209711976 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c1595d0 0x7f857c0ccff0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.271+0000 7f857bfff700 1 -- 172.21.15.170:0/2209711976 >> 172.21.15.170:0/2209711976 conn(0x7f857c0d05f0 msgr2=0x7f857c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.272+0000 7f857bfff700 1 -- 172.21.15.170:0/2209711976 shutdown_connections 2023-10-08T16:12:48.274 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.272+0000 7f857bfff700 1 -- 172.21.15.170:0/2209711976 wait complete. 2023-10-08T16:12:48.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.273+0000 7f857bfff700 1 Processor -- start 2023-10-08T16:12:48.277 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.273+0000 7f857bfff700 1 -- start start 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.273+0000 7f857bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f857c1595d0 0x7f857c144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c15d930 0x7f857c155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c157100 con 0x7f857c0cf480 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c156270 con 0x7f857c15d930 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857c1564f0 con 0x7f857c1595d0 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32832/0 (socket says 172.21.15.170:32832) 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 -- 172.21.15.170:0/11229246 learned_addr learned my addr 172.21.15.170:0/11229246 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857a7fc700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f857c1595d0 0x7f857c144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.278 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857b7fe700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c15d930 0x7f857c155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 -- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f857c1595d0 msgr2=0x7f857c144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f857c1595d0 0x7f857c144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 -- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c15d930 msgr2=0x7f857c155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c15d930 0x7f857c155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 -- 172.21.15.170:0/11229246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c0093c0 con 0x7f857c0cf480 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857affd700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f857c0895f0 tx=0x7f856c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c01d9e0 con 0x7f857c0cf480 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c01db60 con 0x7f857c0cf480 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c1567d0 con 0x7f857c0cf480 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c01b480 con 0x7f857c0cf480 2023-10-08T16:12:48.279 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.274+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c156d90 con 0x7f857c0cf480 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.275+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c04c3c0 con 0x7f857c0cf480 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.276+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f856c01b7f0 con 0x7f857c0cf480 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.276+0000 7f8562ffd700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 0x7f85680697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.276+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f856c086c10 con 0x7f857c0cf480 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.276+0000 7f857a7fc700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 0x7f85680697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.276+0000 7f857a7fc700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 0x7f85680697d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f857c15ae00 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.280 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.280+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f856c0538a0 con 0x7f857c0cf480 2023-10-08T16:12:48.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.541+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f857c0b3860 con 0x7f857c0cf480 2023-10-08T16:12:48.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.544+0000 7f8562ffd700 1 -- 172.21.15.170:0/11229246 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+0 (secure 0 0 0) 0x7f856c056ec0 con 0x7f857c0cf480 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 msgr2=0x7f85680697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 0x7f85680697d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f857c15ae00 tx=0x7f8570009040 comp rx=0 tx=0).stop 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 msgr2=0x7f857c158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f857c0895f0 tx=0x7f856c016000 comp rx=0 tx=0).stop 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 shutdown_connections 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f857c15d930 0x7f857c155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8568067320 0x7f85680697d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f857c0cf480 0x7f857c158780 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 --2- 172.21.15.170:0/11229246 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f857c1595d0 0x7f857c144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 >> 172.21.15.170:0/11229246 conn(0x7f857c0d05f0 msgr2=0x7f857c152dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:48.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 shutdown_connections 2023-10-08T16:12:48.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.546+0000 7f857bfff700 1 -- 172.21.15.170:0/11229246 wait complete. 2023-10-08T16:12:48.548 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:12:48.559 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:48.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.673+0000 7f32483b1700 1 Processor -- start 2023-10-08T16:12:48.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.673+0000 7f32483b1700 1 -- start start 2023-10-08T16:12:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f32483b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 0x7f324015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f32483b1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f324015a160 con 0x7f3240159c60 2023-10-08T16:12:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f32483b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400d13a0 con 0x7f3240069560 2023-10-08T16:12:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f32483b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f324015e140 con 0x7f3240159890 2023-10-08T16:12:48.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f324594c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3240159c60 0x7f324015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39440/0 (socket says 172.21.15.170:39440) 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f324594c700 1 -- 172.21.15.170:0/2536727417 learned_addr learned my addr 172.21.15.170:0/2536727417 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f324614d700 1 --2- 172.21.15.170:0/2536727417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 0x7f324015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94141966 0 0) 0x7f324015a160 con 0x7f3240159c60 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.674+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f322c003570 con 0x7f3240159c60 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353751371 0 0) 0x7f322c003570 con 0x7f3240159c60 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324614d700 1 -- 172.21.15.170:0/2536727417 >> v1:172.21.15.191:6789/0 conn(0x7f3240069560 legacy=0x7f32400ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324614d700 1 -- 172.21.15.170:0/2536727417 >> v1:172.21.15.170:6789/0 conn(0x7f3240159c60 legacy=0x7f324015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324614d700 1 -- 172.21.15.170:0/2536727417 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324015a3d0 con 0x7f3240159890 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324614d700 1 --2- 172.21.15.170:0/2536727417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 0x7f324015abd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3238009e90 tx=0x7f3238009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593d3eda22df5ccc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323800e070 con 0x7f3240159890 2023-10-08T16:12:48.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323800b920 con 0x7f3240159890 2023-10-08T16:12:48.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f324514b700 1 -- 172.21.15.170:0/2536727417 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230001470 con 0x7f3240159890 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f32483b1700 1 -- 172.21.15.170:0/2536727417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 msgr2=0x7f324015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.675+0000 7f32483b1700 1 --2- 172.21.15.170:0/2536727417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 0x7f324015abd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3238009e90 tx=0x7f3238009780 comp rx=0 tx=0).stop 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.676+0000 7f32483b1700 1 -- 172.21.15.170:0/2536727417 shutdown_connections 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.676+0000 7f32483b1700 1 --2- 172.21.15.170:0/2536727417 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240159890 0x7f324015abd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.676+0000 7f32483b1700 1 -- 172.21.15.170:0/2536727417 >> 172.21.15.170:0/2536727417 conn(0x7f3240068e20 msgr2=0x7f32400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.676+0000 7f32483b1700 1 -- 172.21.15.170:0/2536727417 shutdown_connections 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.676+0000 7f32483b1700 1 -- 172.21.15.170:0/2536727417 wait complete. 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 Processor -- start 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 -- start start 2023-10-08T16:12:48.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 0x7f3240169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3240159890 0x7f32400cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32400b3b00 con 0x7f3240159c60 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32400b3de0 con 0x7f3240159890 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.677+0000 7f32483b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32400b40c0 con 0x7f3240069560 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32844/0 (socket says 172.21.15.170:32844) 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 -- 172.21.15.170:0/3671397048 learned_addr learned my addr 172.21.15.170:0/3671397048 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324614d700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 0x7f3240169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324594c700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3240159890 0x7f32400cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 -- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 msgr2=0x7f3240169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 0x7f3240169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 -- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3240159890 msgr2=0x7f32400cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3240159890 0x7f32400cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 -- 172.21.15.170:0/3671397048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238009a20 con 0x7f3240159c60 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324614d700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 0x7f3240169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f324694e700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c600 tx=0x7f323c007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c011590 con 0x7f3240159c60 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323c0116f0 con 0x7f3240159c60 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323c011ab0 con 0x7f3240159c60 2023-10-08T16:12:48.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32400b43a0 con 0x7f3240159c60 2023-10-08T16:12:48.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.678+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32400cfe10 con 0x7f3240159c60 2023-10-08T16:12:48.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.679+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32400b48c0 con 0x7f3240159c60 2023-10-08T16:12:48.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.679+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f323c01b3b0 con 0x7f3240159c60 2023-10-08T16:12:48.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.680+0000 7f32367fc700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 0x7f32240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:48.683 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.680+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f323c084270 con 0x7f3240159c60 2023-10-08T16:12:48.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.680+0000 7f324614d700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 0x7f32240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:48.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.680+0000 7f324614d700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 0x7f32240697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f32380097c0 tx=0x7f3238017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:48.684 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.684+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f323c050f00 con 0x7f3240159c60 2023-10-08T16:12:48.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.907+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f324004c3c0 con 0x7f3240159c60 2023-10-08T16:12:48.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.908+0000 7f32367fc700 1 -- 172.21.15.170:0/3671397048 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4610 (secure 0 0 0) 0x7f323c054520 con 0x7f3240159c60 2023-10-08T16:12:48.908 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:48.908 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":109,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:46.691353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:12:48.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 msgr2=0x7f32240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 0x7f32240697d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f32380097c0 tx=0x7f3238017000 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 msgr2=0x7f324014dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f323c00c600 tx=0x7f323c007e20 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 shutdown_connections 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3224067320 0x7f32240697d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3240159c60 0x7f324014dde0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3240069560 0x7f3240169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 --2- 172.21.15.170:0/3671397048 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3240159890 0x7f32400cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 >> 172.21.15.170:0/3671397048 conn(0x7f3240068e20 msgr2=0x7f3240149dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 shutdown_connections 2023-10-08T16:12:48.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:48.910+0000 7f32483b1700 1 -- 172.21.15.170:0/3671397048 wait complete. 2023-10-08T16:12:48.912 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 109 2023-10-08T16:12:49.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:12:43.032412+0000', 'modified': '2023-10-08T16:12:46.691353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8205}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8205': {'gid': 8205, 'name': 'c', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6836/4170488760', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 4170488760}, {'type': 'v1', 'addr': '172.21.15.170:6836', 'nonce': 4170488760}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:12:49.325 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:12:49.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T16:12:49.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.441+0000 7fcea8d9d700 1 Processor -- start 2023-10-08T16:12:49.442 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.441+0000 7fcea8d9d700 1 -- start start 2023-10-08T16:12:49.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcea4155c10 0x7fcea4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcea40ac230 0x7fcea40ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea41568a0 0x7fcea4156c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40ad930 con 0x7fcea4155c10 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea40acd20 con 0x7fcea40ac230 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea8d9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ad000 con 0x7fcea41568a0 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcea4155c10 0x7fcea4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcea4155c10 0x7fcea4155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32858/0 (socket says 172.21.15.170:32858) 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.442+0000 7fcea259c700 1 -- 172.21.15.170:0/4185909399 learned_addr learned my addr 172.21.15.170:0/4185909399 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fce99d9b700 1 --2- 172.21.15.170:0/4185909399 >> v2:172.21.15.191:3300/0 conn(0x7fcea40ac230 0x7fcea40ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea41568a0 0x7fcea4156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 -- 172.21.15.170:0/4185909399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea41568a0 msgr2=0x7fcea4156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea41568a0 0x7fcea4156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 -- 172.21.15.170:0/4185909399 >> v2:172.21.15.191:3300/0 conn(0x7fcea40ac230 msgr2=0x7fcea40ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 --2- 172.21.15.170:0/4185909399 >> v2:172.21.15.191:3300/0 conn(0x7fcea40ac230 0x7fcea40ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 -- 172.21.15.170:0/4185909399 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea40ad2e0 con 0x7fcea4155c10 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea259c700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea4155c10 0x7fcea4155fe0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcea4065bb0 tx=0x7fce9400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4268b5a24737b18b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea1d9b700 1 -- 172.21.15.170:0/4185909399 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400dc50 con 0x7fcea4155c10 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea1d9b700 1 -- 172.21.15.170:0/4185909399 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400ddb0 con 0x7fcea4155c10 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.443+0000 7fcea1d9b700 1 -- 172.21.15.170:0/4185909399 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce940196d0 con 0x7fcea4155c10 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 -- 172.21.15.170:0/4185909399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea4155c10 msgr2=0x7fcea4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea4155c10 0x7fcea4155fe0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fcea4065bb0 tx=0x7fce9400b4f0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 -- 172.21.15.170:0/4185909399 shutdown_connections 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea41568a0 0x7fcea4156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/4185909399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea4155c10 0x7fcea4155fe0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/4185909399 >> v2:172.21.15.191:3300/0 conn(0x7fcea40ac230 0x7fcea40ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 -- 172.21.15.170:0/4185909399 >> 172.21.15.170:0/4185909399 conn(0x7fcea406a320 msgr2=0x7fcea40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 -- 172.21.15.170:0/4185909399 shutdown_connections 2023-10-08T16:12:49.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.444+0000 7fcea8d9d700 1 -- 172.21.15.170:0/4185909399 wait complete. 2023-10-08T16:12:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.445+0000 7fcea8d9d700 1 Processor -- start 2023-10-08T16:12:49.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.445+0000 7fcea8d9d700 1 -- start start 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea4155c10 0x7fcea414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 0x7fcea414a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea41490a0 con 0x7fcea40ac230 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40b0c90 con 0x7fcea41568a0 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea4147c70 con 0x7fcea4155c10 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32862/0 (socket says 172.21.15.170:32862) 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 -- 172.21.15.170:0/1763890759 learned_addr learned my addr 172.21.15.170:0/1763890759 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 0x7fcea414a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fce99d9b700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea4155c10 0x7fcea414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 -- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea4155c10 msgr2=0x7fcea414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea4155c10 0x7fcea414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 -- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 msgr2=0x7fcea414a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 0x7fcea414a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 -- 172.21.15.170:0/1763890759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9400b790 con 0x7fcea40ac230 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 0x7fcea414a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea259c700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcea4064290 tx=0x7fce9401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fce9affd700 1 -- 172.21.15.170:0/1763890759 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94010850 con 0x7fcea40ac230 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fce9affd700 1 -- 172.21.15.170:0/1763890759 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9401e050 con 0x7fcea40ac230 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea4148490 con 0x7fcea40ac230 2023-10-08T16:12:49.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fce9affd700 1 -- 172.21.15.170:0/1763890759 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94010c00 con 0x7fcea40ac230 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.446+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea4148a50 con 0x7fcea40ac230 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.448+0000 7fce9affd700 1 -- 172.21.15.170:0/1763890759 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fce94024470 con 0x7fcea40ac230 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.448+0000 7fce9affd700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 0x7fce840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.448+0000 7fce9affd700 1 -- 172.21.15.170:0/1763890759 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce94014070 con 0x7fcea40ac230 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.448+0000 7fce99d9b700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 0x7fce840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.449+0000 7fce99d9b700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 0x7fce840697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcea4168db0 tx=0x7fce8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.450+0000 7fceb2095b80 1 Processor -- start 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.450+0000 7fceb2095b80 1 -- start start 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf771880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf6c8a70 con 0x55fecf79b050 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf6c6270 con 0x55fecf771dc0 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fecf6c2060 con 0x55fecf772f80 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32864/0 (socket says 172.21.15.170:32864) 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 -- 172.21.15.170:0/3401272244 learned_addr learned my addr 172.21.15.170:0/3401272244 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 -- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 msgr2=0x55fecf771880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf771880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 -- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 msgr2=0x55fecf7721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 -- 172.21.15.170:0/3401272244 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fecf55c270 con 0x55fecf79b050 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.451+0000 7fcea259c700 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fce9405a210 tx=0x7fce9400d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a892b423cca9a21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fce837fe700 1 -- 172.21.15.170:0/3401272244 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9405a280 con 0x55fecf79b050 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fce837fe700 1 -- 172.21.15.170:0/3401272244 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9405d4d0 con 0x55fecf79b050 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fce837fe700 1 -- 172.21.15.170:0/3401272244 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94061220 con 0x55fecf79b050 2023-10-08T16:12:49.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 -- 172.21.15.170:0/3401272244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 msgr2=0x55fecf79b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fce9405a210 tx=0x7fce9400d8e0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 -- 172.21.15.170:0/3401272244 shutdown_connections 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf79b420 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3401272244 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf771880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 -- 172.21.15.170:0/3401272244 >> 172.21.15.170:0/3401272244 conn(0x55fecf773370 msgr2=0x55fecf773740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 -- 172.21.15.170:0/3401272244 shutdown_connections 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.452+0000 7fceb2095b80 1 -- 172.21.15.170:0/3401272244 wait complete. 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.453+0000 7fceb2095b80 1 -- start start 2023-10-08T16:12:49.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.453+0000 7fceb2095b80 1 Processor -- start 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf7b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf66d890 con 0x55fecf79b050 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf688ea0 con 0x55fecf771dc0 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fecf544e30 con 0x55fecf772f80 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fceb2095b80 10 client.0 _ll_register_callbacks cb 0x55fecf735110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32870/0 (socket says 172.21.15.170:32870) 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 -- 172.21.15.170:0/3293869915 learned_addr learned my addr 172.21.15.170:0/3293869915 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fce99d9b700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf7b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea259c700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 -- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 msgr2=0x55fecf7b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf7b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 -- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 msgr2=0x55fecf7957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.454+0000 7fcea2d9d700 1 -- 172.21.15.170:0/3293869915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea4148a50 con 0x55fecf79b050 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007bb0 tx=0x7fce9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d2d7a17d4de96 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00f850 con 0x55fecf79b050 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce9c00f9b0 con 0x55fecf79b050 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fecf766b70 con 0x55fecf79b050 2023-10-08T16:12:49.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce9c00fd80 con 0x55fecf79b050 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 10 client.8349 fetch_fsmap learned FSMap version 109 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55fecf4fefe0 con 0x55fecf79b050 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fecf766b70 con 0x55fecf79b050 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 109) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce9c016420 con 0x55fecf79b050 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 10 client.8349 fetch_fsmap finished waiting for FSMap version 109 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 10 client.8349 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 20 client.8349 populate_metadata read hostname 'smithi170' 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf7d0080 0x55fecf7d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 4 client.8349 mds_command: new command op to 8205 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 --> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fecf743aa0 con 0x55fecf7d0080 2023-10-08T16:12:49.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.455+0000 7fcea259c700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf7d0080 0x55fecf7d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fce80ff9700 10 client.8349 ms_handle_connect on v2:172.21.15.170:6834/4170488760 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fcea259c700 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf7d0080 0x55fecf7d9c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50bd9bc85dbc099 server_cookie=2a843bd274d6b142 in_seq=0 out_seq=0 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fce80ff9700 1 -- 172.21.15.170:0/3293869915 <== mds.0 v2:172.21.15.170:6834/4170488760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55fecf743aa0 con 0x55fecf7d0080 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fce80ff9700 10 client.8349 handle_command_reply: tid=0 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 client.8349 shutdown 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 msgr2=0x55fecf7b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007bb0 tx=0x7fce9c00b040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 shutdown_connections 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf7d0080 0x55fecf7d9c50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf79b050 0x55fecf7b12e0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf771dc0 0x55fecf7957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.464 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3293869915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf7b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fce80ff9700 0 client.8349 ms_handle_reset on v2:172.21.15.170:6834/4170488760 2023-10-08T16:12:49.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 >> 172.21.15.170:0/3293869915 conn(0x55fecf773370 msgr2=0x55fecf76cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 shutdown_connections 2023-10-08T16:12:49.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.456+0000 7fceb2095b80 1 -- 172.21.15.170:0/3293869915 wait complete. 2023-10-08T16:12:49.465 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.457+0000 7fceb2095b80 20 client.8349 trim_cache size 0 max 16384 2023-10-08T16:12:49.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.473+0000 7fceb2095b80 1 Processor -- start 2023-10-08T16:12:49.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.473+0000 7fceb2095b80 1 -- start start 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf79f7f0 con 0x55fecf771dc0 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf79abc0 con 0x55fecf79b050 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fecf7b2ae0 con 0x55fecf772f80 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32886/0 (socket says 172.21.15.170:32886) 2023-10-08T16:12:49.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 -- 172.21.15.170:0/793882668 learned_addr learned my addr 172.21.15.170:0/793882668 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 -- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 msgr2=0x55fecf773740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 -- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf79b050 msgr2=0x55fecf773c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 -- 172.21.15.170:0/793882668 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fecf690780 con 0x55fecf771dc0 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fcea259c700 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fce940585d0 tx=0x7fce94008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385f0b276e2f2d0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fce83fff700 1 -- 172.21.15.170:0/793882668 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400d8e0 con 0x55fecf771dc0 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fce83fff700 1 -- 172.21.15.170:0/793882668 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9405a280 con 0x55fecf771dc0 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.474+0000 7fce83fff700 1 -- 172.21.15.170:0/793882668 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94061220 con 0x55fecf771dc0 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- 172.21.15.170:0/793882668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 msgr2=0x55fecf7d6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fce940585d0 tx=0x7fce94008040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- 172.21.15.170:0/793882668 shutdown_connections 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf7d6480 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 --2- 172.21.15.170:0/793882668 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- 172.21.15.170:0/793882668 >> 172.21.15.170:0/793882668 conn(0x55fecf773370 msgr2=0x55fecf79ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- 172.21.15.170:0/793882668 shutdown_connections 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- 172.21.15.170:0/793882668 wait complete. 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.475+0000 7fceb2095b80 1 -- start start 2023-10-08T16:12:49.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 Processor -- start 2023-10-08T16:12:49.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf822640 con 0x55fecf771dc0 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fecf7cd9f0 con 0x55fecf772f80 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.476+0000 7fceb2095b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fecf822890 con 0x55fecf79b050 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 10 client.0 _ll_register_callbacks cb 0x55fecf735110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32896/0 (socket says 172.21.15.170:32896) 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 -- 172.21.15.170:0/3399765928 learned_addr learned my addr 172.21.15.170:0/3399765928 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce99d9b700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 -- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 msgr2=0x55fecf773c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 -- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 msgr2=0x55fecf773740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 -- 172.21.15.170:0/3399765928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fecf55c270 con 0x55fecf771dc0 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce99d9b700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea2d9d700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fcea259c700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b530 tx=0x7fce94083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee59538e9b9aed2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94059770 con 0x55fecf771dc0 2023-10-08T16:12:49.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fce940598d0 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce9405d4d0 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce94060fa0 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 10 client.8355 fetch_fsmap learned FSMap version 109 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) v3 -- 0x55fecf677470 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9405d4d0 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 109) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fce94024720 con 0x55fecf771dc0 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 10 client.8355 fetch_fsmap finished waiting for FSMap version 109 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 10 client.8355 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.477+0000 7fceb2095b80 20 client.8355 populate_metadata read hostname 'smithi170' 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf798f90 0x55fecf799360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 4 client.8355 mds_command: new command op to 8205 tid=0[{"prefix": "status"}] 2023-10-08T16:12:49.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 --> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fecf664e00 con 0x55fecf798f90 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce99d9b700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf798f90 0x55fecf799360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce80ff9700 10 client.8355 ms_handle_connect on v2:172.21.15.170:6834/4170488760 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce99d9b700 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf798f90 0x55fecf799360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9445cf689552d157 server_cookie=9ae356d99b2b1370 in_seq=0 out_seq=0 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce80ff9700 1 -- 172.21.15.170:0/3399765928 <== mds.0 v2:172.21.15.170:6834/4170488760 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55fecf664e00 con 0x55fecf798f90 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce80ff9700 10 client.8355 handle_command_reply: tid=0 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 client.8355 shutdown 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 msgr2=0x55fecf81baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b530 tx=0x7fce94083040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 shutdown_connections 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:6834/4170488760,v1:172.21.15.170:6836/4170488760] conn(0x55fecf798f90 0x55fecf799360 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x55fecf79b050 0x55fecf773c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fce80ff9700 0 client.8355 ms_handle_reset on v2:172.21.15.170:6834/4170488760 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fecf771dc0 0x55fecf81baa0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 --2- 172.21.15.170:0/3399765928 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x55fecf772f80 0x55fecf773740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 >> 172.21.15.170:0/3399765928 conn(0x55fecf773370 msgr2=0x55fecf7c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 shutdown_connections 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.478+0000 7fceb2095b80 1 -- 172.21.15.170:0/3399765928 wait complete. 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fceb2095b80 20 client.8355 trim_cache size 0 max 16384 2023-10-08T16:12:49.487 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8205, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.8066913339999999, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 109, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 166, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 166, 2023-10-08T16:12:49.488 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.938146676000001 2023-10-08T16:12:49.489 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:12:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 msgr2=0x7fce840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 0x7fce840697d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcea4168db0 tx=0x7fce8c008040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 msgr2=0x7fcea4147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcea4064290 tx=0x7fce9401b000 comp rx=0 tx=0).stop 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 shutdown_connections 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcea41568a0 0x7fcea414a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fce84067320 0x7fce840697d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcea40ac230 0x7fcea4147730 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 --2- 172.21.15.170:0/1763890759 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcea4155c10 0x7fcea414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 >> 172.21.15.170:0/1763890759 conn(0x7fcea406a320 msgr2=0x7fcea41534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 shutdown_connections 2023-10-08T16:12:49.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.479+0000 7fcea8d9d700 1 -- 172.21.15.170:0/1763890759 wait complete. 2023-10-08T16:12:49.491 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:12:49.513 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:12:49.513+0000 7f7e03f67700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:12:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.605+0000 7f3f1326f700 1 Processor -- start 2023-10-08T16:12:49.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.605+0000 7f3f1326f700 1 -- start start 2023-10-08T16:12:49.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1326f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3f0c159420 0x7f3f0c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1326f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0b2980 con 0x7f3f0c1597f0 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1326f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c0b11d0 con 0x7f3f0c15d230 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1326f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b14b0 con 0x7f3f0c159420 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f11a6c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3f0c1597f0 0x7f3f0c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39444/0 (socket says 172.21.15.170:39444) 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f11a6c700 1 -- 172.21.15.170:0/2680023009 learned_addr learned my addr 172.21.15.170:0/2680023009 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1226d700 1 --2- 172.21.15.170:0/2680023009 >> v2:172.21.15.191:3300/0 conn(0x7f3f0c159420 0x7f3f0c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637089945 0 0) 0x7f3f0c0b2980 con 0x7f3f0c1597f0 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef8003570 con 0x7f3f0c1597f0 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2583732860 0 0) 0x7f3ef8003570 con 0x7f3f0c1597f0 2023-10-08T16:12:49.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 >> v2:172.21.15.191:3300/0 conn(0x7f3f0c159420 msgr2=0x7f3f0c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 --2- 172.21.15.170:0/2680023009 >> v2:172.21.15.191:3300/0 conn(0x7f3f0c159420 0x7f3f0c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 >> v1:172.21.15.191:6789/0 conn(0x7f3f0c15d230 legacy=0x7f3f0c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0b1790 con 0x7f3f0c1597f0 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.606+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f3efc001470 con 0x7f3f0c1597f0 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 >> v1:172.21.15.170:6789/0 conn(0x7f3f0c1597f0 legacy=0x7f3f0c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0b14b0 con 0x7f3ef8004810 2023-10-08T16:12:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800c190 con 0x7f3ef8003a60 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef800c350 con 0x7f3ef80052e0 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1226d700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f12a6e700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 -- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef80052e0 msgr2=0x7f3ef800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3ef80052e0 0x7f3ef800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 -- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8003a60 msgr2=0x7f3ef8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8003a60 0x7f3ef8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f12a6e700 1 -- 172.21.15.170:0/2680023009 reap_dead start 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 -- 172.21.15.170:0/2680023009 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8003570 con 0x7f3ef8004810 2023-10-08T16:12:49.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f11a6c700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c390 tx=0x7f3efc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc60239fa607faab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00cb90 con 0x7f3ef8004810 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc00ccf0 con 0x7f3ef8004810 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.607+0000 7f3f1126b700 1 -- 172.21.15.170:0/2680023009 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc005b50 con 0x7f3ef8004810 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 -- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3ef8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00c390 tx=0x7f3efc001e50 comp rx=0 tx=0).stop 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 -- 172.21.15.170:0/2680023009 shutdown_connections 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 --2- 172.21.15.170:0/2680023009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ef8004810 0x7f3ef8004c80 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 -- 172.21.15.170:0/2680023009 >> 172.21.15.170:0/2680023009 conn(0x7f3f0c0d0660 msgr2=0x7f3f0c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 -- 172.21.15.170:0/2680023009 shutdown_connections 2023-10-08T16:12:49.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.608+0000 7f3f1326f700 1 -- 172.21.15.170:0/2680023009 wait complete. 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.609+0000 7f3f1326f700 1 Processor -- start 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.609+0000 7f3f1326f700 1 -- start start 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8004810 0x7f3f0c1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f0c0c67b0 0x7f3f0c0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c15c060 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c7270 con 0x7f3ef8004810 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c0c7550 con 0x7f3f0c0c67b0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32914/0 (socket says 172.21.15.170:32914) 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 -- 172.21.15.170:0/4067095437 learned_addr learned my addr 172.21.15.170:0/4067095437 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1226d700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8004810 0x7f3f0c1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f11a6c700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f0c0c67b0 0x7f3f0c0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 -- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f0c0c67b0 msgr2=0x7f3f0c0c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f0c0c67b0 0x7f3f0c0c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 -- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8004810 msgr2=0x7f3f0c1552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8004810 0x7f3f0c1552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 -- 172.21.15.170:0/4067095437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0c7830 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f12a6e700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0af930 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080032e0 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08003460 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c0c7b10 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080017d0 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.610+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c0c88b0 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.611+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c04c3c0 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.612+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3f08001bc0 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.612+0000 7f3f027fc700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 0x7f3ef00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.612+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3f08084a80 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.612+0000 7f3f1226d700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 0x7f3ef00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.612+0000 7f3f1226d700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 0x7f3ef00697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006fd0 tx=0x7f3f04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:49.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.616+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f08051710 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.843+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3f0c0b1a70 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.843+0000 7f3f027fc700 1 -- 172.21.15.170:0/4067095437 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3f08054d30 con 0x7f3f0c15b6d0 2023-10-08T16:12:49.843 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 msgr2=0x7f3ef00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 0x7f3ef00697d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f3f04006fd0 tx=0x7f3f04008040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 msgr2=0x7f3f0c15bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0af930 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 shutdown_connections 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3ef0067320 0x7f3ef00697d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f0c15b6d0 0x7f3f0c15bb20 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3ef8004810 0x7f3f0c1552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 --2- 172.21.15.170:0/4067095437 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3f0c0c67b0 0x7f3f0c0c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:49.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 >> 172.21.15.170:0/4067095437 conn(0x7f3f0c0d0660 msgr2=0x7f3f0c0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:49.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 shutdown_connections 2023-10-08T16:12:49.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:49.845+0000 7f3f1326f700 1 -- 172.21.15.170:0/4067095437 wait complete. 2023-10-08T16:12:49.857 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:12:49.857 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:12:49.858 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:49.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:49.858 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:12:49.913 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:12:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:12:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:12:49.913 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:49.914 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41981sec preferred_lft 41981sec 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:12:49.915 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:49.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:49.917 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:12:49.975 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:12:49.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:49.977 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:12:50.033 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:12:50.033 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:12:50.033 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:12:50.033 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:12:50.033 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:12:50.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:12:50.034 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:12:50.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.133 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:12:50.148 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:12:50.148 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:12:50.148 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:50.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:12:50.212 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:50.213 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:50.268 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:12:50.268 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:50.268 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:50.322 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:12:50.322 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:12:50.322 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:50.402 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:50.402 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:50.421 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:50.422 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:50.422 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:50.443 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:50.444 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:12:50.444 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:12:50.444 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:12:50.445 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:12:50.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.445 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:50.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:12:50.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 Processor -- start 2023-10-08T16:12:50.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 -- start start 2023-10-08T16:12:50.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3998157680 0x7f399806e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:50.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39980b00f0 con 0x7f39980af5c0 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399806ebe0 con 0x7f3998157a50 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.665+0000 7f399c937700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399806eec0 con 0x7f3998157680 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39977fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3998157680 0x7f399806e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f3997fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f39980af5c0 0x7f39980b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39450/0 (socket says 172.21.15.170:39450) 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f3997fff700 1 -- 172.21.15.170:0/239338622 learned_addr learned my addr 172.21.15.170:0/239338622 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373361461 0 0) 0x7f39980b00f0 con 0x7f39980af5c0 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3980003570 con 0x7f39980af5c0 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526272451 0 0) 0x7f399806ebe0 con 0x7f3998157a50 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39980b00f0 con 0x7f3998157a50 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39977fe700 1 -- 172.21.15.170:0/239338622 >> v1:172.21.15.191:6789/0 conn(0x7f3998157a50 legacy=0x7f39980aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39977fe700 1 -- 172.21.15.170:0/239338622 >> v1:172.21.15.170:6789/0 conn(0x7f39980af5c0 legacy=0x7f39980b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39977fe700 1 -- 172.21.15.170:0/239338622 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399806f1a0 con 0x7f3998157680 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39977fe700 1 --2- 172.21.15.170:0/239338622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157680 0x7f399806e570 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f39880098e0 tx=0x7f39880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca9e027b6d0a936 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011630 con 0x7f3998157680 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3988011790 con 0x7f3998157680 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.666+0000 7f39967fc700 1 -- 172.21.15.170:0/239338622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988011b30 con 0x7f3998157680 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 -- 172.21.15.170:0/239338622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157680 msgr2=0x7f399806e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 --2- 172.21.15.170:0/239338622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157680 0x7f399806e570 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f39880098e0 tx=0x7f39880091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 -- 172.21.15.170:0/239338622 shutdown_connections 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 --2- 172.21.15.170:0/239338622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157680 0x7f399806e570 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 -- 172.21.15.170:0/239338622 >> 172.21.15.170:0/239338622 conn(0x7f399806b450 msgr2=0x7f39980cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 -- 172.21.15.170:0/239338622 shutdown_connections 2023-10-08T16:12:50.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.667+0000 7f399c937700 1 -- 172.21.15.170:0/239338622 wait complete. 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.668+0000 7f399c937700 1 Processor -- start 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.668+0000 7f399c937700 1 -- start start 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39980af5c0 0x7f399815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3998157680 0x7f39980c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399815bd30 con 0x7f3998157a50 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399815cf90 con 0x7f3998157680 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f399c937700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39980c5180 con 0x7f39980af5c0 2023-10-08T16:12:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32934/0 (socket says 172.21.15.170:32934) 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 -- 172.21.15.170:0/1647455137 learned_addr learned my addr 172.21.15.170:0/1647455137 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3996ffd700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3998157680 0x7f39980c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f39977fe700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39980af5c0 0x7f399815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 -- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39980af5c0 msgr2=0x7f399815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39980af5c0 0x7f399815daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 -- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3998157680 msgr2=0x7f39980c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.669+0000 7f3997fff700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3998157680 0x7f39980c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f3997fff700 1 -- 172.21.15.170:0/1647455137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3988009420 con 0x7f3998157a50 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f3997fff700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f399000cc90 tx=0x7f3990001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000a810 con 0x7f3998157a50 2023-10-08T16:12:50.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399000a990 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399000ad50 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399815bf30 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.670+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399815c4b0 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.671+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399815cc80 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.671+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3990007300 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.672+0000 7f397f7fe700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 0x7f39840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.672+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3990084ef0 con 0x7f3998157a50 2023-10-08T16:12:50.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.672+0000 7f39977fe700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 0x7f39840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:50.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.672+0000 7f39977fe700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 0x7f39840697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f39880093f0 tx=0x7f3988017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:50.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.676+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3990055740 con 0x7f3998157a50 2023-10-08T16:12:50.904 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.904+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f399804c3c0 con 0x7f3998157a50 2023-10-08T16:12:50.905 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.904+0000 7f397f7fe700 1 -- 172.21.15.170:0/1647455137 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v109) v1 ==== 53+0+73 (secure 0 0 0) 0x7f39900552d0 con 0x7f3998157a50 2023-10-08T16:12:50.905 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 msgr2=0x7f39840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 0x7f39840697d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f39880093f0 tx=0x7f3988017000 comp rx=0 tx=0).stop 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 msgr2=0x7f3998156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f399000cc90 tx=0x7f3990001ae0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 shutdown_connections 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3984067320 0x7f39840697d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3998157a50 0x7f3998156230 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f39980af5c0 0x7f399815daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 --2- 172.21.15.170:0/1647455137 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3998157680 0x7f39980c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:50.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 >> 172.21.15.170:0/1647455137 conn(0x7f399806b450 msgr2=0x7f39980cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:50.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 shutdown_connections 2023-10-08T16:12:50.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:50.906+0000 7f399c937700 1 -- 172.21.15.170:0/1647455137 wait complete. 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:12:50.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:50.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.920 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:12:50.938 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:12:50.938 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:12:50.938 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:12:50.939 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41980sec preferred_lft 41980sec 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.940 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:12:50.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:50.942 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:12:51.006 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:12:51.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:51.008 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:12:51.066 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:12:51.066 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:12:51.066 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:12:51.066 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:12:51.066 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:12:51.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:12:51.067 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:12:51.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:51.172 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:12:51.187 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:12:51.187 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:12:51.187 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:51.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:51.188 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:12:51.252 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:51.252 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:51.307 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:12:51.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:12:51.307 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:51.361 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:12:51.677 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:12:51.678 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:51.719 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:51.720 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:51.738 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:51.738 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:51.738 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:51.739 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:12:51.760 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:12:51.761 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:12:51.761 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:12:51.761 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:12:51.762 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:12:51.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:51.762 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:51.826 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:12:51.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.954+0000 7fdee597d700 1 Processor -- start 2023-10-08T16:12:51.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.954+0000 7fdee597d700 1 -- start start 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee597d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdee00b0790 0x7fdee00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee597d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdee01591e0 0x7fdee0159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee597d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee00cfa00 con 0x7fdee00b03c0 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee597d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00cb580 con 0x7fdee01591e0 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee597d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00cf160 con 0x7fdee00b0790 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdee01591e0 0x7fdee0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdee01591e0 0x7fdee0159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32942/0 (socket says 172.21.15.170:32942) 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 -- 172.21.15.170:0/3678357838 learned_addr learned my addr 172.21.15.170:0/3678357838 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fded7fff700 1 --2- 172.21.15.170:0/3678357838 >> v2:172.21.15.191:3300/0 conn(0x7fdee00b0790 0x7fdee00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 -- 172.21.15.170:0/3678357838 >> v2:172.21.15.191:3300/0 conn(0x7fdee00b0790 msgr2=0x7fdee00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 --2- 172.21.15.170:0/3678357838 >> v2:172.21.15.191:3300/0 conn(0x7fdee00b0790 0x7fdee00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 -- 172.21.15.170:0/3678357838 >> v1:172.21.15.191:6789/0 conn(0x7fdee00b03c0 legacy=0x7fdee00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.955+0000 7fdee517c700 1 -- 172.21.15.170:0/3678357838 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00cf440 con 0x7fdee01591e0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee517c700 1 --2- 172.21.15.170:0/3678357838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0159600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fded8009f20 tx=0x7fded8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d98114a3ed03a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fded77fe700 1 -- 172.21.15.170:0/3678357838 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800e070 con 0x7fdee01591e0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fded77fe700 1 -- 172.21.15.170:0/3678357838 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded800ba40 con 0x7fdee01591e0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fded77fe700 1 -- 172.21.15.170:0/3678357838 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded80092e0 con 0x7fdee01591e0 2023-10-08T16:12:51.958 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 -- 172.21.15.170:0/3678357838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 msgr2=0x7fdee0159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 --2- 172.21.15.170:0/3678357838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0159600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fded8009f20 tx=0x7fded8009840 comp rx=0 tx=0).stop 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 -- 172.21.15.170:0/3678357838 shutdown_connections 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 --2- 172.21.15.170:0/3678357838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0159600 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 --2- 172.21.15.170:0/3678357838 >> v2:172.21.15.191:3300/0 conn(0x7fdee00b0790 0x7fdee00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 -- 172.21.15.170:0/3678357838 >> 172.21.15.170:0/3678357838 conn(0x7fdee0063230 msgr2=0x7fdee0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 -- 172.21.15.170:0/3678357838 shutdown_connections 2023-10-08T16:12:51.959 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.956+0000 7fdee597d700 1 -- 172.21.15.170:0/3678357838 wait complete. 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.957+0000 7fdee597d700 1 Processor -- start 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 -- start start 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee00b03c0 0x7fdee015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee00b0790 0x7fdee00c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c5570 con 0x7fdee01591e0 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00c59a0 con 0x7fdee00b03c0 2023-10-08T16:12:51.962 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee597d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0158f60 con 0x7fdee00b0790 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee517c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee517c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32950/0 (socket says 172.21.15.170:32950) 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee517c700 1 -- 172.21.15.170:0/2140044777 learned_addr learned my addr 172.21.15.170:0/2140044777 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fded7fff700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee00b0790 0x7fdee00c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.958+0000 7fdee497b700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee00b03c0 0x7fdee015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 -- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee00b0790 msgr2=0x7fdee00c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee00b0790 0x7fdee00c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 -- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee00b03c0 msgr2=0x7fdee015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee00b03c0 0x7fdee015a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 -- 172.21.15.170:0/2140044777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded8009ae0 con 0x7fdee01591e0 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee517c700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cfe40 tx=0x7fded8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:51.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800be70 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8007ac0 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee015cd30 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8007e80 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.959+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee015cfd0 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.960+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee004c3c0 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.960+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fded801d420 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.961+0000 7fded4ff9700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 0x7fdecc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.961+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fded8086bf0 con 0x7fdee01591e0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.961+0000 7fdee497b700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 0x7fdecc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:51.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.961+0000 7fdee497b700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 0x7fdecc0697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0039e0 tx=0x7fdedc007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:51.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:51.965+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fded8053770 con 0x7fdee01591e0 2023-10-08T16:12:52.190 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.190+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdee00c5780 con 0x7fdee01591e0 2023-10-08T16:12:52.713 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.713+0000 7fded4ff9700 1 -- 172.21.15.170:0/2140044777 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v110) v1 ==== 136+0+0 (secure 0 0 0) 0x7fded8056d90 con 0x7fdee01591e0 2023-10-08T16:12:52.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.715+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 msgr2=0x7fdecc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.715+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 0x7fdecc0697d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0039e0 tx=0x7fdedc007ec0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.715+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 msgr2=0x7fdee0158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fdee00cfe40 tx=0x7fded8018000 comp rx=0 tx=0).stop 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 shutdown_connections 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdecc067320 0x7fdecc0697d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdee01591e0 0x7fdee0158a20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdee00b03c0 0x7fdee015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 --2- 172.21.15.170:0/2140044777 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdee00b0790 0x7fdee00c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 >> 172.21.15.170:0/2140044777 conn(0x7fdee0063230 msgr2=0x7fdee00d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 shutdown_connections 2023-10-08T16:12:52.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.716+0000 7fdee597d700 1 -- 172.21.15.170:0/2140044777 wait complete. 2023-10-08T16:12:52.729 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:12:52.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.845+0000 7f0e5f59e700 1 Processor -- start 2023-10-08T16:12:52.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.845+0000 7f0e5f59e700 1 -- start start 2023-10-08T16:12:52.847 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5f59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0e6015d230 0x7f0e6015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5f59e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600b2980 con 0x7f0e601597f0 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5f59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6015db70 con 0x7f0e60159420 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5f59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600b11d0 con 0x7f0e6015d230 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5ed9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0e6015d230 0x7f0e6015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5dd9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0e601597f0 0x7f0e600b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39452/0 (socket says 172.21.15.170:39452) 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5dd9b700 1 -- 172.21.15.170:0/1721271814 learned_addr learned my addr 172.21.15.170:0/1721271814 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5ed9d700 1 -- 172.21.15.170:0/1721271814 >> v1:172.21.15.191:6789/0 conn(0x7f0e60159420 legacy=0x7f0e600b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5ed9d700 1 -- 172.21.15.170:0/1721271814 >> v1:172.21.15.170:6789/0 conn(0x7f0e601597f0 legacy=0x7f0e600b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5d59a700 1 -- 172.21.15.170:0/1721271814 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810166639 0 0) 0x7f0e600b2980 con 0x7f0e601597f0 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5ed9d700 1 -- 172.21.15.170:0/1721271814 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600b14b0 con 0x7f0e6015d230 2023-10-08T16:12:52.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.846+0000 7f0e5ed9d700 1 --2- 172.21.15.170:0/1721271814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e6015d230 0x7f0e6015d630 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009e60 tx=0x7f0e58009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe7d6b153e3a155 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5d59a700 1 -- 172.21.15.170:0/1721271814 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800e070 con 0x7f0e6015d230 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5d59a700 1 -- 172.21.15.170:0/1721271814 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800bc30 con 0x7f0e6015d230 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5d59a700 1 -- 172.21.15.170:0/1721271814 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58012430 con 0x7f0e6015d230 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 -- 172.21.15.170:0/1721271814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e6015d230 msgr2=0x7f0e6015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/1721271814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e6015d230 0x7f0e6015d630 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009e60 tx=0x7f0e58009780 comp rx=0 tx=0).stop 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 -- 172.21.15.170:0/1721271814 shutdown_connections 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/1721271814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e6015d230 0x7f0e6015d630 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 -- 172.21.15.170:0/1721271814 >> 172.21.15.170:0/1721271814 conn(0x7f0e600d0660 msgr2=0x7f0e600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 -- 172.21.15.170:0/1721271814 shutdown_connections 2023-10-08T16:12:52.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.847+0000 7f0e5f59e700 1 -- 172.21.15.170:0/1721271814 wait complete. 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.848+0000 7f0e5f59e700 1 Processor -- start 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.848+0000 7f0e5f59e700 1 -- start start 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e60159420 0x7f0e600c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e6015d230 0x7f0e60155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6015b600 con 0x7f0e601597f0 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6015a9f0 con 0x7f0e60159420 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5f59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6015acd0 con 0x7f0e6015d230 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32964/0 (socket says 172.21.15.170:32964) 2023-10-08T16:12:52.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 -- 172.21.15.170:0/2820638282 learned_addr learned my addr 172.21.15.170:0/2820638282 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 -- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e6015d230 msgr2=0x7f0e60155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e6015d230 0x7f0e60155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 -- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e60159420 msgr2=0x7f0e600c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e60159420 0x7f0e600c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.849+0000 7f0e5dd9b700 1 -- 172.21.15.170:0/2820638282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009a20 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e5dd9b700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c810 tx=0x7f0e54007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54005c00 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54005d60 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6015afb0 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54007320 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6015ba90 con 0x7f0e601597f0 2023-10-08T16:12:52.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.850+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6004c3c0 con 0x7f0e601597f0 2023-10-08T16:12:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.851+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0e5401b400 con 0x7f0e601597f0 2023-10-08T16:12:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.851+0000 7f0e4e7fc700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 0x7f0e400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.851+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0e54084720 con 0x7f0e601597f0 2023-10-08T16:12:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.851+0000 7f0e5e59c700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 0x7f0e400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:52.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.852+0000 7f0e5e59c700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 0x7f0e400697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0e50001730 tx=0x7f0e50007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:52.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:52.856+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e540514b0 con 0x7f0e601597f0 2023-10-08T16:12:53.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.064+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e6015b740 con 0x7f0e40067320 2023-10-08T16:12:53.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.065+0000 7f0e4e7fc700 1 -- 172.21.15.170:0/2820638282 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0e50001410 con 0x7f0e40067320 2023-10-08T16:12:53.065 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:53.065 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:53.066 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:12:53.066 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:53.066 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 msgr2=0x7f0e400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 0x7f0e400697d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0e50001730 tx=0x7f0e50007b40 comp rx=0 tx=0).stop 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 msgr2=0x7f0e6015c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c810 tx=0x7f0e54007db0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 shutdown_connections 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0e6015d230 0x7f0e60155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0e60159420 0x7f0e600c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0e40067320 0x7f0e400697d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 --2- 172.21.15.170:0/2820638282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0e601597f0 0x7f0e6015c500 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 >> 172.21.15.170:0/2820638282 conn(0x7f0e600d0660 msgr2=0x7f0e600d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:53.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 shutdown_connections 2023-10-08T16:12:53.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.067+0000 7f0e5f59e700 1 -- 172.21.15.170:0/2820638282 wait complete. 2023-10-08T16:12:53.080 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:12:53.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.197+0000 7fb33277a700 1 Processor -- start 2023-10-08T16:12:53.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.197+0000 7fb33277a700 1 -- start start 2023-10-08T16:12:53.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb32c0b4e70 0x7fb32c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 0x7fb32c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0b2080 con 0x7fb32c0b4e70 2023-10-08T16:12:53.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0cfb00 con 0x7fb32c15d240 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb33277a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c0b5c40 con 0x7fb32c0b16d0 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb32c0b4e70 0x7fb32c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb32c0b4e70 0x7fb32c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32966/0 (socket says 172.21.15.170:32966) 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 -- 172.21.15.170:0/368421187 learned_addr learned my addr 172.21.15.170:0/368421187 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 -- 172.21.15.170:0/368421187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 msgr2=0x7fb32c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 --2- 172.21.15.170:0/368421187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 -- 172.21.15.170:0/368421187 >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 msgr2=0x7fb32c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331f79700 1 --2- 172.21.15.170:0/368421187 >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 0x7fb32c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 --2- 172.21.15.170:0/368421187 >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 0x7fb32c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331778700 1 -- 172.21.15.170:0/368421187 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c0b5f20 con 0x7fb32c0b4e70 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.198+0000 7fb331f79700 1 --2- 172.21.15.170:0/368421187 >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 0x7fb32c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb331778700 1 --2- 172.21.15.170:0/368421187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c0b4e70 0x7fb32c0ccff0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb32c15d180 tx=0x7fb328009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60175433b7bf35 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb323fff700 1 -- 172.21.15.170:0/368421187 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32800e070 con 0x7fb32c0b4e70 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb323fff700 1 -- 172.21.15.170:0/368421187 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32800ba50 con 0x7fb32c0b4e70 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb323fff700 1 -- 172.21.15.170:0/368421187 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3280092e0 con 0x7fb32c0b4e70 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 -- 172.21.15.170:0/368421187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c0b4e70 msgr2=0x7fb32c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 --2- 172.21.15.170:0/368421187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c0b4e70 0x7fb32c0ccff0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fb32c15d180 tx=0x7fb328009840 comp rx=0 tx=0).stop 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 -- 172.21.15.170:0/368421187 shutdown_connections 2023-10-08T16:12:53.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 --2- 172.21.15.170:0/368421187 >> v2:172.21.15.191:3300/0 conn(0x7fb32c15d240 0x7fb32c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 --2- 172.21.15.170:0/368421187 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c0b4e70 0x7fb32c0ccff0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 --2- 172.21.15.170:0/368421187 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 -- 172.21.15.170:0/368421187 >> 172.21.15.170:0/368421187 conn(0x7fb32c0d10c0 msgr2=0x7fb32c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:53.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 -- 172.21.15.170:0/368421187 shutdown_connections 2023-10-08T16:12:53.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.199+0000 7fb33277a700 1 -- 172.21.15.170:0/368421187 wait complete. 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.200+0000 7fb33277a700 1 Processor -- start 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.200+0000 7fb33277a700 1 -- start start 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb32c0b4e70 0x7fb32c1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c1476f0 con 0x7fb32c15d240 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c145f20 con 0x7fb32c0b4e70 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb33277a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c146ae0 con 0x7fb32c0b16d0 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.201+0000 7fb331f79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32970/0 (socket says 172.21.15.170:32970) 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 -- 172.21.15.170:0/2422655933 learned_addr learned my addr 172.21.15.170:0/2422655933 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb330f77700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb32c0b4e70 0x7fb32c1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331778700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 -- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 msgr2=0x7fb32c1459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c1459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 -- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb32c0b4e70 msgr2=0x7fb32c1461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb32c0b4e70 0x7fb32c1461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 -- 172.21.15.170:0/2422655933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009ae0 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb331f79700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c850 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324011600 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324011760 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324011b20 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c146dc0 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.202+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c147380 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.203+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c147860 con 0x7fb32c15d240 2023-10-08T16:12:53.208 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.203+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb324018350 con 0x7fb32c15d240 2023-10-08T16:12:53.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.204+0000 7fb3217fa700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 0x7fb3100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.204+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb324084b70 con 0x7fb32c15d240 2023-10-08T16:12:53.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.204+0000 7fb331778700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 0x7fb3100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.204+0000 7fb331778700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 0x7fb3100697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb328009880 tx=0x7fb328018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.209 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.208+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3240516f0 con 0x7fb32c15d240 2023-10-08T16:12:53.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.391+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb32c04bb80 con 0x7fb32c15d240 2023-10-08T16:12:53.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.392+0000 7fb3217fa700 1 -- 172.21.15.170:0/2422655933 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb324054d10 con 0x7fb32c15d240 2023-10-08T16:12:53.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 msgr2=0x7fb3100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 0x7fb3100697d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb328009880 tx=0x7fb328018000 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 msgr2=0x7fb32c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fb32400c850 tx=0x7fb324008040 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 shutdown_connections 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb310067320 0x7fb3100697d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb32c15d240 0x7fb32c068390 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb32c0b16d0 0x7fb32c1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 --2- 172.21.15.170:0/2422655933 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb32c0b4e70 0x7fb32c1461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 >> 172.21.15.170:0/2422655933 conn(0x7fb32c0d10c0 msgr2=0x7fb32c158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 shutdown_connections 2023-10-08T16:12:53.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.394+0000 7fb33277a700 1 -- 172.21.15.170:0/2422655933 wait complete. 2023-10-08T16:12:53.406 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:12:53.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:12:53.407 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:12:53.440 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:12:53.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:53.440 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:12:53.495 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:12:53.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:12:53.495 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:12:53.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:12:53.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:53.539 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:12:53.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:12:53.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:53.596 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T16:12:53.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:12:53.611 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T16:12:53.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:12:53.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:12:53.611 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T16:12:53.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:12:53.652 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T16:12:53.652 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create another-data-pool --pg_num_min 64 2023-10-08T16:12:53.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.772+0000 7f08f2af0700 1 Processor -- start 2023-10-08T16:12:53.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.772+0000 7f08f2af0700 1 -- start start 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f08ec157760 0x7f08ec069870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08ec0cac10 0x7f08ec0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec15c180 con 0x7f08ec0cac10 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec06dfe0 con 0x7f08ec157760 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f2af0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15b8e0 con 0x7f08ec157390 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08ec0cac10 0x7f08ec0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f08ec0cac10 0x7f08ec0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32978/0 (socket says 172.21.15.170:32978) 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 -- 172.21.15.170:0/4023777926 learned_addr learned my addr 172.21.15.170:0/4023777926 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:53.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f12ed700 1 --2- 172.21.15.170:0/4023777926 >> v2:172.21.15.191:3300/0 conn(0x7f08ec157760 0x7f08ec069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f1aee700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 -- 172.21.15.170:0/4023777926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 msgr2=0x7f08ec15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec15b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 -- 172.21.15.170:0/4023777926 >> v2:172.21.15.191:3300/0 conn(0x7f08ec157760 msgr2=0x7f08ec069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 --2- 172.21.15.170:0/4023777926 >> v2:172.21.15.191:3300/0 conn(0x7f08ec157760 0x7f08ec069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 -- 172.21.15.170:0/4023777926 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec15bbc0 con 0x7f08ec0cac10 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.773+0000 7f08f22ef700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0b1d50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f08ec06d830 tx=0x7f08e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73b30d0211944da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f0aec700 1 -- 172.21.15.170:0/4023777926 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400e070 con 0x7f08ec0cac10 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f0aec700 1 -- 172.21.15.170:0/4023777926 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400ba50 con 0x7f08ec0cac10 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f0aec700 1 -- 172.21.15.170:0/4023777926 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40092e0 con 0x7f08ec0cac10 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 -- 172.21.15.170:0/4023777926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 msgr2=0x7f08ec0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0b1d50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f08ec06d830 tx=0x7f08e4009840 comp rx=0 tx=0).stop 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 -- 172.21.15.170:0/4023777926 shutdown_connections 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0b1d50 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4023777926 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4023777926 >> v2:172.21.15.191:3300/0 conn(0x7f08ec157760 0x7f08ec069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 -- 172.21.15.170:0/4023777926 >> 172.21.15.170:0/4023777926 conn(0x7f08ec06adf0 msgr2=0x7f08ec0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 -- 172.21.15.170:0/4023777926 shutdown_connections 2023-10-08T16:12:53.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.774+0000 7f08f2af0700 1 -- 172.21.15.170:0/4023777926 wait complete. 2023-10-08T16:12:53.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.775+0000 7f08f2af0700 1 Processor -- start 2023-10-08T16:12:53.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.775+0000 7f08f2af0700 1 -- start start 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec14d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec157760 0x7f08ec150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c8620 con 0x7f08ec0cac10 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec16d390 con 0x7f08ec157760 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f2af0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec16d670 con 0x7f08ec157390 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:32990/0 (socket says 172.21.15.170:32990) 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 -- 172.21.15.170:0/4090599925 learned_addr learned my addr 172.21.15.170:0/4090599925 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f12ed700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec14d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f22ef700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec157760 0x7f08ec150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 -- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 msgr2=0x7f08ec14d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec14d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 -- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec157760 msgr2=0x7f08ec150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec157760 0x7f08ec150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.776+0000 7f08f1aee700 1 -- 172.21.15.170:0/4090599925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009ae0 con 0x7f08ec0cac10 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08f1aee700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f08e800d890 tx=0x7f08e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800b600 con 0x7f08ec0cac10 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e800b760 con 0x7f08ec0cac10 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec16d950 con 0x7f08ec0cac10 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e800bae0 con 0x7f08ec0cac10 2023-10-08T16:12:53.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.777+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec14d820 con 0x7f08ec0cac10 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.778+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec063230 con 0x7f08ec0cac10 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.778+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f08e801e460 con 0x7f08ec0cac10 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.779+0000 7f08e1ffb700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 0x7f08d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.779+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f08e8085320 con 0x7f08ec0cac10 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.779+0000 7f08f12ed700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 0x7f08d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:53.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.779+0000 7f08f12ed700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 0x7f08d00697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08ec16d170 tx=0x7f08dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:53.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.783+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08e8051fb0 con 0x7f08ec0cac10 2023-10-08T16:12:53.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:53.965+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64} v 0) v1 -- 0x7f08ec0c96e0 con 0x7f08ec0cac10 2023-10-08T16:12:54.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.720+0000 7f08e1ffb700 1 -- 172.21.15.170:0/4090599925 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "another-data-pool", "pg_num_min": 64}]=0 pool 'another-data-pool' created v167) v1 ==== 142+0+0 (secure 0 0 0) 0x7f08e80555d0 con 0x7f08ec0cac10 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 msgr2=0x7f08d00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 0x7f08d00697d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08ec16d170 tx=0x7f08dc008040 comp rx=0 tx=0).stop 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 msgr2=0x7f08ec0c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f08e800d890 tx=0x7f08e8008040 comp rx=0 tx=0).stop 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 shutdown_connections 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f08ec157760 0x7f08ec150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f08d0067320 0x7f08d00697d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08ec0cac10 0x7f08ec0c6790 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 --2- 172.21.15.170:0/4090599925 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f08ec157390 0x7f08ec14d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.724+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 >> 172.21.15.170:0/4090599925 conn(0x7f08ec06adf0 msgr2=0x7f08ec151020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.725+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 shutdown_connections 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.725+0000 7f08f2af0700 1 -- 172.21.15.170:0/4090599925 wait complete. 2023-10-08T16:12:54.726 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'another-data-pool' created 2023-10-08T16:12:54.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs another-data-pool 2023-10-08T16:12:54.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.856+0000 7f4fc4de6700 1 Processor -- start 2023-10-08T16:12:54.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.856+0000 7f4fc4de6700 1 -- start start 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fc4de6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc0141500 0x7f4fc00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fc4de6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4fc006c0a0 0x7f4fc00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fc4de6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc015c2a0 con 0x7f4fc006bcd0 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fc4de6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc00c9880 con 0x7f4fc006c0a0 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fc4de6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015b690 con 0x7f4fc0141500 2023-10-08T16:12:54.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fbe59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4fc006bcd0 0x7f4fc0140df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:39460/0 (socket says 172.21.15.170:39460) 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fbe59c700 1 -- 172.21.15.170:0/2475983447 learned_addr learned my addr 172.21.15.170:0/2475983447 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.857+0000 7f4fbed9d700 1 --2- 172.21.15.170:0/2475983447 >> v2:172.21.15.170:3300/0 conn(0x7f4fc006c0a0 0x7f4fc00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbdd9b700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc0141500 0x7f4fc00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbd59a700 1 -- 172.21.15.170:0/2475983447 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887631148 0 0) 0x7f4fc015c2a0 con 0x7f4fc006bcd0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbd59a700 1 -- 172.21.15.170:0/2475983447 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fa8003570 con 0x7f4fc006bcd0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbed9d700 1 -- 172.21.15.170:0/2475983447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc0141500 msgr2=0x7f4fc00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbed9d700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc0141500 0x7f4fc00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbed9d700 1 -- 172.21.15.170:0/2475983447 >> v1:172.21.15.170:6789/0 conn(0x7f4fc006bcd0 legacy=0x7f4fc0140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbed9d700 1 -- 172.21.15.170:0/2475983447 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc015b970 con 0x7f4fc006c0a0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbed9d700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00c9340 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098e0 tx=0x7f4fb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e5ca1d0adaf426 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:54.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbd59a700 1 -- 172.21.15.170:0/2475983447 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800b640 con 0x7f4fc006c0a0 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbd59a700 1 -- 172.21.15.170:0/2475983447 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800b7a0 con 0x7f4fc006c0a0 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fbd59a700 1 -- 172.21.15.170:0/2475983447 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800bb40 con 0x7f4fc006c0a0 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fc4de6700 1 -- 172.21.15.170:0/2475983447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 msgr2=0x7f4fc00c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.858+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00c9340 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f4fb80098e0 tx=0x7f4fb80091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 -- 172.21.15.170:0/2475983447 shutdown_connections 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00c9340 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/2475983447 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc0141500 0x7f4fc00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 -- 172.21.15.170:0/2475983447 >> 172.21.15.170:0/2475983447 conn(0x7f4fc00ce600 msgr2=0x7f4fc00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 -- 172.21.15.170:0/2475983447 shutdown_connections 2023-10-08T16:12:54.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.859+0000 7f4fc4de6700 1 -- 172.21.15.170:0/2475983447 wait complete. 2023-10-08T16:12:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 Processor -- start 2023-10-08T16:12:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 -- start start 2023-10-08T16:12:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc006bcd0 0x7f4fc013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc0141500 0x7f4fc013d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc013e9a0 con 0x7f4fc006c0a0 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc013dd90 con 0x7f4fc0141500 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fc4de6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc013e070 con 0x7f4fc006bcd0 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fbdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.860+0000 7f4fbdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:33006/0 (socket says 172.21.15.170:33006) 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 -- 172.21.15.170:0/3595558088 learned_addr learned my addr 172.21.15.170:0/3595558088 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbe59c700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc006bcd0 0x7f4fc013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbed9d700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc0141500 0x7f4fc013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 -- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc006bcd0 msgr2=0x7f4fc013ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc006bcd0 0x7f4fc013ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 -- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc0141500 msgr2=0x7f4fc013d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc0141500 0x7f4fc013d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 -- 172.21.15.170:0/3595558088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8009420 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fbdd9b700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d860 tx=0x7f4fb4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb40125f0 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4012750 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc013e350 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4012b10 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.861+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00b02f0 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.862+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc004c3c0 con 0x7f4fc006c0a0 2023-10-08T16:12:54.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.863+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4fb401e460 con 0x7f4fc006c0a0 2023-10-08T16:12:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.863+0000 7f4fae7fc700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 0x7f4fa00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.863+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6077+0+0 (secure 0 0 0) 0x7f4fb4085310 con 0x7f4fc006c0a0 2023-10-08T16:12:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.863+0000 7f4fbe59c700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 0x7f4fa00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.863+0000 7f4fbe59c700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 0x7f4fa00697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005c50 tx=0x7f4fb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:54.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:54.867+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4fb40558c0 con 0x7f4fc006c0a0 2023-10-08T16:12:55.093 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.092+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"} v 0) v1 -- 0x7f4fc015bc50 con 0x7f4fc006c0a0 2023-10-08T16:12:55.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.734+0000 7f4fae7fc700 1 -- 172.21.15.170:0/3595558088 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "another-data-pool"}]=0 added data pool 30 to fsmap v111) v1 ==== 141+0+0 (secure 0 0 0) 0x7f4fb4055450 con 0x7f4fc006c0a0 2023-10-08T16:12:55.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.737+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 msgr2=0x7f4fa00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.737+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 0x7f4fa00697d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005c50 tx=0x7f4fb0009040 comp rx=0 tx=0).stop 2023-10-08T16:12:55.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.737+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 msgr2=0x7f4fc00afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.737+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f4fb400d860 tx=0x7f4fb4008040 comp rx=0 tx=0).stop 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 shutdown_connections 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4fc0141500 0x7f4fc013d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4fc006bcd0 0x7f4fc013ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4fa0067320 0x7f4fa00697d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 --2- 172.21.15.170:0/3595558088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fc006c0a0 0x7f4fc00afdb0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 >> 172.21.15.170:0/3595558088 conn(0x7f4fc00ce600 msgr2=0x7f4fc006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 shutdown_connections 2023-10-08T16:12:55.739 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.738+0000 7f4fc4de6700 1 -- 172.21.15.170:0/3595558088 wait complete. 2023-10-08T16:12:55.740 INFO:teuthology.orchestra.run.smithi170.stderr:added data pool 30 to fsmap 2023-10-08T16:12:55.751 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:55.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.864+0000 7fb3af8de700 1 Processor -- start 2023-10-08T16:12:55.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.864+0000 7fb3af8de700 1 -- start start 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3af8de700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb3a80caa90 0x7fb3a80c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3af8de700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a815d7e0 con 0x7fb3a806c0a0 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3af8de700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a815dac0 con 0x7fb3a806bcd0 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3af8de700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80caef0 con 0x7fb3a80caa90 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3ade7b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb3a80caa90 0x7fb3a80c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:55.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3ace79700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb3a806c0a0 0x7fb3a815df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35194/0 (socket says 172.21.15.170:35194) 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.865+0000 7fb3ace79700 1 -- 172.21.15.170:0/461157376 learned_addr learned my addr 172.21.15.170:0/461157376 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3ade7b700 1 -- 172.21.15.170:0/461157376 >> v1:172.21.15.191:6789/0 conn(0x7fb3a806bcd0 legacy=0x7fb3a815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3ade7b700 1 -- 172.21.15.170:0/461157376 >> v1:172.21.15.170:6789/0 conn(0x7fb3a806c0a0 legacy=0x7fb3a815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3ade7b700 1 -- 172.21.15.170:0/461157376 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a815aa30 con 0x7fb3a80caa90 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3ade7b700 1 --2- 172.21.15.170:0/461157376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a80c8c40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264d817abba9c77f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3a7fff700 1 -- 172.21.15.170:0/461157376 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0009e80 con 0x7fb3a80caa90 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3a7fff700 1 -- 172.21.15.170:0/461157376 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a000b860 con 0x7fb3a80caa90 2023-10-08T16:12:55.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3a7fff700 1 -- 172.21.15.170:0/461157376 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000bbe0 con 0x7fb3a80caa90 2023-10-08T16:12:55.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 -- 172.21.15.170:0/461157376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 msgr2=0x7fb3a80c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 --2- 172.21.15.170:0/461157376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a80c8c40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a00091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 -- 172.21.15.170:0/461157376 shutdown_connections 2023-10-08T16:12:55.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 --2- 172.21.15.170:0/461157376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a80c8c40 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 -- 172.21.15.170:0/461157376 >> 172.21.15.170:0/461157376 conn(0x7fb3a80ce600 msgr2=0x7fb3a80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.866+0000 7fb3af8de700 1 -- 172.21.15.170:0/461157376 shutdown_connections 2023-10-08T16:12:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.867+0000 7fb3af8de700 1 -- 172.21.15.170:0/461157376 wait complete. 2023-10-08T16:12:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 Processor -- start 2023-10-08T16:12:55.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 -- start start 2023-10-08T16:12:55.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 0x7fb3a80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:55.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a806c0a0 0x7fb3a8169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:55.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:55.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.868+0000 7fb3af8de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b1980 con 0x7fb3a80caa90 2023-10-08T16:12:55.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3af8de700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b1c60 con 0x7fb3a806bcd0 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3af8de700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b1f40 con 0x7fb3a806c0a0 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47492/0 (socket says 172.21.15.170:47492) 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 -- 172.21.15.170:0/4122075028 learned_addr learned my addr 172.21.15.170:0/4122075028 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ad67a700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 0x7fb3a80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ace79700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a806c0a0 0x7fb3a8169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 -- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a806c0a0 msgr2=0x7fb3a8169a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a806c0a0 0x7fb3a8169a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 -- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 msgr2=0x7fb3a80b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 0x7fb3a80b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 -- 172.21.15.170:0/4122075028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a0009420 con 0x7fb3a80caa90 2023-10-08T16:12:55.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ad67a700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 0x7fb3a80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3ade7b700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8069220 tx=0x7fb3a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000bb50 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0025420 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00257c0 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a80b2220 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.869+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a8169f90 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.870+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a80cf510 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.871+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb3a0023070 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.871+0000 7fb3a57fa700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 0x7fb3940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.871+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fb3a00863e0 con 0x7fb3a80caa90 2023-10-08T16:12:55.873 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.871+0000 7fb3ad67a700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 0x7fb3940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:55.874 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.871+0000 7fb3ad67a700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 0x7fb3940697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb398007fd0 tx=0x7fb398009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:55.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:55.875+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3a0052ee0 con 0x7fb3a80caa90 2023-10-08T16:12:56.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.101+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3a804c3c0 con 0x7fb3a80caa90 2023-10-08T16:12:56.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.101+0000 7fb3a57fa700 1 -- 172.21.15.170:0/4122075028 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4613 (secure 0 0 0) 0x7fb3a0056500 con 0x7fb3a80caa90 2023-10-08T16:12:56.102 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:56.102 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:55.718968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:12:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.103+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 msgr2=0x7fb3940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.103+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 0x7fb3940697d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb398007fd0 tx=0x7fb398009040 comp rx=0 tx=0).stop 2023-10-08T16:12:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 msgr2=0x7fb3a8069a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8069220 tx=0x7fb3a0017000 comp rx=0 tx=0).stop 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 shutdown_connections 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb394067320 0x7fb3940697d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3a80caa90 0x7fb3a8069a70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb3a806bcd0 0x7fb3a80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 --2- 172.21.15.170:0/4122075028 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb3a806c0a0 0x7fb3a8169a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 >> 172.21.15.170:0/4122075028 conn(0x7fb3a80ce600 msgr2=0x7fb3a80c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 shutdown_connections 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.104+0000 7fb3af8de700 1 -- 172.21.15.170:0/4122075028 wait complete. 2023-10-08T16:12:56.105 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2023-10-08T16:12:56.116 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:56.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.230+0000 7f8470469700 1 Processor -- start 2023-10-08T16:12:56.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.230+0000 7f8470469700 1 -- start start 2023-10-08T16:12:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f8470469700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 0x7f846815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f8470469700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8468069560 0x7f84680699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f8470469700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8468069ee0 con 0x7f8468159c60 2023-10-08T16:12:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f8470469700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846815a160 con 0x7f8468069560 2023-10-08T16:12:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f8470469700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84680d13a0 con 0x7f8468159890 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f846ea06700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8468069560 0x7f84680699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f846ea06700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8468069560 0x7f84680699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47502/0 (socket says 172.21.15.170:47502) 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f846ea06700 1 -- 172.21.15.170:0/32411078 learned_addr learned my addr 172.21.15.170:0/32411078 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.231+0000 7f846e205700 1 --2- 172.21.15.170:0/32411078 >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 0x7f846815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846ea06700 1 -- 172.21.15.170:0/32411078 >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 msgr2=0x7f846815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846ea06700 1 --2- 172.21.15.170:0/32411078 >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 0x7f846815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846ea06700 1 -- 172.21.15.170:0/32411078 >> v1:172.21.15.170:6789/0 conn(0x7f8468159c60 legacy=0x7f846815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846ea06700 1 -- 172.21.15.170:0/32411078 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846815e140 con 0x7f8468069560 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846d203700 1 -- 172.21.15.170:0/32411078 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149222723 0 0) 0x7f8468069ee0 con 0x7f8468159c60 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846e205700 1 --2- 172.21.15.170:0/32411078 >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 0x7f846815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:56.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846ea06700 1 --2- 172.21.15.170:0/32411078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468069560 0x7f84680699a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f84680d1090 tx=0x7f8464009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247d87e6c8d5269 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846d203700 1 -- 172.21.15.170:0/32411078 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400b860 con 0x7f8468069560 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846d203700 1 -- 172.21.15.170:0/32411078 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846400b9c0 con 0x7f8468069560 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f846d203700 1 -- 172.21.15.170:0/32411078 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400bd60 con 0x7f8468069560 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f8470469700 1 -- 172.21.15.170:0/32411078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468069560 msgr2=0x7f84680699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.232+0000 7f8470469700 1 --2- 172.21.15.170:0/32411078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468069560 0x7f84680699a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f84680d1090 tx=0x7f8464009250 comp rx=0 tx=0).stop 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 -- 172.21.15.170:0/32411078 shutdown_connections 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 --2- 172.21.15.170:0/32411078 >> v2:172.21.15.191:3300/0 conn(0x7f8468159890 0x7f846815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 --2- 172.21.15.170:0/32411078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468069560 0x7f84680699a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 -- 172.21.15.170:0/32411078 >> 172.21.15.170:0/32411078 conn(0x7f8468068e20 msgr2=0x7f84680d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 -- 172.21.15.170:0/32411078 shutdown_connections 2023-10-08T16:12:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.233+0000 7f8470469700 1 -- 172.21.15.170:0/32411078 wait complete. 2023-10-08T16:12:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 Processor -- start 2023-10-08T16:12:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 -- start start 2023-10-08T16:12:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8468069560 0x7f846815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8468159c60 0x7f84681542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680ce480 con 0x7f8468159890 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468169930 con 0x7f8468159c60 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f8470469700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846815b8e0 con 0x7f8468069560 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f846da04700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f846da04700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47506/0 (socket says 172.21.15.170:47506) 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.234+0000 7f846da04700 1 -- 172.21.15.170:0/2699068565 learned_addr learned my addr 172.21.15.170:0/2699068565 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846e205700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8468069560 0x7f846815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846ea06700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8468159c60 0x7f84681542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 -- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8468069560 msgr2=0x7f846815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8468069560 0x7f846815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 -- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8468159c60 msgr2=0x7f84681542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8468159c60 0x7f84681542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 -- 172.21.15.170:0/2699068565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640094f0 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f846da04700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f845800c560 tx=0x7f845800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458011590 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8458011710 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8458011ad0 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84680b5a70 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.235+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846815b570 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.236+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846804c3c0 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.236+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f845801b3b0 con 0x7f8468159890 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.237+0000 7f845e7fc700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 0x7f844c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.237+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f8458084200 con 0x7f8468159890 2023-10-08T16:12:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.237+0000 7f846e205700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 0x7f844c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.237+0000 7f846e205700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 0x7f844c0697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8460007740 tx=0x7f84600099d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.241+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8458050d00 con 0x7f8468159890 2023-10-08T16:12:56.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.422+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f846815ba20 con 0x7f8468159890 2023-10-08T16:12:56.423 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.423+0000 7f845e7fc700 1 -- 172.21.15.170:0/2699068565 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) v1 ==== 74+0+16995 (secure 0 0 0) 0x7f8458017090 con 0x7f8468159890 2023-10-08T16:12:56.424 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:56.424 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":168,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:55.093966+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":164,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 msgr2=0x7f844c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 0x7f844c0697d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8460007740 tx=0x7f84600099d0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 msgr2=0x7f84680b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f845800c560 tx=0x7f845800c870 comp rx=0 tx=0).stop 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 shutdown_connections 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8468159c60 0x7f84681542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.426 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8468069560 0x7f846815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f844c067320 0x7f844c0697d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 --2- 172.21.15.170:0/2699068565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8468159890 0x7f84680b5380 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 >> 172.21.15.170:0/2699068565 conn(0x7f8468068e20 msgr2=0x7f8468157550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 shutdown_connections 2023-10-08T16:12:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.425+0000 7f8470469700 1 -- 172.21.15.170:0/2699068565 wait complete. 2023-10-08T16:12:56.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:56.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.551+0000 7f6835732700 1 Processor -- start 2023-10-08T16:12:56.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.551+0000 7f6835732700 1 -- start start 2023-10-08T16:12:56.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f6835732700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 0x7f683015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f6835732700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6830069560 0x7f68300699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f6835732700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683015a160 con 0x7f6830159c60 2023-10-08T16:12:56.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f6835732700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6830069f20 con 0x7f6830069560 2023-10-08T16:12:56.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f6835732700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300d13a0 con 0x7f6830159890 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f682f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6830069560 0x7f68300699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f682e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6830159c60 0x7f683015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35208/0 (socket says 172.21.15.170:35208) 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.552+0000 7f682e7fc700 1 -- 172.21.15.170:0/559923736 learned_addr learned my addr 172.21.15.170:0/559923736 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682effd700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 0x7f683015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682f7fe700 1 -- 172.21.15.170:0/559923736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 msgr2=0x7f683015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682f7fe700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 0x7f683015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682f7fe700 1 -- 172.21.15.170:0/559923736 >> v1:172.21.15.170:6789/0 conn(0x7f6830159c60 legacy=0x7f683015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682dffb700 1 -- 172.21.15.170:0/559923736 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386629984 0 0) 0x7f683015a160 con 0x7f6830159c60 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682f7fe700 1 -- 172.21.15.170:0/559923736 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683015e140 con 0x7f6830069560 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682effd700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 0x7f683015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682f7fe700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830069560 0x7f68300699e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f68280098e0 tx=0x7f68280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fad886154f8402 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682dffb700 1 -- 172.21.15.170:0/559923736 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828009e80 con 0x7f6830069560 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682dffb700 1 -- 172.21.15.170:0/559923736 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800b860 con 0x7f6830069560 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f682dffb700 1 -- 172.21.15.170:0/559923736 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800bbe0 con 0x7f6830069560 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f6835732700 1 -- 172.21.15.170:0/559923736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830069560 msgr2=0x7f68300699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.553+0000 7f6835732700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830069560 0x7f68300699e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f68280098e0 tx=0x7f68280091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 -- 172.21.15.170:0/559923736 shutdown_connections 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830159890 0x7f683015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 --2- 172.21.15.170:0/559923736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830069560 0x7f68300699e0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 -- 172.21.15.170:0/559923736 >> 172.21.15.170:0/559923736 conn(0x7f6830068e20 msgr2=0x7f68300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 -- 172.21.15.170:0/559923736 shutdown_connections 2023-10-08T16:12:56.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.554+0000 7f6835732700 1 -- 172.21.15.170:0/559923736 wait complete. 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.555+0000 7f6835732700 1 Processor -- start 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.555+0000 7f6835732700 1 -- start start 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.555+0000 7f6835732700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830069560 0x7f6830148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830159c60 0x7f68300b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683015ce20 con 0x7f6830159890 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683015c210 con 0x7f6830159c60 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683015c4f0 con 0x7f6830069560 2023-10-08T16:12:56.559 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47510/0 (socket says 172.21.15.170:47510) 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 -- 172.21.15.170:0/883248841 learned_addr learned my addr 172.21.15.170:0/883248841 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682effd700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830069560 0x7f6830148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682f7fe700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830159c60 0x7f68300b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 -- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830069560 msgr2=0x7f6830148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830069560 0x7f6830148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 -- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830159c60 msgr2=0x7f68300b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830159c60 0x7f68300b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 -- 172.21.15.170:0/883248841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009420 con 0x7f6830159890 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f682e7fc700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f681800c810 tx=0x7f6818007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818005c00 con 0x7f6830159890 2023-10-08T16:12:56.560 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818005d60 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683015c7d0 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818007320 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.556+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68301486a0 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.557+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683015cfd0 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.558+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f681801b400 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.558+0000 7f6826ffd700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 0x7f68100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.558+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f68180848b0 con 0x7f6830159890 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.558+0000 7f682effd700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 0x7f68100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.561 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.558+0000 7f682effd700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 0x7f68100697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f682000d560 tx=0x7f68200075a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.562+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f683015cfd0 con 0x7f6830159890 2023-10-08T16:12:56.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.792+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f683004c3c0 con 0x7f6830159890 2023-10-08T16:12:56.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.793+0000 7f6826ffd700 1 -- 172.21.15.170:0/883248841 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4613 (secure 0 0 0) 0x7f68180514b0 con 0x7f6830159890 2023-10-08T16:12:56.795 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:56.795 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":111,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:55.718968+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 msgr2=0x7f68100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 0x7f68100697d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f682000d560 tx=0x7f68200075a0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 msgr2=0x7f6830169d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f681800c810 tx=0x7f6818007db0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 shutdown_connections 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6830159c60 0x7f68300b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6830069560 0x7f6830148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6810067320 0x7f68100697d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 --2- 172.21.15.170:0/883248841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6830159890 0x7f6830169d30 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 >> 172.21.15.170:0/883248841 conn(0x7f6830068e20 msgr2=0x7f6830157950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 shutdown_connections 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.795+0000 7f6835732700 1 -- 172.21.15.170:0/883248841 wait complete. 2023-10-08T16:12:56.797 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 111 2023-10-08T16:12:56.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.923+0000 7fb09241d700 1 Processor -- start 2023-10-08T16:12:56.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.923+0000 7fb09241d700 1 -- start start 2023-10-08T16:12:56.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb09241d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb08c155310 0x7fb08c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb09241d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb09241d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0c77b0 con 0x7fb08c0688d0 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb09241d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c0c8720 con 0x7fb08c155310 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb09241d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c06ac60 con 0x7fb08c06b660 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb08bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb08c155310 0x7fb08c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb0909ba700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb08c0688d0 0x7fb08c15b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35212/0 (socket says 172.21.15.170:35212) 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.924+0000 7fb0909ba700 1 -- 172.21.15.170:0/1981201553 learned_addr learned my addr 172.21.15.170:0/1981201553 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08b7fe700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08bfff700 1 -- 172.21.15.170:0/1981201553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 msgr2=0x7fb08c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08bfff700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08bfff700 1 -- 172.21.15.170:0/1981201553 >> v1:172.21.15.170:6789/0 conn(0x7fb08c0688d0 legacy=0x7fb08c15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08bfff700 1 -- 172.21.15.170:0/1981201553 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c06b1d0 con 0x7fb08c155310 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08affd700 1 -- 172.21.15.170:0/1981201553 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311906742 0 0) 0x7fb08c0c77b0 con 0x7fb08c0688d0 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08b7fe700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08bfff700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0af4c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beac6875576709c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08affd700 1 -- 172.21.15.170:0/1981201553 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c009e80 con 0x7fb08c155310 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08affd700 1 -- 172.21.15.170:0/1981201553 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07c00b8f0 con 0x7fb08c155310 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb08affd700 1 -- 172.21.15.170:0/1981201553 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07c00bc70 con 0x7fb08c155310 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb09241d700 1 -- 172.21.15.170:0/1981201553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 msgr2=0x7fb08c0af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.925+0000 7fb09241d700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0af4c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0098e0 tx=0x7fb07c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 -- 172.21.15.170:0/1981201553 shutdown_connections 2023-10-08T16:12:56.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0af4c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 --2- 172.21.15.170:0/1981201553 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 -- 172.21.15.170:0/1981201553 >> 172.21.15.170:0/1981201553 conn(0x7fb08c0c9860 msgr2=0x7fb08c0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 -- 172.21.15.170:0/1981201553 shutdown_connections 2023-10-08T16:12:56.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.926+0000 7fb09241d700 1 -- 172.21.15.170:0/1981201553 wait complete. 2023-10-08T16:12:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 Processor -- start 2023-10-08T16:12:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 -- start start 2023-10-08T16:12:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 0x7fb08c1479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0b0370 con 0x7fb08c155310 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0afd60 con 0x7fb08c0688d0 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.927+0000 7fb09241d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0b2410 con 0x7fb08c06b660 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47532/0 (socket says 172.21.15.170:47532) 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 -- 172.21.15.170:0/3266428358 learned_addr learned my addr 172.21.15.170:0/3266428358 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb08b7fe700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb08bfff700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 0x7fb08c1479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 -- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 msgr2=0x7fb08c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 -- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 msgr2=0x7fb08c1479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:56.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 0x7fb08c1479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 -- 172.21.15.170:0/3266428358 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c009420 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb08bfff700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 0x7fb08c1479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb08b7fe700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb0909ba700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb08400cc90 tx=0x7fb084001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08400cea0 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08400ab70 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c0b0000 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0840070f0 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.928+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c0b2000 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.929+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08c064d80 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.930+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb0840074a0 con 0x7fb08c155310 2023-10-08T16:12:56.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.930+0000 7fb073fff700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 0x7fb07806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.930+0000 7fb08bfff700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 0x7fb07806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.930+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fb084085320 con 0x7fb08c155310 2023-10-08T16:12:56.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.931+0000 7fb08bfff700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 0x7fb07806dc60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0091e0 tx=0x7fb07c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:56.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:56.935+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb084051e60 con 0x7fb08c155310 2023-10-08T16:12:57.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.119+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb08c04c3c0 con 0x7fb08c155310 2023-10-08T16:12:57.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.120+0000 7fb073fff700 1 -- 172.21.15.170:0/3266428358 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16995 (secure 0 0 0) 0x7fb084055480 con 0x7fb08c155310 2023-10-08T16:12:57.120 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:57.120 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":169,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:56.720052+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"166","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:57.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 msgr2=0x7fb07806dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 0x7fb07806dc60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0091e0 tx=0x7fb07c017000 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 msgr2=0x7fb08c0b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb08400cc90 tx=0x7fb084001a80 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 shutdown_connections 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb07806b7b0 0x7fb07806dc60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb08c155310 0x7fb08c0b0800 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb08c0688d0 0x7fb08c1479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 --2- 172.21.15.170:0/3266428358 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb08c06b660 0x7fb08c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 >> 172.21.15.170:0/3266428358 conn(0x7fb08c0c9860 msgr2=0x7fb08c153930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:57.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 shutdown_connections 2023-10-08T16:12:57.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.122+0000 7fb09241d700 1 -- 172.21.15.170:0/3266428358 wait complete. 2023-10-08T16:12:57.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000975909 --yes-i-really-mean-it 2023-10-08T16:12:57.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.254+0000 7f8beafc5700 1 Processor -- start 2023-10-08T16:12:57.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.254+0000 7f8beafc5700 1 -- start start 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8beafc5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8be40b03c0 0x7f8be40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8beafc5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be40b0790 0x7f8be40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8beafc5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be40cfa00 con 0x7f8be41591e0 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8beafc5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be40cb580 con 0x7f8be40b03c0 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8beafc5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40b2de0 con 0x7f8be40b0790 2023-10-08T16:12:57.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8be40b03c0 0x7f8be40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8be40b03c0 0x7f8be40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47540/0 (socket says 172.21.15.170:47540) 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 -- 172.21.15.170:0/2662264021 learned_addr learned my addr 172.21.15.170:0/2662264021 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be97c2700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be40b0790 0x7f8be40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 -- 172.21.15.170:0/2662264021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be40b0790 msgr2=0x7f8be40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be40b0790 0x7f8be40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 -- 172.21.15.170:0/2662264021 >> v1:172.21.15.191:6789/0 conn(0x7f8be41591e0 legacy=0x7f8be4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.255+0000 7f8be9fc3700 1 -- 172.21.15.170:0/2662264021 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be40cf160 con 0x7f8be40b03c0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8be9fc3700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be40cea70 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8be00099b0 tx=0x7f8be0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366b22c78dc4df49 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8be8fc1700 1 -- 172.21.15.170:0/2662264021 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0009ea0 con 0x7f8be40b03c0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8be8fc1700 1 -- 172.21.15.170:0/2662264021 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be000b860 con 0x7f8be40b03c0 2023-10-08T16:12:57.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8be8fc1700 1 -- 172.21.15.170:0/2662264021 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000bc00 con 0x7f8be40b03c0 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 -- 172.21.15.170:0/2662264021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 msgr2=0x7f8be40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be40cea70 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8be00099b0 tx=0x7f8be0009250 comp rx=0 tx=0).stop 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 -- 172.21.15.170:0/2662264021 shutdown_connections 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be40cea70 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 --2- 172.21.15.170:0/2662264021 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be40b0790 0x7f8be40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 -- 172.21.15.170:0/2662264021 >> 172.21.15.170:0/2662264021 conn(0x7f8be4063230 msgr2=0x7f8be4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 -- 172.21.15.170:0/2662264021 shutdown_connections 2023-10-08T16:12:57.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.256+0000 7f8beafc5700 1 -- 172.21.15.170:0/2662264021 wait complete. 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.257+0000 7f8beafc5700 1 Processor -- start 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 -- start start 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be40b0790 0x7f8be415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8be9fc3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8be9fc3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47552/0 (socket says 172.21.15.170:47552) 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 0x7f8be40b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8be9fc3700 1 -- 172.21.15.170:0/191413249 learned_addr learned my addr 172.21.15.170:0/191413249 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be416da60 con 0x7f8be40b03c0 2023-10-08T16:12:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be415b170 con 0x7f8be40b0790 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be416d130 con 0x7f8be41591e0 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.258+0000 7f8be97c2700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be40b0790 0x7f8be415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8bea7c4700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 0x7f8be40b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 -- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 msgr2=0x7f8be40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 0x7f8be40b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 -- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be40b0790 msgr2=0x7f8be415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be40b0790 0x7f8be415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00094f0 con 0x7f8be40b03c0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8bea7c4700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 0x7f8be40b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8be9fc3700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8be40cfe40 tx=0x7f8be0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000bbe0 con 0x7f8be40b03c0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0011420 con 0x7f8be40b03c0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be416d410 con 0x7f8be40b03c0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be00117c0 con 0x7f8be40b03c0 2023-10-08T16:12:57.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.259+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be416a8d0 con 0x7f8be40b03c0 2023-10-08T16:12:57.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.260+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be404c3c0 con 0x7f8be40b03c0 2023-10-08T16:12:57.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.261+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8be0011bf0 con 0x7f8be40b03c0 2023-10-08T16:12:57.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.261+0000 7f8bd9ffb700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 0x7f8bc80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:57.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.261+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f8be00865d0 con 0x7f8be40b03c0 2023-10-08T16:12:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.261+0000 7f8be97c2700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 0x7f8bc80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.262+0000 7f8be97c2700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 0x7f8bc80697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8be4157310 tx=0x7f8bd40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:57.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.265+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8be00530d0 con 0x7f8be40b03c0 2023-10-08T16:12:57.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:57.478+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000975909", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8be416dba0 con 0x7f8bc8067320 2023-10-08T16:12:58.745 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.744+0000 7f8bd9ffb700 1 -- 172.21.15.170:0/191413249 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+127 (secure 0 0 0) 0x7f8bd4009ad0 con 0x7f8bc8067320 2023-10-08T16:12:58.747 INFO:teuthology.orchestra.run.smithi170.stdout:FS volume 'cephfs' renamed to 'volume_0000000000975909'. But failed to rename data pools as more than one data pool was found. 2023-10-08T16:12:58.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.747+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 msgr2=0x7f8bc80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.747+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 0x7f8bc80697d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8be4157310 tx=0x7f8bd40091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.747+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 msgr2=0x7f8be414ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.747+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8be40cfe40 tx=0x7f8be0017000 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 shutdown_connections 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8be41591e0 0x7f8be40b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8bc8067320 0x7f8bc80697d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8be40b03c0 0x7f8be414ba70 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 --2- 172.21.15.170:0/191413249 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8be40b0790 0x7f8be415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 >> 172.21.15.170:0/191413249 conn(0x7f8be4063230 msgr2=0x7f8be41579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 shutdown_connections 2023-10-08T16:12:58.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.748+0000 7f8beafc5700 1 -- 172.21.15.170:0/191413249 wait complete. 2023-10-08T16:12:58.762 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:12:58.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.881+0000 7ff6ff4b7700 1 Processor -- start 2023-10-08T16:12:58.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.881+0000 7ff6ff4b7700 1 -- start start 2023-10-08T16:12:58.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6ff4b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806bcd0 0x7ff6f80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6ff4b7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6f806c0a0 0x7ff6f815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6ff4b7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f815d7e0 con 0x7ff6f815df50 2023-10-08T16:12:58.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6ff4b7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f815dac0 con 0x7ff6f806c0a0 2023-10-08T16:12:58.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6ff4b7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f815aa30 con 0x7ff6f806bcd0 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6fca52700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6f806c0a0 0x7ff6f815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6fca52700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6f806c0a0 0x7ff6f815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47562/0 (socket says 172.21.15.170:47562) 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.882+0000 7ff6fca52700 1 -- 172.21.15.170:0/1547673915 learned_addr learned my addr 172.21.15.170:0/1547673915 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fd253700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806bcd0 0x7ff6f80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fca52700 1 -- 172.21.15.170:0/1547673915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806bcd0 msgr2=0x7ff6f80caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fca52700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806bcd0 0x7ff6f80caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fca52700 1 -- 172.21.15.170:0/1547673915 >> v1:172.21.15.191:6789/0 conn(0x7ff6f815df50 legacy=0x7ff6f80c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fca52700 1 -- 172.21.15.170:0/1547673915 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f815ad10 con 0x7ff6f806c0a0 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6fca52700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f806c0a0 0x7ff6f815a1e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80099b0 tx=0x7ff6e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae551855f01a0cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6effff700 1 -- 172.21.15.170:0/1547673915 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8009ea0 con 0x7ff6f806c0a0 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6effff700 1 -- 172.21.15.170:0/1547673915 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e800b860 con 0x7ff6f806c0a0 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6effff700 1 -- 172.21.15.170:0/1547673915 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e800bc20 con 0x7ff6f806c0a0 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1547673915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f806c0a0 msgr2=0x7ff6f815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.883+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f806c0a0 0x7ff6f815a1e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff6e80099b0 tx=0x7ff6e8009250 comp rx=0 tx=0).stop 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1547673915 shutdown_connections 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806bcd0 0x7ff6f80caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1547673915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f806c0a0 0x7ff6f815a1e0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1547673915 >> 172.21.15.170:0/1547673915 conn(0x7ff6f80ce600 msgr2=0x7ff6f80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:58.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1547673915 shutdown_connections 2023-10-08T16:12:58.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.884+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1547673915 wait complete. 2023-10-08T16:12:58.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.885+0000 7ff6ff4b7700 1 Processor -- start 2023-10-08T16:12:58.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.885+0000 7ff6ff4b7700 1 -- start start 2023-10-08T16:12:58.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f806bcd0 0x7ff6f80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806c0a0 0x7ff6f80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80c9640 con 0x7ff6f815df50 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f816a270 con 0x7ff6f806bcd0 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80b1980 con 0x7ff6f806c0a0 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47576/0 (socket says 172.21.15.170:47576) 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 -- 172.21.15.170:0/1851868861 learned_addr learned my addr 172.21.15.170:0/1851868861 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fca52700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806c0a0 0x7ff6f80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fd253700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f806bcd0 0x7ff6f80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 -- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806c0a0 msgr2=0x7ff6f80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806c0a0 0x7ff6f80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 -- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f806bcd0 msgr2=0x7ff6f80b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f806bcd0 0x7ff6f80b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 -- 172.21.15.170:0/1851868861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80094f0 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6fda54700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c560 tx=0x7ff6f0007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011590 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00116f0 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80b1c60 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011ab0 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.886+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f80b2220 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.887+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f804c3c0 con 0x7ff6f815df50 2023-10-08T16:12:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.888+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff6f001b3b0 con 0x7ff6f815df50 2023-10-08T16:12:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.888+0000 7ff6ed7fa700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 0x7ff6dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.888+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7ff6f00841c0 con 0x7ff6f815df50 2023-10-08T16:12:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.888+0000 7ff6fd253700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 0x7ff6dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.889+0000 7ff6fd253700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 0x7ff6dc0697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009fd0 tx=0x7ff6f400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:58.892+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6f0050c70 con 0x7ff6f815df50 2023-10-08T16:12:59.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.102+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6f816a3b0 con 0x7ff6dc067320 2023-10-08T16:12:59.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.104+0000 7ff6ed7fa700 1 -- 172.21.15.170:0/1851868861 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7ff6f816a3b0 con 0x7ff6dc067320 2023-10-08T16:12:59.104 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:12:59.104 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:12:59.104 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000975909" 2023-10-08T16:12:59.104 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:12:59.105 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:12:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 msgr2=0x7ff6dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 0x7ff6dc0697d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009fd0 tx=0x7ff6f400b040 comp rx=0 tx=0).stop 2023-10-08T16:12:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 msgr2=0x7ff6f8159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c560 tx=0x7ff6f0007ef0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.107 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 shutdown_connections 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff6dc067320 0x7ff6dc0697d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6f815df50 0x7ff6f8159410 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff6f806bcd0 0x7ff6f80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 --2- 172.21.15.170:0/1851868861 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff6f806c0a0 0x7ff6f80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 >> 172.21.15.170:0/1851868861 conn(0x7ff6f80ce600 msgr2=0x7ff6f80c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:59.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 shutdown_connections 2023-10-08T16:12:59.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.106+0000 7ff6ff4b7700 1 -- 172.21.15.170:0/1851868861 wait complete. 2023-10-08T16:12:59.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:12:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.233+0000 7ff041019700 1 Processor -- start 2023-10-08T16:12:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 -- start start 2023-10-08T16:12:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff03c159c60 0x7ff03c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c15a160 con 0x7ff03c15db10 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c0d13a0 con 0x7ff03c159c60 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff041019700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c15e140 con 0x7ff03c159890 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff03a59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff03c159c60 0x7ff03c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.234+0000 7ff03b59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff03c15db10 0x7ff03c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35220/0 (socket says 172.21.15.170:35220) 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03b59e700 1 -- 172.21.15.170:0/862790242 learned_addr learned my addr 172.21.15.170:0/862790242 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03ad9d700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03a59c700 1 -- 172.21.15.170:0/862790242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 msgr2=0x7ff03c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03a59c700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03a59c700 1 -- 172.21.15.170:0/862790242 >> v1:172.21.15.170:6789/0 conn(0x7ff03c15db10 legacy=0x7ff03c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03a59c700 1 -- 172.21.15.170:0/862790242 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c15a3d0 con 0x7ff03c159c60 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff039d9b700 1 -- 172.21.15.170:0/862790242 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909289187 0 0) 0x7ff03c15a160 con 0x7ff03c15db10 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff03a59c700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c15d5d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098e0 tx=0x7ff0300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b30aee0c5921ac5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff039d9b700 1 -- 172.21.15.170:0/862790242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030009e80 con 0x7ff03c159c60 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff039d9b700 1 -- 172.21.15.170:0/862790242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000b860 con 0x7ff03c159c60 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.235+0000 7ff039d9b700 1 -- 172.21.15.170:0/862790242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000bbe0 con 0x7ff03c159c60 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 -- 172.21.15.170:0/862790242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 msgr2=0x7ff03c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c15d5d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7ff0300098e0 tx=0x7ff0300091a0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 -- 172.21.15.170:0/862790242 shutdown_connections 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 --2- 172.21.15.170:0/862790242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c15d5d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 -- 172.21.15.170:0/862790242 >> 172.21.15.170:0/862790242 conn(0x7ff03c068e20 msgr2=0x7ff03c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 -- 172.21.15.170:0/862790242 shutdown_connections 2023-10-08T16:12:59.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.236+0000 7ff041019700 1 -- 172.21.15.170:0/862790242 wait complete. 2023-10-08T16:12:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.237+0000 7ff041019700 1 Processor -- start 2023-10-08T16:12:59.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.237+0000 7ff041019700 1 -- start start 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03c15db10 0x7ff03c169e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0c9b20 con 0x7ff03c159c60 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c16a960 con 0x7ff03c15db10 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c16ac40 con 0x7ff03c159890 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47590/0 (socket says 172.21.15.170:47590) 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 -- 172.21.15.170:0/2131791538 learned_addr learned my addr 172.21.15.170:0/2131791538 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03ad9d700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03b59e700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03c15db10 0x7ff03c169e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 -- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 msgr2=0x7ff03c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 -- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03c15db10 msgr2=0x7ff03c169e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03c15db10 0x7ff03c169e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 -- 172.21.15.170:0/2131791538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff030009420 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff03a59c700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff030013000 tx=0x7ff030017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03000b9d0 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03000bb30 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030025670 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c0c94d0 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.238+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c0cea90 con 0x7ff03c159c60 2023-10-08T16:12:59.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.239+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c16ae60 con 0x7ff03c159c60 2023-10-08T16:12:59.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.240+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff030023070 con 0x7ff03c159c60 2023-10-08T16:12:59.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.240+0000 7ff022ffd700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 0x7ff0280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.240+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7ff0300864e0 con 0x7ff03c159c60 2023-10-08T16:12:59.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.240+0000 7ff03ad9d700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 0x7ff0280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.241+0000 7ff03ad9d700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 0x7ff0280697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0075a0 tx=0x7ff02c0097a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.245+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff030052f90 con 0x7ff03c159c60 2023-10-08T16:12:59.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.471+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff03c04c3c0 con 0x7ff03c159c60 2023-10-08T16:12:59.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.472+0000 7ff022ffd700 1 -- 172.21.15.170:0/2131791538 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4630 (secure 0 0 0) 0x7ff0300565b0 con 0x7ff03c159c60 2023-10-08T16:12:59.472 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:59.473 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:57.735496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000975909","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:12:59.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 msgr2=0x7ff0280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 0x7ff0280697d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0075a0 tx=0x7ff02c0097a0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 msgr2=0x7ff03c0c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7ff030013000 tx=0x7ff030017000 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 shutdown_connections 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff03c15db10 0x7ff03c169e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff03c159890 0x7ff03c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff028067320 0x7ff0280697d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 --2- 172.21.15.170:0/2131791538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff03c159c60 0x7ff03c0c9fb0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 >> 172.21.15.170:0/2131791538 conn(0x7ff03c068e20 msgr2=0x7ff03c1531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 shutdown_connections 2023-10-08T16:12:59.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.474+0000 7ff041019700 1 -- 172.21.15.170:0/2131791538 wait complete. 2023-10-08T16:12:59.476 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 112 2023-10-08T16:12:59.487 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:12:59.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.603+0000 7fb4c6523700 1 Processor -- start 2023-10-08T16:12:59.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.603+0000 7fb4c6523700 1 -- start start 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c6523700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb4c0069560 0x7fb4c00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c6523700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c0069ee0 con 0x7fb4c0159c60 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c6523700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c015a160 con 0x7fb4c0159890 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c6523700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c00d13a0 con 0x7fb4c0069560 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c4ac0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb4c0069560 0x7fb4c00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4bf7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb4c0159c60 0x7fb4c015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35228/0 (socket says 172.21.15.170:35228) 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4bf7fe700 1 -- 172.21.15.170:0/25904947 learned_addr learned my addr 172.21.15.170:0/25904947 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c4ac0700 1 -- 172.21.15.170:0/25904947 >> v1:172.21.15.191:6789/0 conn(0x7fb4c0159890 legacy=0x7fb4c015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c4ac0700 1 -- 172.21.15.170:0/25904947 >> v1:172.21.15.170:6789/0 conn(0x7fb4c0159c60 legacy=0x7fb4c015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c4ac0700 1 -- 172.21.15.170:0/25904947 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c015e140 con 0x7fb4c0069560 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4beffd700 1 -- 172.21.15.170:0/25904947 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402166246 0 0) 0x7fb4c0069ee0 con 0x7fb4c0159c60 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.604+0000 7fb4c4ac0700 1 --2- 172.21.15.170:0/25904947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0069560 0x7fb4c00699a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009e60 tx=0x7fb4b8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854dff10f213c232 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4beffd700 1 -- 172.21.15.170:0/25904947 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800e070 con 0x7fb4c0069560 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4beffd700 1 -- 172.21.15.170:0/25904947 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800bc30 con 0x7fb4c0069560 2023-10-08T16:12:59.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4beffd700 1 -- 172.21.15.170:0/25904947 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8012430 con 0x7fb4c0069560 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 -- 172.21.15.170:0/25904947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0069560 msgr2=0x7fb4c00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 --2- 172.21.15.170:0/25904947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0069560 0x7fb4c00699a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009e60 tx=0x7fb4b8009780 comp rx=0 tx=0).stop 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 -- 172.21.15.170:0/25904947 shutdown_connections 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 --2- 172.21.15.170:0/25904947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0069560 0x7fb4c00699a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 -- 172.21.15.170:0/25904947 >> 172.21.15.170:0/25904947 conn(0x7fb4c0068e20 msgr2=0x7fb4c00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:59.609 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 -- 172.21.15.170:0/25904947 shutdown_connections 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.605+0000 7fb4c6523700 1 -- 172.21.15.170:0/25904947 wait complete. 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.606+0000 7fb4c6523700 1 Processor -- start 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.606+0000 7fb4c6523700 1 -- start start 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4c0069560 0x7fb4c01542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 0x7fb4c00b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00b3800 con 0x7fb4c0159890 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00cd5c0 con 0x7fb4c0159c60 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c6523700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00cd8a0 con 0x7fb4c0069560 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.610 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47606/0 (socket says 172.21.15.170:47606) 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 -- 172.21.15.170:0/2150728653 learned_addr learned my addr 172.21.15.170:0/2150728653 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c4ac0700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 0x7fb4c00b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bffff700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4c0069560 0x7fb4c01542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 -- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4c0069560 msgr2=0x7fb4c01542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4c0069560 0x7fb4c01542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 -- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 msgr2=0x7fb4c00b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 0x7fb4c00b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4bf7fe700 1 -- 172.21.15.170:0/2150728653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b8009a20 con 0x7fb4c0159890 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.607+0000 7fb4c4ac0700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 0x7fb4c00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4bf7fe700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c5b0 tx=0x7fb4b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.611 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005b30 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4005c90 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c00cdb80 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4007300 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.608+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c0154810 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.609+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c004c3c0 con 0x7fb4c0159890 2023-10-08T16:12:59.612 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.609+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb4b401b3b0 con 0x7fb4c0159890 2023-10-08T16:12:59.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.609+0000 7fb4a7fff700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 0x7fb4ac0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:12:59.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.609+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fb4b4084430 con 0x7fb4c0159890 2023-10-08T16:12:59.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.609+0000 7fb4bffff700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 0x7fb4ac0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:12:59.613 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.610+0000 7fb4bffff700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 0x7fb4ac0697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0003860 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:12:59.614 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.614+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4b4050ee0 con 0x7fb4c0159890 2023-10-08T16:12:59.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.801+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4c006a5b0 con 0x7fb4c0159890 2023-10-08T16:12:59.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.802+0000 7fb4a7fff700 1 -- 172.21.15.170:0/2150728653 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17053 (secure 0 0 0) 0x7fb4b4054500 con 0x7fb4c0159890 2023-10-08T16:12:59.803 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:12:59.803 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":171,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:58.736978+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000975909.meta","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000975909"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 msgr2=0x7fb4ac0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 0x7fb4ac0697d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0003860 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 msgr2=0x7fb4c00c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c5b0 tx=0x7fb4b4001070 comp rx=0 tx=0).stop 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 shutdown_connections 2023-10-08T16:12:59.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4c0159c60 0x7fb4c00b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4c0069560 0x7fb4c01542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb4ac067320 0x7fb4ac0697d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 --2- 172.21.15.170:0/2150728653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c0159890 0x7fb4c00c9540 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 >> 172.21.15.170:0/2150728653 conn(0x7fb4c0068e20 msgr2=0x7fb4c015aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.804+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 shutdown_connections 2023-10-08T16:12:59.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:12:59.805+0000 7fb4c6523700 1 -- 172.21.15.170:0/2150728653 wait complete. 2023-10-08T16:12:59.817 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:12:59.818 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:59.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:12:59.907 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:12:59.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:00.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.021+0000 7fd21a0f4700 1 Processor -- start 2023-10-08T16:13:00.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.021+0000 7fd21a0f4700 1 -- start start 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd21a0f4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd214159890 0x7fd21415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd21a0f4700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd214159c60 0x7fd21415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd21a0f4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21415a160 con 0x7fd21415db10 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd21a0f4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21415df50 con 0x7fd214159890 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd21a0f4700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2140d13a0 con 0x7fd214159c60 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd2137fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd214159890 0x7fd21415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd213fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd21415db10 0x7fd214069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35238/0 (socket says 172.21.15.170:35238) 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd213fff700 1 -- 172.21.15.170:0/2731042610 learned_addr learned my addr 172.21.15.170:0/2731042610 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd20bfff700 1 --2- 172.21.15.170:0/2731042610 >> v2:172.21.15.191:3300/0 conn(0x7fd214159c60 0x7fd21415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd2137fe700 1 -- 172.21.15.170:0/2731042610 >> v2:172.21.15.191:3300/0 conn(0x7fd214159c60 msgr2=0x7fd21415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd2137fe700 1 --2- 172.21.15.170:0/2731042610 >> v2:172.21.15.191:3300/0 conn(0x7fd214159c60 0x7fd21415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd2137fe700 1 -- 172.21.15.170:0/2731042610 >> v1:172.21.15.170:6789/0 conn(0x7fd21415db10 legacy=0x7fd214069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.022+0000 7fd2137fe700 1 -- 172.21.15.170:0/2731042610 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21415e140 con 0x7fd214159890 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd2137fe700 1 --2- 172.21.15.170:0/2731042610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd21415abd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b67f0 tx=0x7fd2040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1287c884d715067 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd212ffd700 1 -- 172.21.15.170:0/2731042610 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400b6b0 con 0x7fd214159890 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd212ffd700 1 -- 172.21.15.170:0/2731042610 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20400b810 con 0x7fd214159890 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd212ffd700 1 -- 172.21.15.170:0/2731042610 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400bbb0 con 0x7fd214159890 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 -- 172.21.15.170:0/2731042610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 msgr2=0x7fd21415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/2731042610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd21415abd0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd2140b67f0 tx=0x7fd2040091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 -- 172.21.15.170:0/2731042610 shutdown_connections 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/2731042610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd21415abd0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/2731042610 >> v2:172.21.15.191:3300/0 conn(0x7fd214159c60 0x7fd21415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 -- 172.21.15.170:0/2731042610 >> 172.21.15.170:0/2731042610 conn(0x7fd214068e20 msgr2=0x7fd2140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 -- 172.21.15.170:0/2731042610 shutdown_connections 2023-10-08T16:13:00.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.023+0000 7fd21a0f4700 1 -- 172.21.15.170:0/2731042610 wait complete. 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.024+0000 7fd21a0f4700 1 Processor -- start 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.024+0000 7fd21a0f4700 1 -- start start 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 0x7fd214158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd21415db10 0x7fd2140c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.029 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140b41c0 con 0x7fd214159890 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2140b4350 con 0x7fd21415db10 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd21a0f4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd214158bb0 con 0x7fd214159c60 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47632/0 (socket says 172.21.15.170:47632) 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 -- 172.21.15.170:0/1114728270 learned_addr learned my addr 172.21.15.170:0/1114728270 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd213fff700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd21415db10 0x7fd2140c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd20bfff700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 0x7fd214158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 -- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 msgr2=0x7fd214158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 0x7fd214158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 -- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd21415db10 msgr2=0x7fd2140c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd21415db10 0x7fd2140c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd2137fe700 1 -- 172.21.15.170:0/1114728270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009420 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.025+0000 7fd20bfff700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 0x7fd214158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd2137fe700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b0e0 tx=0x7fd204017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400bb90 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204011420 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2140c8890 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040117c0 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.026+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2140c8e50 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.027+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21404c3c0 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.027+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd204011bf0 con 0x7fd214159890 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.027+0000 7fd20b7fe700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 0x7fd1f40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.028+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fd204086780 con 0x7fd214159890 2023-10-08T16:13:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.028+0000 7fd20bfff700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 0x7fd1f40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.028+0000 7fd20bfff700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 0x7fd1f40697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd214063ed0 tx=0x7fd1fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.032+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd204053230 con 0x7fd214159890 2023-10-08T16:13:00.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.257+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd21406a5b0 con 0x7fd214159890 2023-10-08T16:13:00.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.258+0000 7fd20b7fe700 1 -- 172.21.15.170:0/1114728270 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4630 (secure 0 0 0) 0x7fd204056850 con 0x7fd214159890 2023-10-08T16:13:00.258 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:00.258 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:57.735496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000975909","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:13:00.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 msgr2=0x7fd1f40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 0x7fd1f40697d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd214063ed0 tx=0x7fd1fc008040 comp rx=0 tx=0).stop 2023-10-08T16:13:00.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 msgr2=0x7fd2140c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b0e0 tx=0x7fd204017000 comp rx=0 tx=0).stop 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 shutdown_connections 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd21415db10 0x7fd2140c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd1f4067320 0x7fd1f40697d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd214159890 0x7fd2140c8260 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 --2- 172.21.15.170:0/1114728270 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd214159c60 0x7fd214158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 >> 172.21.15.170:0/1114728270 conn(0x7fd214068e20 msgr2=0x7fd2140c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 shutdown_connections 2023-10-08T16:13:00.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.260+0000 7fd21a0f4700 1 -- 172.21.15.170:0/1114728270 wait complete. 2023-10-08T16:13:00.262 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 112 2023-10-08T16:13:00.272 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:00.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.387+0000 7f6b18d97700 1 Processor -- start 2023-10-08T16:13:00.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.387+0000 7f6b18d97700 1 -- start start 2023-10-08T16:13:00.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b18d97700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6b14159c60 0x7f6b1415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b18d97700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1415a160 con 0x7f6b1415db10 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b18d97700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b140d13a0 con 0x7f6b14159890 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b18d97700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1415df50 con 0x7f6b14159c60 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b11d9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6b14159c60 0x7f6b1415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b12d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6b1415db10 0x7f6b14069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35248/0 (socket says 172.21.15.170:35248) 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b12d9d700 1 -- 172.21.15.170:0/4228139057 learned_addr learned my addr 172.21.15.170:0/4228139057 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b11d9b700 1 -- 172.21.15.170:0/4228139057 >> v1:172.21.15.191:6789/0 conn(0x7f6b14159890 legacy=0x7f6b1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b11d9b700 1 -- 172.21.15.170:0/4228139057 >> v1:172.21.15.170:6789/0 conn(0x7f6b1415db10 legacy=0x7f6b14069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b11d9b700 1 -- 172.21.15.170:0/4228139057 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1415e140 con 0x7f6b14159c60 2023-10-08T16:13:00.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b1159a700 1 -- 172.21.15.170:0/4228139057 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116771432 0 0) 0x7f6b1415a160 con 0x7f6b1415db10 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b11d9b700 1 --2- 172.21.15.170:0/4228139057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1415d5d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6b080098e0 tx=0x7f6b080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885513787190a1b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b1159a700 1 -- 172.21.15.170:0/4228139057 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08009e80 con 0x7f6b14159c60 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b1159a700 1 -- 172.21.15.170:0/4228139057 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0800b860 con 0x7f6b14159c60 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.388+0000 7f6b1159a700 1 -- 172.21.15.170:0/4228139057 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800bbe0 con 0x7f6b14159c60 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 -- 172.21.15.170:0/4228139057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 msgr2=0x7f6b1415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 --2- 172.21.15.170:0/4228139057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1415d5d0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6b080098e0 tx=0x7f6b080091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 -- 172.21.15.170:0/4228139057 shutdown_connections 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 --2- 172.21.15.170:0/4228139057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1415d5d0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 -- 172.21.15.170:0/4228139057 >> 172.21.15.170:0/4228139057 conn(0x7f6b14068e20 msgr2=0x7f6b140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 -- 172.21.15.170:0/4228139057 shutdown_connections 2023-10-08T16:13:00.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.389+0000 7f6b18d97700 1 -- 172.21.15.170:0/4228139057 wait complete. 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.390+0000 7f6b18d97700 1 Processor -- start 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.390+0000 7f6b18d97700 1 -- start start 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b14159890 0x7f6b1416ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1415db10 0x7f6b14169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1416aea0 con 0x7f6b14159c60 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1416a8e0 con 0x7f6b14159890 2023-10-08T16:13:00.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1416abc0 con 0x7f6b1415db10 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47646/0 (socket says 172.21.15.170:47646) 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 -- 172.21.15.170:0/3065463494 learned_addr learned my addr 172.21.15.170:0/3065463494 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 -- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1415db10 msgr2=0x7f6b14169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1415db10 0x7f6b14169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 -- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b14159890 msgr2=0x7f6b1416ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b1259c700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b14159890 0x7f6b1416ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b14159890 0x7f6b1416ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 -- 172.21.15.170:0/3065463494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08009420 con 0x7f6b14159c60 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b11d9b700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6b14070e20 tx=0x7f6b08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0801f070 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b08011420 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b140b36c0 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800bbe0 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.391+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b140b5330 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.392+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1404c3c0 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.393+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6b08011a50 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.393+0000 7f6b027fc700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 0x7f6af40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.393+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f6b08086480 con 0x7f6b14159c60 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.393+0000 7f6b1259c700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 0x7f6af40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.394+0000 7f6b1259c700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 0x7f6af40697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6b04001760 tx=0x7f6b04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.397+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b08052f30 con 0x7f6b14159c60 2023-10-08T16:13:00.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.581+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b1406a5b0 con 0x7f6b14159c60 2023-10-08T16:13:00.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.582+0000 7f6b027fc700 1 -- 172.21.15.170:0/3065463494 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f6b08056550 con 0x7f6b14159c60 2023-10-08T16:13:00.583 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:00.583 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":171,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:58.736978+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000975909.meta","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000975909"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 msgr2=0x7f6af40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 0x7f6af40697d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6b04001760 tx=0x7f6b04009040 comp rx=0 tx=0).stop 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 msgr2=0x7f6b1416b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6b14070e20 tx=0x7f6b08017000 comp rx=0 tx=0).stop 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 shutdown_connections 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6b1415db10 0x7f6b14169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6b14159890 0x7f6b1416ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6af4067320 0x7f6af40697d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 --2- 172.21.15.170:0/3065463494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b14159c60 0x7f6b1416b080 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 >> 172.21.15.170:0/3065463494 conn(0x7f6b14068e20 msgr2=0x7f6b140c70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 shutdown_connections 2023-10-08T16:13:00.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.584+0000 7f6b18d97700 1 -- 172.21.15.170:0/3065463494 wait complete. 2023-10-08T16:13:00.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 28, 'pool_name': 'cephfs.volume_0000000000975909.meta', 'create_time': '2023-10-08T16:12:41.449554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000975909'}}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:12:41.996094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000975909'}}}, {'pool': 30, 'pool_name': 'another-data-pool', 'create_time': '2023-10-08T16:12:53.983782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000975909'}}}] 2023-10-08T16:13:00.597 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:13:00.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.715+0000 7f99e179f700 1 Processor -- start 2023-10-08T16:13:00.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.715+0000 7f99e179f700 1 -- start start 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e179f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e179f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f99dc0b33b0 0x7f99dc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e179f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0b2080 con 0x7f99dc0b4e70 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e179f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc0cfb00 con 0x7f99dc0b33b0 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e179f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0b5c40 con 0x7f99dc0b16d0 2023-10-08T16:13:00.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f99dc0b33b0 0x7f99dc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f99dc0b33b0 0x7f99dc0b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47660/0 (socket says 172.21.15.170:47660) 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 -- 172.21.15.170:0/1758606334 learned_addr learned my addr 172.21.15.170:0/1758606334 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99db7fe700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 -- 172.21.15.170:0/1758606334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 msgr2=0x7f99dc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.716+0000 7f99e0f9e700 1 -- 172.21.15.170:0/1758606334 >> v1:172.21.15.191:6789/0 conn(0x7f99dc0b4e70 legacy=0x7f99dc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e0f9e700 1 -- 172.21.15.170:0/1758606334 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0b5f20 con 0x7f99dc0b33b0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e0f9e700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc0b3830 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009f20 tx=0x7f99d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c652aef1a36f823a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99daffd700 1 -- 172.21.15.170:0/1758606334 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d400e070 con 0x7f99dc0b33b0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99daffd700 1 -- 172.21.15.170:0/1758606334 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d400ba80 con 0x7f99dc0b33b0 2023-10-08T16:13:00.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99daffd700 1 -- 172.21.15.170:0/1758606334 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40092e0 con 0x7f99dc0b33b0 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 -- 172.21.15.170:0/1758606334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 msgr2=0x7f99dc0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc0b3830 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f99d4009f20 tx=0x7f99d4009840 comp rx=0 tx=0).stop 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 -- 172.21.15.170:0/1758606334 shutdown_connections 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc0b3830 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 --2- 172.21.15.170:0/1758606334 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 -- 172.21.15.170:0/1758606334 >> 172.21.15.170:0/1758606334 conn(0x7f99dc0d10c0 msgr2=0x7f99dc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 -- 172.21.15.170:0/1758606334 shutdown_connections 2023-10-08T16:13:00.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.717+0000 7f99e179f700 1 -- 172.21.15.170:0/1758606334 wait complete. 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.718+0000 7f99e179f700 1 Processor -- start 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 -- start start 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99dc0b4e70 0x7f99dc16d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c7e00 con 0x7f99dc0b33b0 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c80e0 con 0x7f99dc0b4e70 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99e179f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0c83c0 con 0x7f99dc0b16d0 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99db7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47674/0 (socket says 172.21.15.170:47674) 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.719+0000 7f99db7fe700 1 -- 172.21.15.170:0/2373153205 learned_addr learned my addr 172.21.15.170:0/2373153205 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99e0f9e700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99dc0b4e70 0x7f99dc16d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 -- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 msgr2=0x7f99dc15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99dbfff700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 -- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99dc0b4e70 msgr2=0x7f99dc16d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99dc0b4e70 0x7f99dc16d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 -- 172.21.15.170:0/2373153205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d4009ae0 con 0x7f99dc0b33b0 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99dbfff700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99db7fe700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f99d000d890 tx=0x7f99d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000b600 con 0x7f99dc0b33b0 2023-10-08T16:13:00.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d000b760 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc0c86a0 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d000bae0 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.720+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc15eb80 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.721+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc088990 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.721+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f99d001e460 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.722+0000 7f99c3fff700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 0x7f99c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.722+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f99d0085530 con 0x7f99dc0b33b0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.722+0000 7f99dbfff700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 0x7f99c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:00.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.722+0000 7f99dbfff700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 0x7f99c80697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0017a0 tx=0x7f99cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:00.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.726+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99d0051fe0 con 0x7f99dc0b33b0 2023-10-08T16:13:00.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.910+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99dc04bb80 con 0x7f99dc0b33b0 2023-10-08T16:13:00.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.911+0000 7f99c3fff700 1 -- 172.21.15.170:0/2373153205 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99d0055600 con 0x7f99dc0b33b0 2023-10-08T16:13:00.911 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:13:00.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 msgr2=0x7f99c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 0x7f99c80697d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0017a0 tx=0x7f99cc009040 comp rx=0 tx=0).stop 2023-10-08T16:13:00.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 msgr2=0x7f99dc16d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f99d000d890 tx=0x7f99d0008040 comp rx=0 tx=0).stop 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 shutdown_connections 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f99dc0b4e70 0x7f99dc16d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f99dc0b16d0 0x7f99dc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99c8067320 0x7f99c80697d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 --2- 172.21.15.170:0/2373153205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99dc0b33b0 0x7f99dc16d310 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 >> 172.21.15.170:0/2373153205 conn(0x7f99dc0d10c0 msgr2=0x7f99dc158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 shutdown_connections 2023-10-08T16:13:00.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:00.913+0000 7f99e179f700 1 -- 172.21.15.170:0/2373153205 wait complete. 2023-10-08T16:13:00.925 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000975909.meta pg_num 2023-10-08T16:13:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.042+0000 7f010ab1a700 1 Processor -- start 2023-10-08T16:13:01.043 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.042+0000 7f010ab1a700 1 -- start start 2023-10-08T16:13:01.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f010ab1a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f010ab1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f01040699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f010ab1a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0104069ee0 con 0x7f0104159c60 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f010ab1a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010415a160 con 0x7f0104159890 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f010ab1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01040d13a0 con 0x7f0104069560 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f00fbfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0104159c60 0x7f010415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35252/0 (socket says 172.21.15.170:35252) 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f00fbfff700 1 -- 172.21.15.170:0/2980619121 learned_addr learned my addr 172.21.15.170:0/2980619121 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929364886 0 0) 0x7f0104069ee0 con 0x7f0104159c60 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f01088b6700 1 --2- 172.21.15.170:0/2980619121 >> v2:172.21.15.191:3300/0 conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f01090b7700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f01040699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.043+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ec003570 con 0x7f0104159c60 2023-10-08T16:13:01.047 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931444799 0 0) 0x7f00ec003570 con 0x7f0104159c60 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 msgr2=0x7f01040699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f01040699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 >> v2:172.21.15.191:3300/0 conn(0x7f0104159890 msgr2=0x7f010415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 --2- 172.21.15.170:0/2980619121 >> v2:172.21.15.191:3300/0 conn(0x7f0104159890 0x7f010415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010415e140 con 0x7f0104159c60 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f00f4001470 con 0x7f0104159c60 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 >> v1:172.21.15.170:6789/0 conn(0x7f0104159c60 legacy=0x7f010415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec003a60 0x7f00ec003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f00ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010415a160 con 0x7f00ec003a60 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040d13a0 con 0x7f00ec004d50 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec00ba60 con 0x7f0104069560 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec003a60 0x7f00ec003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01090b7700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fbfff700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f00ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 -- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 msgr2=0x7f00ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0104069560 0x7f00ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 -- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec004d50 msgr2=0x7f00ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f00ec004d50 0x7f00ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 -- 172.21.15.170:0/2980619121 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec003570 con 0x7f00ec003a60 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f01088b6700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec003a60 0x7f00ec003e70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f010000e210 tx=0x7f010000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa2b05ac546e397 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000ea60 con 0x7f00ec003a60 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010000f040 con 0x7f00ec003a60 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.044+0000 7f00fb7fe700 1 -- 172.21.15.170:0/2980619121 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010000ee10 con 0x7f00ec003a60 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 -- 172.21.15.170:0/2980619121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec003a60 msgr2=0x7f00ec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 --2- 172.21.15.170:0/2980619121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00ec003a60 0x7f00ec003e70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f010000e210 tx=0x7f010000b040 comp rx=0 tx=0).stop 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f01090b7700 1 -- 172.21.15.170:0/2980619121 reap_dead start 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 -- 172.21.15.170:0/2980619121 shutdown_connections 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 -- 172.21.15.170:0/2980619121 >> 172.21.15.170:0/2980619121 conn(0x7f0104068e20 msgr2=0x7f01040d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 -- 172.21.15.170:0/2980619121 shutdown_connections 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.045+0000 7f010ab1a700 1 -- 172.21.15.170:0/2980619121 wait complete. 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.046+0000 7f010ab1a700 1 Processor -- start 2023-10-08T16:13:01.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.046+0000 7f010ab1a700 1 -- start start 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00ec003a60 0x7f01040b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0104151b50 0x7f0104151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01040b1ad0 con 0x7f01040b2640 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104152510 con 0x7f0104151b50 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f010ab1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01040c9cf0 con 0x7f00ec003a60 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.053 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47700/0 (socket says 172.21.15.170:47700) 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 -- 172.21.15.170:0/4288666080 learned_addr learned my addr 172.21.15.170:0/4288666080 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f01090b7700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0104151b50 0x7f0104151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f01088b6700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00ec003a60 0x7f01040b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 -- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00ec003a60 msgr2=0x7f01040b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00ec003a60 0x7f01040b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 -- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0104151b50 msgr2=0x7f0104151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0104151b50 0x7f0104151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.047+0000 7f00fbfff700 1 -- 172.21.15.170:0/4288666080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01040cda10 con 0x7f01040b2640 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f00fbfff700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f00f400ebc0 tx=0x7f00f4001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004a40 con 0x7f01040b2640 2023-10-08T16:13:01.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4004ba0 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0104150e30 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4005050 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.048+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01040cdcd0 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.049+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01040cde60 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.049+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f00f40053c0 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.049+0000 7f00f8ff9700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 0x7f00f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.049+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f00f4084b50 con 0x7f01040b2640 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.050+0000 7f01088b6700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 0x7f00f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.050+0000 7f01088b6700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 0x7f00f00697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f010000afd0 tx=0x7f010001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.054+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00f4051600 con 0x7f01040b2640 2023-10-08T16:13:01.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.235+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000975909.meta", "var": "pg_num"} v 0) v1 -- 0x7f010404c3c0 con 0x7f01040b2640 2023-10-08T16:13:01.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.235+0000 7f00f8ff9700 1 -- 172.21.15.170:0/4288666080 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000975909.meta", "var": "pg_num"}]=0 v171) v1 ==== 124+0+11 (secure 0 0 0) 0x7f00f4019070 con 0x7f01040b2640 2023-10-08T16:13:01.236 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 msgr2=0x7f00f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 0x7f00f00697d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f010000afd0 tx=0x7f010001a000 comp rx=0 tx=0).stop 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 msgr2=0x7f01041508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f00f400ebc0 tx=0x7f00f4001d40 comp rx=0 tx=0).stop 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 shutdown_connections 2023-10-08T16:13:01.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0104151b50 0x7f0104151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f00ec003a60 0x7f01040b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f00f0067320 0x7f00f00697d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 --2- 172.21.15.170:0/4288666080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01040b2640 0x7f01041508f0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 >> 172.21.15.170:0/4288666080 conn(0x7f0104068e20 msgr2=0x7f010415d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.237+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 shutdown_connections 2023-10-08T16:13:01.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.238+0000 7f010ab1a700 1 -- 172.21.15.170:0/4288666080 wait complete. 2023-10-08T16:13:01.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:13:01.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.368+0000 7fcca8d38700 1 Processor -- start 2023-10-08T16:13:01.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.368+0000 7fcca8d38700 1 -- start start 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca8d38700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcca40b16d0 0x7fcca40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca8d38700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcca40b33b0 0x7fcca40b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca8d38700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca40b2080 con 0x7fcca40b4e70 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca8d38700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40cfb00 con 0x7fcca40b33b0 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca8d38700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40b5c40 con 0x7fcca40b16d0 2023-10-08T16:13:01.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcca40b33b0 0x7fcca40b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcca40b33b0 0x7fcca40b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47714/0 (socket says 172.21.15.170:47714) 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 -- 172.21.15.170:0/209715028 learned_addr learned my addr 172.21.15.170:0/209715028 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcc9affd700 1 --2- 172.21.15.170:0/209715028 >> v2:172.21.15.191:3300/0 conn(0x7fcca40b16d0 0x7fcca40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 -- 172.21.15.170:0/209715028 >> v2:172.21.15.191:3300/0 conn(0x7fcca40b16d0 msgr2=0x7fcca40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 --2- 172.21.15.170:0/209715028 >> v2:172.21.15.191:3300/0 conn(0x7fcca40b16d0 0x7fcca40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 -- 172.21.15.170:0/209715028 >> v1:172.21.15.191:6789/0 conn(0x7fcca40b4e70 legacy=0x7fcca40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.369+0000 7fcca3fff700 1 -- 172.21.15.170:0/209715028 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40b5f20 con 0x7fcca40b33b0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca3fff700 1 --2- 172.21.15.170:0/209715028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca40b37d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcca4089830 tx=0x7fcc9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b57941cb843c4a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca2ffd700 1 -- 172.21.15.170:0/209715028 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00e070 con 0x7fcca40b33b0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca2ffd700 1 -- 172.21.15.170:0/209715028 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c00ba20 con 0x7fcca40b33b0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca2ffd700 1 -- 172.21.15.170:0/209715028 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0092e0 con 0x7fcca40b33b0 2023-10-08T16:13:01.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 -- 172.21.15.170:0/209715028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 msgr2=0x7fcca40b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 --2- 172.21.15.170:0/209715028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca40b37d0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcca4089830 tx=0x7fcc9c009840 comp rx=0 tx=0).stop 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 -- 172.21.15.170:0/209715028 shutdown_connections 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 --2- 172.21.15.170:0/209715028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca40b37d0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 --2- 172.21.15.170:0/209715028 >> v2:172.21.15.191:3300/0 conn(0x7fcca40b16d0 0x7fcca40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 -- 172.21.15.170:0/209715028 >> 172.21.15.170:0/209715028 conn(0x7fcca40d10c0 msgr2=0x7fcca40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 -- 172.21.15.170:0/209715028 shutdown_connections 2023-10-08T16:13:01.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.370+0000 7fcca8d38700 1 -- 172.21.15.170:0/209715028 wait complete. 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.371+0000 7fcca8d38700 1 Processor -- start 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 -- start start 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcca40b16d0 0x7fcca4145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcca40b4e70 0x7fcca415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4148d10 con 0x7fcca40b33b0 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4148100 con 0x7fcca40b16d0 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca8d38700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca41483e0 con 0x7fcca40b4e70 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcc9affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47728/0 (socket says 172.21.15.170:47728) 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcc9affd700 1 -- 172.21.15.170:0/1026950794 learned_addr learned my addr 172.21.15.170:0/1026950794 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca3fff700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcca40b4e70 0x7fcca415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.372+0000 7fcca37fe700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcca40b16d0 0x7fcca4145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 -- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcca40b4e70 msgr2=0x7fcca415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcca40b4e70 0x7fcca415e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 -- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcca40b16d0 msgr2=0x7fcca4145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcca40b16d0 0x7fcca4145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 -- 172.21.15.170:0/1026950794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c009ae0 con 0x7fcca40b33b0 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9affd700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e930 tx=0x7fcc8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c013680 con 0x7fcca40b33b0 2023-10-08T16:13:01.377 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c0137e0 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca41486c0 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c013b80 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.373+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca41461f0 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.374+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4088990 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.375+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcc8c00c3e0 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.375+0000 7fcc9bfff700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 0x7fcc840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.375+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fcc8c084f40 con 0x7fcca40b33b0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.375+0000 7fcca37fe700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 0x7fcc840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.378 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.375+0000 7fcca37fe700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 0x7fcc840697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc94003a80 tx=0x7fcc94007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.379 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.379+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc8c0519f0 con 0x7fcca40b33b0 2023-10-08T16:13:01.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.562+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcca4148e50 con 0x7fcca40b33b0 2023-10-08T16:13:01.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.563+0000 7fcc9bfff700 1 -- 172.21.15.170:0/1026950794 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcc8c055010 con 0x7fcca40b33b0 2023-10-08T16:13:01.563 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 msgr2=0x7fcc840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 0x7fcc840697d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcc94003a80 tx=0x7fcc94007ec0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 msgr2=0x7fcca415e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00e930 tx=0x7fcc8c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 shutdown_connections 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcca40b4e70 0x7fcca415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcca40b16d0 0x7fcca4145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcc84067320 0x7fcc840697d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 --2- 172.21.15.170:0/1026950794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcca40b33b0 0x7fcca415e370 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 >> 172.21.15.170:0/1026950794 conn(0x7fcca40d10c0 msgr2=0x7fcca415c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 shutdown_connections 2023-10-08T16:13:01.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.565+0000 7fcca8d38700 1 -- 172.21.15.170:0/1026950794 wait complete. 2023-10-08T16:13:01.577 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get another-data-pool pg_num 2023-10-08T16:13:01.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.694+0000 7fc23d982700 1 Processor -- start 2023-10-08T16:13:01.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.694+0000 7fc23d982700 1 -- start start 2023-10-08T16:13:01.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc23d982700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 0x7fc23815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc23d982700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc23d982700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc238069ee0 con 0x7fc238159c60 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc23d982700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23815a160 con 0x7fc238159890 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc23d982700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2380d13a0 con 0x7fc238069560 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc236ffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 0x7fc23815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:48768/0 (socket says 172.21.15.170:48768) 2023-10-08T16:13:01.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc2367fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fc238159c60 0x7fc23815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:51636/0 (socket says 172.21.15.170:51636) 2023-10-08T16:13:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.695+0000 7fc2377fe700 1 -- 172.21.15.170:0/1664345066 learned_addr learned my addr 172.21.15.170:0/1664345066 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc2377fe700 1 -- 172.21.15.170:0/1664345066 >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 msgr2=0x7fc23815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc2377fe700 1 --2- 172.21.15.170:0/1664345066 >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 0x7fc23815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc2377fe700 1 -- 172.21.15.170:0/1664345066 >> v1:172.21.15.191:6789/0 conn(0x7fc238159c60 legacy=0x7fc23815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc2377fe700 1 -- 172.21.15.170:0/1664345066 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23815e140 con 0x7fc238069560 2023-10-08T16:13:01.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc236ffd700 1 --2- 172.21.15.170:0/1664345066 >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 0x7fc23815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:01.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc2377fe700 1 --2- 172.21.15.170:0/1664345066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2300099b0 tx=0x7fc230009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49aa1251528a1092 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc235ffb700 1 -- 172.21.15.170:0/1664345066 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc230009ea0 con 0x7fc238069560 2023-10-08T16:13:01.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc235ffb700 1 -- 172.21.15.170:0/1664345066 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23000b860 con 0x7fc238069560 2023-10-08T16:13:01.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc235ffb700 1 -- 172.21.15.170:0/1664345066 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000bc00 con 0x7fc238069560 2023-10-08T16:13:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc23d982700 1 -- 172.21.15.170:0/1664345066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 msgr2=0x7fc2380699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.696+0000 7fc23d982700 1 --2- 172.21.15.170:0/1664345066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2300099b0 tx=0x7fc230009250 comp rx=0 tx=0).stop 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 -- 172.21.15.170:0/1664345066 shutdown_connections 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 --2- 172.21.15.170:0/1664345066 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238069560 0x7fc2380699a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 --2- 172.21.15.170:0/1664345066 >> v2:172.21.15.191:3300/0 conn(0x7fc238159890 0x7fc23815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 -- 172.21.15.170:0/1664345066 >> 172.21.15.170:0/1664345066 conn(0x7fc238068e20 msgr2=0x7fc2380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 -- 172.21.15.170:0/1664345066 shutdown_connections 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.697+0000 7fc23d982700 1 -- 172.21.15.170:0/1664345066 wait complete. 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.698+0000 7fc23d982700 1 Processor -- start 2023-10-08T16:13:01.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.698+0000 7fc23d982700 1 -- start start 2023-10-08T16:13:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.698+0000 7fc23d982700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc238069560 0x7fc2380b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238159c60 0x7fc2380c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23816aee0 con 0x7fc238159890 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23816b1c0 con 0x7fc238069560 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23816a5b0 con 0x7fc238159c60 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47732/0 (socket says 172.21.15.170:47732) 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 -- 172.21.15.170:0/3327325310 learned_addr learned my addr 172.21.15.170:0/3327325310 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2377fe700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238159c60 0x7fc2380c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc236ffd700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc238069560 0x7fc2380b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 -- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238159c60 msgr2=0x7fc2380c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238159c60 0x7fc2380c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 -- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc238069560 msgr2=0x7fc2380b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc238069560 0x7fc2380b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 -- 172.21.15.170:0/3327325310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228009a20 con 0x7fc238159890 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc2367fc700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c7c0 tx=0x7fc220007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220005a70 con 0x7fc238159890 2023-10-08T16:13:01.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc220005bd0 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2300094f0 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200070d0 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.699+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23816ab70 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.700+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23804c3c0 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.701+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc220007440 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.701+0000 7fc22effd700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 0x7fc2180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.701+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fc2200842e0 con 0x7fc238159890 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.701+0000 7fc236ffd700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 0x7fc2180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:01.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.701+0000 7fc236ffd700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 0x7fc2180697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc228009fd0 tx=0x7fc22800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:01.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.705+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc220050d90 con 0x7fc238159890 2023-10-08T16:13:01.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.888+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"} v 0) v1 -- 0x7fc23806a5b0 con 0x7fc238159890 2023-10-08T16:13:01.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.888+0000 7fc22effd700 1 -- 172.21.15.170:0/3327325310 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "another-data-pool", "var": "pg_num"}]=0 v171) v1 ==== 106+0+11 (secure 0 0 0) 0x7fc2200170b0 con 0x7fc238159890 2023-10-08T16:13:01.889 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:13:01.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 msgr2=0x7fc2180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 0x7fc2180697d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fc228009fd0 tx=0x7fc22800b040 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 msgr2=0x7fc238169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c7c0 tx=0x7fc220007f30 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 shutdown_connections 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc238159c60 0x7fc2380c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc238069560 0x7fc2380b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc218067320 0x7fc2180697d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 --2- 172.21.15.170:0/3327325310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc238159890 0x7fc238169cc0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 >> 172.21.15.170:0/3327325310 conn(0x7fc238068e20 msgr2=0x7fc238153420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 shutdown_connections 2023-10-08T16:13:01.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:01.890+0000 7fc23d982700 1 -- 172.21.15.170:0/3327325310 wait complete. 2023-10-08T16:13:01.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:02.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.017+0000 7f724c89f700 1 Processor -- start 2023-10-08T16:13:02.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.017+0000 7f724c89f700 1 -- start start 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f724c89f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7248159420 0x7f72480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f724c89f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f724815d230 0x7f724815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f724c89f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72480b2980 con 0x7f72481597f0 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f724c89f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724815dbb0 con 0x7f7248159420 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f724c89f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72480b11d0 con 0x7f724815d230 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72477fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7248159420 0x7f72480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f7246ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f72481597f0 0x7f72480b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35256/0 (socket says 172.21.15.170:35256) 2023-10-08T16:13:02.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f7246ffd700 1 -- 172.21.15.170:0/3266961883 learned_addr learned my addr 172.21.15.170:0/3266961883 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f7247fff700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f724815d230 0x7f724815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72477fe700 1 -- 172.21.15.170:0/3266961883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f724815d230 msgr2=0x7f724815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72477fe700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f724815d230 0x7f724815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72477fe700 1 -- 172.21.15.170:0/3266961883 >> v1:172.21.15.170:6789/0 conn(0x7f72481597f0 legacy=0x7f72480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72477fe700 1 -- 172.21.15.170:0/3266961883 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480b14b0 con 0x7f7248159420 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.018+0000 7f72467fc700 1 -- 172.21.15.170:0/3266961883 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861812835 0 0) 0x7f72480b2980 con 0x7f72481597f0 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f72477fe700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7248159420 0x7f72480b5f40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7238009e60 tx=0x7f7238009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4501146d7ffbdcb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f72467fc700 1 -- 172.21.15.170:0/3266961883 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800e070 con 0x7f7248159420 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f72467fc700 1 -- 172.21.15.170:0/3266961883 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800bc30 con 0x7f7248159420 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f72467fc700 1 -- 172.21.15.170:0/3266961883 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238012430 con 0x7f7248159420 2023-10-08T16:13:02.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 -- 172.21.15.170:0/3266961883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7248159420 msgr2=0x7f72480b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7248159420 0x7f72480b5f40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7238009e60 tx=0x7f7238009780 comp rx=0 tx=0).stop 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 -- 172.21.15.170:0/3266961883 shutdown_connections 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f724815d230 0x7f724815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 --2- 172.21.15.170:0/3266961883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7248159420 0x7f72480b5f40 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 -- 172.21.15.170:0/3266961883 >> 172.21.15.170:0/3266961883 conn(0x7f72480d0660 msgr2=0x7f72480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 -- 172.21.15.170:0/3266961883 shutdown_connections 2023-10-08T16:13:02.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.019+0000 7f724c89f700 1 -- 172.21.15.170:0/3266961883 wait complete. 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.020+0000 7f724c89f700 1 Processor -- start 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.020+0000 7f724c89f700 1 -- start start 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7248159420 0x7f724816a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72481597f0 0x7f72480b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724815c550 con 0x7f724815d230 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724816a7e0 con 0x7f7248159420 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f724c89f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724816d580 con 0x7f72481597f0 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f7247fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f7247fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47758/0 (socket says 172.21.15.170:47758) 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f7247fff700 1 -- 172.21.15.170:0/1866222592 learned_addr learned my addr 172.21.15.170:0/1866222592 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f7246ffd700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72481597f0 0x7f72480b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.021+0000 7f72477fe700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7248159420 0x7f724816a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 -- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72481597f0 msgr2=0x7f72480b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72481597f0 0x7f72480b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 -- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7248159420 msgr2=0x7f724816a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7248159420 0x7f724816a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 -- 172.21.15.170:0/1866222592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7238009a20 con 0x7f724815d230 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f7247fff700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f724000d860 tx=0x7f7240008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400125f0 con 0x7f724815d230 2023-10-08T16:13:02.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240012790 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724814d2c0 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7240012b50 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.022+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724815c320 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.023+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724804c3c0 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.023+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7240013480 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.024+0000 7f722f7fe700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 0x7f7234069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.024+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f72400850b0 con 0x7f724815d230 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.024+0000 7f72477fe700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 0x7f7234069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.024+0000 7f72477fe700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 0x7f7234069800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7238009e60 tx=0x7f7238019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.028+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7240051b60 con 0x7f724815d230 2023-10-08T16:13:02.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.253+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72480b1820 con 0x7f724815d230 2023-10-08T16:13:02.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.253+0000 7f722f7fe700 1 -- 172.21.15.170:0/1866222592 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4630 (secure 0 0 0) 0x7f7240055180 con 0x7f724815d230 2023-10-08T16:13:02.254 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:02.254 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:57.735496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000975909","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 msgr2=0x7f7234069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 0x7f7234069800 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7238009e60 tx=0x7f7238019000 comp rx=0 tx=0).stop 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 msgr2=0x7f724814c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f724000d860 tx=0x7f7240008040 comp rx=0 tx=0).stop 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 shutdown_connections 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7234067350 0x7f7234069800 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f724815d230 0x7f724814c3e0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7248159420 0x7f724816a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 --2- 172.21.15.170:0/1866222592 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72481597f0 0x7f72480b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 >> 172.21.15.170:0/1866222592 conn(0x7f72480d0660 msgr2=0x7f72480699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 shutdown_connections 2023-10-08T16:13:02.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.255+0000 7f724c89f700 1 -- 172.21.15.170:0/1866222592 wait complete. 2023-10-08T16:13:02.257 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 112 2023-10-08T16:13:02.268 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:02.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.382+0000 7f531e369700 1 Processor -- start 2023-10-08T16:13:02.383 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.382+0000 7f531e369700 1 -- start start 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531e369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 0x7f531815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531e369700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f531815db10 0x7f5318069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531e369700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f531815a160 con 0x7f5318159890 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531e369700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53180d13a0 con 0x7f531815db10 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531e369700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531815e140 con 0x7f5318159c60 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f531c906700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f531815db10 0x7f5318069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f5317fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5318159890 0x7f531815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35266/0 (socket says 172.21.15.170:35266) 2023-10-08T16:13:02.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f5317fff700 1 -- 172.21.15.170:0/3733756715 learned_addr learned my addr 172.21.15.170:0/3733756715 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.383+0000 7f53177fe700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 0x7f531815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531c906700 1 -- 172.21.15.170:0/3733756715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 msgr2=0x7f531815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531c906700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 0x7f531815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531c906700 1 -- 172.21.15.170:0/3733756715 >> v1:172.21.15.170:6789/0 conn(0x7f5318159890 legacy=0x7f531815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531c906700 1 -- 172.21.15.170:0/3733756715 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531815a3d0 con 0x7f531815db10 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f53177fe700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 0x7f531815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f5316ffd700 1 -- 172.21.15.170:0/3733756715 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212676405 0 0) 0x7f531815a160 con 0x7f5318159890 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531c906700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f531815db10 0x7f5318069560 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5310009e60 tx=0x7f5310009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec58dd274adea6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f5316ffd700 1 -- 172.21.15.170:0/3733756715 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000e070 con 0x7f531815db10 2023-10-08T16:13:02.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f5316ffd700 1 -- 172.21.15.170:0/3733756715 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531000bc30 con 0x7f531815db10 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f5316ffd700 1 -- 172.21.15.170:0/3733756715 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310012430 con 0x7f531815db10 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531e369700 1 -- 172.21.15.170:0/3733756715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f531815db10 msgr2=0x7f5318069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531e369700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f531815db10 0x7f5318069560 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f5310009e60 tx=0x7f5310009780 comp rx=0 tx=0).stop 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.384+0000 7f531e369700 1 -- 172.21.15.170:0/3733756715 shutdown_connections 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.385+0000 7f531e369700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f531815db10 0x7f5318069560 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.385+0000 7f531e369700 1 --2- 172.21.15.170:0/3733756715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159c60 0x7f531815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.385+0000 7f531e369700 1 -- 172.21.15.170:0/3733756715 >> 172.21.15.170:0/3733756715 conn(0x7f5318068e20 msgr2=0x7f53180d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.385+0000 7f531e369700 1 -- 172.21.15.170:0/3733756715 shutdown_connections 2023-10-08T16:13:02.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.385+0000 7f531e369700 1 -- 172.21.15.170:0/3733756715 wait complete. 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 Processor -- start 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 -- start start 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 0x7f5318171710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 0x7f5318151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53181726b0 con 0x7f5318159c60 2023-10-08T16:13:02.390 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53180b5010 con 0x7f531815db10 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.386+0000 7f531e369700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53180b52b0 con 0x7f5318159890 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47790/0 (socket says 172.21.15.170:47790) 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 -- 172.21.15.170:0/2890956658 learned_addr learned my addr 172.21.15.170:0/2890956658 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f5317fff700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 0x7f5318171710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f531c906700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 0x7f5318151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 -- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 msgr2=0x7f5318171710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 0x7f5318171710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 -- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 msgr2=0x7f5318151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 0x7f5318151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 -- 172.21.15.170:0/2890956658 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310009a20 con 0x7f5318159c60 2023-10-08T16:13:02.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f5317fff700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 0x7f5318171710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f531c906700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 0x7f5318151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f53177fe700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f530c00e8a0 tx=0x7f530c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00c430 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00c590 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53181720f0 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00c950 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.387+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5318172480 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.388+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53180b58c0 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.389+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f530c00cd40 con 0x7f5318159c60 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.389+0000 7f52fffff700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 0x7f53040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.389+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f530c084cb0 con 0x7f5318159c60 2023-10-08T16:13:02.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.389+0000 7f5317fff700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 0x7f53040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.389+0000 7f5317fff700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 0x7f53040697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f53080011e0 tx=0x7f5308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.393+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f530c051650 con 0x7f5318159c60 2023-10-08T16:13:02.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.579+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f531804c3c0 con 0x7f5318159c60 2023-10-08T16:13:02.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.579+0000 7f52fffff700 1 -- 172.21.15.170:0/2890956658 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f530c054c70 con 0x7f5318159c60 2023-10-08T16:13:02.580 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:02.580 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":171,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:58.736978+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000975909.meta","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000975909"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:02.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 msgr2=0x7f53040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 0x7f53040697d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f53080011e0 tx=0x7f5308009040 comp rx=0 tx=0).stop 2023-10-08T16:13:02.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 msgr2=0x7f5318153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f530c00e8a0 tx=0x7f530c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 shutdown_connections 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f531815db10 0x7f5318151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5318159890 0x7f5318171710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5304067320 0x7f53040697d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 --2- 172.21.15.170:0/2890956658 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5318159c60 0x7f5318153730 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 >> 172.21.15.170:0/2890956658 conn(0x7f5318068e20 msgr2=0x7f53180c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 shutdown_connections 2023-10-08T16:13:02.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.582+0000 7f531e369700 1 -- 172.21.15.170:0/2890956658 wait complete. 2023-10-08T16:13:02.595 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000975909 and related pools 2023-10-08T16:13:02.595 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:02.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.709+0000 7fbb37fff700 1 Processor -- start 2023-10-08T16:13:02.710 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.709+0000 7fbb37fff700 1 -- start start 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb37fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380b11d0 con 0x7fbb38159420 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb37fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb380b14b0 con 0x7fbb3815d230 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb37fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380b1790 con 0x7fbb381597f0 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb36ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbb38159420 0x7fbb380b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35274/0 (socket says 172.21.15.170:35274) 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb36ffd700 1 -- 172.21.15.170:0/2619283329 learned_addr learned my addr 172.21.15.170:0/2619283329 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:02.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396333875 0 0) 0x7fbb380b11d0 con 0x7fbb38159420 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb20003570 con 0x7fbb38159420 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 282168671 0 0) 0x7fbb20003570 con 0x7fbb38159420 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 msgr2=0x7fbb380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 >> v1:172.21.15.191:6789/0 conn(0x7fbb3815d230 legacy=0x7fbb380b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380b1a70 con 0x7fbb38159420 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fbb280017f0 con 0x7fbb38159420 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.710+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 >> v1:172.21.15.170:6789/0 conn(0x7fbb38159420 legacy=0x7fbb380b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20003a90 0x7fbb20003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb20004810 0x7fbb20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380b1790 con 0x7fbb20004810 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2000bb10 con 0x7fbb20003a90 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2000bc80 con 0x7fbb381597f0 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb20004810 0x7fbb20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20003a90 0x7fbb20003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb377fe700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 -- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 msgr2=0x7fbb200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb381597f0 0x7fbb200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 -- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20003a90 msgr2=0x7fbb20003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:02.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20003a90 0x7fbb20003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 -- 172.21.15.170:0/2619283329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009420 con 0x7fbb20004810 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20003a90 0x7fbb20003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb20004810 0x7fbb20004c80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009df0 tx=0x7fbb2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481b62955b5e0ea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c009740 con 0x7fbb20004810 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00e8c0 con 0x7fbb20004810 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.711+0000 7fbb35ffb700 1 -- 172.21.15.170:0/2619283329 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00ec80 con 0x7fbb20004810 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 -- 172.21.15.170:0/2619283329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb20004810 msgr2=0x7fbb20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2619283329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb20004810 0x7fbb20004c80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009df0 tx=0x7fbb2c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb377fe700 1 -- 172.21.15.170:0/2619283329 reap_dead start 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 -- 172.21.15.170:0/2619283329 shutdown_connections 2023-10-08T16:13:02.719 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 -- 172.21.15.170:0/2619283329 >> 172.21.15.170:0/2619283329 conn(0x7fbb380d0660 msgr2=0x7fbb380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 -- 172.21.15.170:0/2619283329 shutdown_connections 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.712+0000 7fbb37fff700 1 -- 172.21.15.170:0/2619283329 wait complete. 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.713+0000 7fbb37fff700 1 Processor -- start 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.713+0000 7fbb37fff700 1 -- start start 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 0x7fbb380c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 0x7fbb38156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3815c580 con 0x7fbb3815bbd0 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c67f0 con 0x7fbb20004810 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c5940 con 0x7fbb38155d90 2023-10-08T16:13:02.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47804/0 (socket says 172.21.15.170:47804) 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 -- 172.21.15.170:0/2995835011 learned_addr learned my addr 172.21.15.170:0/2995835011 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 0x7fbb380c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb377fe700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 0x7fbb38156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 -- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 msgr2=0x7fbb38156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 0x7fbb38156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 -- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 msgr2=0x7fbb380c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 0x7fbb380c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 -- 172.21.15.170:0/2995835011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380c5be0 con 0x7fbb3815bbd0 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 0x7fbb380c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb377fe700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 0x7fbb38156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb367fc700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb20004730 tx=0x7fbb2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c022070 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c000d10 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00ec60 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.714+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb380c5ec0 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.715+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb380c6480 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.715+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3815c580 con 0x7fbb3815bbd0 2023-10-08T16:13:02.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.716+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbb2c026070 con 0x7fbb3815bbd0 2023-10-08T16:13:02.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.716+0000 7fbb1effd700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 0x7fbb240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:02.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.716+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fbb2c0867c0 con 0x7fbb3815bbd0 2023-10-08T16:13:02.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.716+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 0x7fbb240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:02.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.717+0000 7fbb36ffd700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 0x7fbb240697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbb28002080 tx=0x7fbb28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:02.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.720+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb2c053270 con 0x7fbb3815bbd0 2023-10-08T16:13:02.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.949+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb3804c3c0 con 0x7fbb3815bbd0 2023-10-08T16:13:02.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.949+0000 7fbb1effd700 1 -- 172.21.15.170:0/2995835011 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4630 (secure 0 0 0) 0x7fbb2c056890 con 0x7fbb3815bbd0 2023-10-08T16:13:02.950 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:02.950 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:57.735496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000975909","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:13:02.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 msgr2=0x7fbb240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.952 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 0x7fbb240697d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbb28002080 tx=0x7fbb28009040 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 msgr2=0x7fbb3815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb20004730 tx=0x7fbb2c01a000 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 shutdown_connections 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbb38155d90 0x7fbb38156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbb20004810 0x7fbb380c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbb24067320 0x7fbb240697d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 --2- 172.21.15.170:0/2995835011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb3815bbd0 0x7fbb3815c040 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 >> 172.21.15.170:0/2995835011 conn(0x7fbb380d0660 msgr2=0x7fbb38069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 shutdown_connections 2023-10-08T16:13:02.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:02.952+0000 7fbb37fff700 1 -- 172.21.15.170:0/2995835011 wait complete. 2023-10-08T16:13:02.954 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 112 2023-10-08T16:13:02.964 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:03.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.079+0000 7fcef24d5700 1 Processor -- start 2023-10-08T16:13:03.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.079+0000 7fcef24d5700 1 -- start start 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef24d5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fceec15d230 0x7fceec15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef24d5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec0b2980 con 0x7fceec159420 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef24d5700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec15db70 con 0x7fceec1597f0 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef24d5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceec0b11d0 con 0x7fceec15d230 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef14d3700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fceec159420 0x7fceec0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35282/0 (socket says 172.21.15.170:35282) 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef14d3700 1 -- 172.21.15.170:0/2123317158 learned_addr learned my addr 172.21.15.170:0/2123317158 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:03.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcef1cd4700 1 --2- 172.21.15.170:0/2123317158 >> v2:172.21.15.191:3300/0 conn(0x7fceec15d230 0x7fceec15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533814257 0 0) 0x7fceec0b2980 con 0x7fceec159420 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.080+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced8003570 con 0x7fceec159420 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4154611835 0 0) 0x7fced8003570 con 0x7fceec159420 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 >> v2:172.21.15.191:3300/0 conn(0x7fceec15d230 msgr2=0x7fceec15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 --2- 172.21.15.170:0/2123317158 >> v2:172.21.15.191:3300/0 conn(0x7fceec15d230 0x7fceec15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 >> v1:172.21.15.191:6789/0 conn(0x7fceec1597f0 legacy=0x7fceec0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0b14b0 con 0x7fceec159420 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fcee8001470 con 0x7fceec159420 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 >> v1:172.21.15.170:6789/0 conn(0x7fceec159420 legacy=0x7fceec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.084 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8003a60 0x7fced8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 0x7fced8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80051c0 0x7fced800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0b11d0 con 0x7fced8004810 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced800c0d0 con 0x7fced80051c0 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced800c290 con 0x7fced8003a60 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 0x7fced8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef1cd4700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80051c0 0x7fced800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef14d3700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8003a60 0x7fced8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 -- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8003a60 msgr2=0x7fced8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8003a60 0x7fced8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.085 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 -- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80051c0 msgr2=0x7fced800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fced80051c0 0x7fced800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 -- 172.21.15.170:0/2123317158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8003570 con 0x7fced8004810 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef1cd4700 1 -- 172.21.15.170:0/2123317158 reap_dead start 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef14d3700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8003a60 0x7fced8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcef0cd2700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 0x7fced8004c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcedc001880 tx=0x7fcedc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68b322d7eec9216 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc012420 con 0x7fced8004810 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcedc0125a0 con 0x7fced8004810 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.081+0000 7fcee3fff700 1 -- 172.21.15.170:0/2123317158 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcedc012960 con 0x7fced8004810 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 -- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 msgr2=0x7fced8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.086 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 0x7fced8004c80 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fcedc001880 tx=0x7fcedc007d30 comp rx=0 tx=0).stop 2023-10-08T16:13:03.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 -- 172.21.15.170:0/2123317158 shutdown_connections 2023-10-08T16:13:03.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 --2- 172.21.15.170:0/2123317158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced8004810 0x7fced8004c80 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 -- 172.21.15.170:0/2123317158 >> 172.21.15.170:0/2123317158 conn(0x7fceec0d0660 msgr2=0x7fceec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:03.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 -- 172.21.15.170:0/2123317158 shutdown_connections 2023-10-08T16:13:03.087 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.082+0000 7fcef24d5700 1 -- 172.21.15.170:0/2123317158 wait complete. 2023-10-08T16:13:03.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.083+0000 7fcef24d5700 1 Processor -- start 2023-10-08T16:13:03.089 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.083+0000 7fcef24d5700 1 -- start start 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8004810 0x7fceec149340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fceec0c5d80 0x7fceec16d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0c7870 con 0x7fceec16d590 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec149880 con 0x7fceec0c5d80 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef24d5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec16da30 con 0x7fced8004810 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47832/0 (socket says 172.21.15.170:47832) 2023-10-08T16:13:03.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 -- 172.21.15.170:0/1196317653 learned_addr learned my addr 172.21.15.170:0/1196317653 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef14d3700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8004810 0x7fceec149340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef0cd2700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fceec0c5d80 0x7fceec16d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 -- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8004810 msgr2=0x7fceec149340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8004810 0x7fceec149340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 -- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fceec0c5d80 msgr2=0x7fceec16d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fceec0c5d80 0x7fceec16d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.084+0000 7fcef1cd4700 1 -- 172.21.15.170:0/1196317653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec16dd10 con 0x7fceec16d590 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcef1cd4700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fceec15e340 tx=0x7fcee400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400e040 con 0x7fceec16d590 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee400f040 con 0x7fceec16d590 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec0c5580 con 0x7fceec16d590 2023-10-08T16:13:03.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee400fe40 con 0x7fceec16d590 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.085+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec1545f0 con 0x7fceec16d590 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.086+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec0c5aa0 con 0x7fceec16d590 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.086+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcee4015640 con 0x7fceec16d590 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.086+0000 7fcee17fa700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 0x7fced00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.086+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fcee4085a40 con 0x7fceec16d590 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.087+0000 7fcef14d3700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 0x7fced00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.087+0000 7fcef14d3700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 0x7fced00697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcee8001200 tx=0x7fcee8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.092 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.091+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fceec0c5aa0 con 0x7fceec16d590 2023-10-08T16:13:03.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.316+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fceec0c5aa0 con 0x7fceec16d590 2023-10-08T16:13:03.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.317+0000 7fcee17fa700 1 -- 172.21.15.170:0/1196317653 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4630 (secure 0 0 0) 0x7fceec0c5aa0 con 0x7fceec16d590 2023-10-08T16:13:03.317 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:03.318 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:12:43.032412+0000","modified":"2023-10-08T16:12:57.735496+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8205},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8205":{"gid":8205,"name":"c","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/4170488760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":4170488760},{"type":"v1","addr":"172.21.15.170:6836","nonce":4170488760}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29,30],"metadata_pool":28,"enabled":true,"fs_name":"volume_0000000000975909","balancer":"","standby_count_wanted":0},"id":14}]} 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 msgr2=0x7fced00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 0x7fced00697d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fcee8001200 tx=0x7fcee8009040 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 msgr2=0x7fceec0b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fceec15e340 tx=0x7fcee400b040 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 shutdown_connections 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fced0067320 0x7fced00697d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fceec16d590 0x7fceec0b20c0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fced8004810 0x7fceec149340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 --2- 172.21.15.170:0/1196317653 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fceec0c5d80 0x7fceec16d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 >> 172.21.15.170:0/1196317653 conn(0x7fceec0d0660 msgr2=0x7fceec0af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:03.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 shutdown_connections 2023-10-08T16:13:03.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.319+0000 7fcef24d5700 1 -- 172.21.15.170:0/1196317653 wait complete. 2023-10-08T16:13:03.321 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 112 2023-10-08T16:13:03.332 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:03.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.445+0000 7f2660dc1700 1 Processor -- start 2023-10-08T16:13:03.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.445+0000 7f2660dc1700 1 -- start start 2023-10-08T16:13:03.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2660dc1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f265c159890 0x7f265c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2660dc1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f265c069560 0x7f265c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2660dc1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265c069ee0 con 0x7f265c159c60 2023-10-08T16:13:03.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2660dc1700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c15a160 con 0x7f265c159890 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2660dc1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265c0d13a0 con 0x7f265c069560 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f265a59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f265c159890 0x7f265c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2651d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f265c159c60 0x7f265c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35292/0 (socket says 172.21.15.170:35292) 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f2651d9b700 1 -- 172.21.15.170:0/367588200 learned_addr learned my addr 172.21.15.170:0/367588200 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.446+0000 7f265ad9d700 1 --2- 172.21.15.170:0/367588200 >> v2:172.21.15.191:3300/0 conn(0x7f265c069560 0x7f265c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f265a59c700 1 -- 172.21.15.170:0/367588200 >> v2:172.21.15.191:3300/0 conn(0x7f265c069560 msgr2=0x7f265c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f265a59c700 1 --2- 172.21.15.170:0/367588200 >> v2:172.21.15.191:3300/0 conn(0x7f265c069560 0x7f265c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f265a59c700 1 -- 172.21.15.170:0/367588200 >> v1:172.21.15.170:6789/0 conn(0x7f265c159c60 legacy=0x7f265c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f265a59c700 1 -- 172.21.15.170:0/367588200 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c15e140 con 0x7f265c159890 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2659d9b700 1 -- 172.21.15.170:0/367588200 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798807830 0 0) 0x7f265c069ee0 con 0x7f265c159c60 2023-10-08T16:13:03.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f265a59c700 1 --2- 172.21.15.170:0/367588200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159890 0x7f265c15abd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a890 tx=0x7f264c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2ebae2e324f1b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2659d9b700 1 -- 172.21.15.170:0/367588200 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c00e070 con 0x7f265c159890 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2659d9b700 1 -- 172.21.15.170:0/367588200 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c00bc30 con 0x7f265c159890 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2659d9b700 1 -- 172.21.15.170:0/367588200 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c012430 con 0x7f265c159890 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 -- 172.21.15.170:0/367588200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159890 msgr2=0x7f265c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 --2- 172.21.15.170:0/367588200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159890 0x7f265c15abd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a890 tx=0x7f264c009780 comp rx=0 tx=0).stop 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 -- 172.21.15.170:0/367588200 shutdown_connections 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 --2- 172.21.15.170:0/367588200 >> v2:172.21.15.191:3300/0 conn(0x7f265c069560 0x7f265c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 --2- 172.21.15.170:0/367588200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159890 0x7f265c15abd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 -- 172.21.15.170:0/367588200 >> 172.21.15.170:0/367588200 conn(0x7f265c068e20 msgr2=0x7f265c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 -- 172.21.15.170:0/367588200 shutdown_connections 2023-10-08T16:13:03.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.447+0000 7f2660dc1700 1 -- 172.21.15.170:0/367588200 wait complete. 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 Processor -- start 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 -- start start 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f265c069560 0x7f265c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f265c159890 0x7f265c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b3fd0 con 0x7f265c159c60 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b36a0 con 0x7f265c159890 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.449+0000 7f2660dc1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265c0b3980 con 0x7f265c069560 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47856/0 (socket says 172.21.15.170:47856) 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 -- 172.21.15.170:0/3688556145 learned_addr learned my addr 172.21.15.170:0/3688556145 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2651d9b700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f265c159890 0x7f265c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265a59c700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f265c069560 0x7f265c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 -- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f265c069560 msgr2=0x7f265c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f265c069560 0x7f265c156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 -- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f265c159890 msgr2=0x7f265c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f265c159890 0x7f265c0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 -- 172.21.15.170:0/3688556145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c009a20 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f265ad9d700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f265400d890 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26540125f0 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654012770 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265c0b3c60 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654012b30 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.450+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265c156970 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.451+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265c0b4140 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.451+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f265401e460 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.452+0000 7f2652ffd700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 0x7f263c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.452+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7f2654085540 con 0x7f265c159c60 2023-10-08T16:13:03.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.452+0000 7f265a59c700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 0x7f263c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.452+0000 7f265a59c700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 0x7f263c0697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a890 tx=0x7f264c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.456+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2654051ff0 con 0x7f265c159c60 2023-10-08T16:13:03.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.642+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f265c04c3c0 con 0x7f265c159c60 2023-10-08T16:13:03.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.643+0000 7f2652ffd700 1 -- 172.21.15.170:0/3688556145 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+17053 (secure 0 0 0) 0x7f2654055610 con 0x7f265c159c60 2023-10-08T16:13:03.643 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:03.643 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":171,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:12:58.736978+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":28,"pool_name":"cephfs.volume_0000000000975909.meta","create_time":"2023-10-08T16:12:41.449554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000975909"}}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-10-08T16:12:41.996094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}},{"pool":30,"pool_name":"another-data-pool","create_time":"2023-10-08T16:12:53.983782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"volume_0000000000975909"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:03.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 msgr2=0x7f263c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 0x7f263c0697d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f265c06a890 tx=0x7f264c019000 comp rx=0 tx=0).stop 2023-10-08T16:13:03.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 msgr2=0x7f265c153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f265400d890 tx=0x7f2654008040 comp rx=0 tx=0).stop 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 shutdown_connections 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f263c067320 0x7f263c0697d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f265c159c60 0x7f265c153ff0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f265c069560 0x7f265c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 --2- 172.21.15.170:0/3688556145 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f265c159890 0x7f265c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 >> 172.21.15.170:0/3688556145 conn(0x7f265c068e20 msgr2=0x7f265c155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 shutdown_connections 2023-10-08T16:13:03.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.645+0000 7f2660dc1700 1 -- 172.21.15.170:0/3688556145 wait complete. 2023-10-08T16:13:03.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000975909 2023-10-08T16:13:03.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 Processor -- start 2023-10-08T16:13:03.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 -- start start 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe6800caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe68015df50 0x7fe68015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68015d7e0 con 0x7fe68006c0a0 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68015dac0 con 0x7fe68015df50 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.775+0000 7fe686cf0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68015aa30 con 0x7fe68006bcd0 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe68015df50 0x7fe68015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe68015df50 0x7fe68015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47858/0 (socket says 172.21.15.170:47858) 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 -- 172.21.15.170:0/3691998681 learned_addr learned my addr 172.21.15.170:0/3691998681 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe684a8c700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe6800caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.778 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 -- 172.21.15.170:0/3691998681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 msgr2=0x7fe6800caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe6800caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 -- 172.21.15.170:0/3691998681 >> v1:172.21.15.191:6789/0 conn(0x7fe68006c0a0 legacy=0x7fe68015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 -- 172.21.15.170:0/3691998681 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68015ad10 con 0x7fe68015df50 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe68528d700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe68015e3b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe6780099b0 tx=0x7fe678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3782094c16595034 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe6777fe700 1 -- 172.21.15.170:0/3691998681 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800b860 con 0x7fe68015df50 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe6777fe700 1 -- 172.21.15.170:0/3691998681 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800b9c0 con 0x7fe68015df50 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.776+0000 7fe6777fe700 1 -- 172.21.15.170:0/3691998681 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800bd80 con 0x7fe68015df50 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 -- 172.21.15.170:0/3691998681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 msgr2=0x7fe68015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe68015e3b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fe6780099b0 tx=0x7fe678009250 comp rx=0 tx=0).stop 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 -- 172.21.15.170:0/3691998681 shutdown_connections 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe68015e3b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3691998681 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe6800caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 -- 172.21.15.170:0/3691998681 >> 172.21.15.170:0/3691998681 conn(0x7fe6800ce600 msgr2=0x7fe6800cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 -- 172.21.15.170:0/3691998681 shutdown_connections 2023-10-08T16:13:03.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.777+0000 7fe686cf0700 1 -- 172.21.15.170:0/3691998681 wait complete. 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.778+0000 7fe686cf0700 1 Processor -- start 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.778+0000 7fe686cf0700 1 -- start start 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe68016a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68006c0a0 0x7fe6800b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800b2af0 con 0x7fe68015df50 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68016a780 con 0x7fe68006c0a0 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68016aa60 con 0x7fe68006bcd0 2023-10-08T16:13:03.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47868/0 (socket says 172.21.15.170:47868) 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 -- 172.21.15.170:0/3364168149 learned_addr learned my addr 172.21.15.170:0/3364168149 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 -- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 msgr2=0x7fe68016a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe684a8c700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe68016a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe677fff700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68006c0a0 0x7fe6800b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe68016a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 -- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68006c0a0 msgr2=0x7fe6800b25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68006c0a0 0x7fe6800b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 -- 172.21.15.170:0/3364168149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780094f0 con 0x7fe68015df50 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe68528d700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67801f070 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67801a420 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68016ad40 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.779+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800b860 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.780+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68006dd40 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.780+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68016b640 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.781+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe67801aa10 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.781+0000 7fe674ff9700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 0x7fe66c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.781+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6180+0+0 (secure 0 0 0) 0x7fe678086560 con 0x7fe68015df50 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.781+0000 7fe684a8c700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 0x7fe66c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:03.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.782+0000 7fe684a8c700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 0x7fe66c0697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:03.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:03.786+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe678053090 con 0x7fe68015df50 2023-10-08T16:13:04.010 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.009+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000975909"} v 0) v1 -- 0x7fe68004c3c0 con 0x7fe68015df50 2023-10-08T16:13:04.010 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:13:04.010+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:13:04.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.014+0000 7fe674ff9700 1 -- 172.21.15.170:0/3364168149 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000975909"}]=0 volume_0000000000975909 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) v1 ==== 195+0+0 (secure 0 0 0) 0x7fe6780566b0 con 0x7fe68015df50 2023-10-08T16:13:04.015 INFO:tasks.ceph.mds.c.smithi170.stderr: -786> 2023-10-08T16:12:49.513+0000 7f7e03f67700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 msgr2=0x7fe66c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 0x7fe66c0697d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009fd0 tx=0x7fe67c00b040 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 msgr2=0x7fe6800c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678017000 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 shutdown_connections 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe66c067320 0x7fe66c0697d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe68015df50 0x7fe6800c3f20 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe68006bcd0 0x7fe68016a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 --2- 172.21.15.170:0/3364168149 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe68006c0a0 0x7fe6800b25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 >> 172.21.15.170:0/3364168149 conn(0x7fe6800ce600 msgr2=0x7fe680154eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:04.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 shutdown_connections 2023-10-08T16:13:04.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.016+0000 7fe686cf0700 1 -- 172.21.15.170:0/3364168149 wait complete. 2023-10-08T16:13:04.018 INFO:teuthology.orchestra.run.smithi170.stderr:volume_0000000000975909 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:13:04.033 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000975909 --yes-i-really-mean-it 2023-10-08T16:13:04.040 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:13:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.149+0000 7fc46bfca700 1 Processor -- start 2023-10-08T16:13:04.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.150+0000 7fc46bfca700 1 -- start start 2023-10-08T16:13:04.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.150+0000 7fc46bfca700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc4640b5ea0 0x7fc464068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:04.152 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.150+0000 7fc46bfca700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46415c480 con 0x7fc464159c10 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.150+0000 7fc46bfca700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46415b870 con 0x7fc464068bb0 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.150+0000 7fc46bfca700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4640b2a40 con 0x7fc4640b5ea0 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469d66700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc464159c10 0x7fc46415b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:35298/0 (socket says 172.21.15.170:35298) 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469d66700 1 -- 172.21.15.170:0/1842650979 learned_addr learned my addr 172.21.15.170:0/1842650979 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469565700 1 --2- 172.21.15.170:0/1842650979 >> v2:172.21.15.191:3300/0 conn(0x7fc4640b5ea0 0x7fc464068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383939647 0 0) 0x7fc46415c480 con 0x7fc464159c10 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc44c003570 con 0x7fc464159c10 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268000912 0 0) 0x7fc46415b870 con 0x7fc464068bb0 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc46415c480 con 0x7fc464068bb0 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469565700 1 -- 172.21.15.170:0/1842650979 >> v1:172.21.15.191:6789/0 conn(0x7fc464068bb0 legacy=0x7fc46415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469565700 1 -- 172.21.15.170:0/1842650979 >> v1:172.21.15.170:6789/0 conn(0x7fc464159c10 legacy=0x7fc46415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469565700 1 -- 172.21.15.170:0/1842650979 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46415bb50 con 0x7fc4640b5ea0 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc469565700 1 --2- 172.21.15.170:0/1842650979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4640b5ea0 0x7fc464068670 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4580098e0 tx=0x7fc4580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d8fafaa1276a22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458011630 con 0x7fc4640b5ea0 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458011790 con 0x7fc4640b5ea0 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.151+0000 7fc468d64700 1 -- 172.21.15.170:0/1842650979 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458011b10 con 0x7fc4640b5ea0 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 -- 172.21.15.170:0/1842650979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4640b5ea0 msgr2=0x7fc464068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 --2- 172.21.15.170:0/1842650979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4640b5ea0 0x7fc464068670 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc4580098e0 tx=0x7fc4580091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 -- 172.21.15.170:0/1842650979 shutdown_connections 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 --2- 172.21.15.170:0/1842650979 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc4640b5ea0 0x7fc464068670 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 -- 172.21.15.170:0/1842650979 >> 172.21.15.170:0/1842650979 conn(0x7fc464099a20 msgr2=0x7fc4640af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:04.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 -- 172.21.15.170:0/1842650979 shutdown_connections 2023-10-08T16:13:04.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.152+0000 7fc46bfca700 1 -- 172.21.15.170:0/1842650979 wait complete. 2023-10-08T16:13:04.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.153+0000 7fc46bfca700 1 Processor -- start 2023-10-08T16:13:04.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.153+0000 7fc46bfca700 1 -- start start 2023-10-08T16:13:04.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc464068bb0 0x7fc464160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:04.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:04.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc464159c10 0x7fc464069b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640ae5d0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46415e1e0 con 0x7fc464068bb0 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46bfca700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46415e4c0 con 0x7fc464159c10 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:47882/0 (socket says 172.21.15.170:47882) 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 -- 172.21.15.170:0/839775963 learned_addr learned my addr 172.21.15.170:0/839775963 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469d66700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc464068bb0 0x7fc464160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 -- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc464159c10 msgr2=0x7fc464069b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc46a567700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc464159c10 0x7fc464069b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:04.158 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc464159c10 0x7fc464069b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 -- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc464068bb0 msgr2=0x7fc464160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc464068bb0 0x7fc464160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 -- 172.21.15.170:0/839775963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458009420 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.154+0000 7fc469565700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc458009ed0 tx=0x7fc458017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.155+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc458011b10 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.155+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45801a430 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.155+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46415e7a0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.155+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45801a7f0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.155+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4640ae1f0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.156+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4640ae740 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.156+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc45801aba0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.159 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.157+0000 7fc461ffb700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 0x7fc4500697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:04.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.157+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6266+0+0 (secure 0 0 0) 0x7fc458086ab0 con 0x7fc4640b5ea0 2023-10-08T16:13:04.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.157+0000 7fc469d66700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 0x7fc4500697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:04.160 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.157+0000 7fc469d66700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 0x7fc4500697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc454005c50 tx=0x7fc454009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:04.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.161+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc458053510 con 0x7fc4640b5ea0 2023-10-08T16:13:04.385 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:04.384+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000975909", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc46404c3c0 con 0x7fc4640b5ea0 2023-10-08T16:13:05.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.019+0000 7fc461ffb700 1 -- 172.21.15.170:0/839775963 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000975909", "yes_i_really_mean_it": true}]=0 v114) v1 ==== 121+0+0 (secure 0 0 0) 0x7fc458056b30 con 0x7fc4640b5ea0 2023-10-08T16:13:05.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 msgr2=0x7fc4500697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 0x7fc4500697d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fc454005c50 tx=0x7fc454009040 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 msgr2=0x7fc464154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc458009ed0 tx=0x7fc458017000 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 shutdown_connections 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc464159c10 0x7fc464069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc464068bb0 0x7fc464160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc450067320 0x7fc4500697d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 --2- 172.21.15.170:0/839775963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc4640b5ea0 0x7fc464154140 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 >> 172.21.15.170:0/839775963 conn(0x7fc464099a20 msgr2=0x7fc4640b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:05.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 shutdown_connections 2023-10-08T16:13:05.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.022+0000 7fc46bfca700 1 -- 172.21.15.170:0/839775963 wait complete. 2023-10-08T16:13:05.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000975909.meta cephfs.volume_0000000000975909.meta --yes-i-really-really-mean-it 2023-10-08T16:13:05.193 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.193+0000 7fed24ab1700 1 Processor -- start 2023-10-08T16:13:05.194 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.193+0000 7fed24ab1700 1 -- start start 2023-10-08T16:13:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed24ab1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed200b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed24ab1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fed200cf350 0x7fed200cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.197 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed24ab1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200b5b00 con 0x7fed200b53d0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed24ab1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed200b5da0 con 0x7fed200cf350 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed24ab1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed200b6080 con 0x7fed20159ec0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed17fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fed200b53d0 0x7fed200cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43204/0 (socket says 172.21.15.170:43204) 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed17fff700 1 -- 172.21.15.170:0/1288157666 learned_addr learned my addr 172.21.15.170:0/1288157666 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885529462 0 0) 0x7fed200b5b00 con 0x7fed200b53d0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed0c003570 con 0x7fed200b53d0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1ed9d700 1 --2- 172.21.15.170:0/1288157666 >> v2:172.21.15.191:3300/0 conn(0x7fed200cf350 0x7fed200cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed200b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 418354294 0 0) 0x7fed0c003570 con 0x7fed200b53d0 2023-10-08T16:13:05.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 msgr2=0x7fed200b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed200b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 >> v2:172.21.15.191:3300/0 conn(0x7fed200cf350 msgr2=0x7fed200cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 --2- 172.21.15.170:0/1288157666 >> v2:172.21.15.191:3300/0 conn(0x7fed200cf350 0x7fed200cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200cc100 con 0x7fed200b53d0 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.194+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fed08001470 con 0x7fed200b53d0 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 >> v1:172.21.15.170:6789/0 conn(0x7fed200b53d0 legacy=0x7fed200cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c003a60 0x7fed0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed0c004810 0x7fed0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200b5da0 con 0x7fed0c004810 2023-10-08T16:13:05.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200b6080 con 0x7fed0c003a60 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0c00ba60 con 0x7fed20159ec0 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed0c004810 0x7fed0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c003a60 0x7fed0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1ed9d700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 -- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 msgr2=0x7fed0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed20159ec0 0x7fed0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 -- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c003a60 msgr2=0x7fed0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c003a60 0x7fed0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 -- 172.21.15.170:0/1288157666 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0c003570 con 0x7fed0c004810 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed17fff700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed0c004810 0x7fed0c004c80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c3f0 tx=0x7fed08001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f94a9ea73ede865 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:05.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800cc80 con 0x7fed0c004810 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800ce00 con 0x7fed0c004810 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.195+0000 7fed1dd9b700 1 -- 172.21.15.170:0/1288157666 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08005b70 con 0x7fed0c004810 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 -- 172.21.15.170:0/1288157666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed0c004810 msgr2=0x7fed0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1288157666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed0c004810 0x7fed0c004c80 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fed0800c3f0 tx=0x7fed08001e90 comp rx=0 tx=0).stop 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed1ed9d700 1 -- 172.21.15.170:0/1288157666 reap_dead start 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 -- 172.21.15.170:0/1288157666 shutdown_connections 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 -- 172.21.15.170:0/1288157666 >> 172.21.15.170:0/1288157666 conn(0x7fed200999f0 msgr2=0x7fed200b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 -- 172.21.15.170:0/1288157666 shutdown_connections 2023-10-08T16:13:05.201 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.196+0000 7fed24ab1700 1 -- 172.21.15.170:0/1288157666 wait complete. 2023-10-08T16:13:05.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.197+0000 7fed24ab1700 1 Processor -- start 2023-10-08T16:13:05.202 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.197+0000 7fed24ab1700 1 -- start start 2023-10-08T16:13:05.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 0x7fed2006ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed2009a460 0x7fed20068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2009b500 con 0x7fed200688d0 2023-10-08T16:13:05.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20157300 con 0x7fed0c004810 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2006b340 con 0x7fed2009a460 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53198/0 (socket says 172.21.15.170:53198) 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 -- 172.21.15.170:0/1274129 learned_addr learned my addr 172.21.15.170:0/1274129 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed17fff700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed2009a460 0x7fed20068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 0x7fed2006ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 -- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed2009a460 msgr2=0x7fed20068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed2009a460 0x7fed20068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 -- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 msgr2=0x7fed2006ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 0x7fed2006ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:05.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 -- 172.21.15.170:0/1274129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2009a8f0 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 0x7fed2006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed1ed9d700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fed180094d0 tx=0x7fed1800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800e040 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1800f040 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800fe50 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2009abd0 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.198+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2009b190 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.199+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20063c90 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.200+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fed18015640 con 0x7fed200688d0 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.200+0000 7fed167fc700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 0x7fed000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:05.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.200+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 0x7fed000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:05.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.200+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6266+0+0 (secure 0 0 0) 0x7fed18085820 con 0x7fed200688d0 2023-10-08T16:13:05.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.201+0000 7fed1e59c700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 0x7fed000697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fed10009390 tx=0x7fed10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:05.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.205+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed18052270 con 0x7fed200688d0 2023-10-08T16:13:05.389 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:05.389+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000975909.meta", "pool2": "cephfs.volume_0000000000975909.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed200718b0 con 0x7fed200688d0 2023-10-08T16:13:06.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.022+0000 7fed167fc700 1 -- 172.21.15.170:0/1274129 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000975909.meta", "pool2": "cephfs.volume_0000000000975909.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000975909.meta' removed v173) v1 ==== 241+0+0 (secure 0 0 0) 0x7fed18055890 con 0x7fed200688d0 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.025+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 msgr2=0x7fed000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.025+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 0x7fed000697d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fed10009390 tx=0x7fed10008040 comp rx=0 tx=0).stop 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.025+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 msgr2=0x7fed20068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.025+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fed180094d0 tx=0x7fed1800b040 comp rx=0 tx=0).stop 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 shutdown_connections 2023-10-08T16:13:06.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fed00067320 0x7fed000697d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fed200688d0 0x7fed20068d70 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fed0c004810 0x7fed2006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 --2- 172.21.15.170:0/1274129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fed2009a460 0x7fed20068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.026+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 >> 172.21.15.170:0/1274129 conn(0x7fed200999f0 msgr2=0x7fed20158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:06.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.027+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 shutdown_connections 2023-10-08T16:13:06.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.028+0000 7fed24ab1700 1 -- 172.21.15.170:0/1274129 wait complete. 2023-10-08T16:13:06.030 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs.volume_0000000000975909.meta' removed 2023-10-08T16:13:06.041 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:13:06.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.161+0000 7fbd7ebe9700 1 Processor -- start 2023-10-08T16:13:06.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 -- start start 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd780b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd780cf350 0x7fbd780cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd780b5b00 con 0x7fbd780b53d0 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd780b5da0 con 0x7fbd780cf350 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd7ebe9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b6080 con 0x7fbd78159ec0 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd77fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbd780b53d0 0x7fbd780cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43208/0 (socket says 172.21.15.170:43208) 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.162+0000 7fbd77fff700 1 -- 172.21.15.170:0/2574351988 learned_addr learned my addr 172.21.15.170:0/2574351988 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd7d186700 1 --2- 172.21.15.170:0/2574351988 >> v2:172.21.15.191:3300/0 conn(0x7fbd780cf350 0x7fbd780cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249509807 0 0) 0x7fbd780b5b00 con 0x7fbd780b53d0 2023-10-08T16:13:06.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd60003570 con 0x7fbd780b53d0 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd7c985700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd780b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340931255 0 0) 0x7fbd60003570 con 0x7fbd780b53d0 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 msgr2=0x7fbd780b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd780b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 >> v2:172.21.15.191:3300/0 conn(0x7fbd780cf350 msgr2=0x7fbd780cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 --2- 172.21.15.170:0/2574351988 >> v2:172.21.15.191:3300/0 conn(0x7fbd780cf350 0x7fbd780cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd780cc100 con 0x7fbd780b53d0 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fbd6c001470 con 0x7fbd780b53d0 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 >> v1:172.21.15.170:6789/0 conn(0x7fbd780b53d0 legacy=0x7fbd780cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60003a60 0x7fbd60003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60004810 0x7fbd60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b5da0 con 0x7fbd60004810 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd780b6080 con 0x7fbd60003a60 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6000ba60 con 0x7fbd78159ec0 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60004810 0x7fbd60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd7c985700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60003a60 0x7fbd60003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd7d186700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 -- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 msgr2=0x7fbd600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:06.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd78159ec0 0x7fbd600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 -- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60003a60 msgr2=0x7fbd60003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60003a60 0x7fbd60003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.163+0000 7fbd77fff700 1 -- 172.21.15.170:0/2574351988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd60003570 con 0x7fbd60004810 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60004810 0x7fbd60004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c3f0 tx=0x7fbd6c001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53c1d19a648438e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00cc80 con 0x7fbd60004810 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00ce00 con 0x7fbd60004810 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd777fe700 1 -- 172.21.15.170:0/2574351988 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005b70 con 0x7fbd60004810 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2574351988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60004810 msgr2=0x7fbd60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2574351988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60004810 0x7fbd60004c80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c3f0 tx=0x7fbd6c001e90 comp rx=0 tx=0).stop 2023-10-08T16:13:06.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7d186700 1 -- 172.21.15.170:0/2574351988 reap_dead start 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2574351988 shutdown_connections 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2574351988 >> 172.21.15.170:0/2574351988 conn(0x7fbd780999f0 msgr2=0x7fbd780b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2574351988 shutdown_connections 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.164+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2574351988 wait complete. 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.165+0000 7fbd7ebe9700 1 Processor -- start 2023-10-08T16:13:06.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 -- start start 2023-10-08T16:13:06.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd60004810 0x7fbd781480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 0x7fbd78069c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7809d740 con 0x7fbd7809cd90 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7809b350 con 0x7fbd78069810 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7ebe9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7809a460 con 0x7fbd60004810 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7d186700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7d186700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53218/0 (socket says 172.21.15.170:53218) 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.166+0000 7fbd7d186700 1 -- 172.21.15.170:0/2296408099 learned_addr learned my addr 172.21.15.170:0/2296408099 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7c985700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd60004810 0x7fbd781480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 0x7fbd78069c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 -- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd60004810 msgr2=0x7fbd781480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd60004810 0x7fbd781480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 -- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 msgr2=0x7fbd78069c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:06.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 0x7fbd78069c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 -- 172.21.15.170:0/2296408099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7809a740 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd77fff700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 0x7fbd78069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7d186700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fbd78089120 tx=0x7fbd700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000f910 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000fa70 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7000fe10 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7809aa20 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.167+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7809afe0 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.168+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd78063c90 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.168+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd70014650 con 0x7fbd7809cd90 2023-10-08T16:13:06.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.169+0000 7fbd74ff9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 0x7fbd640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.169+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5791+0+0 (secure 0 0 0) 0x7fbd70085320 con 0x7fbd7809cd90 2023-10-08T16:13:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.169+0000 7fbd7c985700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 0x7fbd640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.169+0000 7fbd7c985700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 0x7fbd640697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009390 tx=0x7fbd6800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.173+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd70051fe0 con 0x7fbd7809cd90 2023-10-08T16:13:06.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:06.357+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd780718b0 con 0x7fbd7809cd90 2023-10-08T16:13:07.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.027+0000 7fbd74ff9700 1 -- 172.21.15.170:0/2296408099 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v174) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbd70055600 con 0x7fbd7809cd90 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.032+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 msgr2=0x7fbd640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.032+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 0x7fbd640697d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009390 tx=0x7fbd6800b040 comp rx=0 tx=0).stop 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.032+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 msgr2=0x7fbd7809d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.032+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fbd78089120 tx=0x7fbd700091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 shutdown_connections 2023-10-08T16:13:07.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd64067320 0x7fbd640697d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd7809cd90 0x7fbd7809d200 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd60004810 0x7fbd781480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 --2- 172.21.15.170:0/2296408099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd78069810 0x7fbd78069c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.033+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 >> 172.21.15.170:0/2296408099 conn(0x7fbd780999f0 msgr2=0x7fbd78159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.034+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 shutdown_connections 2023-10-08T16:13:07.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.034+0000 7fbd7ebe9700 1 -- 172.21.15.170:0/2296408099 wait complete. 2023-10-08T16:13:07.037 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:13:07.047 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm another-data-pool another-data-pool --yes-i-really-really-mean-it 2023-10-08T16:13:07.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.167+0000 7fb46ac58700 1 Processor -- start 2023-10-08T16:13:07.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 -- start start 2023-10-08T16:13:07.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb464159ec0 0x7fb4640b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb4640cf350 0x7fb4640cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b5b00 con 0x7fb4640b53d0 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640b5850 con 0x7fb4640cf350 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb46ac58700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640b5da0 con 0x7fb464159ec0 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb4691f5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb4640cf350 0x7fb4640cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.168+0000 7fb463fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb4640b53d0 0x7fb4640cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43216/0 (socket says 172.21.15.170:43216) 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb463fff700 1 -- 172.21.15.170:0/3195544130 learned_addr learned my addr 172.21.15.170:0/3195544130 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4691f5700 1 -- 172.21.15.170:0/3195544130 >> v2:172.21.15.191:3300/0 conn(0x7fb464159ec0 msgr2=0x7fb4640b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4689f4700 1 --2- 172.21.15.170:0/3195544130 >> v2:172.21.15.191:3300/0 conn(0x7fb464159ec0 0x7fb4640b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4691f5700 1 --2- 172.21.15.170:0/3195544130 >> v2:172.21.15.191:3300/0 conn(0x7fb464159ec0 0x7fb4640b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4691f5700 1 -- 172.21.15.170:0/3195544130 >> v1:172.21.15.170:6789/0 conn(0x7fb4640b53d0 legacy=0x7fb4640cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4637fe700 1 -- 172.21.15.170:0/3195544130 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438518750 0 0) 0x7fb4640b5b00 con 0x7fb4640b53d0 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4691f5700 1 -- 172.21.15.170:0/3195544130 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640b6080 con 0x7fb4640cf350 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4691f5700 1 --2- 172.21.15.170:0/3195544130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4640cf350 0x7fb4640cd010 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb464089120 tx=0x7fb45c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6676ccc9e92441bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4637fe700 1 -- 172.21.15.170:0/3195544130 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00b6b0 con 0x7fb4640cf350 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4637fe700 1 -- 172.21.15.170:0/3195544130 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00b810 con 0x7fb4640cf350 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.169+0000 7fb4637fe700 1 -- 172.21.15.170:0/3195544130 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00bbb0 con 0x7fb4640cf350 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 -- 172.21.15.170:0/3195544130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4640cf350 msgr2=0x7fb4640cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3195544130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4640cf350 0x7fb4640cd010 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fb464089120 tx=0x7fb45c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 -- 172.21.15.170:0/3195544130 shutdown_connections 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3195544130 >> v2:172.21.15.191:3300/0 conn(0x7fb464159ec0 0x7fb4640b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3195544130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4640cf350 0x7fb4640cd010 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 -- 172.21.15.170:0/3195544130 >> 172.21.15.170:0/3195544130 conn(0x7fb4640999f0 msgr2=0x7fb4640b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:07.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 -- 172.21.15.170:0/3195544130 shutdown_connections 2023-10-08T16:13:07.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.170+0000 7fb46ac58700 1 -- 172.21.15.170:0/3195544130 wait complete. 2023-10-08T16:13:07.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.171+0000 7fb46ac58700 1 Processor -- start 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.171+0000 7fb46ac58700 1 -- start start 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4640b53d0 0x7fb464155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 0x7fb46406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464153b10 con 0x7fb464159ec0 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464156310 con 0x7fb4640b53d0 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb464156540 con 0x7fb4640cf350 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53234/0 (socket says 172.21.15.170:53234) 2023-10-08T16:13:07.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 -- 172.21.15.170:0/3091094582 learned_addr learned my addr 172.21.15.170:0/3091094582 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4689f4700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4640b53d0 0x7fb464155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb463fff700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 0x7fb46406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 -- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 msgr2=0x7fb46406ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 0x7fb46406ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 -- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4640b53d0 msgr2=0x7fb464155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4640b53d0 0x7fb464155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 -- 172.21.15.170:0/3091094582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c009420 con 0x7fb464159ec0 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb463fff700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 0x7fb46406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb4691f5700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb464089390 tx=0x7fb45c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:07.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c011630 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c011790 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb464156820 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c011b50 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.172+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464156de0 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.173+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb464068390 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.174+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb45c0253f0 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.174+0000 7fb460ff9700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.174+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5360+0+0 (secure 0 0 0) 0x7fb45c086860 con 0x7fb464159ec0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.174+0000 7fb4689f4700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:07.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.175+0000 7fb4689f4700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 0x7fb4440697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e5f0 tx=0x7fb4540076d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:07.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.178+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb464068390 con 0x7fb464159ec0 2023-10-08T16:13:07.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:07.362+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb464068390 con 0x7fb464159ec0 2023-10-08T16:13:08.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.028+0000 7fb460ff9700 1 -- 172.21.15.170:0/3091094582 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "another-data-pool", "pool2": "another-data-pool", "yes_i_really_really_mean_it": true}]=0 pool 'another-data-pool' removed v175) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb464068390 con 0x7fb464159ec0 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 msgr2=0x7fb4440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 0x7fb4440697d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e5f0 tx=0x7fb4540076d0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 msgr2=0x7fb464146d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fb464089390 tx=0x7fb45c017000 comp rx=0 tx=0).stop 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 shutdown_connections 2023-10-08T16:13:08.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb444067320 0x7fb4440697d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb464159ec0 0x7fb464146d70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb4640b53d0 0x7fb464155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 --2- 172.21.15.170:0/3091094582 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb4640cf350 0x7fb46406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.031+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 >> 172.21.15.170:0/3091094582 conn(0x7fb4640999f0 msgr2=0x7fb4640c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.032+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 shutdown_connections 2023-10-08T16:13:08.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.032+0000 7fb46ac58700 1 -- 172.21.15.170:0/3091094582 wait complete. 2023-10-08T16:13:08.034 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'another-data-pool' removed 2023-10-08T16:13:08.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:13:08.161 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.161+0000 7f1e73fff700 1 Processor -- start 2023-10-08T16:13:08.162 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.161+0000 7f1e73fff700 1 -- start start 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b4e70 0x7f1e740ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e73fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1e740b16d0 0x7f1e740b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e73fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740b2080 con 0x7f1e7415d240 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e73fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740cfb00 con 0x7f1e740b16d0 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e73fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740b5c40 con 0x7f1e740b4e70 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1e740b16d0 0x7f1e740b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1e740b16d0 0x7f1e740b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53244/0 (socket says 172.21.15.170:53244) 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 -- 172.21.15.170:0/2112450579 learned_addr learned my addr 172.21.15.170:0/2112450579 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:08.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e72ffd700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b4e70 0x7f1e740ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 -- 172.21.15.170:0/2112450579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b4e70 msgr2=0x7f1e740ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b4e70 0x7f1e740ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 -- 172.21.15.170:0/2112450579 >> v1:172.21.15.191:6789/0 conn(0x7f1e7415d240 legacy=0x7f1e7415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.162+0000 7f1e727fc700 1 -- 172.21.15.170:0/2112450579 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740b5f20 con 0x7f1e740b16d0 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e727fc700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b16d0 0x7f1e740b1b40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1e680099b0 tx=0x7f1e68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649e58089cd37b69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e71ffb700 1 -- 172.21.15.170:0/2112450579 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800b860 con 0x7f1e740b16d0 2023-10-08T16:13:08.166 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e71ffb700 1 -- 172.21.15.170:0/2112450579 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800b9c0 con 0x7f1e740b16d0 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e71ffb700 1 -- 172.21.15.170:0/2112450579 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800bd60 con 0x7f1e740b16d0 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 -- 172.21.15.170:0/2112450579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b16d0 msgr2=0x7f1e740b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b16d0 0x7f1e740b1b40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f1e680099b0 tx=0x7f1e68009250 comp rx=0 tx=0).stop 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 -- 172.21.15.170:0/2112450579 shutdown_connections 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b4e70 0x7f1e740ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2112450579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b16d0 0x7f1e740b1b40 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 -- 172.21.15.170:0/2112450579 >> 172.21.15.170:0/2112450579 conn(0x7f1e740d10c0 msgr2=0x7f1e740affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 -- 172.21.15.170:0/2112450579 shutdown_connections 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.163+0000 7f1e73fff700 1 -- 172.21.15.170:0/2112450579 wait complete. 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.164+0000 7f1e73fff700 1 Processor -- start 2023-10-08T16:13:08.167 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 -- start start 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b16d0 0x7f1e7414c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7415d240 0x7f1e740ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c6db0 con 0x7f1e740b4e70 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c7cc0 con 0x7f1e7415d240 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e73fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7414c5f0 con 0x7f1e740b16d0 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e727fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e727fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53250/0 (socket says 172.21.15.170:53250) 2023-10-08T16:13:08.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e727fc700 1 -- 172.21.15.170:0/2216846647 learned_addr learned my addr 172.21.15.170:0/2216846647 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e72ffd700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b16d0 0x7f1e7414c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.165+0000 7f1e737fe700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7415d240 0x7f1e740ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 -- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b16d0 msgr2=0x7f1e7414c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b16d0 0x7f1e7414c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 -- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7415d240 msgr2=0x7f1e740ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7415d240 0x7f1e740ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 -- 172.21.15.170:0/2216846647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680094f0 con 0x7f1e740b4e70 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e727fc700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1e74070dd0 tx=0x7f1e68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68024440 con 0x7f1e740b4e70 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e680245c0 con 0x7f1e740b4e70 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e740c67f0 con 0x7f1e740b4e70 2023-10-08T16:13:08.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68024980 con 0x7f1e740b4e70 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.166+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e740c7950 con 0x7f1e740b4e70 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.167+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e740c7e30 con 0x7f1e740b4e70 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.167+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e6801a420 con 0x7f1e740b4e70 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.168+0000 7f1e5affd700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 0x7f1e600697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.168+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1e680867b0 con 0x7f1e740b4e70 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.168+0000 7f1e72ffd700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 0x7f1e600697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.168+0000 7f1e72ffd700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 0x7f1e600697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009fd0 tx=0x7f1e64009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.172+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e68053740 con 0x7f1e740b4e70 2023-10-08T16:13:08.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.355+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1e7404bb80 con 0x7f1e740b4e70 2023-10-08T16:13:08.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.358+0000 7f1e5affd700 1 -- 172.21.15.170:0/2216846647 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e68056d60 con 0x7f1e740b4e70 2023-10-08T16:13:08.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.360+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 msgr2=0x7f1e600697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 0x7f1e600697d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009fd0 tx=0x7f1e64009880 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 msgr2=0x7f1e740c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f1e74070dd0 tx=0x7f1e68017000 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 shutdown_connections 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e7415d240 0x7f1e740ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e740b16d0 0x7f1e7414c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e60067320 0x7f1e600697d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 --2- 172.21.15.170:0/2216846647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e740b4e70 0x7f1e740c5d50 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 >> 172.21.15.170:0/2216846647 conn(0x7f1e740d10c0 msgr2=0x7f1e7415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 shutdown_connections 2023-10-08T16:13:08.363 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.361+0000 7f1e73fff700 1 -- 172.21.15.170:0/2216846647 wait complete. 2023-10-08T16:13:08.373 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool' 2023-10-08T16:13:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.487+0000 7f89e7810700 1 Processor -- start 2023-10-08T16:13:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.487+0000 7f89e7810700 1 -- start start 2023-10-08T16:13:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e7810700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f89e0159c60 0x7f89e015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e7810700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e015a160 con 0x7f89e0159890 2023-10-08T16:13:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e7810700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e00d13a0 con 0x7f89e015db10 2023-10-08T16:13:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e7810700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e015df50 con 0x7f89e0159c60 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e55ac700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f89e0159890 0x7f89e015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43224/0 (socket says 172.21.15.170:43224) 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e55ac700 1 -- 172.21.15.170:0/3038305869 learned_addr learned my addr 172.21.15.170:0/3038305869 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89e4dab700 1 --2- 172.21.15.170:0/3038305869 >> v2:172.21.15.191:3300/0 conn(0x7f89e0159c60 0x7f89e015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394818646 0 0) 0x7f89e015a160 con 0x7f89e0159890 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89cc003570 con 0x7f89e0159890 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1746292346 0 0) 0x7f89cc003570 con 0x7f89e0159890 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 >> v2:172.21.15.191:3300/0 conn(0x7f89e0159c60 msgr2=0x7f89e015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 --2- 172.21.15.170:0/3038305869 >> v2:172.21.15.191:3300/0 conn(0x7f89e0159c60 0x7f89e015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 >> v1:172.21.15.191:6789/0 conn(0x7f89e015db10 legacy=0x7f89e0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e015e140 con 0x7f89e0159890 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.488+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f89dc001470 con 0x7f89e0159890 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 >> v1:172.21.15.170:6789/0 conn(0x7f89e0159890 legacy=0x7f89e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 0x7f89cc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89cc004810 0x7f89cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc0051c0 0x7f89cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc00c0a0 con 0x7f89cc003a60 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc00c210 con 0x7f89cc0051c0 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc00c3d0 con 0x7f89cc004810 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 0x7f89cc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e4dab700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89cc004810 0x7f89cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e5dad700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc0051c0 0x7f89cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 -- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89cc004810 msgr2=0x7f89cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89cc004810 0x7f89cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 -- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc0051c0 msgr2=0x7f89cc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc0051c0 0x7f89cc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e5dad700 1 -- 172.21.15.170:0/3038305869 reap_dead start 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 -- 172.21.15.170:0/3038305869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc003570 con 0x7f89cc003a60 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89e55ac700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 0x7f89cc003e70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c3c0 tx=0x7f89dc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361c400a18f05bb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc00cc90 con 0x7f89cc003a60 2023-10-08T16:13:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc00cdf0 con 0x7f89cc003a60 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.489+0000 7f89d7fff700 1 -- 172.21.15.170:0/3038305869 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc005b50 con 0x7f89cc003a60 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 -- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 msgr2=0x7f89cc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 0x7f89cc003e70 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c3c0 tx=0x7f89dc001e50 comp rx=0 tx=0).stop 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 -- 172.21.15.170:0/3038305869 shutdown_connections 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 --2- 172.21.15.170:0/3038305869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89cc003a60 0x7f89cc003e70 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 -- 172.21.15.170:0/3038305869 >> 172.21.15.170:0/3038305869 conn(0x7f89e0068e20 msgr2=0x7f89e00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 -- 172.21.15.170:0/3038305869 shutdown_connections 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.490+0000 7f89e7810700 1 -- 172.21.15.170:0/3038305869 wait complete. 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.491+0000 7f89e7810700 1 Processor -- start 2023-10-08T16:13:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.491+0000 7f89e7810700 1 -- start start 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc003a60 0x7f89e00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e0155ca0 0x7f89e0152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e016dba0 con 0x7f89e016d1c0 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00b4aa0 con 0x7f89cc003a60 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00b5340 con 0x7f89e0155ca0 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53274/0 (socket says 172.21.15.170:53274) 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 -- 172.21.15.170:0/4258718339 learned_addr learned my addr 172.21.15.170:0/4258718339 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 -- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e0155ca0 msgr2=0x7f89e0152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e0155ca0 0x7f89e0152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 -- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc003a60 msgr2=0x7f89e00b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e55ac700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc003a60 0x7f89e00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc003a60 0x7f89e00b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 -- 172.21.15.170:0/4258718339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0152540 con 0x7f89e016d1c0 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e5dad700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cadc0 tx=0x7f89d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8006db0 con 0x7f89e016d1c0 2023-10-08T16:13:08.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8003110 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e00b3d00 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.492+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8001580 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.493+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e00b4fd0 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.493+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e004c3c0 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.494+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89d8001930 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.494+0000 7f89d57fa700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 0x7f89c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.494+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f89d8083ef0 con 0x7f89e016d1c0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.494+0000 7f89e55ac700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 0x7f89c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:08.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.495+0000 7f89e55ac700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 0x7f89c40697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f89e00b36b0 tx=0x7f89dc015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:08.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.499+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89d8050e80 con 0x7f89e016d1c0 2023-10-08T16:13:08.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:08.758+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]} v 0) v1 -- 0x7f89e006a5b0 con 0x7f89e016d1c0 2023-10-08T16:13:09.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.035+0000 7f89d57fa700 1 -- 172.21.15.170:0/4258718339 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_for_more_than_one_data_pool"]}]=0 v434) v1 ==== 163+0+0 (secure 0 0 0) 0x7f89d8019070 con 0x7f89e016d1c0 2023-10-08T16:13:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 msgr2=0x7f89c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 0x7f89c40697d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f89e00b36b0 tx=0x7f89dc015000 comp rx=0 tx=0).stop 2023-10-08T16:13:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 msgr2=0x7f89e016d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.038 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f89e00cadc0 tx=0x7f89d8009040 comp rx=0 tx=0).stop 2023-10-08T16:13:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 shutdown_connections 2023-10-08T16:13:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89c4067320 0x7f89c40697d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89e016d1c0 0x7f89e016d660 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f89cc003a60 0x7f89e00b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 --2- 172.21.15.170:0/4258718339 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f89e0155ca0 0x7f89e0152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 >> 172.21.15.170:0/4258718339 conn(0x7f89e0068e20 msgr2=0x7f89e00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:09.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 shutdown_connections 2023-10-08T16:13:09.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.037+0000 7f89e7810700 1 -- 172.21.15.170:0/4258718339 wait complete. 2023-10-08T16:13:09.050 INFO:tasks.cephfs_test_runner:That renaming a volume with more than one data pool does not change ... ok 2023-10-08T16:13:09.050 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) That volume rename is idempotent. 2023-10-08T16:13:09.050 INFO:tasks.cephfs_test_runner:test_volume_rename_idempotency (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:13:09.051 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-08T16:13:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.164+0000 7fcd7dffb700 1 Processor -- start 2023-10-08T16:13:09.165 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.164+0000 7fcd7dffb700 1 -- start start 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7dffb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7dffb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcd7815d230 0x7fcd7815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7dffb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7815dbd0 con 0x7fcd78159420 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7dffb700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780b11d0 con 0x7fcd7815d230 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7dffb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780b14b0 con 0x7fcd781597f0 2023-10-08T16:13:09.168 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7d7fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcd7815d230 0x7fcd7815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7d7fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcd7815d230 0x7fcd7815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53282/0 (socket says 172.21.15.170:53282) 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7d7fa700 1 -- 172.21.15.170:0/250996593 learned_addr learned my addr 172.21.15.170:0/250996593 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd77fff700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7d7fa700 1 -- 172.21.15.170:0/250996593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 msgr2=0x7fcd780b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.165+0000 7fcd7d7fa700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd780b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7d7fa700 1 -- 172.21.15.170:0/250996593 >> v1:172.21.15.191:6789/0 conn(0x7fcd78159420 legacy=0x7fcd780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7d7fa700 1 -- 172.21.15.170:0/250996593 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780b1790 con 0x7fcd7815d230 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7d7fa700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd7815d230 0x7fcd7815d690 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fcd700099b0 tx=0x7fcd70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e458fb8f8b22b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd777fe700 1 -- 172.21.15.170:0/250996593 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70009ea0 con 0x7fcd7815d230 2023-10-08T16:13:09.169 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd777fe700 1 -- 172.21.15.170:0/250996593 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000b860 con 0x7fcd7815d230 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd777fe700 1 -- 172.21.15.170:0/250996593 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000bc00 con 0x7fcd7815d230 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 -- 172.21.15.170:0/250996593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd7815d230 msgr2=0x7fcd7815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd7815d230 0x7fcd7815d690 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fcd700099b0 tx=0x7fcd70009250 comp rx=0 tx=0).stop 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 -- 172.21.15.170:0/250996593 shutdown_connections 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd7815d230 0x7fcd7815d690 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/250996593 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd780b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 -- 172.21.15.170:0/250996593 >> 172.21.15.170:0/250996593 conn(0x7fcd780d0660 msgr2=0x7fcd780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.166+0000 7fcd7dffb700 1 -- 172.21.15.170:0/250996593 shutdown_connections 2023-10-08T16:13:09.170 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.167+0000 7fcd7dffb700 1 -- 172.21.15.170:0/250996593 wait complete. 2023-10-08T16:13:09.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.168+0000 7fcd7dffb700 1 Processor -- start 2023-10-08T16:13:09.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.168+0000 7fcd7dffb700 1 -- start start 2023-10-08T16:13:09.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.168+0000 7fcd7dffb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.168+0000 7fcd7dffb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd7815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7815d230 0x7fcd78157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780c95c0 con 0x7fcd78159420 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd780c9750 con 0x7fcd7815d230 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815ac90 con 0x7fcd781597f0 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53286/0 (socket says 172.21.15.170:53286) 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 -- 172.21.15.170:0/4225318819 learned_addr learned my addr 172.21.15.170:0/4225318819 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:09.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd77fff700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd7815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7d7fa700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7815d230 0x7fcd78157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 -- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 msgr2=0x7fcd7815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd7815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 -- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7815d230 msgr2=0x7fcd78157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7815d230 0x7fcd78157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 -- 172.21.15.170:0/4225318819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700094f0 con 0x7fcd78159420 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd77fff700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd7815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7cff9700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cb10 tx=0x7fcd680017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800a760 con 0x7fcd78159420 2023-10-08T16:13:09.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6800a8e0 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7815af70 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6800aca0 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.169+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7815b530 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.170+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd780ccf00 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.171+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcd68003030 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.171+0000 7fcd74ff9700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 0x7fcd580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.171+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcd680848a0 con 0x7fcd78159420 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.171+0000 7fcd77fff700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 0x7fcd580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:09.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.171+0000 7fcd77fff700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 0x7fcd580697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcd7816cc80 tx=0x7fcd60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:09.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.175+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd68051830 con 0x7fcd78159420 2023-10-08T16:13:09.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:09.437+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7fcd7815dbd0 con 0x7fcd78159420 2023-10-08T16:13:10.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.036+0000 7fcd74ff9700 1 -- 172.21.15.170:0/4225318819 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v435) v1 ==== 150+0+0 (secure 0 0 0) 0x7fcd7815dbd0 con 0x7fcd78159420 2023-10-08T16:13:10.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 msgr2=0x7fcd580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 0x7fcd580697d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcd7816cc80 tx=0x7fcd60008040 comp rx=0 tx=0).stop 2023-10-08T16:13:10.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 msgr2=0x7fcd7815a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.039 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800cb10 tx=0x7fcd680017b0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 shutdown_connections 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcd7815d230 0x7fcd78157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcd58067320 0x7fcd580697d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcd78159420 0x7fcd7815a6b0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 --2- 172.21.15.170:0/4225318819 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcd781597f0 0x7fcd7815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.038+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 >> 172.21.15.170:0/4225318819 conn(0x7fcd780d0660 msgr2=0x7fcd780b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:10.040 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.039+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 shutdown_connections 2023-10-08T16:13:10.041 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.039+0000 7fcd7dffb700 1 -- 172.21.15.170:0/4225318819 wait complete. 2023-10-08T16:13:10.052 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:13:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.169+0000 7ff9df7e4700 1 Processor -- start 2023-10-08T16:13:10.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.169+0000 7ff9df7e4700 1 -- start start 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9df7e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 0x7ff9d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9df7e4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff9d8069560 0x7ff9d80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9df7e4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d815a160 con 0x7ff9d8159c60 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9df7e4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d8069f20 con 0x7ff9d8069560 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9df7e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80d13a0 con 0x7ff9d8159890 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9ddd81700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff9d8069560 0x7ff9d80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9dcd7f700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff9d8159c60 0x7ff9d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43238/0 (socket says 172.21.15.170:43238) 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.170+0000 7ff9dcd7f700 1 -- 172.21.15.170:0/2158809831 learned_addr learned my addr 172.21.15.170:0/2158809831 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9dd580700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 0x7ff9d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9ddd81700 1 -- 172.21.15.170:0/2158809831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 msgr2=0x7ff9d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:10.173 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9ddd81700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 0x7ff9d815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9ddd81700 1 -- 172.21.15.170:0/2158809831 >> v1:172.21.15.170:6789/0 conn(0x7ff9d8159c60 legacy=0x7ff9d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9ddd81700 1 -- 172.21.15.170:0/2158809831 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d815e140 con 0x7ff9d8069560 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9dd580700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 0x7ff9d815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9ddd81700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8069560 0x7ff9d80699e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009e60 tx=0x7ff9d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e11aaf5bb99bacb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9d7fff700 1 -- 172.21.15.170:0/2158809831 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000e070 con 0x7ff9d8069560 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9d7fff700 1 -- 172.21.15.170:0/2158809831 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d000bb30 con 0x7ff9d8069560 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.171+0000 7ff9d7fff700 1 -- 172.21.15.170:0/2158809831 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0012430 con 0x7ff9d8069560 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 -- 172.21.15.170:0/2158809831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8069560 msgr2=0x7ff9d80699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8069560 0x7ff9d80699e0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009e60 tx=0x7ff9d0009780 comp rx=0 tx=0).stop 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 -- 172.21.15.170:0/2158809831 shutdown_connections 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159890 0x7ff9d815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/2158809831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8069560 0x7ff9d80699e0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 -- 172.21.15.170:0/2158809831 >> 172.21.15.170:0/2158809831 conn(0x7ff9d8068e20 msgr2=0x7ff9d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 -- 172.21.15.170:0/2158809831 shutdown_connections 2023-10-08T16:13:10.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.172+0000 7ff9df7e4700 1 -- 172.21.15.170:0/2158809831 wait complete. 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.173+0000 7ff9df7e4700 1 Processor -- start 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.173+0000 7ff9df7e4700 1 -- start start 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d8069560 0x7ff9d80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159c60 0x7ff9d80cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8153b10 con 0x7ff9d8159890 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d816cec0 con 0x7ff9d8069560 2023-10-08T16:13:10.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d8169e00 con 0x7ff9d8159c60 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53308/0 (socket says 172.21.15.170:53308) 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 -- 172.21.15.170:0/857596706 learned_addr learned my addr 172.21.15.170:0/857596706 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dd580700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d8069560 0x7ff9d80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9ddd81700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159c60 0x7ff9d80cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 -- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159c60 msgr2=0x7ff9d80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159c60 0x7ff9d80cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 -- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d8069560 msgr2=0x7ff9d80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d8069560 0x7ff9d80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 -- 172.21.15.170:0/857596706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0009a20 con 0x7ff9d8159890 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9dcd7f700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c7e0 tx=0x7ff9cc007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc005b30 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc005c90 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d816a0e0 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc007320 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.174+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d80c6f00 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.175+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d816c9b0 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.176+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff9cc01b400 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.176+0000 7ff9d57fa700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 0x7ff9c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.176+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff9cc084540 con 0x7ff9d8159890 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.176+0000 7ff9dd580700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 0x7ff9c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.176+0000 7ff9dd580700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 0x7ff9c40697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006f00 tx=0x7ff9c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.180+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9d816c9b0 con 0x7ff9d8159890 2023-10-08T16:13:10.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.364+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff9d816c9b0 con 0x7ff9d8159890 2023-10-08T16:13:10.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.370+0000 7ff9d57fa700 1 -- 172.21.15.170:0/857596706 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff9d816c9b0 con 0x7ff9d8159890 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 msgr2=0x7ff9c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 0x7ff9c40697d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8006f00 tx=0x7ff9c8008040 comp rx=0 tx=0).stop 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 msgr2=0x7ff9d81693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c7e0 tx=0x7ff9cc007f00 comp rx=0 tx=0).stop 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 shutdown_connections 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff9d8159c60 0x7ff9d80cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff9d8069560 0x7ff9d80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff9c4067320 0x7ff9c40697d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 --2- 172.21.15.170:0/857596706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff9d8159890 0x7ff9d81693b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 >> 172.21.15.170:0/857596706 conn(0x7ff9d8068e20 msgr2=0x7ff9d8155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:10.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 shutdown_connections 2023-10-08T16:13:10.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.372+0000 7ff9df7e4700 1 -- 172.21.15.170:0/857596706 wait complete. 2023-10-08T16:13:10.385 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:13:10.385 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:10.425 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:10.467 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:13:10.467 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:13:10.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:10.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.582+0000 7f064bfff700 1 Processor -- start 2023-10-08T16:13:10.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.582+0000 7f064bfff700 1 -- start start 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f064c1597f0 0x7f064c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064bfff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c0b11d0 con 0x7f064c15d230 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064bfff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c0b14b0 con 0x7f064c159420 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064bfff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c0b1790 con 0x7f064c1597f0 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f064c1597f0 0x7f064c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f064c15d230 0x7f064c15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43240/0 (socket says 172.21.15.170:43240) 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064b7fe700 1 -- 172.21.15.170:0/297668294 learned_addr learned my addr 172.21.15.170:0/297668294 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:10.584 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064a7fc700 1 -- 172.21.15.170:0/297668294 >> v1:172.21.15.191:6789/0 conn(0x7f064c159420 legacy=0x7f064c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064a7fc700 1 -- 172.21.15.170:0/297668294 >> v1:172.21.15.170:6789/0 conn(0x7f064c15d230 legacy=0x7f064c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064a7fc700 1 -- 172.21.15.170:0/297668294 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0b1a70 con 0x7f064c1597f0 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.583+0000 7f064a7fc700 1 --2- 172.21.15.170:0/297668294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c1597f0 0x7f064c0b2270 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f06340098e0 tx=0x7f06340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd1b9a95ca2cc47 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f0649ffb700 1 -- 172.21.15.170:0/297668294 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634009e80 con 0x7f064c1597f0 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f0649ffb700 1 -- 172.21.15.170:0/297668294 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063400b860 con 0x7f064c1597f0 2023-10-08T16:13:10.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f0649ffb700 1 -- 172.21.15.170:0/297668294 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400bbe0 con 0x7f064c1597f0 2023-10-08T16:13:10.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 -- 172.21.15.170:0/297668294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c1597f0 msgr2=0x7f064c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 --2- 172.21.15.170:0/297668294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c1597f0 0x7f064c0b2270 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f06340098e0 tx=0x7f06340091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 -- 172.21.15.170:0/297668294 shutdown_connections 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 --2- 172.21.15.170:0/297668294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c1597f0 0x7f064c0b2270 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 -- 172.21.15.170:0/297668294 >> 172.21.15.170:0/297668294 conn(0x7f064c0d0660 msgr2=0x7f064c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 -- 172.21.15.170:0/297668294 shutdown_connections 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.584+0000 7f064bfff700 1 -- 172.21.15.170:0/297668294 wait complete. 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.585+0000 7f064bfff700 1 Processor -- start 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.585+0000 7f064bfff700 1 -- start start 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f064c1597f0 0x7f064c0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f064c15d230 0x7f064c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c153a80 con 0x7f064c159420 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0c7230 con 0x7f064c1597f0 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064bfff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0c6620 con 0x7f064c15d230 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53320/0 (socket says 172.21.15.170:53320) 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064affd700 1 -- 172.21.15.170:0/3519860258 learned_addr learned my addr 172.21.15.170:0/3519860258 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064a7fc700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f064c1597f0 0x7f064c0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064b7fe700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f064c15d230 0x7f064c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064affd700 1 -- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f064c15d230 msgr2=0x7f064c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.586+0000 7f064affd700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f064c15d230 0x7f064c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064affd700 1 -- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f064c1597f0 msgr2=0x7f064c0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064affd700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f064c1597f0 0x7f064c0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064affd700 1 -- 172.21.15.170:0/3519860258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634009420 con 0x7f064c159420 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064affd700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f063c00cb10 tx=0x7f063c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00ce20 con 0x7f064c159420 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c00a7e0 con 0x7f064c159420 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c0c6900 con 0x7f064c159420 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00aba0 con 0x7f064c159420 2023-10-08T16:13:10.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.587+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c0c6ec0 con 0x7f064c159420 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.588+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c04c3c0 con 0x7f064c159420 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.588+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f063c003030 con 0x7f064c159420 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.588+0000 7f0642ffd700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 0x7f062c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.589+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f063c084900 con 0x7f064c159420 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.589+0000 7f064a7fc700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 0x7f062c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:10.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.589+0000 7f064a7fc700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 0x7f062c0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f064c159c70 tx=0x7f0634017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:10.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.593+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f063c051890 con 0x7f064c159420 2023-10-08T16:13:10.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.816+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f064c0c7510 con 0x7f064c159420 2023-10-08T16:13:10.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.817+0000 7f0642ffd700 1 -- 172.21.15.170:0/3519860258 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+3534 (secure 0 0 0) 0x7f063c054eb0 con 0x7f064c159420 2023-10-08T16:13:10.817 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:10.817 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6662,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/701502996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":701502996},{"type":"v1","addr":"172.21.15.191:6834","nonce":701502996}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":6665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/468246605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":468246605},{"type":"v1","addr":"172.21.15.191:6835","nonce":468246605}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":7378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/347062686","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":347062686},{"type":"v1","addr":"172.21.15.170:6837","nonce":347062686}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104},{"gid":8562,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/250450753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":250450753},{"type":"v1","addr":"172.21.15.170:6836","nonce":250450753}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":114}],"filesystems":[]} 2023-10-08T16:13:10.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 msgr2=0x7f062c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 0x7f062c0697d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f064c159c70 tx=0x7f0634017000 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 msgr2=0x7f064c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f063c00cb10 tx=0x7f063c0017b0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 shutdown_connections 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f064c15d230 0x7f064c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f062c067320 0x7f062c0697d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f064c159420 0x7f064c15afb0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 --2- 172.21.15.170:0/3519860258 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f064c1597f0 0x7f064c0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 >> 172.21.15.170:0/3519860258 conn(0x7f064c0d0660 msgr2=0x7f064c0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 shutdown_connections 2023-10-08T16:13:10.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:10.819+0000 7f064bfff700 1 -- 172.21.15.170:0/3519860258 wait complete. 2023-10-08T16:13:10.821 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 114 2023-10-08T16:13:10.831 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:13:10.831 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:13:10.832 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:13:10.832 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:10.832 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:13:10.832 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:13:10.832 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:13:10.832 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:10.833 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:13:10.833 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:13:10.833 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:13:10.833 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:10.833 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:13:10.833 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:13:10.834 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:13:10.834 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:10.843 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:13:10.843 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:13:10.844 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:13:10.845 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:13:10.846 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:13:10.846 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:13:10.847 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:13:10.847 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:13:10.887 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:13:10.888 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:10.889 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:13:10.890 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:10.891 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:13:10.891 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:13:10.892 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:10.892 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:10.893 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:13:10.960 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:13:10.960 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:13:10.971 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:13:10.972 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:13:10.972 INFO:tasks.ceph.mds.d.smithi191.stderr:2023-10-08T16:13:10.971+0000 7f40c337b700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T16:13:11.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.015+0000 7f920a494700 1 Processor -- start 2023-10-08T16:13:11.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f920a494700 1 -- start start 2023-10-08T16:13:11.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f920a494700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f9204140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:11.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f920a494700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415c2a0 con 0x7f920406c0a0 2023-10-08T16:13:11.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f920a494700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920415b690 con 0x7f92040c9510 2023-10-08T16:13:11.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f920a494700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920415b970 con 0x7f920406bcd0 2023-10-08T16:13:11.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.016+0000 7f92037fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f920406c0a0 0x7f92040c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43250/0 (socket says 172.21.15.170:43250) 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f92037fe700 1 -- 172.21.15.170:0/3651243919 learned_addr learned my addr 172.21.15.170:0/3651243919 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9203fff700 1 --2- 172.21.15.170:0/3651243919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f9204140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521649179 0 0) 0x7f920415c2a0 con 0x7f920406c0a0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91ec003570 con 0x7f920406c0a0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942847043 0 0) 0x7f920415b690 con 0x7f92040c9510 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9203fff700 1 -- 172.21.15.170:0/3651243919 >> v1:172.21.15.191:6789/0 conn(0x7f92040c9510 legacy=0x7f92040cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9203fff700 1 -- 172.21.15.170:0/3651243919 >> v1:172.21.15.170:6789/0 conn(0x7f920406c0a0 legacy=0x7f92040c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9203fff700 1 -- 172.21.15.170:0/3651243919 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920415bc50 con 0x7f920406bcd0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9203fff700 1 --2- 172.21.15.170:0/3651243919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f9204140df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009910 tx=0x7f91f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4533281530ef0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400b680 con 0x7f920406bcd0 2023-10-08T16:13:11.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400b7e0 con 0x7f920406bcd0 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.017+0000 7f9202ffd700 1 -- 172.21.15.170:0/3651243919 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400bba0 con 0x7f920406bcd0 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 -- 172.21.15.170:0/3651243919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 msgr2=0x7f9204140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 --2- 172.21.15.170:0/3651243919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f9204140df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009910 tx=0x7f91f40091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 -- 172.21.15.170:0/3651243919 shutdown_connections 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 --2- 172.21.15.170:0/3651243919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f9204140df0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 -- 172.21.15.170:0/3651243919 >> 172.21.15.170:0/3651243919 conn(0x7f92040ce600 msgr2=0x7f92040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 -- 172.21.15.170:0/3651243919 shutdown_connections 2023-10-08T16:13:11.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.018+0000 7f920a494700 1 -- 172.21.15.170:0/3651243919 wait complete. 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.019+0000 7f920a494700 1 Processor -- start 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.019+0000 7f920a494700 1 -- start start 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f92041689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040c9510 0x7f92040a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:11.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040a5990 con 0x7f920406c0a0 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9204168ef0 con 0x7f92040c9510 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f920a494700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92040a6130 con 0x7f920406bcd0 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53360/0 (socket says 172.21.15.170:53360) 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 -- 172.21.15.170:0/633548212 learned_addr learned my addr 172.21.15.170:0/633548212 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f9203fff700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f92041689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f9208a31700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040c9510 0x7f92040a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 -- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 msgr2=0x7f92041689b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f92041689b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 -- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040c9510 msgr2=0x7f92040a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040c9510 0x7f92040a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 -- 172.21.15.170:0/633548212 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4009420 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.020+0000 7f92037fe700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f91f80011e0 tx=0x7f91f8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8005b30 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8005c90 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92040a6410 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8007300 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92040a69d0 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.021+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920404c3c0 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.022+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f91f8007670 con 0x7f920406c0a0 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.022+0000 7f91ebfff700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 0x7f91f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.022+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f91f8083d50 con 0x7f920406c0a0 2023-10-08T16:13:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.023+0000 7f9203fff700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 0x7f91f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.023+0000 7f9203fff700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 0x7f91f00697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f91f40093f0 tx=0x7f91f4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.027+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91f8050ce0 con 0x7f920406c0a0 2023-10-08T16:13:11.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:11.229+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f920406c9d0 con 0x7f920406c0a0 2023-10-08T16:13:12.057 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.057+0000 7f91ebfff700 1 -- 172.21.15.170:0/633548212 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v180) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91f8017020 con 0x7f920406c0a0 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 msgr2=0x7f91f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 0x7f91f00697d0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f91f40093f0 tx=0x7f91f4017000 comp rx=0 tx=0).stop 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 msgr2=0x7f9204159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f91f80011e0 tx=0x7f91f8007df0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 shutdown_connections 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f92040c9510 0x7f92040a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f920406bcd0 0x7f92041689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f91f0067320 0x7f91f00697d0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 --2- 172.21.15.170:0/633548212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f920406c0a0 0x7f9204159690 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 >> 172.21.15.170:0/633548212 conn(0x7f92040ce600 msgr2=0x7f92040cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 shutdown_connections 2023-10-08T16:13:12.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.060+0000 7f920a494700 1 -- 172.21.15.170:0/633548212 wait complete. 2023-10-08T16:13:12.062 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:13:12.075 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:13:12.195 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.195+0000 7f5e9381c700 1 Processor -- start 2023-10-08T16:13:12.196 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.195+0000 7f5e9381c700 1 -- start start 2023-10-08T16:13:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e9381c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e9381c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e8c068bb0 0x7f5e8c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e9381c700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e8c15c480 con 0x7f5e8c159c10 2023-10-08T16:13:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e9381c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8c0b2a40 con 0x7f5e8c068bb0 2023-10-08T16:13:12.198 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e9381c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c15b870 con 0x7f5e8c0b5ea0 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e91db9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e8c068bb0 0x7f5e8c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e91db9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e8c068bb0 0x7f5e8c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53374/0 (socket says 172.21.15.170:53374) 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e91db9700 1 -- 172.21.15.170:0/1950960867 learned_addr learned my addr 172.21.15.170:0/1950960867 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.196+0000 7f5e90db7700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e91db9700 1 -- 172.21.15.170:0/1950960867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 msgr2=0x7f5e8c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e91db9700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e91db9700 1 -- 172.21.15.170:0/1950960867 >> v1:172.21.15.191:6789/0 conn(0x7f5e8c159c10 legacy=0x7f5e8c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e91db9700 1 -- 172.21.15.170:0/1950960867 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e8c15bb50 con 0x7f5e8c068bb0 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e91db9700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c15c910 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5e840099e0 tx=0x7f5e84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83869637e402199 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.199 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e83fff700 1 -- 172.21.15.170:0/1950960867 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400b820 con 0x7f5e8c068bb0 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e83fff700 1 -- 172.21.15.170:0/1950960867 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8400b980 con 0x7f5e8c068bb0 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e83fff700 1 -- 172.21.15.170:0/1950960867 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400bd20 con 0x7f5e8c068bb0 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 -- 172.21.15.170:0/1950960867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c068bb0 msgr2=0x7f5e8c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c15c910 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f5e840099e0 tx=0x7f5e84009250 comp rx=0 tx=0).stop 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 -- 172.21.15.170:0/1950960867 shutdown_connections 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c15c910 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 --2- 172.21.15.170:0/1950960867 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.197+0000 7f5e9381c700 1 -- 172.21.15.170:0/1950960867 >> 172.21.15.170:0/1950960867 conn(0x7f5e8c099a20 msgr2=0x7f5e8c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.198+0000 7f5e9381c700 1 -- 172.21.15.170:0/1950960867 shutdown_connections 2023-10-08T16:13:12.200 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.198+0000 7f5e9381c700 1 -- 172.21.15.170:0/1950960867 wait complete. 2023-10-08T16:13:12.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.199+0000 7f5e9381c700 1 Processor -- start 2023-10-08T16:13:12.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.199+0000 7f5e9381c700 1 -- start start 2023-10-08T16:13:12.203 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.199+0000 7f5e9381c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.199+0000 7f5e9381c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c0c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c1498e0 con 0x7f5e8c159c10 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c149bc0 con 0x7f5e8c068bb0 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8c149ea0 con 0x7f5e8c0b5ea0 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53380/0 (socket says 172.21.15.170:53380) 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 -- 172.21.15.170:0/2074049233 learned_addr learned my addr 172.21.15.170:0/2074049233 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e915b8700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.204 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e90db7700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c0c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 -- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 msgr2=0x7f5e8c0c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c0c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 -- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e8c068bb0 msgr2=0x7f5e8c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 -- 172.21.15.170:0/2074049233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840094f0 con 0x7f5e8c159c10 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e90db7700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c0c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e91db9700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f5e840099b0 tx=0x7f5e84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8400b820 con 0x7f5e8c159c10 2023-10-08T16:13:12.205 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8401a630 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8401aa10 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8c14a180 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.200+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8c15de50 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.201+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8c14a6a0 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.202+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e8402d440 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.202+0000 7f5e817fa700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 0x7f5e700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.202+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f5e84086550 con 0x7f5e8c159c10 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.202+0000 7f5e915b8700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 0x7f5e700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.206 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.203+0000 7f5e915b8700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 0x7f5e700697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5e88007fd0 tx=0x7f5e88001650 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.207 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.206+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e8c14a6a0 con 0x7f5e8c159c10 2023-10-08T16:13:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.462+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5e8c14a6a0 con 0x7f5e8c159c10 2023-10-08T16:13:12.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.463+0000 7f5e817fa700 1 -- 172.21.15.170:0/2074049233 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5e8c14a6a0 con 0x7f5e8c159c10 2023-10-08T16:13:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.468 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:13:12.479 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 msgr2=0x7f5e700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 0x7f5e700697d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5e88007fd0 tx=0x7f5e88001650 comp rx=0 tx=0).stop 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 msgr2=0x7f5e8c147e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f5e840099b0 tx=0x7f5e84017000 comp rx=0 tx=0).stop 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 shutdown_connections 2023-10-08T16:13:12.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e70067320 0x7f5e700697d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e8c159c10 0x7f5e8c147e90 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e8c068bb0 0x7f5e8c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 --2- 172.21.15.170:0/2074049233 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e8c0b5ea0 0x7f5e8c0c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 >> 172.21.15.170:0/2074049233 conn(0x7f5e8c099a20 msgr2=0x7f5e8c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 shutdown_connections 2023-10-08T16:13:12.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.465+0000 7f5e9381c700 1 -- 172.21.15.170:0/2074049233 wait complete. 2023-10-08T16:13:12.483 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:12.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.596+0000 7f2ddd857700 1 Processor -- start 2023-10-08T16:13:12.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.596+0000 7f2ddd857700 1 -- start start 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2ddd857700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2dd8159c60 0x7f2dd815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2ddd857700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd815a160 con 0x7f2dd8159890 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2ddd857700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80d13a0 con 0x7f2dd815db10 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2ddd857700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd815df50 con 0x7f2dd8159c60 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2dd6ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2dd8159890 0x7f2dd815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43252/0 (socket says 172.21.15.170:43252) 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.597+0000 7f2dd6ffd700 1 -- 172.21.15.170:0/2260711690 learned_addr learned my addr 172.21.15.170:0/2260711690 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dce7fc700 1 --2- 172.21.15.170:0/2260711690 >> v2:172.21.15.191:3300/0 conn(0x7f2dd8159c60 0x7f2dd815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133419769 0 0) 0x7f2dd815a160 con 0x7f2dd8159890 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dc4003570 con 0x7f2dd8159890 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234698280 0 0) 0x7f2dd80d13a0 con 0x7f2dd815db10 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd815a160 con 0x7f2dd815db10 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dce7fc700 1 -- 172.21.15.170:0/2260711690 >> v1:172.21.15.191:6789/0 conn(0x7f2dd815db10 legacy=0x7f2dd8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dce7fc700 1 -- 172.21.15.170:0/2260711690 >> v1:172.21.15.170:6789/0 conn(0x7f2dd8159890 legacy=0x7f2dd815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400526143 0 0) 0x7f2dc4003570 con 0x7f2dd8159890 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dce7fc700 1 -- 172.21.15.170:0/2260711690 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd815e140 con 0x7f2dd8159c60 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dce7fc700 1 --2- 172.21.15.170:0/2260711690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd8159c60 0x7f2dd815d5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00098e0 tx=0x7f2dc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2aebad3b056b64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8001470 con 0x7f2dd8159c60 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc0011420 con 0x7f2dd8159c60 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.598+0000 7f2dd67fc700 1 -- 172.21.15.170:0/2260711690 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00117c0 con 0x7f2dd8159c60 2023-10-08T16:13:12.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 -- 172.21.15.170:0/2260711690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd8159c60 msgr2=0x7f2dd815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 --2- 172.21.15.170:0/2260711690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd8159c60 0x7f2dd815d5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00098e0 tx=0x7f2dc00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 -- 172.21.15.170:0/2260711690 shutdown_connections 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 --2- 172.21.15.170:0/2260711690 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd8159c60 0x7f2dd815d5d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 -- 172.21.15.170:0/2260711690 >> 172.21.15.170:0/2260711690 conn(0x7f2dd8068e20 msgr2=0x7f2dd80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 -- 172.21.15.170:0/2260711690 shutdown_connections 2023-10-08T16:13:12.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.599+0000 7f2ddd857700 1 -- 172.21.15.170:0/2260711690 wait complete. 2023-10-08T16:13:12.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.600+0000 7f2ddd857700 1 Processor -- start 2023-10-08T16:13:12.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.600+0000 7f2ddd857700 1 -- start start 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd8159890 0x7f2dd8153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd815db10 0x7f2dd815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80cf4a0 con 0x7f2dd8159c60 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd815d270 con 0x7f2dd815db10 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80b41b0 con 0x7f2dd8159890 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53388/0 (socket says 172.21.15.170:53388) 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 -- 172.21.15.170:0/1009516776 learned_addr learned my addr 172.21.15.170:0/1009516776 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 -- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd8159890 msgr2=0x7f2dd8153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd8159890 0x7f2dd8153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 -- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd815db10 msgr2=0x7f2dd815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd815db10 0x7f2dd815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 -- 172.21.15.170:0/1009516776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc0009420 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dce7fc700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8070e20 tx=0x7f2dc0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc001cc90 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc001cdf0 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc001a6a0 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.601+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd80ceee0 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.602+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd80cebd0 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.603+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd80b3f50 con 0x7f2dd8159c60 2023-10-08T16:13:12.606 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.603+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2dc001aa90 con 0x7f2dd8159c60 2023-10-08T16:13:12.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.603+0000 7f2dcf7fe700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 0x7f2db80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.603+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2dc00862a0 con 0x7f2dd8159c60 2023-10-08T16:13:12.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.603+0000 7f2dd6ffd700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 0x7f2db80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.607 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.604+0000 7f2dd6ffd700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 0x7f2db80697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8000ff0 tx=0x7f2dc8001430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.608 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.608+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2dc0053280 con 0x7f2dd8159c60 2023-10-08T16:13:12.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.793+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dd80b4340 con 0x7f2dd8159c60 2023-10-08T16:13:12.794 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.794+0000 7f2dcf7fe700 1 -- 172.21.15.170:0/1009516776 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f2dc00568a0 con 0x7f2dd8159c60 2023-10-08T16:13:12.794 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:12.795 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":180,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:12.055063+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 msgr2=0x7f2db80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 0x7f2db80697d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8000ff0 tx=0x7f2dc8001430 comp rx=0 tx=0).stop 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 msgr2=0x7f2dd80cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8070e20 tx=0x7f2dc0016000 comp rx=0 tx=0).stop 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 shutdown_connections 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2dd815db10 0x7f2dd815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2dd8159890 0x7f2dd8153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2db8067320 0x7f2db80697d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 --2- 172.21.15.170:0/1009516776 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2dd8159c60 0x7f2dd80cd220 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 >> 172.21.15.170:0/1009516776 conn(0x7f2dd8068e20 msgr2=0x7f2dd81553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 shutdown_connections 2023-10-08T16:13:12.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.796+0000 7f2ddd857700 1 -- 172.21.15.170:0/1009516776 wait complete. 2023-10-08T16:13:12.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:13:12.809 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:13:12.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.927+0000 7fb119f71700 1 Processor -- start 2023-10-08T16:13:12.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.927+0000 7fb119f71700 1 -- start start 2023-10-08T16:13:12.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb119f71700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb114159890 0x7fb11415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb119f71700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11415a160 con 0x7fb114159c60 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb119f71700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1140d13a0 con 0x7fb114069560 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb119f71700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11415e140 con 0x7fb114159890 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb112ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb114159c60 0x7fb11415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43268/0 (socket says 172.21.15.170:43268) 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb112ffd700 1 -- 172.21.15.170:0/1855189075 learned_addr learned my addr 172.21.15.170:0/1855189075 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649657277 0 0) 0x7fb11415a160 con 0x7fb114159c60 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0fc003570 con 0x7fb114159c60 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.928+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> v2:172.21.15.191:3300/0 conn(0x7fb114159890 0x7fb11415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610672148 0 0) 0x7fb0fc003570 con 0x7fb114159c60 2023-10-08T16:13:12.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 >> v2:172.21.15.191:3300/0 conn(0x7fb114159890 msgr2=0x7fb11415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 --2- 172.21.15.170:0/1855189075 >> v2:172.21.15.191:3300/0 conn(0x7fb114159890 0x7fb11415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 >> v1:172.21.15.191:6789/0 conn(0x7fb114069560 legacy=0x7fb1140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11415a3d0 con 0x7fb114159c60 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fb108001470 con 0x7fb114159c60 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> v2:172.21.15.191:3300/0 conn(0x7fb114159890 0x7fb11415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 >> v1:172.21.15.170:6789/0 conn(0x7fb114159c60 legacy=0x7fb11415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 0x7fb0fc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc004810 0x7fb0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc0052e0 0x7fb0fc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11415e140 con 0x7fb0fc003a60 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc00c190 con 0x7fb0fc0052e0 2023-10-08T16:13:12.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc00c350 con 0x7fb0fc004810 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 0x7fb0fc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb112ffd700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc004810 0x7fb0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb113fff700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc0052e0 0x7fb0fc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 -- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc004810 msgr2=0x7fb0fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc004810 0x7fb0fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 -- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc0052e0 msgr2=0x7fb0fc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc0052e0 0x7fb0fc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 -- 172.21.15.170:0/1855189075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc003570 con 0x7fb0fc003a60 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb112ffd700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb0fc004810 0x7fb0fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb113fff700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc0052e0 0x7fb0fc00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:12.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb113fff700 1 -- 172.21.15.170:0/1855189075 reap_dead start 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1137fe700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 0x7fb0fc003e70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb104007580 tx=0x7fb1040097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cd6b0d292bf738 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104009ae0 con 0x7fb0fc003a60 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb104007dd0 con 0x7fb0fc003a60 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.929+0000 7fb1127fc700 1 -- 172.21.15.170:0/1855189075 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104014630 con 0x7fb0fc003a60 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 -- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 msgr2=0x7fb0fc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 0x7fb0fc003e70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb104007580 tx=0x7fb1040097d0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 -- 172.21.15.170:0/1855189075 shutdown_connections 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 --2- 172.21.15.170:0/1855189075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0fc003a60 0x7fb0fc003e70 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 -- 172.21.15.170:0/1855189075 >> 172.21.15.170:0/1855189075 conn(0x7fb114068e20 msgr2=0x7fb1140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 -- 172.21.15.170:0/1855189075 shutdown_connections 2023-10-08T16:13:12.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.930+0000 7fb119f71700 1 -- 172.21.15.170:0/1855189075 wait complete. 2023-10-08T16:13:12.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.931+0000 7fb119f71700 1 Processor -- start 2023-10-08T16:13:12.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.931+0000 7fb119f71700 1 -- start start 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 0x7fb1140b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11416a0c0 0x7fb11416a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb114152220 con 0x7fb1140cf950 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11416aa80 con 0x7fb0fc003a60 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb119f71700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1140c79d0 con 0x7fb11416a0c0 2023-10-08T16:13:12.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53404/0 (socket says 172.21.15.170:53404) 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 -- 172.21.15.170:0/3036543674 learned_addr learned my addr 172.21.15.170:0/3036543674 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb1137fe700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 0x7fb1140b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 -- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11416a0c0 msgr2=0x7fb11416a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb113fff700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11416a0c0 0x7fb11416a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11416a0c0 0x7fb11416a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 -- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 msgr2=0x7fb1140b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 0x7fb1140b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 -- 172.21.15.170:0/3036543674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1140b6440 con 0x7fb1140cf950 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb1137fe700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 0x7fb1140b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.932+0000 7fb112ffd700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb108001200 tx=0x7fb108005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.933+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10800c990 con 0x7fb1140cf950 2023-10-08T16:13:12.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.933+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10800caf0 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.933+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108013420 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.933+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1140b5ba0 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.933+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1140b6160 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.934+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11404c3c0 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.934+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb108013790 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.934+0000 7fb0fb7fe700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 0x7fb1000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.934+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fb108084c70 con 0x7fb1140cf950 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.934+0000 7fb1137fe700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 0x7fb1000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.935+0000 7fb1137fe700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 0x7fb1000697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb1140b5ad0 tx=0x7fb104018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:12.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:12.939+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb108051c50 con 0x7fb1140cf950 2023-10-08T16:13:13.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.122+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1140c7b10 con 0x7fb1140cf950 2023-10-08T16:13:13.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.122+0000 7fb0fb7fe700 1 -- 172.21.15.170:0/3036543674 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb108055270 con 0x7fb1140cf950 2023-10-08T16:13:13.123 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:13:13.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 msgr2=0x7fb1000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 0x7fb1000697d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb1140b5ad0 tx=0x7fb104018000 comp rx=0 tx=0).stop 2023-10-08T16:13:13.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 msgr2=0x7fb114169b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fb108001200 tx=0x7fb108005840 comp rx=0 tx=0).stop 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 shutdown_connections 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb11416a0c0 0x7fb11416a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb0fc003a60 0x7fb1140b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb100067320 0x7fb1000697d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 --2- 172.21.15.170:0/3036543674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1140cf950 0x7fb114169b80 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 >> 172.21.15.170:0/3036543674 conn(0x7fb114068e20 msgr2=0x7fb11415d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 shutdown_connections 2023-10-08T16:13:13.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.125+0000 7fb119f71700 1 -- 172.21.15.170:0/3036543674 wait complete. 2023-10-08T16:13:13.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:13:13.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.251+0000 7f420278a700 1 Processor -- start 2023-10-08T16:13:13.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.251+0000 7f420278a700 1 -- start start 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f420278a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f41fc159c60 0x7f41fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f420278a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f41fc15db10 0x7f41fc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f420278a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc15a160 con 0x7f41fc159890 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f420278a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc15df50 con 0x7f41fc159c60 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f420278a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc0d13a0 con 0x7f41fc15db10 2023-10-08T16:13:13.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f41fb7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f41fc159c60 0x7f41fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f41fbfff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f41fc159890 0x7f41fc15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43284/0 (socket says 172.21.15.170:43284) 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.252+0000 7f41fbfff700 1 -- 172.21.15.170:0/492607032 learned_addr learned my addr 172.21.15.170:0/492607032 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f4200d27700 1 --2- 172.21.15.170:0/492607032 >> v2:172.21.15.191:3300/0 conn(0x7f41fc15db10 0x7f41fc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41fb7fe700 1 -- 172.21.15.170:0/492607032 >> v2:172.21.15.191:3300/0 conn(0x7f41fc15db10 msgr2=0x7f41fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41fb7fe700 1 --2- 172.21.15.170:0/492607032 >> v2:172.21.15.191:3300/0 conn(0x7f41fc15db10 0x7f41fc069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41fb7fe700 1 -- 172.21.15.170:0/492607032 >> v1:172.21.15.170:6789/0 conn(0x7f41fc159890 legacy=0x7f41fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41faffd700 1 -- 172.21.15.170:0/492607032 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276373529 0 0) 0x7f41fc15a160 con 0x7f41fc159890 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41fb7fe700 1 -- 172.21.15.170:0/492607032 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc15e140 con 0x7f41fc159c60 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41fb7fe700 1 --2- 172.21.15.170:0/492607032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc159c60 0x7f41fc15d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f41fc06a890 tx=0x7f41f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fe43dc623ae37b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41faffd700 1 -- 172.21.15.170:0/492607032 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011630 con 0x7f41fc159c60 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41faffd700 1 -- 172.21.15.170:0/492607032 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f0011790 con 0x7f41fc159c60 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f41faffd700 1 -- 172.21.15.170:0/492607032 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f0011b30 con 0x7f41fc159c60 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f420278a700 1 -- 172.21.15.170:0/492607032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc159c60 msgr2=0x7f41fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.253+0000 7f420278a700 1 --2- 172.21.15.170:0/492607032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc159c60 0x7f41fc15d5d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f41fc06a890 tx=0x7f41f00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 -- 172.21.15.170:0/492607032 shutdown_connections 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 --2- 172.21.15.170:0/492607032 >> v2:172.21.15.191:3300/0 conn(0x7f41fc15db10 0x7f41fc069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 --2- 172.21.15.170:0/492607032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc159c60 0x7f41fc15d5d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 -- 172.21.15.170:0/492607032 >> 172.21.15.170:0/492607032 conn(0x7f41fc068e20 msgr2=0x7f41fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:13.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 -- 172.21.15.170:0/492607032 shutdown_connections 2023-10-08T16:13:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.254+0000 7f420278a700 1 -- 172.21.15.170:0/492607032 wait complete. 2023-10-08T16:13:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.255+0000 7f420278a700 1 Processor -- start 2023-10-08T16:13:13.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.255+0000 7f420278a700 1 -- start start 2023-10-08T16:13:13.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 0x7f41fc14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41fc159c60 0x7f41fc0cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0b1fb0 con 0x7f41fc15db10 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c5ff0 con 0x7f41fc159890 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc0c56c0 con 0x7f41fc159c60 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53418/0 (socket says 172.21.15.170:53418) 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 -- 172.21.15.170:0/1184175176 learned_addr learned my addr 172.21.15.170:0/1184175176 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:13.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41fb7fe700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41fc159c60 0x7f41fc0cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41fbfff700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 0x7f41fc14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 -- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41fc159c60 msgr2=0x7f41fc0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41fc159c60 0x7f41fc0cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 -- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 msgr2=0x7f41fc14b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 0x7f41fc14b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 -- 172.21.15.170:0/1184175176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0009420 con 0x7f41fc15db10 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41fbfff700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 0x7f41fc14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f4200d27700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009df0 tx=0x7f41f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400cb50 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f400f040 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc0c59a0 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014480 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.256+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc14bdd0 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.257+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc0c6160 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.258+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f41f4014830 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.258+0000 7f41e3fff700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 0x7f41e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.258+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f41f40852b0 con 0x7f41fc15db10 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.258+0000 7f41fbfff700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 0x7f41e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.258+0000 7f41fbfff700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 0x7f41e80697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0018d0 tx=0x7f41ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.262+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41f4052280 con 0x7f41fc15db10 2023-10-08T16:13:13.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.452+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41fc04c3c0 con 0x7f41fc15db10 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.452+0000 7f41e3fff700 1 -- 172.21.15.170:0/1184175176 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7f41f40558a0 con 0x7f41fc15db10 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:13:13.453 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:13.454 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:13:13.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.454+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 msgr2=0x7f41e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.454+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 0x7f41e80697d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0018d0 tx=0x7f41ec009040 comp rx=0 tx=0).stop 2023-10-08T16:13:13.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 msgr2=0x7f41fc151d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009df0 tx=0x7f41f400b040 comp rx=0 tx=0).stop 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 shutdown_connections 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41e8067320 0x7f41e80697d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41fc15db10 0x7f41fc151d80 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41fc159890 0x7f41fc14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 --2- 172.21.15.170:0/1184175176 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41fc159c60 0x7f41fc0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 >> 172.21.15.170:0/1184175176 conn(0x7f41fc068e20 msgr2=0x7f41fc1573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 shutdown_connections 2023-10-08T16:13:13.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.455+0000 7f420278a700 1 -- 172.21.15.170:0/1184175176 wait complete. 2023-10-08T16:13:13.467 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:13:13.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:13:13.585 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.585+0000 7fa4d0333700 1 Processor -- start 2023-10-08T16:13:13.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.585+0000 7fa4d0333700 1 -- start start 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4d0333700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa4c80cd910 0x7fa4c80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4d0333700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80b5b00 con 0x7fa4c80b55a0 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4d0333700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80b5da0 con 0x7fa4c8159ec0 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4d0333700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c80b6080 con 0x7fa4c80cd910 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4ce8d0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa4c80b55a0 0x7fa4c80ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43300/0 (socket says 172.21.15.170:43300) 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.586+0000 7fa4ce8d0700 1 -- 172.21.15.170:0/3883196947 learned_addr learned my addr 172.21.15.170:0/3883196947 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/3883196947 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80cd910 0x7fa4c80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641586697 0 0) 0x7fa4c80b5b00 con 0x7fa4c80b55a0 2023-10-08T16:13:13.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b0003570 con 0x7fa4c80b55a0 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991976056 0 0) 0x7fa4b0003570 con 0x7fa4c80b55a0 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80cd910 msgr2=0x7fa4c80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 --2- 172.21.15.170:0/3883196947 >> v2:172.21.15.191:3300/0 conn(0x7fa4c80cd910 0x7fa4c80ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 >> v1:172.21.15.191:6789/0 conn(0x7fa4c8159ec0 legacy=0x7fa4c80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80cc100 con 0x7fa4c80b55a0 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fa4c0001470 con 0x7fa4c80b55a0 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 >> v1:172.21.15.170:6789/0 conn(0x7fa4c80b55a0 legacy=0x7fa4c80ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0004810 0x7fa4b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4b0005210 0x7fa4b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80b6080 con 0x7fa4b0003a90 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000c130 con 0x7fa4b0004810 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000c2c0 con 0x7fa4b0005210 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0004810 0x7fa4b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce8d0700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4b0005210 0x7fa4b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4b0005210 msgr2=0x7fa4b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4b0005210 0x7fa4b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0004810 msgr2=0x7fa4b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:13.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0004810 0x7fa4b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/3883196947 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0003570 con 0x7fa4b0003a90 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4b0004810 0x7fa4b0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce8d0700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4b0005210 0x7fa4b000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce8d0700 1 -- 172.21.15.170:0/3883196947 reap_dead start 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.587+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4b0003ea0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c300 tx=0x7fa4b8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d4c9ce12ca1af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8001d80 con 0x7fa4b0003a90 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b800cb90 con 0x7fa4b0003a90 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4cd0cd700 1 -- 172.21.15.170:0/3883196947 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8012400 con 0x7fa4b0003a90 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 -- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 msgr2=0x7fa4b0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4b0003ea0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c300 tx=0x7fa4b8001a70 comp rx=0 tx=0).stop 2023-10-08T16:13:13.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 -- 172.21.15.170:0/3883196947 shutdown_connections 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 --2- 172.21.15.170:0/3883196947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4b0003ea0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 -- 172.21.15.170:0/3883196947 >> 172.21.15.170:0/3883196947 conn(0x7fa4c80999f0 msgr2=0x7fa4c80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 -- 172.21.15.170:0/3883196947 shutdown_connections 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.588+0000 7fa4d0333700 1 -- 172.21.15.170:0/3883196947 wait complete. 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.589+0000 7fa4d0333700 1 Processor -- start 2023-10-08T16:13:13.594 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.589+0000 7fa4d0333700 1 -- start start 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c81529d0 0x7fa4c8152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 0x7fa4c814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c814eb30 con 0x7fa4b0003a90 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c814e940 con 0x7fa4c81529d0 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4d0333700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c815f5f0 con 0x7fa4c814df90 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53430/0 (socket says 172.21.15.170:53430) 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/466949923 learned_addr learned my addr 172.21.15.170:0/466949923 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c81529d0 0x7fa4c8152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce8d0700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 0x7fa4c814e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 msgr2=0x7fa4c814e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 0x7fa4c814e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c81529d0 msgr2=0x7fa4c8152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c81529d0 0x7fa4c8152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce0cf700 1 -- 172.21.15.170:0/466949923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c8154720 con 0x7fa4b0003a90 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.590+0000 7fa4ce8d0700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 0x7fa4c814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4ce0cf700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c8f0 tx=0x7fa4b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800cd00 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b8021070 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c81549a0 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8026680 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.591+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c8154f60 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.592+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c8063c90 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.592+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa4b8023030 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.592+0000 7fa4c67fc700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 0x7fa4b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.592+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa4b80868f0 con 0x7fa4b0003a90 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.592+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 0x7fa4b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.593+0000 7fa4cd8ce700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 0x7fa4b40697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa4c816d4b0 tx=0x7fa4bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:13.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.597+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4b80539a0 con 0x7fa4b0003a90 2023-10-08T16:13:13.779 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:13.778+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa4c80718b0 con 0x7fa4b0003a90 2023-10-08T16:13:14.519 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.519+0000 7fa4c67fc700 1 -- 172.21.15.170:0/466949923 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa4b8056fc0 con 0x7fa4b0003a90 2023-10-08T16:13:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.522+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 msgr2=0x7fa4b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.522+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 0x7fa4b40697d0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa4c816d4b0 tx=0x7fa4bc008040 comp rx=0 tx=0).stop 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.522+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 msgr2=0x7fa4c81516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.522+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c8f0 tx=0x7fa4b8016000 comp rx=0 tx=0).stop 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 shutdown_connections 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa4c814df90 0x7fa4c814e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa4b4067320 0x7fa4b40697d0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa4b0003a90 0x7fa4c81516c0 secure :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800c8f0 tx=0x7fa4b8016000 comp rx=0 tx=0).stop 2023-10-08T16:13:14.523 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 --2- 172.21.15.170:0/466949923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa4c81529d0 0x7fa4c8152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.523+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 >> 172.21.15.170:0/466949923 conn(0x7fa4c80999f0 msgr2=0x7fa4c80d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.524+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 shutdown_connections 2023-10-08T16:13:14.524 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.524+0000 7fa4d0333700 1 -- 172.21.15.170:0/466949923 wait complete. 2023-10-08T16:13:14.526 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:13:14.537 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:13:14.576 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:13:14.582 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:13:14.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.662+0000 7f34f0797700 1 Processor -- start 2023-10-08T16:13:14.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.662+0000 7f34f0797700 1 -- start start 2023-10-08T16:13:14.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34f0797700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f34e80cbb90 0x7f34e80af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:14.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34f0797700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e80af860 con 0x7f34e8159950 2023-10-08T16:13:14.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34f0797700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e80ce380 con 0x7f34e8069c70 2023-10-08T16:13:14.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34f0797700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e80cec20 con 0x7f34e80cbb90 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34eed34700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f34e80cbb90 0x7f34e80af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34ee533700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f34e8159950 0x7f34e8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43308/0 (socket says 172.21.15.170:43308) 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34ee533700 1 -- 172.21.15.170:0/4039354041 learned_addr learned my addr 172.21.15.170:0/4039354041 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34eed34700 1 -- 172.21.15.170:0/4039354041 >> v1:172.21.15.191:6789/0 conn(0x7f34e8069c70 legacy=0x7f34e80cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34eed34700 1 -- 172.21.15.170:0/4039354041 >> v1:172.21.15.170:6789/0 conn(0x7f34e8159950 legacy=0x7f34e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.663+0000 7f34eed34700 1 -- 172.21.15.170:0/4039354041 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e80cbfb0 con 0x7f34e80cbb90 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34eed34700 1 --2- 172.21.15.170:0/4039354041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e80cbb90 0x7f34e80af320 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009e60 tx=0x7f34e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a986d45effa8bb8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34ed531700 1 -- 172.21.15.170:0/4039354041 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e000e070 con 0x7f34e80cbb90 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34ed531700 1 -- 172.21.15.170:0/4039354041 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e000bbf0 con 0x7f34e80cbb90 2023-10-08T16:13:14.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34ed531700 1 -- 172.21.15.170:0/4039354041 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0012430 con 0x7f34e80cbb90 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 -- 172.21.15.170:0/4039354041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e80cbb90 msgr2=0x7f34e80af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 --2- 172.21.15.170:0/4039354041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e80cbb90 0x7f34e80af320 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009e60 tx=0x7f34e0009780 comp rx=0 tx=0).stop 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 -- 172.21.15.170:0/4039354041 shutdown_connections 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 --2- 172.21.15.170:0/4039354041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e80cbb90 0x7f34e80af320 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 -- 172.21.15.170:0/4039354041 >> 172.21.15.170:0/4039354041 conn(0x7f34e8068e00 msgr2=0x7f34e80d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:14.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 -- 172.21.15.170:0/4039354041 shutdown_connections 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.664+0000 7f34f0797700 1 -- 172.21.15.170:0/4039354041 wait complete. 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 Processor -- start 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 -- start start 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34e8069c70 0x7f34e80c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34e80cbb90 0x7f34e81523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e815b860 con 0x7f34e8159950 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e8152930 con 0x7f34e8069c70 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34f0797700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34e816e450 con 0x7f34e80cbb90 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34eed34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:14.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.666+0000 7f34eed34700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53442/0 (socket says 172.21.15.170:53442) 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 -- 172.21.15.170:0/3312172673 learned_addr learned my addr 172.21.15.170:0/3312172673 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34edd32700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34e80cbb90 0x7f34e81523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34ee533700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34e8069c70 0x7f34e80c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 -- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34e80cbb90 msgr2=0x7f34e81523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34e80cbb90 0x7f34e81523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 -- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34e8069c70 msgr2=0x7f34e80c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34e8069c70 0x7f34e80c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 -- 172.21.15.170:0/3312172673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0009a20 con 0x7f34e8159950 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34eed34700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f34e8070e20 tx=0x7f34e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0012e90 con 0x7f34e8159950 2023-10-08T16:13:14.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e001e640 con 0x7f34e8159950 2023-10-08T16:13:14.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e815aa10 con 0x7f34e8159950 2023-10-08T16:13:14.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e001ea00 con 0x7f34e8159950 2023-10-08T16:13:14.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.667+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e815afd0 con 0x7f34e8159950 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.668+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e80718b0 con 0x7f34e8159950 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.669+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f34e0024070 con 0x7f34e8159950 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.669+0000 7f34de7fc700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 0x7f34cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.669+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f34e0087850 con 0x7f34e8159950 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.669+0000 7f34ee533700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 0x7f34cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:14.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.670+0000 7f34ee533700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 0x7f34cc0697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f34e80c9470 tx=0x7f34e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:14.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.673+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34e00546c0 con 0x7f34e8159950 2023-10-08T16:13:14.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:14.856+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f34e80afa90 con 0x7f34e8159950 2023-10-08T16:13:15.522 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.521+0000 7f34de7fc700 1 -- 172.21.15.170:0/3312172673 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f34e002c0d0 con 0x7f34e8159950 2023-10-08T16:13:15.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.525+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 msgr2=0x7f34cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:15.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.525+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 0x7f34cc0697d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f34e80c9470 tx=0x7f34e4009040 comp rx=0 tx=0).stop 2023-10-08T16:13:15.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.525+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 msgr2=0x7f34e80ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:15.526 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.525+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f34e8070e20 tx=0x7f34e0018000 comp rx=0 tx=0).stop 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 shutdown_connections 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f34cc067320 0x7f34cc0697d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34e8159950 0x7f34e80ca550 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f34e8069c70 0x7f34e80c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 --2- 172.21.15.170:0/3312172673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f34e80cbb90 0x7f34e81523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.528 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.527+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 >> 172.21.15.170:0/3312172673 conn(0x7f34e8068e00 msgr2=0x7f34e80cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:15.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.528+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 shutdown_connections 2023-10-08T16:13:15.529 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.528+0000 7f34f0797700 1 -- 172.21.15.170:0/3312172673 wait complete. 2023-10-08T16:13:15.531 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:13:15.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:13:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.665+0000 7f5062b53700 1 Processor -- start 2023-10-08T16:13:15.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.665+0000 7f5062b53700 1 -- start start 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15df50 con 0x7f505c159c60 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c15a160 con 0x7f505c159890 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f5062b53700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0d13a0 con 0x7f505c15db10 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55802/0 (socket says 172.21.15.170:55802) 2023-10-08T16:13:15.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 -- 172.21.15.170:0/226601240 learned_addr learned my addr 172.21.15.170:0/226601240 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f50610f0700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f50608ef700 1 --2- 172.21.15.170:0/226601240 >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 -- 172.21.15.170:0/226601240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 msgr2=0x7f505c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 -- 172.21.15.170:0/226601240 >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 msgr2=0x7f505c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 --2- 172.21.15.170:0/226601240 >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 -- 172.21.15.170:0/226601240 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c15e140 con 0x7f505c159c60 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f50608ef700 1 --2- 172.21.15.170:0/226601240 >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f50610f0700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505bfff700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f505c0d1090 tx=0x7f5048009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5bc37ac5903d70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:15.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505b7fe700 1 -- 172.21.15.170:0/226601240 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800b860 con 0x7f505c159c60 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505b7fe700 1 -- 172.21.15.170:0/226601240 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504800b9c0 con 0x7f505c159c60 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.666+0000 7f505b7fe700 1 -- 172.21.15.170:0/226601240 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800bd60 con 0x7f505c159c60 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 -- 172.21.15.170:0/226601240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c159c60 msgr2=0x7f505c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f505c0d1090 tx=0x7f5048009250 comp rx=0 tx=0).stop 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 -- 172.21.15.170:0/226601240 shutdown_connections 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c15db10 0x7f505c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 --2- 172.21.15.170:0/226601240 >> v2:172.21.15.191:3300/0 conn(0x7f505c159890 0x7f505c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 --2- 172.21.15.170:0/226601240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c159c60 0x7f505c15d5d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 -- 172.21.15.170:0/226601240 >> 172.21.15.170:0/226601240 conn(0x7f505c068e20 msgr2=0x7f505c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 -- 172.21.15.170:0/226601240 shutdown_connections 2023-10-08T16:13:15.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.667+0000 7f5062b53700 1 -- 172.21.15.170:0/226601240 wait complete. 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.668+0000 7f5062b53700 1 Processor -- start 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.668+0000 7f5062b53700 1 -- start start 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c159890 0x7f505c0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f505c159c60 0x7f505c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c16d280 con 0x7f505c15db10 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c16d560 con 0x7f505c159c60 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f5062b53700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c16d840 con 0x7f505c159890 2023-10-08T16:13:15.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55810/0 (socket says 172.21.15.170:55810) 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 -- 172.21.15.170:0/963755030 learned_addr learned my addr 172.21.15.170:0/963755030 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50608ef700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c159890 0x7f505c0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f505bfff700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f505c159c60 0x7f505c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 -- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c159890 msgr2=0x7f505c0cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c159890 0x7f505c0cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 -- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f505c159c60 msgr2=0x7f505c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f505c159c60 0x7f505c1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 -- 172.21.15.170:0/963755030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50480094f0 con 0x7f505c15db10 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.669+0000 7f50610f0700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f505400d0d0 tx=0x7f50540099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:15.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505400dc10 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505400f040 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c16db20 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5054015440 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c158bf0 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.670+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c16e040 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.671+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f505401e3b0 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.671+0000 7f5058ff9700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 0x7f50400697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.672+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f50540850d0 con 0x7f505c15db10 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.672+0000 7f50608ef700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 0x7f50400697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:15.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.672+0000 7f50608ef700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 0x7f50400697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b65b0 tx=0x7f5050008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:15.676 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.676+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f505c16e040 con 0x7f505c15db10 2023-10-08T16:13:15.902 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:15.902+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f505c16e040 con 0x7f505c15db10 2023-10-08T16:13:16.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.554+0000 7f5058ff9700 1 -- 172.21.15.170:0/963755030 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 31 and data pool 32 v116) v1 ==== 174+0+0 (secure 0 0 0) 0x7f505c16e040 con 0x7f505c15db10 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 msgr2=0x7f50400697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 0x7f50400697d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b65b0 tx=0x7f5050008040 comp rx=0 tx=0).stop 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 msgr2=0x7f505c16c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f505400d0d0 tx=0x7f50540099d0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 shutdown_connections 2023-10-08T16:13:16.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5040067320 0x7f50400697d0 secure :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f505c0b65b0 tx=0x7f5050008040 comp rx=0 tx=0).stop 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f505c15db10 0x7f505c16c5d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f505c159890 0x7f505c0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 --2- 172.21.15.170:0/963755030 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f505c159c60 0x7f505c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 >> 172.21.15.170:0/963755030 conn(0x7f505c068e20 msgr2=0x7f505c0c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 shutdown_connections 2023-10-08T16:13:16.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.566+0000 7f5062b53700 1 -- 172.21.15.170:0/963755030 wait complete. 2023-10-08T16:13:16.568 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 31 and data pool 32 2023-10-08T16:13:16.580 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 Processor -- start 2023-10-08T16:13:16.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 -- start start 2023-10-08T16:13:16.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f040aca80 0x7f4f0406aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4f04069d60 0x7f4f040c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0406bdb0 con 0x7f4f040c81f0 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f040cb370 con 0x7f4f040aca80 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.695+0000 7f4f093fe700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f04159f60 con 0x7f4f04069d60 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f040aca80 0x7f4f0406aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f037fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4f040c81f0 0x7f4f04068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40330/0 (socket says 172.21.15.170:40330) 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f037fe700 1 -- 172.21.15.170:0/4126960811 learned_addr learned my addr 172.21.15.170:0/4126960811 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4126960811 >> v2:172.21.15.191:3300/0 conn(0x7f4f04069d60 msgr2=0x7f4f040c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 --2- 172.21.15.170:0/4126960811 >> v2:172.21.15.191:3300/0 conn(0x7f4f04069d60 0x7f4f040c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4126960811 >> v1:172.21.15.170:6789/0 conn(0x7f4f040c81f0 legacy=0x7f4f04068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4126960811 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f040c8610 con 0x7f4f040aca80 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f01ffb700 1 -- 172.21.15.170:0/4126960811 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810139298 0 0) 0x7f4f0406bdb0 con 0x7f4f040c81f0 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f02ffd700 1 --2- 172.21.15.170:0/4126960811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f040aca80 0x7f4f0406aba0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4f0406e540 tx=0x7f4ef4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37dc94d4062a4cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f01ffb700 1 -- 172.21.15.170:0/4126960811 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400e070 con 0x7f4f040aca80 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f01ffb700 1 -- 172.21.15.170:0/4126960811 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef400bc30 con 0x7f4f040aca80 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.696+0000 7f4f01ffb700 1 -- 172.21.15.170:0/4126960811 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4012430 con 0x7f4f040aca80 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 -- 172.21.15.170:0/4126960811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f040aca80 msgr2=0x7f4f0406aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4126960811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f040aca80 0x7f4f0406aba0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4f0406e540 tx=0x7f4ef4009780 comp rx=0 tx=0).stop 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 -- 172.21.15.170:0/4126960811 shutdown_connections 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4126960811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f040aca80 0x7f4f0406aba0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4126960811 >> v2:172.21.15.191:3300/0 conn(0x7f4f04069d60 0x7f4f040c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 -- 172.21.15.170:0/4126960811 >> 172.21.15.170:0/4126960811 conn(0x7f4f040cd670 msgr2=0x7f4f040cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:16.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 -- 172.21.15.170:0/4126960811 shutdown_connections 2023-10-08T16:13:16.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.697+0000 7f4f093fe700 1 -- 172.21.15.170:0/4126960811 wait complete. 2023-10-08T16:13:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.698+0000 7f4f093fe700 1 Processor -- start 2023-10-08T16:13:16.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.698+0000 7f4f093fe700 1 -- start start 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f040aca80 0x7f4f0416a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f040c81f0 0x7f4f040acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04157ed0 con 0x7f4f04069d60 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040b2450 con 0x7f4f040aca80 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0416ad80 con 0x7f4f040c81f0 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55818/0 (socket says 172.21.15.170:55818) 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4182977731 learned_addr learned my addr 172.21.15.170:0/4182977731 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:16.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f040c81f0 msgr2=0x7f4f040acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f040c81f0 0x7f4f040acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f027fc700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f040aca80 0x7f4f0416a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f040aca80 msgr2=0x7f4f0416a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f040aca80 0x7f4f0416a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 -- 172.21.15.170:0/4182977731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef4009a20 con 0x7f4f04069d60 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f02ffd700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4f0406c6a0 tx=0x7f4ef4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4012e20 con 0x7f4f04069d60 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef401d630 con 0x7f4f04069d60 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f040b0920 con 0x7f4f04069d60 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef401d9f0 con 0x7f4f04069d60 2023-10-08T16:13:16.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.699+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f040afb70 con 0x7f4f04069d60 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.700+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0404c3c0 con 0x7f4f04069d60 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.701+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4ef4024070 con 0x7f4f04069d60 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.701+0000 7f4eeaffd700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 0x7f4ef00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.701+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4ef4087370 con 0x7f4f04069d60 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.701+0000 7f4f027fc700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 0x7f4ef00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:16.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.702+0000 7f4f027fc700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 0x7f4ef00697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006fd0 tx=0x7f4ef8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:16.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.705+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ef4054000 con 0x7f4f04069d60 2023-10-08T16:13:16.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.888+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f0406b510 con 0x7f4f04069d60 2023-10-08T16:13:16.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.889+0000 7f4eeaffd700 1 -- 172.21.15.170:0/4182977731 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f4ef4057620 con 0x7f4f04069d60 2023-10-08T16:13:16.889 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:16.890 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":183,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:15.903162+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:14.874207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 msgr2=0x7f4ef00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 0x7f4ef00697d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8006fd0 tx=0x7f4ef8008040 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 msgr2=0x7f4f04157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f4f0406c6a0 tx=0x7f4ef4019000 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 shutdown_connections 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f040c81f0 0x7f4f040acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4ef0067320 0x7f4ef00697d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f04069d60 0x7f4f04157990 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 --2- 172.21.15.170:0/4182977731 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f040aca80 0x7f4f0416a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 >> 172.21.15.170:0/4182977731 conn(0x7f4f040cd670 msgr2=0x7f4f040c2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 shutdown_connections 2023-10-08T16:13:16.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:16.891+0000 7f4f093fe700 1 -- 172.21.15.170:0/4182977731 wait complete. 2023-10-08T16:13:16.904 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 Processor -- start 2023-10-08T16:13:17.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 -- start start 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f66b0159890 0x7f66b015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b015a160 con 0x7f66b015db10 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b00d13a0 con 0x7f66b0159890 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.018+0000 7f66b57e1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b015e140 con 0x7f66b0159c60 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66ae7fc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f66b0159890 0x7f66b015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66af7fe700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f66b015db10 0x7f66b0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:38554/0 (socket says 172.21.15.170:38554) 2023-10-08T16:13:17.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66af7fe700 1 -- 172.21.15.170:0/55483411 learned_addr learned my addr 172.21.15.170:0/55483411 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 -- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 msgr2=0x7f66b015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 -- 172.21.15.170:0/55483411 >> v1:172.21.15.191:6789/0 conn(0x7f66b015db10 legacy=0x7f66b0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 -- 172.21.15.170:0/55483411 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b015a3d0 con 0x7f66b0159890 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66ae7fc700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66aeffd700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b0159890 0x7f66b015abd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009e60 tx=0x7f66a0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acec0330c201214 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.019+0000 7f66adffb700 1 -- 172.21.15.170:0/55483411 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a000e070 con 0x7f66b0159890 2023-10-08T16:13:17.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66adffb700 1 -- 172.21.15.170:0/55483411 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a000bb60 con 0x7f66b0159890 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66adffb700 1 -- 172.21.15.170:0/55483411 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0012430 con 0x7f66b0159890 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 -- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b0159890 msgr2=0x7f66b015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b0159890 0x7f66b015abd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009e60 tx=0x7f66a0009780 comp rx=0 tx=0).stop 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 -- 172.21.15.170:0/55483411 shutdown_connections 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b0159890 0x7f66b015abd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 --2- 172.21.15.170:0/55483411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 -- 172.21.15.170:0/55483411 >> 172.21.15.170:0/55483411 conn(0x7f66b0068e20 msgr2=0x7f66b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 -- 172.21.15.170:0/55483411 shutdown_connections 2023-10-08T16:13:17.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.020+0000 7f66b57e1700 1 -- 172.21.15.170:0/55483411 wait complete. 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.021+0000 7f66b57e1700 1 Processor -- start 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.021+0000 7f66b57e1700 1 -- start start 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b00b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b015db10 0x7f66b00b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00b3590 con 0x7f66b0159890 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00c79b0 con 0x7f66b015db10 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66b57e1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00b43b0 con 0x7f66b0159c60 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55822/0 (socket says 172.21.15.170:55822) 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 -- 172.21.15.170:0/191300487 learned_addr learned my addr 172.21.15.170:0/191300487 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 -- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 msgr2=0x7f66b00b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b00b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 -- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b015db10 msgr2=0x7f66b00b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b015db10 0x7f66b00b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.022+0000 7f66aeffd700 1 -- 172.21.15.170:0/191300487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009420 con 0x7f66b0159890 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66aeffd700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0012da0 con 0x7f66b0159890 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0027470 con 0x7f66b0159890 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a0009a20 con 0x7f66b0159890 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0027830 con 0x7f66b0159890 2023-10-08T16:13:17.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.023+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b00b37a0 con 0x7f66b0159890 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.024+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b01590e0 con 0x7f66b0159890 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.025+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f66a0024070 con 0x7f66b0159890 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.025+0000 7f66a6ffd700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 0x7f6690069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.025+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f66a0087600 con 0x7f66b0159890 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.025+0000 7f66ae7fc700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 0x7f6690069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.026+0000 7f66ae7fc700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 0x7f6690069850 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f66b015b7b0 tx=0x7f66980091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.028+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f66a00542d0 con 0x7f66b0159890 2023-10-08T16:13:17.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.212+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66b004c3c0 con 0x7f66b0159890 2023-10-08T16:13:17.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.212+0000 7f66a6ffd700 1 -- 172.21.15.170:0/191300487 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f66a00578f0 con 0x7f66b0159890 2023-10-08T16:13:17.213 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:17.213 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":183,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:15.903162+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:14.874207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 msgr2=0x7f6690069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 0x7f6690069850 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f66b015b7b0 tx=0x7f66980091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 msgr2=0x7f66b016e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097c0 tx=0x7f66a0019000 comp rx=0 tx=0).stop 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 shutdown_connections 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f66b015db10 0x7f66b00b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f66900673a0 0x7f6690069850 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66b0159890 0x7f66b016e2e0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 --2- 172.21.15.170:0/191300487 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f66b0159c60 0x7f66b00b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 >> 172.21.15.170:0/191300487 conn(0x7f66b0068e20 msgr2=0x7f66b015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 shutdown_connections 2023-10-08T16:13:17.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.215+0000 7f66b57e1700 1 -- 172.21.15.170:0/191300487 wait complete. 2023-10-08T16:13:17.227 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:13:17.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.345+0000 7faeb5c0d700 1 Processor -- start 2023-10-08T16:13:17.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.345+0000 7faeb5c0d700 1 -- start start 2023-10-08T16:13:17.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeb5c0d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7faeb0159890 0x7faeb015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeb5c0d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeb0159c60 0x7faeb015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeb5c0d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb015a160 con 0x7faeb015db10 2023-10-08T16:13:17.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeb5c0d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb015df50 con 0x7faeb0159c60 2023-10-08T16:13:17.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeb5c0d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb00d13a0 con 0x7faeb0159890 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeaeffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeb0159c60 0x7faeb015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeaffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7faeb015db10 0x7faeb0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40344/0 (socket says 172.21.15.170:40344) 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.346+0000 7faeaffff700 1 -- 172.21.15.170:0/1494664670 learned_addr learned my addr 172.21.15.170:0/1494664670 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaf7fe700 1 --2- 172.21.15.170:0/1494664670 >> v2:172.21.15.191:3300/0 conn(0x7faeb0159890 0x7faeb015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaeffd700 1 -- 172.21.15.170:0/1494664670 >> v2:172.21.15.191:3300/0 conn(0x7faeb0159890 msgr2=0x7faeb015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaeffd700 1 --2- 172.21.15.170:0/1494664670 >> v2:172.21.15.191:3300/0 conn(0x7faeb0159890 0x7faeb015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaeffd700 1 -- 172.21.15.170:0/1494664670 >> v1:172.21.15.170:6789/0 conn(0x7faeb015db10 legacy=0x7faeb0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaeffd700 1 -- 172.21.15.170:0/1494664670 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb015e140 con 0x7faeb0159c60 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeaeffd700 1 --2- 172.21.15.170:0/1494664670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb015d5d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7faeb00d1090 tx=0x7faea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656fb5237bfd370e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeae7fc700 1 -- 172.21.15.170:0/1494664670 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400b6b0 con 0x7faeb0159c60 2023-10-08T16:13:17.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeae7fc700 1 -- 172.21.15.170:0/1494664670 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400b810 con 0x7faeb0159c60 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.347+0000 7faeae7fc700 1 -- 172.21.15.170:0/1494664670 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400bbb0 con 0x7faeb0159c60 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1494664670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 msgr2=0x7faeb015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1494664670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb015d5d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7faeb00d1090 tx=0x7faea40091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1494664670 shutdown_connections 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1494664670 >> v2:172.21.15.191:3300/0 conn(0x7faeb0159890 0x7faeb015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1494664670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb015d5d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1494664670 >> 172.21.15.170:0/1494664670 conn(0x7faeb0068e20 msgr2=0x7faeb00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1494664670 shutdown_connections 2023-10-08T16:13:17.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.348+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1494664670 wait complete. 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.349+0000 7faeb5c0d700 1 Processor -- start 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.349+0000 7faeb5c0d700 1 -- start start 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb0159890 0x7faeb00b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb015db10 0x7faeb00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb016ba10 con 0x7faeb0159c60 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb01478c0 con 0x7faeb0159890 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb016ae00 con 0x7faeb015db10 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55844/0 (socket says 172.21.15.170:55844) 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 -- 172.21.15.170:0/1607252184 learned_addr learned my addr 172.21.15.170:0/1607252184 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaf7fe700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb0159890 0x7faeb00b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaffff700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb015db10 0x7faeb00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 -- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb015db10 msgr2=0x7faeb00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb015db10 0x7faeb00cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 -- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb0159890 msgr2=0x7faeb00b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb0159890 0x7faeb00b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 -- 172.21.15.170:0/1607252184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4009420 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeaeffd700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faeb0070e20 tx=0x7faea4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4011420 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4011580 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb016b0e0 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4011940 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.350+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb016b6a0 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.351+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb004c3c0 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.352+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faea4011d30 con 0x7faeb0159c60 2023-10-08T16:13:17.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.352+0000 7fae977fe700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 0x7fae9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.352+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faea40861a0 con 0x7faeb0159c60 2023-10-08T16:13:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.352+0000 7faeaf7fe700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 0x7fae9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.353+0000 7faeaf7fe700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 0x7fae9c0697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faea0006fd0 tx=0x7faea0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:17.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.356+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faea4052e30 con 0x7faeb0159c60 2023-10-08T16:13:17.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:17.587+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7faeb016bb50 con 0x7faeb0159c60 2023-10-08T16:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.563+0000 7fae977fe700 1 -- 172.21.15.170:0/1607252184 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v119) v1 ==== 118+0+0 (secure 0 0 0) 0x7faea4056450 con 0x7faeb0159c60 2023-10-08T16:13:18.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 msgr2=0x7fae9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 0x7fae9c0697d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7faea0006fd0 tx=0x7faea0008040 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 msgr2=0x7faeb0147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7faeb0070e20 tx=0x7faea4017000 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 shutdown_connections 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeb015db10 0x7faeb00cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeb0159890 0x7faeb00b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae9c067320 0x7fae9c0697d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 --2- 172.21.15.170:0/1607252184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeb0159c60 0x7faeb0147380 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 >> 172.21.15.170:0/1607252184 conn(0x7faeb0068e20 msgr2=0x7faeb0069530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 shutdown_connections 2023-10-08T16:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.565+0000 7faeb5c0d700 1 -- 172.21.15.170:0/1607252184 wait complete. 2023-10-08T16:13:18.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.695+0000 7f36b06d3700 1 Processor -- start 2023-10-08T16:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.695+0000 7f36b06d3700 1 -- start start 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36b06d3700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f36a8159890 0x7f36a815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36b06d3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8069560 0x7f36a80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36b06d3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a8069ee0 con 0x7f36a8159c60 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36b06d3700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a815a160 con 0x7f36a8159890 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36b06d3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80d13a0 con 0x7f36a8069560 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f36a8159890 0x7f36a815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36adc6e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f36a8159c60 0x7f36a815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40350/0 (socket says 172.21.15.170:40350) 2023-10-08T16:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36adc6e700 1 -- 172.21.15.170:0/427970346 learned_addr learned my addr 172.21.15.170:0/427970346 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36aec70700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8069560 0x7f36a80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ad46d700 1 -- 172.21.15.170:0/427970346 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820063060 0 0) 0x7f36a8069ee0 con 0x7f36a8159c60 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ad46d700 1 -- 172.21.15.170:0/427970346 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3694003570 con 0x7f36a8159c60 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 -- 172.21.15.170:0/427970346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8069560 msgr2=0x7f36a80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8069560 0x7f36a80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 -- 172.21.15.170:0/427970346 >> v1:172.21.15.170:6789/0 conn(0x7f36a8159c60 legacy=0x7f36a815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 -- 172.21.15.170:0/427970346 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a815e140 con 0x7f36a8159890 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.696+0000 7f36ae46f700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8159890 0x7f36a815abd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009e60 tx=0x7f36a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee6f79b7f7a8393 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36ad46d700 1 -- 172.21.15.170:0/427970346 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a000e070 con 0x7f36a8159890 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36ad46d700 1 -- 172.21.15.170:0/427970346 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a000bb60 con 0x7f36a8159890 2023-10-08T16:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36ad46d700 1 -- 172.21.15.170:0/427970346 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0012430 con 0x7f36a8159890 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 -- 172.21.15.170:0/427970346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8159890 msgr2=0x7f36a815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8159890 0x7f36a815abd0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009e60 tx=0x7f36a0009780 comp rx=0 tx=0).stop 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 -- 172.21.15.170:0/427970346 shutdown_connections 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8069560 0x7f36a80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 --2- 172.21.15.170:0/427970346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8159890 0x7f36a815abd0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 -- 172.21.15.170:0/427970346 >> 172.21.15.170:0/427970346 conn(0x7f36a8068e20 msgr2=0x7f36a80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 -- 172.21.15.170:0/427970346 shutdown_connections 2023-10-08T16:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.697+0000 7f36b06d3700 1 -- 172.21.15.170:0/427970346 wait complete. 2023-10-08T16:13:18.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.698+0000 7f36b06d3700 1 Processor -- start 2023-10-08T16:13:18.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.698+0000 7f36b06d3700 1 -- start start 2023-10-08T16:13:18.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 0x7f36a80c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 0x7f36a80c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8156c90 con 0x7f36a8069560 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80c8320 con 0x7f36a8159890 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36b06d3700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80cdaa0 con 0x7f36a8159c60 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55856/0 (socket says 172.21.15.170:55856) 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 -- 172.21.15.170:0/1290155217 learned_addr learned my addr 172.21.15.170:0/1290155217 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:18.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36aec70700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 0x7f36a80c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36adc6e700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 0x7f36a80c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 -- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 msgr2=0x7f36a80c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 0x7f36a80c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 -- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 msgr2=0x7f36a80c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 0x7f36a80c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36ae46f700 1 -- 172.21.15.170:0/1290155217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a0009a20 con 0x7f36a8069560 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36adc6e700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 0x7f36a80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.699+0000 7f36aec70700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 0x7f36a80c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f36ae46f700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097c0 tx=0x7f36a0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0012da0 con 0x7f36a8069560 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a0027470 con 0x7f36a8069560 2023-10-08T16:13:18.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a80cd7b0 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a0027830 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.700+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a80caa00 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.701+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a804c3c0 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.701+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f36a0024070 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.701+0000 7f369e7fc700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 0x7f368c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.701+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f36a0087310 con 0x7f36a8069560 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.701+0000 7f36adc6e700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 0x7f368c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:18.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.702+0000 7f36adc6e700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 0x7f368c0697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f36980058d0 tx=0x7f3698009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:18.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.706+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36a0053fa0 con 0x7f36a8069560 2023-10-08T16:13:18.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:18.932+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f36a80b0120 con 0x7f36a8069560 2023-10-08T16:13:19.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.563+0000 7f369e7fc700 1 -- 172.21.15.170:0/1290155217 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v120) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36a00575c0 con 0x7f36a8069560 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 msgr2=0x7f368c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 0x7f368c0697d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f36980058d0 tx=0x7f3698009040 comp rx=0 tx=0).stop 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 msgr2=0x7f36a8143d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f36a00097c0 tx=0x7f36a0019000 comp rx=0 tx=0).stop 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 shutdown_connections 2023-10-08T16:13:19.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f36a8159c60 0x7f36a80c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f368c067320 0x7f368c0697d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8069560 0x7f36a8143d90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 --2- 172.21.15.170:0/1290155217 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f36a8159890 0x7f36a80c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 >> 172.21.15.170:0/1290155217 conn(0x7f36a8068e20 msgr2=0x7f36a81586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 shutdown_connections 2023-10-08T16:13:19.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.565+0000 7f36b06d3700 1 -- 172.21.15.170:0/1290155217 wait complete. 2023-10-08T16:13:19.580 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:19.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.696+0000 7fd500a29700 1 Processor -- start 2023-10-08T16:13:19.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.696+0000 7fd500a29700 1 -- start start 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd500a29700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd4fc1597f0 0x7fd4fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd500a29700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0b11d0 con 0x7fd4fc159420 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd500a29700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4fc0b14b0 con 0x7fd4fc15d230 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd500a29700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4fc0b1790 con 0x7fd4fc1597f0 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4faffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd4fc1597f0 0x7fd4fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4fb7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd4fc159420 0x7fd4fc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40366/0 (socket says 172.21.15.170:40366) 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4fb7fe700 1 -- 172.21.15.170:0/3338443724 learned_addr learned my addr 172.21.15.170:0/3338443724 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:19.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4faffd700 1 -- 172.21.15.170:0/3338443724 >> v1:172.21.15.191:6789/0 conn(0x7fd4fc15d230 legacy=0x7fd4fc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4faffd700 1 -- 172.21.15.170:0/3338443724 >> v1:172.21.15.170:6789/0 conn(0x7fd4fc159420 legacy=0x7fd4fc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4faffd700 1 -- 172.21.15.170:0/3338443724 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4fc0b1a70 con 0x7fd4fc1597f0 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.697+0000 7fd4fa7fc700 1 -- 172.21.15.170:0/3338443724 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6604226 0 0) 0x7fd4fc0b11d0 con 0x7fd4fc159420 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd4faffd700 1 --2- 172.21.15.170:0/3338443724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc0b2270 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00098e0 tx=0x7fd4f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e147ea22ede09d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd4fa7fc700 1 -- 172.21.15.170:0/3338443724 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f0009e80 con 0x7fd4fc1597f0 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd4fa7fc700 1 -- 172.21.15.170:0/3338443724 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f000b820 con 0x7fd4fc1597f0 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd4fa7fc700 1 -- 172.21.15.170:0/3338443724 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000bba0 con 0x7fd4fc1597f0 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 -- 172.21.15.170:0/3338443724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 msgr2=0x7fd4fc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 --2- 172.21.15.170:0/3338443724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc0b2270 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00098e0 tx=0x7fd4f00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 -- 172.21.15.170:0/3338443724 shutdown_connections 2023-10-08T16:13:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 --2- 172.21.15.170:0/3338443724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc0b2270 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 -- 172.21.15.170:0/3338443724 >> 172.21.15.170:0/3338443724 conn(0x7fd4fc0d0660 msgr2=0x7fd4fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 -- 172.21.15.170:0/3338443724 shutdown_connections 2023-10-08T16:13:19.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.698+0000 7fd500a29700 1 -- 172.21.15.170:0/3338443724 wait complete. 2023-10-08T16:13:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.699+0000 7fd500a29700 1 Processor -- start 2023-10-08T16:13:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.699+0000 7fd500a29700 1 -- start start 2023-10-08T16:13:19.704 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 0x7fd4fc15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc15d230 0x7fd4fc155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0ca2e0 con 0x7fd4fc1597f0 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc15bbc0 con 0x7fd4fc15d230 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd500a29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4fc0c5400 con 0x7fd4fc159420 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd4faffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55874/0 (socket says 172.21.15.170:55874) 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd4faffd700 1 -- 172.21.15.170:0/2062454875 learned_addr learned my addr 172.21.15.170:0/2062454875 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd4fbfff700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc15d230 0x7fd4fc155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.700+0000 7fd4fb7fe700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 0x7fd4fc15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:19.705 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 -- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 msgr2=0x7fd4fc15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 0x7fd4fc15b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 -- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc15d230 msgr2=0x7fd4fc155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc15d230 0x7fd4fc155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 -- 172.21.15.170:0/2062454875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0009420 con 0x7fd4fc1597f0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4fb7fe700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 0x7fd4fc15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4faffd700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00030c0 tx=0x7fd4f0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000b990 con 0x7fd4fc1597f0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0020070 con 0x7fd4fc1597f0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4fc0c96d0 con 0x7fd4fc1597f0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f001b450 con 0x7fd4fc1597f0 2023-10-08T16:13:19.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.701+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4fc0c9c90 con 0x7fd4fc1597f0 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.702+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4fc0ca450 con 0x7fd4fc1597f0 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.702+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd4f0022030 con 0x7fd4fc1597f0 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.703+0000 7fd4e37fe700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 0x7fd4e80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.703+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd4f0086b60 con 0x7fd4fc1597f0 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.703+0000 7fd4fb7fe700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 0x7fd4e80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:19.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.703+0000 7fd4fb7fe700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 0x7fd4e80697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec001560 tx=0x7fd4ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:19.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.707+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4f0053900 con 0x7fd4fc1597f0 2023-10-08T16:13:19.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.929+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4fc0ca420 con 0x7fd4fc1597f0 2023-10-08T16:13:19.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.930+0000 7fd4e37fe700 1 -- 172.21.15.170:0/2062454875 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fd4f0056f20 con 0x7fd4fc1597f0 2023-10-08T16:13:19.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:19.934 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":120,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:19.562557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 msgr2=0x7fd4e80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 0x7fd4e80697d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec001560 tx=0x7fd4ec009040 comp rx=0 tx=0).stop 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 msgr2=0x7fd4fc154150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fd4f00030c0 tx=0x7fd4f0016000 comp rx=0 tx=0).stop 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 shutdown_connections 2023-10-08T16:13:19.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd4fc15d230 0x7fd4fc155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd4fc159420 0x7fd4fc15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd4e8067320 0x7fd4e80697d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 --2- 172.21.15.170:0/2062454875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4fc1597f0 0x7fd4fc154150 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 >> 172.21.15.170:0/2062454875 conn(0x7fd4fc0d0660 msgr2=0x7fd4fc069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 shutdown_connections 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:19.932+0000 7fd500a29700 1 -- 172.21.15.170:0/2062454875 wait complete. 2023-10-08T16:13:19.936 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 120 2023-10-08T16:13:19.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:20.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.059+0000 7f0c7ffff700 1 Processor -- start 2023-10-08T16:13:20.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.059+0000 7f0c7ffff700 1 -- start start 2023-10-08T16:13:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7ffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800b11d0 con 0x7f0c80159420 2023-10-08T16:13:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7ffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800b14b0 con 0x7f0c8015d230 2023-10-08T16:13:20.064 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b1790 con 0x7f0c801597f0 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0c80159420 0x7f0c800b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40376/0 (socket says 172.21.15.170:40376) 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7effd700 1 -- 172.21.15.170:0/3474489959 learned_addr learned my addr 172.21.15.170:0/3474489959 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462067404 0 0) 0x7f0c800b11d0 con 0x7f0c80159420 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c68003570 con 0x7f0c80159420 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.060+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4222515705 0 0) 0x7f0c68003570 con 0x7f0c80159420 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 msgr2=0x7f0c800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 >> v1:172.21.15.191:6789/0 conn(0x7f0c8015d230 legacy=0x7f0c800b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800b1a70 con 0x7f0c80159420 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f0c700017f0 con 0x7f0c80159420 2023-10-08T16:13:20.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 >> v1:172.21.15.170:6789/0 conn(0x7f0c80159420 legacy=0x7f0c800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68003a90 0x7f0c68003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68004d50 0x7f0c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b1790 con 0x7f0c68003a90 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6800bb70 con 0x7f0c68004d50 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6800bd30 con 0x7f0c801597f0 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68003a90 0x7f0c68003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7f7fe700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68004d50 0x7f0c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 -- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 msgr2=0x7f0c68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:20.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 -- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68004d50 msgr2=0x7f0c680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68004d50 0x7f0c680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 -- 172.21.15.170:0/3474489959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c68003570 con 0x7f0c68003a90 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c801597f0 0x7f0c68004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7effd700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68003a90 0x7f0c68003ea0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000a930 tx=0x7f0c700021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc3f6574db76db5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700070d0 con 0x7f0c68003a90 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70007230 con 0x7f0c68003a90 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.061+0000 7f0c7dffb700 1 -- 172.21.15.170:0/3474489959 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700075d0 con 0x7f0c68003a90 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 -- 172.21.15.170:0/3474489959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68003a90 msgr2=0x7f0c68003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/3474489959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68003a90 0x7f0c68003ea0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000a930 tx=0x7f0c700021d0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7f7fe700 1 -- 172.21.15.170:0/3474489959 reap_dead start 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 -- 172.21.15.170:0/3474489959 shutdown_connections 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 -- 172.21.15.170:0/3474489959 >> 172.21.15.170:0/3474489959 conn(0x7f0c800d0660 msgr2=0x7f0c800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 -- 172.21.15.170:0/3474489959 shutdown_connections 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.062+0000 7f0c7ffff700 1 -- 172.21.15.170:0/3474489959 wait complete. 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.063+0000 7f0c7ffff700 1 Processor -- start 2023-10-08T16:13:20.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.063+0000 7f0c7ffff700 1 -- start start 2023-10-08T16:13:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68003a90 0x7f0c8016b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c800c7540 0x7f0c800d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.070 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c5530 con 0x7f0c80155260 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c7a50 con 0x7f0c68003a90 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800c7d30 con 0x7f0c800c7540 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55884/0 (socket says 172.21.15.170:55884) 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 -- 172.21.15.170:0/1431913797 learned_addr learned my addr 172.21.15.170:0/1431913797 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7effd700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68003a90 0x7f0c8016b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7f7fe700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c800c7540 0x7f0c800d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 -- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c800c7540 msgr2=0x7f0c800d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c800c7540 0x7f0c800d0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 -- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68003a90 msgr2=0x7f0c8016b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68003a90 0x7f0c8016b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 -- 172.21.15.170:0/1431913797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800c8010 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7e7fc700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0c740118e0 tx=0x7f0c74008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74011e80 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7400f8e0 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c800c82f0 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400fc80 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.064+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c801556d0 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.065+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c800c56a0 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.066+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0c74015460 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.066+0000 7f0c66ffd700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 0x7f0c6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.066+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0c74085680 con 0x7f0c80155260 2023-10-08T16:13:20.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.066+0000 7f0c7effd700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 0x7f0c6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.067+0000 7f0c7effd700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 0x7f0c6c0697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000a3f0 tx=0x7f0c70016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.070+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c74052160 con 0x7f0c80155260 2023-10-08T16:13:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.253+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c8004c3c0 con 0x7f0c80155260 2023-10-08T16:13:20.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.254+0000 7f0c66ffd700 1 -- 172.21.15.170:0/1431913797 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+15412 (secure 0 0 0) 0x7f0c74055780 con 0x7f0c80155260 2023-10-08T16:13:20.256 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:20.256 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":184,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:17.535994+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":32,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:14.874207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 msgr2=0x7f0c6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 0x7f0c6c0697d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000a3f0 tx=0x7f0c70016000 comp rx=0 tx=0).stop 2023-10-08T16:13:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 msgr2=0x7f0c800c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f0c740118e0 tx=0x7f0c74008040 comp rx=0 tx=0).stop 2023-10-08T16:13:20.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 shutdown_connections 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0c800c7540 0x7f0c800d0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0c68003a90 0x7f0c8016b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0c6c067320 0x7f0c6c0697d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 --2- 172.21.15.170:0/1431913797 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c80155260 0x7f0c800c7000 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 >> 172.21.15.170:0/1431913797 conn(0x7f0c800d0660 msgr2=0x7f0c80069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 shutdown_connections 2023-10-08T16:13:20.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.256+0000 7f0c7ffff700 1 -- 172.21.15.170:0/1431913797 wait complete. 2023-10-08T16:13:20.270 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:13:20.270 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:13:20.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 Processor -- start 2023-10-08T16:13:20.384 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 -- start start 2023-10-08T16:13:20.386 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d70159c60 0x7f1d7015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7015a160 con 0x7f1d7015db10 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700d13a0 con 0x7f1d70159890 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.383+0000 7f1d74b24700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7015df50 con 0x7f1d70159c60 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1d7015db10 0x7f1d70069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40382/0 (socket says 172.21.15.170:40382) 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1d70159c60 0x7f1d7015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6ed9d700 1 -- 172.21.15.170:0/3191452995 learned_addr learned my addr 172.21.15.170:0/3191452995 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6dd9b700 1 -- 172.21.15.170:0/3191452995 >> v1:172.21.15.191:6789/0 conn(0x7f1d70159890 legacy=0x7f1d7015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6dd9b700 1 -- 172.21.15.170:0/3191452995 >> v1:172.21.15.170:6789/0 conn(0x7f1d7015db10 legacy=0x7f1d70069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6dd9b700 1 -- 172.21.15.170:0/3191452995 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7015e140 con 0x7f1d70159c60 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6d59a700 1 -- 172.21.15.170:0/3191452995 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806966625 0 0) 0x7f1d7015a160 con 0x7f1d7015db10 2023-10-08T16:13:20.387 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6dd9b700 1 --2- 172.21.15.170:0/3191452995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159c60 0x7f1d7015d5d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098e0 tx=0x7f1d580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65d4884731692a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6d59a700 1 -- 172.21.15.170:0/3191452995 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d58009e80 con 0x7f1d70159c60 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6d59a700 1 -- 172.21.15.170:0/3191452995 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5800b820 con 0x7f1d70159c60 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.384+0000 7f1d6d59a700 1 -- 172.21.15.170:0/3191452995 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5800bba0 con 0x7f1d70159c60 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 -- 172.21.15.170:0/3191452995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159c60 msgr2=0x7f1d7015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 --2- 172.21.15.170:0/3191452995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159c60 0x7f1d7015d5d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1d580098e0 tx=0x7f1d580091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 -- 172.21.15.170:0/3191452995 shutdown_connections 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 --2- 172.21.15.170:0/3191452995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159c60 0x7f1d7015d5d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 -- 172.21.15.170:0/3191452995 >> 172.21.15.170:0/3191452995 conn(0x7f1d70068e20 msgr2=0x7f1d700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 -- 172.21.15.170:0/3191452995 shutdown_connections 2023-10-08T16:13:20.388 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.385+0000 7f1d74b24700 1 -- 172.21.15.170:0/3191452995 wait complete. 2023-10-08T16:13:20.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.386+0000 7f1d74b24700 1 Processor -- start 2023-10-08T16:13:20.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.386+0000 7f1d74b24700 1 -- start start 2023-10-08T16:13:20.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d70159c60 0x7f1d70143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 0x7f1d70144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d701529e0 con 0x7f1d70159890 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700b1100 con 0x7f1d7015db10 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d700b13a0 con 0x7f1d70159c60 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55904/0 (socket says 172.21.15.170:55904) 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 -- 172.21.15.170:0/2626799710 learned_addr learned my addr 172.21.15.170:0/2626799710 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6ed9d700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 0x7f1d70144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6dd9b700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d70159c60 0x7f1d70143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 -- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d70159c60 msgr2=0x7f1d70143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.392 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d70159c60 0x7f1d70143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 -- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 msgr2=0x7f1d70144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 0x7f1d70144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 -- 172.21.15.170:0/2626799710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58009420 con 0x7f1d70159890 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6ed9d700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 0x7f1d70144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d6e59c700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b5e0 tx=0x7f1d60009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000f510 con 0x7f1d70159890 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6000f670 con 0x7f1d70159890 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d700b1680 con 0x7f1d70159890 2023-10-08T16:13:20.393 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6000fa30 con 0x7f1d70159890 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.387+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d700b3be0 con 0x7f1d70159890 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.388+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7004c3c0 con 0x7f1d70159890 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.389+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1d6001a400 con 0x7f1d70159890 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.389+0000 7f1d667fc700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 0x7f1d50069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.389+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1d60084780 con 0x7f1d70159890 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.389+0000 7f1d6dd9b700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 0x7f1d50069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.394 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.390+0000 7f1d6dd9b700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 0x7f1d50069850 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1d580030c0 tx=0x7f1d58016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.393+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d600514b0 con 0x7f1d70159890 2023-10-08T16:13:20.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.573+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1d7006a5b0 con 0x7f1d500673a0 2023-10-08T16:13:20.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.581+0000 7f1d667fc700 1 -- 172.21.15.170:0/2626799710 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289697 (secure 0 0 0) 0x7f1d7006a5b0 con 0x7f1d500673a0 2023-10-08T16:13:20.586 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 msgr2=0x7f1d50069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 0x7f1d50069850 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1d580030c0 tx=0x7f1d58016000 comp rx=0 tx=0).stop 2023-10-08T16:13:20.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 msgr2=0x7f1d700b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000b5e0 tx=0x7f1d60009120 comp rx=0 tx=0).stop 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 shutdown_connections 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d7015db10 0x7f1d70144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d500673a0 0x7f1d50069850 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d70159890 0x7f1d700b3590 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 --2- 172.21.15.170:0/2626799710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1d70159c60 0x7f1d70143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 >> 172.21.15.170:0/2626799710 conn(0x7f1d70068e20 msgr2=0x7f1d70153260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 shutdown_connections 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.584+0000 7f1d74b24700 1 -- 172.21.15.170:0/2626799710 wait complete. 2023-10-08T16:13:20.589 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:13:20.598 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2023-10-08T16:13:20.329277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2442856,"kb_used_data":19496,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752531864,"statfs":{"total":773094113280,"available":770592628736,"internally_reserved":0,"allocated":19963904,"data_stored":9591044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.810989"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.543032+0000","last_change":"2023-10-08T16:13:16.542431+0000","last_active":"2023-10-08T16:13:16.543032+0000","last_peered":"2023-10-08T16:13:16.543032+0000","last_clean":"2023-10-08T16:13:16.543032+0000","last_became_active":"2023-10-08T16:13:16.542202+0000","last_became_peered":"2023-10-08T16:13:16.542202+0000","last_unstale":"2023-10-08T16:13:16.543032+0000","last_undegraded":"2023-10-08T16:13:16.543032+0000","last_fullsized":"2023-10-08T16:13:16.543032+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:39:01.322301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553180+0000","last_change":"2023-10-08T16:13:16.551971+0000","last_active":"2023-10-08T16:13:16.553180+0000","last_peered":"2023-10-08T16:13:16.553180+0000","last_clean":"2023-10-08T16:13:16.553180+0000","last_became_active":"2023-10-08T16:13:16.551539+0000","last_became_peered":"2023-10-08T16:13:16.551539+0000","last_unstale":"2023-10-08T16:13:16.553180+0000","last_undegraded":"2023-10-08T16:13:16.553180+0000","last_fullsized":"2023-10-08T16:13:16.553180+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:34.219003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548911+0000","last_change":"2023-10-08T16:13:16.548248+0000","last_active":"2023-10-08T16:13:16.548911+0000","last_peered":"2023-10-08T16:13:16.548911+0000","last_clean":"2023-10-08T16:13:16.548911+0000","last_became_active":"2023-10-08T16:13:16.547988+0000","last_became_peered":"2023-10-08T16:13:16.547988+0000","last_unstale":"2023-10-08T16:13:16.548911+0000","last_undegraded":"2023-10-08T16:13:16.548911+0000","last_fullsized":"2023-10-08T16:13:16.548911+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:47.159390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562035+0000","last_change":"2023-10-08T16:13:16.555296+0000","last_active":"2023-10-08T16:13:16.562035+0000","last_peered":"2023-10-08T16:13:16.562035+0000","last_clean":"2023-10-08T16:13:16.562035+0000","last_became_active":"2023-10-08T16:13:16.555032+0000","last_became_peered":"2023-10-08T16:13:16.555032+0000","last_unstale":"2023-10-08T16:13:16.562035+0000","last_undegraded":"2023-10-08T16:13:16.562035+0000","last_fullsized":"2023-10-08T16:13:16.562035+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:55.528751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569454+0000","last_change":"2023-10-08T16:13:16.567752+0000","last_active":"2023-10-08T16:13:16.569454+0000","last_peered":"2023-10-08T16:13:16.569454+0000","last_clean":"2023-10-08T16:13:16.569454+0000","last_became_active":"2023-10-08T16:13:16.567502+0000","last_became_peered":"2023-10-08T16:13:16.567502+0000","last_unstale":"2023-10-08T16:13:16.569454+0000","last_undegraded":"2023-10-08T16:13:16.569454+0000","last_fullsized":"2023-10-08T16:13:16.569454+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:46.558442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569401+0000","last_change":"2023-10-08T16:13:16.568373+0000","last_active":"2023-10-08T16:13:16.569401+0000","last_peered":"2023-10-08T16:13:16.569401+0000","last_clean":"2023-10-08T16:13:16.569401+0000","last_became_active":"2023-10-08T16:13:16.568093+0000","last_became_peered":"2023-10-08T16:13:16.568093+0000","last_unstale":"2023-10-08T16:13:16.569401+0000","last_undegraded":"2023-10-08T16:13:16.569401+0000","last_fullsized":"2023-10-08T16:13:16.569401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:06:11.873341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555334+0000","last_change":"2023-10-08T16:13:16.551555+0000","last_active":"2023-10-08T16:13:16.555334+0000","last_peered":"2023-10-08T16:13:16.555334+0000","last_clean":"2023-10-08T16:13:16.555334+0000","last_became_active":"2023-10-08T16:13:16.551300+0000","last_became_peered":"2023-10-08T16:13:16.551300+0000","last_unstale":"2023-10-08T16:13:16.555334+0000","last_undegraded":"2023-10-08T16:13:16.555334+0000","last_fullsized":"2023-10-08T16:13:16.555334+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:39:09.646662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553162+0000","last_change":"2023-10-08T16:13:16.550167+0000","last_active":"2023-10-08T16:13:16.553162+0000","last_peered":"2023-10-08T16:13:16.553162+0000","last_clean":"2023-10-08T16:13:16.553162+0000","last_became_active":"2023-10-08T16:13:16.549784+0000","last_became_peered":"2023-10-08T16:13:16.549784+0000","last_unstale":"2023-10-08T16:13:16.553162+0000","last_undegraded":"2023-10-08T16:13:16.553162+0000","last_fullsized":"2023-10-08T16:13:16.553162+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:54.173138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556272+0000","last_change":"2023-10-08T16:13:16.550674+0000","last_active":"2023-10-08T16:13:16.556272+0000","last_peered":"2023-10-08T16:13:16.556272+0000","last_clean":"2023-10-08T16:13:16.556272+0000","last_became_active":"2023-10-08T16:13:16.550425+0000","last_became_peered":"2023-10-08T16:13:16.550425+0000","last_unstale":"2023-10-08T16:13:16.556272+0000","last_undegraded":"2023-10-08T16:13:16.556272+0000","last_fullsized":"2023-10-08T16:13:16.556272+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:27.527044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548662+0000","last_change":"2023-10-08T16:13:16.547982+0000","last_active":"2023-10-08T16:13:16.548662+0000","last_peered":"2023-10-08T16:13:16.548662+0000","last_clean":"2023-10-08T16:13:16.548662+0000","last_became_active":"2023-10-08T16:13:16.547738+0000","last_became_peered":"2023-10-08T16:13:16.547738+0000","last_unstale":"2023-10-08T16:13:16.548662+0000","last_undegraded":"2023-10-08T16:13:16.548662+0000","last_fullsized":"2023-10-08T16:13:16.548662+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:03.754001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555820+0000","last_change":"2023-10-08T16:13:16.551268+0000","last_active":"2023-10-08T16:13:16.555820+0000","last_peered":"2023-10-08T16:13:16.555820+0000","last_clean":"2023-10-08T16:13:16.555820+0000","last_became_active":"2023-10-08T16:13:16.551016+0000","last_became_peered":"2023-10-08T16:13:16.551016+0000","last_unstale":"2023-10-08T16:13:16.555820+0000","last_undegraded":"2023-10-08T16:13:16.555820+0000","last_fullsized":"2023-10-08T16:13:16.555820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:13:49.509094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566498+0000","last_change":"2023-10-08T16:13:16.562734+0000","last_active":"2023-10-08T16:13:16.566498+0000","last_peered":"2023-10-08T16:13:16.566498+0000","last_clean":"2023-10-08T16:13:16.566498+0000","last_became_active":"2023-10-08T16:13:16.562500+0000","last_became_peered":"2023-10-08T16:13:16.562500+0000","last_unstale":"2023-10-08T16:13:16.566498+0000","last_undegraded":"2023-10-08T16:13:16.566498+0000","last_fullsized":"2023-10-08T16:13:16.566498+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:17:49.339588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562899+0000","last_change":"2023-10-08T16:13:16.559278+0000","last_active":"2023-10-08T16:13:16.562899+0000","last_peered":"2023-10-08T16:13:16.562899+0000","last_clean":"2023-10-08T16:13:16.562899+0000","last_became_active":"2023-10-08T16:13:16.558971+0000","last_became_peered":"2023-10-08T16:13:16.558971+0000","last_unstale":"2023-10-08T16:13:16.562899+0000","last_undegraded":"2023-10-08T16:13:16.562899+0000","last_fullsized":"2023-10-08T16:13:16.562899+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:58.026061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.567122+0000","last_change":"2023-10-08T16:13:16.564642+0000","last_active":"2023-10-08T16:13:16.567122+0000","last_peered":"2023-10-08T16:13:16.567122+0000","last_clean":"2023-10-08T16:13:16.567122+0000","last_became_active":"2023-10-08T16:13:16.564291+0000","last_became_peered":"2023-10-08T16:13:16.564291+0000","last_unstale":"2023-10-08T16:13:16.567122+0000","last_undegraded":"2023-10-08T16:13:16.567122+0000","last_fullsized":"2023-10-08T16:13:16.567122+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:59.204686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570188+0000","last_change":"2023-10-08T16:13:16.565388+0000","last_active":"2023-10-08T16:13:16.570188+0000","last_peered":"2023-10-08T16:13:16.570188+0000","last_clean":"2023-10-08T16:13:16.570188+0000","last_became_active":"2023-10-08T16:13:16.564764+0000","last_became_peered":"2023-10-08T16:13:16.564764+0000","last_unstale":"2023-10-08T16:13:16.570188+0000","last_undegraded":"2023-10-08T16:13:16.570188+0000","last_fullsized":"2023-10-08T16:13:16.570188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:59.844735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557305+0000","last_change":"2023-10-08T16:13:16.554189+0000","last_active":"2023-10-08T16:13:16.557305+0000","last_peered":"2023-10-08T16:13:16.557305+0000","last_clean":"2023-10-08T16:13:16.557305+0000","last_became_active":"2023-10-08T16:13:16.553922+0000","last_became_peered":"2023-10-08T16:13:16.553922+0000","last_unstale":"2023-10-08T16:13:16.557305+0000","last_undegraded":"2023-10-08T16:13:16.557305+0000","last_fullsized":"2023-10-08T16:13:16.557305+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:29.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565680+0000","last_change":"2023-10-08T16:13:16.560911+0000","last_active":"2023-10-08T16:13:16.565680+0000","last_peered":"2023-10-08T16:13:16.565680+0000","last_clean":"2023-10-08T16:13:16.565680+0000","last_became_active":"2023-10-08T16:13:16.560641+0000","last_became_peered":"2023-10-08T16:13:16.560641+0000","last_unstale":"2023-10-08T16:13:16.565680+0000","last_undegraded":"2023-10-08T16:13:16.565680+0000","last_fullsized":"2023-10-08T16:13:16.565680+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:02.693333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557332+0000","last_change":"2023-10-08T16:13:16.554378+0000","last_active":"2023-10-08T16:13:16.557332+0000","last_peered":"2023-10-08T16:13:16.557332+0000","last_clean":"2023-10-08T16:13:16.557332+0000","last_became_active":"2023-10-08T16:13:16.554131+0000","last_became_peered":"2023-10-08T16:13:16.554131+0000","last_unstale":"2023-10-08T16:13:16.557332+0000","last_undegraded":"2023-10-08T16:13:16.557332+0000","last_fullsized":"2023-10-08T16:13:16.557332+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:24.262518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569685+0000","last_change":"2023-10-08T16:13:16.564064+0000","last_active":"2023-10-08T16:13:16.569685+0000","last_peered":"2023-10-08T16:13:16.569685+0000","last_clean":"2023-10-08T16:13:16.569685+0000","last_became_active":"2023-10-08T16:13:16.563805+0000","last_became_peered":"2023-10-08T16:13:16.563805+0000","last_unstale":"2023-10-08T16:13:16.569685+0000","last_undegraded":"2023-10-08T16:13:16.569685+0000","last_fullsized":"2023-10-08T16:13:16.569685+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:31.043915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553082+0000","last_change":"2023-10-08T16:13:16.551375+0000","last_active":"2023-10-08T16:13:16.553082+0000","last_peered":"2023-10-08T16:13:16.553082+0000","last_clean":"2023-10-08T16:13:16.553082+0000","last_became_active":"2023-10-08T16:13:16.551128+0000","last_became_peered":"2023-10-08T16:13:16.551128+0000","last_unstale":"2023-10-08T16:13:16.553082+0000","last_undegraded":"2023-10-08T16:13:16.553082+0000","last_fullsized":"2023-10-08T16:13:16.553082+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:20.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566857+0000","last_change":"2023-10-08T16:13:16.565970+0000","last_active":"2023-10-08T16:13:16.566857+0000","last_peered":"2023-10-08T16:13:16.566857+0000","last_clean":"2023-10-08T16:13:16.566857+0000","last_became_active":"2023-10-08T16:13:16.565686+0000","last_became_peered":"2023-10-08T16:13:16.565686+0000","last_unstale":"2023-10-08T16:13:16.566857+0000","last_undegraded":"2023-10-08T16:13:16.566857+0000","last_fullsized":"2023-10-08T16:13:16.566857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:58:03.187195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563158+0000","last_change":"2023-10-08T16:13:16.557946+0000","last_active":"2023-10-08T16:13:16.563158+0000","last_peered":"2023-10-08T16:13:16.563158+0000","last_clean":"2023-10-08T16:13:16.563158+0000","last_became_active":"2023-10-08T16:13:16.557466+0000","last_became_peered":"2023-10-08T16:13:16.557466+0000","last_unstale":"2023-10-08T16:13:16.563158+0000","last_undegraded":"2023-10-08T16:13:16.563158+0000","last_fullsized":"2023-10-08T16:13:16.563158+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:04.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570312+0000","last_change":"2023-10-08T16:13:16.567053+0000","last_active":"2023-10-08T16:13:16.570312+0000","last_peered":"2023-10-08T16:13:16.570312+0000","last_clean":"2023-10-08T16:13:16.570312+0000","last_became_active":"2023-10-08T16:13:16.565063+0000","last_became_peered":"2023-10-08T16:13:16.565063+0000","last_unstale":"2023-10-08T16:13:16.570312+0000","last_undegraded":"2023-10-08T16:13:16.570312+0000","last_fullsized":"2023-10-08T16:13:16.570312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:13:49.509094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563018+0000","last_change":"2023-10-08T16:13:16.559282+0000","last_active":"2023-10-08T16:13:16.563018+0000","last_peered":"2023-10-08T16:13:16.563018+0000","last_clean":"2023-10-08T16:13:16.563018+0000","last_became_active":"2023-10-08T16:13:16.559059+0000","last_became_peered":"2023-10-08T16:13:16.559059+0000","last_unstale":"2023-10-08T16:13:16.563018+0000","last_undegraded":"2023-10-08T16:13:16.563018+0000","last_fullsized":"2023-10-08T16:13:16.563018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:40:06.617179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560344+0000","last_change":"2023-10-08T16:13:16.554545+0000","last_active":"2023-10-08T16:13:16.560344+0000","last_peered":"2023-10-08T16:13:16.560344+0000","last_clean":"2023-10-08T16:13:16.560344+0000","last_became_active":"2023-10-08T16:13:16.554282+0000","last_became_peered":"2023-10-08T16:13:16.554282+0000","last_unstale":"2023-10-08T16:13:16.560344+0000","last_undegraded":"2023-10-08T16:13:16.560344+0000","last_fullsized":"2023-10-08T16:13:16.560344+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:13.534331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558496+0000","last_change":"2023-10-08T16:13:16.553157+0000","last_active":"2023-10-08T16:13:16.558496+0000","last_peered":"2023-10-08T16:13:16.558496+0000","last_clean":"2023-10-08T16:13:16.558496+0000","last_became_active":"2023-10-08T16:13:16.552874+0000","last_became_peered":"2023-10-08T16:13:16.552874+0000","last_unstale":"2023-10-08T16:13:16.558496+0000","last_undegraded":"2023-10-08T16:13:16.558496+0000","last_fullsized":"2023-10-08T16:13:16.558496+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:05:17.479935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.564405+0000","last_change":"2023-10-08T16:13:16.555271+0000","last_active":"2023-10-08T16:13:16.564405+0000","last_peered":"2023-10-08T16:13:16.564405+0000","last_clean":"2023-10-08T16:13:16.564405+0000","last_became_active":"2023-10-08T16:13:16.554992+0000","last_became_peered":"2023-10-08T16:13:16.554992+0000","last_unstale":"2023-10-08T16:13:16.564405+0000","last_undegraded":"2023-10-08T16:13:16.564405+0000","last_fullsized":"2023-10-08T16:13:16.564405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.983164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556061+0000","last_change":"2023-10-08T16:13:16.552028+0000","last_active":"2023-10-08T16:13:16.556061+0000","last_peered":"2023-10-08T16:13:16.556061+0000","last_clean":"2023-10-08T16:13:16.556061+0000","last_became_active":"2023-10-08T16:13:16.551776+0000","last_became_peered":"2023-10-08T16:13:16.551776+0000","last_unstale":"2023-10-08T16:13:16.556061+0000","last_undegraded":"2023-10-08T16:13:16.556061+0000","last_fullsized":"2023-10-08T16:13:16.556061+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:57.695863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565747+0000","last_change":"2023-10-08T16:13:16.563561+0000","last_active":"2023-10-08T16:13:16.565747+0000","last_peered":"2023-10-08T16:13:16.565747+0000","last_clean":"2023-10-08T16:13:16.565747+0000","last_became_active":"2023-10-08T16:13:16.563307+0000","last_became_peered":"2023-10-08T16:13:16.563307+0000","last_unstale":"2023-10-08T16:13:16.565747+0000","last_undegraded":"2023-10-08T16:13:16.565747+0000","last_fullsized":"2023-10-08T16:13:16.565747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:12.086938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555376+0000","last_change":"2023-10-08T16:13:16.554241+0000","last_active":"2023-10-08T16:13:16.555376+0000","last_peered":"2023-10-08T16:13:16.555376+0000","last_clean":"2023-10-08T16:13:16.555376+0000","last_became_active":"2023-10-08T16:13:16.553968+0000","last_became_peered":"2023-10-08T16:13:16.553968+0000","last_unstale":"2023-10-08T16:13:16.555376+0000","last_undegraded":"2023-10-08T16:13:16.555376+0000","last_fullsized":"2023-10-08T16:13:16.555376+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:01.892483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569330+0000","last_change":"2023-10-08T16:13:16.567041+0000","last_active":"2023-10-08T16:13:16.569330+0000","last_peered":"2023-10-08T16:13:16.569330+0000","last_clean":"2023-10-08T16:13:16.569330+0000","last_became_active":"2023-10-08T16:13:16.566619+0000","last_became_peered":"2023-10-08T16:13:16.566619+0000","last_unstale":"2023-10-08T16:13:16.569330+0000","last_undegraded":"2023-10-08T16:13:16.569330+0000","last_fullsized":"2023-10-08T16:13:16.569330+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:54.402605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556489+0000","last_change":"2023-10-08T16:13:16.552990+0000","last_active":"2023-10-08T16:13:16.556489+0000","last_peered":"2023-10-08T16:13:16.556489+0000","last_clean":"2023-10-08T16:13:16.556489+0000","last_became_active":"2023-10-08T16:13:16.550586+0000","last_became_peered":"2023-10-08T16:13:16.550586+0000","last_unstale":"2023-10-08T16:13:16.556489+0000","last_undegraded":"2023-10-08T16:13:16.556489+0000","last_fullsized":"2023-10-08T16:13:16.556489+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:24.535995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.552993+0000","last_change":"2023-10-08T16:13:16.550927+0000","last_active":"2023-10-08T16:13:16.552993+0000","last_peered":"2023-10-08T16:13:16.552993+0000","last_clean":"2023-10-08T16:13:16.552993+0000","last_became_active":"2023-10-08T16:13:16.550648+0000","last_became_peered":"2023-10-08T16:13:16.550648+0000","last_unstale":"2023-10-08T16:13:16.552993+0000","last_undegraded":"2023-10-08T16:13:16.552993+0000","last_fullsized":"2023-10-08T16:13:16.552993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:21.904163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"183'4","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.571014+0000","last_change":"2023-10-08T16:13:15.538049+0000","last_active":"2023-10-08T16:13:16.571014+0000","last_peered":"2023-10-08T16:13:16.571014+0000","last_clean":"2023-10-08T16:13:16.571014+0000","last_became_active":"2023-10-08T16:13:15.537805+0000","last_became_peered":"2023-10-08T16:13:15.537805+0000","last_unstale":"2023-10-08T16:13:16.571014+0000","last_undegraded":"2023-10-08T16:13:16.571014+0000","last_fullsized":"2023-10-08T16:13:16.571014+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:28.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569718+0000","last_change":"2023-10-08T16:13:16.564952+0000","last_active":"2023-10-08T16:13:16.569718+0000","last_peered":"2023-10-08T16:13:16.569718+0000","last_clean":"2023-10-08T16:13:16.569718+0000","last_became_active":"2023-10-08T16:13:16.564629+0000","last_became_peered":"2023-10-08T16:13:16.564629+0000","last_unstale":"2023-10-08T16:13:16.569718+0000","last_undegraded":"2023-10-08T16:13:16.569718+0000","last_fullsized":"2023-10-08T16:13:16.569718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:03.754001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570321+0000","last_change":"2023-10-08T16:13:15.538058+0000","last_active":"2023-10-08T16:13:16.570321+0000","last_peered":"2023-10-08T16:13:16.570321+0000","last_clean":"2023-10-08T16:13:16.570321+0000","last_became_active":"2023-10-08T16:13:15.537799+0000","last_became_peered":"2023-10-08T16:13:15.537799+0000","last_unstale":"2023-10-08T16:13:16.570321+0000","last_undegraded":"2023-10-08T16:13:16.570321+0000","last_fullsized":"2023-10-08T16:13:16.570321+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:34:25.539622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569442+0000","last_change":"2023-10-08T16:13:16.568411+0000","last_active":"2023-10-08T16:13:16.569442+0000","last_peered":"2023-10-08T16:13:16.569442+0000","last_clean":"2023-10-08T16:13:16.569442+0000","last_became_active":"2023-10-08T16:13:16.568150+0000","last_became_peered":"2023-10-08T16:13:16.568150+0000","last_unstale":"2023-10-08T16:13:16.569442+0000","last_undegraded":"2023-10-08T16:13:16.569442+0000","last_fullsized":"2023-10-08T16:13:16.569442+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:41.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561087+0000","last_change":"2023-10-08T16:13:15.538309+0000","last_active":"2023-10-08T16:13:16.561087+0000","last_peered":"2023-10-08T16:13:16.561087+0000","last_clean":"2023-10-08T16:13:16.561087+0000","last_became_active":"2023-10-08T16:13:15.538044+0000","last_became_peered":"2023-10-08T16:13:15.538044+0000","last_unstale":"2023-10-08T16:13:16.561087+0000","last_undegraded":"2023-10-08T16:13:16.561087+0000","last_fullsized":"2023-10-08T16:13:16.561087+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:23.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556769+0000","last_change":"2023-10-08T16:13:16.554472+0000","last_active":"2023-10-08T16:13:16.556769+0000","last_peered":"2023-10-08T16:13:16.556769+0000","last_clean":"2023-10-08T16:13:16.556769+0000","last_became_active":"2023-10-08T16:13:16.554202+0000","last_became_peered":"2023-10-08T16:13:16.554202+0000","last_unstale":"2023-10-08T16:13:16.556769+0000","last_undegraded":"2023-10-08T16:13:16.556769+0000","last_fullsized":"2023-10-08T16:13:16.556769+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:54.751498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.564973+0000","last_change":"2023-10-08T16:13:15.536795+0000","last_active":"2023-10-08T16:13:16.564973+0000","last_peered":"2023-10-08T16:13:16.564973+0000","last_clean":"2023-10-08T16:13:16.564973+0000","last_became_active":"2023-10-08T16:13:15.536333+0000","last_became_peered":"2023-10-08T16:13:15.536333+0000","last_unstale":"2023-10-08T16:13:16.564973+0000","last_undegraded":"2023-10-08T16:13:16.564973+0000","last_fullsized":"2023-10-08T16:13:16.564973+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:31.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566921+0000","last_change":"2023-10-08T16:13:16.565858+0000","last_active":"2023-10-08T16:13:16.566921+0000","last_peered":"2023-10-08T16:13:16.566921+0000","last_clean":"2023-10-08T16:13:16.566921+0000","last_became_active":"2023-10-08T16:13:16.565571+0000","last_became_peered":"2023-10-08T16:13:16.565571+0000","last_unstale":"2023-10-08T16:13:16.566921+0000","last_undegraded":"2023-10-08T16:13:16.566921+0000","last_fullsized":"2023-10-08T16:13:16.566921+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:27.201767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560739+0000","last_change":"2023-10-08T16:13:15.539561+0000","last_active":"2023-10-08T16:13:16.560739+0000","last_peered":"2023-10-08T16:13:16.560739+0000","last_clean":"2023-10-08T16:13:16.560739+0000","last_became_active":"2023-10-08T16:13:15.539354+0000","last_became_peered":"2023-10-08T16:13:15.539354+0000","last_unstale":"2023-10-08T16:13:16.560739+0000","last_undegraded":"2023-10-08T16:13:16.560739+0000","last_fullsized":"2023-10-08T16:13:16.560739+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:30.041562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565002+0000","last_change":"2023-10-08T16:13:16.562687+0000","last_active":"2023-10-08T16:13:16.565002+0000","last_peered":"2023-10-08T16:13:16.565002+0000","last_clean":"2023-10-08T16:13:16.565002+0000","last_became_active":"2023-10-08T16:13:16.562424+0000","last_became_peered":"2023-10-08T16:13:16.562424+0000","last_unstale":"2023-10-08T16:13:16.565002+0000","last_undegraded":"2023-10-08T16:13:16.565002+0000","last_fullsized":"2023-10-08T16:13:16.565002+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:23:52.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.537897+0000","last_change":"2023-10-08T16:13:15.533033+0000","last_active":"2023-10-08T16:13:16.537897+0000","last_peered":"2023-10-08T16:13:16.537897+0000","last_clean":"2023-10-08T16:13:16.537897+0000","last_became_active":"2023-10-08T16:13:15.532787+0000","last_became_peered":"2023-10-08T16:13:15.532787+0000","last_unstale":"2023-10-08T16:13:16.537897+0000","last_undegraded":"2023-10-08T16:13:16.537897+0000","last_fullsized":"2023-10-08T16:13:16.537897+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:09.787465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562010+0000","last_change":"2023-10-08T16:13:16.555488+0000","last_active":"2023-10-08T16:13:16.562010+0000","last_peered":"2023-10-08T16:13:16.562010+0000","last_clean":"2023-10-08T16:13:16.562010+0000","last_became_active":"2023-10-08T16:13:16.555204+0000","last_became_peered":"2023-10-08T16:13:16.555204+0000","last_unstale":"2023-10-08T16:13:16.562010+0000","last_undegraded":"2023-10-08T16:13:16.562010+0000","last_fullsized":"2023-10-08T16:13:16.562010+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:27:45.695292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.541031+0000","last_change":"2023-10-08T16:13:15.539216+0000","last_active":"2023-10-08T16:13:16.541031+0000","last_peered":"2023-10-08T16:13:16.541031+0000","last_clean":"2023-10-08T16:13:16.541031+0000","last_became_active":"2023-10-08T16:13:15.538974+0000","last_became_peered":"2023-10-08T16:13:15.538974+0000","last_unstale":"2023-10-08T16:13:16.541031+0000","last_undegraded":"2023-10-08T16:13:16.541031+0000","last_fullsized":"2023-10-08T16:13:16.541031+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:30.041562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566874+0000","last_change":"2023-10-08T16:13:16.566086+0000","last_active":"2023-10-08T16:13:16.566874+0000","last_peered":"2023-10-08T16:13:16.566874+0000","last_clean":"2023-10-08T16:13:16.566874+0000","last_became_active":"2023-10-08T16:13:16.565512+0000","last_became_peered":"2023-10-08T16:13:16.565512+0000","last_unstale":"2023-10-08T16:13:16.566874+0000","last_undegraded":"2023-10-08T16:13:16.566874+0000","last_fullsized":"2023-10-08T16:13:16.566874+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:11:23.626656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550149+0000","last_change":"2023-10-08T16:13:15.539170+0000","last_active":"2023-10-08T16:13:16.550149+0000","last_peered":"2023-10-08T16:13:16.550149+0000","last_clean":"2023-10-08T16:13:16.550149+0000","last_became_active":"2023-10-08T16:13:15.538882+0000","last_became_peered":"2023-10-08T16:13:15.538882+0000","last_unstale":"2023-10-08T16:13:16.550149+0000","last_undegraded":"2023-10-08T16:13:16.550149+0000","last_fullsized":"2023-10-08T16:13:16.550149+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:15.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562977+0000","last_change":"2023-10-08T16:13:16.558580+0000","last_active":"2023-10-08T16:13:16.562977+0000","last_peered":"2023-10-08T16:13:16.562977+0000","last_clean":"2023-10-08T16:13:16.562977+0000","last_became_active":"2023-10-08T16:13:16.557017+0000","last_became_peered":"2023-10-08T16:13:16.557017+0000","last_unstale":"2023-10-08T16:13:16.562977+0000","last_undegraded":"2023-10-08T16:13:16.562977+0000","last_fullsized":"2023-10-08T16:13:16.562977+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:27:45.695292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.538227+0000","last_change":"2023-10-08T16:13:15.527932+0000","last_active":"2023-10-08T16:13:16.538227+0000","last_peered":"2023-10-08T16:13:16.538227+0000","last_clean":"2023-10-08T16:13:16.538227+0000","last_became_active":"2023-10-08T16:13:15.527433+0000","last_became_peered":"2023-10-08T16:13:15.527433+0000","last_unstale":"2023-10-08T16:13:16.538227+0000","last_undegraded":"2023-10-08T16:13:16.538227+0000","last_fullsized":"2023-10-08T16:13:16.538227+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:04.321176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557156+0000","last_change":"2023-10-08T16:13:16.551333+0000","last_active":"2023-10-08T16:13:16.557156+0000","last_peered":"2023-10-08T16:13:16.557156+0000","last_clean":"2023-10-08T16:13:16.557156+0000","last_became_active":"2023-10-08T16:13:16.551079+0000","last_became_peered":"2023-10-08T16:13:16.551079+0000","last_unstale":"2023-10-08T16:13:16.557156+0000","last_undegraded":"2023-10-08T16:13:16.557156+0000","last_fullsized":"2023-10-08T16:13:16.557156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:31.043915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"183'3","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570560+0000","last_change":"2023-10-08T16:13:15.536108+0000","last_active":"2023-10-08T16:13:16.570560+0000","last_peered":"2023-10-08T16:13:16.570560+0000","last_clean":"2023-10-08T16:13:16.570560+0000","last_became_active":"2023-10-08T16:13:15.535792+0000","last_became_peered":"2023-10-08T16:13:15.535792+0000","last_unstale":"2023-10-08T16:13:16.570560+0000","last_undegraded":"2023-10-08T16:13:16.570560+0000","last_fullsized":"2023-10-08T16:13:16.570560+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:43.753200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555369+0000","last_change":"2023-10-08T16:13:16.547868+0000","last_active":"2023-10-08T16:13:16.555369+0000","last_peered":"2023-10-08T16:13:16.555369+0000","last_clean":"2023-10-08T16:13:16.555369+0000","last_became_active":"2023-10-08T16:13:16.547513+0000","last_became_peered":"2023-10-08T16:13:16.547513+0000","last_unstale":"2023-10-08T16:13:16.555369+0000","last_undegraded":"2023-10-08T16:13:16.555369+0000","last_fullsized":"2023-10-08T16:13:16.555369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:45.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"183'6","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570662+0000","last_change":"2023-10-08T16:13:15.538790+0000","last_active":"2023-10-08T16:13:16.570662+0000","last_peered":"2023-10-08T16:13:16.570662+0000","last_clean":"2023-10-08T16:13:16.570662+0000","last_became_active":"2023-10-08T16:13:15.538555+0000","last_became_peered":"2023-10-08T16:13:15.538555+0000","last_unstale":"2023-10-08T16:13:16.570662+0000","last_undegraded":"2023-10-08T16:13:16.570662+0000","last_fullsized":"2023-10-08T16:13:16.570662+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:02.751647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550939+0000","last_change":"2023-10-08T16:13:16.547943+0000","last_active":"2023-10-08T16:13:16.550939+0000","last_peered":"2023-10-08T16:13:16.550939+0000","last_clean":"2023-10-08T16:13:16.550939+0000","last_became_active":"2023-10-08T16:13:16.547653+0000","last_became_peered":"2023-10-08T16:13:16.547653+0000","last_unstale":"2023-10-08T16:13:16.550939+0000","last_undegraded":"2023-10-08T16:13:16.550939+0000","last_fullsized":"2023-10-08T16:13:16.550939+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:21.617690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565487+0000","last_change":"2023-10-08T16:13:15.539204+0000","last_active":"2023-10-08T16:13:16.565487+0000","last_peered":"2023-10-08T16:13:16.565487+0000","last_clean":"2023-10-08T16:13:16.565487+0000","last_became_active":"2023-10-08T16:13:15.538927+0000","last_became_peered":"2023-10-08T16:13:15.538927+0000","last_unstale":"2023-10-08T16:13:16.565487+0000","last_undegraded":"2023-10-08T16:13:16.565487+0000","last_fullsized":"2023-10-08T16:13:16.565487+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.169334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560718+0000","last_change":"2023-10-08T16:13:16.558667+0000","last_active":"2023-10-08T16:13:16.560718+0000","last_peered":"2023-10-08T16:13:16.560718+0000","last_clean":"2023-10-08T16:13:16.560718+0000","last_became_active":"2023-10-08T16:13:16.558274+0000","last_became_peered":"2023-10-08T16:13:16.558274+0000","last_unstale":"2023-10-08T16:13:16.560718+0000","last_undegraded":"2023-10-08T16:13:16.560718+0000","last_fullsized":"2023-10-08T16:13:16.560718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:46.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548071+0000","last_change":"2023-10-08T16:13:15.536117+0000","last_active":"2023-10-08T16:13:16.548071+0000","last_peered":"2023-10-08T16:13:16.548071+0000","last_clean":"2023-10-08T16:13:16.548071+0000","last_became_active":"2023-10-08T16:13:15.535753+0000","last_became_peered":"2023-10-08T16:13:15.535753+0000","last_unstale":"2023-10-08T16:13:16.548071+0000","last_undegraded":"2023-10-08T16:13:16.548071+0000","last_fullsized":"2023-10-08T16:13:16.548071+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:13:31.639348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570257+0000","last_change":"2023-10-08T16:13:16.566115+0000","last_active":"2023-10-08T16:13:16.570257+0000","last_peered":"2023-10-08T16:13:16.570257+0000","last_clean":"2023-10-08T16:13:16.570257+0000","last_became_active":"2023-10-08T16:13:16.565853+0000","last_became_peered":"2023-10-08T16:13:16.565853+0000","last_unstale":"2023-10-08T16:13:16.570257+0000","last_undegraded":"2023-10-08T16:13:16.570257+0000","last_fullsized":"2023-10-08T16:13:16.570257+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:24.262518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548029+0000","last_change":"2023-10-08T16:13:15.538483+0000","last_active":"2023-10-08T16:13:16.548029+0000","last_peered":"2023-10-08T16:13:16.548029+0000","last_clean":"2023-10-08T16:13:16.548029+0000","last_became_active":"2023-10-08T16:13:15.538170+0000","last_became_peered":"2023-10-08T16:13:15.538170+0000","last_unstale":"2023-10-08T16:13:16.548029+0000","last_undegraded":"2023-10-08T16:13:16.548029+0000","last_fullsized":"2023-10-08T16:13:16.548029+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:39.503047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561987+0000","last_change":"2023-10-08T16:13:16.553779+0000","last_active":"2023-10-08T16:13:16.561987+0000","last_peered":"2023-10-08T16:13:16.561987+0000","last_clean":"2023-10-08T16:13:16.561987+0000","last_became_active":"2023-10-08T16:13:16.553452+0000","last_became_peered":"2023-10-08T16:13:16.553452+0000","last_unstale":"2023-10-08T16:13:16.561987+0000","last_undegraded":"2023-10-08T16:13:16.561987+0000","last_fullsized":"2023-10-08T16:13:16.561987+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:10.789818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"183'8","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569350+0000","last_change":"2023-10-08T16:13:15.541913+0000","last_active":"2023-10-08T16:13:16.569350+0000","last_peered":"2023-10-08T16:13:16.569350+0000","last_clean":"2023-10-08T16:13:16.569350+0000","last_became_active":"2023-10-08T16:13:15.541623+0000","last_became_peered":"2023-10-08T16:13:15.541623+0000","last_unstale":"2023-10-08T16:13:16.569350+0000","last_undegraded":"2023-10-08T16:13:16.569350+0000","last_fullsized":"2023-10-08T16:13:16.569350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:08.368910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558813+0000","last_change":"2023-10-08T16:13:16.553698+0000","last_active":"2023-10-08T16:13:16.558813+0000","last_peered":"2023-10-08T16:13:16.558813+0000","last_clean":"2023-10-08T16:13:16.558813+0000","last_became_active":"2023-10-08T16:13:16.553415+0000","last_became_peered":"2023-10-08T16:13:16.553415+0000","last_unstale":"2023-10-08T16:13:16.558813+0000","last_undegraded":"2023-10-08T16:13:16.558813+0000","last_fullsized":"2023-10-08T16:13:16.558813+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:53:39.823950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569509+0000","last_change":"2023-10-08T16:13:15.538309+0000","last_active":"2023-10-08T16:13:16.569509+0000","last_peered":"2023-10-08T16:13:16.569509+0000","last_clean":"2023-10-08T16:13:16.569509+0000","last_became_active":"2023-10-08T16:13:15.538022+0000","last_became_peered":"2023-10-08T16:13:15.538022+0000","last_unstale":"2023-10-08T16:13:16.569509+0000","last_undegraded":"2023-10-08T16:13:16.569509+0000","last_fullsized":"2023-10-08T16:13:16.569509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:58.842381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568318+0000","last_change":"2023-10-08T16:13:16.562448+0000","last_active":"2023-10-08T16:13:16.568318+0000","last_peered":"2023-10-08T16:13:16.568318+0000","last_clean":"2023-10-08T16:13:16.568318+0000","last_became_active":"2023-10-08T16:13:16.562183+0000","last_became_peered":"2023-10-08T16:13:16.562183+0000","last_unstale":"2023-10-08T16:13:16.568318+0000","last_undegraded":"2023-10-08T16:13:16.568318+0000","last_fullsized":"2023-10-08T16:13:16.568318+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:04.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.567277+0000","last_change":"2023-10-08T16:13:15.533749+0000","last_active":"2023-10-08T16:13:16.567277+0000","last_peered":"2023-10-08T16:13:16.567277+0000","last_clean":"2023-10-08T16:13:16.567277+0000","last_became_active":"2023-10-08T16:13:15.533490+0000","last_became_peered":"2023-10-08T16:13:15.533490+0000","last_unstale":"2023-10-08T16:13:16.567277+0000","last_undegraded":"2023-10-08T16:13:16.567277+0000","last_fullsized":"2023-10-08T16:13:16.567277+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:15.407146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560352+0000","last_change":"2023-10-08T16:13:16.553594+0000","last_active":"2023-10-08T16:13:16.560352+0000","last_peered":"2023-10-08T16:13:16.560352+0000","last_clean":"2023-10-08T16:13:16.560352+0000","last_became_active":"2023-10-08T16:13:16.553338+0000","last_became_peered":"2023-10-08T16:13:16.553338+0000","last_unstale":"2023-10-08T16:13:16.560352+0000","last_undegraded":"2023-10-08T16:13:16.560352+0000","last_fullsized":"2023-10-08T16:13:16.560352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:40.569851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570436+0000","last_change":"2023-10-08T16:13:15.537983+0000","last_active":"2023-10-08T16:13:16.570436+0000","last_peered":"2023-10-08T16:13:16.570436+0000","last_clean":"2023-10-08T16:13:16.570436+0000","last_became_active":"2023-10-08T16:13:15.537711+0000","last_became_peered":"2023-10-08T16:13:15.537711+0000","last_unstale":"2023-10-08T16:13:16.570436+0000","last_undegraded":"2023-10-08T16:13:16.570436+0000","last_fullsized":"2023-10-08T16:13:16.570436+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:55.761887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557275+0000","last_change":"2023-10-08T16:13:16.555038+0000","last_active":"2023-10-08T16:13:16.557275+0000","last_peered":"2023-10-08T16:13:16.557275+0000","last_clean":"2023-10-08T16:13:16.557275+0000","last_became_active":"2023-10-08T16:13:16.554628+0000","last_became_peered":"2023-10-08T16:13:16.554628+0000","last_unstale":"2023-10-08T16:13:16.557275+0000","last_undegraded":"2023-10-08T16:13:16.557275+0000","last_fullsized":"2023-10-08T16:13:16.557275+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:59.844735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553577+0000","last_change":"2023-10-08T16:13:15.533718+0000","last_active":"2023-10-08T16:13:16.553577+0000","last_peered":"2023-10-08T16:13:16.553577+0000","last_clean":"2023-10-08T16:13:16.553577+0000","last_became_active":"2023-10-08T16:13:15.533383+0000","last_became_peered":"2023-10-08T16:13:15.533383+0000","last_unstale":"2023-10-08T16:13:16.553577+0000","last_undegraded":"2023-10-08T16:13:16.553577+0000","last_fullsized":"2023-10-08T16:13:16.553577+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:38.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562015+0000","last_change":"2023-10-08T16:13:16.552068+0000","last_active":"2023-10-08T16:13:16.562015+0000","last_peered":"2023-10-08T16:13:16.562015+0000","last_clean":"2023-10-08T16:13:16.562015+0000","last_became_active":"2023-10-08T16:13:16.551610+0000","last_became_peered":"2023-10-08T16:13:16.551610+0000","last_unstale":"2023-10-08T16:13:16.562015+0000","last_undegraded":"2023-10-08T16:13:16.562015+0000","last_fullsized":"2023-10-08T16:13:16.562015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:16.578672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568720+0000","last_change":"2023-10-08T16:13:15.541240+0000","last_active":"2023-10-08T16:13:16.568720+0000","last_peered":"2023-10-08T16:13:16.568720+0000","last_clean":"2023-10-08T16:13:16.568720+0000","last_became_active":"2023-10-08T16:13:15.541003+0000","last_became_peered":"2023-10-08T16:13:15.541003+0000","last_unstale":"2023-10-08T16:13:16.568720+0000","last_undegraded":"2023-10-08T16:13:16.568720+0000","last_fullsized":"2023-10-08T16:13:16.568720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550728+0000","last_change":"2023-10-08T16:13:16.547862+0000","last_active":"2023-10-08T16:13:16.550728+0000","last_peered":"2023-10-08T16:13:16.550728+0000","last_clean":"2023-10-08T16:13:16.550728+0000","last_became_active":"2023-10-08T16:13:16.547591+0000","last_became_peered":"2023-10-08T16:13:16.547591+0000","last_unstale":"2023-10-08T16:13:16.550728+0000","last_undegraded":"2023-10-08T16:13:16.550728+0000","last_fullsized":"2023-10-08T16:13:16.550728+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:45.933015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557302+0000","last_change":"2023-10-08T16:13:15.534483+0000","last_active":"2023-10-08T16:13:16.557302+0000","last_peered":"2023-10-08T16:13:16.557302+0000","last_clean":"2023-10-08T16:13:16.557302+0000","last_became_active":"2023-10-08T16:13:15.534030+0000","last_became_peered":"2023-10-08T16:13:15.534030+0000","last_unstale":"2023-10-08T16:13:16.557302+0000","last_undegraded":"2023-10-08T16:13:16.557302+0000","last_fullsized":"2023-10-08T16:13:16.557302+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:43.579186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563058+0000","last_change":"2023-10-08T16:13:16.557905+0000","last_active":"2023-10-08T16:13:16.563058+0000","last_peered":"2023-10-08T16:13:16.563058+0000","last_clean":"2023-10-08T16:13:16.563058+0000","last_became_active":"2023-10-08T16:13:16.557397+0000","last_became_peered":"2023-10-08T16:13:16.557397+0000","last_unstale":"2023-10-08T16:13:16.563058+0000","last_undegraded":"2023-10-08T16:13:16.563058+0000","last_fullsized":"2023-10-08T16:13:16.563058+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:16:22.299401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558271+0000","last_change":"2023-10-08T16:13:15.535401+0000","last_active":"2023-10-08T16:13:16.558271+0000","last_peered":"2023-10-08T16:13:16.558271+0000","last_clean":"2023-10-08T16:13:16.558271+0000","last_became_active":"2023-10-08T16:13:15.535154+0000","last_became_peered":"2023-10-08T16:13:15.535154+0000","last_unstale":"2023-10-08T16:13:16.558271+0000","last_undegraded":"2023-10-08T16:13:16.558271+0000","last_fullsized":"2023-10-08T16:13:16.558271+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:37.367041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568328+0000","last_change":"2023-10-08T16:13:16.566092+0000","last_active":"2023-10-08T16:13:16.568328+0000","last_peered":"2023-10-08T16:13:16.568328+0000","last_clean":"2023-10-08T16:13:16.568328+0000","last_became_active":"2023-10-08T16:13:16.565832+0000","last_became_peered":"2023-10-08T16:13:16.565832+0000","last_unstale":"2023-10-08T16:13:16.568328+0000","last_undegraded":"2023-10-08T16:13:16.568328+0000","last_fullsized":"2023-10-08T16:13:16.568328+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:02:54.584641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.549842+0000","last_change":"2023-10-08T16:13:15.535186+0000","last_active":"2023-10-08T16:13:16.549842+0000","last_peered":"2023-10-08T16:13:16.549842+0000","last_clean":"2023-10-08T16:13:16.549842+0000","last_became_active":"2023-10-08T16:13:15.534944+0000","last_became_peered":"2023-10-08T16:13:15.534944+0000","last_unstale":"2023-10-08T16:13:16.549842+0000","last_undegraded":"2023-10-08T16:13:16.549842+0000","last_fullsized":"2023-10-08T16:13:16.549842+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:06:36.509663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555232+0000","last_change":"2023-10-08T16:13:16.552557+0000","last_active":"2023-10-08T16:13:16.555232+0000","last_peered":"2023-10-08T16:13:16.555232+0000","last_clean":"2023-10-08T16:13:16.555232+0000","last_became_active":"2023-10-08T16:13:16.552289+0000","last_became_peered":"2023-10-08T16:13:16.552289+0000","last_unstale":"2023-10-08T16:13:16.555232+0000","last_undegraded":"2023-10-08T16:13:16.555232+0000","last_fullsized":"2023-10-08T16:13:16.555232+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:28.093263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563806+0000","last_change":"2023-10-08T16:13:15.534638+0000","last_active":"2023-10-08T16:13:16.563806+0000","last_peered":"2023-10-08T16:13:16.563806+0000","last_clean":"2023-10-08T16:13:16.563806+0000","last_became_active":"2023-10-08T16:13:15.534379+0000","last_became_peered":"2023-10-08T16:13:15.534379+0000","last_unstale":"2023-10-08T16:13:16.563806+0000","last_undegraded":"2023-10-08T16:13:16.563806+0000","last_fullsized":"2023-10-08T16:13:16.563806+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:38.851040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570167+0000","last_change":"2023-10-08T16:13:16.565167+0000","last_active":"2023-10-08T16:13:16.570167+0000","last_peered":"2023-10-08T16:13:16.570167+0000","last_clean":"2023-10-08T16:13:16.570167+0000","last_became_active":"2023-10-08T16:13:16.564444+0000","last_became_peered":"2023-10-08T16:13:16.564444+0000","last_unstale":"2023-10-08T16:13:16.570167+0000","last_undegraded":"2023-10-08T16:13:16.570167+0000","last_fullsized":"2023-10-08T16:13:16.570167+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:29.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.547844+0000","last_change":"2023-10-08T16:13:15.535192+0000","last_active":"2023-10-08T16:13:16.547844+0000","last_peered":"2023-10-08T16:13:16.547844+0000","last_clean":"2023-10-08T16:13:16.547844+0000","last_became_active":"2023-10-08T16:13:15.534919+0000","last_became_peered":"2023-10-08T16:13:15.534919+0000","last_unstale":"2023-10-08T16:13:16.547844+0000","last_undegraded":"2023-10-08T16:13:16.547844+0000","last_fullsized":"2023-10-08T16:13:16.547844+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:50.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566779+0000","last_change":"2023-10-08T16:13:16.565986+0000","last_active":"2023-10-08T16:13:16.566779+0000","last_peered":"2023-10-08T16:13:16.566779+0000","last_clean":"2023-10-08T16:13:16.566779+0000","last_became_active":"2023-10-08T16:13:16.565721+0000","last_became_peered":"2023-10-08T16:13:16.565721+0000","last_unstale":"2023-10-08T16:13:16.566779+0000","last_undegraded":"2023-10-08T16:13:16.566779+0000","last_fullsized":"2023-10-08T16:13:16.566779+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:06:37.512016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550551+0000","last_change":"2023-10-08T16:13:15.539117+0000","last_active":"2023-10-08T16:13:16.550551+0000","last_peered":"2023-10-08T16:13:16.550551+0000","last_clean":"2023-10-08T16:13:16.550551+0000","last_became_active":"2023-10-08T16:13:15.538916+0000","last_became_peered":"2023-10-08T16:13:15.538916+0000","last_unstale":"2023-10-08T16:13:16.550551+0000","last_undegraded":"2023-10-08T16:13:16.550551+0000","last_fullsized":"2023-10-08T16:13:16.550551+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:28.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561631+0000","last_change":"2023-10-08T16:13:16.553031+0000","last_active":"2023-10-08T16:13:16.561631+0000","last_peered":"2023-10-08T16:13:16.561631+0000","last_clean":"2023-10-08T16:13:16.561631+0000","last_became_active":"2023-10-08T16:13:16.552711+0000","last_became_peered":"2023-10-08T16:13:16.552711+0000","last_unstale":"2023-10-08T16:13:16.561631+0000","last_undegraded":"2023-10-08T16:13:16.561631+0000","last_fullsized":"2023-10-08T16:13:16.561631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:33:34.713664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550454+0000","last_change":"2023-10-08T16:13:15.536795+0000","last_active":"2023-10-08T16:13:16.550454+0000","last_peered":"2023-10-08T16:13:16.550454+0000","last_clean":"2023-10-08T16:13:16.550454+0000","last_became_active":"2023-10-08T16:13:15.536369+0000","last_became_peered":"2023-10-08T16:13:15.536369+0000","last_unstale":"2023-10-08T16:13:16.550454+0000","last_undegraded":"2023-10-08T16:13:16.550454+0000","last_fullsized":"2023-10-08T16:13:16.550454+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:46.157037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558065+0000","last_change":"2023-10-08T16:13:16.556870+0000","last_active":"2023-10-08T16:13:16.558065+0000","last_peered":"2023-10-08T16:13:16.558065+0000","last_clean":"2023-10-08T16:13:16.558065+0000","last_became_active":"2023-10-08T16:13:16.556612+0000","last_became_peered":"2023-10-08T16:13:16.556612+0000","last_unstale":"2023-10-08T16:13:16.558065+0000","last_undegraded":"2023-10-08T16:13:16.558065+0000","last_fullsized":"2023-10-08T16:13:16.558065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:43.757943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557990+0000","last_change":"2023-10-08T16:13:15.533238+0000","last_active":"2023-10-08T16:13:16.557990+0000","last_peered":"2023-10-08T16:13:16.557990+0000","last_clean":"2023-10-08T16:13:16.557990+0000","last_became_active":"2023-10-08T16:13:15.532936+0000","last_became_peered":"2023-10-08T16:13:15.532936+0000","last_unstale":"2023-10-08T16:13:16.557990+0000","last_undegraded":"2023-10-08T16:13:16.557990+0000","last_fullsized":"2023-10-08T16:13:16.557990+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:48.274831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.559279+0000","last_change":"2023-10-08T16:13:16.558818+0000","last_active":"2023-10-08T16:13:16.559279+0000","last_peered":"2023-10-08T16:13:16.559279+0000","last_clean":"2023-10-08T16:13:16.559279+0000","last_became_active":"2023-10-08T16:13:16.558541+0000","last_became_peered":"2023-10-08T16:13:16.558541+0000","last_unstale":"2023-10-08T16:13:16.559279+0000","last_undegraded":"2023-10-08T16:13:16.559279+0000","last_fullsized":"2023-10-08T16:13:16.559279+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:32:42.833366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548177+0000","last_change":"2023-10-08T16:13:15.532942+0000","last_active":"2023-10-08T16:13:16.548177+0000","last_peered":"2023-10-08T16:13:16.548177+0000","last_clean":"2023-10-08T16:13:16.548177+0000","last_became_active":"2023-10-08T16:13:15.532657+0000","last_became_peered":"2023-10-08T16:13:15.532657+0000","last_unstale":"2023-10-08T16:13:16.548177+0000","last_undegraded":"2023-10-08T16:13:16.548177+0000","last_fullsized":"2023-10-08T16:13:16.548177+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:34:25.539622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553163+0000","last_change":"2023-10-08T16:13:16.551958+0000","last_active":"2023-10-08T16:13:16.553163+0000","last_peered":"2023-10-08T16:13:16.553163+0000","last_clean":"2023-10-08T16:13:16.553163+0000","last_became_active":"2023-10-08T16:13:16.550627+0000","last_became_peered":"2023-10-08T16:13:16.550627+0000","last_unstale":"2023-10-08T16:13:16.553163+0000","last_undegraded":"2023-10-08T16:13:16.553163+0000","last_fullsized":"2023-10-08T16:13:16.553163+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:34.832697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.537676+0000","last_change":"2023-10-08T16:13:15.527840+0000","last_active":"2023-10-08T16:13:16.537676+0000","last_peered":"2023-10-08T16:13:16.537676+0000","last_clean":"2023-10-08T16:13:16.537676+0000","last_became_active":"2023-10-08T16:13:15.527584+0000","last_became_peered":"2023-10-08T16:13:15.527584+0000","last_unstale":"2023-10-08T16:13:16.537676+0000","last_undegraded":"2023-10-08T16:13:16.537676+0000","last_fullsized":"2023-10-08T16:13:16.537676+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:53.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565660+0000","last_change":"2023-10-08T16:13:16.560839+0000","last_active":"2023-10-08T16:13:16.565660+0000","last_peered":"2023-10-08T16:13:16.565660+0000","last_clean":"2023-10-08T16:13:16.565660+0000","last_became_active":"2023-10-08T16:13:16.560535+0000","last_became_peered":"2023-10-08T16:13:16.560535+0000","last_unstale":"2023-10-08T16:13:16.565660+0000","last_undegraded":"2023-10-08T16:13:16.565660+0000","last_fullsized":"2023-10-08T16:13:16.565660+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.983164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557267+0000","last_change":"2023-10-08T16:13:15.528041+0000","last_active":"2023-10-08T16:13:16.557267+0000","last_peered":"2023-10-08T16:13:16.557267+0000","last_clean":"2023-10-08T16:13:16.557267+0000","last_became_active":"2023-10-08T16:13:15.527784+0000","last_became_peered":"2023-10-08T16:13:15.527784+0000","last_unstale":"2023-10-08T16:13:16.557267+0000","last_undegraded":"2023-10-08T16:13:16.557267+0000","last_fullsized":"2023-10-08T16:13:16.557267+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:00.618426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555711+0000","last_change":"2023-10-08T16:13:16.549639+0000","last_active":"2023-10-08T16:13:16.555711+0000","last_peered":"2023-10-08T16:13:16.555711+0000","last_clean":"2023-10-08T16:13:16.555711+0000","last_became_active":"2023-10-08T16:13:16.548935+0000","last_became_peered":"2023-10-08T16:13:16.548935+0000","last_unstale":"2023-10-08T16:13:16.555711+0000","last_undegraded":"2023-10-08T16:13:16.555711+0000","last_fullsized":"2023-10-08T16:13:16.555711+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:25.016780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560945+0000","last_change":"2023-10-08T16:13:15.536599+0000","last_active":"2023-10-08T16:13:16.560945+0000","last_peered":"2023-10-08T16:13:16.560945+0000","last_clean":"2023-10-08T16:13:16.560945+0000","last_became_active":"2023-10-08T16:13:15.536344+0000","last_became_peered":"2023-10-08T16:13:15.536344+0000","last_unstale":"2023-10-08T16:13:16.560945+0000","last_undegraded":"2023-10-08T16:13:16.560945+0000","last_fullsized":"2023-10-08T16:13:16.560945+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:44.930661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":301,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.552077+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:13:16.552077+0000","last_peered":"2023-10-08T16:13:16.552077+0000","last_clean":"2023-10-08T16:13:16.552077+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:13:16.552077+0000","last_undegraded":"2023-10-08T16:13:16.552077+0000","last_fullsized":"2023-10-08T16:13:16.552077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304256,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067584,"statfs":{"total":96636764160,"available":96325206016,"internally_reserved":0,"allocated":2940928,"data_stored":1641204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.86199999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72799999999999998},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.82199999999999995,"15min":1.083},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.64900000000000002,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.70299999999999996,"15min":1.175},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.77900000000000003},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.374,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":1.677,"15min":1.677},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.78100000000000003},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63600000000000001,"5min":0.69899999999999995,"15min":8.3409999999999993},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44400000000000001,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.753,"15min":0.753},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.625}]},{"osd":6,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1050824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.252},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.90600000000000003},"last":0.33900000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.496},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.68500000000000005,"5min":0.747,"15min":0.76700000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.96599999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.94199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.72499999999999998,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61499999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.377},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.82399999999999995,"15min":0.874},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.47799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307764,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064076,"statfs":{"total":96636764160,"available":96321613824,"internally_reserved":0,"allocated":2338816,"data_stored":1050712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.307,"15min":0.312},"min":{"1min":0.245,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.34000000000000002},"min":{"1min":0.253,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.502,"5min":0.627,"15min":0.77500000000000002},"last":0.22700000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.35499999999999998},"min":{"1min":0.27000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52100000000000002,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.311},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.37,"15min":0.374},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.72399999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.38600000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.28699999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.33900000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":1.077},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.84499999999999997,"5min":1.696,"15min":1.696},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.80200000000000005,"5min":1.657,"15min":1.657},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72799999999999998,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85999999999999999,"5min":0.875,"15min":1.1839999999999999},"last":0.622}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":2945024,"data_stored":1642616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.313,"15min":0.48899999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.41399999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.441},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.52000000000000002},"min":{"1min":0.27600000000000002,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.41499999999999998,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.373},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.219},"max":{"1min":0.501,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32700000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.188},"max":{"1min":0.59499999999999997,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.55800000000000005},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.26900000000000002},"max":{"1min":0.42999999999999999,"5min":1.498,"15min":5.5970000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55300000000000005,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":1.847,"15min":3.2890000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77000000000000002,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1051336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.307},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.46400000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.34699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.44400000000000001,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.35999999999999999},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.318}]},{"osd":3,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.34200000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48999999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81499999999999995,"15min":4.1029999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74399999999999999,"5min":0.85199999999999998,"15min":0.871},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307788,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064052,"statfs":{"total":96636764160,"available":96321589248,"internally_reserved":0,"allocated":2363392,"data_stored":1052858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.34399999999999997},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":0.44900000000000001,"5min":0.505,"15min":1.238},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.376},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.35299999999999998},"min":{"1min":0.254,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45800000000000002,"5min":0.47999999999999998,"15min":1.0269999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.39700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.499,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.309}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36399999999999999,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.248},"max":{"1min":0.47699999999999998,"5min":0.53900000000000003,"15min":1.1399999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.28299999999999997,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":1.327},"last":0.376}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.9119999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.099,"15min":1.099},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":1.27},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":1.879},"last":0.65500000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.75700000000000001,"5min":0.81399999999999995,"15min":1.0620000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":1.825},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"max":{"1min":0.81100000000000005,"5min":1.0009999999999999,"15min":1.115},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1050782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.52600000000000002},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.40000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.64000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.70999999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":0.83399999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.60099999999999998},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.34899999999999998},"max":{"1min":0.80400000000000005,"5min":0.875,"15min":0.875},"last":0.749},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.89200000000000002},"last":0.77200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1050712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68700000000000006,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51700000000000002,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73499999999999999,"5min":0.84399999999999997,"15min":12.227},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.871,"15min":1.252},"last":0.68700000000000006}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.314,"15min":0.314},"max":{"1min":0.73899999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.88700000000000001,"15min":1.444},"last":0.72199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.755,"15min":9.3170000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":0.879,"15min":3.8719999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.495,"15min":0.5},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.93799999999999994},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.254,"15min":0.254},"max":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.83199999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.877,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:13:20.604 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:13:20.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.751+0000 7ff7730af700 1 Processor -- start 2023-10-08T16:13:20.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.751+0000 7ff7730af700 1 -- start start 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.752+0000 7ff7730af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7ff76c159890 0x7ff76c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.752+0000 7ff7730af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.752+0000 7ff7730af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff76c15a160 con 0x7ff76c15db10 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.752+0000 7ff7730af700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76c0d13a0 con 0x7ff76c159890 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.752+0000 7ff7730af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c15e140 con 0x7ff76c159c60 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff77164c700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7ff76c15db10 0x7ff76c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:38598/0 (socket says 172.21.15.170:38598) 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76bfff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff77164c700 1 -- 172.21.15.170:0/2868724612 learned_addr learned my addr 172.21.15.170:0/2868724612 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:20.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 --2- 172.21.15.170:0/2868724612 >> v2:172.21.15.191:3300/0 conn(0x7ff76c159890 0x7ff76c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76b7fe700 1 -- 172.21.15.170:0/2868724612 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276589280 0 0) 0x7ff76c15a160 con 0x7ff76c15db10 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76b7fe700 1 -- 172.21.15.170:0/2868724612 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff754003570 con 0x7ff76c15db10 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 -- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 msgr2=0x7ff76c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 -- 172.21.15.170:0/2868724612 >> v1:172.21.15.191:6789/0 conn(0x7ff76c15db10 legacy=0x7ff76c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 -- 172.21.15.170:0/2868724612 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c15a3d0 con 0x7ff76c159890 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76bfff700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff770e4b700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c15abd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009e60 tx=0x7ff75c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76cf3b59e347f821 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76b7fe700 1 -- 172.21.15.170:0/2868724612 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00e070 con 0x7ff76c159890 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76b7fe700 1 -- 172.21.15.170:0/2868724612 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c00bab0 con 0x7ff76c159890 2023-10-08T16:13:20.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.753+0000 7ff76b7fe700 1 -- 172.21.15.170:0/2868724612 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c00be70 con 0x7ff76c159890 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 -- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 msgr2=0x7ff76c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c15abd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009e60 tx=0x7ff75c009780 comp rx=0 tx=0).stop 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 -- 172.21.15.170:0/2868724612 shutdown_connections 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c15abd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 --2- 172.21.15.170:0/2868724612 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 -- 172.21.15.170:0/2868724612 >> 172.21.15.170:0/2868724612 conn(0x7ff76c068e20 msgr2=0x7ff76c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 -- 172.21.15.170:0/2868724612 shutdown_connections 2023-10-08T16:13:20.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.754+0000 7ff7730af700 1 -- 172.21.15.170:0/2868724612 wait complete. 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.755+0000 7ff7730af700 1 Processor -- start 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.755+0000 7ff7730af700 1 -- start start 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c0b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0b4710 con 0x7ff76c15db10 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0b3a90 con 0x7ff76c159890 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff7730af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76c0b3d70 con 0x7ff76c159c60 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.761 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55912/0 (socket says 172.21.15.170:55912) 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 -- 172.21.15.170:0/1394568439 learned_addr learned my addr 172.21.15.170:0/1394568439 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff770e4b700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c0b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff76bfff700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 -- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 msgr2=0x7ff76c169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 -- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 msgr2=0x7ff76c0b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c0b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 -- 172.21.15.170:0/1394568439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff760009420 con 0x7ff76c15db10 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.756+0000 7ff77164c700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff7640010e0 tx=0x7ff764007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.757+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764005c00 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.757+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff764005d60 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.757+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640072e0 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.757+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c009a20 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.757+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c0b4100 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.758+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76c0b4850 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.758+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff764012400 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.758+0000 7ff768ff9700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 0x7ff7580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.759+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff764084050 con 0x7ff76c15db10 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.759+0000 7ff770e4b700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 0x7ff7580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:20.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.759+0000 7ff770e4b700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 0x7ff7580697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0097c0 tx=0x7ff75c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:20.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.763+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff76404b5a0 con 0x7ff76c15db10 2023-10-08T16:13:20.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.946+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff76c0b3f30 con 0x7ff758067320 2023-10-08T16:13:20.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.954+0000 7ff768ff9700 1 -- 172.21.15.170:0/1394568439 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289697 (secure 0 0 0) 0x7ff75c00bd30 con 0x7ff758067320 2023-10-08T16:13:20.961 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:20.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 msgr2=0x7ff7580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 0x7ff7580697d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0097c0 tx=0x7ff75c019000 comp rx=0 tx=0).stop 2023-10-08T16:13:20.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 msgr2=0x7ff76c16a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7ff7640010e0 tx=0x7ff764007dd0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 shutdown_connections 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff758067320 0x7ff7580697d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff76c15db10 0x7ff76c16a130 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff76c159890 0x7ff76c0b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.957+0000 7ff7730af700 1 --2- 172.21.15.170:0/1394568439 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff76c159c60 0x7ff76c169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.958+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 >> 172.21.15.170:0/1394568439 conn(0x7ff76c068e20 msgr2=0x7ff76c152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.958+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 shutdown_connections 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:20.958+0000 7ff7730af700 1 -- 172.21.15.170:0/1394568439 wait complete. 2023-10-08T16:13:20.964 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:13:20.971 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2023-10-08T16:13:20.329277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2442856,"kb_used_data":19496,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752531864,"statfs":{"total":773094113280,"available":770592628736,"internally_reserved":0,"allocated":19963904,"data_stored":9591044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.810989"},"pg_stats":[{"pgid":"32.1f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.543032+0000","last_change":"2023-10-08T16:13:16.542431+0000","last_active":"2023-10-08T16:13:16.543032+0000","last_peered":"2023-10-08T16:13:16.543032+0000","last_clean":"2023-10-08T16:13:16.543032+0000","last_became_active":"2023-10-08T16:13:16.542202+0000","last_became_peered":"2023-10-08T16:13:16.542202+0000","last_unstale":"2023-10-08T16:13:16.543032+0000","last_undegraded":"2023-10-08T16:13:16.543032+0000","last_fullsized":"2023-10-08T16:13:16.543032+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:39:01.322301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553180+0000","last_change":"2023-10-08T16:13:16.551971+0000","last_active":"2023-10-08T16:13:16.553180+0000","last_peered":"2023-10-08T16:13:16.553180+0000","last_clean":"2023-10-08T16:13:16.553180+0000","last_became_active":"2023-10-08T16:13:16.551539+0000","last_became_peered":"2023-10-08T16:13:16.551539+0000","last_unstale":"2023-10-08T16:13:16.553180+0000","last_undegraded":"2023-10-08T16:13:16.553180+0000","last_fullsized":"2023-10-08T16:13:16.553180+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:04:34.219003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548911+0000","last_change":"2023-10-08T16:13:16.548248+0000","last_active":"2023-10-08T16:13:16.548911+0000","last_peered":"2023-10-08T16:13:16.548911+0000","last_clean":"2023-10-08T16:13:16.548911+0000","last_became_active":"2023-10-08T16:13:16.547988+0000","last_became_peered":"2023-10-08T16:13:16.547988+0000","last_unstale":"2023-10-08T16:13:16.548911+0000","last_undegraded":"2023-10-08T16:13:16.548911+0000","last_fullsized":"2023-10-08T16:13:16.548911+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:47.159390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562035+0000","last_change":"2023-10-08T16:13:16.555296+0000","last_active":"2023-10-08T16:13:16.562035+0000","last_peered":"2023-10-08T16:13:16.562035+0000","last_clean":"2023-10-08T16:13:16.562035+0000","last_became_active":"2023-10-08T16:13:16.555032+0000","last_became_peered":"2023-10-08T16:13:16.555032+0000","last_unstale":"2023-10-08T16:13:16.562035+0000","last_undegraded":"2023-10-08T16:13:16.562035+0000","last_fullsized":"2023-10-08T16:13:16.562035+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:55.528751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569454+0000","last_change":"2023-10-08T16:13:16.567752+0000","last_active":"2023-10-08T16:13:16.569454+0000","last_peered":"2023-10-08T16:13:16.569454+0000","last_clean":"2023-10-08T16:13:16.569454+0000","last_became_active":"2023-10-08T16:13:16.567502+0000","last_became_peered":"2023-10-08T16:13:16.567502+0000","last_unstale":"2023-10-08T16:13:16.569454+0000","last_undegraded":"2023-10-08T16:13:16.569454+0000","last_fullsized":"2023-10-08T16:13:16.569454+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:52:46.558442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569401+0000","last_change":"2023-10-08T16:13:16.568373+0000","last_active":"2023-10-08T16:13:16.569401+0000","last_peered":"2023-10-08T16:13:16.569401+0000","last_clean":"2023-10-08T16:13:16.569401+0000","last_became_active":"2023-10-08T16:13:16.568093+0000","last_became_peered":"2023-10-08T16:13:16.568093+0000","last_unstale":"2023-10-08T16:13:16.569401+0000","last_undegraded":"2023-10-08T16:13:16.569401+0000","last_fullsized":"2023-10-08T16:13:16.569401+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:06:11.873341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555334+0000","last_change":"2023-10-08T16:13:16.551555+0000","last_active":"2023-10-08T16:13:16.555334+0000","last_peered":"2023-10-08T16:13:16.555334+0000","last_clean":"2023-10-08T16:13:16.555334+0000","last_became_active":"2023-10-08T16:13:16.551300+0000","last_became_peered":"2023-10-08T16:13:16.551300+0000","last_unstale":"2023-10-08T16:13:16.555334+0000","last_undegraded":"2023-10-08T16:13:16.555334+0000","last_fullsized":"2023-10-08T16:13:16.555334+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:39:09.646662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553162+0000","last_change":"2023-10-08T16:13:16.550167+0000","last_active":"2023-10-08T16:13:16.553162+0000","last_peered":"2023-10-08T16:13:16.553162+0000","last_clean":"2023-10-08T16:13:16.553162+0000","last_became_active":"2023-10-08T16:13:16.549784+0000","last_became_peered":"2023-10-08T16:13:16.549784+0000","last_unstale":"2023-10-08T16:13:16.553162+0000","last_undegraded":"2023-10-08T16:13:16.553162+0000","last_fullsized":"2023-10-08T16:13:16.553162+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:54.173138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556272+0000","last_change":"2023-10-08T16:13:16.550674+0000","last_active":"2023-10-08T16:13:16.556272+0000","last_peered":"2023-10-08T16:13:16.556272+0000","last_clean":"2023-10-08T16:13:16.556272+0000","last_became_active":"2023-10-08T16:13:16.550425+0000","last_became_peered":"2023-10-08T16:13:16.550425+0000","last_unstale":"2023-10-08T16:13:16.556272+0000","last_undegraded":"2023-10-08T16:13:16.556272+0000","last_fullsized":"2023-10-08T16:13:16.556272+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:27.527044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548662+0000","last_change":"2023-10-08T16:13:16.547982+0000","last_active":"2023-10-08T16:13:16.548662+0000","last_peered":"2023-10-08T16:13:16.548662+0000","last_clean":"2023-10-08T16:13:16.548662+0000","last_became_active":"2023-10-08T16:13:16.547738+0000","last_became_peered":"2023-10-08T16:13:16.547738+0000","last_unstale":"2023-10-08T16:13:16.548662+0000","last_undegraded":"2023-10-08T16:13:16.548662+0000","last_fullsized":"2023-10-08T16:13:16.548662+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:03.754001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.15","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555820+0000","last_change":"2023-10-08T16:13:16.551268+0000","last_active":"2023-10-08T16:13:16.555820+0000","last_peered":"2023-10-08T16:13:16.555820+0000","last_clean":"2023-10-08T16:13:16.555820+0000","last_became_active":"2023-10-08T16:13:16.551016+0000","last_became_peered":"2023-10-08T16:13:16.551016+0000","last_unstale":"2023-10-08T16:13:16.555820+0000","last_undegraded":"2023-10-08T16:13:16.555820+0000","last_fullsized":"2023-10-08T16:13:16.555820+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:13:49.509094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.14","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566498+0000","last_change":"2023-10-08T16:13:16.562734+0000","last_active":"2023-10-08T16:13:16.566498+0000","last_peered":"2023-10-08T16:13:16.566498+0000","last_clean":"2023-10-08T16:13:16.566498+0000","last_became_active":"2023-10-08T16:13:16.562500+0000","last_became_peered":"2023-10-08T16:13:16.562500+0000","last_unstale":"2023-10-08T16:13:16.566498+0000","last_undegraded":"2023-10-08T16:13:16.566498+0000","last_fullsized":"2023-10-08T16:13:16.566498+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:17:49.339588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562899+0000","last_change":"2023-10-08T16:13:16.559278+0000","last_active":"2023-10-08T16:13:16.562899+0000","last_peered":"2023-10-08T16:13:16.562899+0000","last_clean":"2023-10-08T16:13:16.562899+0000","last_became_active":"2023-10-08T16:13:16.558971+0000","last_became_peered":"2023-10-08T16:13:16.558971+0000","last_unstale":"2023-10-08T16:13:16.562899+0000","last_undegraded":"2023-10-08T16:13:16.562899+0000","last_fullsized":"2023-10-08T16:13:16.562899+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:42:58.026061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.567122+0000","last_change":"2023-10-08T16:13:16.564642+0000","last_active":"2023-10-08T16:13:16.567122+0000","last_peered":"2023-10-08T16:13:16.567122+0000","last_clean":"2023-10-08T16:13:16.567122+0000","last_became_active":"2023-10-08T16:13:16.564291+0000","last_became_peered":"2023-10-08T16:13:16.564291+0000","last_unstale":"2023-10-08T16:13:16.567122+0000","last_undegraded":"2023-10-08T16:13:16.567122+0000","last_fullsized":"2023-10-08T16:13:16.567122+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:17:59.204686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570188+0000","last_change":"2023-10-08T16:13:16.565388+0000","last_active":"2023-10-08T16:13:16.570188+0000","last_peered":"2023-10-08T16:13:16.570188+0000","last_clean":"2023-10-08T16:13:16.570188+0000","last_became_active":"2023-10-08T16:13:16.564764+0000","last_became_peered":"2023-10-08T16:13:16.564764+0000","last_unstale":"2023-10-08T16:13:16.570188+0000","last_undegraded":"2023-10-08T16:13:16.570188+0000","last_fullsized":"2023-10-08T16:13:16.570188+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:59.844735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557305+0000","last_change":"2023-10-08T16:13:16.554189+0000","last_active":"2023-10-08T16:13:16.557305+0000","last_peered":"2023-10-08T16:13:16.557305+0000","last_clean":"2023-10-08T16:13:16.557305+0000","last_became_active":"2023-10-08T16:13:16.553922+0000","last_became_peered":"2023-10-08T16:13:16.553922+0000","last_unstale":"2023-10-08T16:13:16.557305+0000","last_undegraded":"2023-10-08T16:13:16.557305+0000","last_fullsized":"2023-10-08T16:13:16.557305+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:29.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565680+0000","last_change":"2023-10-08T16:13:16.560911+0000","last_active":"2023-10-08T16:13:16.565680+0000","last_peered":"2023-10-08T16:13:16.565680+0000","last_clean":"2023-10-08T16:13:16.565680+0000","last_became_active":"2023-10-08T16:13:16.560641+0000","last_became_peered":"2023-10-08T16:13:16.560641+0000","last_unstale":"2023-10-08T16:13:16.565680+0000","last_undegraded":"2023-10-08T16:13:16.565680+0000","last_fullsized":"2023-10-08T16:13:16.565680+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:01:02.693333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557332+0000","last_change":"2023-10-08T16:13:16.554378+0000","last_active":"2023-10-08T16:13:16.557332+0000","last_peered":"2023-10-08T16:13:16.557332+0000","last_clean":"2023-10-08T16:13:16.557332+0000","last_became_active":"2023-10-08T16:13:16.554131+0000","last_became_peered":"2023-10-08T16:13:16.554131+0000","last_unstale":"2023-10-08T16:13:16.557332+0000","last_undegraded":"2023-10-08T16:13:16.557332+0000","last_fullsized":"2023-10-08T16:13:16.557332+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:24.262518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569685+0000","last_change":"2023-10-08T16:13:16.564064+0000","last_active":"2023-10-08T16:13:16.569685+0000","last_peered":"2023-10-08T16:13:16.569685+0000","last_clean":"2023-10-08T16:13:16.569685+0000","last_became_active":"2023-10-08T16:13:16.563805+0000","last_became_peered":"2023-10-08T16:13:16.563805+0000","last_unstale":"2023-10-08T16:13:16.569685+0000","last_undegraded":"2023-10-08T16:13:16.569685+0000","last_fullsized":"2023-10-08T16:13:16.569685+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:31.043915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553082+0000","last_change":"2023-10-08T16:13:16.551375+0000","last_active":"2023-10-08T16:13:16.553082+0000","last_peered":"2023-10-08T16:13:16.553082+0000","last_clean":"2023-10-08T16:13:16.553082+0000","last_became_active":"2023-10-08T16:13:16.551128+0000","last_became_peered":"2023-10-08T16:13:16.551128+0000","last_unstale":"2023-10-08T16:13:16.553082+0000","last_undegraded":"2023-10-08T16:13:16.553082+0000","last_fullsized":"2023-10-08T16:13:16.553082+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:11:20.716305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566857+0000","last_change":"2023-10-08T16:13:16.565970+0000","last_active":"2023-10-08T16:13:16.566857+0000","last_peered":"2023-10-08T16:13:16.566857+0000","last_clean":"2023-10-08T16:13:16.566857+0000","last_became_active":"2023-10-08T16:13:16.565686+0000","last_became_peered":"2023-10-08T16:13:16.565686+0000","last_unstale":"2023-10-08T16:13:16.566857+0000","last_undegraded":"2023-10-08T16:13:16.566857+0000","last_fullsized":"2023-10-08T16:13:16.566857+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:58:03.187195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563158+0000","last_change":"2023-10-08T16:13:16.557946+0000","last_active":"2023-10-08T16:13:16.563158+0000","last_peered":"2023-10-08T16:13:16.563158+0000","last_clean":"2023-10-08T16:13:16.563158+0000","last_became_active":"2023-10-08T16:13:16.557466+0000","last_became_peered":"2023-10-08T16:13:16.557466+0000","last_unstale":"2023-10-08T16:13:16.563158+0000","last_undegraded":"2023-10-08T16:13:16.563158+0000","last_fullsized":"2023-10-08T16:13:16.563158+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:04.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570312+0000","last_change":"2023-10-08T16:13:16.567053+0000","last_active":"2023-10-08T16:13:16.570312+0000","last_peered":"2023-10-08T16:13:16.570312+0000","last_clean":"2023-10-08T16:13:16.570312+0000","last_became_active":"2023-10-08T16:13:16.565063+0000","last_became_peered":"2023-10-08T16:13:16.565063+0000","last_unstale":"2023-10-08T16:13:16.570312+0000","last_undegraded":"2023-10-08T16:13:16.570312+0000","last_fullsized":"2023-10-08T16:13:16.570312+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:13:49.509094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563018+0000","last_change":"2023-10-08T16:13:16.559282+0000","last_active":"2023-10-08T16:13:16.563018+0000","last_peered":"2023-10-08T16:13:16.563018+0000","last_clean":"2023-10-08T16:13:16.563018+0000","last_became_active":"2023-10-08T16:13:16.559059+0000","last_became_peered":"2023-10-08T16:13:16.559059+0000","last_unstale":"2023-10-08T16:13:16.563018+0000","last_undegraded":"2023-10-08T16:13:16.563018+0000","last_fullsized":"2023-10-08T16:13:16.563018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:40:06.617179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.7","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560344+0000","last_change":"2023-10-08T16:13:16.554545+0000","last_active":"2023-10-08T16:13:16.560344+0000","last_peered":"2023-10-08T16:13:16.560344+0000","last_clean":"2023-10-08T16:13:16.560344+0000","last_became_active":"2023-10-08T16:13:16.554282+0000","last_became_peered":"2023-10-08T16:13:16.554282+0000","last_unstale":"2023-10-08T16:13:16.560344+0000","last_undegraded":"2023-10-08T16:13:16.560344+0000","last_fullsized":"2023-10-08T16:13:16.560344+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:08:13.534331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558496+0000","last_change":"2023-10-08T16:13:16.553157+0000","last_active":"2023-10-08T16:13:16.558496+0000","last_peered":"2023-10-08T16:13:16.558496+0000","last_clean":"2023-10-08T16:13:16.558496+0000","last_became_active":"2023-10-08T16:13:16.552874+0000","last_became_peered":"2023-10-08T16:13:16.552874+0000","last_unstale":"2023-10-08T16:13:16.558496+0000","last_undegraded":"2023-10-08T16:13:16.558496+0000","last_fullsized":"2023-10-08T16:13:16.558496+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:05:17.479935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.564405+0000","last_change":"2023-10-08T16:13:16.555271+0000","last_active":"2023-10-08T16:13:16.564405+0000","last_peered":"2023-10-08T16:13:16.564405+0000","last_clean":"2023-10-08T16:13:16.564405+0000","last_became_active":"2023-10-08T16:13:16.554992+0000","last_became_peered":"2023-10-08T16:13:16.554992+0000","last_unstale":"2023-10-08T16:13:16.564405+0000","last_undegraded":"2023-10-08T16:13:16.564405+0000","last_fullsized":"2023-10-08T16:13:16.564405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.983164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556061+0000","last_change":"2023-10-08T16:13:16.552028+0000","last_active":"2023-10-08T16:13:16.556061+0000","last_peered":"2023-10-08T16:13:16.556061+0000","last_clean":"2023-10-08T16:13:16.556061+0000","last_became_active":"2023-10-08T16:13:16.551776+0000","last_became_peered":"2023-10-08T16:13:16.551776+0000","last_unstale":"2023-10-08T16:13:16.556061+0000","last_undegraded":"2023-10-08T16:13:16.556061+0000","last_fullsized":"2023-10-08T16:13:16.556061+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:58:57.695863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565747+0000","last_change":"2023-10-08T16:13:16.563561+0000","last_active":"2023-10-08T16:13:16.565747+0000","last_peered":"2023-10-08T16:13:16.565747+0000","last_clean":"2023-10-08T16:13:16.565747+0000","last_became_active":"2023-10-08T16:13:16.563307+0000","last_became_peered":"2023-10-08T16:13:16.563307+0000","last_unstale":"2023-10-08T16:13:16.565747+0000","last_undegraded":"2023-10-08T16:13:16.565747+0000","last_fullsized":"2023-10-08T16:13:16.565747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:12.086938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555376+0000","last_change":"2023-10-08T16:13:16.554241+0000","last_active":"2023-10-08T16:13:16.555376+0000","last_peered":"2023-10-08T16:13:16.555376+0000","last_clean":"2023-10-08T16:13:16.555376+0000","last_became_active":"2023-10-08T16:13:16.553968+0000","last_became_peered":"2023-10-08T16:13:16.553968+0000","last_unstale":"2023-10-08T16:13:16.555376+0000","last_undegraded":"2023-10-08T16:13:16.555376+0000","last_fullsized":"2023-10-08T16:13:16.555376+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:40:01.892483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569330+0000","last_change":"2023-10-08T16:13:16.567041+0000","last_active":"2023-10-08T16:13:16.569330+0000","last_peered":"2023-10-08T16:13:16.569330+0000","last_clean":"2023-10-08T16:13:16.569330+0000","last_became_active":"2023-10-08T16:13:16.566619+0000","last_became_peered":"2023-10-08T16:13:16.566619+0000","last_unstale":"2023-10-08T16:13:16.569330+0000","last_undegraded":"2023-10-08T16:13:16.569330+0000","last_fullsized":"2023-10-08T16:13:16.569330+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:08:54.402605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556489+0000","last_change":"2023-10-08T16:13:16.552990+0000","last_active":"2023-10-08T16:13:16.556489+0000","last_peered":"2023-10-08T16:13:16.556489+0000","last_clean":"2023-10-08T16:13:16.556489+0000","last_became_active":"2023-10-08T16:13:16.550586+0000","last_became_peered":"2023-10-08T16:13:16.550586+0000","last_unstale":"2023-10-08T16:13:16.556489+0000","last_undegraded":"2023-10-08T16:13:16.556489+0000","last_fullsized":"2023-10-08T16:13:16.556489+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:24.535995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.20","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.552993+0000","last_change":"2023-10-08T16:13:16.550927+0000","last_active":"2023-10-08T16:13:16.552993+0000","last_peered":"2023-10-08T16:13:16.552993+0000","last_clean":"2023-10-08T16:13:16.552993+0000","last_became_active":"2023-10-08T16:13:16.550648+0000","last_became_peered":"2023-10-08T16:13:16.550648+0000","last_unstale":"2023-10-08T16:13:16.552993+0000","last_undegraded":"2023-10-08T16:13:16.552993+0000","last_fullsized":"2023-10-08T16:13:16.552993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:23:21.904163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1f","version":"183'4","reported_seq":14,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.571014+0000","last_change":"2023-10-08T16:13:15.538049+0000","last_active":"2023-10-08T16:13:16.571014+0000","last_peered":"2023-10-08T16:13:16.571014+0000","last_clean":"2023-10-08T16:13:16.571014+0000","last_became_active":"2023-10-08T16:13:15.537805+0000","last_became_peered":"2023-10-08T16:13:15.537805+0000","last_unstale":"2023-10-08T16:13:16.571014+0000","last_undegraded":"2023-10-08T16:13:16.571014+0000","last_fullsized":"2023-10-08T16:13:16.571014+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:28.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569718+0000","last_change":"2023-10-08T16:13:16.564952+0000","last_active":"2023-10-08T16:13:16.569718+0000","last_peered":"2023-10-08T16:13:16.569718+0000","last_clean":"2023-10-08T16:13:16.569718+0000","last_became_active":"2023-10-08T16:13:16.564629+0000","last_became_peered":"2023-10-08T16:13:16.564629+0000","last_unstale":"2023-10-08T16:13:16.569718+0000","last_undegraded":"2023-10-08T16:13:16.569718+0000","last_fullsized":"2023-10-08T16:13:16.569718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:03.754001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.1d","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570321+0000","last_change":"2023-10-08T16:13:15.538058+0000","last_active":"2023-10-08T16:13:16.570321+0000","last_peered":"2023-10-08T16:13:16.570321+0000","last_clean":"2023-10-08T16:13:16.570321+0000","last_became_active":"2023-10-08T16:13:15.537799+0000","last_became_peered":"2023-10-08T16:13:15.537799+0000","last_unstale":"2023-10-08T16:13:16.570321+0000","last_undegraded":"2023-10-08T16:13:16.570321+0000","last_fullsized":"2023-10-08T16:13:16.570321+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:34:25.539622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.23","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569442+0000","last_change":"2023-10-08T16:13:16.568411+0000","last_active":"2023-10-08T16:13:16.569442+0000","last_peered":"2023-10-08T16:13:16.569442+0000","last_clean":"2023-10-08T16:13:16.569442+0000","last_became_active":"2023-10-08T16:13:16.568150+0000","last_became_peered":"2023-10-08T16:13:16.568150+0000","last_unstale":"2023-10-08T16:13:16.569442+0000","last_undegraded":"2023-10-08T16:13:16.569442+0000","last_fullsized":"2023-10-08T16:13:16.569442+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:16:41.024028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1c","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561087+0000","last_change":"2023-10-08T16:13:15.538309+0000","last_active":"2023-10-08T16:13:16.561087+0000","last_peered":"2023-10-08T16:13:16.561087+0000","last_clean":"2023-10-08T16:13:16.561087+0000","last_became_active":"2023-10-08T16:13:15.538044+0000","last_became_peered":"2023-10-08T16:13:15.538044+0000","last_unstale":"2023-10-08T16:13:16.561087+0000","last_undegraded":"2023-10-08T16:13:16.561087+0000","last_fullsized":"2023-10-08T16:13:16.561087+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:23.260165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.24","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.556769+0000","last_change":"2023-10-08T16:13:16.554472+0000","last_active":"2023-10-08T16:13:16.556769+0000","last_peered":"2023-10-08T16:13:16.556769+0000","last_clean":"2023-10-08T16:13:16.556769+0000","last_became_active":"2023-10-08T16:13:16.554202+0000","last_became_peered":"2023-10-08T16:13:16.554202+0000","last_unstale":"2023-10-08T16:13:16.556769+0000","last_undegraded":"2023-10-08T16:13:16.556769+0000","last_fullsized":"2023-10-08T16:13:16.556769+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:54.751498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1b","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.564973+0000","last_change":"2023-10-08T16:13:15.536795+0000","last_active":"2023-10-08T16:13:16.564973+0000","last_peered":"2023-10-08T16:13:16.564973+0000","last_clean":"2023-10-08T16:13:16.564973+0000","last_became_active":"2023-10-08T16:13:15.536333+0000","last_became_peered":"2023-10-08T16:13:15.536333+0000","last_unstale":"2023-10-08T16:13:16.564973+0000","last_undegraded":"2023-10-08T16:13:16.564973+0000","last_fullsized":"2023-10-08T16:13:16.564973+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:55:31.187466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.25","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566921+0000","last_change":"2023-10-08T16:13:16.565858+0000","last_active":"2023-10-08T16:13:16.566921+0000","last_peered":"2023-10-08T16:13:16.566921+0000","last_clean":"2023-10-08T16:13:16.566921+0000","last_became_active":"2023-10-08T16:13:16.565571+0000","last_became_peered":"2023-10-08T16:13:16.565571+0000","last_unstale":"2023-10-08T16:13:16.566921+0000","last_undegraded":"2023-10-08T16:13:16.566921+0000","last_fullsized":"2023-10-08T16:13:16.566921+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:29:27.201767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1a","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560739+0000","last_change":"2023-10-08T16:13:15.539561+0000","last_active":"2023-10-08T16:13:16.560739+0000","last_peered":"2023-10-08T16:13:16.560739+0000","last_clean":"2023-10-08T16:13:16.560739+0000","last_became_active":"2023-10-08T16:13:15.539354+0000","last_became_peered":"2023-10-08T16:13:15.539354+0000","last_unstale":"2023-10-08T16:13:16.560739+0000","last_undegraded":"2023-10-08T16:13:16.560739+0000","last_fullsized":"2023-10-08T16:13:16.560739+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:30.041562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.26","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565002+0000","last_change":"2023-10-08T16:13:16.562687+0000","last_active":"2023-10-08T16:13:16.565002+0000","last_peered":"2023-10-08T16:13:16.565002+0000","last_clean":"2023-10-08T16:13:16.565002+0000","last_became_active":"2023-10-08T16:13:16.562424+0000","last_became_peered":"2023-10-08T16:13:16.562424+0000","last_unstale":"2023-10-08T16:13:16.565002+0000","last_undegraded":"2023-10-08T16:13:16.565002+0000","last_fullsized":"2023-10-08T16:13:16.565002+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:23:52.891279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.537897+0000","last_change":"2023-10-08T16:13:15.533033+0000","last_active":"2023-10-08T16:13:16.537897+0000","last_peered":"2023-10-08T16:13:16.537897+0000","last_clean":"2023-10-08T16:13:16.537897+0000","last_became_active":"2023-10-08T16:13:15.532787+0000","last_became_peered":"2023-10-08T16:13:15.532787+0000","last_unstale":"2023-10-08T16:13:16.537897+0000","last_undegraded":"2023-10-08T16:13:16.537897+0000","last_fullsized":"2023-10-08T16:13:16.537897+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:09.787465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.27","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562010+0000","last_change":"2023-10-08T16:13:16.555488+0000","last_active":"2023-10-08T16:13:16.562010+0000","last_peered":"2023-10-08T16:13:16.562010+0000","last_clean":"2023-10-08T16:13:16.562010+0000","last_became_active":"2023-10-08T16:13:16.555204+0000","last_became_peered":"2023-10-08T16:13:16.555204+0000","last_unstale":"2023-10-08T16:13:16.562010+0000","last_undegraded":"2023-10-08T16:13:16.562010+0000","last_fullsized":"2023-10-08T16:13:16.562010+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:27:45.695292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.541031+0000","last_change":"2023-10-08T16:13:15.539216+0000","last_active":"2023-10-08T16:13:16.541031+0000","last_peered":"2023-10-08T16:13:16.541031+0000","last_clean":"2023-10-08T16:13:16.541031+0000","last_became_active":"2023-10-08T16:13:15.538974+0000","last_became_peered":"2023-10-08T16:13:15.538974+0000","last_unstale":"2023-10-08T16:13:16.541031+0000","last_undegraded":"2023-10-08T16:13:16.541031+0000","last_fullsized":"2023-10-08T16:13:16.541031+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:30.041562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.28","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566874+0000","last_change":"2023-10-08T16:13:16.566086+0000","last_active":"2023-10-08T16:13:16.566874+0000","last_peered":"2023-10-08T16:13:16.566874+0000","last_clean":"2023-10-08T16:13:16.566874+0000","last_became_active":"2023-10-08T16:13:16.565512+0000","last_became_peered":"2023-10-08T16:13:16.565512+0000","last_unstale":"2023-10-08T16:13:16.566874+0000","last_undegraded":"2023-10-08T16:13:16.566874+0000","last_fullsized":"2023-10-08T16:13:16.566874+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:11:23.626656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550149+0000","last_change":"2023-10-08T16:13:15.539170+0000","last_active":"2023-10-08T16:13:16.550149+0000","last_peered":"2023-10-08T16:13:16.550149+0000","last_clean":"2023-10-08T16:13:16.550149+0000","last_became_active":"2023-10-08T16:13:15.538882+0000","last_became_peered":"2023-10-08T16:13:15.538882+0000","last_unstale":"2023-10-08T16:13:16.550149+0000","last_undegraded":"2023-10-08T16:13:16.550149+0000","last_fullsized":"2023-10-08T16:13:16.550149+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:15.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.29","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562977+0000","last_change":"2023-10-08T16:13:16.558580+0000","last_active":"2023-10-08T16:13:16.562977+0000","last_peered":"2023-10-08T16:13:16.562977+0000","last_clean":"2023-10-08T16:13:16.562977+0000","last_became_active":"2023-10-08T16:13:16.557017+0000","last_became_peered":"2023-10-08T16:13:16.557017+0000","last_unstale":"2023-10-08T16:13:16.562977+0000","last_undegraded":"2023-10-08T16:13:16.562977+0000","last_fullsized":"2023-10-08T16:13:16.562977+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:27:45.695292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.538227+0000","last_change":"2023-10-08T16:13:15.527932+0000","last_active":"2023-10-08T16:13:16.538227+0000","last_peered":"2023-10-08T16:13:16.538227+0000","last_clean":"2023-10-08T16:13:16.538227+0000","last_became_active":"2023-10-08T16:13:15.527433+0000","last_became_peered":"2023-10-08T16:13:15.527433+0000","last_unstale":"2023-10-08T16:13:16.538227+0000","last_undegraded":"2023-10-08T16:13:16.538227+0000","last_fullsized":"2023-10-08T16:13:16.538227+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:09:04.321176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557156+0000","last_change":"2023-10-08T16:13:16.551333+0000","last_active":"2023-10-08T16:13:16.557156+0000","last_peered":"2023-10-08T16:13:16.557156+0000","last_clean":"2023-10-08T16:13:16.557156+0000","last_became_active":"2023-10-08T16:13:16.551079+0000","last_became_peered":"2023-10-08T16:13:16.551079+0000","last_unstale":"2023-10-08T16:13:16.557156+0000","last_undegraded":"2023-10-08T16:13:16.557156+0000","last_fullsized":"2023-10-08T16:13:16.557156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:54:31.043915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.15","version":"183'3","reported_seq":13,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570560+0000","last_change":"2023-10-08T16:13:15.536108+0000","last_active":"2023-10-08T16:13:16.570560+0000","last_peered":"2023-10-08T16:13:16.570560+0000","last_clean":"2023-10-08T16:13:16.570560+0000","last_became_active":"2023-10-08T16:13:15.535792+0000","last_became_peered":"2023-10-08T16:13:15.535792+0000","last_unstale":"2023-10-08T16:13:16.570560+0000","last_undegraded":"2023-10-08T16:13:16.570560+0000","last_fullsized":"2023-10-08T16:13:16.570560+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:43.753200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555369+0000","last_change":"2023-10-08T16:13:16.547868+0000","last_active":"2023-10-08T16:13:16.555369+0000","last_peered":"2023-10-08T16:13:16.555369+0000","last_clean":"2023-10-08T16:13:16.555369+0000","last_became_active":"2023-10-08T16:13:16.547513+0000","last_became_peered":"2023-10-08T16:13:16.547513+0000","last_unstale":"2023-10-08T16:13:16.555369+0000","last_undegraded":"2023-10-08T16:13:16.555369+0000","last_fullsized":"2023-10-08T16:13:16.555369+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:45.576385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.14","version":"183'6","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570662+0000","last_change":"2023-10-08T16:13:15.538790+0000","last_active":"2023-10-08T16:13:16.570662+0000","last_peered":"2023-10-08T16:13:16.570662+0000","last_clean":"2023-10-08T16:13:16.570662+0000","last_became_active":"2023-10-08T16:13:15.538555+0000","last_became_peered":"2023-10-08T16:13:15.538555+0000","last_unstale":"2023-10-08T16:13:16.570662+0000","last_undegraded":"2023-10-08T16:13:16.570662+0000","last_fullsized":"2023-10-08T16:13:16.570662+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:44:02.751647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550939+0000","last_change":"2023-10-08T16:13:16.547943+0000","last_active":"2023-10-08T16:13:16.550939+0000","last_peered":"2023-10-08T16:13:16.550939+0000","last_clean":"2023-10-08T16:13:16.550939+0000","last_became_active":"2023-10-08T16:13:16.547653+0000","last_became_peered":"2023-10-08T16:13:16.547653+0000","last_unstale":"2023-10-08T16:13:16.550939+0000","last_undegraded":"2023-10-08T16:13:16.550939+0000","last_fullsized":"2023-10-08T16:13:16.550939+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:52:21.617690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.13","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565487+0000","last_change":"2023-10-08T16:13:15.539204+0000","last_active":"2023-10-08T16:13:16.565487+0000","last_peered":"2023-10-08T16:13:16.565487+0000","last_clean":"2023-10-08T16:13:16.565487+0000","last_became_active":"2023-10-08T16:13:15.538927+0000","last_became_peered":"2023-10-08T16:13:15.538927+0000","last_unstale":"2023-10-08T16:13:16.565487+0000","last_undegraded":"2023-10-08T16:13:16.565487+0000","last_fullsized":"2023-10-08T16:13:16.565487+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:30.169334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560718+0000","last_change":"2023-10-08T16:13:16.558667+0000","last_active":"2023-10-08T16:13:16.560718+0000","last_peered":"2023-10-08T16:13:16.560718+0000","last_clean":"2023-10-08T16:13:16.560718+0000","last_became_active":"2023-10-08T16:13:16.558274+0000","last_became_peered":"2023-10-08T16:13:16.558274+0000","last_unstale":"2023-10-08T16:13:16.560718+0000","last_undegraded":"2023-10-08T16:13:16.560718+0000","last_fullsized":"2023-10-08T16:13:16.560718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:14:46.483873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548071+0000","last_change":"2023-10-08T16:13:15.536117+0000","last_active":"2023-10-08T16:13:16.548071+0000","last_peered":"2023-10-08T16:13:16.548071+0000","last_clean":"2023-10-08T16:13:16.548071+0000","last_became_active":"2023-10-08T16:13:15.535753+0000","last_became_peered":"2023-10-08T16:13:15.535753+0000","last_unstale":"2023-10-08T16:13:16.548071+0000","last_undegraded":"2023-10-08T16:13:16.548071+0000","last_fullsized":"2023-10-08T16:13:16.548071+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:13:31.639348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570257+0000","last_change":"2023-10-08T16:13:16.566115+0000","last_active":"2023-10-08T16:13:16.570257+0000","last_peered":"2023-10-08T16:13:16.570257+0000","last_clean":"2023-10-08T16:13:16.570257+0000","last_became_active":"2023-10-08T16:13:16.565853+0000","last_became_peered":"2023-10-08T16:13:16.565853+0000","last_unstale":"2023-10-08T16:13:16.570257+0000","last_undegraded":"2023-10-08T16:13:16.570257+0000","last_fullsized":"2023-10-08T16:13:16.570257+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:24.262518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548029+0000","last_change":"2023-10-08T16:13:15.538483+0000","last_active":"2023-10-08T16:13:16.548029+0000","last_peered":"2023-10-08T16:13:16.548029+0000","last_clean":"2023-10-08T16:13:16.548029+0000","last_became_active":"2023-10-08T16:13:15.538170+0000","last_became_peered":"2023-10-08T16:13:15.538170+0000","last_unstale":"2023-10-08T16:13:16.548029+0000","last_undegraded":"2023-10-08T16:13:16.548029+0000","last_fullsized":"2023-10-08T16:13:16.548029+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:56:39.503047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.2f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561987+0000","last_change":"2023-10-08T16:13:16.553779+0000","last_active":"2023-10-08T16:13:16.561987+0000","last_peered":"2023-10-08T16:13:16.561987+0000","last_clean":"2023-10-08T16:13:16.561987+0000","last_became_active":"2023-10-08T16:13:16.553452+0000","last_became_peered":"2023-10-08T16:13:16.553452+0000","last_unstale":"2023-10-08T16:13:16.561987+0000","last_undegraded":"2023-10-08T16:13:16.561987+0000","last_fullsized":"2023-10-08T16:13:16.561987+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:55:10.789818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.10","version":"183'8","reported_seq":18,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569350+0000","last_change":"2023-10-08T16:13:15.541913+0000","last_active":"2023-10-08T16:13:16.569350+0000","last_peered":"2023-10-08T16:13:16.569350+0000","last_clean":"2023-10-08T16:13:16.569350+0000","last_became_active":"2023-10-08T16:13:15.541623+0000","last_became_peered":"2023-10-08T16:13:15.541623+0000","last_unstale":"2023-10-08T16:13:16.569350+0000","last_undegraded":"2023-10-08T16:13:16.569350+0000","last_fullsized":"2023-10-08T16:13:16.569350+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:07:08.368910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.30","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558813+0000","last_change":"2023-10-08T16:13:16.553698+0000","last_active":"2023-10-08T16:13:16.558813+0000","last_peered":"2023-10-08T16:13:16.558813+0000","last_clean":"2023-10-08T16:13:16.558813+0000","last_became_active":"2023-10-08T16:13:16.553415+0000","last_became_peered":"2023-10-08T16:13:16.553415+0000","last_unstale":"2023-10-08T16:13:16.558813+0000","last_undegraded":"2023-10-08T16:13:16.558813+0000","last_fullsized":"2023-10-08T16:13:16.558813+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:53:39.823950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.f","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.569509+0000","last_change":"2023-10-08T16:13:15.538309+0000","last_active":"2023-10-08T16:13:16.569509+0000","last_peered":"2023-10-08T16:13:16.569509+0000","last_clean":"2023-10-08T16:13:16.569509+0000","last_became_active":"2023-10-08T16:13:15.538022+0000","last_became_peered":"2023-10-08T16:13:15.538022+0000","last_unstale":"2023-10-08T16:13:16.569509+0000","last_undegraded":"2023-10-08T16:13:16.569509+0000","last_fullsized":"2023-10-08T16:13:16.569509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:58.842381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568318+0000","last_change":"2023-10-08T16:13:16.562448+0000","last_active":"2023-10-08T16:13:16.568318+0000","last_peered":"2023-10-08T16:13:16.568318+0000","last_clean":"2023-10-08T16:13:16.568318+0000","last_became_active":"2023-10-08T16:13:16.562183+0000","last_became_peered":"2023-10-08T16:13:16.562183+0000","last_unstale":"2023-10-08T16:13:16.568318+0000","last_undegraded":"2023-10-08T16:13:16.568318+0000","last_fullsized":"2023-10-08T16:13:16.568318+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:04.655263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.e","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.567277+0000","last_change":"2023-10-08T16:13:15.533749+0000","last_active":"2023-10-08T16:13:16.567277+0000","last_peered":"2023-10-08T16:13:16.567277+0000","last_clean":"2023-10-08T16:13:16.567277+0000","last_became_active":"2023-10-08T16:13:15.533490+0000","last_became_peered":"2023-10-08T16:13:15.533490+0000","last_unstale":"2023-10-08T16:13:16.567277+0000","last_undegraded":"2023-10-08T16:13:16.567277+0000","last_fullsized":"2023-10-08T16:13:16.567277+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:17:15.407146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560352+0000","last_change":"2023-10-08T16:13:16.553594+0000","last_active":"2023-10-08T16:13:16.560352+0000","last_peered":"2023-10-08T16:13:16.560352+0000","last_clean":"2023-10-08T16:13:16.560352+0000","last_became_active":"2023-10-08T16:13:16.553338+0000","last_became_peered":"2023-10-08T16:13:16.553338+0000","last_unstale":"2023-10-08T16:13:16.560352+0000","last_undegraded":"2023-10-08T16:13:16.560352+0000","last_fullsized":"2023-10-08T16:13:16.560352+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:43:40.569851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.d","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570436+0000","last_change":"2023-10-08T16:13:15.537983+0000","last_active":"2023-10-08T16:13:16.570436+0000","last_peered":"2023-10-08T16:13:16.570436+0000","last_clean":"2023-10-08T16:13:16.570436+0000","last_became_active":"2023-10-08T16:13:15.537711+0000","last_became_peered":"2023-10-08T16:13:15.537711+0000","last_unstale":"2023-10-08T16:13:16.570436+0000","last_undegraded":"2023-10-08T16:13:16.570436+0000","last_fullsized":"2023-10-08T16:13:16.570436+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:06:55.761887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.33","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557275+0000","last_change":"2023-10-08T16:13:16.555038+0000","last_active":"2023-10-08T16:13:16.557275+0000","last_peered":"2023-10-08T16:13:16.557275+0000","last_clean":"2023-10-08T16:13:16.557275+0000","last_became_active":"2023-10-08T16:13:16.554628+0000","last_became_peered":"2023-10-08T16:13:16.554628+0000","last_unstale":"2023-10-08T16:13:16.557275+0000","last_undegraded":"2023-10-08T16:13:16.557275+0000","last_fullsized":"2023-10-08T16:13:16.557275+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:26:59.844735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553577+0000","last_change":"2023-10-08T16:13:15.533718+0000","last_active":"2023-10-08T16:13:16.553577+0000","last_peered":"2023-10-08T16:13:16.553577+0000","last_clean":"2023-10-08T16:13:16.553577+0000","last_became_active":"2023-10-08T16:13:15.533383+0000","last_became_peered":"2023-10-08T16:13:15.533383+0000","last_unstale":"2023-10-08T16:13:16.553577+0000","last_undegraded":"2023-10-08T16:13:16.553577+0000","last_fullsized":"2023-10-08T16:13:16.553577+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:34:38.210131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.34","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.562015+0000","last_change":"2023-10-08T16:13:16.552068+0000","last_active":"2023-10-08T16:13:16.562015+0000","last_peered":"2023-10-08T16:13:16.562015+0000","last_clean":"2023-10-08T16:13:16.562015+0000","last_became_active":"2023-10-08T16:13:16.551610+0000","last_became_peered":"2023-10-08T16:13:16.551610+0000","last_unstale":"2023-10-08T16:13:16.562015+0000","last_undegraded":"2023-10-08T16:13:16.562015+0000","last_fullsized":"2023-10-08T16:13:16.562015+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:16.578672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.b","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568720+0000","last_change":"2023-10-08T16:13:15.541240+0000","last_active":"2023-10-08T16:13:16.568720+0000","last_peered":"2023-10-08T16:13:16.568720+0000","last_clean":"2023-10-08T16:13:16.568720+0000","last_became_active":"2023-10-08T16:13:15.541003+0000","last_became_peered":"2023-10-08T16:13:15.541003+0000","last_unstale":"2023-10-08T16:13:16.568720+0000","last_undegraded":"2023-10-08T16:13:16.568720+0000","last_fullsized":"2023-10-08T16:13:16.568720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:10.062423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.35","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550728+0000","last_change":"2023-10-08T16:13:16.547862+0000","last_active":"2023-10-08T16:13:16.550728+0000","last_peered":"2023-10-08T16:13:16.550728+0000","last_clean":"2023-10-08T16:13:16.550728+0000","last_became_active":"2023-10-08T16:13:16.547591+0000","last_became_peered":"2023-10-08T16:13:16.547591+0000","last_unstale":"2023-10-08T16:13:16.550728+0000","last_undegraded":"2023-10-08T16:13:16.550728+0000","last_fullsized":"2023-10-08T16:13:16.550728+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:45.933015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557302+0000","last_change":"2023-10-08T16:13:15.534483+0000","last_active":"2023-10-08T16:13:16.557302+0000","last_peered":"2023-10-08T16:13:16.557302+0000","last_clean":"2023-10-08T16:13:16.557302+0000","last_became_active":"2023-10-08T16:13:15.534030+0000","last_became_peered":"2023-10-08T16:13:15.534030+0000","last_unstale":"2023-10-08T16:13:16.557302+0000","last_undegraded":"2023-10-08T16:13:16.557302+0000","last_fullsized":"2023-10-08T16:13:16.557302+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:48:43.579186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563058+0000","last_change":"2023-10-08T16:13:16.557905+0000","last_active":"2023-10-08T16:13:16.563058+0000","last_peered":"2023-10-08T16:13:16.563058+0000","last_clean":"2023-10-08T16:13:16.563058+0000","last_became_active":"2023-10-08T16:13:16.557397+0000","last_became_peered":"2023-10-08T16:13:16.557397+0000","last_unstale":"2023-10-08T16:13:16.563058+0000","last_undegraded":"2023-10-08T16:13:16.563058+0000","last_fullsized":"2023-10-08T16:13:16.563058+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:16:22.299401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558271+0000","last_change":"2023-10-08T16:13:15.535401+0000","last_active":"2023-10-08T16:13:16.558271+0000","last_peered":"2023-10-08T16:13:16.558271+0000","last_clean":"2023-10-08T16:13:16.558271+0000","last_became_active":"2023-10-08T16:13:15.535154+0000","last_became_peered":"2023-10-08T16:13:15.535154+0000","last_unstale":"2023-10-08T16:13:16.558271+0000","last_undegraded":"2023-10-08T16:13:16.558271+0000","last_fullsized":"2023-10-08T16:13:16.558271+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:12:37.367041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.37","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.568328+0000","last_change":"2023-10-08T16:13:16.566092+0000","last_active":"2023-10-08T16:13:16.568328+0000","last_peered":"2023-10-08T16:13:16.568328+0000","last_clean":"2023-10-08T16:13:16.568328+0000","last_became_active":"2023-10-08T16:13:16.565832+0000","last_became_peered":"2023-10-08T16:13:16.565832+0000","last_unstale":"2023-10-08T16:13:16.568328+0000","last_undegraded":"2023-10-08T16:13:16.568328+0000","last_fullsized":"2023-10-08T16:13:16.568328+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:02:54.584641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.549842+0000","last_change":"2023-10-08T16:13:15.535186+0000","last_active":"2023-10-08T16:13:16.549842+0000","last_peered":"2023-10-08T16:13:16.549842+0000","last_clean":"2023-10-08T16:13:16.549842+0000","last_became_active":"2023-10-08T16:13:15.534944+0000","last_became_peered":"2023-10-08T16:13:15.534944+0000","last_unstale":"2023-10-08T16:13:16.549842+0000","last_undegraded":"2023-10-08T16:13:16.549842+0000","last_fullsized":"2023-10-08T16:13:16.549842+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:06:36.509663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555232+0000","last_change":"2023-10-08T16:13:16.552557+0000","last_active":"2023-10-08T16:13:16.555232+0000","last_peered":"2023-10-08T16:13:16.555232+0000","last_clean":"2023-10-08T16:13:16.555232+0000","last_became_active":"2023-10-08T16:13:16.552289+0000","last_became_peered":"2023-10-08T16:13:16.552289+0000","last_unstale":"2023-10-08T16:13:16.555232+0000","last_undegraded":"2023-10-08T16:13:16.555232+0000","last_fullsized":"2023-10-08T16:13:16.555232+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:33:28.093263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.7","version":"183'2","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.563806+0000","last_change":"2023-10-08T16:13:15.534638+0000","last_active":"2023-10-08T16:13:16.563806+0000","last_peered":"2023-10-08T16:13:16.563806+0000","last_clean":"2023-10-08T16:13:16.563806+0000","last_became_active":"2023-10-08T16:13:15.534379+0000","last_became_peered":"2023-10-08T16:13:15.534379+0000","last_unstale":"2023-10-08T16:13:16.563806+0000","last_undegraded":"2023-10-08T16:13:16.563806+0000","last_fullsized":"2023-10-08T16:13:16.563806+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:30:38.851040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.39","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.570167+0000","last_change":"2023-10-08T16:13:16.565167+0000","last_active":"2023-10-08T16:13:16.570167+0000","last_peered":"2023-10-08T16:13:16.570167+0000","last_clean":"2023-10-08T16:13:16.570167+0000","last_became_active":"2023-10-08T16:13:16.564444+0000","last_became_peered":"2023-10-08T16:13:16.564444+0000","last_unstale":"2023-10-08T16:13:16.570167+0000","last_undegraded":"2023-10-08T16:13:16.570167+0000","last_fullsized":"2023-10-08T16:13:16.570167+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:29.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.547844+0000","last_change":"2023-10-08T16:13:15.535192+0000","last_active":"2023-10-08T16:13:16.547844+0000","last_peered":"2023-10-08T16:13:16.547844+0000","last_clean":"2023-10-08T16:13:16.547844+0000","last_became_active":"2023-10-08T16:13:15.534919+0000","last_became_peered":"2023-10-08T16:13:15.534919+0000","last_unstale":"2023-10-08T16:13:16.547844+0000","last_undegraded":"2023-10-08T16:13:16.547844+0000","last_fullsized":"2023-10-08T16:13:16.547844+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:04:50.340346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.566779+0000","last_change":"2023-10-08T16:13:16.565986+0000","last_active":"2023-10-08T16:13:16.566779+0000","last_peered":"2023-10-08T16:13:16.566779+0000","last_clean":"2023-10-08T16:13:16.566779+0000","last_became_active":"2023-10-08T16:13:16.565721+0000","last_became_peered":"2023-10-08T16:13:16.565721+0000","last_unstale":"2023-10-08T16:13:16.566779+0000","last_undegraded":"2023-10-08T16:13:16.566779+0000","last_fullsized":"2023-10-08T16:13:16.566779+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:06:37.512016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550551+0000","last_change":"2023-10-08T16:13:15.539117+0000","last_active":"2023-10-08T16:13:16.550551+0000","last_peered":"2023-10-08T16:13:16.550551+0000","last_clean":"2023-10-08T16:13:16.550551+0000","last_became_active":"2023-10-08T16:13:15.538916+0000","last_became_peered":"2023-10-08T16:13:15.538916+0000","last_unstale":"2023-10-08T16:13:16.550551+0000","last_undegraded":"2023-10-08T16:13:16.550551+0000","last_fullsized":"2023-10-08T16:13:16.550551+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:12:28.665094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.561631+0000","last_change":"2023-10-08T16:13:16.553031+0000","last_active":"2023-10-08T16:13:16.561631+0000","last_peered":"2023-10-08T16:13:16.561631+0000","last_clean":"2023-10-08T16:13:16.561631+0000","last_became_active":"2023-10-08T16:13:16.552711+0000","last_became_peered":"2023-10-08T16:13:16.552711+0000","last_unstale":"2023-10-08T16:13:16.561631+0000","last_undegraded":"2023-10-08T16:13:16.561631+0000","last_fullsized":"2023-10-08T16:13:16.561631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:33:34.713664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.550454+0000","last_change":"2023-10-08T16:13:15.536795+0000","last_active":"2023-10-08T16:13:16.550454+0000","last_peered":"2023-10-08T16:13:16.550454+0000","last_clean":"2023-10-08T16:13:16.550454+0000","last_became_active":"2023-10-08T16:13:15.536369+0000","last_became_peered":"2023-10-08T16:13:15.536369+0000","last_unstale":"2023-10-08T16:13:16.550454+0000","last_undegraded":"2023-10-08T16:13:16.550454+0000","last_fullsized":"2023-10-08T16:13:16.550454+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:42:46.157037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.3c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.558065+0000","last_change":"2023-10-08T16:13:16.556870+0000","last_active":"2023-10-08T16:13:16.558065+0000","last_peered":"2023-10-08T16:13:16.558065+0000","last_clean":"2023-10-08T16:13:16.558065+0000","last_became_active":"2023-10-08T16:13:16.556612+0000","last_became_peered":"2023-10-08T16:13:16.556612+0000","last_unstale":"2023-10-08T16:13:16.558065+0000","last_undegraded":"2023-10-08T16:13:16.558065+0000","last_fullsized":"2023-10-08T16:13:16.558065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:06:43.757943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557990+0000","last_change":"2023-10-08T16:13:15.533238+0000","last_active":"2023-10-08T16:13:16.557990+0000","last_peered":"2023-10-08T16:13:16.557990+0000","last_clean":"2023-10-08T16:13:16.557990+0000","last_became_active":"2023-10-08T16:13:15.532936+0000","last_became_peered":"2023-10-08T16:13:15.532936+0000","last_unstale":"2023-10-08T16:13:16.557990+0000","last_undegraded":"2023-10-08T16:13:16.557990+0000","last_fullsized":"2023-10-08T16:13:16.557990+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:30:48.274831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.3d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.559279+0000","last_change":"2023-10-08T16:13:16.558818+0000","last_active":"2023-10-08T16:13:16.559279+0000","last_peered":"2023-10-08T16:13:16.559279+0000","last_clean":"2023-10-08T16:13:16.559279+0000","last_became_active":"2023-10-08T16:13:16.558541+0000","last_became_peered":"2023-10-08T16:13:16.558541+0000","last_unstale":"2023-10-08T16:13:16.559279+0000","last_undegraded":"2023-10-08T16:13:16.559279+0000","last_fullsized":"2023-10-08T16:13:16.559279+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:32:42.833366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.548177+0000","last_change":"2023-10-08T16:13:15.532942+0000","last_active":"2023-10-08T16:13:16.548177+0000","last_peered":"2023-10-08T16:13:16.548177+0000","last_clean":"2023-10-08T16:13:16.548177+0000","last_became_active":"2023-10-08T16:13:15.532657+0000","last_became_peered":"2023-10-08T16:13:15.532657+0000","last_unstale":"2023-10-08T16:13:16.548177+0000","last_undegraded":"2023-10-08T16:13:16.548177+0000","last_fullsized":"2023-10-08T16:13:16.548177+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:34:25.539622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.3e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.553163+0000","last_change":"2023-10-08T16:13:16.551958+0000","last_active":"2023-10-08T16:13:16.553163+0000","last_peered":"2023-10-08T16:13:16.553163+0000","last_clean":"2023-10-08T16:13:16.553163+0000","last_became_active":"2023-10-08T16:13:16.550627+0000","last_became_peered":"2023-10-08T16:13:16.550627+0000","last_unstale":"2023-10-08T16:13:16.553163+0000","last_undegraded":"2023-10-08T16:13:16.553163+0000","last_fullsized":"2023-10-08T16:13:16.553163+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:07:34.832697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.537676+0000","last_change":"2023-10-08T16:13:15.527840+0000","last_active":"2023-10-08T16:13:16.537676+0000","last_peered":"2023-10-08T16:13:16.537676+0000","last_clean":"2023-10-08T16:13:16.537676+0000","last_became_active":"2023-10-08T16:13:15.527584+0000","last_became_peered":"2023-10-08T16:13:15.527584+0000","last_unstale":"2023-10-08T16:13:16.537676+0000","last_undegraded":"2023-10-08T16:13:16.537676+0000","last_fullsized":"2023-10-08T16:13:16.537676+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:53.170785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.3f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.565660+0000","last_change":"2023-10-08T16:13:16.560839+0000","last_active":"2023-10-08T16:13:16.565660+0000","last_peered":"2023-10-08T16:13:16.565660+0000","last_clean":"2023-10-08T16:13:16.565660+0000","last_became_active":"2023-10-08T16:13:16.560535+0000","last_became_peered":"2023-10-08T16:13:16.560535+0000","last_unstale":"2023-10-08T16:13:16.565660+0000","last_undegraded":"2023-10-08T16:13:16.565660+0000","last_fullsized":"2023-10-08T16:13:16.565660+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:32:33.983164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.557267+0000","last_change":"2023-10-08T16:13:15.528041+0000","last_active":"2023-10-08T16:13:16.557267+0000","last_peered":"2023-10-08T16:13:16.557267+0000","last_clean":"2023-10-08T16:13:16.557267+0000","last_became_active":"2023-10-08T16:13:15.527784+0000","last_became_peered":"2023-10-08T16:13:15.527784+0000","last_unstale":"2023-10-08T16:13:16.557267+0000","last_undegraded":"2023-10-08T16:13:16.557267+0000","last_fullsized":"2023-10-08T16:13:16.557267+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:01:00.618426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.21","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.555711+0000","last_change":"2023-10-08T16:13:16.549639+0000","last_active":"2023-10-08T16:13:16.555711+0000","last_peered":"2023-10-08T16:13:16.555711+0000","last_clean":"2023-10-08T16:13:16.555711+0000","last_became_active":"2023-10-08T16:13:16.548935+0000","last_became_peered":"2023-10-08T16:13:16.548935+0000","last_unstale":"2023-10-08T16:13:16.555711+0000","last_undegraded":"2023-10-08T16:13:16.555711+0000","last_fullsized":"2023-10-08T16:13:16.555711+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:15.517984+0000","last_clean_scrub_stamp":"2023-10-08T16:13:15.517984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:00:25.016780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.1e","version":"183'1","reported_seq":11,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.560945+0000","last_change":"2023-10-08T16:13:15.536599+0000","last_active":"2023-10-08T16:13:16.560945+0000","last_peered":"2023-10-08T16:13:16.560945+0000","last_clean":"2023-10-08T16:13:16.560945+0000","last_became_active":"2023-10-08T16:13:15.536344+0000","last_became_peered":"2023-10-08T16:13:15.536344+0000","last_unstale":"2023-10-08T16:13:16.560945+0000","last_undegraded":"2023-10-08T16:13:16.560945+0000","last_fullsized":"2023-10-08T16:13:16.560945+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:14.515631+0000","last_clean_scrub_stamp":"2023-10-08T16:13:14.515631+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:45:44.930661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":301,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-10-08T16:13:16.552077+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:13:16.552077+0000","last_peered":"2023-10-08T16:13:16.552077+0000","last_clean":"2023-10-08T16:13:16.552077+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:13:16.552077+0000","last_undegraded":"2023-10-08T16:13:16.552077+0000","last_fullsized":"2023-10-08T16:13:16.552077+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":32,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":31,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304256,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067584,"statfs":{"total":96636764160,"available":96325206016,"internally_reserved":0,"allocated":2940928,"data_stored":1641204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52200000000000002,"15min":0.51700000000000002},"min":{"1min":0.40100000000000002,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.62,"5min":0.73599999999999999,"15min":0.86199999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72799999999999998},"min":{"1min":0.40600000000000003,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.61699999999999999,"5min":0.75800000000000001,"15min":8.0939999999999994},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.42799999999999999,"5min":0.35099999999999998,"15min":0.29199999999999998},"max":{"1min":0.67100000000000004,"5min":0.82199999999999995,"15min":1.083},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.64900000000000002,"5min":0.70399999999999996,"15min":8.2040000000000006},"last":0.53900000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.49099999999999999,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.70299999999999996,"15min":1.175},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52800000000000002,"15min":0.77900000000000003},"min":{"1min":0.42099999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.68100000000000005,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.56399999999999995}]},{"osd":3,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52800000000000002,"15min":0.55400000000000005},"min":{"1min":0.38,"5min":0.374,"15min":0.34499999999999997},"max":{"1min":0.67200000000000004,"5min":1.677,"15min":1.677},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52300000000000002,"15min":0.78100000000000003},"min":{"1min":0.34100000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63600000000000001,"5min":0.69899999999999995,"15min":8.3409999999999993},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44400000000000001,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29099999999999998,"15min":0.27500000000000002},"max":{"1min":0.60099999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.376,"5min":0.36799999999999999,"15min":0.33700000000000002},"max":{"1min":0.63200000000000001,"5min":0.753,"15min":0.753},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.503,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.28899999999999998},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.51800000000000002,"15min":0.55000000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65900000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.625}]},{"osd":6,"last update":"Sun Oct 8 16:12:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.56999999999999995,"15min":0.59999999999999998},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.69499999999999995,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55800000000000005,"15min":0.57999999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67700000000000005,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.55400000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1050824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.252},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.90600000000000003},"last":0.33900000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.496},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.68500000000000005,"5min":0.747,"15min":0.76700000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.96599999999999997},"last":0.41999999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.94199999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.72499999999999998,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.47099999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61499999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.377},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.82399999999999995,"15min":0.874},"last":0.52900000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.47799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307764,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064076,"statfs":{"total":96636764160,"available":96321613824,"internally_reserved":0,"allocated":2338816,"data_stored":1050712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.307,"15min":0.312},"min":{"1min":0.245,"5min":0.16900000000000001,"15min":0.16900000000000001},"max":{"1min":0.41399999999999998,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34499999999999997,"15min":0.34000000000000002},"min":{"1min":0.253,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.502,"5min":0.627,"15min":0.77500000000000002},"last":0.22700000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.35499999999999998},"min":{"1min":0.27000000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52100000000000002,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.311},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.37,"15min":0.374},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.72399999999999998},"last":0.39400000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.37,"15min":0.38600000000000001},"min":{"1min":0.30199999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34200000000000003,"15min":0.35699999999999998},"min":{"1min":0.28699999999999998,"5min":0.215,"15min":0.215},"max":{"1min":0.48199999999999998,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.33900000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.39500000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":1.077},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.373,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.84499999999999997,"5min":1.696,"15min":1.696},"last":0.54100000000000004}]},{"osd":5,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.80200000000000005,"5min":1.657,"15min":1.657},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.38600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72799999999999998,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.55000000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58099999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79400000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86899999999999999,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.59999999999999998,"15min":0.58899999999999997},"min":{"1min":0.439,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78900000000000003,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85999999999999999,"5min":0.875,"15min":1.1839999999999999},"last":0.622}]}]},{"osd":0,"up_from":13,"seq":55834574958,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":2945024,"data_stored":1642616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.313,"15min":0.48899999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.41399999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.441},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.52000000000000002},"min":{"1min":0.27600000000000002,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.41499999999999998,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.373},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.219},"max":{"1min":0.501,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.32700000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.188},"max":{"1min":0.59499999999999997,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.55800000000000005},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.26900000000000002},"max":{"1min":0.42999999999999999,"5min":1.498,"15min":5.5970000000000004},"last":0.36099999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55300000000000005,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":0.70799999999999996}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":1.847,"15min":3.2890000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.68400000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.83599999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77000000000000002,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303676,"kb_used_data":2292,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068164,"statfs":{"total":96636764160,"available":96325799936,"internally_reserved":0,"allocated":2347008,"data_stored":1051336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.307},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.46400000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.34399999999999997}]},{"osd":1,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.33300000000000002,"15min":0.34699999999999998},"min":{"1min":0.25600000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.44400000000000001,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.35999999999999999},"min":{"1min":0.27100000000000002,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.318}]},{"osd":3,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.34200000000000003},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48999999999999999,"5min":0.57099999999999995,"15min":0.60599999999999998},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.372,"15min":0.38400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28199999999999997,"15min":0.26800000000000002},"max":{"1min":0.56899999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.40000000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.40400000000000003,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40100000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.78000000000000003,"5min":0.83699999999999997,"15min":0.89100000000000001},"last":0.65000000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53900000000000003,"15min":0.56100000000000005},"min":{"1min":0.42499999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55300000000000005,"15min":0.64800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.81499999999999995,"15min":4.1029999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.41499999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.74399999999999999,"5min":0.85199999999999998,"15min":0.871},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.58499999999999996,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59999999999999998,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307788,"kb_used_data":2308,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064052,"statfs":{"total":96636764160,"available":96321589248,"internally_reserved":0,"allocated":2363392,"data_stored":1052858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.34399999999999997},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":0.44900000000000001,"5min":0.505,"15min":1.238},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.376},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.33700000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.35299999999999998},"min":{"1min":0.254,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45800000000000002,"5min":0.47999999999999998,"15min":1.0269999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.39700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.499,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.309}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36399999999999999,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.248},"max":{"1min":0.47699999999999998,"5min":0.53900000000000003,"15min":1.1399999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.28299999999999997,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":1.327},"last":0.376}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.9119999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.099,"15min":1.099},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":1.27},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":1.879},"last":0.65500000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.75700000000000001,"5min":0.81399999999999995,"15min":1.0620000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":1.825},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"max":{"1min":0.81100000000000005,"5min":1.0009999999999999,"15min":1.115},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303672,"kb_used_data":2288,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068168,"statfs":{"total":96636764160,"available":96325804032,"internally_reserved":0,"allocated":2342912,"data_stored":1050782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.52600000000000002},"min":{"1min":0.377,"5min":0.32100000000000001,"15min":0.28199999999999997},"max":{"1min":0.64600000000000002,"5min":0.68100000000000005,"15min":1.579},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52700000000000002,"15min":0.51100000000000001},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.40000000000000002,"5min":0.318,"15min":0.29899999999999999},"max":{"1min":0.72199999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56599999999999995}]},{"osd":2,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55500000000000005,"15min":0.57999999999999996},"min":{"1min":0.40000000000000002,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.60199999999999998,"15min":0.57599999999999996},"min":{"1min":0.42999999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.64000000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54200000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.435,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69699999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48899999999999999,"15min":0.52000000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64000000000000001,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53900000000000003,"15min":0.55700000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.70999999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.52900000000000003,"15min":0.55700000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68600000000000005,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54700000000000004,"15min":0.57999999999999996},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68200000000000005,"5min":0.76600000000000001,"15min":0.83399999999999996},"last":0.72099999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.60099999999999998},"min":{"1min":0.35799999999999998,"5min":0.35699999999999998,"15min":0.34899999999999998},"max":{"1min":0.80400000000000005,"5min":0.875,"15min":0.875},"last":0.749},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.89200000000000002},"last":0.77200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574958,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303668,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068172,"statfs":{"total":96636764160,"available":96325808128,"internally_reserved":0,"allocated":2338816,"data_stored":1050712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68700000000000006,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51700000000000002,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73499999999999999,"5min":0.84399999999999997,"15min":12.227},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.871,"15min":1.252},"last":0.68700000000000006}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.314,"15min":0.314},"max":{"1min":0.73899999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.88700000000000001,"15min":1.444},"last":0.72199999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.755,"15min":9.3170000000000002},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":0.879,"15min":3.8719999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.495,"15min":0.5},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.93799999999999994},"last":0.496},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.254,"15min":0.254},"max":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.83199999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.877,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:13:20.977 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:13:20.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:13:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.131+0000 7f9c0510e700 1 Processor -- start 2023-10-08T16:13:21.132 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.131+0000 7f9c0510e700 1 -- start start 2023-10-08T16:13:21.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.131+0000 7f9c0510e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00159950 0x7f9c000cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9c0510e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9c000af320 0x7f9c00069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.134 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9c0510e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c000ce380 con 0x7f9c00069aa0 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9c0510e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c00069f20 con 0x7f9c000af320 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9c0510e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000cec20 con 0x7f9c00159950 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9c000af320 0x7f9c00069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9c000af320 0x7f9c00069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55914/0 (socket says 172.21.15.170:55914) 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 -- 172.21.15.170:0/3201625344 learned_addr learned my addr 172.21.15.170:0/3201625344 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfed9d700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00159950 0x7f9c000cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 -- 172.21.15.170:0/3201625344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00159950 msgr2=0x7f9c000cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00159950 0x7f9c000cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 -- 172.21.15.170:0/3201625344 >> v1:172.21.15.191:6789/0 conn(0x7f9c00069aa0 legacy=0x7f9c0015d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 -- 172.21.15.170:0/3201625344 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c000afa90 con 0x7f9c000af320 2023-10-08T16:13:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfe59c700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c000af320 0x7f9c00069560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40099b0 tx=0x7f9bf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8544efa6cc91cddf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfdd9b700 1 -- 172.21.15.170:0/3201625344 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400b860 con 0x7f9c000af320 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfdd9b700 1 -- 172.21.15.170:0/3201625344 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf400b9c0 con 0x7f9c000af320 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.132+0000 7f9bfdd9b700 1 -- 172.21.15.170:0/3201625344 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400bd60 con 0x7f9c000af320 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 -- 172.21.15.170:0/3201625344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c000af320 msgr2=0x7f9c00069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c000af320 0x7f9c00069560 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40099b0 tx=0x7f9bf4009250 comp rx=0 tx=0).stop 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 -- 172.21.15.170:0/3201625344 shutdown_connections 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00159950 0x7f9c000cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 --2- 172.21.15.170:0/3201625344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c000af320 0x7f9c00069560 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 -- 172.21.15.170:0/3201625344 >> 172.21.15.170:0/3201625344 conn(0x7f9c00068e00 msgr2=0x7f9c000d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 -- 172.21.15.170:0/3201625344 shutdown_connections 2023-10-08T16:13:21.136 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.133+0000 7f9c0510e700 1 -- 172.21.15.170:0/3201625344 wait complete. 2023-10-08T16:13:21.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.134+0000 7f9c0510e700 1 Processor -- start 2023-10-08T16:13:21.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.134+0000 7f9c0510e700 1 -- start start 2023-10-08T16:13:21.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 0x7f9c000c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c000af320 0x7f9c0014e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b4e60 con 0x7f9c00159950 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c000b4530 con 0x7f9c000af320 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9c0510e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0016e6f0 con 0x7f9c00069aa0 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55918/0 (socket says 172.21.15.170:55918) 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 -- 172.21.15.170:0/2917268710 learned_addr learned my addr 172.21.15.170:0/2917268710 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bfed9d700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 0x7f9c000c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bfe59c700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c000af320 0x7f9c0014e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 -- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 msgr2=0x7f9c000c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 0x7f9c000c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 -- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c000af320 msgr2=0x7f9c0014e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c000af320 0x7f9c0014e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bff59e700 1 -- 172.21.15.170:0/2917268710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40094f0 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.135+0000 7f9bfed9d700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 0x7f9c000c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9bff59e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c8e0 tx=0x7f9bf8001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800ce80 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf800a7e0 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0016e9d0 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf800ab80 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.136+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c000b4af0 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.137+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c000b4fd0 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.137+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9bf8012420 con 0x7f9c00159950 2023-10-08T16:13:21.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.137+0000 7f9be6ffd700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.137+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9bf8084f60 con 0x7f9c00159950 2023-10-08T16:13:21.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.138+0000 7f9bfed9d700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.142 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.138+0000 7f9bfed9d700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 0x7f9bec0697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009990 tx=0x7f9bf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.143 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.142+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bf8051bf0 con 0x7f9c00159950 2023-10-08T16:13:21.375 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:13:21.374+0000 7f01e4504700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:13:21.399 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.398+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9c000718b0 con 0x7f9c00159950 2023-10-08T16:13:21.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.402+0000 7f9be6ffd700 1 -- 172.21.15.170:0/2917268710 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9bf8055210 con 0x7f9c00159950 2023-10-08T16:13:21.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 msgr2=0x7f9bec0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 0x7f9bec0697d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009990 tx=0x7f9bf0008040 comp rx=0 tx=0).stop 2023-10-08T16:13:21.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 msgr2=0x7f9c0016db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f9bf800c8e0 tx=0x7f9bf8001710 comp rx=0 tx=0).stop 2023-10-08T16:13:21.406 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 shutdown_connections 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9bec067320 0x7f9bec0697d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c00159950 0x7f9c0016db80 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9c00069aa0 0x7f9c000c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 --2- 172.21.15.170:0/2917268710 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9c000af320 0x7f9c0014e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 >> 172.21.15.170:0/2917268710 conn(0x7f9c00068e00 msgr2=0x7f9c000cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 shutdown_connections 2023-10-08T16:13:21.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.404+0000 7f9c0510e700 1 -- 172.21.15.170:0/2917268710 wait complete. 2023-10-08T16:13:21.408 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:13:21.418 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:21.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 Processor -- start 2023-10-08T16:13:21.533 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 -- start start 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fae10069560 0x7fae100699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae10069ee0 con 0x7fae10159890 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1015a160 con 0x7fae10159c60 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.532+0000 7fae155fa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae100d13a0 con 0x7fae10069560 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0f7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fae10069560 0x7fae100699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.535 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0effd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fae10159890 0x7fae1015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40392/0 (socket says 172.21.15.170:40392) 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0effd700 1 -- 172.21.15.170:0/487612357 learned_addr learned my addr 172.21.15.170:0/487612357 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0f7fe700 1 -- 172.21.15.170:0/487612357 >> v1:172.21.15.191:6789/0 conn(0x7fae10159c60 legacy=0x7fae1015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0f7fe700 1 -- 172.21.15.170:0/487612357 >> v1:172.21.15.170:6789/0 conn(0x7fae10159890 legacy=0x7fae1015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0f7fe700 1 -- 172.21.15.170:0/487612357 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1015e140 con 0x7fae10069560 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0e7fc700 1 -- 172.21.15.170:0/487612357 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262625328 0 0) 0x7fae10069ee0 con 0x7fae10159890 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0f7fe700 1 --2- 172.21.15.170:0/487612357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10069560 0x7fae100699a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e60 tx=0x7fae08009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0c3c599e6f54b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0e7fc700 1 -- 172.21.15.170:0/487612357 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800e070 con 0x7fae10069560 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0e7fc700 1 -- 172.21.15.170:0/487612357 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800bc30 con 0x7fae10069560 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.533+0000 7fae0e7fc700 1 -- 172.21.15.170:0/487612357 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08012430 con 0x7fae10069560 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 -- 172.21.15.170:0/487612357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10069560 msgr2=0x7fae100699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.536 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 --2- 172.21.15.170:0/487612357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10069560 0x7fae100699a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fae08009e60 tx=0x7fae08009780 comp rx=0 tx=0).stop 2023-10-08T16:13:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 -- 172.21.15.170:0/487612357 shutdown_connections 2023-10-08T16:13:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 --2- 172.21.15.170:0/487612357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10069560 0x7fae100699a0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 -- 172.21.15.170:0/487612357 >> 172.21.15.170:0/487612357 conn(0x7fae10068e20 msgr2=0x7fae100d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 -- 172.21.15.170:0/487612357 shutdown_connections 2023-10-08T16:13:21.537 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.534+0000 7fae155fa700 1 -- 172.21.15.170:0/487612357 wait complete. 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.535+0000 7fae155fa700 1 Processor -- start 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.535+0000 7fae155fa700 1 -- start start 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae10069560 0x7fae100b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae10159890 0x7fae10140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b4170 con 0x7fae10159c60 2023-10-08T16:13:21.539 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1016dac0 con 0x7fae10069560 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1016dda0 con 0x7fae10159890 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55938/0 (socket says 172.21.15.170:55938) 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 -- 172.21.15.170:0/2924140962 learned_addr learned my addr 172.21.15.170:0/2924140962 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae067fc700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae10159890 0x7fae10140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0effd700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae10069560 0x7fae100b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 -- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae10159890 msgr2=0x7fae10140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae10159890 0x7fae10140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 -- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae10069560 msgr2=0x7fae100b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae10069560 0x7fae100b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 -- 172.21.15.170:0/2924140962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08009a20 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae0f7fe700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fae08013000 tx=0x7fae08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0801b630 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0801b790 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1016e080 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0801bb30 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.536+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae100cd6b0 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.537+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1004c3c0 con 0x7fae10159c60 2023-10-08T16:13:21.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.538+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae0801c400 con 0x7fae10159c60 2023-10-08T16:13:21.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.538+0000 7fae077fe700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 0x7fadf00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:21.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.538+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae08086cf0 con 0x7fae10159c60 2023-10-08T16:13:21.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.538+0000 7fae0effd700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 0x7fadf00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:21.542 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.539+0000 7fae0effd700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 0x7fadf00697d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae000011e0 tx=0x7fae00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:21.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.542+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae08053870 con 0x7fae10159c60 2023-10-08T16:13:21.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.769+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae100cd870 con 0x7fae10159c60 2023-10-08T16:13:21.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.770+0000 7fae077fe700 1 -- 172.21.15.170:0/2924140962 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4619 (secure 0 0 0) 0x7fae0802a020 con 0x7fae10159c60 2023-10-08T16:13:21.771 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:21.771 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":120,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:19.562557+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:21.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 msgr2=0x7fadf00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 0x7fadf00697d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fae000011e0 tx=0x7fae00009040 comp rx=0 tx=0).stop 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 msgr2=0x7fae10157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fae08013000 tx=0x7fae08019000 comp rx=0 tx=0).stop 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 shutdown_connections 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fadf0067320 0x7fadf00697d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae10159c60 0x7fae10157480 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fae10069560 0x7fae100b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 --2- 172.21.15.170:0/2924140962 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fae10159890 0x7fae10140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 >> 172.21.15.170:0/2924140962 conn(0x7fae10068e20 msgr2=0x7fae1015aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 shutdown_connections 2023-10-08T16:13:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:21.772+0000 7fae155fa700 1 -- 172.21.15.170:0/2924140962 wait complete. 2023-10-08T16:13:21.775 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 120 2023-10-08T16:13:22.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:13:15.903470+0000', 'modified': '2023-10-08T16:13:19.562557+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8634}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8634': {'gid': 8634, 'name': 'c', 'rank': 0, 'incarnation': 116, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6836/3034213253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 3034213253}, {'type': 'v1', 'addr': '172.21.15.170:6836', 'nonce': 3034213253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [32], 'metadata_pool': 31, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:13:22.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:13:22.189 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T16:13:22.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.304+0000 7f1e3d05f700 1 Processor -- start 2023-10-08T16:13:22.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.304+0000 7f1e3d05f700 1 -- start start 2023-10-08T16:13:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3d05f700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 0x7f1e3815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3d05f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1e38069560 0x7f1e380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3d05f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e38069ee0 con 0x7f1e38159c60 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3d05f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3815a160 con 0x7f1e38069560 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3d05f700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380d13a0 con 0x7f1e38159890 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e3759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1e38069560 0x7f1e380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e2e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1e38159c60 0x7f1e3815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40402/0 (socket says 172.21.15.170:40402) 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.305+0000 7f1e2e59c700 1 -- 172.21.15.170:0/3824317621 learned_addr learned my addr 172.21.15.170:0/3824317621 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/3824317621 >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 0x7f1e3815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3759e700 1 -- 172.21.15.170:0/3824317621 >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 msgr2=0x7f1e3815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3759e700 1 --2- 172.21.15.170:0/3824317621 >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 0x7f1e3815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3759e700 1 -- 172.21.15.170:0/3824317621 >> v1:172.21.15.170:6789/0 conn(0x7f1e38159c60 legacy=0x7f1e3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3759e700 1 -- 172.21.15.170:0/3824317621 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3815e140 con 0x7f1e38069560 2023-10-08T16:13:22.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3659c700 1 -- 172.21.15.170:0/3824317621 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169322401 0 0) 0x7f1e38069ee0 con 0x7f1e38159c60 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/3824317621 >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 0x7f1e3815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3759e700 1 --2- 172.21.15.170:0/3824317621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e380699a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1e380d1090 tx=0x7f1e30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd14ad79bcc36900 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3659c700 1 -- 172.21.15.170:0/3824317621 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000e070 con 0x7f1e38069560 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3659c700 1 -- 172.21.15.170:0/3824317621 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000bc30 con 0x7f1e38069560 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3659c700 1 -- 172.21.15.170:0/3824317621 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30012430 con 0x7f1e38069560 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 -- 172.21.15.170:0/3824317621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 msgr2=0x7f1e380699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/3824317621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e380699a0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f1e380d1090 tx=0x7f1e30009780 comp rx=0 tx=0).stop 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 -- 172.21.15.170:0/3824317621 shutdown_connections 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/3824317621 >> v2:172.21.15.191:3300/0 conn(0x7f1e38159890 0x7f1e3815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/3824317621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e380699a0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.306+0000 7f1e3d05f700 1 -- 172.21.15.170:0/3824317621 >> 172.21.15.170:0/3824317621 conn(0x7f1e38068e20 msgr2=0x7f1e380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.307+0000 7f1e3d05f700 1 -- 172.21.15.170:0/3824317621 shutdown_connections 2023-10-08T16:13:22.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.307+0000 7f1e3d05f700 1 -- 172.21.15.170:0/3824317621 wait complete. 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 Processor -- start 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 -- start start 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e38159890 0x7f1e38148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e38159c60 0x7f1e38149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380ce1c0 con 0x7f1e38069560 2023-10-08T16:13:22.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380ce4a0 con 0x7f1e38159c60 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.308+0000 7f1e3d05f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380ce780 con 0x7f1e38159890 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55962/0 (socket says 172.21.15.170:55962) 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 -- 172.21.15.170:0/525598415 learned_addr learned my addr 172.21.15.170:0/525598415 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 -- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e38159890 msgr2=0x7f1e38148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e3759e700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e38159c60 0x7f1e38149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e38159890 0x7f1e38148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e38159890 0x7f1e38148f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 -- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e38159c60 msgr2=0x7f1e38149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e38159c60 0x7f1e38149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 -- 172.21.15.170:0/525598415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009a20 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c830 tx=0x7f1e28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e2f7fe700 1 -- 172.21.15.170:0/525598415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011600 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e2f7fe700 1 -- 172.21.15.170:0/525598415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28011760 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e380cea60 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e2f7fe700 1 -- 172.21.15.170:0/525598415 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011b20 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.309+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e38158e90 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.310+0000 7f1e2f7fe700 1 -- 172.21.15.170:0/525598415 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1e2800a3e0 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.311+0000 7f1e2f7fe700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 0x7f1e180697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.311+0000 7f1e2f7fe700 1 -- 172.21.15.170:0/525598415 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e28085240 con 0x7f1e38069560 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.311+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 0x7f1e180697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.311+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 0x7f1e180697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1e381592b0 tx=0x7f1e20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.313+0000 7f1e46357b80 1 Processor -- start 2023-10-08T16:13:22.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.313+0000 7f1e46357b80 1 -- start start 2023-10-08T16:13:22.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c70e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.321 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2c6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2bc7a70 con 0x5638d2cbad00 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2bc5270 con 0x5638d2c70a50 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d2bc1060 con 0x5638d2c99f60 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55972/0 (socket says 172.21.15.170:55972) 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 -- 172.21.15.170:0/1404355099 learned_addr learned my addr 172.21.15.170:0/1404355099 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2c6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c70e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 -- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 msgr2=0x5638d2c6ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.322 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2c6ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 -- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 msgr2=0x5638d2c70e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c70e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 -- 172.21.15.170:0/1404355099 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d2a5b270 con 0x5638d2cbad00 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2c6ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005f70 tx=0x7f1e20012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfa5175980c647e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e17fff700 1 -- 172.21.15.170:0/1404355099 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200144b0 con 0x5638d2cbad00 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e17fff700 1 -- 172.21.15.170:0/1404355099 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20014610 con 0x5638d2cbad00 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.314+0000 7f1e17fff700 1 -- 172.21.15.170:0/1404355099 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e200149b0 con 0x5638d2cbad00 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 -- 172.21.15.170:0/1404355099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 msgr2=0x5638d2c99a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.323 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f1e20005f70 tx=0x7f1e20012000 comp rx=0 tx=0).stop 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 -- 172.21.15.170:0/1404355099 shutdown_connections 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2c6ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c70e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1404355099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2c99a20 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 -- 172.21.15.170:0/1404355099 >> 172.21.15.170:0/1404355099 conn(0x5638d2c72370 msgr2=0x5638d2c72740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 -- 172.21.15.170:0/1404355099 shutdown_connections 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.315+0000 7f1e46357b80 1 -- 172.21.15.170:0/1404355099 wait complete. 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.316+0000 7f1e46357b80 1 -- start start 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.316+0000 7f1e46357b80 1 Processor -- start 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2cc7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cc8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.324 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2b6c890 con 0x5638d2cbad00 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2b87ea0 con 0x5638d2c70a50 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d2a43e30 con 0x5638d2c99f60 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 10 client.0 _ll_register_callbacks cb 0x5638d2c34110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55982/0 (socket says 172.21.15.170:55982) 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 -- 172.21.15.170:0/1244431995 learned_addr learned my addr 172.21.15.170:0/1244431995 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cc8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2cc7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 -- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 msgr2=0x5638d2cc8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.325 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cc8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 -- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 msgr2=0x5638d2cc7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2cc7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 -- 172.21.15.170:0/1244431995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e20005ad0 con 0x5638d2cbad00 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1e30013000 tx=0x7f1e30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4d3214a2f45b87 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e157fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30021070 con 0x5638d2cbad00 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e30026470 con 0x5638d2cbad00 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.317+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638d2c65b70 con 0x5638d2cbad00 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e30026840 con 0x5638d2cbad00 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 10 client.8793 fetch_fsmap learned FSMap version 120 2023-10-08T16:13:22.326 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x5638d29fdfe0 con 0x5638d2cbad00 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5638d2c65b70 con 0x5638d2cbad00 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 120) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1e300269a0 con 0x5638d2cbad00 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 10 client.8793 fetch_fsmap finished waiting for FSMap version 120 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 10 client.8793 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 20 client.8793 populate_metadata read hostname 'smithi170' 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2cc86a0 0x5638d2c6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 4 client.8793 mds_command: new command op to 8634 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 --> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638d2c42aa0 con 0x5638d2cc86a0 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2cc86a0 0x5638d2c6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e157fa700 10 client.8793 ms_handle_connect on v2:172.21.15.170:6834/3034213253 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.318+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2cc86a0 0x5638d2c6d8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3a09ad1bbb55bb server_cookie=b8365cfd914f63ec in_seq=0 out_seq=0 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e157fa700 1 -- 172.21.15.170:0/1244431995 <== mds.0 v2:172.21.15.170:6834/3034213253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5638d2c42aa0 con 0x5638d2cc86a0 2023-10-08T16:13:22.327 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e157fa700 10 client.8793 handle_command_reply: tid=0 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 client.8793 shutdown 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 msgr2=0x5638d2cba170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1e30013000 tx=0x7f1e30019000 comp rx=0 tx=0).stop 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 shutdown_connections 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2cc86a0 0x5638d2c6d8b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cba170 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2cc7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e157fa700 0 client.8793 ms_handle_reset on v2:172.21.15.170:6834/3034213253 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1244431995 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cc8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 >> 172.21.15.170:0/1244431995 conn(0x5638d2c72370 msgr2=0x5638d2c71350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 shutdown_connections 2023-10-08T16:13:22.328 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 1 -- 172.21.15.170:0/1244431995 wait complete. 2023-10-08T16:13:22.329 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.319+0000 7f1e46357b80 20 client.8793 trim_cache size 0 max 16384 2023-10-08T16:13:22.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.336+0000 7f1e46357b80 1 Processor -- start 2023-10-08T16:13:22.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.336+0000 7f1e46357b80 1 -- start start 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cb6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2cc4400 con 0x5638d2cbad00 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2c73ff0 con 0x5638d2c70a50 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d2c50510 con 0x5638d2c99f60 2023-10-08T16:13:22.340 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55990/0 (socket says 172.21.15.170:55990) 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 -- 172.21.15.170:0/424564280 learned_addr learned my addr 172.21.15.170:0/424564280 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cb6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 -- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 msgr2=0x5638d2cb6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cb6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 -- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 msgr2=0x5638d2c71880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c71880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 -- 172.21.15.170:0/424564280 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638d2b8f780 con 0x5638d2cbad00 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:22.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e3759e700 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f1e30026e30 tx=0x7f1e3001e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15371eacd6a1bc6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e2cd99700 1 -- 172.21.15.170:0/424564280 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30021070 con 0x5638d2cbad00 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e2cd99700 1 -- 172.21.15.170:0/424564280 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30009880 con 0x5638d2cbad00 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.337+0000 7f1e2cd99700 1 -- 172.21.15.170:0/424564280 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30026470 con 0x5638d2cbad00 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- 172.21.15.170:0/424564280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 msgr2=0x5638d2cb69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f1e30026e30 tx=0x7f1e3001e070 comp rx=0 tx=0).stop 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- 172.21.15.170:0/424564280 shutdown_connections 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cb69a0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c70a50 0x5638d2c71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 --2- 172.21.15.170:0/424564280 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c99f60 0x5638d2cb6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- 172.21.15.170:0/424564280 >> 172.21.15.170:0/424564280 conn(0x5638d2c72370 msgr2=0x5638d2c72740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- 172.21.15.170:0/424564280 shutdown_connections 2023-10-08T16:13:22.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- 172.21.15.170:0/424564280 wait complete. 2023-10-08T16:13:22.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.338+0000 7f1e46357b80 1 -- start start 2023-10-08T16:13:22.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 Processor -- start 2023-10-08T16:13:22.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c70a50 0x5638d2d1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.346 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c99f60 0x5638d2cbb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2cb8df0 con 0x5638d2cbad00 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638d2c6f9b0 con 0x5638d2c99f60 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638d2ccfe10 con 0x5638d2c70a50 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.339+0000 7f1e46357b80 10 client.0 _ll_register_callbacks cb 0x5638d2c34110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55992/0 (socket says 172.21.15.170:55992) 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 -- 172.21.15.170:0/1678160551 learned_addr learned my addr 172.21.15.170:0/1678160551 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c70a50 0x5638d2d1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e2e59c700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c99f60 0x5638d2cbb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.347 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 -- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c70a50 msgr2=0x5638d2d1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c70a50 0x5638d2d1cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 -- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c99f60 msgr2=0x5638d2cbb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c99f60 0x5638d2cbb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 -- 172.21.15.170:0/1678160551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638d29fdfe0 con 0x5638d2cbad00 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e3759e700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f1e30013fd0 tx=0x7f1e30036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855f08822ed54fed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30021040 con 0x5638d2cbad00 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1e3000bda0 con 0x5638d2cbad00 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e20014610 con 0x5638d2cbad00 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e30040cc0 con 0x5638d2cbad00 2023-10-08T16:13:22.348 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 10 client.8799 fetch_fsmap learned FSMap version 120 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x5638d2b76470 con 0x5638d2cbad00 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20014610 con 0x5638d2cbad00 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 120) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1e3003e070 con 0x5638d2cbad00 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 10 client.8799 fetch_fsmap finished waiting for FSMap version 120 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 10 client.8799 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 20 client.8799 populate_metadata read hostname 'smithi170' 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2d206c0 0x5638d2d20a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 4 client.8799 mds_command: new command op to 8634 tid=0[{"prefix": "status"}] 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.340+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 --> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5638d2b63e00 con 0x5638d2d206c0 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2d206c0 0x5638d2d20a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e157fa700 10 client.8799 ms_handle_connect on v2:172.21.15.170:6834/3034213253 2023-10-08T16:13:22.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e36d9d700 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2d206c0 0x5638d2d20a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd1514bdb543f5 server_cookie=c3d5747bd6523790 in_seq=0 out_seq=0 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e157fa700 1 -- 172.21.15.170:0/1678160551 <== mds.0 v2:172.21.15.170:6834/3034213253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5638d2b63e00 con 0x5638d2d206c0 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e157fa700 10 client.8799 handle_command_reply: tid=0 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 client.8799 shutdown 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 msgr2=0x5638d2cbb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f1e30013fd0 tx=0x7f1e30036000 comp rx=0 tx=0).stop 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 shutdown_connections 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:6834/3034213253,v1:172.21.15.170:6836/3034213253] conn(0x5638d2d206c0 0x5638d2d20a90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5638d2cbad00 0x5638d2cbb610 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x5638d2c70a50 0x5638d2d1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e157fa700 0 client.8799 ms_handle_reset on v2:172.21.15.170:6834/3034213253 2023-10-08T16:13:22.350 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 --2- 172.21.15.170:0/1678160551 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x5638d2c99f60 0x5638d2cbb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 >> 172.21.15.170:0/1678160551 conn(0x5638d2c72370 msgr2=0x5638d2cb6460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 shutdown_connections 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 1 -- 172.21.15.170:0/1678160551 wait complete. 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.341+0000 7f1e46357b80 20 client.8799 trim_cache size 0 max 16384 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8634, 2023-10-08T16:13:22.351 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.79270336, 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 120, 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 184, 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 184, 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 11.381248537999999 2023-10-08T16:13:22.352 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 msgr2=0x7f1e180697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 0x7f1e180697d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1e381592b0 tx=0x7f1e20009040 comp rx=0 tx=0).stop 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 msgr2=0x7f1e38158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f1e2800c830 tx=0x7f1e28008040 comp rx=0 tx=0).stop 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 shutdown_connections 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1e38159c60 0x7f1e38149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1e18067320 0x7f1e180697d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1e38069560 0x7f1e38158950 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 --2- 172.21.15.170:0/525598415 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1e38159890 0x7f1e38148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 >> 172.21.15.170:0/525598415 conn(0x7f1e38068e20 msgr2=0x7f1e38157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 shutdown_connections 2023-10-08T16:13:22.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.342+0000 7f1e3d05f700 1 -- 172.21.15.170:0/525598415 wait complete. 2023-10-08T16:13:22.355 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:13:22.468 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.467+0000 7f2d9e6bb700 1 Processor -- start 2023-10-08T16:13:22.469 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.467+0000 7f2d9e6bb700 1 -- start start 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9e6bb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2d98159c60 0x7f2d9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9e6bb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9815a160 con 0x7f2d9815db10 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9e6bb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d980d13a0 con 0x7f2d98159890 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9e6bb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9815df50 con 0x7f2d98159c60 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9cc58700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2d9815db10 0x7f2d98069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40404/0 (socket says 172.21.15.170:40404) 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d9cc58700 1 -- 172.21.15.170:0/2373505491 learned_addr learned my addr 172.21.15.170:0/2373505491 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:22.471 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d977fe700 1 --2- 172.21.15.170:0/2373505491 >> v2:172.21.15.191:3300/0 conn(0x7f2d98159c60 0x7f2d9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164799781 0 0) 0x7f2d9815a160 con 0x7f2d9815db10 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.468+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d80003570 con 0x7f2d9815db10 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d977fe700 1 -- 172.21.15.170:0/2373505491 >> v1:172.21.15.191:6789/0 conn(0x7f2d98159890 legacy=0x7f2d9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2238668843 0 0) 0x7f2d80003570 con 0x7f2d9815db10 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d977fe700 1 -- 172.21.15.170:0/2373505491 >> v1:172.21.15.170:6789/0 conn(0x7f2d9815db10 legacy=0x7f2d98069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d977fe700 1 -- 172.21.15.170:0/2373505491 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9815e140 con 0x7f2d98159c60 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d977fe700 1 --2- 172.21.15.170:0/2373505491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98159c60 0x7f2d9815d5d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0098e0 tx=0x7f2d8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ecf3f26157088c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d900017f0 con 0x7f2d98159c60 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c011420 con 0x7f2d98159c60 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d96ffd700 1 -- 172.21.15.170:0/2373505491 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0117c0 con 0x7f2d98159c60 2023-10-08T16:13:22.472 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/2373505491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98159c60 msgr2=0x7f2d9815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.469+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/2373505491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98159c60 0x7f2d9815d5d0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0098e0 tx=0x7f2d8c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.470+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/2373505491 shutdown_connections 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.470+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/2373505491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d98159c60 0x7f2d9815d5d0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.470+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/2373505491 >> 172.21.15.170:0/2373505491 conn(0x7f2d98068e20 msgr2=0x7f2d980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.470+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/2373505491 shutdown_connections 2023-10-08T16:13:22.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.470+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/2373505491 wait complete. 2023-10-08T16:13:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.471+0000 7f2d9e6bb700 1 Processor -- start 2023-10-08T16:13:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.471+0000 7f2d9e6bb700 1 -- start start 2023-10-08T16:13:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d98159890 0x7f2d980ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d9815db10 0x7f2d980b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9816d690 con 0x7f2d98159c60 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9816d970 con 0x7f2d9815db10 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9816dc50 con 0x7f2d98159890 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:55994/0 (socket says 172.21.15.170:55994) 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 -- 172.21.15.170:0/1605297491 learned_addr learned my addr 172.21.15.170:0/1605297491 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d97fff700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d98159890 0x7f2d980ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9cc58700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d9815db10 0x7f2d980b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 -- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d98159890 msgr2=0x7f2d980ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d98159890 0x7f2d980ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 -- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d9815db10 msgr2=0x7f2d980b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d9815db10 0x7f2d980b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 -- 172.21.15.170:0/1605297491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c009420 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d977fe700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0030c0 tx=0x7f2d8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c010c20 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c010d80 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9816df30 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c01d660 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.472+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9816d2b0 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.473+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d980ceb50 con 0x7f2d98159c60 2023-10-08T16:13:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.474+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d8c01d9d0 con 0x7f2d98159c60 2023-10-08T16:13:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.474+0000 7f2d7ffff700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 0x7f2d840697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.474+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d8c086690 con 0x7f2d98159c60 2023-10-08T16:13:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.474+0000 7f2d97fff700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 0x7f2d840697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.474+0000 7f2d97fff700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 0x7f2d840697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000f60 tx=0x7f2d88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.478+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d980ceb50 con 0x7f2d98159c60 2023-10-08T16:13:22.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.707+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d980ceb50 con 0x7f2d98159c60 2023-10-08T16:13:22.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.707+0000 7f2d7ffff700 1 -- 172.21.15.170:0/1605297491 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v120) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d980ceb50 con 0x7f2d98159c60 2023-10-08T16:13:22.709 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 msgr2=0x7f2d840697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 0x7f2d840697d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2d88000f60 tx=0x7f2d88009040 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 msgr2=0x7f2d980b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0030c0 tx=0x7f2d8c017000 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 shutdown_connections 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d9815db10 0x7f2d980b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d98159890 0x7f2d980ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2d84067320 0x7f2d840697d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 --2- 172.21.15.170:0/1605297491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d98159c60 0x7f2d980b1590 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 >> 172.21.15.170:0/1605297491 conn(0x7f2d98068e20 msgr2=0x7f2d9815cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:22.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 shutdown_connections 2023-10-08T16:13:22.712 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:22.709+0000 7f2d9e6bb700 1 -- 172.21.15.170:0/1605297491 wait complete. 2023-10-08T16:13:22.722 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:13:22.722 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:13:22.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:13:22.723 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:22.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.723 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:13:22.738 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:13:22.738 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:13:22.738 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:22.739 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41948sec preferred_lft 41948sec 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.740 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:22.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.741 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:13:22.801 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:13:22.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.805 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:13:22.861 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:13:22.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.862 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:13:22.862 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:13:22.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:13:22.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:13:22.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:13:22.863 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:13:22.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.960 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:13:22.975 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:13:22.975 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:13:22.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:22.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:22.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:13:23.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:23.039 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:23.095 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:23.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:23.095 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:23.149 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:13:23.150 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:13:23.150 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:23.230 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:23.230 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:23.248 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:23.249 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:23.269 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:13:23.270 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:13:23.271 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:13:23.272 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:13:23.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:23.272 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:23.337 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:13:23.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.490+0000 7f26ae5e1700 1 Processor -- start 2023-10-08T16:13:23.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.490+0000 7f26ae5e1700 1 -- start start 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.491+0000 7f26ae5e1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.491+0000 7f26ae5e1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 0x7f26a80cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.491+0000 7f26ae5e1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a806c750 con 0x7f26a80cbc10 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.491+0000 7f26ae5e1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a806ca30 con 0x7f26a806c1e0 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.491+0000 7f26ae5e1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a806cd10 con 0x7f26a8157940 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a77fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 0x7f26a80cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:54312/0 (socket says 172.21.15.170:54312) 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 -- 172.21.15.170:0/3944904101 learned_addr learned my addr 172.21.15.170:0/3944904101 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 -- 172.21.15.170:0/3944904101 >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 msgr2=0x7f26a80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 --2- 172.21.15.170:0/3944904101 >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 0x7f26a80cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 -- 172.21.15.170:0/3944904101 >> v1:172.21.15.191:6789/0 conn(0x7f26a80cbc10 legacy=0x7f26a815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 -- 172.21.15.170:0/3944904101 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a806cff0 con 0x7f26a8157940 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a77fe700 1 --2- 172.21.15.170:0/3944904101 >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 0x7f26a80cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a7fff700 1 --2- 172.21.15.170:0/3944904101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26980099b0 tx=0x7f2698009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a650c2a16710994 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.492+0000 7f26a6ffd700 1 -- 172.21.15.170:0/3944904101 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698009ea0 con 0x7f26a8157940 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26a6ffd700 1 -- 172.21.15.170:0/3944904101 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269800b860 con 0x7f26a8157940 2023-10-08T16:13:23.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26a6ffd700 1 -- 172.21.15.170:0/3944904101 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269800bc00 con 0x7f26a8157940 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 -- 172.21.15.170:0/3944904101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 msgr2=0x7f26a806f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/3944904101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f26980099b0 tx=0x7f2698009250 comp rx=0 tx=0).stop 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 -- 172.21.15.170:0/3944904101 shutdown_connections 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/3944904101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a8157940 0x7f26a806f5a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/3944904101 >> v2:172.21.15.191:3300/0 conn(0x7f26a806c1e0 0x7f26a80cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 -- 172.21.15.170:0/3944904101 >> 172.21.15.170:0/3944904101 conn(0x7f26a80cd480 msgr2=0x7f26a80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 -- 172.21.15.170:0/3944904101 shutdown_connections 2023-10-08T16:13:23.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.493+0000 7f26ae5e1700 1 -- 172.21.15.170:0/3944904101 wait complete. 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.494+0000 7f26ae5e1700 1 Processor -- start 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.494+0000 7f26ae5e1700 1 -- start start 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a806c1e0 0x7f26a80c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a8157940 0x7f26a816b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a80c7a50 con 0x7f26a80cbc10 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a81531e0 con 0x7f26a8157940 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26ae5e1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a816c0d0 con 0x7f26a806c1e0 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26a77fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56002/0 (socket says 172.21.15.170:56002) 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26a77fe700 1 -- 172.21.15.170:0/423803836 learned_addr learned my addr 172.21.15.170:0/423803836 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:23.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26a7fff700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a806c1e0 0x7f26a80c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.495+0000 7f26acb7e700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a8157940 0x7f26a816b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 -- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a806c1e0 msgr2=0x7f26a80c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a806c1e0 0x7f26a80c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 -- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a8157940 msgr2=0x7f26a816b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a8157940 0x7f26a816b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 -- 172.21.15.170:0/423803836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c009420 con 0x7f26a80cbc10 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26a77fe700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ec80 tx=0x7f269c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00c920 con 0x7f26a80cbc10 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00ca80 con 0x7f26a80cbc10 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26980094f0 con 0x7f26a80cbc10 2023-10-08T16:13:23.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00ce00 con 0x7f26a80cbc10 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.496+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a80c7d10 con 0x7f26a80cbc10 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.497+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a80c8370 con 0x7f26a80cbc10 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.497+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f269c013680 con 0x7f26a80cbc10 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.498+0000 7f268ffff700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 0x7f26940697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.498+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f269c085380 con 0x7f26a80cbc10 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.498+0000 7f26a7fff700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 0x7f26940697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:23.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.498+0000 7f26a7fff700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 0x7f26940697d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26980099b0 tx=0x7f2698017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:23.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.502+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f269c052010 con 0x7f26a80cbc10 2023-10-08T16:13:23.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.732+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f26a804c3c0 con 0x7f26a80cbc10 2023-10-08T16:13:23.733 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.732+0000 7f268ffff700 1 -- 172.21.15.170:0/423803836 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v120) v1 ==== 53+0+73 (secure 0 0 0) 0x7f269c055630 con 0x7f26a80cbc10 2023-10-08T16:13:23.734 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 msgr2=0x7f26940697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 0x7f26940697d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f26980099b0 tx=0x7f2698017000 comp rx=0 tx=0).stop 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 msgr2=0x7f26a8142090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f269c00ec80 tx=0x7f269c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 shutdown_connections 2023-10-08T16:13:23.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f26a8157940 0x7f26a816b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f26a806c1e0 0x7f26a80c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2694067320 0x7f26940697d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 --2- 172.21.15.170:0/423803836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26a80cbc10 0x7f26a8142090 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 >> 172.21.15.170:0/423803836 conn(0x7f26a80cd480 msgr2=0x7f26a80b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.734+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 shutdown_connections 2023-10-08T16:13:23.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:23.735+0000 7f26ae5e1700 1 -- 172.21.15.170:0/423803836 wait complete. 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:13:23.748 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:23.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:23.749 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:13:23.767 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:13:23.767 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:13:23.767 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:23.768 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41948sec preferred_lft 41948sec 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:13:23.769 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:23.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:23.770 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:13:23.835 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:13:23.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:23.837 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:13:23.895 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:13:23.895 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:13:23.895 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:13:23.895 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:13:23.895 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:13:23.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:13:23.896 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:13:23.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:24.000 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:13:24.014 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:13:24.014 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:13:24.015 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:24.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:24.015 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:13:24.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:24.079 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:24.134 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:24.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:24.135 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:24.189 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:13:24.189 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:13:24.189 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:24.268 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:24.268 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:24.286 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:24.287 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:24.308 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:24.308 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:13:24.309 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:13:24.311 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:13:24.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:24.311 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:24.375 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:13:24.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.494+0000 7f069e26b700 1 Processor -- start 2023-10-08T16:13:24.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.494+0000 7f069e26b700 1 -- start start 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069e26b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069e26b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069815b560 con 0x7f06980c94c0 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069e26b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069815b840 con 0x7f0698157540 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069e26b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06980cbbe0 con 0x7f06980cd6a0 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069ca68700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f06980c94c0 0x7f06980cadd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:40416/0 (socket says 172.21.15.170:40416) 2023-10-08T16:13:24.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.495+0000 7f069ca68700 1 -- 172.21.15.170:0/2638674509 learned_addr learned my addr 172.21.15.170:0/2638674509 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f069da6a700 1 --2- 172.21.15.170:0/2638674509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794971123 0 0) 0x7f069815b560 con 0x7f06980c94c0 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0684003570 con 0x7f06980c94c0 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505914322 0 0) 0x7f069815b840 con 0x7f0698157540 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f069815b560 con 0x7f0698157540 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2984643502 0 0) 0x7f0684003570 con 0x7f06980c94c0 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f069da6a700 1 -- 172.21.15.170:0/2638674509 >> v1:172.21.15.191:6789/0 conn(0x7f0698157540 legacy=0x7f06980c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f069da6a700 1 -- 172.21.15.170:0/2638674509 >> v1:172.21.15.170:6789/0 conn(0x7f06980c94c0 legacy=0x7f06980cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f069da6a700 1 -- 172.21.15.170:0/2638674509 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069815bb20 con 0x7f06980cd6a0 2023-10-08T16:13:24.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f069da6a700 1 --2- 172.21.15.170:0/2638674509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980cb4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e90 tx=0x7f0690009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22eaa44fa9e8d498 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:24.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000e070 con 0x7f06980cd6a0 2023-10-08T16:13:24.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000b870 con 0x7f06980cd6a0 2023-10-08T16:13:24.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.496+0000 7f068ffff700 1 -- 172.21.15.170:0/2638674509 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688001470 con 0x7f06980cd6a0 2023-10-08T16:13:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 -- 172.21.15.170:0/2638674509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 msgr2=0x7f06980cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:24.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 --2- 172.21.15.170:0/2638674509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980cb4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0690009e90 tx=0x7f0690009780 comp rx=0 tx=0).stop 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 -- 172.21.15.170:0/2638674509 shutdown_connections 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 --2- 172.21.15.170:0/2638674509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980cb4e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 -- 172.21.15.170:0/2638674509 >> 172.21.15.170:0/2638674509 conn(0x7f069806baf0 msgr2=0x7f069806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 -- 172.21.15.170:0/2638674509 shutdown_connections 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.497+0000 7f069e26b700 1 -- 172.21.15.170:0/2638674509 wait complete. 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.498+0000 7f069e26b700 1 Processor -- start 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.498+0000 7f069e26b700 1 -- start start 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:24.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0698157540 0x7f0698153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06981531b0 con 0x7f06980c94c0 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698152350 con 0x7f0698157540 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069e26b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06981525a0 con 0x7f06980cd6a0 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56008/0 (socket says 172.21.15.170:56008) 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 -- 172.21.15.170:0/4213989120 learned_addr learned my addr 172.21.15.170:0/4213989120 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069da6a700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0698157540 0x7f0698153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 -- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 msgr2=0x7f06980c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 -- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0698157540 msgr2=0x7f0698153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0698157540 0x7f0698153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:24.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 -- 172.21.15.170:0/4213989120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690009a20 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.499+0000 7f069d269700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f069400c580 tx=0x7f0694007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.500+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694010510 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.500+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0694010670 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.500+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0698152880 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.500+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694010a30 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.500+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0698152e40 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.501+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069804c3c0 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.501+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f069401b400 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.501+0000 7f068d7fa700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 0x7f067c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.501+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f06940848d0 con 0x7f06980c94c0 2023-10-08T16:13:24.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.501+0000 7f069ca68700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 0x7f067c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.502+0000 7f069ca68700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 0x7f067c0697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f06880011e0 tx=0x7f0688001320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:24.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.506+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0694051560 con 0x7f06980c94c0 2023-10-08T16:13:24.744 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:24.743+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06980b09d0 con 0x7f06980c94c0 2023-10-08T16:13:25.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.585+0000 7f068d7fa700 1 -- 172.21.15.170:0/4213989120 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v121) v1 ==== 136+0+0 (secure 0 0 0) 0x7f0694054b80 con 0x7f06980c94c0 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 msgr2=0x7f067c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 0x7f067c0697d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f06880011e0 tx=0x7f0688001320 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 msgr2=0x7f0698149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f069400c580 tx=0x7f0694007e10 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 shutdown_connections 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0698157540 0x7f0698153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f067c067320 0x7f067c0697d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f06980c94c0 0x7f0698149360 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 --2- 172.21.15.170:0/4213989120 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f06980cd6a0 0x7f06980c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 >> 172.21.15.170:0/4213989120 conn(0x7f069806baf0 msgr2=0x7f06980ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:25.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 shutdown_connections 2023-10-08T16:13:25.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.588+0000 7f069e26b700 1 -- 172.21.15.170:0/4213989120 wait complete. 2023-10-08T16:13:25.602 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:13:25.717 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.716+0000 7f4f84b7b700 1 Processor -- start 2023-10-08T16:13:25.718 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.716+0000 7f4f84b7b700 1 -- start start 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f84b7b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f84b7b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 0x7f4f800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f84b7b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f800b11d0 con 0x7f4f8015d230 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f84b7b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f800b14b0 con 0x7f4f801597f0 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f84b7b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800b1790 con 0x7f4f80159420 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4f8015d230 0x7f4f800b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45648/0 (socket says 172.21.15.170:45648) 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7ffff700 1 -- 172.21.15.170:0/3789727031 learned_addr learned my addr 172.21.15.170:0/3789727031 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:25.720 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7effd700 1 --2- 172.21.15.170:0/3789727031 >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 0x7f4f800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7f7fe700 1 --2- 172.21.15.170:0/3789727031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7e7fc700 1 -- 172.21.15.170:0/3789727031 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094065419 0 0) 0x7f4f800b11d0 con 0x7f4f8015d230 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7e7fc700 1 -- 172.21.15.170:0/3789727031 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f68003570 con 0x7f4f8015d230 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7f7fe700 1 -- 172.21.15.170:0/3789727031 >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 msgr2=0x7f4f800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7f7fe700 1 --2- 172.21.15.170:0/3789727031 >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 0x7f4f800b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7f7fe700 1 -- 172.21.15.170:0/3789727031 >> v1:172.21.15.170:6789/0 conn(0x7f4f8015d230 legacy=0x7f4f800b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7f7fe700 1 -- 172.21.15.170:0/3789727031 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800b1a70 con 0x7f4f80159420 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.717+0000 7f4f7effd700 1 --2- 172.21.15.170:0/3789727031 >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 0x7f4f800b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f7f7fe700 1 --2- 172.21.15.170:0/3789727031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800b5f40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e60 tx=0x7f4f70009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b87e225de032ba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f7e7fc700 1 -- 172.21.15.170:0/3789727031 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000e070 con 0x7f4f80159420 2023-10-08T16:13:25.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f7e7fc700 1 -- 172.21.15.170:0/3789727031 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7000bab0 con 0x7f4f80159420 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f7e7fc700 1 -- 172.21.15.170:0/3789727031 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7000be70 con 0x7f4f80159420 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 -- 172.21.15.170:0/3789727031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 msgr2=0x7f4f800b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/3789727031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800b5f40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4f70009e60 tx=0x7f4f70009780 comp rx=0 tx=0).stop 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 -- 172.21.15.170:0/3789727031 shutdown_connections 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/3789727031 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800b5f40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/3789727031 >> v2:172.21.15.191:3300/0 conn(0x7f4f801597f0 0x7f4f800b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 -- 172.21.15.170:0/3789727031 >> 172.21.15.170:0/3789727031 conn(0x7f4f800d0660 msgr2=0x7f4f800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 -- 172.21.15.170:0/3789727031 shutdown_connections 2023-10-08T16:13:25.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.718+0000 7f4f84b7b700 1 -- 172.21.15.170:0/3789727031 wait complete. 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 Processor -- start 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 -- start start 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f801597f0 0x7f4f8014bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 0x7f4f800cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f800c7710 con 0x7f4f8015d230 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8016d070 con 0x7f4f801597f0 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.720+0000 7f4f84b7b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8016d350 con 0x7f4f80159420 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 0x7f4f800cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7f7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:38624/0 (socket says 172.21.15.170:38624) 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7f7fe700 1 -- 172.21.15.170:0/430133507 learned_addr learned my addr 172.21.15.170:0/430133507 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7effd700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f801597f0 0x7f4f8014bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 -- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 msgr2=0x7f4f800c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 -- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f801597f0 msgr2=0x7f4f8014bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f801597f0 0x7f4f8014bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 -- 172.21.15.170:0/430133507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74009420 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f7ffff700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 0x7f4f800cbbe0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4f780058d0 tx=0x7f4f78007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7800ce10 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78005b30 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780070d0 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f70009a20 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.721+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8016d910 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.722+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f80169940 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.722+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4f78007480 con 0x7f4f8015d230 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.723+0000 7f4f677fe700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 0x7f4f6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:25.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.723+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f4f78083e80 con 0x7f4f8015d230 2023-10-08T16:13:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.723+0000 7f4f7f7fe700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 0x7f4f6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.723+0000 7f4f7f7fe700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 0x7f4f6c0697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4f70003440 tx=0x7f4f70019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:25.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.727+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f7804b5a0 con 0x7f4f8015d230 2023-10-08T16:13:25.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.935+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f8004c3c0 con 0x7f4f6c067320 2023-10-08T16:13:25.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.936+0000 7f4f677fe700 1 -- 172.21.15.170:0/430133507 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4f8004c3c0 con 0x7f4f6c067320 2023-10-08T16:13:25.937 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:13:25.937 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:25.937 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:13:25.937 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:25.938 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:13:25.939 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 msgr2=0x7f4f6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 0x7f4f6c0697d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4f70003440 tx=0x7f4f70019000 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 msgr2=0x7f4f800cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 0x7f4f800cbbe0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f4f780058d0 tx=0x7f4f78007d10 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 shutdown_connections 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4f6c067320 0x7f4f6c0697d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f8015d230 0x7f4f800cbbe0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4f80159420 0x7f4f800c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 --2- 172.21.15.170:0/430133507 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4f801597f0 0x7f4f8014bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 >> 172.21.15.170:0/430133507 conn(0x7f4f800d0660 msgr2=0x7f4f800b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:25.940 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 shutdown_connections 2023-10-08T16:13:25.941 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:25.938+0000 7f4f84b7b700 1 -- 172.21.15.170:0/430133507 wait complete. 2023-10-08T16:13:25.951 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.067+0000 7f1ee9279700 1 Processor -- start 2023-10-08T16:13:26.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.067+0000 7f1ee9279700 1 -- start start 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee9279700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ee4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee9279700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee415c2a0 con 0x7f1ee406c0a0 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee9279700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee415b690 con 0x7f1ee40c9510 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee9279700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415b970 con 0x7f1ee406bcd0 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee27fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1ee406c0a0 0x7f1ee40c8e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45664/0 (socket says 172.21.15.170:45664) 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee27fc700 1 -- 172.21.15.170:0/826437036 learned_addr learned my addr 172.21.15.170:0/826437036 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.068+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162899819 0 0) 0x7f1ee415c2a0 con 0x7f1ee406c0a0 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ecc003570 con 0x7f1ee406c0a0 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ee4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.077 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1551708165 0 0) 0x7f1ecc003570 con 0x7f1ee406c0a0 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 msgr2=0x7f1ee4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ee4140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 >> v1:172.21.15.191:6789/0 conn(0x7f1ee40c9510 legacy=0x7f1ee40cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee415bc50 con 0x7f1ee406c0a0 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f1ed8001470 con 0x7f1ee406c0a0 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 >> v1:172.21.15.170:6789/0 conn(0x7f1ee406c0a0 legacy=0x7f1ee40c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ecc003a90 0x7f1ecc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ecc004810 0x7f1ecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ecc0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415b970 con 0x7f1ecc004810 2023-10-08T16:13:26.078 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00bb60 con 0x7f1ecc003a90 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00bd20 con 0x7f1ee406bcd0 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ecc004810 0x7f1ecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ecc003a90 0x7f1ecc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee37fe700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ecc0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 -- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 msgr2=0x7f1ecc0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ee406bcd0 0x7f1ecc0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 -- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ecc003a90 msgr2=0x7f1ecc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ecc003a90 0x7f1ecc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 -- 172.21.15.170:0/826437036 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc003570 con 0x7f1ecc004810 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ecc003a90 0x7f1ecc003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.069+0000 7f1ee27fc700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ecc004810 0x7f1ecc004c80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c390 tx=0x7f1ee4158030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809622acff26dc73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.079 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed800cc60 con 0x7f1ecc004810 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed800cdc0 con 0x7f1ecc004810 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee1ffb700 1 -- 172.21.15.170:0/826437036 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8005b70 con 0x7f1ecc004810 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 -- 172.21.15.170:0/826437036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ecc004810 msgr2=0x7f1ecc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 --2- 172.21.15.170:0/826437036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ecc004810 0x7f1ecc004c80 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c390 tx=0x7f1ee4158030 comp rx=0 tx=0).stop 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee37fe700 1 -- 172.21.15.170:0/826437036 reap_dead start 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 -- 172.21.15.170:0/826437036 shutdown_connections 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 -- 172.21.15.170:0/826437036 >> 172.21.15.170:0/826437036 conn(0x7f1ee40ce600 msgr2=0x7f1ee40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 -- 172.21.15.170:0/826437036 shutdown_connections 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.070+0000 7f1ee9279700 1 -- 172.21.15.170:0/826437036 wait complete. 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.071+0000 7f1ee9279700 1 Processor -- start 2023-10-08T16:13:26.080 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.071+0000 7f1ee9279700 1 -- start start 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ecc004810 0x7f1ee40a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee40a9b20 0x7f1ee4158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40a63c0 con 0x7f1ee4168e10 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee416b070 con 0x7f1ee40a9b20 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee9279700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4158dd0 con 0x7f1ecc004810 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46020/0 (socket says 172.21.15.170:46020) 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 -- 172.21.15.170:0/606049448 learned_addr learned my addr 172.21.15.170:0/606049448 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee27fc700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee40a9b20 0x7f1ee4158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ecc004810 0x7f1ee40a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.081 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 -- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ecc004810 msgr2=0x7f1ee40a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ecc004810 0x7f1ee40a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 -- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee40a9b20 msgr2=0x7f1ee4158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee40a9b20 0x7f1ee4158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.072+0000 7f1ee37fe700 1 -- 172.21.15.170:0/606049448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40a65b0 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ee37fe700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40cf210 tx=0x7f1edc001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc019070 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00ee70 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee40a6af0 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc006dd0 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.073+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee40a70b0 con 0x7f1ee4168e10 2023-10-08T16:13:26.082 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.074+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee404c3c0 con 0x7f1ee4168e10 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.074+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1edc00c9c0 con 0x7f1ee4168e10 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.074+0000 7f1ecaffd700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 0x7f1ed00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.075+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f1edc084d10 con 0x7f1ee4168e10 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.075+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 0x7f1ed00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.075+0000 7f1ee2ffd700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 0x7f1ed00697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006f00 tx=0x7f1ed40091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.079+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1edc051960 con 0x7f1ee4168e10 2023-10-08T16:13:26.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.264+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ee415bf30 con 0x7f1ee4168e10 2023-10-08T16:13:26.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.264+0000 7f1ecaffd700 1 -- 172.21.15.170:0/606049448 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1edc054f80 con 0x7f1ee4168e10 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 msgr2=0x7f1ed00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 0x7f1ed00697d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4006f00 tx=0x7f1ed40091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 msgr2=0x7f1ee4159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40cf210 tx=0x7f1edc001640 comp rx=0 tx=0).stop 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 shutdown_connections 2023-10-08T16:13:26.268 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1ed0067320 0x7f1ed00697d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee4168e10 0x7f1ee4159470 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1ecc004810 0x7f1ee40a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 --2- 172.21.15.170:0/606049448 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1ee40a9b20 0x7f1ee4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 >> 172.21.15.170:0/606049448 conn(0x7f1ee40ce600 msgr2=0x7f1ee40b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 shutdown_connections 2023-10-08T16:13:26.269 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.266+0000 7f1ee9279700 1 -- 172.21.15.170:0/606049448 wait complete. 2023-10-08T16:13:26.280 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:13:26.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:13:26.280 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:26.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:13:26.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:26.319 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:13:26.374 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:13:26.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:13:26.374 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:26.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:13:26.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:26.416 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:13:26.473 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:13:26.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:26.473 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T16:13:26.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:13:26.488 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T16:13:26.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:13:26.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:26.488 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T16:13:26.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:13:26.529 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T16:13:26.529 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000316055 --yes-i-really-mean-it 2023-10-08T16:13:26.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.657+0000 7f18bf15d700 1 Processor -- start 2023-10-08T16:13:26.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.657+0000 7f18bf15d700 1 -- start start 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 0x7f18b80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f18b80b2100 0x7f18b8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 0x7f18b8069800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b80acc00 con 0x7f18b80b2100 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b806d6a0 con 0x7f18b80688d0 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bf15d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8068d50 con 0x7f18b8157740 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f18b80b2100 0x7f18b8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f18b80b2100 0x7f18b8068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46036/0 (socket says 172.21.15.170:46036) 2023-10-08T16:13:26.661 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 -- 172.21.15.170:0/2112495687 learned_addr learned my addr 172.21.15.170:0/2112495687 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bd6fa700 1 --2- 172.21.15.170:0/2112495687 >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 0x7f18b8069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 -- 172.21.15.170:0/2112495687 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 msgr2=0x7f18b80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bcef9700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 0x7f18b80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 0x7f18b80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 -- 172.21.15.170:0/2112495687 >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 msgr2=0x7f18b8069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 --2- 172.21.15.170:0/2112495687 >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 0x7f18b8069800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 -- 172.21.15.170:0/2112495687 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b806dc20 con 0x7f18b80b2100 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bd6fa700 1 --2- 172.21.15.170:0/2112495687 >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 0x7f18b8069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18bcef9700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 0x7f18b80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.658+0000 7f18b7fff700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8068390 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f18b80c9fd0 tx=0x7f18a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabea7ba8af4bebf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18b77fe700 1 -- 172.21.15.170:0/2112495687 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400b820 con 0x7f18b80b2100 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18b77fe700 1 -- 172.21.15.170:0/2112495687 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400b980 con 0x7f18b80b2100 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18b77fe700 1 -- 172.21.15.170:0/2112495687 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400bd20 con 0x7f18b80b2100 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 -- 172.21.15.170:0/2112495687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 msgr2=0x7f18b8068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8068390 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f18b80c9fd0 tx=0x7f18a4009250 comp rx=0 tx=0).stop 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 -- 172.21.15.170:0/2112495687 shutdown_connections 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 --2- 172.21.15.170:0/2112495687 >> v2:172.21.15.191:3300/0 conn(0x7f18b80688d0 0x7f18b8069800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b8157740 0x7f18b80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 --2- 172.21.15.170:0/2112495687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8068390 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 -- 172.21.15.170:0/2112495687 >> 172.21.15.170:0/2112495687 conn(0x7f18b80c8f90 msgr2=0x7f18b80c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 -- 172.21.15.170:0/2112495687 shutdown_connections 2023-10-08T16:13:26.663 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.659+0000 7f18bf15d700 1 -- 172.21.15.170:0/2112495687 wait complete. 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.660+0000 7f18bf15d700 1 Processor -- start 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.660+0000 7f18bf15d700 1 -- start start 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b80688d0 0x7f18b80a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b8157740 0x7f18b8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80c8590 con 0x7f18b80b2100 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80c8a20 con 0x7f18b8157740 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bf15d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b80c7980 con 0x7f18b80688d0 2023-10-08T16:13:26.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18b7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46046/0 (socket says 172.21.15.170:46046) 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18b7fff700 1 -- 172.21.15.170:0/1931864538 learned_addr learned my addr 172.21.15.170:0/1931864538 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bcef9700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b80688d0 0x7f18b80a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.661+0000 7f18bd6fa700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b8157740 0x7f18b8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 -- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b80688d0 msgr2=0x7f18b80a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b80688d0 0x7f18b80a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 -- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b8157740 msgr2=0x7f18b8156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b8157740 0x7f18b8156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 -- 172.21.15.170:0/1931864538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40094f0 con 0x7f18b80b2100 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b7fff700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f18a40094c0 tx=0x7f18a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4011e40 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a4024440 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b80c7c60 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4024800 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.662+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b80c8220 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.663+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b80c83b0 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.663+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f18a401a420 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.664+0000 7f18b4ff9700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 0x7f189c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.664+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5763+0+0 (secure 0 0 0) 0x7f18a40866b0 con 0x7f18b80b2100 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.664+0000 7f18bcef9700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 0x7f189c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:26.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.664+0000 7f18bcef9700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 0x7f189c0697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:26.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.668+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a4053300 con 0x7f18b80b2100 2023-10-08T16:13:26.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:26.877+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000316055", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f18b80718b0 con 0x7f189c067320 2023-10-08T16:13:29.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.620+0000 7f18b4ff9700 1 -- 172.21.15.170:0/1931864538 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f18b80718b0 con 0x7f189c067320 2023-10-08T16:13:29.622 INFO:teuthology.orchestra.run.smithi170.stdout:FS volume 'cephfs' renamed to 'volume_0000000000316055' 2023-10-08T16:13:29.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 msgr2=0x7f189c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 0x7f189c0697d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac008040 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 msgr2=0x7f18b8156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f18a40094c0 tx=0x7f18a4017000 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 shutdown_connections 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f18b8157740 0x7f18b8156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f18b80688d0 0x7f18b80a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f189c067320 0x7f189c0697d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 --2- 172.21.15.170:0/1931864538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18b80b2100 0x7f18b8156650 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 >> 172.21.15.170:0/1931864538 conn(0x7f18b80c8f90 msgr2=0x7f18b80caee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 shutdown_connections 2023-10-08T16:13:29.625 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.623+0000 7f18bf15d700 1 -- 172.21.15.170:0/1931864538 wait complete. 2023-10-08T16:13:29.639 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rename cephfs volume_0000000000316055 --yes-i-really-mean-it 2023-10-08T16:13:29.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.763+0000 7f81bd777700 1 Processor -- start 2023-10-08T16:13:29.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.763+0000 7f81bd777700 1 -- start start 2023-10-08T16:13:29.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81bd777700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f81b80b55a0 0x7f81b8164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81bd777700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80cfa60 con 0x7f81b8159ec0 2023-10-08T16:13:29.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81bd777700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80b5b00 con 0x7f81b80cd910 2023-10-08T16:13:29.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81bd777700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b80b5da0 con 0x7f81b80b55a0 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81b6ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f81b8159ec0 0x7f81b80b4e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45680/0 (socket says 172.21.15.170:45680) 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.764+0000 7f81b6ffd700 1 -- 172.21.15.170:0/3108888309 learned_addr learned my addr 172.21.15.170:0/3108888309 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812906657 0 0) 0x7f81b80cfa60 con 0x7f81b8159ec0 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 --2- 172.21.15.170:0/3108888309 >> v2:172.21.15.191:3300/0 conn(0x7f81b80b55a0 0x7f81b8164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0003570 con 0x7f81b8159ec0 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 559015884 0 0) 0x7f81a0003570 con 0x7f81b8159ec0 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 >> v2:172.21.15.191:3300/0 conn(0x7f81b80b55a0 msgr2=0x7f81b8164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 --2- 172.21.15.170:0/3108888309 >> v2:172.21.15.191:3300/0 conn(0x7f81b80b55a0 0x7f81b8164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 >> v1:172.21.15.191:6789/0 conn(0x7f81b80cd910 legacy=0x7f81b80ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80b6080 con 0x7f81b8159ec0 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f81a8001970 con 0x7f81b8159ec0 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 >> v1:172.21.15.170:6789/0 conn(0x7f81b8159ec0 legacy=0x7f81b80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0003a60 0x7f81a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81a0004810 0x7f81a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 0x7f81a000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80b5da0 con 0x7f81a0005210 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a000c130 con 0x7f81a0003a60 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a000c2c0 con 0x7f81a0004810 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 0x7f81a000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b6ffd700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0003a60 0x7f81a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b67fc700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81a0004810 0x7f81a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 -- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81a0004810 msgr2=0x7f81a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81a0004810 0x7f81a0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 -- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0003a60 msgr2=0x7f81a0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0003a60 0x7f81a0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b77fe700 1 -- 172.21.15.170:0/3108888309 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a0003570 con 0x7f81a0005210 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.765+0000 7f81b67fc700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81a0004810 0x7f81a0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81b77fe700 1 -- 172.21.15.170:0/3108888309 reap_dead start 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81b77fe700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 0x7f81a000ba60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009c30 tx=0x7f81b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cc5fdf5823ce90 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00095b0 con 0x7f81a0005210 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b000e710 con 0x7f81a0005210 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81b5ffb700 1 -- 172.21.15.170:0/3108888309 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b000eaf0 con 0x7f81a0005210 2023-10-08T16:13:29.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 -- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 msgr2=0x7f81a000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 0x7f81a000ba60 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009c30 tx=0x7f81b00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 -- 172.21.15.170:0/3108888309 shutdown_connections 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 --2- 172.21.15.170:0/3108888309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81a0005210 0x7f81a000ba60 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 -- 172.21.15.170:0/3108888309 >> 172.21.15.170:0/3108888309 conn(0x7f81b80999f0 msgr2=0x7f81b80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 -- 172.21.15.170:0/3108888309 shutdown_connections 2023-10-08T16:13:29.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.766+0000 7f81bd777700 1 -- 172.21.15.170:0/3108888309 wait complete. 2023-10-08T16:13:29.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 Processor -- start 2023-10-08T16:13:29.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 -- start start 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0005210 0x7f81b8150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81b8169930 0x7f81b80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b81549a0 con 0x7f81b816cf60 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8068390 con 0x7f81a0005210 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81bd777700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b816d3f0 con 0x7f81b8169930 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81b67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81b67fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46064/0 (socket says 172.21.15.170:46064) 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.768+0000 7f81b67fc700 1 -- 172.21.15.170:0/1791567869 learned_addr learned my addr 172.21.15.170:0/1791567869 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b6ffd700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0005210 0x7f81b8150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b77fe700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81b8169930 0x7f81b80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 -- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81b8169930 msgr2=0x7f81b80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81b8169930 0x7f81b80b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 -- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0005210 msgr2=0x7f81b8150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0005210 0x7f81b8150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 -- 172.21.15.170:0/1791567869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b816d6d0 con 0x7f81b816cf60 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81b67fc700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00e580 tx=0x7f81ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac011460 con 0x7f81b816cf60 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ac010040 con 0x7f81b816cf60 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b816d9b0 con 0x7f81b816cf60 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ac0117f0 con 0x7f81b816cf60 2023-10-08T16:13:29.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.769+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b806a3b0 con 0x7f81b816cf60 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.770+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b8063c90 con 0x7f81b816cf60 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.771+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f81ac011be0 con 0x7f81b816cf60 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.771+0000 7f819effd700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 0x7f81a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.771+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f81ac083f00 con 0x7f81b816cf60 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.771+0000 7f81b6ffd700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 0x7f81a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.771+0000 7f81b6ffd700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 0x7f81a40697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007d30 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:29.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.775+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81ac04d5a0 con 0x7f81b816cf60 2023-10-08T16:13:29.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:29.986+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rename", "vol_name": "cephfs", "new_vol_name": "volume_0000000000316055", "yes_i_really_mean_it": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f81b80718b0 con 0x7f81a4067320 2023-10-08T16:13:30.626 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.625+0000 7f819effd700 1 -- 172.21.15.170:0/1791567869 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+56 (secure 0 0 0) 0x7f81b80718b0 con 0x7f81a4067320 2023-10-08T16:13:30.626 INFO:teuthology.orchestra.run.smithi170.stdout:FS volume 'cephfs' renamed to 'volume_0000000000316055' 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 msgr2=0x7f81a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 0x7f81a40697d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f81a8007d30 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 msgr2=0x7f81b8069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f81ac00e580 tx=0x7f81ac009040 comp rx=0 tx=0).stop 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 shutdown_connections 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f81b8169930 0x7f81b80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81a0005210 0x7f81b8150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f81a4067320 0x7f81a40697d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 --2- 172.21.15.170:0/1791567869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81b816cf60 0x7f81b8069e70 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 >> 172.21.15.170:0/1791567869 conn(0x7f81b80999f0 msgr2=0x7f81b8152050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:30.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 shutdown_connections 2023-10-08T16:13:30.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.628+0000 7f81bd777700 1 -- 172.21.15.170:0/1791567869 wait complete. 2023-10-08T16:13:30.642 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:13:30.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.758+0000 7fe89f80b700 1 Processor -- start 2023-10-08T16:13:30.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.758+0000 7fe89f80b700 1 -- start start 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 0x7fe89815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe898159c60 0x7fe89815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe89815db10 0x7fe898069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89815a160 con 0x7fe89815db10 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8980d13a0 con 0x7fe898159890 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89f80b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89815e140 con 0x7fe898159c60 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89dda8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe89815db10 0x7fe898069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89dda8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe89815db10 0x7fe898069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46076/0 (socket says 172.21.15.170:46076) 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.759+0000 7fe89dda8700 1 -- 172.21.15.170:0/2914865811 learned_addr learned my addr 172.21.15.170:0/2914865811 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89d5a7700 1 --2- 172.21.15.170:0/2914865811 >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 0x7fe89815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89cda6700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe898159c60 0x7fe89815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 -- 172.21.15.170:0/2914865811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe898159c60 msgr2=0x7fe89815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe898159c60 0x7fe89815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 -- 172.21.15.170:0/2914865811 >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 msgr2=0x7fe89815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 --2- 172.21.15.170:0/2914865811 >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 0x7fe89815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 -- 172.21.15.170:0/2914865811 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89815a3d0 con 0x7fe89815db10 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89d5a7700 1 --2- 172.21.15.170:0/2914865811 >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 0x7fe89815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89dda8700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe89815db10 0x7fe898069560 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe8980d1090 tx=0x7fe890009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f31dd5d8ad2ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe88ffff700 1 -- 172.21.15.170:0/2914865811 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000e070 con 0x7fe89815db10 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe88ffff700 1 -- 172.21.15.170:0/2914865811 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89000ba50 con 0x7fe89815db10 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe88ffff700 1 -- 172.21.15.170:0/2914865811 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8900092e0 con 0x7fe89815db10 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89f80b700 1 -- 172.21.15.170:0/2914865811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe89815db10 msgr2=0x7fe898069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.760+0000 7fe89f80b700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe89815db10 0x7fe898069560 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe8980d1090 tx=0x7fe890009840 comp rx=0 tx=0).stop 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 -- 172.21.15.170:0/2914865811 shutdown_connections 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe89815db10 0x7fe898069560 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 --2- 172.21.15.170:0/2914865811 >> v2:172.21.15.191:3300/0 conn(0x7fe898159890 0x7fe89815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 --2- 172.21.15.170:0/2914865811 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe898159c60 0x7fe89815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 -- 172.21.15.170:0/2914865811 >> 172.21.15.170:0/2914865811 conn(0x7fe898068e20 msgr2=0x7fe8980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 -- 172.21.15.170:0/2914865811 shutdown_connections 2023-10-08T16:13:30.765 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.761+0000 7fe89f80b700 1 -- 172.21.15.170:0/2914865811 wait complete. 2023-10-08T16:13:30.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.762+0000 7fe89f80b700 1 Processor -- start 2023-10-08T16:13:30.766 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.762+0000 7fe89f80b700 1 -- start start 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 0x7fe898153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 0x7fe898142230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe898153e50 con 0x7fe898159890 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89814f540 con 0x7fe898159c60 2023-10-08T16:13:30.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe898154060 con 0x7fe89815db10 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46092/0 (socket says 172.21.15.170:46092) 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 -- 172.21.15.170:0/4266415585 learned_addr learned my addr 172.21.15.170:0/4266415585 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89dda8700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 0x7fe898142230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89cda6700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 0x7fe898153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 -- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 msgr2=0x7fe898142230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 0x7fe898142230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 -- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 msgr2=0x7fe898153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 0x7fe898153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 -- 172.21.15.170:0/4266415585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe890009ae0 con 0x7fe898159890 2023-10-08T16:13:30.769 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89dda8700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 0x7fe898142230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89cda6700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 0x7fe898153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89d5a7700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c830 tx=0x7fe894008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011600 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894011760 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe898154340 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011b20 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.763+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89814f1d0 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.764+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89804c3c0 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.765+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe89400a3e0 con 0x7fe898159890 2023-10-08T16:13:30.770 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.765+0000 7fe88d7fa700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 0x7fe87c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:30.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.765+0000 7fe89cda6700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 0x7fe87c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:30.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.765+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7fe894085320 con 0x7fe898159890 2023-10-08T16:13:30.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.766+0000 7fe89cda6700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 0x7fe87c0697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe8980d16f0 tx=0x7fe888008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:30.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.769+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe894051e10 con 0x7fe898159890 2023-10-08T16:13:30.981 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.980+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe89814f680 con 0x7fe87c067320 2023-10-08T16:13:30.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.981+0000 7fe88d7fa700 1 -- 172.21.15.170:0/4266415585 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+58 (secure 0 0 0) 0x7fe88800f6b0 con 0x7fe87c067320 2023-10-08T16:13:30.983 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:13:30.983 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:30.983 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "volume_0000000000316055" 2023-10-08T16:13:30.983 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:30.983 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:13:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.983+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 msgr2=0x7fe87c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.983+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 0x7fe87c0697d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe8980d16f0 tx=0x7fe888008040 comp rx=0 tx=0).stop 2023-10-08T16:13:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.983+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 msgr2=0x7fe89816a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:30.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.983+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c830 tx=0x7fe894008040 comp rx=0 tx=0).stop 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 shutdown_connections 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe89815db10 0x7fe898142230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe87c067320 0x7fe87c0697d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe898159890 0x7fe89816a390 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 --2- 172.21.15.170:0/4266415585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe898159c60 0x7fe898153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 >> 172.21.15.170:0/4266415585 conn(0x7fe898068e20 msgr2=0x7fe8980cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 shutdown_connections 2023-10-08T16:13:30.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:30.984+0000 7fe89f80b700 1 -- 172.21.15.170:0/4266415585 wait complete. 2023-10-08T16:13:30.997 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:31.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.109+0000 7f9e7764f700 1 Processor -- start 2023-10-08T16:13:31.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.109+0000 7f9e7764f700 1 -- start start 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9e70159890 0x7f9e7015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7015a160 con 0x7f9e70159c60 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e700d13a0 con 0x7f9e70069560 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e7015e140 con 0x7f9e70159890 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9e70159890 0x7f9e7015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e74bea700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9e70159c60 0x7f9e7015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45690/0 (socket says 172.21.15.170:45690) 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e74bea700 1 -- 172.21.15.170:0/1341494270 learned_addr learned my addr 172.21.15.170:0/1341494270 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:31.113 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9e70159890 0x7f9e7015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46096/0 (socket says 172.21.15.170:46096) 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 -- 172.21.15.170:0/1341494270 >> v1:172.21.15.191:6789/0 conn(0x7f9e70069560 legacy=0x7f9e700ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 -- 172.21.15.170:0/1341494270 >> v1:172.21.15.170:6789/0 conn(0x7f9e70159c60 legacy=0x7f9e7015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 -- 172.21.15.170:0/1341494270 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7015a3d0 con 0x7f9e70159890 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e67fff700 1 -- 172.21.15.170:0/1341494270 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150978279 0 0) 0x7f9e7015a160 con 0x7f9e70159c60 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e753eb700 1 --2- 172.21.15.170:0/1341494270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e7015abd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e60 tx=0x7f9e6c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e378769e65925c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e67fff700 1 -- 172.21.15.170:0/1341494270 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c00e070 con 0x7f9e70159890 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e67fff700 1 -- 172.21.15.170:0/1341494270 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6c00bc30 con 0x7f9e70159890 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e67fff700 1 -- 172.21.15.170:0/1341494270 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c012430 con 0x7f9e70159890 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- 172.21.15.170:0/1341494270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 msgr2=0x7f9e7015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 --2- 172.21.15.170:0/1341494270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e7015abd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e60 tx=0x7f9e6c009780 comp rx=0 tx=0).stop 2023-10-08T16:13:31.114 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- 172.21.15.170:0/1341494270 shutdown_connections 2023-10-08T16:13:31.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 --2- 172.21.15.170:0/1341494270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e7015abd0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.111+0000 7f9e7764f700 1 -- 172.21.15.170:0/1341494270 >> 172.21.15.170:0/1341494270 conn(0x7f9e70068e20 msgr2=0x7f9e700d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:31.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.112+0000 7f9e7764f700 1 -- 172.21.15.170:0/1341494270 shutdown_connections 2023-10-08T16:13:31.115 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.112+0000 7f9e7764f700 1 -- 172.21.15.170:0/1341494270 wait complete. 2023-10-08T16:13:31.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.113+0000 7f9e7764f700 1 Processor -- start 2023-10-08T16:13:31.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.113+0000 7f9e7764f700 1 -- start start 2023-10-08T16:13:31.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.113+0000 7f9e7764f700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e70069560 0x7f9e70152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.113+0000 7f9e7764f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 0x7f9e700b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e70153190 con 0x7f9e70159890 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700c96f0 con 0x7f9e70069560 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700c99d0 con 0x7f9e70159c60 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46106/0 (socket says 172.21.15.170:46106) 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 -- 172.21.15.170:0/3280638585 learned_addr learned my addr 172.21.15.170:0/3280638585 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e753eb700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e70069560 0x7f9e70152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e75bec700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 0x7f9e700b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 -- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 msgr2=0x7f9e700b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 0x7f9e700b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 -- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e70069560 msgr2=0x7f9e70152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e70069560 0x7f9e70152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 -- 172.21.15.170:0/3280638585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c009a20 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e75bec700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 0x7f9e700b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e74bea700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c5b0 tx=0x7f9e60001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60005b30 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e60005c90 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e700c9cb0 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60007300 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e700c93e0 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.115+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7004c3c0 con 0x7f9e70159890 2023-10-08T16:13:31.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.116+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9e6001b3b0 con 0x7f9e70159890 2023-10-08T16:13:31.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.116+0000 7f9e657fa700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 0x7f9e540697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.116+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f9e600841e0 con 0x7f9e70159890 2023-10-08T16:13:31.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.116+0000 7f9e753eb700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 0x7f9e540697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.117+0000 7f9e753eb700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 0x7f9e540697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e60 tx=0x7f9e6c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.121+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e60050de0 con 0x7f9e70159890 2023-10-08T16:13:31.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.347+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e700b1360 con 0x7f9e70159890 2023-10-08T16:13:31.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.348+0000 7f9e657fa700 1 -- 172.21.15.170:0/3280638585 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4636 (secure 0 0 0) 0x7f9e60054400 con 0x7f9e70159890 2023-10-08T16:13:31.349 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:31.350 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:27.591305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000316055","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:31.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 msgr2=0x7f9e540697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 0x7f9e540697d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009e60 tx=0x7f9e6c019000 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 msgr2=0x7f9e700c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000c5b0 tx=0x7f9e60001070 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 shutdown_connections 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9e70159c60 0x7f9e700b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9e70069560 0x7f9e70152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9e54067320 0x7f9e540697d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 --2- 172.21.15.170:0/3280638585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9e70159890 0x7f9e700c8ea0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 >> 172.21.15.170:0/3280638585 conn(0x7f9e70068e20 msgr2=0x7f9e700b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 shutdown_connections 2023-10-08T16:13:31.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.351+0000 7f9e7764f700 1 -- 172.21.15.170:0/3280638585 wait complete. 2023-10-08T16:13:31.353 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2023-10-08T16:13:31.363 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:31.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.477+0000 7f814457d700 1 Processor -- start 2023-10-08T16:13:31.478 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.477+0000 7f814457d700 1 -- start start 2023-10-08T16:13:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f814457d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f813c141500 0x7f813c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f814457d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c15c2a0 con 0x7f813c06c0a0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f814457d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c15b690 con 0x7f813c06bcd0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f814457d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c15b970 con 0x7f813c141500 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8142b1a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f813c06c0a0 0x7f813c0c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45692/0 (socket says 172.21.15.170:45692) 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8142b1a700 1 -- 172.21.15.170:0/2206899712 learned_addr learned my addr 172.21.15.170:0/2206899712 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141b18700 1 --2- 172.21.15.170:0/2206899712 >> v2:172.21.15.191:3300/0 conn(0x7f813c141500 0x7f813c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990203275 0 0) 0x7f813c15c2a0 con 0x7f813c06c0a0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8128003570 con 0x7f813c06c0a0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858869319 0 0) 0x7f8128003570 con 0x7f813c06c0a0 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 >> v2:172.21.15.191:3300/0 conn(0x7f813c141500 msgr2=0x7f813c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 --2- 172.21.15.170:0/2206899712 >> v2:172.21.15.191:3300/0 conn(0x7f813c141500 0x7f813c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.482 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 >> v1:172.21.15.191:6789/0 conn(0x7f813c06bcd0 legacy=0x7f813c140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c15bc50 con 0x7f813c06c0a0 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f81340017f0 con 0x7f813c06c0a0 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 >> v1:172.21.15.170:6789/0 conn(0x7f813c06c0a0 legacy=0x7f813c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128003a60 0x7f8128003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 0x7f8128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81280051c0 0x7f812800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c15b970 con 0x7f8128004810 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812800c050 con 0x7f81280051c0 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812800c210 con 0x7f8128003a60 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.478+0000 7f8141b18700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 0x7f8128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8142319700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128003a60 0x7f8128003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8142b1a700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81280051c0 0x7f812800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 -- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128003a60 msgr2=0x7f8128003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128003a60 0x7f8128003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 -- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81280051c0 msgr2=0x7f812800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81280051c0 0x7f812800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 -- 172.21.15.170:0/2206899712 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128003570 con 0x7f8128004810 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8142b1a700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f81280051c0 0x7f812800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8142b1a700 1 -- 172.21.15.170:0/2206899712 reap_dead start 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141b18700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 0x7f8128004c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f8138009c00 tx=0x7f81380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d8656982740d18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800e710 con 0x7f8128004810 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813800e870 con 0x7f8128004810 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f8141317700 1 -- 172.21.15.170:0/2206899712 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800ec30 con 0x7f8128004810 2023-10-08T16:13:31.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f814457d700 1 -- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 msgr2=0x7f8128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.479+0000 7f814457d700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 0x7f8128004c80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f8138009c00 tx=0x7f81380091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.480+0000 7f814457d700 1 -- 172.21.15.170:0/2206899712 shutdown_connections 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.480+0000 7f814457d700 1 --2- 172.21.15.170:0/2206899712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128004810 0x7f8128004c80 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.480+0000 7f814457d700 1 -- 172.21.15.170:0/2206899712 >> 172.21.15.170:0/2206899712 conn(0x7f813c0ce600 msgr2=0x7f813c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.480+0000 7f814457d700 1 -- 172.21.15.170:0/2206899712 shutdown_connections 2023-10-08T16:13:31.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.480+0000 7f814457d700 1 -- 172.21.15.170:0/2206899712 wait complete. 2023-10-08T16:13:31.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 Processor -- start 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 -- start start 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128004810 0x7f813c13ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f813c069bd0 0x7f813c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0694a0 con 0x7f813c13ea10 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0b0490 con 0x7f813c069bd0 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f814457d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0c8560 con 0x7f8128004810 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.481+0000 7f8141b18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46118/0 (socket says 172.21.15.170:46118) 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 -- 172.21.15.170:0/868129083 learned_addr learned my addr 172.21.15.170:0/868129083 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8142319700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128004810 0x7f813c13ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8142b1a700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f813c069bd0 0x7f813c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 -- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128004810 msgr2=0x7f813c13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128004810 0x7f813c13ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 -- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f813c069bd0 msgr2=0x7f813c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f813c069bd0 0x7f813c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 -- 172.21.15.170:0/868129083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0c8840 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f8141b18700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8138013f50 tx=0x7f813801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138022070 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813800e9e0 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c16a950 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81380276a0 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.482+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c0696b0 con 0x7f813c13ea10 2023-10-08T16:13:31.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.483+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c04c3c0 con 0x7f813c13ea10 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.483+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8138027ad0 con 0x7f813c13ea10 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.484+0000 7f81327fc700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 0x7f81200697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.484+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f81380866f0 con 0x7f813c13ea10 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.484+0000 7f8142319700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 0x7f81200697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.484+0000 7f8142319700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 0x7f81200697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f812c003c90 tx=0x7f812c007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.489+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81380533b0 con 0x7f813c13ea10 2023-10-08T16:13:31.677 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.677+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f813c0694a0 con 0x7f813c13ea10 2023-10-08T16:13:31.678 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.677+0000 7f81327fc700 1 -- 172.21.15.170:0/868129083 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15586 (secure 0 0 0) 0x7f81380569d0 con 0x7f813c13ea10 2023-10-08T16:13:31.678 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:31.678 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":189,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:29.612723+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000316055.meta","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000316055"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000316055.data","create_time":"2023-10-08T16:13:14.874207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000316055"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.c","mappings":[{"from":2,"to":0}]},{"pgid":"32.f","mappings":[{"from":6,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 msgr2=0x7f81200697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 0x7f81200697d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f812c003c90 tx=0x7f812c007d10 comp rx=0 tx=0).stop 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 msgr2=0x7f813c0aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f8138013f50 tx=0x7f813801a000 comp rx=0 tx=0).stop 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 shutdown_connections 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f813c069bd0 0x7f813c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128004810 0x7f813c13ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.680 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8120067320 0x7f81200697d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 --2- 172.21.15.170:0/868129083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f813c13ea10 0x7f813c0aff50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 >> 172.21.15.170:0/868129083 conn(0x7f813c0ce600 msgr2=0x7f813c0cc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 shutdown_connections 2023-10-08T16:13:31.681 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.679+0000 7f814457d700 1 -- 172.21.15.170:0/868129083 wait complete. 2023-10-08T16:13:31.692 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:13:31.692 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:31.731 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:31.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:13:31.773 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:31.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.887+0000 7ff791425700 1 Processor -- start 2023-10-08T16:13:31.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.887+0000 7ff791425700 1 -- start start 2023-10-08T16:13:31.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff791425700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff78c0b5ea0 0x7ff78c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff791425700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c15c480 con 0x7ff78c068bb0 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff791425700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c15b870 con 0x7ff78c159c10 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff791425700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0b2a40 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff7827fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff78c0b5ea0 0x7ff78c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff78b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff78c068bb0 0x7ff78c15c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45696/0 (socket says 172.21.15.170:45696) 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff78b7fe700 1 -- 172.21.15.170:0/3650356665 learned_addr learned my addr 172.21.15.170:0/3650356665 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff7827fc700 1 -- 172.21.15.170:0/3650356665 >> v1:172.21.15.191:6789/0 conn(0x7ff78c159c10 legacy=0x7ff78c15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff7827fc700 1 -- 172.21.15.170:0/3650356665 >> v1:172.21.15.170:6789/0 conn(0x7ff78c068bb0 legacy=0x7ff78c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.888+0000 7ff7827fc700 1 -- 172.21.15.170:0/3650356665 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c15bb50 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff7827fc700 1 --2- 172.21.15.170:0/3650356665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c068670 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098e0 tx=0x7ff7740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b973c6f9da91ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff78a7fc700 1 -- 172.21.15.170:0/3650356665 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774009e80 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff78a7fc700 1 -- 172.21.15.170:0/3650356665 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77400b820 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff78a7fc700 1 -- 172.21.15.170:0/3650356665 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400bba0 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 -- 172.21.15.170:0/3650356665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 msgr2=0x7ff78c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 --2- 172.21.15.170:0/3650356665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c068670 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098e0 tx=0x7ff7740091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 -- 172.21.15.170:0/3650356665 shutdown_connections 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 --2- 172.21.15.170:0/3650356665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c068670 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 -- 172.21.15.170:0/3650356665 >> 172.21.15.170:0/3650356665 conn(0x7ff78c099a20 msgr2=0x7ff78c0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 -- 172.21.15.170:0/3650356665 shutdown_connections 2023-10-08T16:13:31.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.889+0000 7ff791425700 1 -- 172.21.15.170:0/3650356665 wait complete. 2023-10-08T16:13:31.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.890+0000 7ff791425700 1 Processor -- start 2023-10-08T16:13:31.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 -- start start 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff78c068bb0 0x7ff78c1484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff78c159c10 0x7ff78c09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c1489f0 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c14b000 con 0x7ff78c068bb0 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff791425700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c14b2e0 con 0x7ff78c159c10 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff7827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff7827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46148/0 (socket says 172.21.15.170:46148) 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff7827fc700 1 -- 172.21.15.170:0/871099823 learned_addr learned my addr 172.21.15.170:0/871099823 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:31.894 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff78affd700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff78c068bb0 0x7ff78c1484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.891+0000 7ff78b7fe700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff78c159c10 0x7ff78c09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 -- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff78c159c10 msgr2=0x7ff78c09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff78c159c10 0x7ff78c09a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 -- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff78c068bb0 msgr2=0x7ff78c1484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff78c068bb0 0x7ff78c1484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 -- 172.21.15.170:0/871099823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009420 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7827fc700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff7740030c0 tx=0x7ff774016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77400b990 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774020070 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774025480 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c14b5c0 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.892+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c14bb80 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.893+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c0b6580 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.893+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff774022030 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.894+0000 7ff7837fe700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 0x7ff76c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:31.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.894+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7ff774086870 con 0x7ff78c0b5ea0 2023-10-08T16:13:31.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.894+0000 7ff78affd700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 0x7ff76c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:31.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.894+0000 7ff78affd700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 0x7ff76c0697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0093e0 tx=0x7ff77c015040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:31.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:31.898+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff774053580 con 0x7ff78c0b5ea0 2023-10-08T16:13:32.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.126+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff78c04c3c0 con 0x7ff78c0b5ea0 2023-10-08T16:13:32.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.126+0000 7ff7837fe700 1 -- 172.21.15.170:0/871099823 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4636 (secure 0 0 0) 0x7ff77402a090 con 0x7ff78c0b5ea0 2023-10-08T16:13:32.127 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:32.127 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:27.591305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000316055","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:32.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 msgr2=0x7ff76c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 0x7ff76c0697d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0093e0 tx=0x7ff77c015040 comp rx=0 tx=0).stop 2023-10-08T16:13:32.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 msgr2=0x7ff78c09a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff7740030c0 tx=0x7ff774016000 comp rx=0 tx=0).stop 2023-10-08T16:13:32.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 shutdown_connections 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff78c159c10 0x7ff78c09a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff78c068bb0 0x7ff78c1484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff76c067320 0x7ff76c0697d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 --2- 172.21.15.170:0/871099823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff78c0b5ea0 0x7ff78c09a490 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 >> 172.21.15.170:0/871099823 conn(0x7ff78c099a20 msgr2=0x7ff78c09d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 shutdown_connections 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.129+0000 7ff791425700 1 -- 172.21.15.170:0/871099823 wait complete. 2023-10-08T16:13:32.130 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2023-10-08T16:13:32.141 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:32.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.255+0000 7f787e024700 1 Processor -- start 2023-10-08T16:13:32.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.255+0000 7f787e024700 1 -- start start 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f787e024700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7878159c60 0x7f787815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f787e024700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787815a160 con 0x7f7878159890 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f787e024700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78780d13a0 con 0x7f787815db10 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f787e024700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787815df50 con 0x7f7878159c60 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f78777fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7878159890 0x7f787815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45704/0 (socket says 172.21.15.170:45704) 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f78777fe700 1 -- 172.21.15.170:0/3358626172 learned_addr learned my addr 172.21.15.170:0/3358626172 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:32.258 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f7876ffd700 1 --2- 172.21.15.170:0/3358626172 >> v2:172.21.15.191:3300/0 conn(0x7f7878159c60 0x7f787815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252050763 0 0) 0x7f787815a160 con 0x7f7878159890 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.256+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7860003570 con 0x7f7878159890 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f7876ffd700 1 -- 172.21.15.170:0/3358626172 >> v1:172.21.15.191:6789/0 conn(0x7f787815db10 legacy=0x7f7878069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3826262226 0 0) 0x7f7860003570 con 0x7f7878159890 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f7876ffd700 1 -- 172.21.15.170:0/3358626172 >> v1:172.21.15.170:6789/0 conn(0x7f7878159890 legacy=0x7f787815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f7876ffd700 1 -- 172.21.15.170:0/3358626172 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787815e140 con 0x7f7878159c60 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f7876ffd700 1 --2- 172.21.15.170:0/3358626172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f787815d5d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098e0 tx=0x7f786c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e380a16a95d72c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680017f0 con 0x7f7878159c60 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c011420 con 0x7f7878159c60 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f78767fc700 1 -- 172.21.15.170:0/3358626172 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0117a0 con 0x7f7878159c60 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f787e024700 1 -- 172.21.15.170:0/3358626172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 msgr2=0x7f787815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.257+0000 7f787e024700 1 --2- 172.21.15.170:0/3358626172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f787815d5d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098e0 tx=0x7f786c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.258+0000 7f787e024700 1 -- 172.21.15.170:0/3358626172 shutdown_connections 2023-10-08T16:13:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.258+0000 7f787e024700 1 --2- 172.21.15.170:0/3358626172 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f787815d5d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.258+0000 7f787e024700 1 -- 172.21.15.170:0/3358626172 >> 172.21.15.170:0/3358626172 conn(0x7f7878068e20 msgr2=0x7f78780d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.258+0000 7f787e024700 1 -- 172.21.15.170:0/3358626172 shutdown_connections 2023-10-08T16:13:32.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.258+0000 7f787e024700 1 -- 172.21.15.170:0/3358626172 wait complete. 2023-10-08T16:13:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 Processor -- start 2023-10-08T16:13:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 -- start start 2023-10-08T16:13:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 0x7f78780cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f7878145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780cf040 con 0x7f787815db10 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c6560 con 0x7f7878159c60 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.259+0000 7f787e024700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780cd640 con 0x7f7878159890 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46152/0 (socket says 172.21.15.170:46152) 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 -- 172.21.15.170:0/533126940 learned_addr learned my addr 172.21.15.170:0/533126940 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f78777fe700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 0x7f78780cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7876ffd700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f7878145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 -- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 msgr2=0x7f78780cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 0x7f78780cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 -- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 msgr2=0x7f7878145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f7878145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 -- 172.21.15.170:0/533126940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009420 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f78777fe700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 0x7f78780cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f7877fff700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f787000c4c0 tx=0x7f7870007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870011590 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870011710 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780c5950 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870011ad0 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.260+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780c5f10 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.261+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787804c3c0 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.262+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f787001b3b0 con 0x7f787815db10 2023-10-08T16:13:32.264 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.262+0000 7f785f7fe700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.262+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f78700841a0 con 0x7f787815db10 2023-10-08T16:13:32.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.262+0000 7f78777fe700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.265 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.262+0000 7f78777fe700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 0x7f78640697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7868001ec0 tx=0x7f7868000f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.266 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.266+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7870050da0 con 0x7f787815db10 2023-10-08T16:13:32.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.451+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7878089680 con 0x7f787815db10 2023-10-08T16:13:32.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.452+0000 7f785f7fe700 1 -- 172.21.15.170:0/533126940 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15586 (secure 0 0 0) 0x7f7870017090 con 0x7f787815db10 2023-10-08T16:13:32.453 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:32.453 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":189,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:29.612723+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000316055.meta","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000316055"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000316055.data","create_time":"2023-10-08T16:13:14.874207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000316055"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.c","mappings":[{"from":2,"to":0}]},{"pgid":"32.f","mappings":[{"from":6,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 msgr2=0x7f78640697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 0x7f78640697d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7868001ec0 tx=0x7f7868000f40 comp rx=0 tx=0).stop 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 msgr2=0x7f7878154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f787000c4c0 tx=0x7f7870007f50 comp rx=0 tx=0).stop 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 shutdown_connections 2023-10-08T16:13:32.455 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7864067320 0x7f78640697d0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f787815db10 0x7f7878154280 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7878159890 0x7f78780cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 --2- 172.21.15.170:0/533126940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7878159c60 0x7f7878145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 >> 172.21.15.170:0/533126940 conn(0x7f7878068e20 msgr2=0x7f78781554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 shutdown_connections 2023-10-08T16:13:32.456 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.454+0000 7f787e024700 1 -- 172.21.15.170:0/533126940 wait complete. 2023-10-08T16:13:32.467 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 31, 'pool_name': 'cephfs.volume_0000000000316055.meta', 'create_time': '2023-10-08T16:13:13.796888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'volume_0000000000316055'}}}, {'pool': 32, 'pool_name': 'cephfs.volume_0000000000316055.data', 'create_time': '2023-10-08T16:13:14.874207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '187', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'volume_0000000000316055'}}}] 2023-10-08T16:13:32.468 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:13:32.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.588+0000 7f595b59e700 1 Processor -- start 2023-10-08T16:13:32.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.588+0000 7f595b59e700 1 -- start start 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b4e70 0x7f595c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595b59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f595c0b1b40 0x7f595c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595b59e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c0cfb00 con 0x7f595c0b16d0 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595b59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595c0b5c40 con 0x7f595c0b1b40 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0b5f20 con 0x7f595c0b4e70 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f595c0b1b40 0x7f595c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f5959d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f595c0b16d0 0x7f595c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45716/0 (socket says 172.21.15.170:45716) 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f5959d9b700 1 -- 172.21.15.170:0/1087408106 learned_addr learned my addr 172.21.15.170:0/1087408106 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595a59c700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b4e70 0x7f595c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 -- 172.21.15.170:0/1087408106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b4e70 msgr2=0x7f595c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b4e70 0x7f595c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 -- 172.21.15.170:0/1087408106 >> v1:172.21.15.170:6789/0 conn(0x7f595c0b16d0 legacy=0x7f595c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 -- 172.21.15.170:0/1087408106 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c0b6200 con 0x7f595c0b1b40 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595959a700 1 -- 172.21.15.170:0/1087408106 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615601099 0 0) 0x7f595c0cfb00 con 0x7f595c0b16d0 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595ad9d700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b1b40 0x7f595c15d950 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5954009e60 tx=0x7f5954009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f5545431141efc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.589+0000 7f595959a700 1 -- 172.21.15.170:0/1087408106 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595400e070 con 0x7f595c0b1b40 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595959a700 1 -- 172.21.15.170:0/1087408106 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595400bc30 con 0x7f595c0b1b40 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595959a700 1 -- 172.21.15.170:0/1087408106 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954012430 con 0x7f595c0b1b40 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 -- 172.21.15.170:0/1087408106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b1b40 msgr2=0x7f595c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b1b40 0x7f595c15d950 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5954009e60 tx=0x7f5954009780 comp rx=0 tx=0).stop 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 -- 172.21.15.170:0/1087408106 shutdown_connections 2023-10-08T16:13:32.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b4e70 0x7f595c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 --2- 172.21.15.170:0/1087408106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b1b40 0x7f595c15d950 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 -- 172.21.15.170:0/1087408106 >> 172.21.15.170:0/1087408106 conn(0x7f595c0d10c0 msgr2=0x7f595c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 -- 172.21.15.170:0/1087408106 shutdown_connections 2023-10-08T16:13:32.593 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.590+0000 7f595b59e700 1 -- 172.21.15.170:0/1087408106 wait complete. 2023-10-08T16:13:32.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.591+0000 7f595b59e700 1 Processor -- start 2023-10-08T16:13:32.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.591+0000 7f595b59e700 1 -- start start 2023-10-08T16:13:32.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b16d0 0x7f595c0af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 0x7f595c0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0af5a0 con 0x7f595c0b4e70 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0c9e10 con 0x7f595c0b1b40 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595b59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0ca0f0 con 0x7f595c0b16d0 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46164/0 (socket says 172.21.15.170:46164) 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 -- 172.21.15.170:0/2200751732 learned_addr learned my addr 172.21.15.170:0/2200751732 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595a59c700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b16d0 0x7f595c0af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f5959d9b700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 0x7f595c0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 -- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b16d0 msgr2=0x7f595c0af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b16d0 0x7f595c0af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 -- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 msgr2=0x7f595c0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 0x7f595c0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 -- 172.21.15.170:0/2200751732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5954009a20 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f5959d9b700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 0x7f595c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f595ad9d700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5954013000 tx=0x7f5954019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.592+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954012e70 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.593+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595401d630 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.593+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c0ca3d0 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.593+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595401d9d0 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.593+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c0c7ea0 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.594+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c0b1510 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.594+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5954024070 con 0x7f595c0b4e70 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.594+0000 7f594a7fc700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.594+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f59540874f0 con 0x7f595c0b4e70 2023-10-08T16:13:32.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.595+0000 7f595a59c700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.595+0000 7f595a59c700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 0x7f593c0697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f594c006fd0 tx=0x7f594c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.599+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59540540f0 con 0x7f595c0b4e70 2023-10-08T16:13:32.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.783+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f595c0b5a20 con 0x7f595c0b4e70 2023-10-08T16:13:32.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.783+0000 7f594a7fc700 1 -- 172.21.15.170:0/2200751732 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5954057710 con 0x7f595c0b4e70 2023-10-08T16:13:32.784 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 msgr2=0x7f593c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 0x7f593c0697d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f594c006fd0 tx=0x7f594c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 msgr2=0x7f595c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f5954013000 tx=0x7f5954019000 comp rx=0 tx=0).stop 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 shutdown_connections 2023-10-08T16:13:32.786 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f593c067320 0x7f593c0697d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f595c0b4e70 0x7f595c0ca700 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f595c0b16d0 0x7f595c0af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 --2- 172.21.15.170:0/2200751732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f595c0b1b40 0x7f595c0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.785+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 >> 172.21.15.170:0/2200751732 conn(0x7f595c0d10c0 msgr2=0x7f595c0b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.786+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 shutdown_connections 2023-10-08T16:13:32.787 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.786+0000 7f595b59e700 1 -- 172.21.15.170:0/2200751732 wait complete. 2023-10-08T16:13:32.798 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000316055.meta pg_num 2023-10-08T16:13:32.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.914+0000 7f7a1baf8700 1 Processor -- start 2023-10-08T16:13:32.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a1baf8700 1 -- start start 2023-10-08T16:13:32.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a1baf8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7a14159890 0x7f7a1415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a1baf8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a1415a160 con 0x7f7a14159c60 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a1baf8700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a140d13a0 con 0x7f7a14069560 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a1baf8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a1415e140 con 0x7f7a14159890 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a19093700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f7a14159c60 0x7f7a1415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45732/0 (socket says 172.21.15.170:45732) 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.915+0000 7f7a19093700 1 -- 172.21.15.170:0/3717645235 learned_addr learned my addr 172.21.15.170:0/3717645235 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261988362 0 0) 0x7f7a1415a160 con 0x7f7a14159c60 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a19894700 1 --2- 172.21.15.170:0/3717645235 >> v2:172.21.15.191:3300/0 conn(0x7f7a14159890 0x7f7a1415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a00003570 con 0x7f7a14159c60 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038435553 0 0) 0x7f7a00003570 con 0x7f7a14159c60 2023-10-08T16:13:32.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 >> v2:172.21.15.191:3300/0 conn(0x7f7a14159890 msgr2=0x7f7a1415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 --2- 172.21.15.170:0/3717645235 >> v2:172.21.15.191:3300/0 conn(0x7f7a14159890 0x7f7a1415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 >> v1:172.21.15.191:6789/0 conn(0x7f7a14069560 legacy=0x7f7a140ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1415a3d0 con 0x7f7a14159c60 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f7a04001470 con 0x7f7a14159c60 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 >> v1:172.21.15.170:6789/0 conn(0x7f7a14159c60 legacy=0x7f7a1415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a00003a60 0x7f7a00003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a00004810 0x7f7a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 0x7f7a0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1415e140 con 0x7f7a000051c0 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0000c0d0 con 0x7f7a00004810 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0000c290 con 0x7f7a00003a60 2023-10-08T16:13:32.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a1a095700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 0x7f7a0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a19894700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a00003a60 0x7f7a00003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a19093700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a00004810 0x7f7a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a1a095700 1 -- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a00003a60 msgr2=0x7f7a00003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.916+0000 7f7a1a095700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a00003a60 0x7f7a00003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1a095700 1 -- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a00004810 msgr2=0x7f7a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1a095700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a00004810 0x7f7a00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1a095700 1 -- 172.21.15.170:0/3717645235 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00003570 con 0x7f7a000051c0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1a095700 1 -- 172.21.15.170:0/3717645235 reap_dead start 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1a095700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 0x7f7a0000ba60 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c001fc0 tx=0x7f7a0c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d3abaee00ae819 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c001950 con 0x7f7a000051c0 2023-10-08T16:13:32.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c00c8b0 con 0x7f7a000051c0 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a18892700 1 -- 172.21.15.170:0/3717645235 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00cc90 con 0x7f7a000051c0 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 -- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 msgr2=0x7f7a0000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 0x7f7a0000ba60 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c001fc0 tx=0x7f7a0c001640 comp rx=0 tx=0).stop 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 -- 172.21.15.170:0/3717645235 shutdown_connections 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/3717645235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a000051c0 0x7f7a0000ba60 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 -- 172.21.15.170:0/3717645235 >> 172.21.15.170:0/3717645235 conn(0x7f7a14068e20 msgr2=0x7f7a140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 -- 172.21.15.170:0/3717645235 shutdown_connections 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.917+0000 7f7a1baf8700 1 -- 172.21.15.170:0/3717645235 wait complete. 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.918+0000 7f7a1baf8700 1 Processor -- start 2023-10-08T16:13:32.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 -- start start 2023-10-08T16:13:32.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a000051c0 0x7f7a140ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14149210 0x7f7a140b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140ce820 con 0x7f7a140b2120 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140b25a0 con 0x7f7a14149210 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1baf8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a140cde50 con 0x7f7a000051c0 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1a095700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1a095700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46182/0 (socket says 172.21.15.170:46182) 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a1a095700 1 -- 172.21.15.170:0/574079804 learned_addr learned my addr 172.21.15.170:0/574079804 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:32.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a19093700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14149210 0x7f7a140b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.919+0000 7f7a19894700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a000051c0 0x7f7a140ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 -- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a000051c0 msgr2=0x7f7a140ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a000051c0 0x7f7a140ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 -- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14149210 msgr2=0x7f7a140b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14149210 0x7f7a140b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 -- 172.21.15.170:0/574079804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a140cd240 con 0x7f7a140b2120 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1a095700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c6a0 tx=0x7f7a0c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c00c6d0 con 0x7f7a140b2120 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c012e40 con 0x7f7a140b2120 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c01e660 con 0x7f7a140b2120 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a140cd520 con 0x7f7a140b2120 2023-10-08T16:13:32.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.920+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a140cdae0 con 0x7f7a140b2120 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.921+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a140cdfe0 con 0x7f7a140b2120 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.921+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a0c01ea50 con 0x7f7a140b2120 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.922+0000 7f7a09ffb700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 0x7f79f80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.922+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f7a0c086700 con 0x7f7a140b2120 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.922+0000 7f7a19894700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 0x7f79f80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.922+0000 7f7a19894700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 0x7f79f80697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7a10006f00 tx=0x7f7a100091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:32.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:32.926+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a0c053300 con 0x7f7a140b2120 2023-10-08T16:13:33.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.107+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000316055.meta", "var": "pg_num"} v 0) v1 -- 0x7f7a1404c3c0 con 0x7f7a140b2120 2023-10-08T16:13:33.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.108+0000 7f7a09ffb700 1 -- 172.21.15.170:0/574079804 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000316055.meta", "var": "pg_num"}]=0 v189) v1 ==== 124+0+11 (secure 0 0 0) 0x7f7a0c017020 con 0x7f7a140b2120 2023-10-08T16:13:33.108 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 msgr2=0x7f79f80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 0x7f79f80697d0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7a10006f00 tx=0x7f7a100091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 msgr2=0x7f7a140b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00c6a0 tx=0x7f7a0c015000 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 shutdown_connections 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f79f8067320 0x7f79f80697d0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a140b2120 0x7f7a140b4050 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a000051c0 0x7f7a140ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 --2- 172.21.15.170:0/574079804 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a14149210 0x7f7a140b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 >> 172.21.15.170:0/574079804 conn(0x7f7a14068e20 msgr2=0x7f7a14149c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 shutdown_connections 2023-10-08T16:13:33.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.110+0000 7f7a1baf8700 1 -- 172.21.15.170:0/574079804 wait complete. 2023-10-08T16:13:33.122 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.volume_0000000000316055.data pg_num 2023-10-08T16:13:33.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.239+0000 7fd9bc513700 1 Processor -- start 2023-10-08T16:13:33.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.239+0000 7fd9bc513700 1 -- start start 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9bc513700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b4159890 0x7fd9b415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9bc513700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9bc513700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b4069ee0 con 0x7fd9b4159c60 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9bc513700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b415a160 con 0x7fd9b4159890 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9bc513700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40d13a0 con 0x7fd9b4069560 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9baab0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b4159890 0x7fd9b415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9baab0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:38810/0 (socket says 172.21.15.170:38810) 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fd9b4159890 0x7fd9b415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59774/0 (socket says 172.21.15.170:59774) 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9baab0700 1 -- 172.21.15.170:0/1645779129 learned_addr learned my addr 172.21.15.170:0/1645779129 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9b9aae700 1 --1- 172.21.15.170:0/1645779129 >> v1:172.21.15.191:6789/0 conn(0x7fd9b4159c60 0x7fd9b415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58530/0 (socket says 172.21.15.170:58530) 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 -- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 msgr2=0x7fd9b40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 -- 172.21.15.170:0/1645779129 >> v1:172.21.15.191:6789/0 conn(0x7fd9b4159c60 legacy=0x7fd9b415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9ba2af700 1 -- 172.21.15.170:0/1645779129 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b415e140 con 0x7fd9b4159890 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.240+0000 7fd9baab0700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9ba2af700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4159890 0x7fd9b415abd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098e0 tx=0x7fd9b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc460ddb455a2243 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9b92ad700 1 -- 172.21.15.170:0/1645779129 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000b640 con 0x7fd9b4159890 2023-10-08T16:13:33.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9b92ad700 1 -- 172.21.15.170:0/1645779129 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000b7a0 con 0x7fd9b4159890 2023-10-08T16:13:33.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9b92ad700 1 -- 172.21.15.170:0/1645779129 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000bb40 con 0x7fd9b4159890 2023-10-08T16:13:33.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 -- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4159890 msgr2=0x7fd9b415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.245 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4159890 0x7fd9b415abd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098e0 tx=0x7fd9b00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 -- 172.21.15.170:0/1645779129 shutdown_connections 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4069560 0x7fd9b40699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1645779129 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4159890 0x7fd9b415abd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 -- 172.21.15.170:0/1645779129 >> 172.21.15.170:0/1645779129 conn(0x7fd9b4068e20 msgr2=0x7fd9b40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 -- 172.21.15.170:0/1645779129 shutdown_connections 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.241+0000 7fd9bc513700 1 -- 172.21.15.170:0/1645779129 wait complete. 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.242+0000 7fd9bc513700 1 Processor -- start 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.242+0000 7fd9bc513700 1 -- start start 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4159890 0x7fd9b4149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cd260 con 0x7fd9b4159c60 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cd540 con 0x7fd9b4069560 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9bc513700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40cd820 con 0x7fd9b4159890 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9baab0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9ba2af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9ba2af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59788/0 (socket says 172.21.15.170:59788) 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9baab0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46186/0 (socket says 172.21.15.170:46186) 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9ba2af700 1 -- 172.21.15.170:0/1894943017 learned_addr learned my addr 172.21.15.170:0/1894943017 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:33.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.243+0000 7fd9b9aae700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4159890 0x7fd9b4149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 -- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4159890 msgr2=0x7fd9b4149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4159890 0x7fd9b4149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 -- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 msgr2=0x7fd9b4153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 -- 172.21.15.170:0/1894943017 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac009ae0 con 0x7fd9b4159c60 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9ba2af700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9baab0700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009e70 tx=0x7fd9ac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac018070 con 0x7fd9b4159c60 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac00f040 con 0x7fd9b4159c60 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b0009420 con 0x7fd9b4159c60 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00cb00 con 0x7fd9b4159c60 2023-10-08T16:13:33.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.244+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b40ce020 con 0x7fd9b4159c60 2023-10-08T16:13:33.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.245+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b404c3c0 con 0x7fd9b4159c60 2023-10-08T16:13:33.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.245+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fd9ac003020 con 0x7fd9b4159c60 2023-10-08T16:13:33.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.246+0000 7fd9aa7fc700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 0x7fd9980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.246+0000 7fd9ba2af700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 0x7fd9980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.246+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7fd9ac085100 con 0x7fd9b4159c60 2023-10-08T16:13:33.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.246+0000 7fd9ba2af700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 0x7fd9980697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003080 tx=0x7fd9b0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.250+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9ac051bf0 con 0x7fd9b4159c60 2023-10-08T16:13:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.433+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.volume_0000000000316055.data", "var": "pg_num"} v 0) v1 -- 0x7fd9b406a5b0 con 0x7fd9b4159c60 2023-10-08T16:13:33.433 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.433+0000 7fd9aa7fc700 1 -- 172.21.15.170:0/1894943017 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.volume_0000000000316055.data", "var": "pg_num"}]=0 v189) v1 ==== 124+0+11 (secure 0 0 0) 0x7fd9ac055210 con 0x7fd9b4159c60 2023-10-08T16:13:33.434 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:13:33.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 msgr2=0x7fd9980697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 0x7fd9980697d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0003080 tx=0x7fd9b0017000 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 msgr2=0x7fd9b414a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009e70 tx=0x7fd9ac00b040 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 shutdown_connections 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fd998067320 0x7fd9980697d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9b4159c60 0x7fd9b414a3e0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fd9b4069560 0x7fd9b4153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 --2- 172.21.15.170:0/1894943017 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fd9b4159890 0x7fd9b4149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 >> 172.21.15.170:0/1894943017 conn(0x7fd9b4068e20 msgr2=0x7fd9b415b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 shutdown_connections 2023-10-08T16:13:33.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.435+0000 7fd9bc513700 1 -- 172.21.15.170:0/1894943017 wait complete. 2023-10-08T16:13:33.447 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:33.562 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.562+0000 7f51a4d20700 1 Processor -- start 2023-10-08T16:13:33.563 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.562+0000 7f51a4d20700 1 -- start start 2023-10-08T16:13:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f51a4d20700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f51a0159890 0x7f51a015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f51a4d20700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f51a4d20700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a0069ee0 con 0x7f51a0159c60 2023-10-08T16:13:33.565 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f51a4d20700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a015a160 con 0x7f51a0159890 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f51a4d20700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00d13a0 con 0x7f51a0069560 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f5195d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f51a0159c60 0x7f51a015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45742/0 (socket says 172.21.15.170:45742) 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f5195d9b700 1 -- 172.21.15.170:0/3622458251 learned_addr learned my addr 172.21.15.170:0/3622458251 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f519e59c700 1 --2- 172.21.15.170:0/3622458251 >> v2:172.21.15.191:3300/0 conn(0x7f51a0159890 0x7f51a015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3622458251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905276189 0 0) 0x7f51a0069ee0 con 0x7f51a0159c60 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.563+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c003570 con 0x7f51a0159c60 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642026289 0 0) 0x7f518c003570 con 0x7f51a0159c60 2023-10-08T16:13:33.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519ed9d700 1 -- 172.21.15.170:0/3622458251 >> v2:172.21.15.191:3300/0 conn(0x7f51a0159890 msgr2=0x7f51a015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3622458251 >> v2:172.21.15.191:3300/0 conn(0x7f51a0159890 0x7f51a015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519ed9d700 1 -- 172.21.15.170:0/3622458251 >> v1:172.21.15.170:6789/0 conn(0x7f51a0159c60 legacy=0x7f51a015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519ed9d700 1 -- 172.21.15.170:0/3622458251 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a015e140 con 0x7f51a0069560 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3622458251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a00699a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5198009e60 tx=0x7f5198009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cccd2a2e3067da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800e070 con 0x7f51a0069560 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800b7a0 con 0x7f51a0069560 2023-10-08T16:13:33.567 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f519dd9b700 1 -- 172.21.15.170:0/3622458251 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188001470 con 0x7f51a0069560 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f51a4d20700 1 -- 172.21.15.170:0/3622458251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 msgr2=0x7f51a00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.564+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3622458251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a00699a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5198009e60 tx=0x7f5198009780 comp rx=0 tx=0).stop 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 -- 172.21.15.170:0/3622458251 shutdown_connections 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3622458251 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a00699a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3622458251 >> v2:172.21.15.191:3300/0 conn(0x7f51a0159890 0x7f51a015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 -- 172.21.15.170:0/3622458251 >> 172.21.15.170:0/3622458251 conn(0x7f51a0068e20 msgr2=0x7f51a00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 -- 172.21.15.170:0/3622458251 shutdown_connections 2023-10-08T16:13:33.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.565+0000 7f51a4d20700 1 -- 172.21.15.170:0/3622458251 wait complete. 2023-10-08T16:13:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 Processor -- start 2023-10-08T16:13:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 -- start start 2023-10-08T16:13:33.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a016db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a0159890 0x7f51a00c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a016d100 con 0x7f51a0159c60 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a016d6c0 con 0x7f51a0159890 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.566+0000 7f51a4d20700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a016a5e0 con 0x7f51a0069560 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46196/0 (socket says 172.21.15.170:46196) 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 -- 172.21.15.170:0/3330385753 learned_addr learned my addr 172.21.15.170:0/3330385753 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:33.572 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519e59c700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a016db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f5195d9b700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a0159890 0x7f51a00c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 -- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 msgr2=0x7f51a016db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a016db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 -- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a0159890 msgr2=0x7f51a00c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a0159890 0x7f51a00c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 -- 172.21.15.170:0/3330385753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198009a20 con 0x7f51a0159c60 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519e59c700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a016db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f519ed9d700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f51a0070e20 tx=0x7f5198018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.573 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198012d20 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519801e430 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a016a8c0 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519801e7f0 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.567+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a00c94a0 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.568+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a016d830 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.568+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f519801eb60 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.569+0000 7f5196ffd700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 0x7f51800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.569+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f5198086ff0 con 0x7f51a0159c60 2023-10-08T16:13:33.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.569+0000 7f519e59c700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 0x7f51800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.569+0000 7f519e59c700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 0x7f51800697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51a00b6630 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.573+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5198053ae0 con 0x7f51a0159c60 2023-10-08T16:13:33.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.797+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51a004c3c0 con 0x7f51a0159c60 2023-10-08T16:13:33.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.798+0000 7f5196ffd700 1 -- 172.21.15.170:0/3330385753 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4636 (secure 0 0 0) 0x7f5198057100 con 0x7f51a0159c60 2023-10-08T16:13:33.798 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:33.798 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:27.591305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000316055","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:33.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 msgr2=0x7f51800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 0x7f51800697d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51a00b6630 tx=0x7f5190008040 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 msgr2=0x7f51a0169b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f51a0070e20 tx=0x7f5198018000 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 shutdown_connections 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5180067320 0x7f51800697d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f51a0159c60 0x7f51a0169b20 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f51a0069560 0x7f51a016db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 --2- 172.21.15.170:0/3330385753 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f51a0159890 0x7f51a00c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 >> 172.21.15.170:0/3330385753 conn(0x7f51a0068e20 msgr2=0x7f51a0152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 shutdown_connections 2023-10-08T16:13:33.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.800+0000 7f51a4d20700 1 -- 172.21.15.170:0/3330385753 wait complete. 2023-10-08T16:13:33.802 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2023-10-08T16:13:33.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:33.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.927+0000 7f7a5fcae700 1 Processor -- start 2023-10-08T16:13:33.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.927+0000 7f7a5fcae700 1 -- start start 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5fcae700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7a58159890 0x7f7a5815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5fcae700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a58069560 0x7f7a580699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5fcae700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a58069ee0 con 0x7f7a58159c60 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5fcae700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5815a160 con 0x7f7a58069560 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5fcae700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580d13a0 con 0x7f7a58159890 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a58069560 0x7f7a580699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7a58069560 0x7f7a580699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46212/0 (socket says 172.21.15.170:46212) 2023-10-08T16:13:33.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 -- 172.21.15.170:0/751798983 learned_addr learned my addr 172.21.15.170:0/751798983 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5da4a700 1 --2- 172.21.15.170:0/751798983 >> v2:172.21.15.191:3300/0 conn(0x7f7a58159890 0x7f7a5815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 -- 172.21.15.170:0/751798983 >> v2:172.21.15.191:3300/0 conn(0x7f7a58159890 msgr2=0x7f7a5815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 --2- 172.21.15.170:0/751798983 >> v2:172.21.15.191:3300/0 conn(0x7f7a58159890 0x7f7a5815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 -- 172.21.15.170:0/751798983 >> v1:172.21.15.191:6789/0 conn(0x7f7a58159c60 legacy=0x7f7a5815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 -- 172.21.15.170:0/751798983 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5815e140 con 0x7f7a58069560 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5e24b700 1 --2- 172.21.15.170:0/751798983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58069560 0x7f7a580699a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7a580d1090 tx=0x7f7a50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99707ae26967d5e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5ca48700 1 -- 172.21.15.170:0/751798983 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000e070 con 0x7f7a58069560 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5ca48700 1 -- 172.21.15.170:0/751798983 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000ba20 con 0x7f7a58069560 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.928+0000 7f7a5ca48700 1 -- 172.21.15.170:0/751798983 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500092e0 con 0x7f7a58069560 2023-10-08T16:13:33.931 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 -- 172.21.15.170:0/751798983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58069560 msgr2=0x7f7a580699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/751798983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58069560 0x7f7a580699a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7a580d1090 tx=0x7f7a50009840 comp rx=0 tx=0).stop 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 -- 172.21.15.170:0/751798983 shutdown_connections 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/751798983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58069560 0x7f7a580699a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/751798983 >> v2:172.21.15.191:3300/0 conn(0x7f7a58159890 0x7f7a5815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 -- 172.21.15.170:0/751798983 >> 172.21.15.170:0/751798983 conn(0x7f7a58068e20 msgr2=0x7f7a580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 -- 172.21.15.170:0/751798983 shutdown_connections 2023-10-08T16:13:33.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.929+0000 7f7a5fcae700 1 -- 172.21.15.170:0/751798983 wait complete. 2023-10-08T16:13:33.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.930+0000 7f7a5fcae700 1 Processor -- start 2023-10-08T16:13:33.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.930+0000 7f7a5fcae700 1 -- start start 2023-10-08T16:13:33.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 0x7f7a580b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 0x7f7a580c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580b6270 con 0x7f7a58159890 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5816d270 con 0x7f7a58159c60 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5fcae700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5816d550 con 0x7f7a58069560 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46222/0 (socket says 172.21.15.170:46222) 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 -- 172.21.15.170:0/567800402 learned_addr learned my addr 172.21.15.170:0/567800402 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5e24b700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 0x7f7a580c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5da4a700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 0x7f7a580b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 -- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 msgr2=0x7f7a580b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 0x7f7a580b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 -- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 msgr2=0x7f7a580c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 0x7f7a580c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 -- 172.21.15.170:0/567800402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009ae0 con 0x7f7a58159890 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5da4a700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 0x7f7a580b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5e24b700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 0x7f7a580c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a5d249700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c4e0 tx=0x7f7a4800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48011590 con 0x7f7a58159890 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.931+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48011710 con 0x7f7a58159890 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.932+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5816d830 con 0x7f7a58159890 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.932+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48011ad0 con 0x7f7a58159890 2023-10-08T16:13:33.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.932+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a580b46c0 con 0x7f7a58159890 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.933+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5816dd50 con 0x7f7a58159890 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.933+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7a4801b3b0 con 0x7f7a58159890 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.933+0000 7f7a4dffb700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 0x7f7a3c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.933+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f7a48084240 con 0x7f7a58159890 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.933+0000 7f7a5da4a700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 0x7f7a3c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.934+0000 7f7a5da4a700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 0x7f7a3c0697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d80 tx=0x7f7a54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:33.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:33.937+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a5816dd50 con 0x7f7a58159890 2023-10-08T16:13:34.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.119+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a5816dd50 con 0x7f7a58159890 2023-10-08T16:13:34.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.120+0000 7f7a4dffb700 1 -- 172.21.15.170:0/567800402 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15586 (secure 0 0 0) 0x7f7a5816dd50 con 0x7f7a58159890 2023-10-08T16:13:34.120 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:34.120 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":189,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:29.612723+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000316055.meta","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000316055"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000316055.data","create_time":"2023-10-08T16:13:14.874207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000316055"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.c","mappings":[{"from":2,"to":0}]},{"pgid":"32.f","mappings":[{"from":6,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:34.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 msgr2=0x7f7a3c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 0x7f7a3c0697d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009d80 tx=0x7f7a54008040 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 msgr2=0x7f7a580b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c4e0 tx=0x7f7a4800c8a0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 shutdown_connections 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7a58159c60 0x7f7a580c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7a58069560 0x7f7a580b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7a3c067320 0x7f7a3c0697d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 --2- 172.21.15.170:0/567800402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7a58159890 0x7f7a580b4180 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 >> 172.21.15.170:0/567800402 conn(0x7f7a58068e20 msgr2=0x7f7a5814d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 shutdown_connections 2023-10-08T16:13:34.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.122+0000 7f7a5fcae700 1 -- 172.21.15.170:0/567800402 wait complete. 2023-10-08T16:13:34.135 INFO:tasks.cephfs.filesystem:Destroying file system volume_0000000000316055 and related pools 2023-10-08T16:13:34.135 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.250+0000 7fa89aaa0700 1 Processor -- start 2023-10-08T16:13:34.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.250+0000 7fa89aaa0700 1 -- start start 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89aaa0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89aaa0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa89415db10 0x7fa894069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89aaa0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89415a160 con 0x7fa894159890 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89aaa0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8940d13a0 con 0x7fa89415db10 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89aaa0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89415e140 con 0x7fa894159c60 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa89415db10 0x7fa894069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa893fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.253 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fa89415db10 0x7fa894069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:59812/0 (socket says 172.21.15.170:59812) 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa893fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:38884/0 (socket says 172.21.15.170:38884) 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 -- 172.21.15.170:0/3884462739 learned_addr learned my addr 172.21.15.170:0/3884462739 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89883c700 1 --1- 172.21.15.170:0/3884462739 >> v1:172.21.15.191:6789/0 conn(0x7fa894159890 0x7fa89415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:58552/0 (socket says 172.21.15.170:58552) 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 -- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 msgr2=0x7fa89415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 -- 172.21.15.170:0/3884462739 >> v1:172.21.15.191:6789/0 conn(0x7fa894159890 legacy=0x7fa89415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa89903d700 1 -- 172.21.15.170:0/3884462739 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89415a3d0 con 0x7fa89415db10 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.251+0000 7fa893fff700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:34.254 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89903d700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa894069560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0099b0 tx=0x7fa87c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4439dde4d5475e21 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa8937fe700 1 -- 172.21.15.170:0/3884462739 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c009ea0 con 0x7fa89415db10 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa8937fe700 1 -- 172.21.15.170:0/3884462739 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c00b860 con 0x7fa89415db10 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa8937fe700 1 -- 172.21.15.170:0/3884462739 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00bc00 con 0x7fa89415db10 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 -- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 msgr2=0x7fa894069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa894069560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0099b0 tx=0x7fa87c009250 comp rx=0 tx=0).stop 2023-10-08T16:13:34.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 -- 172.21.15.170:0/3884462739 shutdown_connections 2023-10-08T16:13:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa894069560 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/3884462739 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa89415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 -- 172.21.15.170:0/3884462739 >> 172.21.15.170:0/3884462739 conn(0x7fa894068e20 msgr2=0x7fa8940d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 -- 172.21.15.170:0/3884462739 shutdown_connections 2023-10-08T16:13:34.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.252+0000 7fa89aaa0700 1 -- 172.21.15.170:0/3884462739 wait complete. 2023-10-08T16:13:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.253+0000 7fa89aaa0700 1 Processor -- start 2023-10-08T16:13:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 -- start start 2023-10-08T16:13:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa8940cf750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa89416d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89416abd0 con 0x7fa894159890 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa894169f50 con 0x7fa89415db10 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.254+0000 7fa89aaa0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89416a230 con 0x7fa894159c60 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46228/0 (socket says 172.21.15.170:46228) 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 -- 172.21.15.170:0/359615484 learned_addr learned my addr 172.21.15.170:0/359615484 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89903d700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa89416d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa893fff700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa8940cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 -- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 msgr2=0x7fa8940cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa8940cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 -- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 msgr2=0x7fa89416d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa89416d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 -- 172.21.15.170:0/359615484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884009a20 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89883c700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c7c0 tx=0x7fa88c007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c005a70 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c005bd0 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0094f0 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0070f0 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.255+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89416a7f0 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.256+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89404c3c0 con 0x7fa894159890 2023-10-08T16:13:34.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.257+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa88c007460 con 0x7fa894159890 2023-10-08T16:13:34.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.257+0000 7fa890ff9700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 0x7fa8740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.257+0000 7fa893fff700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 0x7fa8740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.257+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7fa88c0846d0 con 0x7fa894159890 2023-10-08T16:13:34.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.257+0000 7fa893fff700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 0x7fa8740697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa8840099f0 tx=0x7fa88400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.261+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa88c051350 con 0x7fa894159890 2023-10-08T16:13:34.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.488+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8940b0120 con 0x7fa894159890 2023-10-08T16:13:34.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.488+0000 7fa890ff9700 1 -- 172.21.15.170:0/359615484 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4636 (secure 0 0 0) 0x7fa88c054970 con 0x7fa894159890 2023-10-08T16:13:34.489 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:34.489 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:27.591305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000316055","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:34.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 msgr2=0x7fa8740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 0x7fa8740697d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa8840099f0 tx=0x7fa88400b040 comp rx=0 tx=0).stop 2023-10-08T16:13:34.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 msgr2=0x7fa8940cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c7c0 tx=0x7fa88c007f70 comp rx=0 tx=0).stop 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 shutdown_connections 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa89415db10 0x7fa89416d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa874067320 0x7fa8740697d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa894159890 0x7fa8940cf210 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 --2- 172.21.15.170:0/359615484 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa894159c60 0x7fa8940cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 >> 172.21.15.170:0/359615484 conn(0x7fa894068e20 msgr2=0x7fa894152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 shutdown_connections 2023-10-08T16:13:34.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.491+0000 7fa89aaa0700 1 -- 172.21.15.170:0/359615484 wait complete. 2023-10-08T16:13:34.493 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2023-10-08T16:13:34.503 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:34.618 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.618+0000 7f65ef7cd700 1 Processor -- start 2023-10-08T16:13:34.619 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.618+0000 7f65ef7cd700 1 -- start start 2023-10-08T16:13:34.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ef7cd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f65e8069560 0x7f65e80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.621 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ef7cd700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e8069ee0 con 0x7f65e8159890 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ef7cd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e815a160 con 0x7f65e8159c60 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ef7cd700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e80d13a0 con 0x7f65e8069560 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ed569700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f65e8159890 0x7f65e815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45744/0 (socket says 172.21.15.170:45744) 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65ed569700 1 -- 172.21.15.170:0/2491103816 learned_addr learned my addr 172.21.15.170:0/2491103816 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65edd6a700 1 --2- 172.21.15.170:0/2491103816 >> v2:172.21.15.191:3300/0 conn(0x7f65e8069560 0x7f65e80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.619+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154553799 0 0) 0x7f65e8069ee0 con 0x7f65e8159890 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d4003570 con 0x7f65e8159890 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798959581 0 0) 0x7f65e815a160 con 0x7f65e8159c60 2023-10-08T16:13:34.622 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8069ee0 con 0x7f65e8159c60 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65edd6a700 1 -- 172.21.15.170:0/2491103816 >> v1:172.21.15.191:6789/0 conn(0x7f65e8159c60 legacy=0x7f65e815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65edd6a700 1 -- 172.21.15.170:0/2491103816 >> v1:172.21.15.170:6789/0 conn(0x7f65e8159890 legacy=0x7f65e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434392802 0 0) 0x7f65d4003570 con 0x7f65e8159890 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65edd6a700 1 -- 172.21.15.170:0/2491103816 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e815e140 con 0x7f65e8069560 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65edd6a700 1 --2- 172.21.15.170:0/2491103816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8069560 0x7f65e80699a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009e60 tx=0x7f65e0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c61699c8d33d6922 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e000e070 con 0x7f65e8069560 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e000b7a0 con 0x7f65e8069560 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.620+0000 7f65dffff700 1 -- 172.21.15.170:0/2491103816 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4001470 con 0x7f65e8069560 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2491103816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8069560 msgr2=0x7f65e80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2491103816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8069560 0x7f65e80699a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009e60 tx=0x7f65e0009780 comp rx=0 tx=0).stop 2023-10-08T16:13:34.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2491103816 shutdown_connections 2023-10-08T16:13:34.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2491103816 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8069560 0x7f65e80699a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2491103816 >> 172.21.15.170:0/2491103816 conn(0x7f65e8068e20 msgr2=0x7f65e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2491103816 shutdown_connections 2023-10-08T16:13:34.624 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.621+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2491103816 wait complete. 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.622+0000 7f65ef7cd700 1 Processor -- start 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.622+0000 7f65ef7cd700 1 -- start start 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8159890 0x7f65e816d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 0x7f65e80ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e816de00 con 0x7f65e8069560 2023-10-08T16:13:34.627 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80cd910 con 0x7f65e8159890 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80ce1b0 con 0x7f65e8159c60 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46234/0 (socket says 172.21.15.170:46234) 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 -- 172.21.15.170:0/2525176497 learned_addr learned my addr 172.21.15.170:0/2525176497 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65edd6a700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 0x7f65e80ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ecd68700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8159890 0x7f65e816d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 -- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 msgr2=0x7f65e80ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 0x7f65e80ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 -- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8159890 msgr2=0x7f65e816d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.628 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8159890 0x7f65e816d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 -- 172.21.15.170:0/2525176497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e0009a20 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65edd6a700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 0x7f65e80ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ed569700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f65e400cb70 tx=0x7f65e4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4007360 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e40074e0 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e80cfd80 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004830 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.623+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e80cde40 con 0x7f65e8069560 2023-10-08T16:13:34.629 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.624+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e80ce490 con 0x7f65e8069560 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.625+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f65e4004ba0 con 0x7f65e8069560 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.625+0000 7f65dd7fa700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.625+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7f65e40841c0 con 0x7f65e8069560 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.625+0000 7f65ecd68700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.626+0000 7f65ecd68700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 0x7f65cc0697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f65d80037e0 tx=0x7f65d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.629+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65e4050dc0 con 0x7f65e8069560 2023-10-08T16:13:34.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.854+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e804c3c0 con 0x7f65e8069560 2023-10-08T16:13:34.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.855+0000 7f65dd7fa700 1 -- 172.21.15.170:0/2525176497 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+4636 (secure 0 0 0) 0x7f65e40543e0 con 0x7f65e8069560 2023-10-08T16:13:34.855 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:34.855 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":123,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115}],"filesystems":[{"mdsmap":{"epoch":122,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:15.903470+0000","modified":"2023-10-08T16:13:27.591305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8634},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8634":{"gid":8634,"name":"c","rank":0,"incarnation":116,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6836/3034213253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3034213253},{"type":"v1","addr":"172.21.15.170:6836","nonce":3034213253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[32],"metadata_pool":31,"enabled":true,"fs_name":"volume_0000000000316055","balancer":"","standby_count_wanted":0},"id":15}]} 2023-10-08T16:13:34.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 msgr2=0x7f65cc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 0x7f65cc0697d0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f65d80037e0 tx=0x7f65d8009040 comp rx=0 tx=0).stop 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 msgr2=0x7f65e8153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f65e400cb70 tx=0x7f65e4001b50 comp rx=0 tx=0).stop 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 shutdown_connections 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f65e8159c60 0x7f65e80ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f65cc067320 0x7f65cc0697d0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e8069560 0x7f65e8153650 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 --2- 172.21.15.170:0/2525176497 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f65e8159890 0x7f65e816d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 >> 172.21.15.170:0/2525176497 conn(0x7f65e8068e20 msgr2=0x7f65e815d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 shutdown_connections 2023-10-08T16:13:34.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.857+0000 7f65ef7cd700 1 -- 172.21.15.170:0/2525176497 wait complete. 2023-10-08T16:13:34.859 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 123 2023-10-08T16:13:34.869 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:34.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.983+0000 7fcb6bbe2700 1 Processor -- start 2023-10-08T16:13:34.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.983+0000 7fcb6bbe2700 1 -- start start 2023-10-08T16:13:34.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6bbe2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb6415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6bbe2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcb64069560 0x7fcb640699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6bbe2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6415a160 con 0x7fcb64159c60 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6bbe2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb64069f20 con 0x7fcb64069560 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6bbe2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640d13a0 con 0x7fcb64159890 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6917d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcb64159c60 0x7fcb6415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:45746/0 (socket says 172.21.15.170:45746) 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.984+0000 7fcb6917d700 1 -- 172.21.15.170:0/2347745036 learned_addr learned my addr 172.21.15.170:0/2347745036 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590872209 0 0) 0x7fcb6415a160 con 0x7fcb64159c60 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/2347745036 >> v2:172.21.15.191:3300/0 conn(0x7fcb64069560 0x7fcb640699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6997e700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb6415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb50003570 con 0x7fcb64159c60 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394297531 0 0) 0x7fcb50003570 con 0x7fcb64159c60 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 msgr2=0x7fcb6415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb6415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 >> v2:172.21.15.191:3300/0 conn(0x7fcb64069560 msgr2=0x7fcb640699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 --2- 172.21.15.170:0/2347745036 >> v2:172.21.15.191:3300/0 conn(0x7fcb64069560 0x7fcb640699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415e140 con 0x7fcb64159c60 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fcb54001470 con 0x7fcb64159c60 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 >> v1:172.21.15.170:6789/0 conn(0x7fcb64159c60 legacy=0x7fcb6415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb50003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004090 0x7fcb50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb50004d50 0x7fcb500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64069f20 con 0x7fcb50004d50 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640d13a0 con 0x7fcb50004090 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5000bbf0 con 0x7fcb64159890 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb50004d50 0x7fcb500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6997e700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb50003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6917d700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004090 0x7fcb50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 -- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 msgr2=0x7fcb50003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64159890 0x7fcb50003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 -- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004090 msgr2=0x7fcb50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004090 0x7fcb50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.990 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 -- 172.21.15.170:0/2347745036 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb50003570 con 0x7fcb50004d50 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.985+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb50004d50 0x7fcb500051c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00bb90 tx=0x7fcb5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54801d39fc37ef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00d5d0 con 0x7fcb50004d50 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00d730 con 0x7fcb50004d50 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6897c700 1 -- 172.21.15.170:0/2347745036 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00daf0 con 0x7fcb50004d50 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/2347745036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb50004d50 msgr2=0x7fcb500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/2347745036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb50004d50 0x7fcb500051c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00bb90 tx=0x7fcb5c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6a17f700 1 -- 172.21.15.170:0/2347745036 reap_dead start 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/2347745036 shutdown_connections 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/2347745036 >> 172.21.15.170:0/2347745036 conn(0x7fcb64068e20 msgr2=0x7fcb640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/2347745036 shutdown_connections 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.986+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/2347745036 wait complete. 2023-10-08T16:13:34.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.987+0000 7fcb6bbe2700 1 Processor -- start 2023-10-08T16:13:34.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.987+0000 7fcb6bbe2700 1 -- start start 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004d50 0x7fcb64151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64151a40 0x7fcb64150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640ce080 con 0x7fcb640ce4d0 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6415ce90 con 0x7fcb50004d50 2023-10-08T16:13:34.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6bbe2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cfea0 con 0x7fcb64151a40 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46246/0 (socket says 172.21.15.170:46246) 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 -- 172.21.15.170:0/197040038 learned_addr learned my addr 172.21.15.170:0/197040038 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6997e700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004d50 0x7fcb64151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6917d700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64151a40 0x7fcb64150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 -- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64151a40 msgr2=0x7fcb64150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64151a40 0x7fcb64150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 -- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004d50 msgr2=0x7fcb64151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004d50 0x7fcb64151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.988+0000 7fcb6a17f700 1 -- 172.21.15.170:0/197040038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640b3630 con 0x7fcb640ce4d0 2023-10-08T16:13:34.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb6a17f700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000bb40 tx=0x7fcb5c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00a050 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c00de60 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00d5d0 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640b3910 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.989+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640ce940 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.990+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb640ce1f0 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.990+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb5c025410 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.991+0000 7fcb59ffb700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.991+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7fcb5c085f80 con 0x7fcb640ce4d0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.991+0000 7fcb6997e700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:34.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.991+0000 7fcb6997e700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 0x7fcb480697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcb600076e0 tx=0x7fcb60009710 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:34.997 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:34.995+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb640ce1f0 con 0x7fcb640ce4d0 2023-10-08T16:13:35.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.179+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb640ce1f0 con 0x7fcb640ce4d0 2023-10-08T16:13:35.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.179+0000 7fcb59ffb700 1 -- 172.21.15.170:0/197040038 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+15586 (secure 0 0 0) 0x7fcb5c052c90 con 0x7fcb640ce4d0 2023-10-08T16:13:35.180 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:35.180 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":189,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:29.612723+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":31,"pool_name":"cephfs.volume_0000000000316055.meta","create_time":"2023-10-08T16:13:13.796888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"volume_0000000000316055"}}},{"pool":32,"pool_name":"cephfs.volume_0000000000316055.data","create_time":"2023-10-08T16:13:14.874207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"volume_0000000000316055"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"32.c","mappings":[{"from":2,"to":0}]},{"pgid":"32.f","mappings":[{"from":6,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:35.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.181+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 msgr2=0x7fcb480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.182 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.181+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 0x7fcb480697d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcb600076e0 tx=0x7fcb60009710 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.181+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 msgr2=0x7fcb6415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.181+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000bb40 tx=0x7fcb5c01b000 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.181+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 shutdown_connections 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb48067320 0x7fcb480697d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb640ce4d0 0x7fcb6415b4d0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcb50004d50 0x7fcb64151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 --2- 172.21.15.170:0/197040038 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcb64151a40 0x7fcb64150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 >> 172.21.15.170:0/197040038 conn(0x7fcb64068e20 msgr2=0x7fcb64152370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 shutdown_connections 2023-10-08T16:13:35.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.182+0000 7fcb6bbe2700 1 -- 172.21.15.170:0/197040038 wait complete. 2023-10-08T16:13:35.194 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail volume_0000000000316055 2023-10-08T16:13:35.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.309+0000 7efe93fff700 1 Processor -- start 2023-10-08T16:13:35.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.309+0000 7efe93fff700 1 -- start start 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe93fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe941597f0 0x7efe940b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe93fff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940b11d0 con 0x7efe9415d230 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe93fff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940b14b0 con 0x7efe94159420 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe93fff700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe940b1790 con 0x7efe941597f0 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe8a7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efe941597f0 0x7efe940b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe937fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efe9415d230 0x7efe9415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51910/0 (socket says 172.21.15.170:51910) 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe937fe700 1 -- 172.21.15.170:0/2097526979 learned_addr learned my addr 172.21.15.170:0/2097526979 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe8a7fc700 1 -- 172.21.15.170:0/2097526979 >> v1:172.21.15.191:6789/0 conn(0x7efe94159420 legacy=0x7efe940b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe8a7fc700 1 -- 172.21.15.170:0/2097526979 >> v1:172.21.15.170:6789/0 conn(0x7efe9415d230 legacy=0x7efe9415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe8a7fc700 1 -- 172.21.15.170:0/2097526979 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940b1a70 con 0x7efe941597f0 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe927fc700 1 -- 172.21.15.170:0/2097526979 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196093666 0 0) 0x7efe940b11d0 con 0x7efe9415d230 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.310+0000 7efe8a7fc700 1 --2- 172.21.15.170:0/2097526979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941597f0 0x7efe940b2270 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098e0 tx=0x7efe7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97665b212463678 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe927fc700 1 -- 172.21.15.170:0/2097526979 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c009e80 con 0x7efe941597f0 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe927fc700 1 -- 172.21.15.170:0/2097526979 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c00b860 con 0x7efe941597f0 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe927fc700 1 -- 172.21.15.170:0/2097526979 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00bbe0 con 0x7efe941597f0 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 -- 172.21.15.170:0/2097526979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941597f0 msgr2=0x7efe940b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 --2- 172.21.15.170:0/2097526979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941597f0 0x7efe940b2270 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0098e0 tx=0x7efe7c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 -- 172.21.15.170:0/2097526979 shutdown_connections 2023-10-08T16:13:35.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 --2- 172.21.15.170:0/2097526979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941597f0 0x7efe940b2270 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 -- 172.21.15.170:0/2097526979 >> 172.21.15.170:0/2097526979 conn(0x7efe940d0660 msgr2=0x7efe940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:35.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 -- 172.21.15.170:0/2097526979 shutdown_connections 2023-10-08T16:13:35.314 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.311+0000 7efe93fff700 1 -- 172.21.15.170:0/2097526979 wait complete. 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.312+0000 7efe93fff700 1 Processor -- start 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.312+0000 7efe93fff700 1 -- start start 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe941597f0 0x7efe94153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9415d230 0x7efe9416aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9415b2c0 con 0x7efe94159420 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe94154050 con 0x7efe941597f0 2023-10-08T16:13:35.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe93fff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9416b920 con 0x7efe9415d230 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe92ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe92ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49874/0 (socket says 172.21.15.170:49874) 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe92ffd700 1 -- 172.21.15.170:0/1005905290 learned_addr learned my addr 172.21.15.170:0/1005905290 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe937fe700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9415d230 0x7efe9416aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe8a7fc700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe941597f0 0x7efe94153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.313+0000 7efe92ffd700 1 -- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9415d230 msgr2=0x7efe9416aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe92ffd700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9415d230 0x7efe9416aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe92ffd700 1 -- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe941597f0 msgr2=0x7efe94153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe92ffd700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe941597f0 0x7efe94153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe92ffd700 1 -- 172.21.15.170:0/1005905290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c009420 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe92ffd700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c590 tx=0x7efe8400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84011590 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe840116f0 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9415a990 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84011ab0 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.314+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9415af50 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.315+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9404c3c0 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.315+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efe8401b3b0 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.316+0000 7efe8b7fe700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 0x7efe740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.316+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5841+0+0 (secure 0 0 0) 0x7efe84084340 con 0x7efe94159420 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.316+0000 7efe8a7fc700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 0x7efe740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.319 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.316+0000 7efe8a7fc700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 0x7efe740697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efe94159c70 tx=0x7efe7c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.320 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.320+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe84050f40 con 0x7efe94159420 2023-10-08T16:13:35.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.546+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "volume_0000000000316055"} v 0) v1 -- 0x7efe940b1d50 con 0x7efe94159420 2023-10-08T16:13:35.547 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:13:35.547+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:13:35.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.551+0000 7efe8b7fe700 1 -- 172.21.15.170:0/1005905290 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "volume_0000000000316055"}]=0 volume_0000000000316055 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v124) v1 ==== 195+0+0 (secure 0 0 0) 0x7efe84054560 con 0x7efe94159420 2023-10-08T16:13:35.552 INFO:tasks.ceph.mds.c.smithi170.stderr: -791> 2023-10-08T16:13:21.374+0000 7f01e4504700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 msgr2=0x7efe740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 0x7efe740697d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efe94159c70 tx=0x7efe7c017000 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 msgr2=0x7efe941547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7efe8400c590 tx=0x7efe8400c950 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 shutdown_connections 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efe9415d230 0x7efe9416aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efe74067320 0x7efe740697d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe94159420 0x7efe941547b0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 --2- 172.21.15.170:0/1005905290 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efe941597f0 0x7efe94153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 >> 172.21.15.170:0/1005905290 conn(0x7efe940d0660 msgr2=0x7efe940b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:35.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 shutdown_connections 2023-10-08T16:13:35.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.554+0000 7efe93fff700 1 -- 172.21.15.170:0/1005905290 wait complete. 2023-10-08T16:13:35.556 INFO:teuthology.orchestra.run.smithi170.stderr:volume_0000000000316055 marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:13:35.568 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm volume_0000000000316055 --yes-i-really-mean-it 2023-10-08T16:13:35.572 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:13:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.687+0000 7f205ec5d700 1 Processor -- start 2023-10-08T16:13:35.687 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.687+0000 7f205ec5d700 1 -- start start 2023-10-08T16:13:35.689 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205ec5d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f20580b4e70 0x7f20580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205ec5d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f20580b16d0 0x7f20580b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205ec5d700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20580b2080 con 0x7f205815d240 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205ec5d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20580cfb00 con 0x7f20580b4e70 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205ec5d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20580b5c40 con 0x7f20580b16d0 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f20580b4e70 0x7f20580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.690 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f20580b4e70 0x7f20580ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49896/0 (socket says 172.21.15.170:49896) 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 -- 172.21.15.170:0/1141691815 learned_addr learned my addr 172.21.15.170:0/1141691815 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205d45a700 1 --2- 172.21.15.170:0/1141691815 >> v2:172.21.15.191:3300/0 conn(0x7f20580b16d0 0x7f20580b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 -- 172.21.15.170:0/1141691815 >> v2:172.21.15.191:3300/0 conn(0x7f20580b16d0 msgr2=0x7f20580b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 --2- 172.21.15.170:0/1141691815 >> v2:172.21.15.191:3300/0 conn(0x7f20580b16d0 0x7f20580b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 -- 172.21.15.170:0/1141691815 >> v1:172.21.15.191:6789/0 conn(0x7f205815d240 legacy=0x7f205815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 -- 172.21.15.170:0/1141691815 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580b5f20 con 0x7f20580b4e70 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205dc5b700 1 --2- 172.21.15.170:0/1141691815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b4e70 0x7f20580ccff0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2058089830 tx=0x7f2054009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af173fcd8265b6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205cc59700 1 -- 172.21.15.170:0/1141691815 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400e070 con 0x7f20580b4e70 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205cc59700 1 -- 172.21.15.170:0/1141691815 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205400ba20 con 0x7f20580b4e70 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.688+0000 7f205cc59700 1 -- 172.21.15.170:0/1141691815 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20540092e0 con 0x7f20580b4e70 2023-10-08T16:13:35.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 -- 172.21.15.170:0/1141691815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b4e70 msgr2=0x7f20580ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 --2- 172.21.15.170:0/1141691815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b4e70 0x7f20580ccff0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2058089830 tx=0x7f2054009840 comp rx=0 tx=0).stop 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 -- 172.21.15.170:0/1141691815 shutdown_connections 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 --2- 172.21.15.170:0/1141691815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b4e70 0x7f20580ccff0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 --2- 172.21.15.170:0/1141691815 >> v2:172.21.15.191:3300/0 conn(0x7f20580b16d0 0x7f20580b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 -- 172.21.15.170:0/1141691815 >> 172.21.15.170:0/1141691815 conn(0x7f20580d10c0 msgr2=0x7f20580affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 -- 172.21.15.170:0/1141691815 shutdown_connections 2023-10-08T16:13:35.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.689+0000 7f205ec5d700 1 -- 172.21.15.170:0/1141691815 wait complete. 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.690+0000 7f205ec5d700 1 Processor -- start 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.690+0000 7f205ec5d700 1 -- start start 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 0x7f205815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205815d240 0x7f20580c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20580c5ea0 con 0x7f20580b16d0 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058157740 con 0x7f20580b4e70 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205ec5d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205815b730 con 0x7f205815d240 2023-10-08T16:13:35.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49902/0 (socket says 172.21.15.170:49902) 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 -- 172.21.15.170:0/353162390 learned_addr learned my addr 172.21.15.170:0/353162390 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205e45c700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205815d240 0x7f20580c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205d45a700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 0x7f205815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 -- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205815d240 msgr2=0x7f20580c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205815d240 0x7f20580c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 -- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 msgr2=0x7f205815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 0x7f205815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 -- 172.21.15.170:0/353162390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2054009ae0 con 0x7f20580b16d0 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205d45a700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 0x7f205815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:35.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.691+0000 7f205dc5b700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2058070dd0 tx=0x7f2054018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.692+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054008a80 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.692+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2054008c00 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.692+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205815ba10 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.692+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2054026460 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.692+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205815bfd0 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.693+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058088990 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.693+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2054003040 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.694+0000 7f204dffb700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 0x7f203c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.694+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5927+0+0 (secure 0 0 0) 0x7f2054086c90 con 0x7f20580b16d0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.694+0000 7f205d45a700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 0x7f203c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:35.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.694+0000 7f205d45a700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 0x7f203c0697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f20580c5580 tx=0x7f2048008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:35.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.698+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2054053730 con 0x7f20580b16d0 2023-10-08T16:13:35.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:35.925+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "volume_0000000000316055", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f205804bb80 con 0x7f20580b16d0 2023-10-08T16:13:36.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.552+0000 7f204dffb700 1 -- 172.21.15.170:0/353162390 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "volume_0000000000316055", "yes_i_really_mean_it": true}]=0 v125) v1 ==== 121+0+0 (secure 0 0 0) 0x7f2054056d50 con 0x7f20580b16d0 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 msgr2=0x7f203c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 0x7f203c0697d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f20580c5580 tx=0x7f2048008040 comp rx=0 tx=0).stop 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 msgr2=0x7f2058157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f2058070dd0 tx=0x7f2054018000 comp rx=0 tx=0).stop 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 shutdown_connections 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f205815d240 0x7f20580c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f203c067320 0x7f203c0697d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20580b16d0 0x7f2058157200 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 --2- 172.21.15.170:0/353162390 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f20580b4e70 0x7f205815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 >> 172.21.15.170:0/353162390 conn(0x7f20580d10c0 msgr2=0x7f2058158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:36.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 shutdown_connections 2023-10-08T16:13:36.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.554+0000 7f205ec5d700 1 -- 172.21.15.170:0/353162390 wait complete. 2023-10-08T16:13:36.567 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000316055.meta cephfs.volume_0000000000316055.meta --yes-i-really-really-mean-it 2023-10-08T16:13:36.727 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.727+0000 7f05b8ea2700 1 Processor -- start 2023-10-08T16:13:36.728 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.727+0000 7f05b8ea2700 1 -- start start 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b8ea2700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f05b40cca20 0x7f05b40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b8ea2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b40cfa00 con 0x7f05b40b0790 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b8ea2700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b40cb580 con 0x7f05b40b03c0 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b8ea2700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b40cf160 con 0x7f05b40cca20 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b2ffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f05b40b0790 0x7f05b41591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51916/0 (socket says 172.21.15.170:51916) 2023-10-08T16:13:36.730 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b2ffd700 1 -- 172.21.15.170:0/3575977923 learned_addr learned my addr 172.21.15.170:0/3575977923 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b3fff700 1 --2- 172.21.15.170:0/3575977923 >> v2:172.21.15.191:3300/0 conn(0x7f05b40cca20 0x7f05b40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131298052 0 0) 0x7f05b40cfa00 con 0x7f05b40b0790 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f059c003570 con 0x7f05b40b0790 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1372371798 0 0) 0x7f05b40cb580 con 0x7f05b40b03c0 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05b40cfa00 con 0x7f05b40b03c0 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b3fff700 1 -- 172.21.15.170:0/3575977923 >> v1:172.21.15.191:6789/0 conn(0x7f05b40b03c0 legacy=0x7f05b40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487025381 0 0) 0x7f059c003570 con 0x7f05b40b0790 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b3fff700 1 -- 172.21.15.170:0/3575977923 >> v1:172.21.15.170:6789/0 conn(0x7f05b40b0790 legacy=0x7f05b41591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.728+0000 7f05b3fff700 1 -- 172.21.15.170:0/3575977923 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b40cf440 con 0x7f05b40cca20 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b3fff700 1 --2- 172.21.15.170:0/3575977923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40cca20 0x7f05b40cea70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009910 tx=0x7f05ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a529985406fbbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:36.731 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8001470 con 0x7f05b40cca20 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac011420 con 0x7f05b40cca20 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b27fc700 1 -- 172.21.15.170:0/3575977923 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0117a0 con 0x7f05b40cca20 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 -- 172.21.15.170:0/3575977923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40cca20 msgr2=0x7f05b40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/3575977923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40cca20 0x7f05b40cea70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009910 tx=0x7f05ac0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 -- 172.21.15.170:0/3575977923 shutdown_connections 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/3575977923 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40cca20 0x7f05b40cea70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 -- 172.21.15.170:0/3575977923 >> 172.21.15.170:0/3575977923 conn(0x7f05b4063230 msgr2=0x7f05b4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 -- 172.21.15.170:0/3575977923 shutdown_connections 2023-10-08T16:13:36.732 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.729+0000 7f05b8ea2700 1 -- 172.21.15.170:0/3575977923 wait complete. 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 Processor -- start 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 -- start start 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40b0790 0x7f05b4153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b40cca20 0x7f05b415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4155310 con 0x7f05b40b03c0 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4151ff0 con 0x7f05b40b0790 2023-10-08T16:13:36.735 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b8ea2700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4154700 con 0x7f05b40cca20 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49914/0 (socket says 172.21.15.170:49914) 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b37fe700 1 -- 172.21.15.170:0/479269330 learned_addr learned my addr 172.21.15.170:0/479269330 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b3fff700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b40cca20 0x7f05b415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.731+0000 7f05b2ffd700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40b0790 0x7f05b4153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 -- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b40cca20 msgr2=0x7f05b415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b40cca20 0x7f05b415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 -- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40b0790 msgr2=0x7f05b4153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40b0790 0x7f05b4153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 -- 172.21.15.170:0/479269330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac009420 con 0x7f05b40b03c0 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b37fe700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c900 tx=0x7f05a4002780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:36.736 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012630 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4012790 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012b50 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b41549e0 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.732+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b4154fa0 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.733+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b40cdb90 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.734+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f05a401b400 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.734+0000 7f059b7fe700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 0x7f05a00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.734+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5927+0+0 (secure 0 0 0) 0x7f05a4085010 con 0x7f05b40b03c0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.734+0000 7f05b2ffd700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 0x7f05a00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:36.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.734+0000 7f05b2ffd700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 0x7f05a00697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f05b416d2a0 tx=0x7f05a8000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:36.738 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.738+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a4051bc0 con 0x7f05b40b03c0 2023-10-08T16:13:36.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:36.922+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000316055.meta", "pool2": "cephfs.volume_0000000000316055.meta", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f05b404c3c0 con 0x7f05b40b03c0 2023-10-08T16:13:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.646+0000 7f059b7fe700 1 -- 172.21.15.170:0/479269330 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000316055.meta", "pool2": "cephfs.volume_0000000000316055.meta", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000316055.meta' removed v191) v1 ==== 241+0+0 (secure 0 0 0) 0x7f05a40551e0 con 0x7f05b40b03c0 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 msgr2=0x7f05a00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 0x7f05a00697d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f05b416d2a0 tx=0x7f05a8000bc0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 msgr2=0x7f05b4151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c900 tx=0x7f05a4002780 comp rx=0 tx=0).stop 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 shutdown_connections 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f05b40cca20 0x7f05b415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f05a0067320 0x7f05a00697d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05b40b03c0 0x7f05b4151ab0 secure :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c900 tx=0x7f05a4002780 comp rx=0 tx=0).stop 2023-10-08T16:13:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 --2- 172.21.15.170:0/479269330 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f05b40b0790 0x7f05b4153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.651+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 >> 172.21.15.170:0/479269330 conn(0x7f05b4063230 msgr2=0x7f05b415c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.654+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 shutdown_connections 2023-10-08T16:13:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.654+0000 7f05b8ea2700 1 -- 172.21.15.170:0/479269330 wait complete. 2023-10-08T16:13:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs.volume_0000000000316055.meta' removed 2023-10-08T16:13:37.669 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs.volume_0000000000316055.data cephfs.volume_0000000000316055.data --yes-i-really-really-mean-it 2023-10-08T16:13:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.791+0000 7feb25e9e700 1 Processor -- start 2023-10-08T16:13:37.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb25e9e700 1 -- start start 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb25e9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb200699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb25e9e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb20068900 con 0x7feb200af8c0 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb25e9e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb20068be0 con 0x7feb20068390 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb25e9e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb20068ec0 con 0x7feb2006ab40 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.792+0000 7feb1ffff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7feb200af8c0 0x7feb20159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51922/0 (socket says 172.21.15.170:51922) 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1ffff700 1 -- 172.21.15.170:0/851102781 learned_addr learned my addr 172.21.15.170:0/851102781 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb200699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397154995 0 0) 0x7feb20068900 con 0x7feb200af8c0 2023-10-08T16:13:37.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb08003570 con 0x7feb200af8c0 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1448810435 0 0) 0x7feb08003570 con 0x7feb200af8c0 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 msgr2=0x7feb200699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb200699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 >> v1:172.21.15.191:6789/0 conn(0x7feb20068390 legacy=0x7feb200c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200691a0 con 0x7feb200af8c0 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7feb180017f0 con 0x7feb200af8c0 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 >> v1:172.21.15.170:6789/0 conn(0x7feb200af8c0 legacy=0x7feb20159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08003a90 0x7feb08003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb08004d50 0x7feb080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb20068ec0 con 0x7feb08003a90 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0800bb70 con 0x7feb08004d50 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0800bd30 con 0x7feb2006ab40 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08003a90 0x7feb08003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1ffff700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb08004d50 0x7feb080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1effd700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 -- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 msgr2=0x7feb08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb2006ab40 0x7feb08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 -- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb08004d50 msgr2=0x7feb080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb08004d50 0x7feb080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.793+0000 7feb1f7fe700 1 -- 172.21.15.170:0/851102781 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb08003570 con 0x7feb08003a90 2023-10-08T16:13:37.798 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08003a90 0x7feb08003ea0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7feb10009c00 tx=0x7feb100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eed63d6cab738b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000e710 con 0x7feb08003a90 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1000e870 con 0x7feb08003a90 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb1e7fc700 1 -- 172.21.15.170:0/851102781 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000ec30 con 0x7feb08003a90 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 -- 172.21.15.170:0/851102781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08003a90 msgr2=0x7feb08003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 --2- 172.21.15.170:0/851102781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb08003a90 0x7feb08003ea0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7feb10009c00 tx=0x7feb100091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb1ffff700 1 -- 172.21.15.170:0/851102781 reap_dead start 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 -- 172.21.15.170:0/851102781 shutdown_connections 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 -- 172.21.15.170:0/851102781 >> 172.21.15.170:0/851102781 conn(0x7feb200cd640 msgr2=0x7feb200cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 -- 172.21.15.170:0/851102781 shutdown_connections 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.794+0000 7feb25e9e700 1 -- 172.21.15.170:0/851102781 wait complete. 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.795+0000 7feb25e9e700 1 Processor -- start 2023-10-08T16:13:37.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.795+0000 7feb25e9e700 1 -- start start 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 0x7feb20158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb200c6d50 0x7feb200c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c7700 con 0x7feb2015b2e0 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c5340 con 0x7feb200c6d50 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb25e9e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c5620 con 0x7feb08003a90 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb1effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb1effd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49936/0 (socket says 172.21.15.170:49936) 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb1effd700 1 -- 172.21.15.170:0/3322980033 learned_addr learned my addr 172.21.15.170:0/3322980033 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 0x7feb20158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.796+0000 7feb1ffff700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb200c6d50 0x7feb200c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 -- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 msgr2=0x7feb20158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 0x7feb20158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 -- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb200c6d50 msgr2=0x7feb200c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb200c6d50 0x7feb200c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 -- 172.21.15.170:0/3322980033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200c5900 con 0x7feb2015b2e0 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 0x7feb20158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb1effd700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e5f0 tx=0x7feb14009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:37.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb140115c0 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb14011740 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200c5be0 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14011b00 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.797+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200c5000 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.798+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200c5f50 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.798+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7feb14014480 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.799+0000 7feb077fe700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 0x7feb0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.799+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 0x7feb0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.799+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5452+0+0 (secure 0 0 0) 0x7feb14084810 con 0x7feb2015b2e0 2023-10-08T16:13:37.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.799+0000 7feb1f7fe700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 0x7feb0c0697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feb100091e0 tx=0x7feb1001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:37.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.803+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb1404d5a0 con 0x7feb2015b2e0 2023-10-08T16:13:37.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:37.984+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000316055.data", "pool2": "cephfs.volume_0000000000316055.data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feb200718b0 con 0x7feb2015b2e0 2023-10-08T16:13:38.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.661+0000 7feb077fe700 1 -- 172.21.15.170:0/3322980033 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs.volume_0000000000316055.data", "pool2": "cephfs.volume_0000000000316055.data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs.volume_0000000000316055.data' removed v192) v1 ==== 241+0+0 (secure 0 0 0) 0x7feb14054e50 con 0x7feb2015b2e0 2023-10-08T16:13:38.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 msgr2=0x7feb0c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 0x7feb0c0697d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7feb100091e0 tx=0x7feb1001a000 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 msgr2=0x7feb2015b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7feb1400e5f0 tx=0x7feb14009040 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 shutdown_connections 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7feb200c6d50 0x7feb200c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7feb08003a90 0x7feb20158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7feb0c067320 0x7feb0c0697d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 --2- 172.21.15.170:0/3322980033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb2015b2e0 0x7feb2015b750 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.669+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 >> 172.21.15.170:0/3322980033 conn(0x7feb200cd640 msgr2=0x7feb200698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:38.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.670+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 shutdown_connections 2023-10-08T16:13:38.671 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.670+0000 7feb25e9e700 1 -- 172.21.15.170:0/3322980033 wait complete. 2023-10-08T16:13:38.675 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs.volume_0000000000316055.data' removed 2023-10-08T16:13:38.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:13:38.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.806+0000 7f76a2d6e700 1 Processor -- start 2023-10-08T16:13:38.806 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.806+0000 7f76a2d6e700 1 -- start start 2023-10-08T16:13:38.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a2d6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f769c159890 0x7f769c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:38.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a2d6e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c15a160 con 0x7f769c159c60 2023-10-08T16:13:38.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a2d6e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f769c0d13a0 con 0x7f769c069560 2023-10-08T16:13:38.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a2d6e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c15e140 con 0x7f769c159890 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a0b0a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f769c159890 0x7f769c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f7693fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f769c159c60 0x7f769c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51930/0 (socket says 172.21.15.170:51930) 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f7693fff700 1 -- 172.21.15.170:0/589750103 learned_addr learned my addr 172.21.15.170:0/589750103 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a0b0a700 1 -- 172.21.15.170:0/589750103 >> v1:172.21.15.191:6789/0 conn(0x7f769c069560 legacy=0x7f769c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a0b0a700 1 -- 172.21.15.170:0/589750103 >> v1:172.21.15.170:6789/0 conn(0x7f769c159c60 legacy=0x7f769c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a0b0a700 1 -- 172.21.15.170:0/589750103 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c15a3d0 con 0x7f769c159890 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76937fe700 1 -- 172.21.15.170:0/589750103 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 642028582 0 0) 0x7f769c15a160 con 0x7f769c159c60 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76a0b0a700 1 --2- 172.21.15.170:0/589750103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c159890 0x7f769c15abd0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e60 tx=0x7f7698009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b708e1bbf4173d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76937fe700 1 -- 172.21.15.170:0/589750103 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800e070 con 0x7f769c159890 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76937fe700 1 -- 172.21.15.170:0/589750103 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800bc30 con 0x7f769c159890 2023-10-08T16:13:38.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.807+0000 7f76937fe700 1 -- 172.21.15.170:0/589750103 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698012430 con 0x7f769c159890 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 -- 172.21.15.170:0/589750103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c159890 msgr2=0x7f769c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/589750103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c159890 0x7f769c15abd0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f7698009e60 tx=0x7f7698009780 comp rx=0 tx=0).stop 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 -- 172.21.15.170:0/589750103 shutdown_connections 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/589750103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c159890 0x7f769c15abd0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 -- 172.21.15.170:0/589750103 >> 172.21.15.170:0/589750103 conn(0x7f769c068e20 msgr2=0x7f769c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 -- 172.21.15.170:0/589750103 shutdown_connections 2023-10-08T16:13:38.810 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.808+0000 7f76a2d6e700 1 -- 172.21.15.170:0/589750103 wait complete. 2023-10-08T16:13:38.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.809+0000 7f76a2d6e700 1 Processor -- start 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.809+0000 7f76a2d6e700 1 -- start start 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 0x7f769c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f769c159c60 0x7f769c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15de90 con 0x7f769c069560 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c152730 con 0x7f769c159890 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a2d6e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c151b20 con 0x7f769c159c60 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49952/0 (socket says 172.21.15.170:49952) 2023-10-08T16:13:38.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 -- 172.21.15.170:0/2749231101 learned_addr learned my addr 172.21.15.170:0/2749231101 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a130b700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f769c159c60 0x7f769c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f7693fff700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 0x7f769c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 -- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f769c159c60 msgr2=0x7f769c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f769c159c60 0x7f769c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 -- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 msgr2=0x7f769c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 0x7f769c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 -- 172.21.15.170:0/2749231101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7698009a20 con 0x7f769c069560 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f7693fff700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 0x7f769c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.810+0000 7f76a0b0a700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f76980099f0 tx=0x7f7698019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.811+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769801b420 con 0x7f769c069560 2023-10-08T16:13:38.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.811+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769801b580 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.811+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769801b940 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.811+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c151e00 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.811+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c1523c0 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.812+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c152a10 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.812+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f769801bcf0 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.812+0000 7f7690ff9700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 0x7f76880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.813+0000 7f7693fff700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 0x7f76880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.813+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7698086560 con 0x7f769c069560 2023-10-08T16:13:38.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.813+0000 7f7693fff700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 0x7f76880697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f769c0cc720 tx=0x7f768c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:38.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:38.817+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76980533a0 con 0x7f769c069560 2023-10-08T16:13:39.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.001+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f769c04c3c0 con 0x7f769c069560 2023-10-08T16:13:39.004 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.004+0000 7f7690ff9700 1 -- 172.21.15.170:0/2749231101 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f769802a0a0 con 0x7f769c069560 2023-10-08T16:13:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 msgr2=0x7f76880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 0x7f76880697d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f769c0cc720 tx=0x7f768c009040 comp rx=0 tx=0).stop 2023-10-08T16:13:39.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 msgr2=0x7f769c151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f76980099f0 tx=0x7f7698019000 comp rx=0 tx=0).stop 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 shutdown_connections 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f769c159c60 0x7f769c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7688067320 0x7f76880697d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f769c069560 0x7f769c151540 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.006+0000 7f76a2d6e700 1 --2- 172.21.15.170:0/2749231101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f769c159890 0x7f769c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.007+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 >> 172.21.15.170:0/2749231101 conn(0x7f769c068e20 msgr2=0x7f769c0c7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.007+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 shutdown_connections 2023-10-08T16:13:39.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.007+0000 7f76a2d6e700 1 -- 172.21.15.170:0/2749231101 wait complete. 2023-10-08T16:13:39.019 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency' 2023-10-08T16:13:39.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.136+0000 7f7266bab700 1 Processor -- start 2023-10-08T16:13:39.137 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.136+0000 7f7266bab700 1 -- start start 2023-10-08T16:13:39.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7266bab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f72600aca80 0x7f726006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7266bab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7260069d60 0x7f72600c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.139 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7266bab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726006bdb0 con 0x7f72600c81f0 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7266bab700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72600c8670 con 0x7f72600aca80 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7266bab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72600cb370 con 0x7f7260069d60 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7264947700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f72600aca80 0x7f726006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7264947700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f72600aca80 0x7f726006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49968/0 (socket says 172.21.15.170:49968) 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.137+0000 7f7264947700 1 -- 172.21.15.170:0/4117697953 learned_addr learned my addr 172.21.15.170:0/4117697953 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7264947700 1 -- 172.21.15.170:0/4117697953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7260069d60 msgr2=0x7f72600c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7264947700 1 --2- 172.21.15.170:0/4117697953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7260069d60 0x7f72600c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7264947700 1 -- 172.21.15.170:0/4117697953 >> v1:172.21.15.191:6789/0 conn(0x7f72600c81f0 legacy=0x7f7260068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7264947700 1 -- 172.21.15.170:0/4117697953 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7260159f60 con 0x7f72600aca80 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7264947700 1 --2- 172.21.15.170:0/4117697953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72600aca80 0x7f726006aba0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7254009990 tx=0x7f7254009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddbf8ac71d7f0b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.140 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f725f7fe700 1 -- 172.21.15.170:0/4117697953 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254005c10 con 0x7f72600aca80 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f725f7fe700 1 -- 172.21.15.170:0/4117697953 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254005d70 con 0x7f72600aca80 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f725f7fe700 1 -- 172.21.15.170:0/4117697953 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725400b840 con 0x7f72600aca80 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 -- 172.21.15.170:0/4117697953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72600aca80 msgr2=0x7f726006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 --2- 172.21.15.170:0/4117697953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72600aca80 0x7f726006aba0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7254009990 tx=0x7f7254009220 comp rx=0 tx=0).stop 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 -- 172.21.15.170:0/4117697953 shutdown_connections 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 --2- 172.21.15.170:0/4117697953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72600aca80 0x7f726006aba0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 --2- 172.21.15.170:0/4117697953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7260069d60 0x7f72600c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.138+0000 7f7266bab700 1 -- 172.21.15.170:0/4117697953 >> 172.21.15.170:0/4117697953 conn(0x7f72600cd670 msgr2=0x7f72600cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.139+0000 7f7266bab700 1 -- 172.21.15.170:0/4117697953 shutdown_connections 2023-10-08T16:13:39.141 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.139+0000 7f7266bab700 1 -- 172.21.15.170:0/4117697953 wait complete. 2023-10-08T16:13:39.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.140+0000 7f7266bab700 1 Processor -- start 2023-10-08T16:13:39.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.140+0000 7f7266bab700 1 -- start start 2023-10-08T16:13:39.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.140+0000 7f7266bab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.140+0000 7f7266bab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 0x7f72600acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 0x7f726016a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600adfc0 con 0x7f7260069d60 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72600ad720 con 0x7f72600c81f0 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72600ada00 con 0x7f72600aca80 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49980/0 (socket says 172.21.15.170:49980) 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 -- 172.21.15.170:0/1126027401 learned_addr learned my addr 172.21.15.170:0/1126027401 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f725ffff700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 0x7f72600acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7265148700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 0x7f726016a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 -- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 msgr2=0x7f72600acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 0x7f72600acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.145 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 -- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 msgr2=0x7f726016a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 0x7f726016a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 -- 172.21.15.170:0/1126027401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254009480 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f725ffff700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 0x7f72600acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7265148700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 0x7f726016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7264947700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7254009260 tx=0x7f7254015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254000d10 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254011c80 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725401d430 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72600adce0 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.141+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72600ad430 con 0x7f7260069d60 2023-10-08T16:13:39.146 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.142+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72600ae990 con 0x7f7260069d60 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.143+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f725401d820 con 0x7f7260069d60 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.143+0000 7f725cff9700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 0x7f72440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.143+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f7254086570 con 0x7f7260069d60 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.143+0000 7f725ffff700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 0x7f72440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.143+0000 7f725ffff700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 0x7f72440697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f72600ae850 tx=0x7f724c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.147+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72600ae990 con 0x7f7260069d60 2023-10-08T16:13:39.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.403+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]} v 0) v1 -- 0x7f72600aed70 con 0x7f7260069d60 2023-10-08T16:13:39.662 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.661+0000 7f725cff9700 1 -- 172.21.15.170:0/1126027401 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rename_idempotency"]}]=0 v464) v1 ==== 147+0+0 (secure 0 0 0) 0x7f72600aed70 con 0x7f7260069d60 2023-10-08T16:13:39.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 msgr2=0x7f72440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 0x7f72440697d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f72600ae850 tx=0x7f724c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 msgr2=0x7f72600b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7254009260 tx=0x7f7254015000 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 shutdown_connections 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f72600c81f0 0x7f726016a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7244067320 0x7f72440697d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7260069d60 0x7f72600b0410 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 --2- 172.21.15.170:0/1126027401 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f72600aca80 0x7f72600acef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 >> 172.21.15.170:0/1126027401 conn(0x7f72600cd670 msgr2=0x7f726006a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 shutdown_connections 2023-10-08T16:13:39.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.664+0000 7f7266bab700 1 -- 172.21.15.170:0/1126027401 wait complete. 2023-10-08T16:13:39.676 INFO:tasks.cephfs_test_runner:That volume rename is idempotent. ... ok 2023-10-08T16:13:39.676 INFO:tasks.cephfs_test_runner:Starting test: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-08T16:13:39.676 INFO:tasks.cephfs_test_runner:test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:13:39.677 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-08T16:13:39.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.791+0000 7f584f8af700 1 Processor -- start 2023-10-08T16:13:39.792 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.792+0000 7f584f8af700 1 -- start start 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.792+0000 7f584f8af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 0x7f584815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.792+0000 7f584f8af700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584815a160 con 0x7f5848069560 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.792+0000 7f584f8af700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480d13a0 con 0x7f5848159c60 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.792+0000 7f584f8af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584815e140 con 0x7f5848159890 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584de4c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5848069560 0x7f58480ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51944/0 (socket says 172.21.15.170:51944) 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584de4c700 1 -- 172.21.15.170:0/1054464724 learned_addr learned my addr 172.21.15.170:0/1054464724 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584d64b700 1 --2- 172.21.15.170:0/1054464724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 0x7f584815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49398122 0 0) 0x7f584815a160 con 0x7f5848069560 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5834003570 con 0x7f5848069560 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885534357 0 0) 0x7f58480d13a0 con 0x7f5848159c60 2023-10-08T16:13:39.795 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f584815a160 con 0x7f5848159c60 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3292088495 0 0) 0x7f5834003570 con 0x7f5848069560 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584d64b700 1 -- 172.21.15.170:0/1054464724 >> v1:172.21.15.191:6789/0 conn(0x7f5848159c60 legacy=0x7f584815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584d64b700 1 -- 172.21.15.170:0/1054464724 >> v1:172.21.15.170:6789/0 conn(0x7f5848069560 legacy=0x7f58480ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584d64b700 1 -- 172.21.15.170:0/1054464724 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584815a3d0 con 0x7f5848159890 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f584d64b700 1 --2- 172.21.15.170:0/1054464724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 0x7f584815abd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5844009910 tx=0x7f58440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa3aad0d3b86688f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400017f0 con 0x7f5848159890 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844011420 con 0x7f5848159890 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.793+0000 7f583ffff700 1 -- 172.21.15.170:0/1054464724 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440117c0 con 0x7f5848159890 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 -- 172.21.15.170:0/1054464724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 msgr2=0x7f584815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 --2- 172.21.15.170:0/1054464724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 0x7f584815abd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5844009910 tx=0x7f58440091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.796 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 -- 172.21.15.170:0/1054464724 shutdown_connections 2023-10-08T16:13:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 --2- 172.21.15.170:0/1054464724 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159890 0x7f584815abd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 -- 172.21.15.170:0/1054464724 >> 172.21.15.170:0/1054464724 conn(0x7f5848068e20 msgr2=0x7f58480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 -- 172.21.15.170:0/1054464724 shutdown_connections 2023-10-08T16:13:39.797 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.794+0000 7f584f8af700 1 -- 172.21.15.170:0/1054464724 wait complete. 2023-10-08T16:13:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.795+0000 7f584f8af700 1 Processor -- start 2023-10-08T16:13:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.795+0000 7f584f8af700 1 -- start start 2023-10-08T16:13:39.799 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5848159890 0x7f58480cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159c60 0x7f5848169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480ce070 con 0x7f5848069560 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480ce350 con 0x7f5848159890 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584f8af700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480ce630 con 0x7f5848159c60 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49990/0 (socket says 172.21.15.170:49990) 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 -- 172.21.15.170:0/3422379012 learned_addr learned my addr 172.21.15.170:0/3422379012 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584de4c700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159c60 0x7f5848169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584ce4a700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5848159890 0x7f58480cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.800 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 -- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159c60 msgr2=0x7f5848169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159c60 0x7f5848169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 -- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5848159890 msgr2=0x7f58480cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5848159890 0x7f58480cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.796+0000 7f584d64b700 1 -- 172.21.15.170:0/3422379012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844009420 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f584d64b700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f58440091e0 tx=0x7f5844016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584401cb30 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584401cc90 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58480ce910 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584401a6a0 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.797+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58481570d0 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.798+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58481572c0 con 0x7f5848069560 2023-10-08T16:13:39.801 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.798+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f584401aad0 con 0x7f5848069560 2023-10-08T16:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.798+0000 7f583d7fa700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 0x7f582c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.798+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5844086620 con 0x7f5848069560 2023-10-08T16:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.798+0000 7f584ce4a700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 0x7f582c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:39.802 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.799+0000 7f584ce4a700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 0x7f582c0697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5848153bc0 tx=0x7f5838007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:39.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:39.803+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58440536c0 con 0x7f5848069560 2023-10-08T16:13:40.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.059+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7f584804c3c0 con 0x7f5848069560 2023-10-08T16:13:40.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.666+0000 7f583d7fa700 1 -- 172.21.15.170:0/3422379012 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v465) v1 ==== 134+0+0 (secure 0 0 0) 0x7f5844056ce0 con 0x7f5848069560 2023-10-08T16:13:40.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 msgr2=0x7f582c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 0x7f582c0697d0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f5848153bc0 tx=0x7f5838007d50 comp rx=0 tx=0).stop 2023-10-08T16:13:40.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 msgr2=0x7f58480cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f58440091e0 tx=0x7f5844016000 comp rx=0 tx=0).stop 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 shutdown_connections 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5848159c60 0x7f5848169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f582c067320 0x7f582c0697d0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5848069560 0x7f58480cd2e0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 --2- 172.21.15.170:0/3422379012 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5848159890 0x7f58480cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 >> 172.21.15.170:0/3422379012 conn(0x7f5848068e20 msgr2=0x7f58480b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 shutdown_connections 2023-10-08T16:13:40.670 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.669+0000 7f584f8af700 1 -- 172.21.15.170:0/3422379012 wait complete. 2023-10-08T16:13:40.683 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:13:40.803 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.803+0000 7ff7f845a700 1 Processor -- start 2023-10-08T16:13:40.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.803+0000 7ff7f845a700 1 -- start start 2023-10-08T16:13:40.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f845a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff7f0069560 0x7ff7f00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:40.804 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f845a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f0069ee0 con 0x7ff7f0159c60 2023-10-08T16:13:40.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f845a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f015a160 con 0x7ff7f0159890 2023-10-08T16:13:40.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f845a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00d13a0 con 0x7ff7f0069560 2023-10-08T16:13:40.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f69f7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff7f0069560 0x7ff7f00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:40.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f59f5700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff7f0159c60 0x7ff7f015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51958/0 (socket says 172.21.15.170:51958) 2023-10-08T16:13:40.805 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.804+0000 7ff7f59f5700 1 -- 172.21.15.170:0/387821764 learned_addr learned my addr 172.21.15.170:0/387821764 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f69f7700 1 -- 172.21.15.170:0/387821764 >> v1:172.21.15.191:6789/0 conn(0x7ff7f0159890 legacy=0x7ff7f015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f69f7700 1 -- 172.21.15.170:0/387821764 >> v1:172.21.15.170:6789/0 conn(0x7ff7f0159c60 legacy=0x7ff7f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f51f4700 1 -- 172.21.15.170:0/387821764 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078112370 0 0) 0x7ff7f0069ee0 con 0x7ff7f0159c60 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f69f7700 1 -- 172.21.15.170:0/387821764 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f015e140 con 0x7ff7f0069560 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f69f7700 1 --2- 172.21.15.170:0/387821764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f00699a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009e60 tx=0x7ff7e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adda47aef3d3c3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f51f4700 1 -- 172.21.15.170:0/387821764 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800e070 con 0x7ff7f0069560 2023-10-08T16:13:40.807 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f51f4700 1 -- 172.21.15.170:0/387821764 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e800bc30 con 0x7ff7f0069560 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f51f4700 1 -- 172.21.15.170:0/387821764 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8012430 con 0x7ff7f0069560 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f845a700 1 -- 172.21.15.170:0/387821764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 msgr2=0x7ff7f00699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f845a700 1 --2- 172.21.15.170:0/387821764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f00699a0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009e60 tx=0x7ff7e8009780 comp rx=0 tx=0).stop 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f845a700 1 -- 172.21.15.170:0/387821764 shutdown_connections 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f845a700 1 --2- 172.21.15.170:0/387821764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f00699a0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.805+0000 7ff7f845a700 1 -- 172.21.15.170:0/387821764 >> 172.21.15.170:0/387821764 conn(0x7ff7f0068e20 msgr2=0x7ff7f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.806+0000 7ff7f845a700 1 -- 172.21.15.170:0/387821764 shutdown_connections 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.806+0000 7ff7f845a700 1 -- 172.21.15.170:0/387821764 wait complete. 2023-10-08T16:13:40.808 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 Processor -- start 2023-10-08T16:13:40.809 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 -- start start 2023-10-08T16:13:40.811 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7f0159890 0x7ff7f015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7f0159c60 0x7ff7f00ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00ce930 con 0x7ff7f0069560 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cfbc0 con 0x7ff7f0159890 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f845a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00cf730 con 0x7ff7f0159c60 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f61f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f61f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50016/0 (socket says 172.21.15.170:50016) 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.807+0000 7ff7f61f6700 1 -- 172.21.15.170:0/3071450291 learned_addr learned my addr 172.21.15.170:0/3071450291 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:40.812 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f59f5700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7f0159890 0x7ff7f015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f69f7700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7f0159c60 0x7ff7f00ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 -- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7f0159c60 msgr2=0x7ff7f00ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7f0159c60 0x7ff7f00ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 -- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7f0159890 msgr2=0x7ff7f015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7f0159890 0x7ff7f015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 -- 172.21.15.170:0/3071450291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8009a20 con 0x7ff7f0069560 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f61f6700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c5c0 tx=0x7ff7ec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011590 con 0x7ff7f0069560 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec011710 con 0x7ff7f0069560 2023-10-08T16:13:40.813 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f00ceb20 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec011ad0 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.808+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f00cf160 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.809+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f00cfa10 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.810+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff7ec01b3b0 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.810+0000 7ff7e67fc700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 0x7ff7d40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.810+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff7ec083d80 con 0x7ff7f0069560 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.810+0000 7ff7f59f5700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 0x7ff7d40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:40.814 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.810+0000 7ff7f59f5700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 0x7ff7d40697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7f015da00 tx=0x7ff7e0007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:40.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.814+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7ec050d10 con 0x7ff7f0069560 2023-10-08T16:13:40.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.994+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff7f004c3c0 con 0x7ff7f0069560 2023-10-08T16:13:40.998 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:40.998+0000 7ff7e67fc700 1 -- 172.21.15.170:0/3071450291 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff7ec017070 con 0x7ff7f0069560 2023-10-08T16:13:41.000 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 msgr2=0x7ff7d40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 0x7ff7d40697d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff7f015da00 tx=0x7ff7e0007f50 comp rx=0 tx=0).stop 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 msgr2=0x7ff7f0154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c5c0 tx=0x7ff7ec00c8d0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 shutdown_connections 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff7f0159c60 0x7ff7f00ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff7d4067320 0x7ff7d40697d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.001 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7f0069560 0x7ff7f0154090 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 --2- 172.21.15.170:0/3071450291 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff7f0159890 0x7ff7f015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 >> 172.21.15.170:0/3071450291 conn(0x7ff7f0068e20 msgr2=0x7ff7f01576a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:41.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 shutdown_connections 2023-10-08T16:13:41.002 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.000+0000 7ff7f845a700 1 -- 172.21.15.170:0/3071450291 wait complete. 2023-10-08T16:13:41.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:13:41.013 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:41.056 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:13:41.098 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:13:41.098 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T16:13:41.099 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:41.212 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.212+0000 7f0d4d142700 1 Processor -- start 2023-10-08T16:13:41.213 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.212+0000 7f0d4d142700 1 -- start start 2023-10-08T16:13:41.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d4d142700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f0d48069560 0x7f0d480699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d4d142700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d48069ee0 con 0x7f0d48159c60 2023-10-08T16:13:41.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d4d142700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4815a160 con 0x7f0d48159890 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d4d142700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480d13a0 con 0x7f0d48069560 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d3e59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0d48159c60 0x7f0d4815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51970/0 (socket says 172.21.15.170:51970) 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.213+0000 7f0d3e59c700 1 -- 172.21.15.170:0/3769670945 learned_addr learned my addr 172.21.15.170:0/3769670945 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025075437 0 0) 0x7f0d48069ee0 con 0x7f0d48159c60 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d34003570 con 0x7f0d48159c60 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4759e700 1 --2- 172.21.15.170:0/3769670945 >> v2:172.21.15.191:3300/0 conn(0x7f0d48069560 0x7f0d480699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1177524820 0 0) 0x7f0d34003570 con 0x7f0d48159c60 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 >> v2:172.21.15.191:3300/0 conn(0x7f0d48069560 msgr2=0x7f0d480699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:41.217 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 --2- 172.21.15.170:0/3769670945 >> v2:172.21.15.191:3300/0 conn(0x7f0d48069560 0x7f0d480699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 >> v1:172.21.15.191:6789/0 conn(0x7f0d48159890 legacy=0x7f0d4815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4815e140 con 0x7f0d48159c60 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f0d30001470 con 0x7f0d48159c60 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 >> v1:172.21.15.170:6789/0 conn(0x7f0d48159c60 legacy=0x7f0d4815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d34003a60 0x7f0d34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d340051c0 0x7f0d3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480d13a0 con 0x7f0d34004810 2023-10-08T16:13:41.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3400c0d0 con 0x7f0d340051c0 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3400c290 con 0x7f0d34003a60 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4759e700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d340051c0 0x7f0d3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d46d9d700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d34003a60 0x7f0d34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 -- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d34003a60 msgr2=0x7f0d34003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d34003a60 0x7f0d34003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 -- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d340051c0 msgr2=0x7f0d3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d340051c0 0x7f0d3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 -- 172.21.15.170:0/3769670945 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d34003570 con 0x7f0d34004810 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4759e700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d340051c0 0x7f0d3400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d4759e700 1 -- 172.21.15.170:0/3769670945 reap_dead start 2023-10-08T16:13:41.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.214+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d34004c80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c3c0 tx=0x7f0d30001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbae5ec4faede10 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000cc90 con 0x7f0d34004810 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3000cdf0 con 0x7f0d34004810 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4659c700 1 -- 172.21.15.170:0/3769670945 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30005b50 con 0x7f0d34004810 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 -- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 msgr2=0x7f0d34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d34004c80 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000c3c0 tx=0x7f0d30001e50 comp rx=0 tx=0).stop 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 -- 172.21.15.170:0/3769670945 shutdown_connections 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 --2- 172.21.15.170:0/3769670945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d34004c80 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 -- 172.21.15.170:0/3769670945 >> 172.21.15.170:0/3769670945 conn(0x7f0d48068e20 msgr2=0x7f0d480d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 -- 172.21.15.170:0/3769670945 shutdown_connections 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.215+0000 7f0d4d142700 1 -- 172.21.15.170:0/3769670945 wait complete. 2023-10-08T16:13:41.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.216+0000 7f0d4d142700 1 Processor -- start 2023-10-08T16:13:41.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.216+0000 7f0d4d142700 1 -- start start 2023-10-08T16:13:41.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d48155b60 0x7f0d480ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d480ceea0 0x7f0d480b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480b4240 con 0x7f0d34004810 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48155fd0 con 0x7f0d480ceea0 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d4d142700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480b3630 con 0x7f0d48155b60 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50038/0 (socket says 172.21.15.170:50038) 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 -- 172.21.15.170:0/1929979026 learned_addr learned my addr 172.21.15.170:0/1929979026 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 -- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d48155b60 msgr2=0x7f0d480ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d48155b60 0x7f0d480ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 -- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d480ceea0 msgr2=0x7f0d480b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d480ceea0 0x7f0d480b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.217+0000 7f0d46d9d700 1 -- 172.21.15.170:0/1929979026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480b3910 con 0x7f0d34004810 2023-10-08T16:13:41.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d46d9d700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e870 tx=0x7f0d38009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38013630 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38013790 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d480b3bf0 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38013b50 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.218+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d480cfa60 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.219+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d480b4090 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.219+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d3801c400 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.219+0000 7f0d3f7fe700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 0x7f0d280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.219+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0d38084b90 con 0x7f0d34004810 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.219+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 0x7f0d280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.220+0000 7f0d3e59c700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 0x7f0d280697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0d480cbd50 tx=0x7f0d30015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.224+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d38051b20 con 0x7f0d34004810 2023-10-08T16:13:41.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.450+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d48156110 con 0x7f0d34004810 2023-10-08T16:13:41.450 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.450+0000 7f0d3f7fe700 1 -- 172.21.15.170:0/1929979026 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+3546 (secure 0 0 0) 0x7f0d38055140 con 0x7f0d34004810 2023-10-08T16:13:41.451 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:41.451 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":125,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6872,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3792762463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":3792762463},{"type":"v1","addr":"172.21.15.170:6837","nonce":3792762463}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/2966489556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":2966489556},{"type":"v1","addr":"172.21.15.191:6834","nonce":2966489556}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":6881,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/2468627952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":2468627952},{"type":"v1","addr":"172.21.15.191:6835","nonce":2468627952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":115},{"gid":8934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/3706235783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3706235783},{"type":"v1","addr":"172.21.15.170:6836","nonce":3706235783}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":125}],"filesystems":[]} 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 msgr2=0x7f0d280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 0x7f0d280697d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0d480cbd50 tx=0x7f0d30015000 comp rx=0 tx=0).stop 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 msgr2=0x7f0d4816ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800e870 tx=0x7f0d38009040 comp rx=0 tx=0).stop 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 shutdown_connections 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d480ceea0 0x7f0d480b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.453 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d28067320 0x7f0d280697d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d34004810 0x7f0d4816ad90 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 --2- 172.21.15.170:0/1929979026 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d48155b60 0x7f0d480ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 >> 172.21.15.170:0/1929979026 conn(0x7f0d48068e20 msgr2=0x7f0d481589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 shutdown_connections 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.452+0000 7f0d4d142700 1 -- 172.21.15.170:0/1929979026 wait complete. 2023-10-08T16:13:41.454 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 125 2023-10-08T16:13:41.465 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T16:13:41.465 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T16:13:41.466 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:13:41.466 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:41.466 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T16:13:41.466 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T16:13:41.466 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:13:41.466 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:41.467 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T16:13:41.467 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T16:13:41.467 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:13:41.467 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:41.467 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T16:13:41.467 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T16:13:41.468 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:13:41.468 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:13:41.474 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:13:41.475 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T16:13:41.476 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:13:41.476 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T16:13:41.478 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:13:41.479 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T16:13:41.480 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:13:41.480 DEBUG:teuthology.orchestra.run.smithi191:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T16:13:41.518 INFO:tasks.ceph.mds.a:Started 2023-10-08T16:13:41.518 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:41.519 INFO:tasks.ceph.mds.c:Started 2023-10-08T16:13:41.520 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:41.522 INFO:tasks.ceph.mds.b:Started 2023-10-08T16:13:41.522 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:41.522 INFO:tasks.ceph.mds.d:Started 2023-10-08T16:13:41.523 DEBUG:teuthology.parallel:result is None 2023-10-08T16:13:41.523 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T16:13:41.591 INFO:tasks.ceph.mds.a.smithi170.stdout:starting mds.a at 2023-10-08T16:13:41.591 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:13:41.606 INFO:tasks.ceph.mds.d.smithi191.stdout:starting mds.d at 2023-10-08T16:13:41.606 INFO:tasks.ceph.mds.b.smithi191.stdout:starting mds.b at 2023-10-08T16:13:41.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.645+0000 7f744759e700 1 Processor -- start 2023-10-08T16:13:41.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.645+0000 7f744759e700 1 -- start start 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744759e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7448159420 0x7f74480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 0x7f74480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744759e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74480b11d0 con 0x7f744815d230 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744759e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74480b14b0 con 0x7f7448159420 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480b1790 con 0x7f74481597f0 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7448159420 0x7f74480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7448159420 0x7f74480b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50072/0 (socket says 172.21.15.170:50072) 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 -- 172.21.15.170:0/2070761304 learned_addr learned my addr 172.21.15.170:0/2070761304 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f7445d9b700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 0x7f74480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 -- 172.21.15.170:0/2070761304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 msgr2=0x7f74480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 0x7f74480b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 -- 172.21.15.170:0/2070761304 >> v1:172.21.15.191:6789/0 conn(0x7f744815d230 legacy=0x7f74480b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f744659c700 1 -- 172.21.15.170:0/2070761304 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480b1a70 con 0x7f7448159420 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.646+0000 7f7445d9b700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 0x7f74480b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744659c700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7448159420 0x7f74480b5f40 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f7438009f20 tx=0x7f7438009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeca73a69fc9df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744559a700 1 -- 172.21.15.170:0/2070761304 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743800e070 con 0x7f7448159420 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744559a700 1 -- 172.21.15.170:0/2070761304 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743800ba40 con 0x7f7448159420 2023-10-08T16:13:41.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744559a700 1 -- 172.21.15.170:0/2070761304 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74380092e0 con 0x7f7448159420 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 -- 172.21.15.170:0/2070761304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7448159420 msgr2=0x7f74480b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7448159420 0x7f74480b5f40 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f7438009f20 tx=0x7f7438009840 comp rx=0 tx=0).stop 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 -- 172.21.15.170:0/2070761304 shutdown_connections 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7448159420 0x7f74480b5f40 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 --2- 172.21.15.170:0/2070761304 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f74481597f0 0x7f74480b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 -- 172.21.15.170:0/2070761304 >> 172.21.15.170:0/2070761304 conn(0x7f74480d0660 msgr2=0x7f74480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 -- 172.21.15.170:0/2070761304 shutdown_connections 2023-10-08T16:13:41.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.647+0000 7f744759e700 1 -- 172.21.15.170:0/2070761304 wait complete. 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.648+0000 7f744759e700 1 Processor -- start 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.648+0000 7f744759e700 1 -- start start 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7448159420 0x7f7448157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f744815d230 0x7f744815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744815a910 con 0x7f74481597f0 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448159d00 con 0x7f744815d230 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744759e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7448159fe0 con 0x7f7448159420 2023-10-08T16:13:41.653 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50088/0 (socket says 172.21.15.170:50088) 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 -- 172.21.15.170:0/66810762 learned_addr learned my addr 172.21.15.170:0/66810762 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f744659c700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7448159420 0x7f7448157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7446d9d700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f744815d230 0x7f744815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 -- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7448159420 msgr2=0x7f7448157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7448159420 0x7f7448157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 -- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f744815d230 msgr2=0x7f744815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f744815d230 0x7f744815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.649+0000 7f7445d9b700 1 -- 172.21.15.170:0/66810762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438009ae0 con 0x7f74481597f0 2023-10-08T16:13:41.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f7445d9b700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c830 tx=0x7f743c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c011600 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c011760 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744815a2c0 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c011b20 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.650+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7448157970 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.651+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744815aac0 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.651+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f743c018360 con 0x7f74481597f0 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.651+0000 7f74367fc700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 0x7f74280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:41.655 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.652+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f743c0849c0 con 0x7f74481597f0 2023-10-08T16:13:41.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.652+0000 7f744659c700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 0x7f74280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:41.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.652+0000 7f744659c700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 0x7f74280697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7438009880 tx=0x7f7438017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:41.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.656+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f744815aac0 con 0x7f74481597f0 2023-10-08T16:13:41.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:41.855+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f744804c3c0 con 0x7f74481597f0 2023-10-08T16:13:42.682 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.682+0000 7f74367fc700 1 -- 172.21.15.170:0/66810762 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v197) v1 ==== 97+0+0 (secure 0 0 0) 0x7f743c015080 con 0x7f74481597f0 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.684+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 msgr2=0x7f74280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.684+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 0x7f74280697d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7438009880 tx=0x7f7438017000 comp rx=0 tx=0).stop 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.684+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 msgr2=0x7f744815ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.684+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c830 tx=0x7f743c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 shutdown_connections 2023-10-08T16:13:42.685 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f744815d230 0x7f744815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7448159420 0x7f7448157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7428067320 0x7f74280697d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 --2- 172.21.15.170:0/66810762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74481597f0 0x7f744815ada0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 >> 172.21.15.170:0/66810762 conn(0x7f74480d0660 msgr2=0x7f744815d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 shutdown_connections 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.685+0000 7f744759e700 1 -- 172.21.15.170:0/66810762 wait complete. 2023-10-08T16:13:42.686 INFO:teuthology.orchestra.run.smithi170.stderr: removed all blocklist entries 2023-10-08T16:13:42.698 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T16:13:42.816 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.816+0000 7f8a081b6700 1 Processor -- start 2023-10-08T16:13:42.817 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.816+0000 7f8a081b6700 1 -- start start 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a081b6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8a00069560 0x7f8a000699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a081b6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a00069ee0 con 0x7f8a00159c60 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a081b6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0015a160 con 0x7f8a00159890 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a081b6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a000d13a0 con 0x7f8a00069560 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a06753700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8a00069560 0x7f8a000699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a05751700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8a00159c60 0x7f8a0015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51972/0 (socket says 172.21.15.170:51972) 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.817+0000 7f8a05751700 1 -- 172.21.15.170:0/3628161622 learned_addr learned my addr 172.21.15.170:0/3628161622 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:42.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a06753700 1 -- 172.21.15.170:0/3628161622 >> v1:172.21.15.191:6789/0 conn(0x7f8a00159890 legacy=0x7f8a0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a06753700 1 -- 172.21.15.170:0/3628161622 >> v1:172.21.15.170:6789/0 conn(0x7f8a00159c60 legacy=0x7f8a0015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a06753700 1 -- 172.21.15.170:0/3628161622 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0015e140 con 0x7f8a00069560 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a04f50700 1 -- 172.21.15.170:0/3628161622 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170091800 0 0) 0x7f8a00069ee0 con 0x7f8a00159c60 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a06753700 1 --2- 172.21.15.170:0/3628161622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00069560 0x7f8a000699a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098e0 tx=0x7f89f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a416d9001fa86f81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a04f50700 1 -- 172.21.15.170:0/3628161622 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8009e80 con 0x7f8a00069560 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a04f50700 1 -- 172.21.15.170:0/3628161622 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f800b820 con 0x7f8a00069560 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a04f50700 1 -- 172.21.15.170:0/3628161622 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800bba0 con 0x7f8a00069560 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 -- 172.21.15.170:0/3628161622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00069560 msgr2=0x7f8a000699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 --2- 172.21.15.170:0/3628161622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00069560 0x7f8a000699a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f89f80098e0 tx=0x7f89f80091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.820 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 -- 172.21.15.170:0/3628161622 shutdown_connections 2023-10-08T16:13:42.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 --2- 172.21.15.170:0/3628161622 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00069560 0x7f8a000699a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 -- 172.21.15.170:0/3628161622 >> 172.21.15.170:0/3628161622 conn(0x7f8a00068e20 msgr2=0x7f8a000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:42.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.818+0000 7f8a081b6700 1 -- 172.21.15.170:0/3628161622 shutdown_connections 2023-10-08T16:13:42.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.819+0000 7f8a081b6700 1 -- 172.21.15.170:0/3628161622 wait complete. 2023-10-08T16:13:42.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 Processor -- start 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 -- start start 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a00069560 0x7f8a0016dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a00159c60 0x7f8a00144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c78a0 con 0x7f8a00159890 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00144780 con 0x7f8a00069560 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a081b6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c7af0 con 0x7f8a00159c60 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a05751700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a05751700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50106/0 (socket says 172.21.15.170:50106) 2023-10-08T16:13:42.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.820+0000 7f8a05751700 1 -- 172.21.15.170:0/595452547 learned_addr learned my addr 172.21.15.170:0/595452547 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a06753700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a00159c60 0x7f8a00144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05f52700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a00069560 0x7f8a0016dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 -- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a00159c60 msgr2=0x7f8a00144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a00159c60 0x7f8a00144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 -- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a00069560 msgr2=0x7f8a0016dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a00069560 0x7f8a0016dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 -- 172.21.15.170:0/595452547 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8009420 con 0x7f8a00159890 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a05751700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f89f000c5b0 tx=0x7f89f0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0005b30 con 0x7f8a00159890 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f0005c90 con 0x7f8a00159890 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000c7dd0 con 0x7f8a00159890 2023-10-08T16:13:42.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0007300 con 0x7f8a00159890 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.821+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000cea20 con 0x7f8a00159890 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.822+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a000cec10 con 0x7f8a00159890 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.822+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f89f001b3b0 con 0x7f8a00159890 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.823+0000 7f89f5ffb700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.823+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f89f0083d50 con 0x7f8a00159890 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.823+0000 7f8a05f52700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:42.826 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.823+0000 7f8a05f52700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 0x7f89e40697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0075c0 tx=0x7f89fc009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:42.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:42.827+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f0050d30 con 0x7f8a00159890 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.082+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8a0004c3c0 con 0x7f8a00159890 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.082+0000 7f89f5ffb700 1 -- 172.21.15.170:0/595452547 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f89f0017090 con 0x7f8a00159890 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stdout: "auth_dump": [ 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.083 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.a", 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJl2UdrMxAAUSx6LGQol/3eIYJVdn296g==", 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.084 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.b", 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJl9VYtAxAAlvNTUaZn/zPgt7HoFLeEIw==", 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.085 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.c", 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJllMNXOhAAJFS/NQ/nLej+vpOXTej3Rg==", 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.086 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mds.d", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlb67mCRAAyZA7nSPmw5QTi+kmJmyF2Q==", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.0", 2023-10-08T16:13:43.087 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDk0iJldOtPLBAAQ7HiC7CHfUlc2I7xoEdxAA==", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.1", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDn0iJlQAyLKhAAX2QEXs+iM4EoJVeatNBTSQ==", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.088 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.2", 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDq0iJlM49HKRAA3kGtjeFPx1W1tdRJkWJ3Xg==", 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.089 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.3", 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDt0iJlLzxpKBAAdJHlgJynFwYEWDaGkU8PVQ==", 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.090 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.4", 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD00iJl+lY/AhAA8PR4jo4ne49sDHU4DCofbw==", 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.091 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.5", 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD20iJlfRtlOxAAKXHv8UyOB2rUJaCfoAhVTg==", 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.092 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.6", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD50iJl4r7LOhAAtEt0q1eHGk6ZiGMpcRQ6fw==", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "osd.7", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQD80iJltEJNOBAANZIvmP4GyCWHrnYXeD21fg==", 2023-10-08T16:13:43.093 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow profile osd", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile osd", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.0", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlLUmuDRAALPU8AbplL54D0pf5fqcjkg==", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow r", 2023-10-08T16:13:43.094 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.1", 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlEDQFFhAAG+qtZ6xTKudci5OV0oyobg==", 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.095 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.2", 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlbo4KExAAPkHMHSfhbu1iRG0+Q0S4zQ==", 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.096 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.3", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDi0iJlsj9cGxAAfF2coT1PbilBSxs3xdXY3g==", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow r", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow rw", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow rwx" 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "client.admin", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDg0iJlXUrONRAA8hzJVnTdTtyfln+uXbCrIg==", 2023-10-08T16:13:43.097 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "mgr": "allow *", 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow *", 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.x", 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJldhcUMBAASGBKchn0vBuTmzJ9g8j7IQ==", 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.098 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "entity": "mgr.y", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "key": "AQDh0iJloFcCLBAA2mY2M5dIRWl5h6caAMz7JA==", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": { 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": "allow *", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "mon": "allow profile mgr", 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: "osd": "allow *" 2023-10-08T16:13:43.099 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.100 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:43.100 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2023-10-08T16:13:43.100 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:13:43.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 msgr2=0x7f89e40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 0x7f89e40697d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0075c0 tx=0x7f89fc009780 comp rx=0 tx=0).stop 2023-10-08T16:13:43.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 msgr2=0x7f8a000ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f89f000c5b0 tx=0x7f89f0001070 comp rx=0 tx=0).stop 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 shutdown_connections 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8a00159c60 0x7f8a00144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8a00069560 0x7f8a0016dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f89e4067320 0x7f89e40697d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 --2- 172.21.15.170:0/595452547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a00159890 0x7f8a000ca210 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.084+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 >> 172.21.15.170:0/595452547 conn(0x7f8a00068e20 msgr2=0x7f8a00152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.085+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 shutdown_connections 2023-10-08T16:13:43.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.085+0000 7f8a081b6700 1 -- 172.21.15.170:0/595452547 wait complete. 2023-10-08T16:13:43.103 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:43.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.219+0000 7f59fecf9700 1 Processor -- start 2023-10-08T16:13:43.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.219+0000 7f59fecf9700 1 -- start start 2023-10-08T16:13:43.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fecf9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f59f8159420 0x7f59f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fecf9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fecf9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f80b2980 con 0x7f59f81597f0 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fecf9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f815dbb0 con 0x7f59f8159420 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fecf9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80b11d0 con 0x7f59f815d230 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f59f8159420 0x7f59f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fd4f6700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f59f81597f0 0x7f59f80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51976/0 (socket says 172.21.15.170:51976) 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fd4f6700 1 -- 172.21.15.170:0/3210694543 learned_addr learned my addr 172.21.15.170:0/3210694543 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fe4f8700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 -- 172.21.15.170:0/3210694543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 msgr2=0x7f59f815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 -- 172.21.15.170:0/3210694543 >> v1:172.21.15.170:6789/0 conn(0x7f59f81597f0 legacy=0x7f59f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 -- 172.21.15.170:0/3210694543 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80b14b0 con 0x7f59f8159420 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fccf5700 1 -- 172.21.15.170:0/3210694543 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365524333 0 0) 0x7f59f80b2980 con 0x7f59f81597f0 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f8159420 0x7f59f80b5f40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098e0 tx=0x7f59f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2023c9959a8069fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.220+0000 7f59fccf5700 1 -- 172.21.15.170:0/3210694543 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4009e80 con 0x7f59f8159420 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fccf5700 1 -- 172.21.15.170:0/3210694543 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f400b860 con 0x7f59f8159420 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fccf5700 1 -- 172.21.15.170:0/3210694543 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f400bbe0 con 0x7f59f8159420 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 -- 172.21.15.170:0/3210694543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f8159420 msgr2=0x7f59f80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f8159420 0x7f59f80b5f40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f59f40098e0 tx=0x7f59f40091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 -- 172.21.15.170:0/3210694543 shutdown_connections 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3210694543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f8159420 0x7f59f80b5f40 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 -- 172.21.15.170:0/3210694543 >> 172.21.15.170:0/3210694543 conn(0x7f59f80d0660 msgr2=0x7f59f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 -- 172.21.15.170:0/3210694543 shutdown_connections 2023-10-08T16:13:43.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.221+0000 7f59fecf9700 1 -- 172.21.15.170:0/3210694543 wait complete. 2023-10-08T16:13:43.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.222+0000 7f59fecf9700 1 Processor -- start 2023-10-08T16:13:43.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.222+0000 7f59fecf9700 1 -- start start 2023-10-08T16:13:43.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 0x7f59f816a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f816a9e0 con 0x7f59f81597f0 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8152590 con 0x7f59f8159420 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fecf9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8152870 con 0x7f59f815d230 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50130/0 (socket says 172.21.15.170:50130) 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 -- 172.21.15.170:0/3314994940 learned_addr learned my addr 172.21.15.170:0/3314994940 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fe4f8700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 0x7f59f816a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 -- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 msgr2=0x7f59f80b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f80b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 -- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 msgr2=0x7f59f816a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 0x7f59f816a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 -- 172.21.15.170:0/3314994940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4009420 con 0x7f59f81597f0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 0x7f59f816a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.223+0000 7f59fd4f6700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c810 tx=0x7f59e8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.224+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8005c00 con 0x7f59f81597f0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.224+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8005d60 con 0x7f59f81597f0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.224+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f8152b50 con 0x7f59f81597f0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.224+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e80072e0 con 0x7f59f81597f0 2023-10-08T16:13:43.228 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.224+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f8153110 con 0x7f59f81597f0 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.225+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f8153330 con 0x7f59f81597f0 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.225+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f59e8007690 con 0x7f59f81597f0 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.225+0000 7f59edffb700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 0x7f59dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.225+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f59e8083d40 con 0x7f59f81597f0 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.225+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 0x7f59dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.229 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.226+0000 7f59fdcf7700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 0x7f59dc0697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f59f40091e0 tx=0x7f59f4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.230+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e804b5a0 con 0x7f59f81597f0 2023-10-08T16:13:43.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.412+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59f804c3c0 con 0x7f59f81597f0 2023-10-08T16:13:43.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.413+0000 7f59edffb700 1 -- 172.21.15.170:0/3314994940 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+12192 (secure 0 0 0) 0x7f59e8017020 con 0x7f59f81597f0 2023-10-08T16:13:43.413 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:43.414 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":197,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:42.680131+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":32,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 msgr2=0x7f59dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 0x7f59dc0697d0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f59f40091e0 tx=0x7f59f4017000 comp rx=0 tx=0).stop 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 msgr2=0x7f59f814cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f59e800c810 tx=0x7f59e8007db0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 shutdown_connections 2023-10-08T16:13:43.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f59f815d230 0x7f59f80b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f59f8159420 0x7f59f816a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f59dc067320 0x7f59dc0697d0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 --2- 172.21.15.170:0/3314994940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59f81597f0 0x7f59f814cb90 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 >> 172.21.15.170:0/3314994940 conn(0x7f59f80d0660 msgr2=0x7f59f80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 shutdown_connections 2023-10-08T16:13:43.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.415+0000 7f59fecf9700 1 -- 172.21.15.170:0/3314994940 wait complete. 2023-10-08T16:13:43.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T16:13:43.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:13:43.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.546+0000 7fe103b68700 1 Processor -- start 2023-10-08T16:13:43.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.546+0000 7fe103b68700 1 -- start start 2023-10-08T16:13:43.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe103b68700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe0fc159c10 0x7fe0fc15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe103b68700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0fc15c910 0x7fe0fc15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe103b68700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0fc15c480 con 0x7fe0fc0b5ea0 2023-10-08T16:13:43.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe103b68700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc0b2a40 con 0x7fe0fc15c910 2023-10-08T16:13:43.549 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe103b68700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0fc15d290 con 0x7fe0fc159c10 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe102105700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0fc15c910 0x7fe0fc15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe102105700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0fc15c910 0x7fe0fc15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50134/0 (socket says 172.21.15.170:50134) 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe102105700 1 -- 172.21.15.170:0/3752758035 learned_addr learned my addr 172.21.15.170:0/3752758035 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.547+0000 7fe101904700 1 --2- 172.21.15.170:0/3752758035 >> v2:172.21.15.191:3300/0 conn(0x7fe0fc159c10 0x7fe0fc15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe102105700 1 -- 172.21.15.170:0/3752758035 >> v2:172.21.15.191:3300/0 conn(0x7fe0fc159c10 msgr2=0x7fe0fc15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe102105700 1 --2- 172.21.15.170:0/3752758035 >> v2:172.21.15.191:3300/0 conn(0x7fe0fc159c10 0x7fe0fc15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe102105700 1 -- 172.21.15.170:0/3752758035 >> v1:172.21.15.191:6789/0 conn(0x7fe0fc0b5ea0 legacy=0x7fe0fc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe102105700 1 -- 172.21.15.170:0/3752758035 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc15b5d0 con 0x7fe0fc15c910 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe102105700 1 --2- 172.21.15.170:0/3752758035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc15cd50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc099f10 tx=0x7fe0f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fcb3b21902574e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe100902700 1 -- 172.21.15.170:0/3752758035 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400b820 con 0x7fe0fc15c910 2023-10-08T16:13:43.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe100902700 1 -- 172.21.15.170:0/3752758035 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f400b980 con 0x7fe0fc15c910 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe100902700 1 -- 172.21.15.170:0/3752758035 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f400bd20 con 0x7fe0fc15c910 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 -- 172.21.15.170:0/3752758035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 msgr2=0x7fe0fc15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 --2- 172.21.15.170:0/3752758035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc15cd50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc099f10 tx=0x7fe0f4009250 comp rx=0 tx=0).stop 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 -- 172.21.15.170:0/3752758035 shutdown_connections 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 --2- 172.21.15.170:0/3752758035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc15cd50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 --2- 172.21.15.170:0/3752758035 >> v2:172.21.15.191:3300/0 conn(0x7fe0fc159c10 0x7fe0fc15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.548+0000 7fe103b68700 1 -- 172.21.15.170:0/3752758035 >> 172.21.15.170:0/3752758035 conn(0x7fe0fc099a20 msgr2=0x7fe0fc0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.549+0000 7fe103b68700 1 -- 172.21.15.170:0/3752758035 shutdown_connections 2023-10-08T16:13:43.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.549+0000 7fe103b68700 1 -- 172.21.15.170:0/3752758035 wait complete. 2023-10-08T16:13:43.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 Processor -- start 2023-10-08T16:13:43.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 -- start start 2023-10-08T16:13:43.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0fc0b5ea0 0x7fe0fc1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0fc159c10 0x7fe0fc169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc153b10 con 0x7fe0fc15c910 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc09e2b0 con 0x7fe0fc159c10 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.550+0000 7fe103b68700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0fc16a3c0 con 0x7fe0fc0b5ea0 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50140/0 (socket says 172.21.15.170:50140) 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 -- 172.21.15.170:0/3858458718 learned_addr learned my addr 172.21.15.170:0/3858458718 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe101904700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0fc0b5ea0 0x7fe0fc1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe101103700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0fc159c10 0x7fe0fc169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 -- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0fc0b5ea0 msgr2=0x7fe0fc1535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0fc0b5ea0 0x7fe0fc1535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 -- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0fc159c10 msgr2=0x7fe0fc169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0fc159c10 0x7fe0fc169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 -- 172.21.15.170:0/3858458718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40094f0 con 0x7fe0fc15c910 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe102105700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009290 tx=0x7fe0f4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f401b680 con 0x7fe0fc15c910 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f401b7e0 con 0x7fe0fc15c910 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0fc155020 con 0x7fe0fc15c910 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f401bba0 con 0x7fe0fc15c910 2023-10-08T16:13:43.556 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.551+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0fc1544c0 con 0x7fe0fc15c910 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.552+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0fc1551b0 con 0x7fe0fc15c910 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.553+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe0f401c450 con 0x7fe0fc15c910 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.553+0000 7fe0f1ffb700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 0x7fe0e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.553+0000 7fe101904700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 0x7fe0e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.553+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe0f40866b0 con 0x7fe0fc15c910 2023-10-08T16:13:43.557 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.553+0000 7fe101904700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 0x7fe0e00697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009fd0 tx=0x7fe0f8009880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.558 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.558+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0f4053690 con 0x7fe0fc15c910 2023-10-08T16:13:43.746 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.746+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0fc15b870 con 0x7fe0fc15c910 2023-10-08T16:13:43.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.746+0000 7fe0f1ffb700 1 -- 172.21.15.170:0/3858458718 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v197) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0f402a020 con 0x7fe0fc15c910 2023-10-08T16:13:43.747 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:13:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 msgr2=0x7fe0e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 0x7fe0e00697d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009fd0 tx=0x7fe0f8009880 comp rx=0 tx=0).stop 2023-10-08T16:13:43.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 msgr2=0x7fe0fc155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009290 tx=0x7fe0f4016000 comp rx=0 tx=0).stop 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 shutdown_connections 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe0e0067320 0x7fe0e00697d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0fc15c910 0x7fe0fc155a20 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe0fc0b5ea0 0x7fe0fc1535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 --2- 172.21.15.170:0/3858458718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe0fc159c10 0x7fe0fc169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 >> 172.21.15.170:0/3858458718 conn(0x7fe0fc099a20 msgr2=0x7fe0fc09b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 shutdown_connections 2023-10-08T16:13:43.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.749+0000 7fe103b68700 1 -- 172.21.15.170:0/3858458718 wait complete. 2023-10-08T16:13:43.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T16:13:43.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.875+0000 7f149a261700 1 Processor -- start 2023-10-08T16:13:43.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f149a261700 1 -- start start 2023-10-08T16:13:43.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f149a261700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f14940b0a10 0x7f1494155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f149a261700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14940ad930 con 0x7f1494155c10 2023-10-08T16:13:43.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f149a261700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14940acd20 con 0x7f14941578b0 2023-10-08T16:13:43.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f149a261700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940ad000 con 0x7f14940b0a10 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.876+0000 7f1493fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f14940b0a10 0x7f1494155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14937fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1494155c10 0x7f14940ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51980/0 (socket says 172.21.15.170:51980) 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f1493fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f14940b0a10 0x7f1494155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50148/0 (socket says 172.21.15.170:50148) 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14937fe700 1 -- 172.21.15.170:0/3840060360 learned_addr learned my addr 172.21.15.170:0/3840060360 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f1493fff700 1 -- 172.21.15.170:0/3840060360 >> v1:172.21.15.191:6789/0 conn(0x7f14941578b0 legacy=0x7f14940af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f1493fff700 1 -- 172.21.15.170:0/3840060360 >> v1:172.21.15.170:6789/0 conn(0x7f1494155c10 legacy=0x7f14940ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f1493fff700 1 -- 172.21.15.170:0/3840060360 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940ad2e0 con 0x7f14940b0a10 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14927fc700 1 -- 172.21.15.170:0/3840060360 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218708509 0 0) 0x7f14940ad930 con 0x7f1494155c10 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f1493fff700 1 --2- 172.21.15.170:0/3840060360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14940b0a10 0x7f1494155fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098e0 tx=0x7f148c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5e253d7e460093 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14927fc700 1 -- 172.21.15.170:0/3840060360 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c009e80 con 0x7f14940b0a10 2023-10-08T16:13:43.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14927fc700 1 -- 172.21.15.170:0/3840060360 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00b860 con 0x7f14940b0a10 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.877+0000 7f14927fc700 1 -- 172.21.15.170:0/3840060360 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00bbe0 con 0x7f14940b0a10 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 -- 172.21.15.170:0/3840060360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14940b0a10 msgr2=0x7f1494155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 --2- 172.21.15.170:0/3840060360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14940b0a10 0x7f1494155fe0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f148c0098e0 tx=0x7f148c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 -- 172.21.15.170:0/3840060360 shutdown_connections 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 --2- 172.21.15.170:0/3840060360 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14940b0a10 0x7f1494155fe0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 -- 172.21.15.170:0/3840060360 >> 172.21.15.170:0/3840060360 conn(0x7f149406a320 msgr2=0x7f14940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:43.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 -- 172.21.15.170:0/3840060360 shutdown_connections 2023-10-08T16:13:43.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.878+0000 7f149a261700 1 -- 172.21.15.170:0/3840060360 wait complete. 2023-10-08T16:13:43.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.879+0000 7f149a261700 1 Processor -- start 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.879+0000 7f149a261700 1 -- start start 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14940b0a10 0x7f14940cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14941578b0 0x7f14940cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1494147840 con 0x7f1494155c10 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cd230 con 0x7f14940b0a10 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cad80 con 0x7f14941578b0 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50154/0 (socket says 172.21.15.170:50154) 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 -- 172.21.15.170:0/3267037117 learned_addr learned my addr 172.21.15.170:0/3267037117 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:43.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1493fff700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14941578b0 0x7f14940cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f14937fe700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14940b0a10 0x7f14940cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 -- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14941578b0 msgr2=0x7f14940cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14941578b0 0x7f14940cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 -- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14940b0a10 msgr2=0x7f14940cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14940b0a10 0x7f14940cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 -- 172.21.15.170:0/3267037117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009420 con 0x7f1494155c10 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f1492ffd700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f148800c4e0 tx=0x7f148800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488011590 con 0x7f1494155c10 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488011710 con 0x7f1494155c10 2023-10-08T16:13:43.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14940c8550 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488011ad0 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.880+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14940ca9c0 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.881+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14940c8940 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.882+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f148801b3b0 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.882+0000 7f147b7fe700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 0x7f1480069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.882+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1488083e60 con 0x7f1494155c10 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.882+0000 7f14937fe700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 0x7f1480069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:43.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.883+0000 7f14937fe700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 0x7f1480069800 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1484000f80 tx=0x7f1484009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:43.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:43.886+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1488050e40 con 0x7f1494155c10 2023-10-08T16:13:44.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.071+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f149404c3c0 con 0x7f1494155c10 2023-10-08T16:13:44.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.072+0000 7f147b7fe700 1 -- 172.21.15.170:0/3267037117 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v197) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1488017070 con 0x7f1494155c10 2023-10-08T16:13:44.072 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "poolnum": 1, 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: "poolname": ".mgr" 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:44.073 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 msgr2=0x7f1480069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 0x7f1480069800 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1484000f80 tx=0x7f1484009040 comp rx=0 tx=0).stop 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 msgr2=0x7f14940a70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f148800c4e0 tx=0x7f148800c8a0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 shutdown_connections 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f14941578b0 0x7f14940cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f14940b0a10 0x7f14940cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1480067350 0x7f1480069800 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 --2- 172.21.15.170:0/3267037117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1494155c10 0x7f14940a70c0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 >> 172.21.15.170:0/3267037117 conn(0x7f149406a320 msgr2=0x7f149406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:44.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 shutdown_connections 2023-10-08T16:13:44.076 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.074+0000 7f149a261700 1 -- 172.21.15.170:0/3267037117 wait complete. 2023-10-08T16:13:44.087 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T16:13:44.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T16:13:44.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.216+0000 7fe92328e700 1 Processor -- start 2023-10-08T16:13:44.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.216+0000 7fe92328e700 1 -- start start 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92328e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe91c159ec0 0x7fe91c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92328e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 0x7fe91c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92328e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91c0b5b00 con 0x7fe91c0cce40 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92328e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91c0b5850 con 0x7fe91c159ec0 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92328e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c0b5da0 con 0x7fe91c0b53d0 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe91c159ec0 0x7fe91c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92182b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fe91c0cce40 0x7fe91c0cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:51992/0 (socket says 172.21.15.170:51992) 2023-10-08T16:13:44.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92182b700 1 -- 172.21.15.170:0/2749662450 learned_addr learned my addr 172.21.15.170:0/2749662450 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe91c159ec0 0x7fe91c0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50158/0 (socket says 172.21.15.170:50158) 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe920829700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 0x7fe91c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 -- 172.21.15.170:0/2749662450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 msgr2=0x7fe91c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 0x7fe91c0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 -- 172.21.15.170:0/2749662450 >> v1:172.21.15.170:6789/0 conn(0x7fe91c0cce40 legacy=0x7fe91c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe913fff700 1 -- 172.21.15.170:0/2749662450 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292654685 0 0) 0x7fe91c0b5b00 con 0x7fe91c0cce40 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 -- 172.21.15.170:0/2749662450 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c0b6080 con 0x7fe91c159ec0 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe920829700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 0x7fe91c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe92102a700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c0b4e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe918009eb0 tx=0x7fe918009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df90d0973a9d481b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe913fff700 1 -- 172.21.15.170:0/2749662450 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800e070 con 0x7fe91c159ec0 2023-10-08T16:13:44.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe913fff700 1 -- 172.21.15.170:0/2749662450 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91800bc30 con 0x7fe91c159ec0 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.217+0000 7fe913fff700 1 -- 172.21.15.170:0/2749662450 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918012430 con 0x7fe91c159ec0 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 -- 172.21.15.170:0/2749662450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 msgr2=0x7fe91c0b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c0b4e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fe918009eb0 tx=0x7fe918009780 comp rx=0 tx=0).stop 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 -- 172.21.15.170:0/2749662450 shutdown_connections 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c0b4e90 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 --2- 172.21.15.170:0/2749662450 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0b53d0 0x7fe91c0cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 -- 172.21.15.170:0/2749662450 >> 172.21.15.170:0/2749662450 conn(0x7fe91c0999f0 msgr2=0x7fe91c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 -- 172.21.15.170:0/2749662450 shutdown_connections 2023-10-08T16:13:44.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.218+0000 7fe92328e700 1 -- 172.21.15.170:0/2749662450 wait complete. 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.219+0000 7fe92328e700 1 Processor -- start 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.219+0000 7fe92328e700 1 -- start start 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe91c0b53d0 0x7fe91c14e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0cce40 0x7fe91c06a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c1524a0 con 0x7fe91c159ec0 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c152780 con 0x7fe91c0b53d0 2023-10-08T16:13:44.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92328e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c068430 con 0x7fe91c0cce40 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50174/0 (socket says 172.21.15.170:50174) 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 -- 172.21.15.170:0/4184637745 learned_addr learned my addr 172.21.15.170:0/4184637745 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe920829700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0cce40 0x7fe91c06a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 -- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0cce40 msgr2=0x7fe91c06a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92102a700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe91c0b53d0 0x7fe91c14e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0cce40 0x7fe91c06a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 -- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe91c0b53d0 msgr2=0x7fe91c14e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe91c0b53d0 0x7fe91c14e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.220+0000 7fe92182b700 1 -- 172.21.15.170:0/4184637745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918009a20 con 0x7fe91c159ec0 2023-10-08T16:13:44.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe92182b700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c7e0 tx=0x7fe914007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914005c00 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe914005d60 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914004a80 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c068710 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.221+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c06a850 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.222+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91c0cbd30 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.222+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe914005030 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.223+0000 7fe9117fa700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 0x7fe9000697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.223+0000 7fe92102a700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 0x7fe9000697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.223+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe914083d90 con 0x7fe91c159ec0 2023-10-08T16:13:44.226 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.223+0000 7fe92102a700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 0x7fe9000697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe918009eb0 tx=0x7fe918017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.227 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.227+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe91404b5a0 con 0x7fe91c159ec0 2023-10-08T16:13:44.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.409+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe91c063c90 con 0x7fe91c159ec0 2023-10-08T16:13:44.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:13:44.689 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:13:44.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.689+0000 7fe9117fa700 1 -- 172.21.15.170:0/4184637745 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v198) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe914050d70 con 0x7fe91c159ec0 2023-10-08T16:13:44.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 msgr2=0x7fe9000697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 0x7fe9000697d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe918009eb0 tx=0x7fe918017000 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 msgr2=0x7fe91c154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fe91400c7e0 tx=0x7fe914007d90 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 shutdown_connections 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe900067320 0x7fe9000697d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe91c159ec0 0x7fe91c154020 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe91c0b53d0 0x7fe91c14e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 --2- 172.21.15.170:0/4184637745 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe91c0cce40 0x7fe91c06a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.696 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 >> 172.21.15.170:0/4184637745 conn(0x7fe91c0999f0 msgr2=0x7fe91c0cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:44.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.695+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 shutdown_connections 2023-10-08T16:13:44.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.696+0000 7fe92328e700 1 -- 172.21.15.170:0/4184637745 wait complete. 2023-10-08T16:13:44.698 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' created 2023-10-08T16:13:44.710 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T16:13:44.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 Processor -- start 2023-10-08T16:13:44.908 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 -- start start 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f594c159950 0x7f594c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c069c70 0x7f594c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594c0ce380 con 0x7f594c0af320 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c0cec20 con 0x7f594c159950 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.908+0000 7f59515b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0afa90 con 0x7f594c069c70 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594b7fe700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c069c70 0x7f594c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f594c159950 0x7f594c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.910 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594a7fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f594c0af320 0x7f594c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:43714/0 (socket says 172.21.15.170:43714) 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594a7fc700 1 -- 172.21.15.170:0/3957891630 learned_addr learned my addr 172.21.15.170:0/3957891630 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f5949ffb700 1 -- 172.21.15.170:0/3957891630 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384638164 0 0) 0x7f594c0ce380 con 0x7f594c0af320 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f5949ffb700 1 -- 172.21.15.170:0/3957891630 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5934003570 con 0x7f594c0af320 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 -- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c069c70 msgr2=0x7f594c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 --2- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c069c70 0x7f594c0cf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 -- 172.21.15.170:0/3957891630 >> v1:172.21.15.191:6789/0 conn(0x7f594c0af320 legacy=0x7f594c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 -- 172.21.15.170:0/3957891630 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c0ce5d0 con 0x7f594c159950 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f594affd700 1 --2- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c0cbb90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f593c009e60 tx=0x7f593c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11fcb9cf86233895 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.909+0000 7f5949ffb700 1 -- 172.21.15.170:0/3957891630 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00e070 con 0x7f594c159950 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f5949ffb700 1 -- 172.21.15.170:0/3957891630 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c00bab0 con 0x7f594c159950 2023-10-08T16:13:44.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f5949ffb700 1 -- 172.21.15.170:0/3957891630 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c00be70 con 0x7f594c159950 2023-10-08T16:13:44.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 -- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 msgr2=0x7f594c0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 --2- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c0cbb90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f593c009e60 tx=0x7f593c009780 comp rx=0 tx=0).stop 2023-10-08T16:13:44.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 -- 172.21.15.170:0/3957891630 shutdown_connections 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 --2- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c069c70 0x7f594c0cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 --2- 172.21.15.170:0/3957891630 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c0cbb90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 -- 172.21.15.170:0/3957891630 >> 172.21.15.170:0/3957891630 conn(0x7f594c068e00 msgr2=0x7f594c0d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 -- 172.21.15.170:0/3957891630 shutdown_connections 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.910+0000 7f59515b1700 1 -- 172.21.15.170:0/3957891630 wait complete. 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.911+0000 7f59515b1700 1 Processor -- start 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.911+0000 7f59515b1700 1 -- start start 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c0af320 0x7f594c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c15b610 con 0x7f594c069c70 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0b5c90 con 0x7f594c159950 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f59515b1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0b6490 con 0x7f594c0af320 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50190/0 (socket says 172.21.15.170:50190) 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 -- 172.21.15.170:0/2645509527 learned_addr learned my addr 172.21.15.170:0/2645509527 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594a7fc700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c0af320 0x7f594c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594b7fe700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 -- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c0af320 msgr2=0x7f594c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c0af320 0x7f594c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 -- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 msgr2=0x7f594c169c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:44.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c169c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594affd700 1 -- 172.21.15.170:0/2645509527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009a20 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.912+0000 7f594b7fe700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f594affd700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f593c0097c0 tx=0x7f593c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c012ba0 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c012d20 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c15b8d0 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0276c0 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.913+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c0b61b0 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.914+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c0718b0 con 0x7f594c069c70 2023-10-08T16:13:44.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.914+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f593c024070 con 0x7f594c069c70 2023-10-08T16:13:44.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.915+0000 7f5932ffd700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 0x7f59380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:44.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.915+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 5223+0+0 (secure 0 0 0) 0x7f593c086f10 con 0x7f594c069c70 2023-10-08T16:13:44.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.915+0000 7f594a7fc700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 0x7f59380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:44.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.915+0000 7f594a7fc700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 0x7f59380697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f594c158610 tx=0x7f5940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:44.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:44.919+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f593c053d40 con 0x7f594c069c70 2023-10-08T16:13:45.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.101+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f594c0ce8b0 con 0x7f594c069c70 2023-10-08T16:13:45.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.694+0000 7f5932ffd700 1 -- 172.21.15.170:0/2645509527 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v199) v1 ==== 170+0+0 (secure 0 0 0) 0x7f593c02c0d0 con 0x7f594c069c70 2023-10-08T16:13:45.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 msgr2=0x7f59380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 0x7f59380697d0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f594c158610 tx=0x7f5940009040 comp rx=0 tx=0).stop 2023-10-08T16:13:45.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 msgr2=0x7f594c152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f593c0097c0 tx=0x7f593c019000 comp rx=0 tx=0).stop 2023-10-08T16:13:45.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 shutdown_connections 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f594c159950 0x7f594c169c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5938067320 0x7f59380697d0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f594c069c70 0x7f594c152360 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 --2- 172.21.15.170:0/2645509527 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f594c0af320 0x7f594c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.699+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 >> 172.21.15.170:0/2645509527 conn(0x7f594c068e00 msgr2=0x7f594c0cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.700+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 shutdown_connections 2023-10-08T16:13:45.700 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.700+0000 7f59515b1700 1 -- 172.21.15.170:0/2645509527 wait complete. 2023-10-08T16:13:45.713 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' created 2023-10-08T16:13:45.720 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T16:13:45.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.838+0000 7fdaf5b3b700 1 Processor -- start 2023-10-08T16:13:45.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.838+0000 7fdaf5b3b700 1 -- start start 2023-10-08T16:13:45.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaf5b3b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdaf015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaf5b3b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf015a160 con 0x7fdaf0159890 2023-10-08T16:13:45.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaf5b3b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00d13a0 con 0x7fdaf015db10 2023-10-08T16:13:45.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaf5b3b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015e140 con 0x7fdaf0159c60 2023-10-08T16:13:45.842 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaef7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdaf0159890 0x7fdaf015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55580/0 (socket says 172.21.15.170:55580) 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaef7fe700 1 -- 172.21.15.170:0/917176285 learned_addr learned my addr 172.21.15.170:0/917176285 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdaf015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895074525 0 0) 0x7fdaf015a160 con 0x7fdaf0159890 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8003570 con 0x7fdaf0159890 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833563429 0 0) 0x7fdad8003570 con 0x7fdaf0159890 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 msgr2=0x7fdaf015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdaf015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 >> v1:172.21.15.191:6789/0 conn(0x7fdaf015db10 legacy=0x7fdaf0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf015a3d0 con 0x7fdaf0159890 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fdae4001470 con 0x7fdaf0159890 2023-10-08T16:13:45.843 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 >> v1:172.21.15.170:6789/0 conn(0x7fdaf0159890 legacy=0x7fdaf015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdad8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdad8003fd0 0x7fdad8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8004d50 0x7fdad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015e140 con 0x7fdad8003fd0 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800bb40 con 0x7fdad8004d50 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.839+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800bd00 con 0x7fdaf0159c60 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdad8003fd0 0x7fdad8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaef7fe700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdad8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeffff700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8004d50 0x7fdad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 -- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 msgr2=0x7fdad8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:45.844 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf0159c60 0x7fdad8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 -- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8004d50 msgr2=0x7fdad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8004d50 0x7fdad80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 -- 172.21.15.170:0/917176285 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8003570 con 0x7fdad8003fd0 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdad8003fd0 0x7fdad8004810 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fdaf015a090 tx=0x7fdae00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fabc1e90be2f953 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000e710 con 0x7fdad8003fd0 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae000e870 con 0x7fdad8003fd0 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.840+0000 7fdaee7fc700 1 -- 172.21.15.170:0/917176285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000ec30 con 0x7fdad8003fd0 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/917176285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdad8003fd0 msgr2=0x7fdad8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/917176285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdad8003fd0 0x7fdad8004810 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fdaf015a090 tx=0x7fdae00091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.845 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaeffff700 1 -- 172.21.15.170:0/917176285 reap_dead start 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/917176285 shutdown_connections 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/917176285 >> 172.21.15.170:0/917176285 conn(0x7fdaf0068e20 msgr2=0x7fdaf00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/917176285 shutdown_connections 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.841+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/917176285 wait complete. 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.842+0000 7fdaf5b3b700 1 Processor -- start 2023-10-08T16:13:45.846 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.842+0000 7fdaf5b3b700 1 -- start start 2023-10-08T16:13:45.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8003fd0 0x7fdaf0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf015d470 0x7fdaf015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0152440 con 0x7fdaf00cf430 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015c950 con 0x7fdad8003fd0 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf015bd40 con 0x7fdaf015d470 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52212/0 (socket says 172.21.15.170:52212) 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 -- 172.21.15.170:0/1629068453 learned_addr learned my addr 172.21.15.170:0/1629068453 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaef7fe700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8003fd0 0x7fdaf0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeeffd700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf015d470 0x7fdaf015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 -- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf015d470 msgr2=0x7fdaf015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf015d470 0x7fdaf015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 -- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8003fd0 msgr2=0x7fdaf0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8003fd0 0x7fdaf0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 -- 172.21.15.170:0/1629068453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf015c020 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaeffff700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00d03b0 tx=0x7fdae80017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800c750 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae800c8d0 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf015c300 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800cc90 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.843+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf015ba10 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.844+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf015cce0 con 0x7fdaf00cf430 2023-10-08T16:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.845+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdae8015640 con 0x7fdaf00cf430 2023-10-08T16:13:45.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.845+0000 7fdad77fe700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 0x7fdadc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:45.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.845+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fdae8085180 con 0x7fdaf00cf430 2023-10-08T16:13:45.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.845+0000 7fdaef7fe700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 0x7fdadc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:45.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.845+0000 7fdaef7fe700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 0x7fdadc0697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdae4007e90 tx=0x7fdae4001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:45.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:45.849+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdae8051e70 con 0x7fdaf00cf430 2023-10-08T16:13:46.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.074+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdaf00b2aa0 con 0x7fdaf00cf430 2023-10-08T16:13:46.711 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.710+0000 7fdad77fe700 1 -- 172.21.15.170:0/1629068453 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 33 and data pool 34 v127) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdae8055490 con 0x7fdaf00cf430 2023-10-08T16:13:46.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.715+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 msgr2=0x7fdadc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.715 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.715+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 0x7fdadc0697d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdae4007e90 tx=0x7fdae4001df0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.715+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 msgr2=0x7fdaf00cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.716 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.715+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00d03b0 tx=0x7fdae80017d0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 shutdown_connections 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdadc067320 0x7fdadc0697d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaf00cf430 0x7fdaf00cf8a0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdad8003fd0 0x7fdaf0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 --2- 172.21.15.170:0/1629068453 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdaf015d470 0x7fdaf015b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 >> 172.21.15.170:0/1629068453 conn(0x7fdaf0068e20 msgr2=0x7fdaf0156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:46.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 shutdown_connections 2023-10-08T16:13:46.722 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.720+0000 7fdaf5b3b700 1 -- 172.21.15.170:0/1629068453 wait complete. 2023-10-08T16:13:46.722 INFO:teuthology.orchestra.run.smithi170.stderr:new fs with metadata pool 33 and data pool 34 2023-10-08T16:13:46.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:46.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.850+0000 7f2d1badc700 1 Processor -- start 2023-10-08T16:13:46.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.850+0000 7f2d1badc700 1 -- start start 2023-10-08T16:13:46.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d1badc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2d1415dba0 0x7f2d1406ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:46.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d1badc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140c74b0 con 0x7f2d1415d730 2023-10-08T16:13:46.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d1badc700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d140c8700 con 0x7f2d14159b10 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d1badc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d140c9070 con 0x7f2d1415dba0 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d19077700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2d1415d730 0x7f2d1406a410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55596/0 (socket says 172.21.15.170:55596) 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d19077700 1 -- 172.21.15.170:0/3406415516 learned_addr learned my addr 172.21.15.170:0/3406415516 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.851+0000 7f2d1a079700 1 --2- 172.21.15.170:0/3406415516 >> v2:172.21.15.170:3300/0 conn(0x7f2d1415dba0 0x7f2d1406ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d1a079700 1 -- 172.21.15.170:0/3406415516 >> v1:172.21.15.191:6789/0 conn(0x7f2d14159b10 legacy=0x7f2d140cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d1a079700 1 -- 172.21.15.170:0/3406415516 >> v1:172.21.15.170:6789/0 conn(0x7f2d1415d730 legacy=0x7f2d1406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d1a079700 1 -- 172.21.15.170:0/3406415516 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140c80f0 con 0x7f2d1415dba0 2023-10-08T16:13:46.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d1a079700 1 --2- 172.21.15.170:0/3406415516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d1415dba0 0x7f2d1406ab20 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098e0 tx=0x7f2d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dc47a5945721e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:46.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d18876700 1 -- 172.21.15.170:0/3406415516 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c009e80 con 0x7f2d1415dba0 2023-10-08T16:13:46.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d18876700 1 -- 172.21.15.170:0/3406415516 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0c00b8f0 con 0x7f2d1415dba0 2023-10-08T16:13:46.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.852+0000 7f2d18876700 1 -- 172.21.15.170:0/3406415516 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00bc70 con 0x7f2d1415dba0 2023-10-08T16:13:46.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 -- 172.21.15.170:0/3406415516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d1415dba0 msgr2=0x7f2d1406ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.855 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 --2- 172.21.15.170:0/3406415516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d1415dba0 0x7f2d1406ab20 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098e0 tx=0x7f2d0c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 -- 172.21.15.170:0/3406415516 shutdown_connections 2023-10-08T16:13:46.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 --2- 172.21.15.170:0/3406415516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d1415dba0 0x7f2d1406ab20 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 -- 172.21.15.170:0/3406415516 >> 172.21.15.170:0/3406415516 conn(0x7f2d140c9870 msgr2=0x7f2d140c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:46.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 -- 172.21.15.170:0/3406415516 shutdown_connections 2023-10-08T16:13:46.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.853+0000 7f2d1badc700 1 -- 172.21.15.170:0/3406415516 wait complete. 2023-10-08T16:13:46.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 Processor -- start 2023-10-08T16:13:46.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 -- start start 2023-10-08T16:13:46.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d1415d730 0x7f2d1415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1415dba0 0x7f2d14140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c1ba0 con 0x7f2d14159b10 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c63b0 con 0x7f2d1415d730 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.854+0000 7f2d1badc700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d140c6690 con 0x7f2d1415dba0 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52232/0 (socket says 172.21.15.170:52232) 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 -- 172.21.15.170:0/398748676 learned_addr learned my addr 172.21.15.170:0/398748676 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19077700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d1415d730 0x7f2d1415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d1a079700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1415dba0 0x7f2d14140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:46.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 -- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1415dba0 msgr2=0x7f2d14140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1415dba0 0x7f2d14140b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 -- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d1415d730 msgr2=0x7f2d1415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d1415d730 0x7f2d1415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 -- 172.21.15.170:0/398748676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c009420 con 0x7f2d14159b10 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d19878700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800cb10 tx=0x7f2d080017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800a760 con 0x7f2d14159b10 2023-10-08T16:13:46.862 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0800a8c0 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d140c6970 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800ac80 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.855+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1415b840 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.856+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1404c3c0 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.857+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2d08003030 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.857+0000 7f2d11ffb700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 0x7f2cf80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.857+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2d08084fb0 con 0x7f2d14159b10 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.857+0000 7f2d19077700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 0x7f2cf80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:46.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.857+0000 7f2d19077700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 0x7f2cf80697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2d140697e0 tx=0x7f2d00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:46.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:46.861+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d08051a30 con 0x7f2d14159b10 2023-10-08T16:13:47.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.045+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d140c7640 con 0x7f2d14159b10 2023-10-08T16:13:47.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.045+0000 7f2d11ffb700 1 -- 172.21.15.170:0/398748676 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f2d08055050 con 0x7f2d14159b10 2023-10-08T16:13:47.046 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:47.046 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":200,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:46.076037+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:47.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 msgr2=0x7f2cf80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.048 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 0x7f2cf80697d0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2d140697e0 tx=0x7f2d00009040 comp rx=0 tx=0).stop 2023-10-08T16:13:47.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 msgr2=0x7f2d141545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800cb10 tx=0x7f2d080017f0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 shutdown_connections 2023-10-08T16:13:47.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2d1415dba0 0x7f2d14140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2cf8067320 0x7f2cf80697d0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d14159b10 0x7f2d141545a0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 --2- 172.21.15.170:0/398748676 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2d1415d730 0x7f2d1415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 >> 172.21.15.170:0/398748676 conn(0x7f2d140c9870 msgr2=0x7f2d1406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:47.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 shutdown_connections 2023-10-08T16:13:47.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.048+0000 7f2d1badc700 1 -- 172.21.15.170:0/398748676 wait complete. 2023-10-08T16:13:47.060 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:47.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.175+0000 7f7ac33d4700 1 Processor -- start 2023-10-08T16:13:47.176 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.176+0000 7f7ac33d4700 1 -- start start 2023-10-08T16:13:47.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.176+0000 7f7ac33d4700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7abc159890 0x7f7abc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.176+0000 7f7ac33d4700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc15a160 con 0x7f7abc069560 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.176+0000 7f7ac33d4700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7abc0d13a0 con 0x7f7abc159c60 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.176+0000 7f7ac33d4700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc15e140 con 0x7f7abc159890 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7abc159890 0x7f7abc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7abc159890 0x7f7abc15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52244/0 (socket says 172.21.15.170:52244) 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 -- 172.21.15.170:0/1587503765 learned_addr learned my addr 172.21.15.170:0/1587503765 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1971700 1 --1- 172.21.15.170:0/1587503765 >> v1:172.21.15.170:6789/0 conn(0x7f7abc069560 0x7f7abc0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55598/0 (socket says 172.21.15.170:55598) 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 -- 172.21.15.170:0/1587503765 >> v1:172.21.15.191:6789/0 conn(0x7f7abc159c60 legacy=0x7f7abc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 -- 172.21.15.170:0/1587503765 >> v1:172.21.15.170:6789/0 conn(0x7f7abc069560 legacy=0x7f7abc0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 -- 172.21.15.170:0/1587503765 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc15a3d0 con 0x7f7abc159890 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ac1170700 1 --2- 172.21.15.170:0/1587503765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc159890 0x7f7abc15abd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f20 tx=0x7f7ab8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d891e957c498ea13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ab3fff700 1 -- 172.21.15.170:0/1587503765 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab800e070 con 0x7f7abc159890 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ab3fff700 1 -- 172.21.15.170:0/1587503765 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab800ba40 con 0x7f7abc159890 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.177+0000 7f7ab3fff700 1 -- 172.21.15.170:0/1587503765 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80092e0 con 0x7f7abc159890 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 -- 172.21.15.170:0/1587503765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc159890 msgr2=0x7f7abc15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/1587503765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc159890 0x7f7abc15abd0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009f20 tx=0x7f7ab8009840 comp rx=0 tx=0).stop 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 -- 172.21.15.170:0/1587503765 shutdown_connections 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/1587503765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc159890 0x7f7abc15abd0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 -- 172.21.15.170:0/1587503765 >> 172.21.15.170:0/1587503765 conn(0x7f7abc068e20 msgr2=0x7f7abc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 -- 172.21.15.170:0/1587503765 shutdown_connections 2023-10-08T16:13:47.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.178+0000 7f7ac33d4700 1 -- 172.21.15.170:0/1587503765 wait complete. 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.179+0000 7f7ac33d4700 1 Processor -- start 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.179+0000 7f7ac33d4700 1 -- start start 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7abc159890 0x7f7abc16d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 0x7f7abc16d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc16b700 con 0x7f7abc069560 2023-10-08T16:13:47.183 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0c92f0 con 0x7f7abc159c60 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc16add0 con 0x7f7abc159890 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52246/0 (socket says 172.21.15.170:52246) 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 -- 172.21.15.170:0/3231046958 learned_addr learned my addr 172.21.15.170:0/3231046958 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 -- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7abc159890 msgr2=0x7f7abc16d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7abc159890 0x7f7abc16d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 -- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 msgr2=0x7f7abc16d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1971700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 0x7f7abc16d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 0x7f7abc16d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 -- 172.21.15.170:0/3231046958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8009ae0 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1971700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 0x7f7abc16d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac1170700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7abc070e20 tx=0x7f7ab8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8012ae0 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab8012c60 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc16b0b0 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab801d420 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.180+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc0c8590 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.182+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc04c3c0 con 0x7f7abc069560 2023-10-08T16:13:47.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.182+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7ab801d850 con 0x7f7abc069560 2023-10-08T16:13:47.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.182+0000 7f7ab17fa700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 0x7f7aa00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.182+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ab8086550 con 0x7f7abc069560 2023-10-08T16:13:47.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.182+0000 7f7ac096f700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 0x7f7aa00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.183+0000 7f7ac096f700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 0x7f7aa00697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7aac007fd0 tx=0x7f7aac007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.186+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ab80530d0 con 0x7f7abc069560 2023-10-08T16:13:47.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.371+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7abc15b4d0 con 0x7f7abc069560 2023-10-08T16:13:47.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.372+0000 7f7ab17fa700 1 -- 172.21.15.170:0/3231046958 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+15425 (secure 0 0 0) 0x7f7ab80566f0 con 0x7f7abc069560 2023-10-08T16:13:47.373 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:47.373 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":200,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:46.076037+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 msgr2=0x7f7aa00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 0x7f7aa00697d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7aac007fd0 tx=0x7f7aac007b20 comp rx=0 tx=0).stop 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 msgr2=0x7f7abc140f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7abc070e20 tx=0x7f7ab8017000 comp rx=0 tx=0).stop 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 shutdown_connections 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7abc159c60 0x7f7abc16d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7aa0067320 0x7f7aa00697d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7abc069560 0x7f7abc140f80 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.375 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 --2- 172.21.15.170:0/3231046958 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7abc159890 0x7f7abc16d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 >> 172.21.15.170:0/3231046958 conn(0x7f7abc068e20 msgr2=0x7f7abc0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 shutdown_connections 2023-10-08T16:13:47.376 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.374+0000 7f7ac33d4700 1 -- 172.21.15.170:0/3231046958 wait complete. 2023-10-08T16:13:47.387 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T16:13:47.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.504+0000 7f432359e700 1 Processor -- start 2023-10-08T16:13:47.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.504+0000 7f432359e700 1 -- start start 2023-10-08T16:13:47.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f43240b4e70 0x7f43240ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43240cfb00 con 0x7f43240b1b40 2023-10-08T16:13:47.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432359e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432415e060 con 0x7f43240b16d0 2023-10-08T16:13:47.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240b5c40 con 0x7f43240b4e70 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f4322d9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f43240b1b40 0x7f432415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55614/0 (socket says 172.21.15.170:55614) 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f4322d9d700 1 -- 172.21.15.170:0/2452118190 learned_addr learned my addr 172.21.15.170:0/2452118190 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432259c700 1 --2- 172.21.15.170:0/2452118190 >> v2:172.21.15.191:3300/0 conn(0x7f43240b4e70 0x7f43240ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250500491 0 0) 0x7f43240cfb00 con 0x7f43240b1b40 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003570 con 0x7f43240b1b40 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.505+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518317622 0 0) 0x7f4310003570 con 0x7f43240b1b40 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 >> v2:172.21.15.191:3300/0 conn(0x7f43240b4e70 msgr2=0x7f43240ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 --2- 172.21.15.170:0/2452118190 >> v2:172.21.15.191:3300/0 conn(0x7f43240b4e70 0x7f43240ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 >> v1:172.21.15.191:6789/0 conn(0x7f43240b16d0 legacy=0x7f432415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240b5f20 con 0x7f43240b1b40 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f4314001470 con 0x7f43240b1b40 2023-10-08T16:13:47.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 >> v1:172.21.15.170:6789/0 conn(0x7f43240b1b40 legacy=0x7f432415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4310003a60 0x7f4310003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 0x7f4310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43100051c0 0x7f431000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240b5c40 con 0x7f4310004810 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431000c160 con 0x7f43100051c0 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431000c320 con 0x7f4310003a60 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 0x7f4310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432259c700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4310003a60 0x7f4310003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4322d9d700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43100051c0 0x7f431000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 -- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4310003a60 msgr2=0x7f4310003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4310003a60 0x7f4310003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 -- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43100051c0 msgr2=0x7f431000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f43100051c0 0x7f431000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4322d9d700 1 -- 172.21.15.170:0/2452118190 reap_dead start 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 -- 172.21.15.170:0/2452118190 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310003570 con 0x7f4310004810 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f4321d9b700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 0x7f4310004c80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f430c0018b0 tx=0x7f430c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f0b9a21496f5a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c012420 con 0x7f4310004810 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c0125a0 con 0x7f4310004810 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.506+0000 7f432159a700 1 -- 172.21.15.170:0/2452118190 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c012960 con 0x7f4310004810 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 -- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 msgr2=0x7f4310004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 0x7f4310004c80 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f430c0018b0 tx=0x7f430c007d30 comp rx=0 tx=0).stop 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 -- 172.21.15.170:0/2452118190 shutdown_connections 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 --2- 172.21.15.170:0/2452118190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4310004810 0x7f4310004c80 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 -- 172.21.15.170:0/2452118190 >> 172.21.15.170:0/2452118190 conn(0x7f43240d10c0 msgr2=0x7f43240affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 -- 172.21.15.170:0/2452118190 shutdown_connections 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.507+0000 7f432359e700 1 -- 172.21.15.170:0/2452118190 wait complete. 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.508+0000 7f432359e700 1 Processor -- start 2023-10-08T16:13:47.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.508+0000 7f432359e700 1 -- start start 2023-10-08T16:13:47.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4310004810 0x7f43240c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c86a0 0x7f43241699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c9720 con 0x7f4324068fd0 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240a7110 con 0x7f4310004810 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240ca2d0 con 0x7f43240c86a0 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52256/0 (socket says 172.21.15.170:52256) 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 -- 172.21.15.170:0/2083937732 learned_addr learned my addr 172.21.15.170:0/2083937732 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f432259c700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4310004810 0x7f43240c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4321d9b700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c86a0 0x7f43241699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 -- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c86a0 msgr2=0x7f43241699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c86a0 0x7f43241699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 -- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4310004810 msgr2=0x7f43240c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4310004810 0x7f43240c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 -- 172.21.15.170:0/2083937732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240c8e80 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f4322d9d700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f43240b3130 tx=0x7f4314001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.509+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314018030 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.510+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431400f690 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.510+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240c9160 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.510+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400fa30 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.510+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43240c8af0 con 0x7f4324068fd0 2023-10-08T16:13:47.516 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.511+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240ca0a0 con 0x7f4324068fd0 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.511+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f431401b070 con 0x7f4324068fd0 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.511+0000 7f431a7fc700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 0x7f43040697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.511+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4314019070 con 0x7f4324068fd0 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.511+0000 7f432259c700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 0x7f43040697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.512+0000 7f432259c700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 0x7f43040697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4324069c90 tx=0x7f431c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:47.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.516+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43240ca0a0 con 0x7f4324068fd0 2023-10-08T16:13:47.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:47.747+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f43240ca0a0 con 0x7f4324068fd0 2023-10-08T16:13:48.721 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.721+0000 7f431a7fc700 1 -- 172.21.15.170:0/2083937732 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v130) v1 ==== 118+0+0 (secure 0 0 0) 0x7f43240ca0a0 con 0x7f4324068fd0 2023-10-08T16:13:48.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 msgr2=0x7f43040697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 0x7f43040697d0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4324069c90 tx=0x7f431c00b040 comp rx=0 tx=0).stop 2023-10-08T16:13:48.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 msgr2=0x7f4324157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f43240b3130 tx=0x7f4314001e50 comp rx=0 tx=0).stop 2023-10-08T16:13:48.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 shutdown_connections 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4304067320 0x7f43040697d0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4324068fd0 0x7f4324157500 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4310004810 0x7f43240c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 --2- 172.21.15.170:0/2083937732 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f43240c86a0 0x7f43241699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 >> 172.21.15.170:0/2083937732 conn(0x7f43240d10c0 msgr2=0x7f4324068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 shutdown_connections 2023-10-08T16:13:48.724 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.723+0000 7f432359e700 1 -- 172.21.15.170:0/2083937732 wait complete. 2023-10-08T16:13:48.736 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T16:13:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.854+0000 7f127b341700 1 Processor -- start 2023-10-08T16:13:48.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.854+0000 7f127b341700 1 -- start start 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f127b341700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 0x7f127415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f127b341700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f127415db10 0x7f1274069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f127b341700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127415a160 con 0x7f1274159890 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f127b341700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12740d13a0 con 0x7f127415db10 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f127b341700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127415e140 con 0x7f1274159c60 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12798de700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f127415db10 0x7f1274069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12790dd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1274159890 0x7f127415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55628/0 (socket says 172.21.15.170:55628) 2023-10-08T16:13:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12790dd700 1 -- 172.21.15.170:0/2090808505 learned_addr learned my addr 172.21.15.170:0/2090808505 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12788dc700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 0x7f127415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12798de700 1 -- 172.21.15.170:0/2090808505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 msgr2=0x7f127415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12798de700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 0x7f127415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12798de700 1 -- 172.21.15.170:0/2090808505 >> v1:172.21.15.170:6789/0 conn(0x7f1274159890 legacy=0x7f127415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12798de700 1 -- 172.21.15.170:0/2090808505 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127415a3d0 con 0x7f127415db10 2023-10-08T16:13:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.855+0000 7f12788dc700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 0x7f127415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f12798de700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127415db10 0x7f1274069560 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f126c009e60 tx=0x7f126c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15940606fc1c718 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f126bfff700 1 -- 172.21.15.170:0/2090808505 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c00e070 con 0x7f127415db10 2023-10-08T16:13:48.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f126bfff700 1 -- 172.21.15.170:0/2090808505 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c00bc30 con 0x7f127415db10 2023-10-08T16:13:48.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f126bfff700 1 -- 172.21.15.170:0/2090808505 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c012430 con 0x7f127415db10 2023-10-08T16:13:48.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 -- 172.21.15.170:0/2090808505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127415db10 msgr2=0x7f1274069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127415db10 0x7f1274069560 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f126c009e60 tx=0x7f126c009780 comp rx=0 tx=0).stop 2023-10-08T16:13:48.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 -- 172.21.15.170:0/2090808505 shutdown_connections 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f127415db10 0x7f1274069560 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 --2- 172.21.15.170:0/2090808505 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1274159c60 0x7f127415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 -- 172.21.15.170:0/2090808505 >> 172.21.15.170:0/2090808505 conn(0x7f1274068e20 msgr2=0x7f12740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 -- 172.21.15.170:0/2090808505 shutdown_connections 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.856+0000 7f127b341700 1 -- 172.21.15.170:0/2090808505 wait complete. 2023-10-08T16:13:48.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.857+0000 7f127b341700 1 Processor -- start 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.857+0000 7f127b341700 1 -- start start 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1274159c60 0x7f12740c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127415db10 0x7f1274169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740b5590 con 0x7f1274159890 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274152d30 con 0x7f1274159c60 2023-10-08T16:13:48.868 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f127b341700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740b4980 con 0x7f127415db10 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f12790dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f12790dd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52270/0 (socket says 172.21.15.170:52270) 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f12790dd700 1 -- 172.21.15.170:0/1625606898 learned_addr learned my addr 172.21.15.170:0/1625606898 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.858+0000 7f12798de700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127415db10 0x7f1274169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12788dc700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1274159c60 0x7f12740c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.869 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 -- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127415db10 msgr2=0x7f1274169810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127415db10 0x7f1274169810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 -- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1274159c60 msgr2=0x7f12740c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1274159c60 0x7f12740c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 -- 172.21.15.170:0/1625606898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c009a20 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12790dd700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f127000c7c0 tx=0x7f1270007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270005b30 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1270005c90 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12740b4c60 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270004a80 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.859+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12740b5220 con 0x7f1274159890 2023-10-08T16:13:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.860+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12740b5870 con 0x7f1274159890 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.860+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f12700070d0 con 0x7f1274159890 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.861+0000 7f12697fa700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 0x7f12580697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.861+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1270083f50 con 0x7f1274159890 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.861+0000 7f12788dc700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 0x7f12580697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.861+0000 7f12788dc700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 0x7f12580697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f12740c7c60 tx=0x7f1264006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:48.865+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f127004b5a0 con 0x7f1274159890 2023-10-08T16:13:49.090 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.089+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f127404c3c0 con 0x7f1274159890 2023-10-08T16:13:49.723 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.722+0000 7f12697fa700 1 -- 172.21.15.170:0/1625606898 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v131) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1270050be0 con 0x7f1274159890 2023-10-08T16:13:49.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 msgr2=0x7f12580697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:49.725 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 0x7f12580697d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f12740c7c60 tx=0x7f1264006f90 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 msgr2=0x7f1274158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f127000c7c0 tx=0x7f1270007f70 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 shutdown_connections 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f127415db10 0x7f1274169810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1258067320 0x7f12580697d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1274159890 0x7f1274158430 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 --2- 172.21.15.170:0/1625606898 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1274159c60 0x7f12740c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 >> 172.21.15.170:0/1625606898 conn(0x7f1274068e20 msgr2=0x7f12740b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 shutdown_connections 2023-10-08T16:13:49.726 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.725+0000 7f127b341700 1 -- 172.21.15.170:0/1625606898 wait complete. 2023-10-08T16:13:49.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:49.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.856+0000 7f40d2c38700 1 Processor -- start 2023-10-08T16:13:49.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.856+0000 7f40d2c38700 1 -- start start 2023-10-08T16:13:49.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2c38700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f40cc159420 0x7f40cc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2c38700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f40cc15d230 0x7f40cc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2c38700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40cc0b2980 con 0x7f40cc1597f0 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2c38700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc15db70 con 0x7f40cc15d230 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2c38700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40cc0b11d0 con 0x7f40cc159420 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f40cc15d230 0x7f40cc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f40cc15d230 0x7f40cc15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52274/0 (socket says 172.21.15.170:52274) 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 -- 172.21.15.170:0/224885584 learned_addr learned my addr 172.21.15.170:0/224885584 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d1c36700 1 --2- 172.21.15.170:0/224885584 >> v2:172.21.15.191:3300/0 conn(0x7f40cc159420 0x7f40cc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 -- 172.21.15.170:0/224885584 >> v2:172.21.15.191:3300/0 conn(0x7f40cc159420 msgr2=0x7f40cc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 --2- 172.21.15.170:0/224885584 >> v2:172.21.15.191:3300/0 conn(0x7f40cc159420 0x7f40cc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 -- 172.21.15.170:0/224885584 >> v1:172.21.15.191:6789/0 conn(0x7f40cc1597f0 legacy=0x7f40cc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:49.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 -- 172.21.15.170:0/224885584 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0b14b0 con 0x7f40cc15d230 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.857+0000 7f40d2437700 1 --2- 172.21.15.170:0/224885584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc15d230 0x7f40cc15d630 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0b2f20 tx=0x7f40c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec6f3ab7437098c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d0c34700 1 -- 172.21.15.170:0/224885584 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c400e070 con 0x7f40cc15d230 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d0c34700 1 -- 172.21.15.170:0/224885584 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c400ba10 con 0x7f40cc15d230 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d0c34700 1 -- 172.21.15.170:0/224885584 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c40092e0 con 0x7f40cc15d230 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 -- 172.21.15.170:0/224885584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc15d230 msgr2=0x7f40cc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 --2- 172.21.15.170:0/224885584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc15d230 0x7f40cc15d630 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0b2f20 tx=0x7f40c4009840 comp rx=0 tx=0).stop 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 -- 172.21.15.170:0/224885584 shutdown_connections 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 --2- 172.21.15.170:0/224885584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc15d230 0x7f40cc15d630 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 --2- 172.21.15.170:0/224885584 >> v2:172.21.15.191:3300/0 conn(0x7f40cc159420 0x7f40cc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 -- 172.21.15.170:0/224885584 >> 172.21.15.170:0/224885584 conn(0x7f40cc0d0660 msgr2=0x7f40cc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 -- 172.21.15.170:0/224885584 shutdown_connections 2023-10-08T16:13:49.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.858+0000 7f40d2c38700 1 -- 172.21.15.170:0/224885584 wait complete. 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.859+0000 7f40d2c38700 1 Processor -- start 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.859+0000 7f40d2c38700 1 -- start start 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc1597f0 0x7f40cc0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 0x7f40cc154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.863 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc169550 con 0x7f40cc159420 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc16da80 con 0x7f40cc15d230 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2c38700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc15aaa0 con 0x7f40cc1597f0 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52284/0 (socket says 172.21.15.170:52284) 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 -- 172.21.15.170:0/3278342506 learned_addr learned my addr 172.21.15.170:0/3278342506 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1435700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc1597f0 0x7f40cc0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2437700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 0x7f40cc154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 -- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc1597f0 msgr2=0x7f40cc0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc1597f0 0x7f40cc0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 -- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 msgr2=0x7f40cc154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 0x7f40cc154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:49.864 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d1c36700 1 -- 172.21.15.170:0/3278342506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4009ae0 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.860+0000 7f40d2437700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 0x7f40cc154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40d1c36700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d890 tx=0x7f40c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c80125f0 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8012770 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc16d1e0 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8012b30 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.861+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc16d7a0 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.862+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc04c3c0 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.862+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f40c801e460 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.863+0000 7f40c1ffb700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 0x7f40b00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.863+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f40c8085320 con 0x7f40cc159420 2023-10-08T16:13:49.865 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.863+0000 7f40d1435700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 0x7f40b00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:49.866 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.863+0000 7f40d1435700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 0x7f40b00697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0af650 tx=0x7f40bc007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:49.867 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:49.867+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40c8051fb0 con 0x7f40cc159420 2023-10-08T16:13:50.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.090+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40cc0b37c0 con 0x7f40cc159420 2023-10-08T16:13:50.091 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.091+0000 7f40c1ffb700 1 -- 172.21.15.170:0/3278342506 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f40c80555d0 con 0x7f40cc159420 2023-10-08T16:13:50.091 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:50.091 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 msgr2=0x7f40b00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 0x7f40b00697d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f40cc0af650 tx=0x7f40bc007ea0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 msgr2=0x7f40cc16cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f40c800d890 tx=0x7f40c8008040 comp rx=0 tx=0).stop 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 shutdown_connections 2023-10-08T16:13:50.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f40cc15d230 0x7f40cc154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f40b0067320 0x7f40b00697d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f40cc159420 0x7f40cc16cb50 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 --2- 172.21.15.170:0/3278342506 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f40cc1597f0 0x7f40cc0c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 >> 172.21.15.170:0/3278342506 conn(0x7f40cc0d0660 msgr2=0x7f40cc157450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 shutdown_connections 2023-10-08T16:13:50.095 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.093+0000 7f40d2c38700 1 -- 172.21.15.170:0/3278342506 wait complete. 2023-10-08T16:13:50.096 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 131 2023-10-08T16:13:50.106 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:13:50.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.220+0000 7fc6006ab700 1 Processor -- start 2023-10-08T16:13:50.221 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.220+0000 7fc6006ab700 1 -- start start 2023-10-08T16:13:50.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc6006ab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc5f8159890 0x7fc5f815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc6006ab700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 0x7fc5f80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc6006ab700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f8069ee0 con 0x7fc5f8159c60 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc6006ab700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f815a160 con 0x7fc5f8159890 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc6006ab700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80d13a0 con 0x7fc5f8069560 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc5fe447700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc5f8159890 0x7fc5f815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.221+0000 7fc5fe447700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc5f8159890 0x7fc5f815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52296/0 (socket says 172.21.15.170:52296) 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 -- 172.21.15.170:0/3378329043 learned_addr learned my addr 172.21.15.170:0/3378329043 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fec48700 1 --2- 172.21.15.170:0/3378329043 >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 0x7fc5f80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 -- 172.21.15.170:0/3378329043 >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 msgr2=0x7fc5f80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 --2- 172.21.15.170:0/3378329043 >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 0x7fc5f80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 -- 172.21.15.170:0/3378329043 >> v1:172.21.15.191:6789/0 conn(0x7fc5f8159c60 legacy=0x7fc5f815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 -- 172.21.15.170:0/3378329043 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f815e140 con 0x7fc5f8159890 2023-10-08T16:13:50.224 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fec48700 1 --2- 172.21.15.170:0/3378329043 >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 0x7fc5f80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fe447700 1 --2- 172.21.15.170:0/3378329043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f815abd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806a890 tx=0x7fc5f4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939d404ca35e1e92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fd445700 1 -- 172.21.15.170:0/3378329043 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400b820 con 0x7fc5f8159890 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fd445700 1 -- 172.21.15.170:0/3378329043 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f400b980 con 0x7fc5f8159890 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.222+0000 7fc5fd445700 1 -- 172.21.15.170:0/3378329043 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400bd20 con 0x7fc5f8159890 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 -- 172.21.15.170:0/3378329043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 msgr2=0x7fc5f815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 --2- 172.21.15.170:0/3378329043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f815abd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806a890 tx=0x7fc5f4009250 comp rx=0 tx=0).stop 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 -- 172.21.15.170:0/3378329043 shutdown_connections 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 --2- 172.21.15.170:0/3378329043 >> v2:172.21.15.191:3300/0 conn(0x7fc5f8069560 0x7fc5f80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 --2- 172.21.15.170:0/3378329043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f815abd0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 -- 172.21.15.170:0/3378329043 >> 172.21.15.170:0/3378329043 conn(0x7fc5f8068e20 msgr2=0x7fc5f80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 -- 172.21.15.170:0/3378329043 shutdown_connections 2023-10-08T16:13:50.225 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.223+0000 7fc6006ab700 1 -- 172.21.15.170:0/3378329043 wait complete. 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.224+0000 7fc6006ab700 1 Processor -- start 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.224+0000 7fc6006ab700 1 -- start start 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f8069560 0x7fc5f8147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5f8159c60 0x7fc5f80b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c85a0 con 0x7fc5f8159890 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f81477d0 con 0x7fc5f8069560 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c8f70 con 0x7fc5f8159c60 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.230 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52306/0 (socket says 172.21.15.170:52306) 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 -- 172.21.15.170:0/1610692836 learned_addr learned my addr 172.21.15.170:0/1610692836 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fec48700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5f8159c60 0x7fc5f80b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fe447700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f8069560 0x7fc5f8147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 -- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5f8159c60 msgr2=0x7fc5f80b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5f8159c60 0x7fc5f80b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 -- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f8069560 msgr2=0x7fc5f8147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f8069560 0x7fc5f8147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 -- 172.21.15.170:0/1610692836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f40094f0 con 0x7fc5f8159890 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5fdc46700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b5e0 tx=0x7fc5e8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800f510 con 0x7fc5f8159890 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800f670 con 0x7fc5f8159890 2023-10-08T16:13:50.231 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80c79d0 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800fa30 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.225+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80c7f50 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.226+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80c8710 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.227+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc5e801a400 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.227+0000 7fc5ee7fc700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 0x7fc5dc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.227+0000 7fc5fe447700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 0x7fc5dc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.227+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc5e80845c0 con 0x7fc5f8159890 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.228+0000 7fc5fe447700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 0x7fc5dc0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806a890 tx=0x7fc5f4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.232+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc5e80513d0 con 0x7fc5f8159890 2023-10-08T16:13:50.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.415+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5f804c3c0 con 0x7fc5f8159890 2023-10-08T16:13:50.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.415+0000 7fc5ee7fc700 1 -- 172.21.15.170:0/1610692836 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+15412 (secure 0 0 0) 0x7fc5e80549f0 con 0x7fc5f8159890 2023-10-08T16:13:50.416 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:50.416 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":201,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:13:47.699427+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 msgr2=0x7fc5dc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 0x7fc5dc0697d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5f806a890 tx=0x7fc5f4016000 comp rx=0 tx=0).stop 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 msgr2=0x7fc5f80c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800b5e0 tx=0x7fc5e8009120 comp rx=0 tx=0).stop 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 shutdown_connections 2023-10-08T16:13:50.418 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc5f8159c60 0x7fc5f80b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc5f8069560 0x7fc5f8147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc5dc067320 0x7fc5dc0697d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 --2- 172.21.15.170:0/1610692836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5f8159890 0x7fc5f80c8a30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 >> 172.21.15.170:0/1610692836 conn(0x7fc5f8068e20 msgr2=0x7fc5f8153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.417+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 shutdown_connections 2023-10-08T16:13:50.419 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.418+0000 7fc6006ab700 1 -- 172.21.15.170:0/1610692836 wait complete. 2023-10-08T16:13:50.430 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T16:13:50.430 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:13:50.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.543+0000 7faeab59e700 1 Processor -- start 2023-10-08T16:13:50.543 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.543+0000 7faeab59e700 1 -- start start 2023-10-08T16:13:50.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeab59e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeac159420 0x7faeac0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeab59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeab59e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac0b11d0 con 0x7faeac15d230 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeab59e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac0b14b0 con 0x7faeac159420 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeab59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0b1790 con 0x7faeac1597f0 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeac159420 0x7faeac0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faeac159420 0x7faeac0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52318/0 (socket says 172.21.15.170:52318) 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 -- 172.21.15.170:0/496523350 learned_addr learned my addr 172.21.15.170:0/496523350 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faea9d9b700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 -- 172.21.15.170:0/496523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 msgr2=0x7faeac0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:50.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 -- 172.21.15.170:0/496523350 >> v1:172.21.15.191:6789/0 conn(0x7faeac15d230 legacy=0x7faeac0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 -- 172.21.15.170:0/496523350 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0b1a70 con 0x7faeac159420 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.544+0000 7faeaa59c700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac159420 0x7faeac0b5f40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7faea40099b0 tx=0x7faea4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a95b894ece96b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faea959a700 1 -- 172.21.15.170:0/496523350 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400b820 con 0x7faeac159420 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faea959a700 1 -- 172.21.15.170:0/496523350 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400b980 con 0x7faeac159420 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faea959a700 1 -- 172.21.15.170:0/496523350 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400bd20 con 0x7faeac159420 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 -- 172.21.15.170:0/496523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac159420 msgr2=0x7faeac0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac159420 0x7faeac0b5f40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7faea40099b0 tx=0x7faea4009250 comp rx=0 tx=0).stop 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 -- 172.21.15.170:0/496523350 shutdown_connections 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac159420 0x7faeac0b5f40 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 --2- 172.21.15.170:0/496523350 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 -- 172.21.15.170:0/496523350 >> 172.21.15.170:0/496523350 conn(0x7faeac0d0660 msgr2=0x7faeac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 -- 172.21.15.170:0/496523350 shutdown_connections 2023-10-08T16:13:50.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.545+0000 7faeab59e700 1 -- 172.21.15.170:0/496523350 wait complete. 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.546+0000 7faeab59e700 1 Processor -- start 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.546+0000 7faeab59e700 1 -- start start 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeac159420 0x7faeac16a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cf870 con 0x7faeac15d230 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cfb50 con 0x7faeac159420 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeab59e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cfe30 con 0x7faeac1597f0 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeaad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeaad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52330/0 (socket says 172.21.15.170:52330) 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeaad9d700 1 -- 172.21.15.170:0/2055932899 learned_addr learned my addr 172.21.15.170:0/2055932899 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faea9d9b700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.547+0000 7faeaa59c700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeac159420 0x7faeac16a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 -- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 msgr2=0x7faeac0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 -- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeac159420 msgr2=0x7faeac16a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeac159420 0x7faeac16a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 -- 172.21.15.170:0/2055932899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40094f0 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeaad9d700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b4e0 tx=0x7fae9c006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00f5e0 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9c00f740 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac0d0110 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c00fb00 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.548+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac0c9fe0 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.549+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac04c3c0 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.549+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fae9c01a400 con 0x7faeac15d230 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.549+0000 7fae9a7fc700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 0x7fae8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.550+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fae9c084820 con 0x7faeac15d230 2023-10-08T16:13:50.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.550+0000 7faeaa59c700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 0x7fae8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.550+0000 7faeaa59c700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 0x7fae8c0697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7faea40099b0 tx=0x7faea4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.554+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae9c0514f0 con 0x7faeac15d230 2023-10-08T16:13:50.734 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.734+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faeac0cfcc0 con 0x7fae8c067320 2023-10-08T16:13:50.742 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.742+0000 7fae9a7fc700 1 -- 172.21.15.170:0/2055932899 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289481 (secure 0 0 0) 0x7faeac0cfcc0 con 0x7fae8c067320 2023-10-08T16:13:50.745 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 msgr2=0x7fae8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 0x7fae8c0697d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7faea40099b0 tx=0x7faea4016000 comp rx=0 tx=0).stop 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 msgr2=0x7faeac0c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b4e0 tx=0x7fae9c006f90 comp rx=0 tx=0).stop 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 shutdown_connections 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fae8c067320 0x7fae8c0697d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faeac15d230 0x7faeac0c84c0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faeac159420 0x7faeac16a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 --2- 172.21.15.170:0/2055932899 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faeac1597f0 0x7faeac0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 >> 172.21.15.170:0/2055932899 conn(0x7faeac0d0660 msgr2=0x7faeac156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 shutdown_connections 2023-10-08T16:13:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.745+0000 7faeab59e700 1 -- 172.21.15.170:0/2055932899 wait complete. 2023-10-08T16:13:50.749 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:13:50.758 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2023-10-08T16:13:50.334941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444552,"kb_used_data":21192,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752530168,"statfs":{"total":773094113280,"available":770590892032,"internally_reserved":0,"allocated":21700608,"data_stored":10388348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.646426"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719631+0000","last_change":"2023-10-08T16:13:46.716965+0000","last_active":"2023-10-08T16:13:46.719631+0000","last_peered":"2023-10-08T16:13:46.719631+0000","last_clean":"2023-10-08T16:13:46.719631+0000","last_became_active":"2023-10-08T16:13:46.716533+0000","last_became_peered":"2023-10-08T16:13:46.716533+0000","last_unstale":"2023-10-08T16:13:46.719631+0000","last_undegraded":"2023-10-08T16:13:46.719631+0000","last_fullsized":"2023-10-08T16:13:46.719631+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:15:49.088630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720216+0000","last_change":"2023-10-08T16:13:46.712321+0000","last_active":"2023-10-08T16:13:46.720216+0000","last_peered":"2023-10-08T16:13:46.720216+0000","last_clean":"2023-10-08T16:13:46.720216+0000","last_became_active":"2023-10-08T16:13:46.710565+0000","last_became_peered":"2023-10-08T16:13:46.710565+0000","last_unstale":"2023-10-08T16:13:46.720216+0000","last_undegraded":"2023-10-08T16:13:46.720216+0000","last_fullsized":"2023-10-08T16:13:46.720216+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:57.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719777+0000","last_change":"2023-10-08T16:13:46.711501+0000","last_active":"2023-10-08T16:13:46.719777+0000","last_peered":"2023-10-08T16:13:46.719777+0000","last_clean":"2023-10-08T16:13:46.719777+0000","last_became_active":"2023-10-08T16:13:46.711217+0000","last_became_peered":"2023-10-08T16:13:46.711217+0000","last_unstale":"2023-10-08T16:13:46.719777+0000","last_undegraded":"2023-10-08T16:13:46.719777+0000","last_fullsized":"2023-10-08T16:13:46.719777+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:50.720056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726238+0000","last_change":"2023-10-08T16:13:46.723579+0000","last_active":"2023-10-08T16:13:46.726238+0000","last_peered":"2023-10-08T16:13:46.726238+0000","last_clean":"2023-10-08T16:13:46.726238+0000","last_became_active":"2023-10-08T16:13:46.723334+0000","last_became_peered":"2023-10-08T16:13:46.723334+0000","last_unstale":"2023-10-08T16:13:46.726238+0000","last_undegraded":"2023-10-08T16:13:46.726238+0000","last_fullsized":"2023-10-08T16:13:46.726238+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:18:19.423200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718449+0000","last_change":"2023-10-08T16:13:46.716220+0000","last_active":"2023-10-08T16:13:46.718449+0000","last_peered":"2023-10-08T16:13:46.718449+0000","last_clean":"2023-10-08T16:13:46.718449+0000","last_became_active":"2023-10-08T16:13:46.715977+0000","last_became_peered":"2023-10-08T16:13:46.715977+0000","last_unstale":"2023-10-08T16:13:46.718449+0000","last_undegraded":"2023-10-08T16:13:46.718449+0000","last_fullsized":"2023-10-08T16:13:46.718449+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:09.661956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720450+0000","last_change":"2023-10-08T16:13:46.717466+0000","last_active":"2023-10-08T16:13:46.720450+0000","last_peered":"2023-10-08T16:13:46.720450+0000","last_clean":"2023-10-08T16:13:46.720450+0000","last_became_active":"2023-10-08T16:13:46.717104+0000","last_became_peered":"2023-10-08T16:13:46.717104+0000","last_unstale":"2023-10-08T16:13:46.720450+0000","last_undegraded":"2023-10-08T16:13:46.720450+0000","last_fullsized":"2023-10-08T16:13:46.720450+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:33:48.165199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720895+0000","last_change":"2023-10-08T16:13:46.716979+0000","last_active":"2023-10-08T16:13:46.720895+0000","last_peered":"2023-10-08T16:13:46.720895+0000","last_clean":"2023-10-08T16:13:46.720895+0000","last_became_active":"2023-10-08T16:13:46.716575+0000","last_became_peered":"2023-10-08T16:13:46.716575+0000","last_unstale":"2023-10-08T16:13:46.720895+0000","last_undegraded":"2023-10-08T16:13:46.720895+0000","last_fullsized":"2023-10-08T16:13:46.720895+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:54.321204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.725669+0000","last_change":"2023-10-08T16:13:46.719186+0000","last_active":"2023-10-08T16:13:46.725669+0000","last_peered":"2023-10-08T16:13:46.725669+0000","last_clean":"2023-10-08T16:13:46.725669+0000","last_became_active":"2023-10-08T16:13:46.718970+0000","last_became_peered":"2023-10-08T16:13:46.718970+0000","last_unstale":"2023-10-08T16:13:46.725669+0000","last_undegraded":"2023-10-08T16:13:46.725669+0000","last_fullsized":"2023-10-08T16:13:46.725669+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:42.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719377+0000","last_change":"2023-10-08T16:13:46.713041+0000","last_active":"2023-10-08T16:13:46.719377+0000","last_peered":"2023-10-08T16:13:46.719377+0000","last_clean":"2023-10-08T16:13:46.719377+0000","last_became_active":"2023-10-08T16:13:46.712750+0000","last_became_peered":"2023-10-08T16:13:46.712750+0000","last_unstale":"2023-10-08T16:13:46.719377+0000","last_undegraded":"2023-10-08T16:13:46.719377+0000","last_fullsized":"2023-10-08T16:13:46.719377+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:18.258158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711018+0000","last_change":"2023-10-08T16:13:46.710226+0000","last_active":"2023-10-08T16:13:46.711018+0000","last_peered":"2023-10-08T16:13:46.711018+0000","last_clean":"2023-10-08T16:13:46.711018+0000","last_became_active":"2023-10-08T16:13:46.709775+0000","last_became_peered":"2023-10-08T16:13:46.709775+0000","last_unstale":"2023-10-08T16:13:46.711018+0000","last_undegraded":"2023-10-08T16:13:46.711018+0000","last_fullsized":"2023-10-08T16:13:46.711018+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:51.276191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719315+0000","last_change":"2023-10-08T16:13:46.714419+0000","last_active":"2023-10-08T16:13:46.719315+0000","last_peered":"2023-10-08T16:13:46.719315+0000","last_clean":"2023-10-08T16:13:46.719315+0000","last_became_active":"2023-10-08T16:13:46.714049+0000","last_became_peered":"2023-10-08T16:13:46.714049+0000","last_unstale":"2023-10-08T16:13:46.719315+0000","last_undegraded":"2023-10-08T16:13:46.719315+0000","last_fullsized":"2023-10-08T16:13:46.719315+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:13.999024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719734+0000","last_change":"2023-10-08T16:13:46.714854+0000","last_active":"2023-10-08T16:13:46.719734+0000","last_peered":"2023-10-08T16:13:46.719734+0000","last_clean":"2023-10-08T16:13:46.719734+0000","last_became_active":"2023-10-08T16:13:46.714401+0000","last_became_peered":"2023-10-08T16:13:46.714401+0000","last_unstale":"2023-10-08T16:13:46.719734+0000","last_undegraded":"2023-10-08T16:13:46.719734+0000","last_fullsized":"2023-10-08T16:13:46.719734+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:13.109279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726380+0000","last_change":"2023-10-08T16:13:46.724430+0000","last_active":"2023-10-08T16:13:46.726380+0000","last_peered":"2023-10-08T16:13:46.726380+0000","last_clean":"2023-10-08T16:13:46.726380+0000","last_became_active":"2023-10-08T16:13:46.723527+0000","last_became_peered":"2023-10-08T16:13:46.723527+0000","last_unstale":"2023-10-08T16:13:46.726380+0000","last_undegraded":"2023-10-08T16:13:46.726380+0000","last_fullsized":"2023-10-08T16:13:46.726380+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:10.385487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.725890+0000","last_change":"2023-10-08T16:13:46.724989+0000","last_active":"2023-10-08T16:13:46.725890+0000","last_peered":"2023-10-08T16:13:46.725890+0000","last_clean":"2023-10-08T16:13:46.725890+0000","last_became_active":"2023-10-08T16:13:46.724529+0000","last_became_peered":"2023-10-08T16:13:46.724529+0000","last_unstale":"2023-10-08T16:13:46.725890+0000","last_undegraded":"2023-10-08T16:13:46.725890+0000","last_fullsized":"2023-10-08T16:13:46.725890+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:17.387236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720748+0000","last_change":"2023-10-08T16:13:46.716730+0000","last_active":"2023-10-08T16:13:46.720748+0000","last_peered":"2023-10-08T16:13:46.720748+0000","last_clean":"2023-10-08T16:13:46.720748+0000","last_became_active":"2023-10-08T16:13:46.716428+0000","last_became_peered":"2023-10-08T16:13:46.716428+0000","last_unstale":"2023-10-08T16:13:46.720748+0000","last_undegraded":"2023-10-08T16:13:46.720748+0000","last_fullsized":"2023-10-08T16:13:46.720748+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.000625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717786+0000","last_change":"2023-10-08T16:13:46.714598+0000","last_active":"2023-10-08T16:13:46.717786+0000","last_peered":"2023-10-08T16:13:46.717786+0000","last_clean":"2023-10-08T16:13:46.717786+0000","last_became_active":"2023-10-08T16:13:46.714338+0000","last_became_peered":"2023-10-08T16:13:46.714338+0000","last_unstale":"2023-10-08T16:13:46.717786+0000","last_undegraded":"2023-10-08T16:13:46.717786+0000","last_fullsized":"2023-10-08T16:13:46.717786+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:56.307779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.710993+0000","last_change":"2023-10-08T16:13:46.709772+0000","last_active":"2023-10-08T16:13:46.710993+0000","last_peered":"2023-10-08T16:13:46.710993+0000","last_clean":"2023-10-08T16:13:46.710993+0000","last_became_active":"2023-10-08T16:13:46.709469+0000","last_became_peered":"2023-10-08T16:13:46.709469+0000","last_unstale":"2023-10-08T16:13:46.710993+0000","last_undegraded":"2023-10-08T16:13:46.710993+0000","last_fullsized":"2023-10-08T16:13:46.710993+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:18.832660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.705502+0000","last_change":"2023-10-08T16:13:46.701770+0000","last_active":"2023-10-08T16:13:46.705502+0000","last_peered":"2023-10-08T16:13:46.705502+0000","last_clean":"2023-10-08T16:13:46.705502+0000","last_became_active":"2023-10-08T16:13:46.701495+0000","last_became_peered":"2023-10-08T16:13:46.701495+0000","last_unstale":"2023-10-08T16:13:46.705502+0000","last_undegraded":"2023-10-08T16:13:46.705502+0000","last_fullsized":"2023-10-08T16:13:46.705502+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:16:26.230719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.716722+0000","last_change":"2023-10-08T16:13:46.713079+0000","last_active":"2023-10-08T16:13:46.716722+0000","last_peered":"2023-10-08T16:13:46.716722+0000","last_clean":"2023-10-08T16:13:46.716722+0000","last_became_active":"2023-10-08T16:13:46.712881+0000","last_became_peered":"2023-10-08T16:13:46.712881+0000","last_unstale":"2023-10-08T16:13:46.716722+0000","last_undegraded":"2023-10-08T16:13:46.716722+0000","last_fullsized":"2023-10-08T16:13:46.716722+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:13.999024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720842+0000","last_change":"2023-10-08T16:13:46.718098+0000","last_active":"2023-10-08T16:13:46.720842+0000","last_peered":"2023-10-08T16:13:46.720842+0000","last_clean":"2023-10-08T16:13:46.720842+0000","last_became_active":"2023-10-08T16:13:46.717371+0000","last_became_peered":"2023-10-08T16:13:46.717371+0000","last_unstale":"2023-10-08T16:13:46.720842+0000","last_undegraded":"2023-10-08T16:13:46.720842+0000","last_fullsized":"2023-10-08T16:13:46.720842+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721296+0000","last_change":"2023-10-08T16:13:46.719405+0000","last_active":"2023-10-08T16:13:46.721296+0000","last_peered":"2023-10-08T16:13:46.721296+0000","last_clean":"2023-10-08T16:13:46.721296+0000","last_became_active":"2023-10-08T16:13:46.718899+0000","last_became_peered":"2023-10-08T16:13:46.718899+0000","last_unstale":"2023-10-08T16:13:46.721296+0000","last_undegraded":"2023-10-08T16:13:46.721296+0000","last_fullsized":"2023-10-08T16:13:46.721296+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:29:19.761812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719727+0000","last_change":"2023-10-08T16:13:46.716500+0000","last_active":"2023-10-08T16:13:46.719727+0000","last_peered":"2023-10-08T16:13:46.719727+0000","last_clean":"2023-10-08T16:13:46.719727+0000","last_became_active":"2023-10-08T16:13:46.716179+0000","last_became_peered":"2023-10-08T16:13:46.716179+0000","last_unstale":"2023-10-08T16:13:46.719727+0000","last_undegraded":"2023-10-08T16:13:46.719727+0000","last_fullsized":"2023-10-08T16:13:46.719727+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:17.903574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720783+0000","last_change":"2023-10-08T16:13:46.717103+0000","last_active":"2023-10-08T16:13:46.720783+0000","last_peered":"2023-10-08T16:13:46.720783+0000","last_clean":"2023-10-08T16:13:46.720783+0000","last_became_active":"2023-10-08T16:13:46.716751+0000","last_became_peered":"2023-10-08T16:13:46.716751+0000","last_unstale":"2023-10-08T16:13:46.720783+0000","last_undegraded":"2023-10-08T16:13:46.720783+0000","last_fullsized":"2023-10-08T16:13:46.720783+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:50:50.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711930+0000","last_change":"2023-10-08T16:13:46.708707+0000","last_active":"2023-10-08T16:13:46.711930+0000","last_peered":"2023-10-08T16:13:46.711930+0000","last_clean":"2023-10-08T16:13:46.711930+0000","last_became_active":"2023-10-08T16:13:46.707058+0000","last_became_peered":"2023-10-08T16:13:46.707058+0000","last_unstale":"2023-10-08T16:13:46.711930+0000","last_undegraded":"2023-10-08T16:13:46.711930+0000","last_fullsized":"2023-10-08T16:13:46.711930+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:51.276191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.722608+0000","last_change":"2023-10-08T16:13:46.720029+0000","last_active":"2023-10-08T16:13:46.722608+0000","last_peered":"2023-10-08T16:13:46.722608+0000","last_clean":"2023-10-08T16:13:46.722608+0000","last_became_active":"2023-10-08T16:13:46.719722+0000","last_became_peered":"2023-10-08T16:13:46.719722+0000","last_unstale":"2023-10-08T16:13:46.722608+0000","last_undegraded":"2023-10-08T16:13:46.722608+0000","last_fullsized":"2023-10-08T16:13:46.722608+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:49.593058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720236+0000","last_change":"2023-10-08T16:13:46.712522+0000","last_active":"2023-10-08T16:13:46.720236+0000","last_peered":"2023-10-08T16:13:46.720236+0000","last_clean":"2023-10-08T16:13:46.720236+0000","last_became_active":"2023-10-08T16:13:46.710314+0000","last_became_peered":"2023-10-08T16:13:46.710314+0000","last_unstale":"2023-10-08T16:13:46.720236+0000","last_undegraded":"2023-10-08T16:13:46.720236+0000","last_fullsized":"2023-10-08T16:13:46.720236+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:29.865417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720792+0000","last_change":"2023-10-08T16:13:46.719679+0000","last_active":"2023-10-08T16:13:46.720792+0000","last_peered":"2023-10-08T16:13:46.720792+0000","last_clean":"2023-10-08T16:13:46.720792+0000","last_became_active":"2023-10-08T16:13:46.719142+0000","last_became_peered":"2023-10-08T16:13:46.719142+0000","last_unstale":"2023-10-08T16:13:46.720792+0000","last_undegraded":"2023-10-08T16:13:46.720792+0000","last_fullsized":"2023-10-08T16:13:46.720792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:39.994481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719652+0000","last_change":"2023-10-08T16:13:46.717328+0000","last_active":"2023-10-08T16:13:46.719652+0000","last_peered":"2023-10-08T16:13:46.719652+0000","last_clean":"2023-10-08T16:13:46.719652+0000","last_became_active":"2023-10-08T16:13:46.716219+0000","last_became_peered":"2023-10-08T16:13:46.716219+0000","last_unstale":"2023-10-08T16:13:46.719652+0000","last_undegraded":"2023-10-08T16:13:46.719652+0000","last_fullsized":"2023-10-08T16:13:46.719652+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:08:52.774909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719427+0000","last_change":"2023-10-08T16:13:46.712537+0000","last_active":"2023-10-08T16:13:46.719427+0000","last_peered":"2023-10-08T16:13:46.719427+0000","last_clean":"2023-10-08T16:13:46.719427+0000","last_became_active":"2023-10-08T16:13:46.712160+0000","last_became_peered":"2023-10-08T16:13:46.712160+0000","last_unstale":"2023-10-08T16:13:46.719427+0000","last_undegraded":"2023-10-08T16:13:46.719427+0000","last_fullsized":"2023-10-08T16:13:46.719427+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:12:14.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728143+0000","last_change":"2023-10-08T16:13:46.725629+0000","last_active":"2023-10-08T16:13:46.728143+0000","last_peered":"2023-10-08T16:13:46.728143+0000","last_clean":"2023-10-08T16:13:46.728143+0000","last_became_active":"2023-10-08T16:13:46.725221+0000","last_became_peered":"2023-10-08T16:13:46.725221+0000","last_unstale":"2023-10-08T16:13:46.728143+0000","last_undegraded":"2023-10-08T16:13:46.728143+0000","last_fullsized":"2023-10-08T16:13:46.728143+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:36:56.467137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.712259+0000","last_change":"2023-10-08T16:13:46.709459+0000","last_active":"2023-10-08T16:13:46.712259+0000","last_peered":"2023-10-08T16:13:46.712259+0000","last_clean":"2023-10-08T16:13:46.712259+0000","last_became_active":"2023-10-08T16:13:46.709245+0000","last_became_peered":"2023-10-08T16:13:46.709245+0000","last_unstale":"2023-10-08T16:13:46.712259+0000","last_undegraded":"2023-10-08T16:13:46.712259+0000","last_fullsized":"2023-10-08T16:13:46.712259+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:28:03.814244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717086+0000","last_change":"2023-10-08T16:13:46.714125+0000","last_active":"2023-10-08T16:13:46.717086+0000","last_peered":"2023-10-08T16:13:46.717086+0000","last_clean":"2023-10-08T16:13:46.717086+0000","last_became_active":"2023-10-08T16:13:46.713869+0000","last_became_peered":"2023-10-08T16:13:46.713869+0000","last_unstale":"2023-10-08T16:13:46.717086+0000","last_undegraded":"2023-10-08T16:13:46.717086+0000","last_fullsized":"2023-10-08T16:13:46.717086+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:56.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"200'4","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720227+0000","last_change":"2023-10-08T16:13:45.712063+0000","last_active":"2023-10-08T16:13:46.720227+0000","last_peered":"2023-10-08T16:13:46.720227+0000","last_clean":"2023-10-08T16:13:46.720227+0000","last_became_active":"2023-10-08T16:13:45.711548+0000","last_became_peered":"2023-10-08T16:13:45.711548+0000","last_unstale":"2023-10-08T16:13:46.720227+0000","last_undegraded":"2023-10-08T16:13:46.720227+0000","last_fullsized":"2023-10-08T16:13:46.720227+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:08.377590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721524+0000","last_change":"2023-10-08T16:13:46.715069+0000","last_active":"2023-10-08T16:13:46.721524+0000","last_peered":"2023-10-08T16:13:46.721524+0000","last_clean":"2023-10-08T16:13:46.721524+0000","last_became_active":"2023-10-08T16:13:46.714767+0000","last_became_peered":"2023-10-08T16:13:46.714767+0000","last_unstale":"2023-10-08T16:13:46.721524+0000","last_undegraded":"2023-10-08T16:13:46.721524+0000","last_fullsized":"2023-10-08T16:13:46.721524+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:17.329894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720655+0000","last_change":"2023-10-08T16:13:45.711057+0000","last_active":"2023-10-08T16:13:46.720655+0000","last_peered":"2023-10-08T16:13:46.720655+0000","last_clean":"2023-10-08T16:13:46.720655+0000","last_became_active":"2023-10-08T16:13:45.710601+0000","last_became_peered":"2023-10-08T16:13:45.710601+0000","last_unstale":"2023-10-08T16:13:46.720655+0000","last_undegraded":"2023-10-08T16:13:46.720655+0000","last_fullsized":"2023-10-08T16:13:46.720655+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:26.860970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721544+0000","last_change":"2023-10-08T16:13:46.715207+0000","last_active":"2023-10-08T16:13:46.721544+0000","last_peered":"2023-10-08T16:13:46.721544+0000","last_clean":"2023-10-08T16:13:46.721544+0000","last_became_active":"2023-10-08T16:13:46.714963+0000","last_became_peered":"2023-10-08T16:13:46.714963+0000","last_unstale":"2023-10-08T16:13:46.721544+0000","last_undegraded":"2023-10-08T16:13:46.721544+0000","last_fullsized":"2023-10-08T16:13:46.721544+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:07.682520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721133+0000","last_change":"2023-10-08T16:13:45.708676+0000","last_active":"2023-10-08T16:13:46.721133+0000","last_peered":"2023-10-08T16:13:46.721133+0000","last_clean":"2023-10-08T16:13:46.721133+0000","last_became_active":"2023-10-08T16:13:45.708415+0000","last_became_peered":"2023-10-08T16:13:45.708415+0000","last_unstale":"2023-10-08T16:13:46.721133+0000","last_undegraded":"2023-10-08T16:13:46.721133+0000","last_fullsized":"2023-10-08T16:13:46.721133+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:56.692151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720569+0000","last_change":"2023-10-08T16:13:46.716597+0000","last_active":"2023-10-08T16:13:46.720569+0000","last_peered":"2023-10-08T16:13:46.720569+0000","last_clean":"2023-10-08T16:13:46.720569+0000","last_became_active":"2023-10-08T16:13:46.716315+0000","last_became_peered":"2023-10-08T16:13:46.716315+0000","last_unstale":"2023-10-08T16:13:46.720569+0000","last_undegraded":"2023-10-08T16:13:46.720569+0000","last_fullsized":"2023-10-08T16:13:46.720569+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:23.158046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719132+0000","last_change":"2023-10-08T16:13:45.704516+0000","last_active":"2023-10-08T16:13:46.719132+0000","last_peered":"2023-10-08T16:13:46.719132+0000","last_clean":"2023-10-08T16:13:46.719132+0000","last_became_active":"2023-10-08T16:13:45.704300+0000","last_became_peered":"2023-10-08T16:13:45.704300+0000","last_unstale":"2023-10-08T16:13:46.719132+0000","last_undegraded":"2023-10-08T16:13:46.719132+0000","last_fullsized":"2023-10-08T16:13:46.719132+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:11.998472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721053+0000","last_change":"2023-10-08T16:13:46.718188+0000","last_active":"2023-10-08T16:13:46.721053+0000","last_peered":"2023-10-08T16:13:46.721053+0000","last_clean":"2023-10-08T16:13:46.721053+0000","last_became_active":"2023-10-08T16:13:46.717108+0000","last_became_peered":"2023-10-08T16:13:46.717108+0000","last_unstale":"2023-10-08T16:13:46.721053+0000","last_undegraded":"2023-10-08T16:13:46.721053+0000","last_fullsized":"2023-10-08T16:13:46.721053+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:46.340315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717253+0000","last_change":"2023-10-08T16:13:45.714455+0000","last_active":"2023-10-08T16:13:46.717253+0000","last_peered":"2023-10-08T16:13:46.717253+0000","last_clean":"2023-10-08T16:13:46.717253+0000","last_became_active":"2023-10-08T16:13:45.714223+0000","last_became_peered":"2023-10-08T16:13:45.714223+0000","last_unstale":"2023-10-08T16:13:46.717253+0000","last_undegraded":"2023-10-08T16:13:46.717253+0000","last_fullsized":"2023-10-08T16:13:46.717253+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:12:13.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720729+0000","last_change":"2023-10-08T16:13:46.715968+0000","last_active":"2023-10-08T16:13:46.720729+0000","last_peered":"2023-10-08T16:13:46.720729+0000","last_clean":"2023-10-08T16:13:46.720729+0000","last_became_active":"2023-10-08T16:13:46.715676+0000","last_became_peered":"2023-10-08T16:13:46.715676+0000","last_unstale":"2023-10-08T16:13:46.720729+0000","last_undegraded":"2023-10-08T16:13:46.720729+0000","last_fullsized":"2023-10-08T16:13:46.720729+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:39:04.379764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726527+0000","last_change":"2023-10-08T16:13:45.712951+0000","last_active":"2023-10-08T16:13:46.726527+0000","last_peered":"2023-10-08T16:13:46.726527+0000","last_clean":"2023-10-08T16:13:46.726527+0000","last_became_active":"2023-10-08T16:13:45.712689+0000","last_became_peered":"2023-10-08T16:13:45.712689+0000","last_unstale":"2023-10-08T16:13:46.726527+0000","last_undegraded":"2023-10-08T16:13:46.726527+0000","last_fullsized":"2023-10-08T16:13:46.726527+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:31.089140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720868+0000","last_change":"2023-10-08T16:13:46.718873+0000","last_active":"2023-10-08T16:13:46.720868+0000","last_peered":"2023-10-08T16:13:46.720868+0000","last_clean":"2023-10-08T16:13:46.720868+0000","last_became_active":"2023-10-08T16:13:46.718361+0000","last_became_peered":"2023-10-08T16:13:46.718361+0000","last_unstale":"2023-10-08T16:13:46.720868+0000","last_undegraded":"2023-10-08T16:13:46.720868+0000","last_fullsized":"2023-10-08T16:13:46.720868+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:42.257442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700490+0000","last_change":"2023-10-08T16:13:45.710004+0000","last_active":"2023-10-08T16:13:46.700490+0000","last_peered":"2023-10-08T16:13:46.700490+0000","last_clean":"2023-10-08T16:13:46.700490+0000","last_became_active":"2023-10-08T16:13:45.709690+0000","last_became_peered":"2023-10-08T16:13:45.709690+0000","last_unstale":"2023-10-08T16:13:46.700490+0000","last_undegraded":"2023-10-08T16:13:46.700490+0000","last_fullsized":"2023-10-08T16:13:46.700490+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:53.701102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726362+0000","last_change":"2023-10-08T16:13:46.724299+0000","last_active":"2023-10-08T16:13:46.726362+0000","last_peered":"2023-10-08T16:13:46.726362+0000","last_clean":"2023-10-08T16:13:46.726362+0000","last_became_active":"2023-10-08T16:13:46.723274+0000","last_became_peered":"2023-10-08T16:13:46.723274+0000","last_unstale":"2023-10-08T16:13:46.726362+0000","last_undegraded":"2023-10-08T16:13:46.726362+0000","last_fullsized":"2023-10-08T16:13:46.726362+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:45.904403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.696743+0000","last_change":"2023-10-08T16:13:45.710836+0000","last_active":"2023-10-08T16:13:46.696743+0000","last_peered":"2023-10-08T16:13:46.696743+0000","last_clean":"2023-10-08T16:13:46.696743+0000","last_became_active":"2023-10-08T16:13:45.710547+0000","last_became_peered":"2023-10-08T16:13:45.710547+0000","last_unstale":"2023-10-08T16:13:46.696743+0000","last_undegraded":"2023-10-08T16:13:46.696743+0000","last_fullsized":"2023-10-08T16:13:46.696743+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:30:44.465783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726132+0000","last_change":"2023-10-08T16:13:46.722962+0000","last_active":"2023-10-08T16:13:46.726132+0000","last_peered":"2023-10-08T16:13:46.726132+0000","last_clean":"2023-10-08T16:13:46.726132+0000","last_became_active":"2023-10-08T16:13:46.722743+0000","last_became_peered":"2023-10-08T16:13:46.722743+0000","last_unstale":"2023-10-08T16:13:46.726132+0000","last_undegraded":"2023-10-08T16:13:46.726132+0000","last_fullsized":"2023-10-08T16:13:46.726132+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:57.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700057+0000","last_change":"2023-10-08T16:13:45.710255+0000","last_active":"2023-10-08T16:13:46.700057+0000","last_peered":"2023-10-08T16:13:46.700057+0000","last_clean":"2023-10-08T16:13:46.700057+0000","last_became_active":"2023-10-08T16:13:45.710024+0000","last_became_peered":"2023-10-08T16:13:45.710024+0000","last_unstale":"2023-10-08T16:13:46.700057+0000","last_undegraded":"2023-10-08T16:13:46.700057+0000","last_fullsized":"2023-10-08T16:13:46.700057+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:01:34.963368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.715296+0000","last_change":"2023-10-08T16:13:46.711432+0000","last_active":"2023-10-08T16:13:46.715296+0000","last_peered":"2023-10-08T16:13:46.715296+0000","last_clean":"2023-10-08T16:13:46.715296+0000","last_became_active":"2023-10-08T16:13:46.711173+0000","last_became_peered":"2023-10-08T16:13:46.711173+0000","last_unstale":"2023-10-08T16:13:46.715296+0000","last_undegraded":"2023-10-08T16:13:46.715296+0000","last_fullsized":"2023-10-08T16:13:46.715296+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:25.699255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700269+0000","last_change":"2023-10-08T16:13:45.710114+0000","last_active":"2023-10-08T16:13:46.700269+0000","last_peered":"2023-10-08T16:13:46.700269+0000","last_clean":"2023-10-08T16:13:46.700269+0000","last_became_active":"2023-10-08T16:13:45.709779+0000","last_became_peered":"2023-10-08T16:13:45.709779+0000","last_unstale":"2023-10-08T16:13:46.700269+0000","last_undegraded":"2023-10-08T16:13:46.700269+0000","last_fullsized":"2023-10-08T16:13:46.700269+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:06.536547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719556+0000","last_change":"2023-10-08T16:13:46.713366+0000","last_active":"2023-10-08T16:13:46.719556+0000","last_peered":"2023-10-08T16:13:46.719556+0000","last_clean":"2023-10-08T16:13:46.719556+0000","last_became_active":"2023-10-08T16:13:46.713107+0000","last_became_peered":"2023-10-08T16:13:46.713107+0000","last_unstale":"2023-10-08T16:13:46.719556+0000","last_undegraded":"2023-10-08T16:13:46.719556+0000","last_fullsized":"2023-10-08T16:13:46.719556+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:16:27.649039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"200'3","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726213+0000","last_change":"2023-10-08T16:13:45.710943+0000","last_active":"2023-10-08T16:13:46.726213+0000","last_peered":"2023-10-08T16:13:46.726213+0000","last_clean":"2023-10-08T16:13:46.726213+0000","last_became_active":"2023-10-08T16:13:45.710658+0000","last_became_peered":"2023-10-08T16:13:45.710658+0000","last_unstale":"2023-10-08T16:13:46.726213+0000","last_undegraded":"2023-10-08T16:13:46.726213+0000","last_fullsized":"2023-10-08T16:13:46.726213+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:01.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711914+0000","last_change":"2023-10-08T16:13:46.708242+0000","last_active":"2023-10-08T16:13:46.711914+0000","last_peered":"2023-10-08T16:13:46.711914+0000","last_clean":"2023-10-08T16:13:46.711914+0000","last_became_active":"2023-10-08T16:13:46.707988+0000","last_became_peered":"2023-10-08T16:13:46.707988+0000","last_unstale":"2023-10-08T16:13:46.711914+0000","last_undegraded":"2023-10-08T16:13:46.711914+0000","last_fullsized":"2023-10-08T16:13:46.711914+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:05.366328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"200'6","reported_seq":16,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.723445+0000","last_change":"2023-10-08T16:13:45.715559+0000","last_active":"2023-10-08T16:13:46.723445+0000","last_peered":"2023-10-08T16:13:46.723445+0000","last_clean":"2023-10-08T16:13:46.723445+0000","last_became_active":"2023-10-08T16:13:45.715312+0000","last_became_peered":"2023-10-08T16:13:45.715312+0000","last_unstale":"2023-10-08T16:13:46.723445+0000","last_undegraded":"2023-10-08T16:13:46.723445+0000","last_fullsized":"2023-10-08T16:13:46.723445+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:40.229883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718209+0000","last_change":"2023-10-08T16:13:46.714266+0000","last_active":"2023-10-08T16:13:46.718209+0000","last_peered":"2023-10-08T16:13:46.718209+0000","last_clean":"2023-10-08T16:13:46.718209+0000","last_became_active":"2023-10-08T16:13:46.713993+0000","last_became_peered":"2023-10-08T16:13:46.713993+0000","last_unstale":"2023-10-08T16:13:46.718209+0000","last_undegraded":"2023-10-08T16:13:46.718209+0000","last_fullsized":"2023-10-08T16:13:46.718209+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:18.258158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719261+0000","last_change":"2023-10-08T16:13:45.705570+0000","last_active":"2023-10-08T16:13:46.719261+0000","last_peered":"2023-10-08T16:13:46.719261+0000","last_clean":"2023-10-08T16:13:46.719261+0000","last_became_active":"2023-10-08T16:13:45.705322+0000","last_became_peered":"2023-10-08T16:13:45.705322+0000","last_unstale":"2023-10-08T16:13:46.719261+0000","last_undegraded":"2023-10-08T16:13:46.719261+0000","last_fullsized":"2023-10-08T16:13:46.719261+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:56.447630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719607+0000","last_change":"2023-10-08T16:13:46.717364+0000","last_active":"2023-10-08T16:13:46.719607+0000","last_peered":"2023-10-08T16:13:46.719607+0000","last_clean":"2023-10-08T16:13:46.719607+0000","last_became_active":"2023-10-08T16:13:46.717025+0000","last_became_peered":"2023-10-08T16:13:46.717025+0000","last_unstale":"2023-10-08T16:13:46.719607+0000","last_undegraded":"2023-10-08T16:13:46.719607+0000","last_fullsized":"2023-10-08T16:13:46.719607+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:46.143513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700655+0000","last_change":"2023-10-08T16:13:45.709755+0000","last_active":"2023-10-08T16:13:46.700655+0000","last_peered":"2023-10-08T16:13:46.700655+0000","last_clean":"2023-10-08T16:13:46.700655+0000","last_became_active":"2023-10-08T16:13:45.709498+0000","last_became_peered":"2023-10-08T16:13:45.709498+0000","last_unstale":"2023-10-08T16:13:46.700655+0000","last_undegraded":"2023-10-08T16:13:46.700655+0000","last_fullsized":"2023-10-08T16:13:46.700655+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:00.914379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.707810+0000","last_change":"2023-10-08T16:13:46.707449+0000","last_active":"2023-10-08T16:13:46.707810+0000","last_peered":"2023-10-08T16:13:46.707810+0000","last_clean":"2023-10-08T16:13:46.707810+0000","last_became_active":"2023-10-08T16:13:46.707255+0000","last_became_peered":"2023-10-08T16:13:46.707255+0000","last_unstale":"2023-10-08T16:13:46.707810+0000","last_undegraded":"2023-10-08T16:13:46.707810+0000","last_fullsized":"2023-10-08T16:13:46.707810+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.699981+0000","last_change":"2023-10-08T16:13:45.711503+0000","last_active":"2023-10-08T16:13:46.699981+0000","last_peered":"2023-10-08T16:13:46.699981+0000","last_clean":"2023-10-08T16:13:46.699981+0000","last_became_active":"2023-10-08T16:13:45.711261+0000","last_became_peered":"2023-10-08T16:13:45.711261+0000","last_unstale":"2023-10-08T16:13:46.699981+0000","last_undegraded":"2023-10-08T16:13:46.699981+0000","last_fullsized":"2023-10-08T16:13:46.699981+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:29.269360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718431+0000","last_change":"2023-10-08T16:13:46.714853+0000","last_active":"2023-10-08T16:13:46.718431+0000","last_peered":"2023-10-08T16:13:46.718431+0000","last_clean":"2023-10-08T16:13:46.718431+0000","last_became_active":"2023-10-08T16:13:46.714601+0000","last_became_peered":"2023-10-08T16:13:46.714601+0000","last_unstale":"2023-10-08T16:13:46.718431+0000","last_undegraded":"2023-10-08T16:13:46.718431+0000","last_fullsized":"2023-10-08T16:13:46.718431+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:33:48.165199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"200'8","reported_seq":18,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.727675+0000","last_change":"2023-10-08T16:13:45.713037+0000","last_active":"2023-10-08T16:13:46.727675+0000","last_peered":"2023-10-08T16:13:46.727675+0000","last_clean":"2023-10-08T16:13:46.727675+0000","last_became_active":"2023-10-08T16:13:45.707929+0000","last_became_peered":"2023-10-08T16:13:45.707929+0000","last_unstale":"2023-10-08T16:13:46.727675+0000","last_undegraded":"2023-10-08T16:13:46.727675+0000","last_fullsized":"2023-10-08T16:13:46.727675+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:29.315045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718962+0000","last_change":"2023-10-08T16:13:46.714344+0000","last_active":"2023-10-08T16:13:46.718962+0000","last_peered":"2023-10-08T16:13:46.718962+0000","last_clean":"2023-10-08T16:13:46.718962+0000","last_became_active":"2023-10-08T16:13:46.713700+0000","last_became_peered":"2023-10-08T16:13:46.713700+0000","last_unstale":"2023-10-08T16:13:46.718962+0000","last_undegraded":"2023-10-08T16:13:46.718962+0000","last_fullsized":"2023-10-08T16:13:46.718962+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:19.227974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721806+0000","last_change":"2023-10-08T16:13:45.711223+0000","last_active":"2023-10-08T16:13:46.721806+0000","last_peered":"2023-10-08T16:13:46.721806+0000","last_clean":"2023-10-08T16:13:46.721806+0000","last_became_active":"2023-10-08T16:13:45.710228+0000","last_became_peered":"2023-10-08T16:13:45.710228+0000","last_unstale":"2023-10-08T16:13:46.721806+0000","last_undegraded":"2023-10-08T16:13:46.721806+0000","last_fullsized":"2023-10-08T16:13:46.721806+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:38.143990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719333+0000","last_change":"2023-10-08T16:13:46.715455+0000","last_active":"2023-10-08T16:13:46.719333+0000","last_peered":"2023-10-08T16:13:46.719333+0000","last_clean":"2023-10-08T16:13:46.719333+0000","last_became_active":"2023-10-08T16:13:46.715051+0000","last_became_peered":"2023-10-08T16:13:46.715051+0000","last_unstale":"2023-10-08T16:13:46.719333+0000","last_undegraded":"2023-10-08T16:13:46.719333+0000","last_fullsized":"2023-10-08T16:13:46.719333+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:56.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721494+0000","last_change":"2023-10-08T16:13:45.709874+0000","last_active":"2023-10-08T16:13:46.721494+0000","last_peered":"2023-10-08T16:13:46.721494+0000","last_clean":"2023-10-08T16:13:46.721494+0000","last_became_active":"2023-10-08T16:13:45.709603+0000","last_became_peered":"2023-10-08T16:13:45.709603+0000","last_unstale":"2023-10-08T16:13:46.721494+0000","last_undegraded":"2023-10-08T16:13:46.721494+0000","last_fullsized":"2023-10-08T16:13:46.721494+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:36.367531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726305+0000","last_change":"2023-10-08T16:13:46.723933+0000","last_active":"2023-10-08T16:13:46.726305+0000","last_peered":"2023-10-08T16:13:46.726305+0000","last_clean":"2023-10-08T16:13:46.726305+0000","last_became_active":"2023-10-08T16:13:46.723685+0000","last_became_peered":"2023-10-08T16:13:46.723685+0000","last_unstale":"2023-10-08T16:13:46.726305+0000","last_undegraded":"2023-10-08T16:13:46.726305+0000","last_fullsized":"2023-10-08T16:13:46.726305+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:09.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721286+0000","last_change":"2023-10-08T16:13:45.710450+0000","last_active":"2023-10-08T16:13:46.721286+0000","last_peered":"2023-10-08T16:13:46.721286+0000","last_clean":"2023-10-08T16:13:46.721286+0000","last_became_active":"2023-10-08T16:13:45.709126+0000","last_became_peered":"2023-10-08T16:13:45.709126+0000","last_unstale":"2023-10-08T16:13:46.721286+0000","last_undegraded":"2023-10-08T16:13:46.721286+0000","last_fullsized":"2023-10-08T16:13:46.721286+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:16.324496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719951+0000","last_change":"2023-10-08T16:13:46.717609+0000","last_active":"2023-10-08T16:13:46.719951+0000","last_peered":"2023-10-08T16:13:46.719951+0000","last_clean":"2023-10-08T16:13:46.719951+0000","last_became_active":"2023-10-08T16:13:46.717267+0000","last_became_peered":"2023-10-08T16:13:46.717267+0000","last_unstale":"2023-10-08T16:13:46.719951+0000","last_undegraded":"2023-10-08T16:13:46.719951+0000","last_fullsized":"2023-10-08T16:13:46.719951+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:39.149387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697992+0000","last_change":"2023-10-08T16:13:45.710038+0000","last_active":"2023-10-08T16:13:46.697992+0000","last_peered":"2023-10-08T16:13:46.697992+0000","last_clean":"2023-10-08T16:13:46.697992+0000","last_became_active":"2023-10-08T16:13:45.709768+0000","last_became_peered":"2023-10-08T16:13:45.709768+0000","last_unstale":"2023-10-08T16:13:46.697992+0000","last_undegraded":"2023-10-08T16:13:46.697992+0000","last_fullsized":"2023-10-08T16:13:46.697992+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:08.247815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720593+0000","last_change":"2023-10-08T16:13:46.713653+0000","last_active":"2023-10-08T16:13:46.720593+0000","last_peered":"2023-10-08T16:13:46.720593+0000","last_clean":"2023-10-08T16:13:46.720593+0000","last_became_active":"2023-10-08T16:13:46.713363+0000","last_became_peered":"2023-10-08T16:13:46.713363+0000","last_unstale":"2023-10-08T16:13:46.720593+0000","last_undegraded":"2023-10-08T16:13:46.720593+0000","last_fullsized":"2023-10-08T16:13:46.720593+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:02:05.867238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728291+0000","last_change":"2023-10-08T16:13:45.713664+0000","last_active":"2023-10-08T16:13:46.728291+0000","last_peered":"2023-10-08T16:13:46.728291+0000","last_clean":"2023-10-08T16:13:46.728291+0000","last_became_active":"2023-10-08T16:13:45.713430+0000","last_became_peered":"2023-10-08T16:13:45.713430+0000","last_unstale":"2023-10-08T16:13:46.728291+0000","last_undegraded":"2023-10-08T16:13:46.728291+0000","last_fullsized":"2023-10-08T16:13:46.728291+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:44.438570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726972+0000","last_change":"2023-10-08T16:13:46.726090+0000","last_active":"2023-10-08T16:13:46.726972+0000","last_peered":"2023-10-08T16:13:46.726972+0000","last_clean":"2023-10-08T16:13:46.726972+0000","last_became_active":"2023-10-08T16:13:46.725767+0000","last_became_peered":"2023-10-08T16:13:46.725767+0000","last_unstale":"2023-10-08T16:13:46.726972+0000","last_undegraded":"2023-10-08T16:13:46.726972+0000","last_fullsized":"2023-10-08T16:13:46.726972+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:56.307779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.696854+0000","last_change":"2023-10-08T16:13:45.701528+0000","last_active":"2023-10-08T16:13:46.696854+0000","last_peered":"2023-10-08T16:13:46.696854+0000","last_clean":"2023-10-08T16:13:46.696854+0000","last_became_active":"2023-10-08T16:13:45.701281+0000","last_became_peered":"2023-10-08T16:13:45.701281+0000","last_unstale":"2023-10-08T16:13:46.696854+0000","last_undegraded":"2023-10-08T16:13:46.696854+0000","last_fullsized":"2023-10-08T16:13:46.696854+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:15:48.083233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726279+0000","last_change":"2023-10-08T16:13:46.723692+0000","last_active":"2023-10-08T16:13:46.726279+0000","last_peered":"2023-10-08T16:13:46.726279+0000","last_clean":"2023-10-08T16:13:46.726279+0000","last_became_active":"2023-10-08T16:13:46.722805+0000","last_became_peered":"2023-10-08T16:13:46.722805+0000","last_unstale":"2023-10-08T16:13:46.726279+0000","last_undegraded":"2023-10-08T16:13:46.726279+0000","last_fullsized":"2023-10-08T16:13:46.726279+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:37.815837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.709087+0000","last_change":"2023-10-08T16:13:45.716060+0000","last_active":"2023-10-08T16:13:46.709087+0000","last_peered":"2023-10-08T16:13:46.709087+0000","last_clean":"2023-10-08T16:13:46.709087+0000","last_became_active":"2023-10-08T16:13:45.715810+0000","last_became_peered":"2023-10-08T16:13:45.715810+0000","last_unstale":"2023-10-08T16:13:46.709087+0000","last_undegraded":"2023-10-08T16:13:46.709087+0000","last_fullsized":"2023-10-08T16:13:46.709087+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:32.344606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.712491+0000","last_change":"2023-10-08T16:13:46.708479+0000","last_active":"2023-10-08T16:13:46.712491+0000","last_peered":"2023-10-08T16:13:46.712491+0000","last_clean":"2023-10-08T16:13:46.712491+0000","last_became_active":"2023-10-08T16:13:46.708142+0000","last_became_peered":"2023-10-08T16:13:46.708142+0000","last_unstale":"2023-10-08T16:13:46.712491+0000","last_undegraded":"2023-10-08T16:13:46.712491+0000","last_fullsized":"2023-10-08T16:13:46.712491+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:04.597091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700628+0000","last_change":"2023-10-08T16:13:45.713097+0000","last_active":"2023-10-08T16:13:46.700628+0000","last_peered":"2023-10-08T16:13:46.700628+0000","last_clean":"2023-10-08T16:13:46.700628+0000","last_became_active":"2023-10-08T16:13:45.712834+0000","last_became_peered":"2023-10-08T16:13:45.712834+0000","last_unstale":"2023-10-08T16:13:46.700628+0000","last_undegraded":"2023-10-08T16:13:46.700628+0000","last_fullsized":"2023-10-08T16:13:46.700628+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720823+0000","last_change":"2023-10-08T16:13:46.717904+0000","last_active":"2023-10-08T16:13:46.720823+0000","last_peered":"2023-10-08T16:13:46.720823+0000","last_clean":"2023-10-08T16:13:46.720823+0000","last_became_active":"2023-10-08T16:13:46.717618+0000","last_became_peered":"2023-10-08T16:13:46.717618+0000","last_unstale":"2023-10-08T16:13:46.720823+0000","last_undegraded":"2023-10-08T16:13:46.720823+0000","last_fullsized":"2023-10-08T16:13:46.720823+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:33.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.724135+0000","last_change":"2023-10-08T16:13:45.712846+0000","last_active":"2023-10-08T16:13:46.724135+0000","last_peered":"2023-10-08T16:13:46.724135+0000","last_clean":"2023-10-08T16:13:46.724135+0000","last_became_active":"2023-10-08T16:13:45.712606+0000","last_became_peered":"2023-10-08T16:13:45.712606+0000","last_unstale":"2023-10-08T16:13:46.724135+0000","last_undegraded":"2023-10-08T16:13:46.724135+0000","last_fullsized":"2023-10-08T16:13:46.724135+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:28.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726956+0000","last_change":"2023-10-08T16:13:46.726347+0000","last_active":"2023-10-08T16:13:46.726956+0000","last_peered":"2023-10-08T16:13:46.726956+0000","last_clean":"2023-10-08T16:13:46.726956+0000","last_became_active":"2023-10-08T16:13:46.725786+0000","last_became_peered":"2023-10-08T16:13:46.725786+0000","last_unstale":"2023-10-08T16:13:46.726956+0000","last_undegraded":"2023-10-08T16:13:46.726956+0000","last_fullsized":"2023-10-08T16:13:46.726956+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:23.549077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.698338+0000","last_change":"2023-10-08T16:13:45.708520+0000","last_active":"2023-10-08T16:13:46.698338+0000","last_peered":"2023-10-08T16:13:46.698338+0000","last_clean":"2023-10-08T16:13:46.698338+0000","last_became_active":"2023-10-08T16:13:45.708294+0000","last_became_peered":"2023-10-08T16:13:45.708294+0000","last_unstale":"2023-10-08T16:13:46.698338+0000","last_undegraded":"2023-10-08T16:13:46.698338+0000","last_fullsized":"2023-10-08T16:13:46.698338+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:31.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.713646+0000","last_change":"2023-10-08T16:13:46.706525+0000","last_active":"2023-10-08T16:13:46.713646+0000","last_peered":"2023-10-08T16:13:46.713646+0000","last_clean":"2023-10-08T16:13:46.713646+0000","last_became_active":"2023-10-08T16:13:46.706242+0000","last_became_peered":"2023-10-08T16:13:46.706242+0000","last_unstale":"2023-10-08T16:13:46.713646+0000","last_undegraded":"2023-10-08T16:13:46.713646+0000","last_fullsized":"2023-10-08T16:13:46.713646+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:29.375190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.701210+0000","last_change":"2023-10-08T16:13:45.714624+0000","last_active":"2023-10-08T16:13:46.701210+0000","last_peered":"2023-10-08T16:13:46.701210+0000","last_clean":"2023-10-08T16:13:46.701210+0000","last_became_active":"2023-10-08T16:13:45.714387+0000","last_became_peered":"2023-10-08T16:13:45.714387+0000","last_unstale":"2023-10-08T16:13:46.701210+0000","last_undegraded":"2023-10-08T16:13:46.701210+0000","last_fullsized":"2023-10-08T16:13:46.701210+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:06.677122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719397+0000","last_change":"2023-10-08T16:13:46.713337+0000","last_active":"2023-10-08T16:13:46.719397+0000","last_peered":"2023-10-08T16:13:46.719397+0000","last_clean":"2023-10-08T16:13:46.719397+0000","last_became_active":"2023-10-08T16:13:46.713033+0000","last_became_peered":"2023-10-08T16:13:46.713033+0000","last_unstale":"2023-10-08T16:13:46.719397+0000","last_undegraded":"2023-10-08T16:13:46.719397+0000","last_fullsized":"2023-10-08T16:13:46.719397+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:32.094537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.699889+0000","last_change":"2023-10-08T16:13:45.708389+0000","last_active":"2023-10-08T16:13:46.699889+0000","last_peered":"2023-10-08T16:13:46.699889+0000","last_clean":"2023-10-08T16:13:46.699889+0000","last_became_active":"2023-10-08T16:13:45.708121+0000","last_became_peered":"2023-10-08T16:13:45.708121+0000","last_unstale":"2023-10-08T16:13:46.699889+0000","last_undegraded":"2023-10-08T16:13:46.699889+0000","last_fullsized":"2023-10-08T16:13:46.699889+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:53.427625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.722988+0000","last_change":"2023-10-08T16:13:46.705169+0000","last_active":"2023-10-08T16:13:46.722988+0000","last_peered":"2023-10-08T16:13:46.722988+0000","last_clean":"2023-10-08T16:13:46.722988+0000","last_became_active":"2023-10-08T16:13:46.704909+0000","last_became_peered":"2023-10-08T16:13:46.704909+0000","last_unstale":"2023-10-08T16:13:46.722988+0000","last_undegraded":"2023-10-08T16:13:46.722988+0000","last_fullsized":"2023-10-08T16:13:46.722988+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:50:50.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697967+0000","last_change":"2023-10-08T16:13:45.709209+0000","last_active":"2023-10-08T16:13:46.697967+0000","last_peered":"2023-10-08T16:13:46.697967+0000","last_clean":"2023-10-08T16:13:46.697967+0000","last_became_active":"2023-10-08T16:13:45.708951+0000","last_became_peered":"2023-10-08T16:13:45.708951+0000","last_unstale":"2023-10-08T16:13:46.697967+0000","last_undegraded":"2023-10-08T16:13:46.697967+0000","last_fullsized":"2023-10-08T16:13:46.697967+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:50.264735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728161+0000","last_change":"2023-10-08T16:13:46.726205+0000","last_active":"2023-10-08T16:13:46.728161+0000","last_peered":"2023-10-08T16:13:46.728161+0000","last_clean":"2023-10-08T16:13:46.728161+0000","last_became_active":"2023-10-08T16:13:46.725905+0000","last_became_peered":"2023-10-08T16:13:46.725905+0000","last_unstale":"2023-10-08T16:13:46.728161+0000","last_undegraded":"2023-10-08T16:13:46.728161+0000","last_fullsized":"2023-10-08T16:13:46.728161+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:29.861214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697743+0000","last_change":"2023-10-08T16:13:45.714988+0000","last_active":"2023-10-08T16:13:46.697743+0000","last_peered":"2023-10-08T16:13:46.697743+0000","last_clean":"2023-10-08T16:13:46.697743+0000","last_became_active":"2023-10-08T16:13:45.714625+0000","last_became_peered":"2023-10-08T16:13:45.714625+0000","last_unstale":"2023-10-08T16:13:46.697743+0000","last_undegraded":"2023-10-08T16:13:46.697743+0000","last_fullsized":"2023-10-08T16:13:46.697743+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:51.253894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720409+0000","last_change":"2023-10-08T16:13:46.716158+0000","last_active":"2023-10-08T16:13:46.720409+0000","last_peered":"2023-10-08T16:13:46.720409+0000","last_clean":"2023-10-08T16:13:46.720409+0000","last_became_active":"2023-10-08T16:13:46.715914+0000","last_became_peered":"2023-10-08T16:13:46.715914+0000","last_unstale":"2023-10-08T16:13:46.720409+0000","last_undegraded":"2023-10-08T16:13:46.720409+0000","last_fullsized":"2023-10-08T16:13:46.720409+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:05.622697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.703750+0000","last_change":"2023-10-08T16:13:45.708973+0000","last_active":"2023-10-08T16:13:46.703750+0000","last_peered":"2023-10-08T16:13:46.703750+0000","last_clean":"2023-10-08T16:13:46.703750+0000","last_became_active":"2023-10-08T16:13:45.708721+0000","last_became_peered":"2023-10-08T16:13:45.708721+0000","last_unstale":"2023-10-08T16:13:46.703750+0000","last_undegraded":"2023-10-08T16:13:46.703750+0000","last_fullsized":"2023-10-08T16:13:46.703750+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:45.743779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717432+0000","last_change":"2023-10-08T16:13:46.712153+0000","last_active":"2023-10-08T16:13:46.717432+0000","last_peered":"2023-10-08T16:13:46.717432+0000","last_clean":"2023-10-08T16:13:46.717432+0000","last_became_active":"2023-10-08T16:13:46.710495+0000","last_became_peered":"2023-10-08T16:13:46.710495+0000","last_unstale":"2023-10-08T16:13:46.717432+0000","last_undegraded":"2023-10-08T16:13:46.717432+0000","last_fullsized":"2023-10-08T16:13:46.717432+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:05.366328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697844+0000","last_change":"2023-10-08T16:13:45.709550+0000","last_active":"2023-10-08T16:13:46.697844+0000","last_peered":"2023-10-08T16:13:46.697844+0000","last_clean":"2023-10-08T16:13:46.697844+0000","last_became_active":"2023-10-08T16:13:45.709297+0000","last_became_peered":"2023-10-08T16:13:45.709297+0000","last_unstale":"2023-10-08T16:13:46.697844+0000","last_undegraded":"2023-10-08T16:13:46.697844+0000","last_fullsized":"2023-10-08T16:13:46.697844+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:31:09.018500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726327+0000","last_change":"2023-10-08T16:13:46.724173+0000","last_active":"2023-10-08T16:13:46.726327+0000","last_peered":"2023-10-08T16:13:46.726327+0000","last_clean":"2023-10-08T16:13:46.726327+0000","last_became_active":"2023-10-08T16:13:46.722757+0000","last_became_peered":"2023-10-08T16:13:46.722757+0000","last_unstale":"2023-10-08T16:13:46.726327+0000","last_undegraded":"2023-10-08T16:13:46.726327+0000","last_fullsized":"2023-10-08T16:13:46.726327+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:02:45.349897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":316,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.703558+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:13:46.703558+0000","last_peered":"2023-10-08T16:13:46.703558+0000","last_clean":"2023-10-08T16:13:46.703558+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:13:46.703558+0000","last_undegraded":"2023-10-08T16:13:46.703558+0000","last_fullsized":"2023-10-08T16:13:46.703558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304468,"kb_used_data":3084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067372,"statfs":{"total":96636764160,"available":96324988928,"internally_reserved":0,"allocated":3158016,"data_stored":1741319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.69999999999999996},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.74099999999999999,"15min":8.0939999999999994},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29199999999999998},"max":{"1min":0.66200000000000003,"5min":0.82199999999999995,"15min":1.083},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49099999999999999,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":1.175},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.44}]},{"osd":3,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.753,"5min":1.677,"15min":1.677},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.751},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":8.3409999999999993},"last":0.496}]},{"osd":4,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.753,"15min":0.753},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72999999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.78000000000000003,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.57699999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.90600000000000003},"last":0.63100000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.496},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.68500000000000005,"5min":0.747,"15min":0.76700000000000002},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.495},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.96599999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.94199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.72499999999999998,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61499999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.82399999999999995,"15min":0.874},"last":0.37}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.48299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307984,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063856,"statfs":{"total":96636764160,"available":96321388544,"internally_reserved":0,"allocated":2564096,"data_stored":1150499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.311,"15min":0.31},"min":{"1min":0.192,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20300000000000001},"max":{"1min":0.42499999999999999,"5min":0.627,"15min":0.77500000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34000000000000002,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.38600000000000001,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.372,"15min":0.371},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.627,"15min":0.72399999999999998},"last":0.27000000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.378,"15min":0.38200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45300000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34499999999999997,"15min":0.35299999999999998},"min":{"1min":0.248,"5min":0.215,"15min":0.215},"max":{"1min":0.373,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":1.077},"last":0.441},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78300000000000003,"5min":1.696,"15min":1.696},"last":0.48299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":1.657,"15min":1.657},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.501}]},{"osd":6,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.57299999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72399999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60099999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76900000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.875,"15min":1.1839999999999999},"last":0.46000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308560,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063280,"statfs":{"total":96636764160,"available":96320798720,"internally_reserved":0,"allocated":3153920,"data_stored":1740813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.313,"15min":0.48899999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.41399999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.441},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.52000000000000002},"min":{"1min":0.27600000000000002,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.41499999999999998,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.373},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.219},"max":{"1min":0.501,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.377}]},{"osd":3,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.188},"max":{"1min":0.59499999999999997,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.55800000000000005},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.26900000000000002},"max":{"1min":0.42999999999999999,"5min":1.498,"15min":5.5970000000000004},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55300000000000005,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":1.1379999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":1.847,"15min":3.2890000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.63700000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.751},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77000000000000002,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":1.1699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1152487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.312,"15min":0.308},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.48199999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.312,"5min":0.32100000000000001,"15min":0.34200000000000003},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.189},"max":{"1min":0.439,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.247}]},{"osd":1,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.34599999999999997},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.30599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.34200000000000003},"min":{"1min":0.24399999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60599999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.28999999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.37}]},{"osd":4,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.76200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.254},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.89100000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56999999999999995,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":4.1029999999999998},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.44500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.871},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.47499999999999998,"5min":0.434,"15min":0.36299999999999999},"max":{"1min":0.88,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307980,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063860,"statfs":{"total":96636764160,"available":96321392640,"internally_reserved":0,"allocated":2560000,"data_stored":1150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.34399999999999997},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":0.44900000000000001,"5min":0.505,"15min":1.238},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.376},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.35299999999999998},"min":{"1min":0.254,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45800000000000002,"5min":0.47999999999999998,"15min":1.0269999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.39700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.499,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36399999999999999,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.248},"max":{"1min":0.47699999999999998,"5min":0.53900000000000003,"15min":1.1399999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.28299999999999997,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":1.327},"last":0.317}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.9119999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.099,"15min":1.099},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":1.27},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":1.879},"last":0.432}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.75700000000000001,"5min":0.81399999999999995,"15min":1.0620000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":1.825},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"max":{"1min":0.81100000000000005,"5min":1.0009999999999999,"15min":1.115},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303892,"kb_used_data":2508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067948,"statfs":{"total":96636764160,"available":96325578752,"internally_reserved":0,"allocated":2568192,"data_stored":1151911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.58999999999999997,"5min":0.68100000000000005,"15min":1.579},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51700000000000002,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.57399999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.63,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63500000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.51500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.74199999999999999,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.55600000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.57599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.83399999999999996},"last":0.69799999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.875,"15min":0.875},"last":0.747},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.89200000000000002},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68700000000000006,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.55900000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51700000000000002,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73499999999999999,"5min":0.84399999999999997,"15min":12.227},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.871,"15min":1.252},"last":0.46800000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.314,"15min":0.314},"max":{"1min":0.73899999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.88700000000000001,"15min":1.444},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.755,"15min":9.3170000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":0.879,"15min":3.8719999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.495,"15min":0.5},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.93799999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.377}]},{"osd":6,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.254,"15min":0.254},"max":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53400000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.83199999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.877,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.501}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:13:50.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:13:50.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.911+0000 7f970251a700 1 Processor -- start 2023-10-08T16:13:50.911 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.911+0000 7f970251a700 1 -- start start 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f970251a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 0x7f96fc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f970251a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96fc1597f0 0x7f96fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f970251a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96fc0b11d0 con 0x7f96fc15d230 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f970251a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc0b14b0 con 0x7f96fc1597f0 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f970251a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96fc0b1790 con 0x7f96fc159420 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96fc1597f0 0x7f96fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96fc1597f0 0x7f96fc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52342/0 (socket says 172.21.15.170:52342) 2023-10-08T16:13:50.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 -- 172.21.15.170:0/2544610750 learned_addr learned my addr 172.21.15.170:0/2544610750 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9701518700 1 --2- 172.21.15.170:0/2544610750 >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 0x7f96fc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 -- 172.21.15.170:0/2544610750 >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 msgr2=0x7f96fc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 --2- 172.21.15.170:0/2544610750 >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 0x7f96fc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 -- 172.21.15.170:0/2544610750 >> v1:172.21.15.191:6789/0 conn(0x7f96fc15d230 legacy=0x7f96fc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 -- 172.21.15.170:0/2544610750 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96fc0b1a70 con 0x7f96fc1597f0 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9701518700 1 --2- 172.21.15.170:0/2544610750 >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 0x7f96fc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.912+0000 7f9700d17700 1 --2- 172.21.15.170:0/2544610750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc1597f0 0x7f96fc0b2270 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0b2f20 tx=0x7f96ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2f09d4cdbe5fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f96f3fff700 1 -- 172.21.15.170:0/2544610750 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00b860 con 0x7f96fc1597f0 2023-10-08T16:13:50.915 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f96f3fff700 1 -- 172.21.15.170:0/2544610750 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec00b9c0 con 0x7f96fc1597f0 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f96f3fff700 1 -- 172.21.15.170:0/2544610750 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00bd60 con 0x7f96fc1597f0 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 -- 172.21.15.170:0/2544610750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc1597f0 msgr2=0x7f96fc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 --2- 172.21.15.170:0/2544610750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc1597f0 0x7f96fc0b2270 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0b2f20 tx=0x7f96ec009250 comp rx=0 tx=0).stop 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 -- 172.21.15.170:0/2544610750 shutdown_connections 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 --2- 172.21.15.170:0/2544610750 >> v2:172.21.15.191:3300/0 conn(0x7f96fc159420 0x7f96fc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 --2- 172.21.15.170:0/2544610750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc1597f0 0x7f96fc0b2270 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 -- 172.21.15.170:0/2544610750 >> 172.21.15.170:0/2544610750 conn(0x7f96fc0d0660 msgr2=0x7f96fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 -- 172.21.15.170:0/2544610750 shutdown_connections 2023-10-08T16:13:50.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.913+0000 7f970251a700 1 -- 172.21.15.170:0/2544610750 wait complete. 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.914+0000 7f970251a700 1 Processor -- start 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.914+0000 7f970251a700 1 -- start start 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96fc159420 0x7f96fc0c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc1597f0 0x7f96fc0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0ca730 con 0x7f96fc15d230 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0c9b20 con 0x7f96fc1597f0 2023-10-08T16:13:50.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f970251a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96fc0c9e00 con 0x7f96fc159420 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f9701d19700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f9701d19700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52346/0 (socket says 172.21.15.170:52346) 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f9701d19700 1 -- 172.21.15.170:0/795008242 learned_addr learned my addr 172.21.15.170:0/795008242 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f9701518700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96fc159420 0x7f96fc0c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.915+0000 7f9700d17700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc1597f0 0x7f96fc0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 -- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96fc159420 msgr2=0x7f96fc0c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96fc159420 0x7f96fc0c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.920 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 -- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc1597f0 msgr2=0x7f96fc0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc1597f0 0x7f96fc0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 -- 172.21.15.170:0/795008242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0094f0 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f9701d19700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c810 tx=0x7f96f8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8012630 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f80127b0 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8012b70 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96fc0ca0e0 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.916+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96fc15bc00 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.917+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96fc0ca8a0 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.917+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f96f8003030 con 0x7f96fc15d230 2023-10-08T16:13:50.921 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.918+0000 7f96f17fa700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 0x7f96e0069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.918+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f96f8084b60 con 0x7f96fc15d230 2023-10-08T16:13:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.918+0000 7f9701518700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 0x7f96e0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.918+0000 7f9701518700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 0x7f96e0069850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006f00 tx=0x7f96f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:50.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:50.922+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96f80517b0 con 0x7f96fc15d230 2023-10-08T16:13:51.102 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.102+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96fc04c3c0 con 0x7f96e00673a0 2023-10-08T16:13:51.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.110+0000 7f96f17fa700 1 -- 172.21.15.170:0/795008242 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289481 (secure 0 0 0) 0x7f96fc04c3c0 con 0x7f96e00673a0 2023-10-08T16:13:51.113 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 msgr2=0x7f96e0069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 0x7f96e0069850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96f4006f00 tx=0x7f96f4008040 comp rx=0 tx=0).stop 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 msgr2=0x7f96fc15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c810 tx=0x7f96f8001650 comp rx=0 tx=0).stop 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 shutdown_connections 2023-10-08T16:13:51.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f96e00673a0 0x7f96e0069850 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96fc15d230 0x7f96fc15b6c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f96fc159420 0x7f96fc0c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 --2- 172.21.15.170:0/795008242 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f96fc1597f0 0x7f96fc0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 >> 172.21.15.170:0/795008242 conn(0x7f96fc0d0660 msgr2=0x7f96fc0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 shutdown_connections 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.113+0000 7f970251a700 1 -- 172.21.15.170:0/795008242 wait complete. 2023-10-08T16:13:51.117 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:13:51.125 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2023-10-08T16:13:50.334941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444552,"kb_used_data":21192,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752530168,"statfs":{"total":773094113280,"available":770590892032,"internally_reserved":0,"allocated":21700608,"data_stored":10388348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.646426"},"pg_stats":[{"pgid":"34.3f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719631+0000","last_change":"2023-10-08T16:13:46.716965+0000","last_active":"2023-10-08T16:13:46.719631+0000","last_peered":"2023-10-08T16:13:46.719631+0000","last_clean":"2023-10-08T16:13:46.719631+0000","last_became_active":"2023-10-08T16:13:46.716533+0000","last_became_peered":"2023-10-08T16:13:46.716533+0000","last_unstale":"2023-10-08T16:13:46.719631+0000","last_undegraded":"2023-10-08T16:13:46.719631+0000","last_fullsized":"2023-10-08T16:13:46.719631+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:15:49.088630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.3e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720216+0000","last_change":"2023-10-08T16:13:46.712321+0000","last_active":"2023-10-08T16:13:46.720216+0000","last_peered":"2023-10-08T16:13:46.720216+0000","last_clean":"2023-10-08T16:13:46.720216+0000","last_became_active":"2023-10-08T16:13:46.710565+0000","last_became_peered":"2023-10-08T16:13:46.710565+0000","last_unstale":"2023-10-08T16:13:46.720216+0000","last_undegraded":"2023-10-08T16:13:46.720216+0000","last_fullsized":"2023-10-08T16:13:46.720216+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:57.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.3d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719777+0000","last_change":"2023-10-08T16:13:46.711501+0000","last_active":"2023-10-08T16:13:46.719777+0000","last_peered":"2023-10-08T16:13:46.719777+0000","last_clean":"2023-10-08T16:13:46.719777+0000","last_became_active":"2023-10-08T16:13:46.711217+0000","last_became_peered":"2023-10-08T16:13:46.711217+0000","last_unstale":"2023-10-08T16:13:46.719777+0000","last_undegraded":"2023-10-08T16:13:46.719777+0000","last_fullsized":"2023-10-08T16:13:46.719777+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:53:50.720056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726238+0000","last_change":"2023-10-08T16:13:46.723579+0000","last_active":"2023-10-08T16:13:46.726238+0000","last_peered":"2023-10-08T16:13:46.726238+0000","last_clean":"2023-10-08T16:13:46.726238+0000","last_became_active":"2023-10-08T16:13:46.723334+0000","last_became_peered":"2023-10-08T16:13:46.723334+0000","last_unstale":"2023-10-08T16:13:46.726238+0000","last_undegraded":"2023-10-08T16:13:46.726238+0000","last_fullsized":"2023-10-08T16:13:46.726238+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:18:19.423200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.3b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718449+0000","last_change":"2023-10-08T16:13:46.716220+0000","last_active":"2023-10-08T16:13:46.718449+0000","last_peered":"2023-10-08T16:13:46.718449+0000","last_clean":"2023-10-08T16:13:46.718449+0000","last_became_active":"2023-10-08T16:13:46.715977+0000","last_became_peered":"2023-10-08T16:13:46.715977+0000","last_unstale":"2023-10-08T16:13:46.718449+0000","last_undegraded":"2023-10-08T16:13:46.718449+0000","last_fullsized":"2023-10-08T16:13:46.718449+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:19:09.661956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.3a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720450+0000","last_change":"2023-10-08T16:13:46.717466+0000","last_active":"2023-10-08T16:13:46.720450+0000","last_peered":"2023-10-08T16:13:46.720450+0000","last_clean":"2023-10-08T16:13:46.720450+0000","last_became_active":"2023-10-08T16:13:46.717104+0000","last_became_peered":"2023-10-08T16:13:46.717104+0000","last_unstale":"2023-10-08T16:13:46.720450+0000","last_undegraded":"2023-10-08T16:13:46.720450+0000","last_fullsized":"2023-10-08T16:13:46.720450+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:33:48.165199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.39","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720895+0000","last_change":"2023-10-08T16:13:46.716979+0000","last_active":"2023-10-08T16:13:46.720895+0000","last_peered":"2023-10-08T16:13:46.720895+0000","last_clean":"2023-10-08T16:13:46.720895+0000","last_became_active":"2023-10-08T16:13:46.716575+0000","last_became_peered":"2023-10-08T16:13:46.716575+0000","last_unstale":"2023-10-08T16:13:46.720895+0000","last_undegraded":"2023-10-08T16:13:46.720895+0000","last_fullsized":"2023-10-08T16:13:46.720895+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:07:54.321204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.38","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.725669+0000","last_change":"2023-10-08T16:13:46.719186+0000","last_active":"2023-10-08T16:13:46.725669+0000","last_peered":"2023-10-08T16:13:46.725669+0000","last_clean":"2023-10-08T16:13:46.725669+0000","last_became_active":"2023-10-08T16:13:46.718970+0000","last_became_peered":"2023-10-08T16:13:46.718970+0000","last_unstale":"2023-10-08T16:13:46.725669+0000","last_undegraded":"2023-10-08T16:13:46.725669+0000","last_fullsized":"2023-10-08T16:13:46.725669+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:42:42.873165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.37","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719377+0000","last_change":"2023-10-08T16:13:46.713041+0000","last_active":"2023-10-08T16:13:46.719377+0000","last_peered":"2023-10-08T16:13:46.719377+0000","last_clean":"2023-10-08T16:13:46.719377+0000","last_became_active":"2023-10-08T16:13:46.712750+0000","last_became_peered":"2023-10-08T16:13:46.712750+0000","last_unstale":"2023-10-08T16:13:46.719377+0000","last_undegraded":"2023-10-08T16:13:46.719377+0000","last_fullsized":"2023-10-08T16:13:46.719377+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:18.258158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.36","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711018+0000","last_change":"2023-10-08T16:13:46.710226+0000","last_active":"2023-10-08T16:13:46.711018+0000","last_peered":"2023-10-08T16:13:46.711018+0000","last_clean":"2023-10-08T16:13:46.711018+0000","last_became_active":"2023-10-08T16:13:46.709775+0000","last_became_peered":"2023-10-08T16:13:46.709775+0000","last_unstale":"2023-10-08T16:13:46.711018+0000","last_undegraded":"2023-10-08T16:13:46.711018+0000","last_fullsized":"2023-10-08T16:13:46.711018+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:51.276191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.35","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719315+0000","last_change":"2023-10-08T16:13:46.714419+0000","last_active":"2023-10-08T16:13:46.719315+0000","last_peered":"2023-10-08T16:13:46.719315+0000","last_clean":"2023-10-08T16:13:46.719315+0000","last_became_active":"2023-10-08T16:13:46.714049+0000","last_became_peered":"2023-10-08T16:13:46.714049+0000","last_unstale":"2023-10-08T16:13:46.719315+0000","last_undegraded":"2023-10-08T16:13:46.719315+0000","last_fullsized":"2023-10-08T16:13:46.719315+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:13.999024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.34","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719734+0000","last_change":"2023-10-08T16:13:46.714854+0000","last_active":"2023-10-08T16:13:46.719734+0000","last_peered":"2023-10-08T16:13:46.719734+0000","last_clean":"2023-10-08T16:13:46.719734+0000","last_became_active":"2023-10-08T16:13:46.714401+0000","last_became_peered":"2023-10-08T16:13:46.714401+0000","last_unstale":"2023-10-08T16:13:46.719734+0000","last_undegraded":"2023-10-08T16:13:46.719734+0000","last_fullsized":"2023-10-08T16:13:46.719734+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:15:13.109279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.33","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726380+0000","last_change":"2023-10-08T16:13:46.724430+0000","last_active":"2023-10-08T16:13:46.726380+0000","last_peered":"2023-10-08T16:13:46.726380+0000","last_clean":"2023-10-08T16:13:46.726380+0000","last_became_active":"2023-10-08T16:13:46.723527+0000","last_became_peered":"2023-10-08T16:13:46.723527+0000","last_unstale":"2023-10-08T16:13:46.726380+0000","last_undegraded":"2023-10-08T16:13:46.726380+0000","last_fullsized":"2023-10-08T16:13:46.726380+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:51:10.385487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.32","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.725890+0000","last_change":"2023-10-08T16:13:46.724989+0000","last_active":"2023-10-08T16:13:46.725890+0000","last_peered":"2023-10-08T16:13:46.725890+0000","last_clean":"2023-10-08T16:13:46.725890+0000","last_became_active":"2023-10-08T16:13:46.724529+0000","last_became_peered":"2023-10-08T16:13:46.724529+0000","last_unstale":"2023-10-08T16:13:46.725890+0000","last_undegraded":"2023-10-08T16:13:46.725890+0000","last_fullsized":"2023-10-08T16:13:46.725890+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:07:17.387236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.31","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720748+0000","last_change":"2023-10-08T16:13:46.716730+0000","last_active":"2023-10-08T16:13:46.720748+0000","last_peered":"2023-10-08T16:13:46.720748+0000","last_clean":"2023-10-08T16:13:46.720748+0000","last_became_active":"2023-10-08T16:13:46.716428+0000","last_became_peered":"2023-10-08T16:13:46.716428+0000","last_unstale":"2023-10-08T16:13:46.720748+0000","last_undegraded":"2023-10-08T16:13:46.720748+0000","last_fullsized":"2023-10-08T16:13:46.720748+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:37:52.000625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.30","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717786+0000","last_change":"2023-10-08T16:13:46.714598+0000","last_active":"2023-10-08T16:13:46.717786+0000","last_peered":"2023-10-08T16:13:46.717786+0000","last_clean":"2023-10-08T16:13:46.717786+0000","last_became_active":"2023-10-08T16:13:46.714338+0000","last_became_peered":"2023-10-08T16:13:46.714338+0000","last_unstale":"2023-10-08T16:13:46.717786+0000","last_undegraded":"2023-10-08T16:13:46.717786+0000","last_fullsized":"2023-10-08T16:13:46.717786+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:56.307779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.710993+0000","last_change":"2023-10-08T16:13:46.709772+0000","last_active":"2023-10-08T16:13:46.710993+0000","last_peered":"2023-10-08T16:13:46.710993+0000","last_clean":"2023-10-08T16:13:46.710993+0000","last_became_active":"2023-10-08T16:13:46.709469+0000","last_became_peered":"2023-10-08T16:13:46.709469+0000","last_unstale":"2023-10-08T16:13:46.710993+0000","last_undegraded":"2023-10-08T16:13:46.710993+0000","last_fullsized":"2023-10-08T16:13:46.710993+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:27:18.832660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.705502+0000","last_change":"2023-10-08T16:13:46.701770+0000","last_active":"2023-10-08T16:13:46.705502+0000","last_peered":"2023-10-08T16:13:46.705502+0000","last_clean":"2023-10-08T16:13:46.705502+0000","last_became_active":"2023-10-08T16:13:46.701495+0000","last_became_peered":"2023-10-08T16:13:46.701495+0000","last_unstale":"2023-10-08T16:13:46.705502+0000","last_undegraded":"2023-10-08T16:13:46.705502+0000","last_fullsized":"2023-10-08T16:13:46.705502+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:16:26.230719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.716722+0000","last_change":"2023-10-08T16:13:46.713079+0000","last_active":"2023-10-08T16:13:46.716722+0000","last_peered":"2023-10-08T16:13:46.716722+0000","last_clean":"2023-10-08T16:13:46.716722+0000","last_became_active":"2023-10-08T16:13:46.712881+0000","last_became_peered":"2023-10-08T16:13:46.712881+0000","last_unstale":"2023-10-08T16:13:46.716722+0000","last_undegraded":"2023-10-08T16:13:46.716722+0000","last_fullsized":"2023-10-08T16:13:46.716722+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:53:13.999024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.2c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720842+0000","last_change":"2023-10-08T16:13:46.718098+0000","last_active":"2023-10-08T16:13:46.720842+0000","last_peered":"2023-10-08T16:13:46.720842+0000","last_clean":"2023-10-08T16:13:46.720842+0000","last_became_active":"2023-10-08T16:13:46.717371+0000","last_became_peered":"2023-10-08T16:13:46.717371+0000","last_unstale":"2023-10-08T16:13:46.720842+0000","last_undegraded":"2023-10-08T16:13:46.720842+0000","last_fullsized":"2023-10-08T16:13:46.720842+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.2b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721296+0000","last_change":"2023-10-08T16:13:46.719405+0000","last_active":"2023-10-08T16:13:46.721296+0000","last_peered":"2023-10-08T16:13:46.721296+0000","last_clean":"2023-10-08T16:13:46.721296+0000","last_became_active":"2023-10-08T16:13:46.718899+0000","last_became_peered":"2023-10-08T16:13:46.718899+0000","last_unstale":"2023-10-08T16:13:46.721296+0000","last_undegraded":"2023-10-08T16:13:46.721296+0000","last_fullsized":"2023-10-08T16:13:46.721296+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:29:19.761812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.2a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719727+0000","last_change":"2023-10-08T16:13:46.716500+0000","last_active":"2023-10-08T16:13:46.719727+0000","last_peered":"2023-10-08T16:13:46.719727+0000","last_clean":"2023-10-08T16:13:46.719727+0000","last_became_active":"2023-10-08T16:13:46.716179+0000","last_became_peered":"2023-10-08T16:13:46.716179+0000","last_unstale":"2023-10-08T16:13:46.719727+0000","last_undegraded":"2023-10-08T16:13:46.719727+0000","last_fullsized":"2023-10-08T16:13:46.719727+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:29:17.903574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.29","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720783+0000","last_change":"2023-10-08T16:13:46.717103+0000","last_active":"2023-10-08T16:13:46.720783+0000","last_peered":"2023-10-08T16:13:46.720783+0000","last_clean":"2023-10-08T16:13:46.720783+0000","last_became_active":"2023-10-08T16:13:46.716751+0000","last_became_peered":"2023-10-08T16:13:46.716751+0000","last_unstale":"2023-10-08T16:13:46.720783+0000","last_undegraded":"2023-10-08T16:13:46.720783+0000","last_fullsized":"2023-10-08T16:13:46.720783+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:50:50.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.28","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711930+0000","last_change":"2023-10-08T16:13:46.708707+0000","last_active":"2023-10-08T16:13:46.711930+0000","last_peered":"2023-10-08T16:13:46.711930+0000","last_clean":"2023-10-08T16:13:46.711930+0000","last_became_active":"2023-10-08T16:13:46.707058+0000","last_became_peered":"2023-10-08T16:13:46.707058+0000","last_unstale":"2023-10-08T16:13:46.711930+0000","last_undegraded":"2023-10-08T16:13:46.711930+0000","last_fullsized":"2023-10-08T16:13:46.711930+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:09:51.276191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.27","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.722608+0000","last_change":"2023-10-08T16:13:46.720029+0000","last_active":"2023-10-08T16:13:46.722608+0000","last_peered":"2023-10-08T16:13:46.722608+0000","last_clean":"2023-10-08T16:13:46.722608+0000","last_became_active":"2023-10-08T16:13:46.719722+0000","last_became_peered":"2023-10-08T16:13:46.719722+0000","last_unstale":"2023-10-08T16:13:46.722608+0000","last_undegraded":"2023-10-08T16:13:46.722608+0000","last_fullsized":"2023-10-08T16:13:46.722608+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:49:49.593058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.26","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720236+0000","last_change":"2023-10-08T16:13:46.712522+0000","last_active":"2023-10-08T16:13:46.720236+0000","last_peered":"2023-10-08T16:13:46.720236+0000","last_clean":"2023-10-08T16:13:46.720236+0000","last_became_active":"2023-10-08T16:13:46.710314+0000","last_became_peered":"2023-10-08T16:13:46.710314+0000","last_unstale":"2023-10-08T16:13:46.720236+0000","last_undegraded":"2023-10-08T16:13:46.720236+0000","last_fullsized":"2023-10-08T16:13:46.720236+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:29.865417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.25","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720792+0000","last_change":"2023-10-08T16:13:46.719679+0000","last_active":"2023-10-08T16:13:46.720792+0000","last_peered":"2023-10-08T16:13:46.720792+0000","last_clean":"2023-10-08T16:13:46.720792+0000","last_became_active":"2023-10-08T16:13:46.719142+0000","last_became_peered":"2023-10-08T16:13:46.719142+0000","last_unstale":"2023-10-08T16:13:46.720792+0000","last_undegraded":"2023-10-08T16:13:46.720792+0000","last_fullsized":"2023-10-08T16:13:46.720792+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:54:39.994481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.24","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719652+0000","last_change":"2023-10-08T16:13:46.717328+0000","last_active":"2023-10-08T16:13:46.719652+0000","last_peered":"2023-10-08T16:13:46.719652+0000","last_clean":"2023-10-08T16:13:46.719652+0000","last_became_active":"2023-10-08T16:13:46.716219+0000","last_became_peered":"2023-10-08T16:13:46.716219+0000","last_unstale":"2023-10-08T16:13:46.719652+0000","last_undegraded":"2023-10-08T16:13:46.719652+0000","last_fullsized":"2023-10-08T16:13:46.719652+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:08:52.774909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.22","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719427+0000","last_change":"2023-10-08T16:13:46.712537+0000","last_active":"2023-10-08T16:13:46.719427+0000","last_peered":"2023-10-08T16:13:46.719427+0000","last_clean":"2023-10-08T16:13:46.719427+0000","last_became_active":"2023-10-08T16:13:46.712160+0000","last_became_peered":"2023-10-08T16:13:46.712160+0000","last_unstale":"2023-10-08T16:13:46.719427+0000","last_undegraded":"2023-10-08T16:13:46.719427+0000","last_fullsized":"2023-10-08T16:13:46.719427+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:12:14.531087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.21","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728143+0000","last_change":"2023-10-08T16:13:46.725629+0000","last_active":"2023-10-08T16:13:46.728143+0000","last_peered":"2023-10-08T16:13:46.728143+0000","last_clean":"2023-10-08T16:13:46.728143+0000","last_became_active":"2023-10-08T16:13:46.725221+0000","last_became_peered":"2023-10-08T16:13:46.725221+0000","last_unstale":"2023-10-08T16:13:46.728143+0000","last_undegraded":"2023-10-08T16:13:46.728143+0000","last_fullsized":"2023-10-08T16:13:46.728143+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:36:56.467137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.20","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.712259+0000","last_change":"2023-10-08T16:13:46.709459+0000","last_active":"2023-10-08T16:13:46.712259+0000","last_peered":"2023-10-08T16:13:46.712259+0000","last_clean":"2023-10-08T16:13:46.712259+0000","last_became_active":"2023-10-08T16:13:46.709245+0000","last_became_peered":"2023-10-08T16:13:46.709245+0000","last_unstale":"2023-10-08T16:13:46.712259+0000","last_undegraded":"2023-10-08T16:13:46.712259+0000","last_fullsized":"2023-10-08T16:13:46.712259+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:28:03.814244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717086+0000","last_change":"2023-10-08T16:13:46.714125+0000","last_active":"2023-10-08T16:13:46.717086+0000","last_peered":"2023-10-08T16:13:46.717086+0000","last_clean":"2023-10-08T16:13:46.717086+0000","last_became_active":"2023-10-08T16:13:46.713869+0000","last_became_peered":"2023-10-08T16:13:46.713869+0000","last_unstale":"2023-10-08T16:13:46.717086+0000","last_undegraded":"2023-10-08T16:13:46.717086+0000","last_fullsized":"2023-10-08T16:13:46.717086+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:56.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1f","version":"200'4","reported_seq":14,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720227+0000","last_change":"2023-10-08T16:13:45.712063+0000","last_active":"2023-10-08T16:13:46.720227+0000","last_peered":"2023-10-08T16:13:46.720227+0000","last_clean":"2023-10-08T16:13:46.720227+0000","last_became_active":"2023-10-08T16:13:45.711548+0000","last_became_peered":"2023-10-08T16:13:45.711548+0000","last_unstale":"2023-10-08T16:13:46.720227+0000","last_undegraded":"2023-10-08T16:13:46.720227+0000","last_fullsized":"2023-10-08T16:13:46.720227+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:35:08.377590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721524+0000","last_change":"2023-10-08T16:13:46.715069+0000","last_active":"2023-10-08T16:13:46.721524+0000","last_peered":"2023-10-08T16:13:46.721524+0000","last_clean":"2023-10-08T16:13:46.721524+0000","last_became_active":"2023-10-08T16:13:46.714767+0000","last_became_peered":"2023-10-08T16:13:46.714767+0000","last_unstale":"2023-10-08T16:13:46.721524+0000","last_undegraded":"2023-10-08T16:13:46.721524+0000","last_fullsized":"2023-10-08T16:13:46.721524+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:17.329894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1e","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720655+0000","last_change":"2023-10-08T16:13:45.711057+0000","last_active":"2023-10-08T16:13:46.720655+0000","last_peered":"2023-10-08T16:13:46.720655+0000","last_clean":"2023-10-08T16:13:46.720655+0000","last_became_active":"2023-10-08T16:13:45.710601+0000","last_became_peered":"2023-10-08T16:13:45.710601+0000","last_unstale":"2023-10-08T16:13:46.720655+0000","last_undegraded":"2023-10-08T16:13:46.720655+0000","last_fullsized":"2023-10-08T16:13:46.720655+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:26.860970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721544+0000","last_change":"2023-10-08T16:13:46.715207+0000","last_active":"2023-10-08T16:13:46.721544+0000","last_peered":"2023-10-08T16:13:46.721544+0000","last_clean":"2023-10-08T16:13:46.721544+0000","last_became_active":"2023-10-08T16:13:46.714963+0000","last_became_peered":"2023-10-08T16:13:46.714963+0000","last_unstale":"2023-10-08T16:13:46.721544+0000","last_undegraded":"2023-10-08T16:13:46.721544+0000","last_fullsized":"2023-10-08T16:13:46.721544+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:07.682520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1d","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721133+0000","last_change":"2023-10-08T16:13:45.708676+0000","last_active":"2023-10-08T16:13:46.721133+0000","last_peered":"2023-10-08T16:13:46.721133+0000","last_clean":"2023-10-08T16:13:46.721133+0000","last_became_active":"2023-10-08T16:13:45.708415+0000","last_became_peered":"2023-10-08T16:13:45.708415+0000","last_unstale":"2023-10-08T16:13:46.721133+0000","last_undegraded":"2023-10-08T16:13:46.721133+0000","last_fullsized":"2023-10-08T16:13:46.721133+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:28:56.692151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"34.1f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720569+0000","last_change":"2023-10-08T16:13:46.716597+0000","last_active":"2023-10-08T16:13:46.720569+0000","last_peered":"2023-10-08T16:13:46.720569+0000","last_clean":"2023-10-08T16:13:46.720569+0000","last_became_active":"2023-10-08T16:13:46.716315+0000","last_became_peered":"2023-10-08T16:13:46.716315+0000","last_unstale":"2023-10-08T16:13:46.720569+0000","last_undegraded":"2023-10-08T16:13:46.720569+0000","last_fullsized":"2023-10-08T16:13:46.720569+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:39:23.158046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1c","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719132+0000","last_change":"2023-10-08T16:13:45.704516+0000","last_active":"2023-10-08T16:13:46.719132+0000","last_peered":"2023-10-08T16:13:46.719132+0000","last_clean":"2023-10-08T16:13:46.719132+0000","last_became_active":"2023-10-08T16:13:45.704300+0000","last_became_peered":"2023-10-08T16:13:45.704300+0000","last_unstale":"2023-10-08T16:13:46.719132+0000","last_undegraded":"2023-10-08T16:13:46.719132+0000","last_fullsized":"2023-10-08T16:13:46.719132+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:33:11.998472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.18","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721053+0000","last_change":"2023-10-08T16:13:46.718188+0000","last_active":"2023-10-08T16:13:46.721053+0000","last_peered":"2023-10-08T16:13:46.721053+0000","last_clean":"2023-10-08T16:13:46.721053+0000","last_became_active":"2023-10-08T16:13:46.717108+0000","last_became_peered":"2023-10-08T16:13:46.717108+0000","last_unstale":"2023-10-08T16:13:46.721053+0000","last_undegraded":"2023-10-08T16:13:46.721053+0000","last_fullsized":"2023-10-08T16:13:46.721053+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:41:46.340315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1b","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717253+0000","last_change":"2023-10-08T16:13:45.714455+0000","last_active":"2023-10-08T16:13:46.717253+0000","last_peered":"2023-10-08T16:13:46.717253+0000","last_clean":"2023-10-08T16:13:46.717253+0000","last_became_active":"2023-10-08T16:13:45.714223+0000","last_became_peered":"2023-10-08T16:13:45.714223+0000","last_unstale":"2023-10-08T16:13:46.717253+0000","last_undegraded":"2023-10-08T16:13:46.717253+0000","last_fullsized":"2023-10-08T16:13:46.717253+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:12:13.525689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.19","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720729+0000","last_change":"2023-10-08T16:13:46.715968+0000","last_active":"2023-10-08T16:13:46.720729+0000","last_peered":"2023-10-08T16:13:46.720729+0000","last_clean":"2023-10-08T16:13:46.720729+0000","last_became_active":"2023-10-08T16:13:46.715676+0000","last_became_peered":"2023-10-08T16:13:46.715676+0000","last_unstale":"2023-10-08T16:13:46.720729+0000","last_undegraded":"2023-10-08T16:13:46.720729+0000","last_fullsized":"2023-10-08T16:13:46.720729+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:39:04.379764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1a","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726527+0000","last_change":"2023-10-08T16:13:45.712951+0000","last_active":"2023-10-08T16:13:46.726527+0000","last_peered":"2023-10-08T16:13:46.726527+0000","last_clean":"2023-10-08T16:13:46.726527+0000","last_became_active":"2023-10-08T16:13:45.712689+0000","last_became_peered":"2023-10-08T16:13:45.712689+0000","last_unstale":"2023-10-08T16:13:46.726527+0000","last_undegraded":"2023-10-08T16:13:46.726527+0000","last_fullsized":"2023-10-08T16:13:46.726527+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:31.089140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.1a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720868+0000","last_change":"2023-10-08T16:13:46.718873+0000","last_active":"2023-10-08T16:13:46.720868+0000","last_peered":"2023-10-08T16:13:46.720868+0000","last_clean":"2023-10-08T16:13:46.720868+0000","last_became_active":"2023-10-08T16:13:46.718361+0000","last_became_peered":"2023-10-08T16:13:46.718361+0000","last_unstale":"2023-10-08T16:13:46.720868+0000","last_undegraded":"2023-10-08T16:13:46.720868+0000","last_fullsized":"2023-10-08T16:13:46.720868+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:44:42.257442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700490+0000","last_change":"2023-10-08T16:13:45.710004+0000","last_active":"2023-10-08T16:13:46.700490+0000","last_peered":"2023-10-08T16:13:46.700490+0000","last_clean":"2023-10-08T16:13:46.700490+0000","last_became_active":"2023-10-08T16:13:45.709690+0000","last_became_peered":"2023-10-08T16:13:45.709690+0000","last_unstale":"2023-10-08T16:13:46.700490+0000","last_undegraded":"2023-10-08T16:13:46.700490+0000","last_fullsized":"2023-10-08T16:13:46.700490+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:13:53.701102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.1b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726362+0000","last_change":"2023-10-08T16:13:46.724299+0000","last_active":"2023-10-08T16:13:46.726362+0000","last_peered":"2023-10-08T16:13:46.726362+0000","last_clean":"2023-10-08T16:13:46.726362+0000","last_became_active":"2023-10-08T16:13:46.723274+0000","last_became_peered":"2023-10-08T16:13:46.723274+0000","last_unstale":"2023-10-08T16:13:46.726362+0000","last_undegraded":"2023-10-08T16:13:46.726362+0000","last_fullsized":"2023-10-08T16:13:46.726362+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:11:45.904403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.696743+0000","last_change":"2023-10-08T16:13:45.710836+0000","last_active":"2023-10-08T16:13:46.696743+0000","last_peered":"2023-10-08T16:13:46.696743+0000","last_clean":"2023-10-08T16:13:46.696743+0000","last_became_active":"2023-10-08T16:13:45.710547+0000","last_became_peered":"2023-10-08T16:13:45.710547+0000","last_unstale":"2023-10-08T16:13:46.696743+0000","last_undegraded":"2023-10-08T16:13:46.696743+0000","last_fullsized":"2023-10-08T16:13:46.696743+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:30:44.465783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.14","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726132+0000","last_change":"2023-10-08T16:13:46.722962+0000","last_active":"2023-10-08T16:13:46.726132+0000","last_peered":"2023-10-08T16:13:46.726132+0000","last_clean":"2023-10-08T16:13:46.726132+0000","last_became_active":"2023-10-08T16:13:46.722743+0000","last_became_peered":"2023-10-08T16:13:46.722743+0000","last_unstale":"2023-10-08T16:13:46.726132+0000","last_undegraded":"2023-10-08T16:13:46.726132+0000","last_fullsized":"2023-10-08T16:13:46.726132+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:37:57.282005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700057+0000","last_change":"2023-10-08T16:13:45.710255+0000","last_active":"2023-10-08T16:13:46.700057+0000","last_peered":"2023-10-08T16:13:46.700057+0000","last_clean":"2023-10-08T16:13:46.700057+0000","last_became_active":"2023-10-08T16:13:45.710024+0000","last_became_peered":"2023-10-08T16:13:45.710024+0000","last_unstale":"2023-10-08T16:13:46.700057+0000","last_undegraded":"2023-10-08T16:13:46.700057+0000","last_fullsized":"2023-10-08T16:13:46.700057+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:01:34.963368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.15","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.715296+0000","last_change":"2023-10-08T16:13:46.711432+0000","last_active":"2023-10-08T16:13:46.715296+0000","last_peered":"2023-10-08T16:13:46.715296+0000","last_clean":"2023-10-08T16:13:46.715296+0000","last_became_active":"2023-10-08T16:13:46.711173+0000","last_became_peered":"2023-10-08T16:13:46.711173+0000","last_unstale":"2023-10-08T16:13:46.715296+0000","last_undegraded":"2023-10-08T16:13:46.715296+0000","last_fullsized":"2023-10-08T16:13:46.715296+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:30:25.699255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700269+0000","last_change":"2023-10-08T16:13:45.710114+0000","last_active":"2023-10-08T16:13:46.700269+0000","last_peered":"2023-10-08T16:13:46.700269+0000","last_clean":"2023-10-08T16:13:46.700269+0000","last_became_active":"2023-10-08T16:13:45.709779+0000","last_became_peered":"2023-10-08T16:13:45.709779+0000","last_unstale":"2023-10-08T16:13:46.700269+0000","last_undegraded":"2023-10-08T16:13:46.700269+0000","last_fullsized":"2023-10-08T16:13:46.700269+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:20:06.536547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"34.16","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719556+0000","last_change":"2023-10-08T16:13:46.713366+0000","last_active":"2023-10-08T16:13:46.719556+0000","last_peered":"2023-10-08T16:13:46.719556+0000","last_clean":"2023-10-08T16:13:46.719556+0000","last_became_active":"2023-10-08T16:13:46.713107+0000","last_became_peered":"2023-10-08T16:13:46.713107+0000","last_unstale":"2023-10-08T16:13:46.719556+0000","last_undegraded":"2023-10-08T16:13:46.719556+0000","last_fullsized":"2023-10-08T16:13:46.719556+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:16:27.649039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.15","version":"200'3","reported_seq":13,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726213+0000","last_change":"2023-10-08T16:13:45.710943+0000","last_active":"2023-10-08T16:13:46.726213+0000","last_peered":"2023-10-08T16:13:46.726213+0000","last_clean":"2023-10-08T16:13:46.726213+0000","last_became_active":"2023-10-08T16:13:45.710658+0000","last_became_peered":"2023-10-08T16:13:45.710658+0000","last_unstale":"2023-10-08T16:13:46.726213+0000","last_undegraded":"2023-10-08T16:13:46.726213+0000","last_fullsized":"2023-10-08T16:13:46.726213+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:00:01.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.17","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.711914+0000","last_change":"2023-10-08T16:13:46.708242+0000","last_active":"2023-10-08T16:13:46.711914+0000","last_peered":"2023-10-08T16:13:46.711914+0000","last_clean":"2023-10-08T16:13:46.711914+0000","last_became_active":"2023-10-08T16:13:46.707988+0000","last_became_peered":"2023-10-08T16:13:46.707988+0000","last_unstale":"2023-10-08T16:13:46.711914+0000","last_undegraded":"2023-10-08T16:13:46.711914+0000","last_fullsized":"2023-10-08T16:13:46.711914+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:05.366328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.14","version":"200'6","reported_seq":16,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.723445+0000","last_change":"2023-10-08T16:13:45.715559+0000","last_active":"2023-10-08T16:13:46.723445+0000","last_peered":"2023-10-08T16:13:46.723445+0000","last_clean":"2023-10-08T16:13:46.723445+0000","last_became_active":"2023-10-08T16:13:45.715312+0000","last_became_peered":"2023-10-08T16:13:45.715312+0000","last_unstale":"2023-10-08T16:13:46.723445+0000","last_undegraded":"2023-10-08T16:13:46.723445+0000","last_fullsized":"2023-10-08T16:13:46.723445+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:13:40.229883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.10","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718209+0000","last_change":"2023-10-08T16:13:46.714266+0000","last_active":"2023-10-08T16:13:46.718209+0000","last_peered":"2023-10-08T16:13:46.718209+0000","last_clean":"2023-10-08T16:13:46.718209+0000","last_became_active":"2023-10-08T16:13:46.713993+0000","last_became_peered":"2023-10-08T16:13:46.713993+0000","last_unstale":"2023-10-08T16:13:46.718209+0000","last_undegraded":"2023-10-08T16:13:46.718209+0000","last_fullsized":"2023-10-08T16:13:46.718209+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:05:18.258158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.13","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719261+0000","last_change":"2023-10-08T16:13:45.705570+0000","last_active":"2023-10-08T16:13:46.719261+0000","last_peered":"2023-10-08T16:13:46.719261+0000","last_clean":"2023-10-08T16:13:46.719261+0000","last_became_active":"2023-10-08T16:13:45.705322+0000","last_became_peered":"2023-10-08T16:13:45.705322+0000","last_unstale":"2023-10-08T16:13:46.719261+0000","last_undegraded":"2023-10-08T16:13:46.719261+0000","last_fullsized":"2023-10-08T16:13:46.719261+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:23:56.447630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.11","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719607+0000","last_change":"2023-10-08T16:13:46.717364+0000","last_active":"2023-10-08T16:13:46.719607+0000","last_peered":"2023-10-08T16:13:46.719607+0000","last_clean":"2023-10-08T16:13:46.719607+0000","last_became_active":"2023-10-08T16:13:46.717025+0000","last_became_peered":"2023-10-08T16:13:46.717025+0000","last_unstale":"2023-10-08T16:13:46.719607+0000","last_undegraded":"2023-10-08T16:13:46.719607+0000","last_fullsized":"2023-10-08T16:13:46.719607+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:22:46.143513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700655+0000","last_change":"2023-10-08T16:13:45.709755+0000","last_active":"2023-10-08T16:13:46.700655+0000","last_peered":"2023-10-08T16:13:46.700655+0000","last_clean":"2023-10-08T16:13:46.700655+0000","last_became_active":"2023-10-08T16:13:45.709498+0000","last_became_peered":"2023-10-08T16:13:45.709498+0000","last_unstale":"2023-10-08T16:13:46.700655+0000","last_undegraded":"2023-10-08T16:13:46.700655+0000","last_fullsized":"2023-10-08T16:13:46.700655+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:43:00.914379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.12","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.707810+0000","last_change":"2023-10-08T16:13:46.707449+0000","last_active":"2023-10-08T16:13:46.707810+0000","last_peered":"2023-10-08T16:13:46.707810+0000","last_clean":"2023-10-08T16:13:46.707810+0000","last_became_active":"2023-10-08T16:13:46.707255+0000","last_became_peered":"2023-10-08T16:13:46.707255+0000","last_unstale":"2023-10-08T16:13:46.707810+0000","last_undegraded":"2023-10-08T16:13:46.707810+0000","last_fullsized":"2023-10-08T16:13:46.707810+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:47:34.310886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.699981+0000","last_change":"2023-10-08T16:13:45.711503+0000","last_active":"2023-10-08T16:13:46.699981+0000","last_peered":"2023-10-08T16:13:46.699981+0000","last_clean":"2023-10-08T16:13:46.699981+0000","last_became_active":"2023-10-08T16:13:45.711261+0000","last_became_peered":"2023-10-08T16:13:45.711261+0000","last_unstale":"2023-10-08T16:13:46.699981+0000","last_undegraded":"2023-10-08T16:13:46.699981+0000","last_fullsized":"2023-10-08T16:13:46.699981+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:42:29.269360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.13","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718431+0000","last_change":"2023-10-08T16:13:46.714853+0000","last_active":"2023-10-08T16:13:46.718431+0000","last_peered":"2023-10-08T16:13:46.718431+0000","last_clean":"2023-10-08T16:13:46.718431+0000","last_became_active":"2023-10-08T16:13:46.714601+0000","last_became_peered":"2023-10-08T16:13:46.714601+0000","last_unstale":"2023-10-08T16:13:46.718431+0000","last_undegraded":"2023-10-08T16:13:46.718431+0000","last_fullsized":"2023-10-08T16:13:46.718431+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:33:48.165199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.10","version":"200'8","reported_seq":18,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.727675+0000","last_change":"2023-10-08T16:13:45.713037+0000","last_active":"2023-10-08T16:13:46.727675+0000","last_peered":"2023-10-08T16:13:46.727675+0000","last_clean":"2023-10-08T16:13:46.727675+0000","last_became_active":"2023-10-08T16:13:45.707929+0000","last_became_peered":"2023-10-08T16:13:45.707929+0000","last_unstale":"2023-10-08T16:13:46.727675+0000","last_undegraded":"2023-10-08T16:13:46.727675+0000","last_fullsized":"2023-10-08T16:13:46.727675+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:08:29.315045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.c","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.718962+0000","last_change":"2023-10-08T16:13:46.714344+0000","last_active":"2023-10-08T16:13:46.718962+0000","last_peered":"2023-10-08T16:13:46.718962+0000","last_clean":"2023-10-08T16:13:46.718962+0000","last_became_active":"2023-10-08T16:13:46.713700+0000","last_became_peered":"2023-10-08T16:13:46.713700+0000","last_unstale":"2023-10-08T16:13:46.718962+0000","last_undegraded":"2023-10-08T16:13:46.718962+0000","last_fullsized":"2023-10-08T16:13:46.718962+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:59:19.227974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721806+0000","last_change":"2023-10-08T16:13:45.711223+0000","last_active":"2023-10-08T16:13:46.721806+0000","last_peered":"2023-10-08T16:13:46.721806+0000","last_clean":"2023-10-08T16:13:46.721806+0000","last_became_active":"2023-10-08T16:13:45.710228+0000","last_became_peered":"2023-10-08T16:13:45.710228+0000","last_unstale":"2023-10-08T16:13:46.721806+0000","last_undegraded":"2023-10-08T16:13:46.721806+0000","last_fullsized":"2023-10-08T16:13:46.721806+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:38.143990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"34.d","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719333+0000","last_change":"2023-10-08T16:13:46.715455+0000","last_active":"2023-10-08T16:13:46.719333+0000","last_peered":"2023-10-08T16:13:46.719333+0000","last_clean":"2023-10-08T16:13:46.719333+0000","last_became_active":"2023-10-08T16:13:46.715051+0000","last_became_peered":"2023-10-08T16:13:46.715051+0000","last_unstale":"2023-10-08T16:13:46.719333+0000","last_undegraded":"2023-10-08T16:13:46.719333+0000","last_fullsized":"2023-10-08T16:13:46.719333+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:37:56.205955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.e","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721494+0000","last_change":"2023-10-08T16:13:45.709874+0000","last_active":"2023-10-08T16:13:46.721494+0000","last_peered":"2023-10-08T16:13:46.721494+0000","last_clean":"2023-10-08T16:13:46.721494+0000","last_became_active":"2023-10-08T16:13:45.709603+0000","last_became_peered":"2023-10-08T16:13:45.709603+0000","last_unstale":"2023-10-08T16:13:46.721494+0000","last_undegraded":"2023-10-08T16:13:46.721494+0000","last_fullsized":"2023-10-08T16:13:46.721494+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:12:36.367531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.e","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726305+0000","last_change":"2023-10-08T16:13:46.723933+0000","last_active":"2023-10-08T16:13:46.726305+0000","last_peered":"2023-10-08T16:13:46.726305+0000","last_clean":"2023-10-08T16:13:46.726305+0000","last_became_active":"2023-10-08T16:13:46.723685+0000","last_became_peered":"2023-10-08T16:13:46.723685+0000","last_unstale":"2023-10-08T16:13:46.726305+0000","last_undegraded":"2023-10-08T16:13:46.726305+0000","last_fullsized":"2023-10-08T16:13:46.726305+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T19:17:09.877871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.d","version":"200'1","reported_seq":11,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.721286+0000","last_change":"2023-10-08T16:13:45.710450+0000","last_active":"2023-10-08T16:13:46.721286+0000","last_peered":"2023-10-08T16:13:46.721286+0000","last_clean":"2023-10-08T16:13:46.721286+0000","last_became_active":"2023-10-08T16:13:45.709126+0000","last_became_peered":"2023-10-08T16:13:45.709126+0000","last_unstale":"2023-10-08T16:13:46.721286+0000","last_undegraded":"2023-10-08T16:13:46.721286+0000","last_fullsized":"2023-10-08T16:13:46.721286+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:43:16.324496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.f","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719951+0000","last_change":"2023-10-08T16:13:46.717609+0000","last_active":"2023-10-08T16:13:46.719951+0000","last_peered":"2023-10-08T16:13:46.719951+0000","last_clean":"2023-10-08T16:13:46.719951+0000","last_became_active":"2023-10-08T16:13:46.717267+0000","last_became_peered":"2023-10-08T16:13:46.717267+0000","last_unstale":"2023-10-08T16:13:46.719951+0000","last_undegraded":"2023-10-08T16:13:46.719951+0000","last_fullsized":"2023-10-08T16:13:46.719951+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:35:39.149387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697992+0000","last_change":"2023-10-08T16:13:45.710038+0000","last_active":"2023-10-08T16:13:46.697992+0000","last_peered":"2023-10-08T16:13:46.697992+0000","last_clean":"2023-10-08T16:13:46.697992+0000","last_became_active":"2023-10-08T16:13:45.709768+0000","last_became_peered":"2023-10-08T16:13:45.709768+0000","last_unstale":"2023-10-08T16:13:46.697992+0000","last_undegraded":"2023-10-08T16:13:46.697992+0000","last_fullsized":"2023-10-08T16:13:46.697992+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:27:08.247815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.8","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720593+0000","last_change":"2023-10-08T16:13:46.713653+0000","last_active":"2023-10-08T16:13:46.720593+0000","last_peered":"2023-10-08T16:13:46.720593+0000","last_clean":"2023-10-08T16:13:46.720593+0000","last_became_active":"2023-10-08T16:13:46.713363+0000","last_became_peered":"2023-10-08T16:13:46.713363+0000","last_unstale":"2023-10-08T16:13:46.720593+0000","last_undegraded":"2023-10-08T16:13:46.720593+0000","last_fullsized":"2023-10-08T16:13:46.720593+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:02:05.867238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.b","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728291+0000","last_change":"2023-10-08T16:13:45.713664+0000","last_active":"2023-10-08T16:13:46.728291+0000","last_peered":"2023-10-08T16:13:46.728291+0000","last_clean":"2023-10-08T16:13:46.728291+0000","last_became_active":"2023-10-08T16:13:45.713430+0000","last_became_peered":"2023-10-08T16:13:45.713430+0000","last_unstale":"2023-10-08T16:13:46.728291+0000","last_undegraded":"2023-10-08T16:13:46.728291+0000","last_fullsized":"2023-10-08T16:13:46.728291+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T23:08:44.438570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.9","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726972+0000","last_change":"2023-10-08T16:13:46.726090+0000","last_active":"2023-10-08T16:13:46.726972+0000","last_peered":"2023-10-08T16:13:46.726972+0000","last_clean":"2023-10-08T16:13:46.726972+0000","last_became_active":"2023-10-08T16:13:46.725767+0000","last_became_peered":"2023-10-08T16:13:46.725767+0000","last_unstale":"2023-10-08T16:13:46.726972+0000","last_undegraded":"2023-10-08T16:13:46.726972+0000","last_fullsized":"2023-10-08T16:13:46.726972+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:49:56.307779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.696854+0000","last_change":"2023-10-08T16:13:45.701528+0000","last_active":"2023-10-08T16:13:46.696854+0000","last_peered":"2023-10-08T16:13:46.696854+0000","last_clean":"2023-10-08T16:13:46.696854+0000","last_became_active":"2023-10-08T16:13:45.701281+0000","last_became_peered":"2023-10-08T16:13:45.701281+0000","last_unstale":"2023-10-08T16:13:46.696854+0000","last_undegraded":"2023-10-08T16:13:46.696854+0000","last_fullsized":"2023-10-08T16:13:46.696854+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:15:48.083233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"34.a","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726279+0000","last_change":"2023-10-08T16:13:46.723692+0000","last_active":"2023-10-08T16:13:46.726279+0000","last_peered":"2023-10-08T16:13:46.726279+0000","last_clean":"2023-10-08T16:13:46.726279+0000","last_became_active":"2023-10-08T16:13:46.722805+0000","last_became_peered":"2023-10-08T16:13:46.722805+0000","last_unstale":"2023-10-08T16:13:46.726279+0000","last_undegraded":"2023-10-08T16:13:46.726279+0000","last_fullsized":"2023-10-08T16:13:46.726279+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:59:37.815837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.709087+0000","last_change":"2023-10-08T16:13:45.716060+0000","last_active":"2023-10-08T16:13:46.709087+0000","last_peered":"2023-10-08T16:13:46.709087+0000","last_clean":"2023-10-08T16:13:46.709087+0000","last_became_active":"2023-10-08T16:13:45.715810+0000","last_became_peered":"2023-10-08T16:13:45.715810+0000","last_unstale":"2023-10-08T16:13:46.709087+0000","last_undegraded":"2023-10-08T16:13:46.709087+0000","last_fullsized":"2023-10-08T16:13:46.709087+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:32.344606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"34.b","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.712491+0000","last_change":"2023-10-08T16:13:46.708479+0000","last_active":"2023-10-08T16:13:46.712491+0000","last_peered":"2023-10-08T16:13:46.712491+0000","last_clean":"2023-10-08T16:13:46.712491+0000","last_became_active":"2023-10-08T16:13:46.708142+0000","last_became_peered":"2023-10-08T16:13:46.708142+0000","last_unstale":"2023-10-08T16:13:46.712491+0000","last_undegraded":"2023-10-08T16:13:46.712491+0000","last_fullsized":"2023-10-08T16:13:46.712491+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:04.597091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.700628+0000","last_change":"2023-10-08T16:13:45.713097+0000","last_active":"2023-10-08T16:13:46.700628+0000","last_peered":"2023-10-08T16:13:46.700628+0000","last_clean":"2023-10-08T16:13:46.700628+0000","last_became_active":"2023-10-08T16:13:45.712834+0000","last_became_peered":"2023-10-08T16:13:45.712834+0000","last_unstale":"2023-10-08T16:13:46.700628+0000","last_undegraded":"2023-10-08T16:13:46.700628+0000","last_fullsized":"2023-10-08T16:13:46.700628+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:38:08.000780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.4","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720823+0000","last_change":"2023-10-08T16:13:46.717904+0000","last_active":"2023-10-08T16:13:46.720823+0000","last_peered":"2023-10-08T16:13:46.720823+0000","last_clean":"2023-10-08T16:13:46.720823+0000","last_became_active":"2023-10-08T16:13:46.717618+0000","last_became_peered":"2023-10-08T16:13:46.717618+0000","last_unstale":"2023-10-08T16:13:46.720823+0000","last_undegraded":"2023-10-08T16:13:46.720823+0000","last_fullsized":"2023-10-08T16:13:46.720823+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T02:22:33.350003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.7","version":"200'2","reported_seq":12,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.724135+0000","last_change":"2023-10-08T16:13:45.712846+0000","last_active":"2023-10-08T16:13:46.724135+0000","last_peered":"2023-10-08T16:13:46.724135+0000","last_clean":"2023-10-08T16:13:46.724135+0000","last_became_active":"2023-10-08T16:13:45.712606+0000","last_became_peered":"2023-10-08T16:13:45.712606+0000","last_unstale":"2023-10-08T16:13:46.724135+0000","last_undegraded":"2023-10-08T16:13:46.724135+0000","last_fullsized":"2023-10-08T16:13:46.724135+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:59:28.860020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"34.5","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726956+0000","last_change":"2023-10-08T16:13:46.726347+0000","last_active":"2023-10-08T16:13:46.726956+0000","last_peered":"2023-10-08T16:13:46.726956+0000","last_clean":"2023-10-08T16:13:46.726956+0000","last_became_active":"2023-10-08T16:13:46.725786+0000","last_became_peered":"2023-10-08T16:13:46.725786+0000","last_unstale":"2023-10-08T16:13:46.726956+0000","last_undegraded":"2023-10-08T16:13:46.726956+0000","last_fullsized":"2023-10-08T16:13:46.726956+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:12:23.549077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.698338+0000","last_change":"2023-10-08T16:13:45.708520+0000","last_active":"2023-10-08T16:13:46.698338+0000","last_peered":"2023-10-08T16:13:46.698338+0000","last_clean":"2023-10-08T16:13:46.698338+0000","last_became_active":"2023-10-08T16:13:45.708294+0000","last_became_peered":"2023-10-08T16:13:45.708294+0000","last_unstale":"2023-10-08T16:13:46.698338+0000","last_undegraded":"2023-10-08T16:13:46.698338+0000","last_fullsized":"2023-10-08T16:13:46.698338+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:26:31.099225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.6","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.713646+0000","last_change":"2023-10-08T16:13:46.706525+0000","last_active":"2023-10-08T16:13:46.713646+0000","last_peered":"2023-10-08T16:13:46.713646+0000","last_clean":"2023-10-08T16:13:46.713646+0000","last_became_active":"2023-10-08T16:13:46.706242+0000","last_became_peered":"2023-10-08T16:13:46.706242+0000","last_unstale":"2023-10-08T16:13:46.713646+0000","last_undegraded":"2023-10-08T16:13:46.713646+0000","last_fullsized":"2023-10-08T16:13:46.713646+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:18:29.375190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.701210+0000","last_change":"2023-10-08T16:13:45.714624+0000","last_active":"2023-10-08T16:13:46.701210+0000","last_peered":"2023-10-08T16:13:46.701210+0000","last_clean":"2023-10-08T16:13:46.701210+0000","last_became_active":"2023-10-08T16:13:45.714387+0000","last_became_peered":"2023-10-08T16:13:45.714387+0000","last_unstale":"2023-10-08T16:13:46.701210+0000","last_undegraded":"2023-10-08T16:13:46.701210+0000","last_fullsized":"2023-10-08T16:13:46.701210+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T04:07:06.677122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.7","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.719397+0000","last_change":"2023-10-08T16:13:46.713337+0000","last_active":"2023-10-08T16:13:46.719397+0000","last_peered":"2023-10-08T16:13:46.719397+0000","last_clean":"2023-10-08T16:13:46.719397+0000","last_became_active":"2023-10-08T16:13:46.713033+0000","last_became_peered":"2023-10-08T16:13:46.713033+0000","last_unstale":"2023-10-08T16:13:46.719397+0000","last_undegraded":"2023-10-08T16:13:46.719397+0000","last_fullsized":"2023-10-08T16:13:46.719397+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:19:32.094537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.699889+0000","last_change":"2023-10-08T16:13:45.708389+0000","last_active":"2023-10-08T16:13:46.699889+0000","last_peered":"2023-10-08T16:13:46.699889+0000","last_clean":"2023-10-08T16:13:46.699889+0000","last_became_active":"2023-10-08T16:13:45.708121+0000","last_became_peered":"2023-10-08T16:13:45.708121+0000","last_unstale":"2023-10-08T16:13:46.699889+0000","last_undegraded":"2023-10-08T16:13:46.699889+0000","last_fullsized":"2023-10-08T16:13:46.699889+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T18:51:53.427625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.0","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.722988+0000","last_change":"2023-10-08T16:13:46.705169+0000","last_active":"2023-10-08T16:13:46.722988+0000","last_peered":"2023-10-08T16:13:46.722988+0000","last_clean":"2023-10-08T16:13:46.722988+0000","last_became_active":"2023-10-08T16:13:46.704909+0000","last_became_peered":"2023-10-08T16:13:46.704909+0000","last_unstale":"2023-10-08T16:13:46.722988+0000","last_undegraded":"2023-10-08T16:13:46.722988+0000","last_fullsized":"2023-10-08T16:13:46.722988+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T00:50:50.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697967+0000","last_change":"2023-10-08T16:13:45.709209+0000","last_active":"2023-10-08T16:13:46.697967+0000","last_peered":"2023-10-08T16:13:46.697967+0000","last_clean":"2023-10-08T16:13:46.697967+0000","last_became_active":"2023-10-08T16:13:45.708951+0000","last_became_peered":"2023-10-08T16:13:45.708951+0000","last_unstale":"2023-10-08T16:13:46.697967+0000","last_undegraded":"2023-10-08T16:13:46.697967+0000","last_fullsized":"2023-10-08T16:13:46.697967+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:24:50.264735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.1","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.728161+0000","last_change":"2023-10-08T16:13:46.726205+0000","last_active":"2023-10-08T16:13:46.728161+0000","last_peered":"2023-10-08T16:13:46.728161+0000","last_clean":"2023-10-08T16:13:46.728161+0000","last_became_active":"2023-10-08T16:13:46.725905+0000","last_became_peered":"2023-10-08T16:13:46.725905+0000","last_unstale":"2023-10-08T16:13:46.728161+0000","last_undegraded":"2023-10-08T16:13:46.728161+0000","last_fullsized":"2023-10-08T16:13:46.728161+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T16:36:29.861214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697743+0000","last_change":"2023-10-08T16:13:45.714988+0000","last_active":"2023-10-08T16:13:46.697743+0000","last_peered":"2023-10-08T16:13:46.697743+0000","last_clean":"2023-10-08T16:13:46.697743+0000","last_became_active":"2023-10-08T16:13:45.714625+0000","last_became_peered":"2023-10-08T16:13:45.714625+0000","last_unstale":"2023-10-08T16:13:46.697743+0000","last_undegraded":"2023-10-08T16:13:46.697743+0000","last_fullsized":"2023-10-08T16:13:46.697743+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T21:34:51.253894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.2","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.720409+0000","last_change":"2023-10-08T16:13:46.716158+0000","last_active":"2023-10-08T16:13:46.720409+0000","last_peered":"2023-10-08T16:13:46.720409+0000","last_clean":"2023-10-08T16:13:46.720409+0000","last_became_active":"2023-10-08T16:13:46.715914+0000","last_became_peered":"2023-10-08T16:13:46.715914+0000","last_unstale":"2023-10-08T16:13:46.720409+0000","last_undegraded":"2023-10-08T16:13:46.720409+0000","last_fullsized":"2023-10-08T16:13:46.720409+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T22:57:05.622697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.703750+0000","last_change":"2023-10-08T16:13:45.708973+0000","last_active":"2023-10-08T16:13:46.703750+0000","last_peered":"2023-10-08T16:13:46.703750+0000","last_clean":"2023-10-08T16:13:46.703750+0000","last_became_active":"2023-10-08T16:13:45.708721+0000","last_became_peered":"2023-10-08T16:13:45.708721+0000","last_unstale":"2023-10-08T16:13:46.703750+0000","last_undegraded":"2023-10-08T16:13:46.703750+0000","last_fullsized":"2023-10-08T16:13:46.703750+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:56:45.743779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"34.3","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.717432+0000","last_change":"2023-10-08T16:13:46.712153+0000","last_active":"2023-10-08T16:13:46.717432+0000","last_peered":"2023-10-08T16:13:46.717432+0000","last_clean":"2023-10-08T16:13:46.717432+0000","last_became_active":"2023-10-08T16:13:46.710495+0000","last_became_peered":"2023-10-08T16:13:46.710495+0000","last_unstale":"2023-10-08T16:13:46.717432+0000","last_undegraded":"2023-10-08T16:13:46.717432+0000","last_fullsized":"2023-10-08T16:13:46.717432+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T20:59:05.366328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.697844+0000","last_change":"2023-10-08T16:13:45.709550+0000","last_active":"2023-10-08T16:13:46.697844+0000","last_peered":"2023-10-08T16:13:46.697844+0000","last_clean":"2023-10-08T16:13:46.697844+0000","last_became_active":"2023-10-08T16:13:45.709297+0000","last_became_peered":"2023-10-08T16:13:45.709297+0000","last_unstale":"2023-10-08T16:13:46.697844+0000","last_undegraded":"2023-10-08T16:13:46.697844+0000","last_fullsized":"2023-10-08T16:13:46.697844+0000","mapping_epoch":198,"log_start":"0'0","ondisk_log_start":"0'0","created":198,"last_epoch_clean":199,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:44.683091+0000","last_clean_scrub_stamp":"2023-10-08T16:13:44.683091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T01:31:09.018500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"34.23","version":"0'0","reported_seq":9,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.726327+0000","last_change":"2023-10-08T16:13:46.724173+0000","last_active":"2023-10-08T16:13:46.726327+0000","last_peered":"2023-10-08T16:13:46.726327+0000","last_clean":"2023-10-08T16:13:46.726327+0000","last_became_active":"2023-10-08T16:13:46.722757+0000","last_became_peered":"2023-10-08T16:13:46.722757+0000","last_unstale":"2023-10-08T16:13:46.726327+0000","last_undegraded":"2023-10-08T16:13:46.726327+0000","last_fullsized":"2023-10-08T16:13:46.726327+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:13:45.688488+0000","last_clean_scrub_stamp":"2023-10-08T16:13:45.688488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:02:45.349897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":316,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-10-08T16:13:46.703558+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:13:46.703558+0000","last_peered":"2023-10-08T16:13:46.703558+0000","last_clean":"2023-10-08T16:13:46.703558+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:13:46.703558+0000","last_undegraded":"2023-10-08T16:13:46.703558+0000","last_fullsized":"2023-10-08T16:13:46.703558+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":34,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":33,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304468,"kb_used_data":3084,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067372,"statfs":{"total":96636764160,"available":96324988928,"internally_reserved":0,"allocated":3158016,"data_stored":1741319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53200000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.496,"5min":0.50900000000000001,"15min":0.69999999999999996},"min":{"1min":0.32500000000000001,"5min":0.308,"15min":0.30499999999999999},"max":{"1min":0.68100000000000005,"5min":0.74099999999999999,"15min":8.0939999999999994},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.53900000000000003,"15min":0.54000000000000004},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.29199999999999998},"max":{"1min":0.66200000000000003,"5min":0.82199999999999995,"15min":1.083},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.72499999999999998},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.30399999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.46300000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.49099999999999999,"15min":0.52600000000000002},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":1.175},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53300000000000003,"15min":0.749},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.79100000000000004,"5min":0.89000000000000001,"15min":8.4339999999999993},"last":0.44}]},{"osd":3,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.55000000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.753,"5min":1.677,"15min":1.677},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.52600000000000002,"15min":0.751},"min":{"1min":0.38900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":8.3409999999999993},"last":0.496}]},{"osd":4,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.45900000000000002,"15min":0.47999999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71799999999999997,"5min":0.753,"15min":0.753},"last":0.52100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.51800000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52700000000000002,"15min":0.54700000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.72999999999999998,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:13:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.78000000000000003,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56200000000000006,"15min":0.57699999999999996},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.27200000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51200000000000001,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.308,"15min":0.29299999999999998},"max":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.90600000000000003},"last":0.63100000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.505,"15min":0.496},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.70099999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.68500000000000005,"5min":0.747,"15min":0.76700000000000002},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53700000000000003},"min":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.495},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.379,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69999999999999996,"5min":0.75,"15min":0.96599999999999997},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52800000000000002,"15min":0.55000000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.94199999999999995},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71599999999999997,"5min":0.73899999999999999,"15min":0.76600000000000001},"last":0.51100000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.48799999999999999,"15min":0.51200000000000001},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.26200000000000001},"max":{"1min":0.72499999999999998,"5min":0.76900000000000002,"15min":0.80700000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52700000000000002,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.30099999999999999,"15min":0.28899999999999998},"max":{"1min":0.754,"5min":0.77100000000000002,"15min":0.89200000000000002},"last":0.46100000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.61499999999999999,"5min":0.79400000000000004,"15min":0.81899999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.44,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.82399999999999995,"15min":0.874},"last":0.37}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.33400000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79600000000000004,"5min":0.80700000000000005,"15min":0.93999999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.45200000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.746,"5min":0.81899999999999995,"15min":0.85599999999999998},"last":0.48299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307984,"kb_used_data":2504,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063856,"statfs":{"total":96636764160,"available":96321388544,"internally_reserved":0,"allocated":2564096,"data_stored":1150499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.28899999999999998,"5min":0.311,"15min":0.31},"min":{"1min":0.192,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":1.2150000000000001},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.22700000000000001,"5min":0.219,"15min":0.20300000000000001},"max":{"1min":0.42499999999999999,"5min":0.627,"15min":0.77500000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.34000000000000002,"15min":0.35099999999999998},"min":{"1min":0.23599999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.38600000000000001,"5min":0.57299999999999995,"15min":1.2230000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.372,"15min":0.371},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.40999999999999998,"5min":0.627,"15min":0.72399999999999998},"last":0.27000000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.378,"15min":0.38200000000000001},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45300000000000001,"5min":0.63100000000000001,"15min":0.70399999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.31900000000000001,"5min":0.34499999999999997,"15min":0.35299999999999998},"min":{"1min":0.248,"5min":0.215,"15min":0.215},"max":{"1min":0.373,"5min":0.56399999999999995,"15min":0.72699999999999998},"last":0.34499999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.53500000000000003},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.84099999999999997,"15min":1.077},"last":0.441},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.34300000000000003,"5min":0.33700000000000002,"15min":0.33500000000000002},"max":{"1min":0.78300000000000003,"5min":1.696,"15min":1.696},"last":0.48299999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57199999999999995,"15min":0.56599999999999995},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":1.657,"15min":1.657},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.40200000000000002,"5min":0.378,"15min":0.32300000000000001},"max":{"1min":0.73299999999999998,"5min":0.82099999999999995,"15min":0.82899999999999996},"last":0.501}]},{"osd":6,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.57299999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.72399999999999998,"5min":0.84999999999999998,"15min":1.099},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.626,"15min":0.61199999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81499999999999995,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60099999999999998,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.76900000000000002,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59399999999999997},"min":{"1min":0.442,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.747,"5min":0.875,"15min":1.1839999999999999},"last":0.46000000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574964,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308560,"kb_used_data":3080,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063280,"statfs":{"total":96636764160,"available":96320798720,"internally_reserved":0,"allocated":3153920,"data_stored":1740813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.313,"15min":0.48899999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.41399999999999998,"5min":0.59999999999999998,"15min":5.5599999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.441},"min":{"1min":0.23000000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.46100000000000002,"5min":0.53800000000000003,"15min":3.3220000000000001},"last":0.44900000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.33700000000000002,"15min":0.52000000000000002},"min":{"1min":0.27600000000000002,"5min":0.22900000000000001,"15min":0.22600000000000001},"max":{"1min":0.41499999999999998,"5min":0.53000000000000003,"15min":5.6769999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.34999999999999998,"15min":0.373},"min":{"1min":0.28000000000000003,"5min":0.253,"15min":0.219},"max":{"1min":0.501,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.377}]},{"osd":3,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.34499999999999997,"15min":0.45100000000000001},"min":{"1min":0.27000000000000002,"5min":0.215,"15min":0.188},"max":{"1min":0.59499999999999997,"5min":1.6519999999999999,"15min":3.2160000000000002},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38300000000000001,"15min":0.55800000000000005},"min":{"1min":0.29799999999999999,"5min":0.28100000000000003,"15min":0.26900000000000002},"max":{"1min":0.42999999999999999,"5min":1.498,"15min":5.5970000000000004},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78800000000000003,"5min":0.82599999999999996,"15min":0.86299999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55300000000000005,"15min":0.63900000000000001},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76500000000000001,"5min":1.7649999999999999,"15min":3.2519999999999998},"last":1.1379999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56100000000000005,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70699999999999996,"5min":1.847,"15min":3.2890000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":5.6500000000000004},"last":0.63700000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82699999999999996,"5min":0.86299999999999999,"15min":5.6239999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.58899999999999997},"min":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59199999999999997,"15min":0.60199999999999998},"min":{"1min":0.41699999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88800000000000001},"last":0.751},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.70299999999999996},"min":{"1min":0.42299999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.77000000000000002,"5min":1.8979999999999999,"15min":3.5049999999999999},"last":1.1699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303900,"kb_used_data":2516,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067940,"statfs":{"total":96636764160,"available":96325570560,"internally_reserved":0,"allocated":2576384,"data_stored":1152487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.312,"15min":0.308},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.48199999999999998,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.312,"5min":0.32100000000000001,"15min":0.34200000000000003},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.189},"max":{"1min":0.439,"5min":0.58799999999999997,"15min":0.64300000000000002},"last":0.247}]},{"osd":1,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.33900000000000002,"15min":0.34599999999999997},"min":{"1min":0.26400000000000001,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.45700000000000002,"5min":0.55300000000000005,"15min":0.58499999999999996},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.30599999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.48599999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.32500000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.32600000000000001,"15min":0.34200000000000003},"min":{"1min":0.24399999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60599999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.379,"15min":0.38500000000000001},"min":{"1min":0.28999999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.37}]},{"osd":4,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.40999999999999998,"5min":0.34100000000000003,"15min":0.33300000000000002},"max":{"1min":0.76200000000000001,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.56399999999999995,"15min":0.54800000000000004},"min":{"1min":0.34100000000000003,"5min":0.313,"15min":0.254},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.89100000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.315},"max":{"1min":0.80700000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.56999999999999995,"15min":0.64000000000000001},"min":{"1min":0.435,"5min":0.35799999999999998,"15min":0.30299999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":4.1029999999999998},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.41599999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88300000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.59099999999999997},"min":{"1min":0.44500000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.871},"last":0.67200000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.42799999999999999,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.86799999999999999,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.47499999999999998,"5min":0.434,"15min":0.36299999999999999},"max":{"1min":0.88,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.71499999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307980,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063860,"statfs":{"total":96636764160,"available":96321392640,"internally_reserved":0,"allocated":2560000,"data_stored":1150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.307,"15min":0.34399999999999997},"min":{"1min":0.215,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":0.44900000000000001,"5min":0.505,"15min":1.238},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.32400000000000001,"15min":0.376},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.56599999999999995,"15min":1.9470000000000001},"last":0.26900000000000002}]},{"osd":1,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.33200000000000002,"15min":0.35299999999999998},"min":{"1min":0.254,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45800000000000002,"5min":0.47999999999999998,"15min":1.0269999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.39700000000000002},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.23999999999999999},"max":{"1min":0.499,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.41999999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36399999999999999,"15min":0.39100000000000001},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.248},"max":{"1min":0.47699999999999998,"5min":0.53900000000000003,"15min":1.1399999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.41299999999999998},"min":{"1min":0.28299999999999997,"5min":0.25900000000000001,"15min":0.25800000000000001},"max":{"1min":0.56100000000000005,"5min":0.56499999999999995,"15min":1.327},"last":0.317}]},{"osd":4,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.9119999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":1.099,"15min":1.099},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55000000000000004,"15min":0.57799999999999996},"min":{"1min":0.41999999999999998,"5min":0.35099999999999998,"15min":0.34599999999999997},"max":{"1min":0.76400000000000001,"5min":0.77900000000000003,"15min":1.27},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57199999999999995,"15min":0.60799999999999998},"min":{"1min":0.39700000000000002,"5min":0.35099999999999998,"15min":0.34300000000000003},"max":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":1.879},"last":0.432}]},{"osd":6,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58699999999999997,"15min":0.60299999999999998},"min":{"1min":0.38400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.75700000000000001,"5min":0.81399999999999995,"15min":1.0620000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60699999999999998,"15min":0.62},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.79100000000000004,"5min":0.81200000000000006,"15min":1.2969999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.78500000000000003,"5min":0.97199999999999998,"15min":1.825},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60399999999999998,"15min":0.624},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40000000000000002},"max":{"1min":0.81100000000000005,"5min":1.0009999999999999,"15min":1.115},"last":0.54600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303892,"kb_used_data":2508,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067948,"statfs":{"total":96636764160,"available":96325578752,"internally_reserved":0,"allocated":2568192,"data_stored":1151911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.50600000000000001,"15min":0.52200000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.58999999999999997,"5min":0.68100000000000005,"15min":1.579},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61499999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.51700000000000002,"15min":0.49399999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.57399999999999995,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.63,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63500000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.56200000000000006,"15min":0.57499999999999996},"min":{"1min":0.38700000000000001,"5min":0.33600000000000002,"15min":0.254},"max":{"1min":0.67400000000000004,"5min":0.83699999999999997,"15min":1.5629999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60599999999999998,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.69099999999999995,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.80700000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.69699999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.48299999999999998,"15min":0.51500000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70299999999999996,"5min":0.72999999999999998,"15min":0.77100000000000002},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53400000000000003,"15min":0.55400000000000005},"min":{"1min":0.33200000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.74199999999999999,"5min":0.80000000000000004,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53200000000000003,"15min":0.55600000000000005},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":0.84399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54400000000000004,"15min":0.57599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.83399999999999996},"last":0.69799999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:13:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.58099999999999996,"15min":0.59999999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82899999999999996,"5min":0.875,"15min":0.875},"last":0.747},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59699999999999998,"15min":0.61799999999999999},"min":{"1min":0.40999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.89200000000000002},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303884,"kb_used_data":2500,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067956,"statfs":{"total":96636764160,"available":96325586944,"internally_reserved":0,"allocated":2560000,"data_stored":1150445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76000000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52100000000000002,"15min":0.61899999999999999},"min":{"1min":0.39000000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68700000000000006,"5min":0.80400000000000005,"15min":4.0119999999999996},"last":0.55900000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.51700000000000002,"15min":0.91300000000000003},"min":{"1min":0.373,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73499999999999999,"5min":0.84399999999999997,"15min":12.227},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.38400000000000001,"15min":0.378},"max":{"1min":0.76300000000000001,"5min":0.871,"15min":1.252},"last":0.46800000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53700000000000003,"15min":0.64000000000000001},"min":{"1min":0.436,"5min":0.314,"15min":0.314},"max":{"1min":0.73899999999999999,"5min":0.79300000000000004,"15min":3.9510000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52700000000000002,"15min":0.53800000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.88700000000000001,"15min":1.444},"last":0.443}]},{"osd":3,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.68200000000000005,"5min":0.755,"15min":9.3170000000000002},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57899999999999996,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.78100000000000003,"5min":0.879,"15min":3.8719999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.495,"15min":0.5},"min":{"1min":0.373,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.93799999999999994},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.86599999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.377}]},{"osd":6,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51800000000000002,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.254,"15min":0.254},"max":{"1min":0.79200000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60699999999999998,"15min":0.60799999999999998},"min":{"1min":0.40899999999999997,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.89000000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53400000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.83199999999999996,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.47999999999999998,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.877,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.501}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":34,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:13:51.169 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T16:13:51.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T16:13:51.292 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.292+0000 7fc5615be700 1 Processor -- start 2023-10-08T16:13:51.293 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.292+0000 7fc5615be700 1 -- start start 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc5615be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c1595d0 0x7fc55c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc5615be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc55c0cf480 0x7fc55c15d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc5615be700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c15dbc0 con 0x7fc55c15d760 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc5615be700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c0b10f0 con 0x7fc55c0cf480 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc5615be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0cd530 con 0x7fc55c1595d0 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc55c0cf480 0x7fc55c15d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc55c0cf480 0x7fc55c15d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52360/0 (socket says 172.21.15.170:52360) 2023-10-08T16:13:51.295 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 -- 172.21.15.170:0/2016518044 learned_addr learned my addr 172.21.15.170:0/2016518044 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55bfff700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c1595d0 0x7fc55c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 -- 172.21.15.170:0/2016518044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c1595d0 msgr2=0x7fc55c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c1595d0 0x7fc55c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 -- 172.21.15.170:0/2016518044 >> v1:172.21.15.191:6789/0 conn(0x7fc55c15d760 legacy=0x7fc55c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.293+0000 7fc55b7fe700 1 -- 172.21.15.170:0/2016518044 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c15e000 con 0x7fc55c0cf480 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc55b7fe700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c15d220 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc5500099b0 tx=0x7fc550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e20ff8d9876a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc55affd700 1 -- 172.21.15.170:0/2016518044 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000b820 con 0x7fc55c0cf480 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc55affd700 1 -- 172.21.15.170:0/2016518044 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000b980 con 0x7fc55c0cf480 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc55affd700 1 -- 172.21.15.170:0/2016518044 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000bd20 con 0x7fc55c0cf480 2023-10-08T16:13:51.296 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 -- 172.21.15.170:0/2016518044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 msgr2=0x7fc55c15d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c15d220 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fc5500099b0 tx=0x7fc550009250 comp rx=0 tx=0).stop 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 -- 172.21.15.170:0/2016518044 shutdown_connections 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c1595d0 0x7fc55c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 --2- 172.21.15.170:0/2016518044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c15d220 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 -- 172.21.15.170:0/2016518044 >> 172.21.15.170:0/2016518044 conn(0x7fc55c0d05f0 msgr2=0x7fc55c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 -- 172.21.15.170:0/2016518044 shutdown_connections 2023-10-08T16:13:51.297 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.294+0000 7fc5615be700 1 -- 172.21.15.170:0/2016518044 wait complete. 2023-10-08T16:13:51.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.295+0000 7fc5615be700 1 Processor -- start 2023-10-08T16:13:51.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 -- start start 2023-10-08T16:13:51.299 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc55c1595d0 0x7fc55c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c15d760 0x7fc55c15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c1558c0 con 0x7fc55c0cf480 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c61a0 con 0x7fc55c1595d0 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc5615be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c155d50 con 0x7fc55c15d760 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc55bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52374/0 (socket says 172.21.15.170:52374) 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc55bfff700 1 -- 172.21.15.170:0/919416411 learned_addr learned my addr 172.21.15.170:0/919416411 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc55b7fe700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc55c1595d0 0x7fc55c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.296+0000 7fc560dbd700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c15d760 0x7fc55c15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 -- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c15d760 msgr2=0x7fc55c15c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c15d760 0x7fc55c15c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 -- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc55c1595d0 msgr2=0x7fc55c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc55c1595d0 0x7fc55c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 -- 172.21.15.170:0/919416411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5500094f0 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc55bfff700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00d840 tx=0x7fc54c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c016d20 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c01a070 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc55c154f90 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c0216d0 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.297+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c155550 con 0x7fc55c0cf480 2023-10-08T16:13:51.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.298+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c04c3c0 con 0x7fc55c0cf480 2023-10-08T16:13:51.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.298+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc54c021a40 con 0x7fc55c0cf480 2023-10-08T16:13:51.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.299+0000 7fc543fff700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 0x7fc5480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.299+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc54c01c070 con 0x7fc55c0cf480 2023-10-08T16:13:51.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.299+0000 7fc55b7fe700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 0x7fc5480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.302 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.299+0000 7fc55b7fe700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 0x7fc5480697d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc5500099b0 tx=0x7fc550016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.303+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc54c059ce0 con 0x7fc55c0cf480 2023-10-08T16:13:51.564 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.564+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc55c0b3860 con 0x7fc55c0cf480 2023-10-08T16:13:51.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.568+0000 7fc543fff700 1 -- 172.21.15.170:0/919416411 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v27) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc54c026070 con 0x7fc55c0cf480 2023-10-08T16:13:51.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 msgr2=0x7fc5480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 0x7fc5480697d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fc5500099b0 tx=0x7fc550016000 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 msgr2=0x7fc55c140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc54c00d840 tx=0x7fc54c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 shutdown_connections 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc55c15d760 0x7fc55c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc548067320 0x7fc5480697d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc55c0cf480 0x7fc55c140990 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 --2- 172.21.15.170:0/919416411 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc55c1595d0 0x7fc55c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 >> 172.21.15.170:0/919416411 conn(0x7fc55c0d05f0 msgr2=0x7fc55c0b5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 shutdown_connections 2023-10-08T16:13:51.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.570+0000 7fc5615be700 1 -- 172.21.15.170:0/919416411 wait complete. 2023-10-08T16:13:51.572 INFO:teuthology.orchestra.run.smithi170.stderr:updated caps for client.0 2023-10-08T16:13:51.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:13:51.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.698+0000 7f812fc6b700 1 Processor -- start 2023-10-08T16:13:51.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.698+0000 7f812fc6b700 1 -- start start 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812fc6b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8128159890 0x7f812815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812fc6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 0x7f81280699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812fc6b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8128069ee0 con 0x7f8128159c60 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812fc6b700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812815a160 con 0x7f8128159890 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812fc6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280d13a0 con 0x7f8128069560 2023-10-08T16:13:51.701 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8128159890 0x7f812815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8128159890 0x7f812815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52376/0 (socket says 172.21.15.170:52376) 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 -- 172.21.15.170:0/3137519418 learned_addr learned my addr 172.21.15.170:0/3137519418 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812e208700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 0x7f81280699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 -- 172.21.15.170:0/3137519418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 msgr2=0x7f81280699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 0x7f81280699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.699+0000 7f812da07700 1 -- 172.21.15.170:0/3137519418 >> v1:172.21.15.191:6789/0 conn(0x7f8128159c60 legacy=0x7f812815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812da07700 1 -- 172.21.15.170:0/3137519418 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812815e140 con 0x7f8128159890 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812e208700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 0x7f81280699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812da07700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128159890 0x7f812815abd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8124009f20 tx=0x7f8124009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b8b6212c99dd3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.702 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812ca05700 1 -- 172.21.15.170:0/3137519418 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400e070 con 0x7f8128159890 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812ca05700 1 -- 172.21.15.170:0/3137519418 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812400ba40 con 0x7f8128159890 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812ca05700 1 -- 172.21.15.170:0/3137519418 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240092e0 con 0x7f8128159890 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 -- 172.21.15.170:0/3137519418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128159890 msgr2=0x7f812815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128159890 0x7f812815abd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f8124009f20 tx=0x7f8124009840 comp rx=0 tx=0).stop 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 -- 172.21.15.170:0/3137519418 shutdown_connections 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128069560 0x7f81280699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3137519418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128159890 0x7f812815abd0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 -- 172.21.15.170:0/3137519418 >> 172.21.15.170:0/3137519418 conn(0x7f8128068e20 msgr2=0x7f81280d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 -- 172.21.15.170:0/3137519418 shutdown_connections 2023-10-08T16:13:51.703 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.700+0000 7f812fc6b700 1 -- 172.21.15.170:0/3137519418 wait complete. 2023-10-08T16:13:51.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.701+0000 7f812fc6b700 1 Processor -- start 2023-10-08T16:13:51.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.701+0000 7f812fc6b700 1 -- start start 2023-10-08T16:13:51.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.706 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8128159890 0x7f812816f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128159c60 0x7f81280b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c79b0 con 0x7f8128069560 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c7c90 con 0x7f8128159890 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812fc6b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81280c7f70 con 0x7f8128159c60 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812da07700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812da07700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52382/0 (socket says 172.21.15.170:52382) 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812da07700 1 -- 172.21.15.170:0/3669280431 learned_addr learned my addr 172.21.15.170:0/3669280431 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812d206700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8128159890 0x7f812816f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.702+0000 7f812e208700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128159c60 0x7f81280b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 -- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128159c60 msgr2=0x7f81280b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128159c60 0x7f81280b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.707 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 -- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8128159890 msgr2=0x7f812816f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8128159890 0x7f812816f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 -- 172.21.15.170:0/3669280431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124009ae0 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812da07700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f81280899c0 tx=0x7f8124017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81240128d0 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124012a30 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81280c8250 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124012df0 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.703+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812816f6c0 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.704+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81280c8770 con 0x7f8128069560 2023-10-08T16:13:51.708 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.704+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f812401d630 con 0x7f8128069560 2023-10-08T16:13:51.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.705+0000 7f811dffb700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 0x7f810c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:51.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.705+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8124086480 con 0x7f8128069560 2023-10-08T16:13:51.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.705+0000 7f812d206700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 0x7f810c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:51.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.705+0000 7f812d206700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 0x7f810c0697d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81280b5e30 tx=0x7f8118007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:51.709 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.709+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81280c8770 con 0x7f8128069560 2023-10-08T16:13:51.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.932+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81280c8770 con 0x7f8128069560 2023-10-08T16:13:51.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.933+0000 7f811dffb700 1 -- 172.21.15.170:0/3669280431 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f81280c8770 con 0x7f8128069560 2023-10-08T16:13:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:13:51.934 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":131,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:13:51.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 msgr2=0x7f810c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 0x7f810c0697d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f81280b5e30 tx=0x7f8118007ea0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 msgr2=0x7f8128153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f81280899c0 tx=0x7f8124017000 comp rx=0 tx=0).stop 2023-10-08T16:13:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 shutdown_connections 2023-10-08T16:13:51.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8128159c60 0x7f81280b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f810c067320 0x7f810c0697d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8128069560 0x7f8128153930 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 --2- 172.21.15.170:0/3669280431 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8128159890 0x7f812816f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 >> 172.21.15.170:0/3669280431 conn(0x7f8128068e20 msgr2=0x7f8128154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 shutdown_connections 2023-10-08T16:13:51.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:51.935+0000 7f812fc6b700 1 -- 172.21.15.170:0/3669280431 wait complete. 2023-10-08T16:13:51.939 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 131 2023-10-08T16:13:52.004 INFO:tasks.ceph.mds.c.smithi170.stderr:2023-10-08T16:13:52.003+0000 7fdb92328700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:13:52.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 131, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T16:13:46.076345+0000', 'modified': '2023-10-08T16:13:49.720431+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8997}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8997': {'gid': 8997, 'name': 'c', 'rank': 0, 'incarnation': 127, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6837/1430346956', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6835', 'nonce': 1430346956}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1430346956}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [34], 'metadata_pool': 33, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T16:13:52.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T16:13:52.352 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T16:13:52.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.469+0000 7f865e1bb700 1 Processor -- start 2023-10-08T16:13:52.470 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.469+0000 7f865e1bb700 1 -- start start 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f865e1bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8658159c60 0x7f865815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f865e1bb700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f865815db10 0x7f8658069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f865e1bb700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865815a160 con 0x7f8658159890 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f865e1bb700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86580d13a0 con 0x7f865815db10 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f865e1bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865815e140 con 0x7f8658159c60 2023-10-08T16:13:52.473 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f8657fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f865815db10 0x7f8658069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f86577fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8658159890 0x7f865815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55632/0 (socket says 172.21.15.170:55632) 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f86577fe700 1 -- 172.21.15.170:0/1521330955 learned_addr learned my addr 172.21.15.170:0/1521330955 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.470+0000 7f8656ffd700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8658159c60 0x7f865815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f8657fff700 1 -- 172.21.15.170:0/1521330955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8658159c60 msgr2=0x7f865815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f8657fff700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8658159c60 0x7f865815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f8657fff700 1 -- 172.21.15.170:0/1521330955 >> v1:172.21.15.170:6789/0 conn(0x7f8658159890 legacy=0x7f865815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f86567fc700 1 -- 172.21.15.170:0/1521330955 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583653985 0 0) 0x7f865815a160 con 0x7f8658159890 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f8657fff700 1 -- 172.21.15.170:0/1521330955 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865815a3d0 con 0x7f865815db10 2023-10-08T16:13:52.474 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f8657fff700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f865815db10 0x7f8658069560 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f86500098e0 tx=0x7f86500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95322bd3ccf08bdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f86567fc700 1 -- 172.21.15.170:0/1521330955 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650009e80 con 0x7f865815db10 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f86567fc700 1 -- 172.21.15.170:0/1521330955 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865000b860 con 0x7f865815db10 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f86567fc700 1 -- 172.21.15.170:0/1521330955 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000bbe0 con 0x7f865815db10 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 -- 172.21.15.170:0/1521330955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f865815db10 msgr2=0x7f8658069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f865815db10 0x7f8658069560 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f86500098e0 tx=0x7f86500091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 -- 172.21.15.170:0/1521330955 shutdown_connections 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f865815db10 0x7f8658069560 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1521330955 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8658159c60 0x7f865815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.475 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 -- 172.21.15.170:0/1521330955 >> 172.21.15.170:0/1521330955 conn(0x7f8658068e20 msgr2=0x7f86580d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 -- 172.21.15.170:0/1521330955 shutdown_connections 2023-10-08T16:13:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.471+0000 7f865e1bb700 1 -- 172.21.15.170:0/1521330955 wait complete. 2023-10-08T16:13:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 Processor -- start 2023-10-08T16:13:52.476 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 -- start start 2023-10-08T16:13:52.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8658159890 0x7f865814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 0x7f86581582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658152540 con 0x7f8658159c60 2023-10-08T16:13:52.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8658151930 con 0x7f8658159890 2023-10-08T16:13:52.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f865e1bb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8658151c10 con 0x7f865815db10 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.473+0000 7f8656ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52400/0 (socket says 172.21.15.170:52400) 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 -- 172.21.15.170:0/1851129570 learned_addr learned my addr 172.21.15.170:0/1851129570 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8657fff700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 0x7f86581582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f86577fe700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8658159890 0x7f865814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 -- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 msgr2=0x7f86581582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 0x7f86581582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 -- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8658159890 msgr2=0x7f865814c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.489 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8658159890 0x7f865814c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 -- 172.21.15.170:0/1851129570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650009420 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8657fff700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 0x7f86581582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f8656ffd700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f864c00d860 tx=0x7f864c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f863f7fe700 1 -- 172.21.15.170:0/1851129570 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00b630 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f863f7fe700 1 -- 172.21.15.170:0/1851129570 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c00b790 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8658151ef0 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f863f7fe700 1 -- 172.21.15.170:0/1851129570 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c00bb10 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.474+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865814c930 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.475+0000 7f863f7fe700 1 -- 172.21.15.170:0/1851129570 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f864c01e460 con 0x7f8658159c60 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.476+0000 7f863f7fe700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 0x7f8644069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.476+0000 7f863f7fe700 1 -- 172.21.15.170:0/1851129570 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f864c085380 con 0x7f8658159c60 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.476+0000 7f86577fe700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 0x7f8644069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.476+0000 7f86577fe700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 0x7f8644069800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8648002220 tx=0x7f8648009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 Processor -- start 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 -- start start 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e13889e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e13885cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e137e0a70 con 0x561e138d3d00 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.478+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e137de270 con 0x561e138b2f60 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e137da060 con 0x561e13889a50 2023-10-08T16:13:52.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52414/0 (socket says 172.21.15.170:52414) 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 -- 172.21.15.170:0/3440202715 learned_addr learned my addr 172.21.15.170:0/3440202715 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8657fff700 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e13885cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f86577fe700 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e13889e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 -- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 msgr2=0x561e13889e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e13889e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 -- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 msgr2=0x561e13885cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e13885cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 -- 172.21.15.170:0/3440202715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e13674270 con 0x561e138d3d00 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f8656ffd700 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f864c006f00 tx=0x7f864c055b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d369031dd91b142 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f863cff9700 1 -- 172.21.15.170:0/3440202715 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c058fa0 con 0x561e138d3d00 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f863cff9700 1 -- 172.21.15.170:0/3440202715 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c05cc70 con 0x561e138d3d00 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.479+0000 7f863cff9700 1 -- 172.21.15.170:0/3440202715 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c055f30 con 0x561e138d3d00 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- 172.21.15.170:0/3440202715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 msgr2=0x561e138b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f864c006f00 tx=0x7f864c055b50 comp rx=0 tx=0).stop 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- 172.21.15.170:0/3440202715 shutdown_connections 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e13885cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e13889e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 --2- 172.21.15.170:0/3440202715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138b2a20 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- 172.21.15.170:0/3440202715 >> 172.21.15.170:0/3440202715 conn(0x561e1388b370 msgr2=0x561e1388b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- 172.21.15.170:0/3440202715 shutdown_connections 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- 172.21.15.170:0/3440202715 wait complete. 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.480+0000 7f86674b3b80 1 -- start start 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 Processor -- start 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e138cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e13785890 con 0x561e138d3d00 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e137a0ea0 con 0x561e138b2f60 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e1365ce30 con 0x561e13889a50 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86674b3b80 10 client.0 _ll_register_callbacks cb 0x561e1384d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f8657fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f8657fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52422/0 (socket says 172.21.15.170:52422) 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f8657fff700 1 -- 172.21.15.170:0/602845378 learned_addr learned my addr 172.21.15.170:0/602845378 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f86577fe700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e138cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.481+0000 7f8656ffd700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 -- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 msgr2=0x561e138cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e138cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 -- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 msgr2=0x561e138e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 -- 172.21.15.170:0/602845378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865814c930 con 0x561e138d3d00 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8657fff700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8650013000 tx=0x7f8650017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423fba4dc2c2cd74 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650021070 con 0x561e138d3d00 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f865001b640 con 0x561e138d3d00 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e1387eb70 con 0x561e138d3d00 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865001ba30 con 0x561e138d3d00 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 10 client.9153 fetch_fsmap learned FSMap version 131 2023-10-08T16:13:52.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x561e13616fe0 con 0x561e138d3d00 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e1387eb70 con 0x561e138d3d00 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 131) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f865001bb90 con 0x561e138d3d00 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 10 client.9153 fetch_fsmap finished waiting for FSMap version 131 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 10 client.9153 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 20 client.9153 populate_metadata read hostname 'smithi170' 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e13884d20 0x561e138850f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 4 client.9153 mds_command: new command op to 8997 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.482+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 --> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e1385baa0 con 0x561e13884d20 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f86577fe700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e13884d20 0x561e138850f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f86577fe700 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e13884d20 0x561e138850f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95d8a7e52eb3d56 server_cookie=a64088631474cd57 in_seq=0 out_seq=0 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f8635ffb700 10 client.9153 ms_handle_connect on v2:172.21.15.170:6835/1430346956 2023-10-08T16:13:52.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f8635ffb700 1 -- 172.21.15.170:0/602845378 <== mds.0 v2:172.21.15.170:6835/1430346956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561e1385baa0 con 0x561e13884d20 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f8635ffb700 10 client.9153 handle_command_reply: tid=0 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.483+0000 7f86674b3b80 1 client.9153 shutdown 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 msgr2=0x561e138d28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f8650013000 tx=0x7f8650017000 comp rx=0 tx=0).stop 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 shutdown_connections 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e13884d20 0x561e138850f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d28e0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e13889a50 0x561e138cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.498 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f8635ffb700 0 client.9153 ms_handle_reset on v2:172.21.15.170:6835/1430346956 2023-10-08T16:13:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 --2- 172.21.15.170:0/602845378 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 >> 172.21.15.170:0/602845378 conn(0x561e1388b370 msgr2=0x561e13886850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 shutdown_connections 2023-10-08T16:13:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 1 -- 172.21.15.170:0/602845378 wait complete. 2023-10-08T16:13:52.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.484+0000 7f86674b3b80 20 client.9153 trim_cache size 0 max 16384 2023-10-08T16:13:52.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.500+0000 7f86674b3b80 1 Processor -- start 2023-10-08T16:13:52.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.500+0000 7f86674b3b80 1 -- start start 2023-10-08T16:13:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138d3d00 0x561e138cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e138d1330 con 0x561e13889a50 2023-10-08T16:13:52.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e1388a720 con 0x561e138b2f60 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e1388a940 con 0x561e138d3d00 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52438/0 (socket says 172.21.15.170:52438) 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 -- 172.21.15.170:0/2790547594 learned_addr learned my addr 172.21.15.170:0/2790547594 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 -- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138d3d00 msgr2=0x561e138cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138d3d00 0x561e138cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f8656ffd700 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 -- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 msgr2=0x561e138cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.501+0000 7f86577fe700 1 -- 172.21.15.170:0/2790547594 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e137a8780 con 0x561e13889a50 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86577fe700 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x561e138856f0 tx=0x7f8648012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152871ba4fe3a6c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f863d7fa700 1 -- 172.21.15.170:0/2790547594 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864800dbb0 con 0x561e13889a50 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f863d7fa700 1 -- 172.21.15.170:0/2790547594 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86480260b0 con 0x561e13889a50 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f863d7fa700 1 -- 172.21.15.170:0/2790547594 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648026cc0 con 0x561e13889a50 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 -- 172.21.15.170:0/2790547594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 msgr2=0x561e138cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x561e138856f0 tx=0x7f8648012000 comp rx=0 tx=0).stop 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 -- 172.21.15.170:0/2790547594 shutdown_connections 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138d3d00 0x561e138cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e13889a50 0x561e138cf420 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2790547594 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e138b2f60 0x561e138cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 -- 172.21.15.170:0/2790547594 >> 172.21.15.170:0/2790547594 conn(0x561e1388b370 msgr2=0x561e138824b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 -- 172.21.15.170:0/2790547594 shutdown_connections 2023-10-08T16:13:52.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.502+0000 7f86674b3b80 1 -- 172.21.15.170:0/2790547594 wait complete. 2023-10-08T16:13:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.503+0000 7f86674b3b80 1 -- start start 2023-10-08T16:13:52.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.503+0000 7f86674b3b80 1 Processor -- start 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e13889a50 0x561e138e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 0x561e138cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e138e54e0 con 0x561e138d3d00 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e138e62c0 con 0x561e13889a50 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e13936ed0 con 0x561e138b2f60 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 10 client.0 _ll_register_callbacks cb 0x561e1384d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52440/0 (socket says 172.21.15.170:52440) 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 -- 172.21.15.170:0/2907121914 learned_addr learned my addr 172.21.15.170:0/2907121914 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86577fe700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e13889a50 0x561e138e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8656ffd700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 0x561e138cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 -- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 msgr2=0x561e138cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 0x561e138cfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 -- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e13889a50 msgr2=0x561e138e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e13889a50 0x561e138e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 -- 172.21.15.170:0/2907121914 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864801c040 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8656ffd700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 0x561e138cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8657fff700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8650001eb0 tx=0x7f8650021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cfbe10f2525a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8635ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865000bbe0 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mon.0 v2:172.21.15.170:3300/0 2 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f865000bd40 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f864c05cc70 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f865001a980 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 10 client.9159 fetch_fsmap learned FSMap version 131 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.504+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=131,monmap=2+}) v3 -- 0x561e1378f470 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mon.0 v2:172.21.15.170:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c05cc70 con 0x561e138d3d00 2023-10-08T16:13:52.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mon.0 v2:172.21.15.170:3300/0 5 ==== fsmap(e 131) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f865001aae0 con 0x561e138d3d00 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 10 client.9159 fetch_fsmap finished waiting for FSMap version 131 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 10 client.9159 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 20 client.9159 populate_metadata read hostname 'smithi170' 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e138d4670 0x561e138d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 4 client.9159 mds_command: new command op to 8997 tid=0[{"prefix": "status"}] 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 --> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e1377ce00 con 0x561e138d4670 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86577fe700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e138d4670 0x561e138d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 10 client.9159 ms_handle_connect on v2:172.21.15.170:6835/1430346956 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86577fe700 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e138d4670 0x561e138d4a40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7613035bdeb485 server_cookie=e798ca59a1ba0a11 in_seq=0 out_seq=0 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 1 -- 172.21.15.170:0/2907121914 <== mds.0 v2:172.21.15.170:6835/1430346956 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x561e1377ce00 con 0x561e138d4670 2023-10-08T16:13:52.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 10 client.9159 handle_command_reply: tid=0 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 client.9159 shutdown 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 msgr2=0x561e138d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8650001eb0 tx=0x7f8650021000 comp rx=0 tx=0).stop 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 shutdown_connections 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:6835/1430346956,v1:172.21.15.170:6837/1430346956] conn(0x561e138d4670 0x561e138d4a40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561e138d3d00 0x561e138d0400 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f8635ffb700 0 client.9159 ms_handle_reset on v2:172.21.15.170:6835/1430346956 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x561e13889a50 0x561e138e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 --2- 172.21.15.170:0/2907121914 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x561e138b2f60 0x561e138cfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.505+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 >> 172.21.15.170:0/2907121914 conn(0x561e1388b370 msgr2=0x561e13885c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.514 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 shutdown_connections 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f86674b3b80 1 -- 172.21.15.170:0/2907121914 wait complete. 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f86674b3b80 20 client.9159 trim_cache size 0 max 16384 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "cluster_fsid": "a2f97f1a-22f1-43be-8b23-d4161b44866d", 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "whoami": 0, 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "id": 8997, 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "want_state": "up:active", 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "state": "up:active", 2023-10-08T16:13:52.515 INFO:teuthology.orchestra.run.smithi170.stdout: "fs_name": "cephfs", 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout: "rank_uptime": 5.8051595689999997, 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout: "mdsmap_epoch": 131, 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch": 201, 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_epoch_barrier": 201, 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout: "uptime": 10.912761967 2023-10-08T16:13:52.516 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 msgr2=0x7f8644069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 0x7f8644069800 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8648002220 tx=0x7f8648009040 comp rx=0 tx=0).stop 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 msgr2=0x7f8658149e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f864c00d860 tx=0x7f864c008040 comp rx=0 tx=0).stop 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 shutdown_connections 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f865815db10 0x7f86581582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8658159890 0x7f865814c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8644067350 0x7f8644069800 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.517 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 --2- 172.21.15.170:0/1851129570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8658159c60 0x7f8658149e50 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 >> 172.21.15.170:0/1851129570 conn(0x7f8658068e20 msgr2=0x7f865814d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 shutdown_connections 2023-10-08T16:13:52.518 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.506+0000 7f865e1bb700 1 -- 172.21.15.170:0/1851129570 wait complete. 2023-10-08T16:13:52.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:13:52.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.633+0000 7f12dcc15700 1 Processor -- start 2023-10-08T16:13:52.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.633+0000 7f12dcc15700 1 -- start start 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12dcc15700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f12d8069560 0x7f12d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12dcc15700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d8069ee0 con 0x7f12d8159890 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12dcc15700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d815a160 con 0x7f12d8159c60 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12dcc15700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80d13a0 con 0x7f12d8069560 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d659c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f12d8159890 0x7f12d815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55634/0 (socket says 172.21.15.170:55634) 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d659c700 1 -- 172.21.15.170:0/1659200608 learned_addr learned my addr 172.21.15.170:0/1659200608 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d6d9d700 1 --2- 172.21.15.170:0/1659200608 >> v2:172.21.15.191:3300/0 conn(0x7f12d8069560 0x7f12d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488245552 0 0) 0x7f12d8069ee0 con 0x7f12d8159890 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c0003570 con 0x7f12d8159890 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015996737 0 0) 0x7f12d815a160 con 0x7f12d8159c60 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d8069ee0 con 0x7f12d8159c60 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d6d9d700 1 -- 172.21.15.170:0/1659200608 >> v1:172.21.15.191:6789/0 conn(0x7f12d8159c60 legacy=0x7f12d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d6d9d700 1 -- 172.21.15.170:0/1659200608 >> v1:172.21.15.170:6789/0 conn(0x7f12d8159890 legacy=0x7f12d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.634+0000 7f12d6d9d700 1 -- 172.21.15.170:0/1659200608 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d815e140 con 0x7f12d8069560 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12d6d9d700 1 --2- 172.21.15.170:0/1659200608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8069560 0x7f12d80699a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009e60 tx=0x7f12c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae3c76e84e56189f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800e070 con 0x7f12d8069560 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c800bab0 con 0x7f12d8069560 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12d559a700 1 -- 172.21.15.170:0/1659200608 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c800be30 con 0x7f12d8069560 2023-10-08T16:13:52.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 -- 172.21.15.170:0/1659200608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8069560 msgr2=0x7f12d80699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 --2- 172.21.15.170:0/1659200608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8069560 0x7f12d80699a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f12c8009e60 tx=0x7f12c8009780 comp rx=0 tx=0).stop 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 -- 172.21.15.170:0/1659200608 shutdown_connections 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 --2- 172.21.15.170:0/1659200608 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8069560 0x7f12d80699a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 -- 172.21.15.170:0/1659200608 >> 172.21.15.170:0/1659200608 conn(0x7f12d8068e20 msgr2=0x7f12d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 -- 172.21.15.170:0/1659200608 shutdown_connections 2023-10-08T16:13:52.638 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.635+0000 7f12dcc15700 1 -- 172.21.15.170:0/1659200608 wait complete. 2023-10-08T16:13:52.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.636+0000 7f12dcc15700 1 Processor -- start 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 -- start start 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 0x7f12d80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12d8159c60 0x7f12d80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80cdb00 con 0x7f12d8069560 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d81558a0 con 0x7f12d8159890 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12dcc15700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80caad0 con 0x7f12d8159c60 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12d659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12d659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52448/0 (socket says 172.21.15.170:52448) 2023-10-08T16:13:52.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12d659c700 1 -- 172.21.15.170:0/2955607325 learned_addr learned my addr 172.21.15.170:0/2955607325 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12d6d9d700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12d8159c60 0x7f12d80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.637+0000 7f12d5d9b700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 0x7f12d80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 -- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12d8159c60 msgr2=0x7f12d80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12d8159c60 0x7f12d80b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 -- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 msgr2=0x7f12d80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 0x7f12d80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 -- 172.21.15.170:0/2955607325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c8009a20 con 0x7f12d8069560 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d5d9b700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 0x7f12d80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12d659c700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c790 tx=0x7f12d0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0005b30 con 0x7f12d8069560 2023-10-08T16:13:52.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0005c90 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d815d390 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0007300 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.638+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d815d950 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.639+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d80cc530 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.639+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f12d001b400 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.640+0000 7f12c67fc700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 0x7f12b80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.640+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f12d0084710 con 0x7f12d8069560 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.640+0000 7f12d5d9b700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 0x7f12b80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:52.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.640+0000 7f12d5d9b700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 0x7f12b80697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f12d80cd7e0 tx=0x7f12cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:52.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.644+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12d0054fe0 con 0x7f12d8069560 2023-10-08T16:13:52.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.871+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12d804c3c0 con 0x7f12d8069560 2023-10-08T16:13:52.872 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.872+0000 7f12c67fc700 1 -- 172.21.15.170:0/2955607325 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12d00076f0 con 0x7f12d8069560 2023-10-08T16:13:52.872 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 msgr2=0x7f12b80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 0x7f12b80697d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f12d80cd7e0 tx=0x7f12cc009040 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 msgr2=0x7f12d8155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f12d000c790 tx=0x7f12d0007ea0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 shutdown_connections 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f12d8159c60 0x7f12d80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f12b8067320 0x7f12b80697d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12d8069560 0x7f12d8155360 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 --2- 172.21.15.170:0/2955607325 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f12d8159890 0x7f12d80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 >> 172.21.15.170:0/2955607325 conn(0x7f12d8068e20 msgr2=0x7f12d80c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:52.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 shutdown_connections 2023-10-08T16:13:52.876 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:52.874+0000 7f12dcc15700 1 -- 172.21.15.170:0/2955607325 wait complete. 2023-10-08T16:13:52.886 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:13:52.886 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:13:52.886 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi170.front.sepia.ceph.com 2023-10-08T16:13:52.886 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T16:13:52.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:52.887 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:13:52.887 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:13:52.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:13:52.887 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:52.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:52.887 DEBUG:teuthology.orchestra.run.smithi170:> ip addr 2023-10-08T16:13:52.902 INFO:teuthology.orchestra.run.smithi170.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:13:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:13:52.902 INFO:teuthology.orchestra.run.smithi170.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 ::1/128 scope host 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f0 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:d9:3b:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: altname enp7s0f1 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:52.903 INFO:teuthology.orchestra.run.smithi170.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 0c:c4:7a:88:72:55 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: inet 172.21.15.170/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft 41918sec preferred_lft 41918sec 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::ec4:7aff:fe88:7255/64 scope link 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::207d:74ff:fe24:6765/64 scope link 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:52.904 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:53:08:09:0d:54 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::6853:8ff:fe09:d54/64 scope link 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: link/ether 6a:fb:69:cc:19:a8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: inet6 fe80::68fb:69ff:fecc:19a8/64 scope link 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run.smithi170.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:52.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:52.905 DEBUG:teuthology.orchestra.run.smithi170:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:13:52.965 INFO:teuthology.orchestra.run.smithi170.stdout:1 2023-10-08T16:13:52.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:52.966 DEBUG:teuthology.orchestra.run.smithi170:> route 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:Kernel IP routing table 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run.smithi170.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:13:53.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> (cd / && exec bash -c ' 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> set -e 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:13:53.023 DEBUG:teuthology.orchestra.run.smithi170:> ') 2023-10-08T16:13:53.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.122 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:13:53.136 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:13:53.136 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:13:53.136 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:53.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.136 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link set brx.0 up 2023-10-08T16:13:53.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:53.201 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:53.254 INFO:teuthology.orchestra.run.smithi170.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T16:13:53.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:53.255 DEBUG:teuthology.orchestra.run.smithi170:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:53.309 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T16:13:53.309 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:13:53.309 DEBUG:teuthology.orchestra.run.smithi170:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:53.388 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:53.388 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:53.407 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:53.408 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:53.429 INFO:teuthology.orchestra.run.smithi170.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:53.429 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with new device syntax: 0@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:13:53.430 INFO:teuthology.orchestra.run.smithi170.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:13:53.432 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:13:53.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.432 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:53.497 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T16:13:53.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.653+0000 7f6fc24bd700 1 Processor -- start 2023-10-08T16:13:53.654 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.653+0000 7f6fc24bd700 1 -- start start 2023-10-08T16:13:53.656 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fc24bd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6fbc0b5ea0 0x7f6fbc068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fc24bd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 0x7f6fbc15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fc24bd700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc15c480 con 0x7f6fbc159c10 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fc24bd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b2a40 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fc24bd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc15b870 con 0x7f6fbc068bb0 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fbb7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6fbc0b5ea0 0x7f6fbc068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.657 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fbb7fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6fbc0b5ea0 0x7f6fbc068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52458/0 (socket says 172.21.15.170:52458) 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.654+0000 7f6fbb7fe700 1 -- 172.21.15.170:0/1959037799 learned_addr learned my addr 172.21.15.170:0/1959037799 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc0a5a700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 0x7f6fbc15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbb7fe700 1 -- 172.21.15.170:0/1959037799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 msgr2=0x7f6fbc15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbb7fe700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 0x7f6fbc15c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbb7fe700 1 -- 172.21.15.170:0/1959037799 >> v1:172.21.15.191:6789/0 conn(0x7f6fbc159c10 legacy=0x7f6fbc15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbb7fe700 1 -- 172.21.15.170:0/1959037799 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc15bb50 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc0a5a700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 0x7f6fbc15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:53.658 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbb7fe700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc068670 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323a20f643b2cad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbaffd700 1 -- 172.21.15.170:0/1959037799 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000b820 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbaffd700 1 -- 172.21.15.170:0/1959037799 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000b980 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fbaffd700 1 -- 172.21.15.170:0/1959037799 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000bd20 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1959037799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc0b5ea0 msgr2=0x7f6fbc068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc068670 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0009250 comp rx=0 tx=0).stop 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1959037799 shutdown_connections 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc068bb0 0x7f6fbc15c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1959037799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc068670 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.659 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1959037799 >> 172.21.15.170:0/1959037799 conn(0x7f6fbc099a20 msgr2=0x7f6fbc0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:53.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.655+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1959037799 shutdown_connections 2023-10-08T16:13:53.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.656+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1959037799 wait complete. 2023-10-08T16:13:53.660 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 Processor -- start 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 -- start start 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc09a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc159c10 0x7f6fbc09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc16cb40 con 0x7f6fbc068bb0 2023-10-08T16:13:53.664 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c9180 con 0x7f6fbc0b5ea0 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.657+0000 7f6fc24bd700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc16cd90 con 0x7f6fbc159c10 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52474/0 (socket says 172.21.15.170:52474) 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 -- 172.21.15.170:0/1657882749 learned_addr learned my addr 172.21.15.170:0/1657882749 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbb7fe700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc09a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fc0a5a700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc159c10 0x7f6fbc09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 -- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc159c10 msgr2=0x7f6fbc09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc159c10 0x7f6fbc09a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 -- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fbc0b5ea0 msgr2=0x7f6fbc09a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc09a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 -- 172.21.15.170:0/1657882749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb00094f0 con 0x7f6fbc068bb0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fbbfff700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00b5e0 tx=0x7f6fac009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00f510 con 0x7f6fbc068bb0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac00f670 con 0x7f6fbc068bb0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc16d070 con 0x7f6fbc068bb0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac00fa30 con 0x7f6fbc068bb0 2023-10-08T16:13:53.666 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.658+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0c8d50 con 0x7f6fbc068bb0 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.659+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc04c3c0 con 0x7f6fbc068bb0 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.660+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f6fac01a400 con 0x7f6fbc068bb0 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.660+0000 7f6fa3fff700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 0x7f6fa80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.660+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6fac084720 con 0x7f6fbc068bb0 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.660+0000 7f6fbb7fe700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 0x7f6fa80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:53.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.660+0000 7f6fbb7fe700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 0x7f6fa80697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:53.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.664+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6fac0514b0 con 0x7f6fbc068bb0 2023-10-08T16:13:53.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.895+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6fbc15be30 con 0x7f6fbc068bb0 2023-10-08T16:13:53.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.895+0000 7f6fa3fff700 1 -- 172.21.15.170:0/1657882749 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v131) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6fac054ad0 con 0x7f6fbc068bb0 2023-10-08T16:13:53.896 INFO:teuthology.orchestra.run.smithi170.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T16:13:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 msgr2=0x7f6fa80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 0x7f6fa80697d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6fb00099b0 tx=0x7f6fb0016000 comp rx=0 tx=0).stop 2023-10-08T16:13:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 msgr2=0x7f6fbc0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:53.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f6fac00b5e0 tx=0x7f6fac009120 comp rx=0 tx=0).stop 2023-10-08T16:13:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 shutdown_connections 2023-10-08T16:13:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f6fbc159c10 0x7f6fbc09a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6fa8067320 0x7f6fa80697d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6fbc068bb0 0x7f6fbc0c8810 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 --2- 172.21.15.170:0/1657882749 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f6fbc0b5ea0 0x7f6fbc09a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 >> 172.21.15.170:0/1657882749 conn(0x7f6fbc099a20 msgr2=0x7f6fbc09b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 shutdown_connections 2023-10-08T16:13:53.900 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:53.897+0000 7f6fc24bd700 1 -- 172.21.15.170:0/1657882749 wait complete. 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi191.front.sepia.ceph.com 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T16:13:53.910 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:53.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.911 DEBUG:teuthology.orchestra.run.smithi191:> ip addr 2023-10-08T16:13:53.929 INFO:teuthology.orchestra.run.smithi191.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 ::1/128 scope host 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f6 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f0 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:53.930 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:d9:37:f7 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: altname enp7s0f1 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:10 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 0c:c4:7a:88:72:11 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: inet 172.21.15.191/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft 41917sec preferred_lft 41917sec 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::ec4:7aff:fe88:7211/64 scope link 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T16:13:53.931 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::708d:3bff:febd:7ae9/64 scope link 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 1e:89:7f:50:88:0c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::1c89:7fff:fe50:880c/64 scope link 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: link/ether 02:fc:67:c9:8a:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:13:53.932 INFO:teuthology.orchestra.run.smithi191.stdout: inet6 fe80::fc:67ff:fec9:8a24/64 scope link 2023-10-08T16:13:53.933 INFO:teuthology.orchestra.run.smithi191.stdout: valid_lft forever preferred_lft forever 2023-10-08T16:13:53.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.933 DEBUG:teuthology.orchestra.run.smithi191:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T16:13:53.997 INFO:teuthology.orchestra.run.smithi191.stdout:1 2023-10-08T16:13:53.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:53.999 DEBUG:teuthology.orchestra.run.smithi191:> route 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run.smithi191.stdout:Kernel IP routing table 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run.smithi191.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run.smithi191.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run.smithi191.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run.smithi191.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T16:13:54.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> (cd / && exec bash -c ' 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> set -e 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T16:13:54.059 DEBUG:teuthology.orchestra.run.smithi191:> ') 2023-10-08T16:13:54.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:54.165 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:13:54.181 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:13:54.181 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:13:54.181 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:54.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:54.182 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link set brx.0 up 2023-10-08T16:13:54.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:54.247 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:54.302 INFO:teuthology.orchestra.run.smithi191.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T16:13:54.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T16:13:54.302 DEBUG:teuthology.orchestra.run.smithi191:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:54.356 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T16:13:54.357 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T16:13:54.357 DEBUG:teuthology.orchestra.run.smithi191:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:54.437 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:54.437 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:54.456 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:54.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:54.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:54.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:54.456 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:54.457 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:54.457 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:54.457 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:3300,172.21.15.191:3300,172.21.15.191:3301" 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with new device syntax: 1@a2f97f1a-22f1-43be-8b23-d4161b44866d.cephfs=/ 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.170:3300/172.21.15.191:3300/172.21.15.191:3301" will pass to kernel 2023-10-08T16:13:54.478 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: v2 mount support check returned 2 2023-10-08T16:13:54.479 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: kernel does not support v2 syntax 2023-10-08T16:13:54.479 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-08T16:13:54.479 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: resolved to: "172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790" 2023-10-08T16:13:54.479 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.170:6789,172.21.15.191:6789,172.21.15.191:6790:/ 2023-10-08T16:13:54.479 INFO:teuthology.orchestra.run.smithi191.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=a2f97f1a-22f1-43be-8b23-d4161b44866d" will pass to kernel 2023-10-08T16:13:54.480 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T16:13:54.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:54.480 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:54.544 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-10-08T16:13:54.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.664+0000 7fa109f64700 1 Processor -- start 2023-10-08T16:13:54.665 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.664+0000 7fa109f64700 1 -- start start 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109f64700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa1040cca20 0x7fa1040cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109f64700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040cfa00 con 0x7fa1040b0790 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109f64700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040cb580 con 0x7fa1040b03c0 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109f64700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040cf160 con 0x7fa1040cca20 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109763700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fa1040cca20 0x7fa1040cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa103fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa1040b0790 0x7fa1041591e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55636/0 (socket says 172.21.15.170:55636) 2023-10-08T16:13:54.667 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa103fff700 1 -- 172.21.15.170:0/762340690 learned_addr learned my addr 172.21.15.170:0/762340690 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109763700 1 -- 172.21.15.170:0/762340690 >> v1:172.21.15.191:6789/0 conn(0x7fa1040b03c0 legacy=0x7fa1040b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109763700 1 -- 172.21.15.170:0/762340690 >> v1:172.21.15.170:6789/0 conn(0x7fa1040b0790 legacy=0x7fa1041591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa1037fe700 1 -- 172.21.15.170:0/762340690 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196826587 0 0) 0x7fa1040cfa00 con 0x7fa1040b0790 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.665+0000 7fa109763700 1 -- 172.21.15.170:0/762340690 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040cf440 con 0x7fa1040cca20 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109763700 1 --2- 172.21.15.170:0/762340690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040cca20 0x7fa1040cea70 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e90 tx=0x7fa0fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b39ca7f140792 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa1037fe700 1 -- 172.21.15.170:0/762340690 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc00e070 con 0x7fa1040cca20 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa1037fe700 1 -- 172.21.15.170:0/762340690 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00bbf0 con 0x7fa1040cca20 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa1037fe700 1 -- 172.21.15.170:0/762340690 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc012430 con 0x7fa1040cca20 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 -- 172.21.15.170:0/762340690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040cca20 msgr2=0x7fa1040cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 --2- 172.21.15.170:0/762340690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040cca20 0x7fa1040cea70 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009e90 tx=0x7fa0fc009780 comp rx=0 tx=0).stop 2023-10-08T16:13:54.668 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 -- 172.21.15.170:0/762340690 shutdown_connections 2023-10-08T16:13:54.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 --2- 172.21.15.170:0/762340690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040cca20 0x7fa1040cea70 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:54.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 -- 172.21.15.170:0/762340690 >> 172.21.15.170:0/762340690 conn(0x7fa104063230 msgr2=0x7fa104063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:54.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 -- 172.21.15.170:0/762340690 shutdown_connections 2023-10-08T16:13:54.669 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.666+0000 7fa109f64700 1 -- 172.21.15.170:0/762340690 wait complete. 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.667+0000 7fa109f64700 1 Processor -- start 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 -- start start 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 0x7fa1040c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1040cca20 0x7fa1040c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040c9130 con 0x7fa1040b03c0 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104169880 con 0x7fa1040cca20 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109f64700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10415a2c0 con 0x7fa1040b0790 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa108f62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:54.672 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa108f62700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:52490/0 (socket says 172.21.15.170:52490) 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa108f62700 1 -- 172.21.15.170:0/335441273 learned_addr learned my addr 172.21.15.170:0/335441273 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa109763700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1040cca20 0x7fa1040c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.668+0000 7fa103fff700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 0x7fa1040c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 -- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 msgr2=0x7fa1040c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 0x7fa1040c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 -- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1040cca20 msgr2=0x7fa1040c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1040cca20 0x7fa1040c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 -- 172.21.15.170:0/335441273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc009a20 con 0x7fa1040b03c0 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa103fff700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 0x7fa1040c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa108f62700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400c540 tx=0x7fa0f400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:54.673 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4011590 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f4011710 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040c8890 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4011ad0 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.669+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1040c8e50 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.670+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104169650 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.670+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fa0f401b3b0 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.671+0000 7fa100ff9700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 0x7fa0f00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.671+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa0f4084100 con 0x7fa1040b03c0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.671+0000 7fa103fff700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 0x7fa0f00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:54.674 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.671+0000 7fa103fff700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 0x7fa0f00697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa1040c3cb0 tx=0x7fa0f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:54.675 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.675+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0f4050d90 con 0x7fa1040b03c0 2023-10-08T16:13:54.907 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:54.907+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa10404c3c0 con 0x7fa1040b03c0 2023-10-08T16:13:55.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.748+0000 7fa100ff9700 1 -- 172.21.15.170:0/335441273 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 136+0+0 (secure 0 0 0) 0x7fa0f40543b0 con 0x7fa1040b03c0 2023-10-08T16:13:55.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.750+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 msgr2=0x7fa0f00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.750+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 0x7fa0f00697d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa1040c3cb0 tx=0x7fa0f8009040 comp rx=0 tx=0).stop 2023-10-08T16:13:55.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 msgr2=0x7fa104159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400c540 tx=0x7fa0f400c850 comp rx=0 tx=0).stop 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 shutdown_connections 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fa1040cca20 0x7fa1040c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fa0f0067320 0x7fa0f00697d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1040b03c0 0x7fa104159d80 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 --2- 172.21.15.170:0/335441273 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fa1040b0790 0x7fa1040c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 >> 172.21.15.170:0/335441273 conn(0x7fa104063230 msgr2=0x7fa1040b4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 shutdown_connections 2023-10-08T16:13:55.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.751+0000 7fa109f64700 1 -- 172.21.15.170:0/335441273 wait complete. 2023-10-08T16:13:55.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T16:13:55.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.880+0000 7f4b9359e700 1 Processor -- start 2023-10-08T16:13:55.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.880+0000 7f4b9359e700 1 -- start start 2023-10-08T16:13:55.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9359e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4b94159420 0x7f4b940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9359e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4b9415d230 0x7f4b9415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9359e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b940b2980 con 0x7f4b941597f0 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9359e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9415db70 con 0x7f4b94159420 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9359e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b940b11d0 con 0x7f4b9415d230 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4b94159420 0x7f4b940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b91d9b700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4b941597f0 0x7f4b940b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48606/0 (socket says 172.21.15.170:48606) 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b91d9b700 1 -- 172.21.15.170:0/625861416 learned_addr learned my addr 172.21.15.170:0/625861416 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b92d9d700 1 --2- 172.21.15.170:0/625861416 >> v2:172.21.15.191:3300/0 conn(0x7f4b9415d230 0x7f4b9415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 -- 172.21.15.170:0/625861416 >> v2:172.21.15.191:3300/0 conn(0x7f4b9415d230 msgr2=0x7f4b9415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 --2- 172.21.15.170:0/625861416 >> v2:172.21.15.191:3300/0 conn(0x7f4b9415d230 0x7f4b9415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 -- 172.21.15.170:0/625861416 >> v1:172.21.15.170:6789/0 conn(0x7f4b941597f0 legacy=0x7f4b940b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 -- 172.21.15.170:0/625861416 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b940b14b0 con 0x7f4b94159420 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.881+0000 7f4b9259c700 1 --2- 172.21.15.170:0/625861416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b94159420 0x7f4b940b5f40 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4b9415d170 tx=0x7f4b840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e959c560676c6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9159a700 1 -- 172.21.15.170:0/625861416 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400b6b0 con 0x7f4b94159420 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9159a700 1 -- 172.21.15.170:0/625861416 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8400b810 con 0x7f4b94159420 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9159a700 1 -- 172.21.15.170:0/625861416 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8400bbb0 con 0x7f4b94159420 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 -- 172.21.15.170:0/625861416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b94159420 msgr2=0x7f4b940b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 --2- 172.21.15.170:0/625861416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b94159420 0x7f4b940b5f40 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f4b9415d170 tx=0x7f4b840091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 -- 172.21.15.170:0/625861416 shutdown_connections 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 --2- 172.21.15.170:0/625861416 >> v2:172.21.15.191:3300/0 conn(0x7f4b9415d230 0x7f4b9415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 --2- 172.21.15.170:0/625861416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b94159420 0x7f4b940b5f40 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 -- 172.21.15.170:0/625861416 >> 172.21.15.170:0/625861416 conn(0x7f4b940d0660 msgr2=0x7f4b940d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 -- 172.21.15.170:0/625861416 shutdown_connections 2023-10-08T16:13:55.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.882+0000 7f4b9359e700 1 -- 172.21.15.170:0/625861416 wait complete. 2023-10-08T16:13:55.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.883+0000 7f4b9359e700 1 Processor -- start 2023-10-08T16:13:55.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.883+0000 7f4b9359e700 1 -- start start 2023-10-08T16:13:55.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b94159420 0x7f4b94155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 0x7f4b9416a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b941566b0 con 0x7f4b941597f0 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9416a7b0 con 0x7f4b9415d230 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b9359e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9416aa90 con 0x7f4b94159420 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56998/0 (socket says 172.21.15.170:56998) 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 -- 172.21.15.170:0/989024075 learned_addr learned my addr 172.21.15.170:0/989024075 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b92d9d700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 0x7f4b9416a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 -- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b94159420 msgr2=0x7f4b94155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b94159420 0x7f4b94155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 -- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 msgr2=0x7f4b9416a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:55.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 0x7f4b9416a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b91d9b700 1 -- 172.21.15.170:0/989024075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84009420 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.884+0000 7f4b92d9d700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 0x7f4b9416a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b91d9b700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800c790 tx=0x7f4b88007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88005b30 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b88005cb0 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b88004a80 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b94159ca0 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.885+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9415a260 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.886+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9404c3c0 con 0x7f4b941597f0 2023-10-08T16:13:55.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.886+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4b88005030 con 0x7f4b941597f0 2023-10-08T16:13:55.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.887+0000 7f4b827fc700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 0x7f4b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:55.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.887+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4b88084610 con 0x7f4b941597f0 2023-10-08T16:13:55.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.887+0000 7f4b9259c700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 0x7f4b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:55.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.887+0000 7f4b9259c700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 0x7f4b740697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b940c6970 tx=0x7f4b84017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:55.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:55.891+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b880512a0 con 0x7f4b941597f0 2023-10-08T16:13:56.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.099+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b940b1790 con 0x7f4b74067320 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.100+0000 7f4b827fc700 1 -- 172.21.15.170:0/989024075 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4b940b1790 con 0x7f4b74067320 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stdout:[ 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stdout: { 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stdout: "name": "cephfs" 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stdout: } 2023-10-08T16:13:56.101 INFO:teuthology.orchestra.run.smithi170.stdout:] 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 msgr2=0x7f4b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 0x7f4b740697d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f4b940c6970 tx=0x7f4b84017000 comp rx=0 tx=0).stop 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 msgr2=0x7f4b94169bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800c790 tx=0x7f4b88007f40 comp rx=0 tx=0).stop 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 shutdown_connections 2023-10-08T16:13:56.103 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4b9415d230 0x7f4b9416a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f4b94159420 0x7f4b94155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4b74067320 0x7f4b740697d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 --2- 172.21.15.170:0/989024075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b941597f0 0x7f4b94169bd0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 >> 172.21.15.170:0/989024075 conn(0x7f4b940d0660 msgr2=0x7f4b94069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.102+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 shutdown_connections 2023-10-08T16:13:56.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.103+0000 7f4b9359e700 1 -- 172.21.15.170:0/989024075 wait complete. 2023-10-08T16:13:56.115 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T16:13:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.232+0000 7f63b59c5700 1 Processor -- start 2023-10-08T16:13:56.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.232+0000 7f63b59c5700 1 -- start start 2023-10-08T16:13:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63b59c5700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63b0159c60 0x7f63b015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63b59c5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 0x7f63b0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63b59c5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b015a160 con 0x7f63b0159890 2023-10-08T16:13:56.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63b59c5700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b015df50 con 0x7f63b0159c60 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63b59c5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00d13a0 con 0x7f63b015db10 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63ae7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63b0159c60 0x7f63b015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63aeffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f63b0159890 0x7f63b015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48614/0 (socket says 172.21.15.170:48614) 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63ae7fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63b0159c60 0x7f63b015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57002/0 (socket says 172.21.15.170:57002) 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.233+0000 7f63aeffd700 1 -- 172.21.15.170:0/3528706476 learned_addr learned my addr 172.21.15.170:0/3528706476 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63ae7fc700 1 -- 172.21.15.170:0/3528706476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 msgr2=0x7f63b0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63af7fe700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 0x7f63b0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63adffb700 1 -- 172.21.15.170:0/3528706476 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222155002 0 0) 0x7f63b015a160 con 0x7f63b0159890 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63ae7fc700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 0x7f63b0069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63ae7fc700 1 -- 172.21.15.170:0/3528706476 >> v1:172.21.15.170:6789/0 conn(0x7f63b0159890 legacy=0x7f63b015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63ae7fc700 1 -- 172.21.15.170:0/3528706476 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b015e140 con 0x7f63b0159c60 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63af7fe700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 0x7f63b0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63ae7fc700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b015d5d0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098e0 tx=0x7f63a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53db96d59bd1731 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63adffb700 1 -- 172.21.15.170:0/3528706476 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4009e80 con 0x7f63b0159c60 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63adffb700 1 -- 172.21.15.170:0/3528706476 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400b860 con 0x7f63b0159c60 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63adffb700 1 -- 172.21.15.170:0/3528706476 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400bbe0 con 0x7f63b0159c60 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63b59c5700 1 -- 172.21.15.170:0/3528706476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 msgr2=0x7f63b015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.234+0000 7f63b59c5700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b015d5d0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f63a40098e0 tx=0x7f63a40091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 -- 172.21.15.170:0/3528706476 shutdown_connections 2023-10-08T16:13:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b015db10 0x7f63b0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 --2- 172.21.15.170:0/3528706476 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b015d5d0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 -- 172.21.15.170:0/3528706476 >> 172.21.15.170:0/3528706476 conn(0x7f63b0068e20 msgr2=0x7f63b00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 -- 172.21.15.170:0/3528706476 shutdown_connections 2023-10-08T16:13:56.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.235+0000 7f63b59c5700 1 -- 172.21.15.170:0/3528706476 wait complete. 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.236+0000 7f63b59c5700 1 Processor -- start 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.236+0000 7f63b59c5700 1 -- start start 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.236+0000 7f63b59c5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 0x7f63b0151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.236+0000 7f63b59c5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 0x7f63b00b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b014b090 con 0x7f63b0159c60 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0152340 con 0x7f63b015db10 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0152620 con 0x7f63b0159890 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57010/0 (socket says 172.21.15.170:57010) 2023-10-08T16:13:56.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 -- 172.21.15.170:0/897694852 learned_addr learned my addr 172.21.15.170:0/897694852 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 -- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 msgr2=0x7f63b0151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63aeffd700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 0x7f63b0151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63af7fe700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 0x7f63b00b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 0x7f63b0151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 -- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 msgr2=0x7f63b00b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 0x7f63b00b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 -- 172.21.15.170:0/897694852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4009420 con 0x7f63b0159c60 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63af7fe700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 0x7f63b00b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63aeffd700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 0x7f63b0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63ae7fc700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f63b00899c0 tx=0x7f63a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a400bbe0 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a4011420 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b0152900 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40117e0 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.237+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b014aa20 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.238+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b004c3c0 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.239+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63a4011bd0 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.239+0000 7f6396ffd700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.239+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f63a40867e0 con 0x7f63b0159c60 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.239+0000 7f63aeffd700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.239+0000 7f63aeffd700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 0x7f639c0697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d1a0 tx=0x7f63a0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.243+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63a4053470 con 0x7f63b0159c60 2023-10-08T16:13:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.427+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f63b014ae60 con 0x7f63b0159c60 2023-10-08T16:13:56.427 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.427+0000 7f6396ffd700 1 -- 172.21.15.170:0/897694852 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63a4056a90 con 0x7f63b0159c60 2023-10-08T16:13:56.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 msgr2=0x7f639c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 0x7f639c0697d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d1a0 tx=0x7f63a0009040 comp rx=0 tx=0).stop 2023-10-08T16:13:56.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 msgr2=0x7f63b014a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.430 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f63b00899c0 tx=0x7f63a4017000 comp rx=0 tx=0).stop 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 shutdown_connections 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63b015db10 0x7f63b00b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f63b0159890 0x7f63b0151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f639c067320 0x7f639c0697d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.429+0000 7f63b59c5700 1 --2- 172.21.15.170:0/897694852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0159c60 0x7f63b014a4e0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.430+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 >> 172.21.15.170:0/897694852 conn(0x7f63b0068e20 msgr2=0x7f63b014d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.430+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 shutdown_connections 2023-10-08T16:13:56.431 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.430+0000 7f63b59c5700 1 -- 172.21.15.170:0/897694852 wait complete. 2023-10-08T16:13:56.442 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T16:13:56.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:13:56.442 DEBUG:teuthology.orchestra.run.smithi170:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T16:13:56.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:13:56.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:56.493 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T16:13:56.507 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T16:13:56.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T16:13:56.508 DEBUG:teuthology.orchestra.run.smithi191:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-10-08T16:13:56.558 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:13:56.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:56.558 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T16:13:56.575 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:13:56.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:56.575 DEBUG:teuthology.orchestra.run.smithi170:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T16:13:56.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:13:56.590 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-08T16:13:56.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:13:56.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:13:56.590 DEBUG:teuthology.orchestra.run.smithi191:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T16:13:56.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:13:56.631 INFO:teuthology.orchestra.run.smithi191.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-08T16:13:56.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs 2023-10-08T16:13:56.747 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.747+0000 7f3c1ffff700 1 Processor -- start 2023-10-08T16:13:56.748 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.747+0000 7f3c1ffff700 1 -- start start 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1ffff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f3c200b4e70 0x7f3c200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1ffff700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c200cfb00 con 0x7f3c200b16d0 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1ffff700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2015e060 con 0x7f3c200b1b40 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1ffff700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200b5c40 con 0x7f3c200b4e70 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1e7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3c200b16d0 0x7f3c2015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48626/0 (socket says 172.21.15.170:48626) 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1e7fc700 1 -- 172.21.15.170:0/3949695260 learned_addr learned my addr 172.21.15.170:0/3949695260 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.748+0000 7f3c1effd700 1 --2- 172.21.15.170:0/3949695260 >> v2:172.21.15.191:3300/0 conn(0x7f3c200b4e70 0x7f3c200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.749 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.749+0000 7f3c1effd700 1 -- 172.21.15.170:0/3949695260 >> v1:172.21.15.191:6789/0 conn(0x7f3c200b1b40 legacy=0x7f3c2015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.749+0000 7f3c1effd700 1 -- 172.21.15.170:0/3949695260 >> v1:172.21.15.170:6789/0 conn(0x7f3c200b16d0 legacy=0x7f3c2015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.749+0000 7f3c1effd700 1 -- 172.21.15.170:0/3949695260 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200b5f20 con 0x7f3c200b4e70 2023-10-08T16:13:56.750 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.749+0000 7f3c1effd700 1 --2- 172.21.15.170:0/3949695260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b4e70 0x7f3c200ccff0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098e0 tx=0x7f3c100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1419ee371496d56 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.751 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.751+0000 7f3c1dffb700 1 -- 172.21.15.170:0/3949695260 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c10009e80 con 0x7f3c200b4e70 2023-10-08T16:13:56.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.751+0000 7f3c1dffb700 1 -- 172.21.15.170:0/3949695260 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1000b860 con 0x7f3c200b4e70 2023-10-08T16:13:56.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.751+0000 7f3c1dffb700 1 -- 172.21.15.170:0/3949695260 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1000bbe0 con 0x7f3c200b4e70 2023-10-08T16:13:56.752 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3949695260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b4e70 msgr2=0x7f3c200ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 --2- 172.21.15.170:0/3949695260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b4e70 0x7f3c200ccff0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3c100098e0 tx=0x7f3c100091a0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3949695260 shutdown_connections 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 --2- 172.21.15.170:0/3949695260 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b4e70 0x7f3c200ccff0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3949695260 >> 172.21.15.170:0/3949695260 conn(0x7f3c200d10c0 msgr2=0x7f3c200affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3949695260 shutdown_connections 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.752+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3949695260 wait complete. 2023-10-08T16:13:56.753 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.753+0000 7f3c1ffff700 1 Processor -- start 2023-10-08T16:13:56.754 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.753+0000 7f3c1ffff700 1 -- start start 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b16d0 0x7f3c20169fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c200b1b40 0x7f3c2016cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c200b4e70 0x7f3c200c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200c5e20 con 0x7f3c200b4e70 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200caff0 con 0x7f3c200b16d0 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1ffff700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2016a510 con 0x7f3c200b1b40 2023-10-08T16:13:56.756 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c200b4e70 0x7f3c200c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c200b4e70 0x7f3c200c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57014/0 (socket says 172.21.15.170:57014) 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 -- 172.21.15.170:0/3134580855 learned_addr learned my addr 172.21.15.170:0/3134580855 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1effd700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b16d0 0x7f3c20169fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1e7fc700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c200b1b40 0x7f3c2016cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 -- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c200b1b40 msgr2=0x7f3c2016cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3c200b1b40 0x7f3c2016cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 -- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b16d0 msgr2=0x7f3c20169fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3c200b16d0 0x7f3c20169fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 -- 172.21.15.170:0/3134580855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10009420 con 0x7f3c200b4e70 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.754+0000 7f3c1f7fe700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c200b4e70 0x7f3c200c8c50 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800c8e0 tx=0x7f3c18001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.757 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18012630 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c18012790 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3134580855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c200c7ce0 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18012b50 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3134580855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c200c5760 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.755+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3134580855 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c200c5f60 con 0x7f3c200b4e70 2023-10-08T16:13:56.758 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.756+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3c1801a3a0 con 0x7f3c200b4e70 2023-10-08T16:13:56.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.756+0000 7f3c06ffd700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3c0c067320 0x7f3c0c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:56.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.756+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c18084de0 con 0x7f3c200b4e70 2023-10-08T16:13:56.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.756+0000 7f3c1effd700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3c0c067320 0x7f3c0c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:56.759 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.757+0000 7f3c1effd700 1 --2- 172.21.15.170:0/3134580855 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3c0c067320 0x7f3c0c0697d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3c100093f0 tx=0x7f3c10017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:56.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.760+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c200c5f60 con 0x7f3c200b4e70 2023-10-08T16:13:56.970 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.969+0000 7f3c1ffff700 1 -- 172.21.15.170:0/3134580855 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c200c5f60 con 0x7f3c0c067320 2023-10-08T16:13:56.970 INFO:tasks.ceph.mgr.y.smithi170.stderr:2023-10-08T16:13:56.970+0000 7f60a202f700 -1 mgr.server reply reply (1) Operation not permitted WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-08T16:13:56.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:56.970+0000 7f3c06ffd700 1 -- 172.21.15.170:0/3134580855 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: -1 WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it.) v1 ==== 204+0+0 (secure 0 0 0) 0x7f3c200c5f60 con 0x7f3c0c067320 2023-10-08T16:13:56.972 INFO:teuthology.orchestra.run.smithi170.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in the filesystem 'cephfs'. If you are *ABSOLUTELY CERTAIN* that is what you want, re-issue the command followed by --yes-i-really-mean-it. 2023-10-08T16:13:56.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:13:56.976 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2023-10-08T16:13:57.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.093+0000 7f03b2be5700 1 Processor -- start 2023-10-08T16:13:57.094 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.093+0000 7f03b2be5700 1 -- start start 2023-10-08T16:13:57.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b2be5700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f03ac159c60 0x7f03ac15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.097 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b2be5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f03ac069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b2be5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac15a160 con 0x7f03ac159890 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b2be5700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac15df50 con 0x7f03ac159c60 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b2be5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0d13a0 con 0x7f03ac15db10 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b0981700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f03ac159890 0x7f03ac15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48640/0 (socket says 172.21.15.170:48640) 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b0981700 1 -- 172.21.15.170:0/1883285786 learned_addr learned my addr 172.21.15.170:0/1883285786 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03b1182700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f03ac069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.094+0000 7f03a3fff700 1 --2- 172.21.15.170:0/1883285786 >> v2:172.21.15.191:3300/0 conn(0x7f03ac159c60 0x7f03ac15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619953938 0 0) 0x7f03ac15a160 con 0x7f03ac159890 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0394003570 con 0x7f03ac159890 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844612783 0 0) 0x7f0394003570 con 0x7f03ac159890 2023-10-08T16:13:57.098 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 msgr2=0x7f03ac069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f03ac069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 >> v2:172.21.15.191:3300/0 conn(0x7f03ac159c60 msgr2=0x7f03ac15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 --2- 172.21.15.170:0/1883285786 >> v2:172.21.15.191:3300/0 conn(0x7f03ac159c60 0x7f03ac15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac15e140 con 0x7f03ac159890 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f03a8001470 con 0x7f03ac159890 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 >> v1:172.21.15.170:6789/0 conn(0x7f03ac159890 legacy=0x7f03ac15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0394003a60 0x7f0394003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f0394004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394004d50 0x7f03940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0d13a0 con 0x7f0394003a60 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039400ba60 con 0x7f0394004d50 2023-10-08T16:13:57.099 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039400bba0 con 0x7f03ac15db10 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0394003a60 0x7f0394003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a3fff700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f0394004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b1182700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394004d50 0x7f03940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 -- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 msgr2=0x7f0394004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac15db10 0x7f0394004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 -- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394004d50 msgr2=0x7f03940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394004d50 0x7f03940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 -- 172.21.15.170:0/1883285786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394003570 con 0x7f0394003a60 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b1182700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394004d50 0x7f03940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03b0981700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0394003a60 0x7f0394003e70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c3c0 tx=0x7f03a8001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd6569a5a7ff81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:57.100 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800cc30 con 0x7f0394003a60 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800cdb0 con 0x7f0394003a60 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.095+0000 7f03a37fe700 1 -- 172.21.15.170:0/1883285786 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8005b70 con 0x7f0394003a60 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 -- 172.21.15.170:0/1883285786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0394003a60 msgr2=0x7f0394003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 --2- 172.21.15.170:0/1883285786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0394003a60 0x7f0394003e70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c3c0 tx=0x7f03a8001e90 comp rx=0 tx=0).stop 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b1182700 1 -- 172.21.15.170:0/1883285786 reap_dead start 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 -- 172.21.15.170:0/1883285786 shutdown_connections 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 -- 172.21.15.170:0/1883285786 >> 172.21.15.170:0/1883285786 conn(0x7f03ac068e20 msgr2=0x7f03ac0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 -- 172.21.15.170:0/1883285786 shutdown_connections 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.096+0000 7f03b2be5700 1 -- 172.21.15.170:0/1883285786 wait complete. 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.097+0000 7f03b2be5700 1 Processor -- start 2023-10-08T16:13:57.101 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.097+0000 7f03b2be5700 1 -- start start 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394003a60 0x7f03ac14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ac0c78a0 0x7f03ac0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac0b4490 0x7f03ac15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac14b4d0 con 0x7f03ac0c78a0 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0cebe0 con 0x7f0394003a60 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b2be5700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0ce2b0 con 0x7f03ac0b4490 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ac0c78a0 0x7f03ac0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ac0c78a0 0x7f03ac0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57042/0 (socket says 172.21.15.170:57042) 2023-10-08T16:13:57.104 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 -- 172.21.15.170:0/1860846140 learned_addr learned my addr 172.21.15.170:0/1860846140 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b1182700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac0b4490 0x7f03ac15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03b0981700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394003a60 0x7f03ac14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 -- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac0b4490 msgr2=0x7f03ac15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f03ac0b4490 0x7f03ac15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 -- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394003a60 msgr2=0x7f03ac14af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0394003a60 0x7f03ac14af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.098+0000 7f03a3fff700 1 -- 172.21.15.170:0/1860846140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0ce590 con 0x7f03ac0c78a0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03a3fff700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03ac0c78a0 0x7f03ac0b3f50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f039c011cc0 tx=0x7f039c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00f910 con 0x7f03ac0c78a0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c00fa70 con 0x7f03ac0c78a0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03b2be5700 1 -- 172.21.15.170:0/1860846140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0ce870 con 0x7f03ac0c78a0 2023-10-08T16:13:57.105 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c00fdf0 con 0x7f03ac0c78a0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.099+0000 7f03b2be5700 1 -- 172.21.15.170:0/1860846140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac14b800 con 0x7f03ac0c78a0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.100+0000 7f03b2be5700 1 -- 172.21.15.170:0/1860846140 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac0ced70 con 0x7f03ac0c78a0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.100+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f039c014650 con 0x7f03ac0c78a0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.101+0000 7f03a0ff9700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0398067320 0x7f03980697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.101+0000 7f03b0981700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0398067320 0x7f03980697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.101+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f039c018030 con 0x7f03ac0c78a0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.101+0000 7f03b0981700 1 --2- 172.21.15.170:0/1860846140 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0398067320 0x7f03980697d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f03a8001ed0 tx=0x7f03a8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:13:57.106 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.105+0000 7f03a0ff9700 1 -- 172.21.15.170:0/1860846140 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f039c0521b0 con 0x7f03ac0c78a0 2023-10-08T16:13:57.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:13:57.316+0000 7f03b2be5700 1 -- 172.21.15.170:0/1860846140 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f03ac04c3c0 con 0x7f0398067320 2023-10-08T16:14:14.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:14:14.758 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:14:44.846 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:14:44.849 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:15:14.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:15:14.923 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:15:44.983 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:15:44.986 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:15:57.005 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-10-08T16:15:57.006 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T16:15:57.006 DEBUG:teuthology.orchestra.run.smithi170:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:15:57.051 DEBUG:teuthology.orchestra.run.smithi191:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T16:15:57.093 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T16:15:57.097 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:15:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.221+0000 7f7cd29be700 1 Processor -- start 2023-10-08T16:15:57.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.221+0000 7f7cd29be700 1 -- start start 2023-10-08T16:15:57.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.232 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f7ccc159c60 0x7f7ccc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ccc15db10 0x7f7ccc069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc15a160 con 0x7f7ccc15db10 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0d13a0 con 0x7f7ccc159c60 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd29be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc15e140 con 0x7f7ccc159890 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd0f5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ccc15db10 0x7f7ccc069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.233 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd0f5b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ccc15db10 0x7f7ccc069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56030/0 (socket says 172.21.15.170:56030) 2023-10-08T16:15:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.222+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1790369561 learned_addr learned my addr 172.21.15.170:0/1790369561 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccbfff700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccb7fe700 1 --2- 172.21.15.170:0/1790369561 >> v2:172.21.15.191:3300/0 conn(0x7f7ccc159c60 0x7f7ccc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1790369561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 msgr2=0x7f7ccc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.234 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1790369561 >> v2:172.21.15.191:3300/0 conn(0x7f7ccc159c60 msgr2=0x7f7ccc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1790369561 >> v2:172.21.15.191:3300/0 conn(0x7f7ccc159c60 0x7f7ccc15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1790369561 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc15a3d0 con 0x7f7ccc15db10 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccbfff700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc069560 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0650f0 tx=0x7f7cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6b0b09879ae353 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.235 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccaffd700 1 -- 172.21.15.170:0/1790369561 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400e070 con 0x7f7ccc15db10 2023-10-08T16:15:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccaffd700 1 -- 172.21.15.170:0/1790369561 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc400ba50 con 0x7f7ccc15db10 2023-10-08T16:15:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.223+0000 7f7ccaffd700 1 -- 172.21.15.170:0/1790369561 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40092e0 con 0x7f7ccc15db10 2023-10-08T16:15:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 -- 172.21.15.170:0/1790369561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 msgr2=0x7f7ccc069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc069560 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc0650f0 tx=0x7f7cc4009840 comp rx=0 tx=0).stop 2023-10-08T16:15:57.236 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 -- 172.21.15.170:0/1790369561 shutdown_connections 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc069560 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1790369561 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1790369561 >> v2:172.21.15.191:3300/0 conn(0x7f7ccc159c60 0x7f7ccc15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 -- 172.21.15.170:0/1790369561 >> 172.21.15.170:0/1790369561 conn(0x7f7ccc068e20 msgr2=0x7f7ccc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 -- 172.21.15.170:0/1790369561 shutdown_connections 2023-10-08T16:15:57.237 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.224+0000 7f7cd29be700 1 -- 172.21.15.170:0/1790369561 wait complete. 2023-10-08T16:15:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.225+0000 7f7cd29be700 1 Processor -- start 2023-10-08T16:15:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.225+0000 7f7cd29be700 1 -- start start 2023-10-08T16:15:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 0x7f7ccc16cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.238 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b6480 con 0x7f7ccc15db10 2023-10-08T16:15:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b1820 con 0x7f7ccc159c60 2023-10-08T16:15:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0b1b00 con 0x7f7ccc159890 2023-10-08T16:15:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.239 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56044/0 (socket says 172.21.15.170:56044) 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1403392085 learned_addr learned my addr 172.21.15.170:0/1403392085 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7ccbfff700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7ccb7fe700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 0x7f7ccc16cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 msgr2=0x7f7ccc0b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc0b5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 msgr2=0x7f7ccc16cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 0x7f7ccc16cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 -- 172.21.15.170:0/1403392085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009ae0 con 0x7f7ccc15db10 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7ccb7fe700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 0x7f7ccc16cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd0f5b700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40032f0 con 0x7f7ccc15db10 2023-10-08T16:15:57.241 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40094e0 con 0x7f7ccc15db10 2023-10-08T16:15:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc0b1de0 con 0x7f7ccc15db10 2023-10-08T16:15:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc4025460 con 0x7f7ccc15db10 2023-10-08T16:15:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.226+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc0b60e0 con 0x7f7ccc15db10 2023-10-08T16:15:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.227+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc0b2300 con 0x7f7ccc15db10 2023-10-08T16:15:57.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.228+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f7cc4003040 con 0x7f7ccc15db10 2023-10-08T16:15:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.228+0000 7f7cb3fff700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 0x7f7cb80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.228+0000 7f7ccbfff700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 0x7f7cb80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.228+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f7cc4086370 con 0x7f7ccc15db10 2023-10-08T16:15:57.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.229+0000 7f7ccbfff700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 0x7f7cb80697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc006f00 tx=0x7f7cbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.244 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.233+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ccc0b2300 con 0x7f7ccc15db10 2023-10-08T16:15:57.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.457+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ccc0b2300 con 0x7f7ccc15db10 2023-10-08T16:15:57.458 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.457+0000 7f7cb3fff700 1 -- 172.21.15.170:0/1403392085 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f7ccc0b2300 con 0x7f7ccc15db10 2023-10-08T16:15:57.459 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:15:57.459 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:15:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 msgr2=0x7f7cb80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 0x7f7cb80697d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc006f00 tx=0x7f7cbc008040 comp rx=0 tx=0).stop 2023-10-08T16:15:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 msgr2=0x7f7ccc16d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009fd0 tx=0x7f7cc4018000 comp rx=0 tx=0).stop 2023-10-08T16:15:57.461 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 shutdown_connections 2023-10-08T16:15:57.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f7cb8067320 0x7f7cb80697d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ccc15db10 0x7f7ccc16d430 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f7ccc159890 0x7f7ccc0b5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 --2- 172.21.15.170:0/1403392085 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f7ccc159c60 0x7f7ccc16cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.462 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 >> 172.21.15.170:0/1403392085 conn(0x7f7ccc068e20 msgr2=0x7f7ccc15b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:57.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 shutdown_connections 2023-10-08T16:15:57.463 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.460+0000 7f7cd29be700 1 -- 172.21.15.170:0/1403392085 wait complete. 2023-10-08T16:15:57.463 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 132 2023-10-08T16:15:57.473 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:15:57.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.588+0000 7fb29fe6e700 1 Processor -- start 2023-10-08T16:15:57.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.588+0000 7fb29fe6e700 1 -- start start 2023-10-08T16:15:57.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29fe6e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb298159c60 0x7fb29815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29fe6e700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 0x7fb298069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29fe6e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29815a160 con 0x7fb298159890 2023-10-08T16:15:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29fe6e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29815df50 con 0x7fb298159c60 2023-10-08T16:15:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29fe6e700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2980d13a0 con 0x7fb29815db10 2023-10-08T16:15:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb298159c60 0x7fb29815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb298159c60 0x7fb29815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56052/0 (socket says 172.21.15.170:56052) 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 -- 172.21.15.170:0/2911681985 learned_addr learned my addr 172.21.15.170:0/2911681985 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29e40b700 1 --2- 172.21.15.170:0/2911681985 >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 0x7fb298069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 -- 172.21.15.170:0/2911681985 >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 msgr2=0x7fb298069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 --2- 172.21.15.170:0/2911681985 >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 0x7fb298069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 -- 172.21.15.170:0/2911681985 >> v1:172.21.15.191:6789/0 conn(0x7fb298159890 legacy=0x7fb29815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 -- 172.21.15.170:0/2911681985 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29815e140 con 0x7fb298159c60 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29e40b700 1 --2- 172.21.15.170:0/2911681985 >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 0x7fb298069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29d409700 1 --2- 172.21.15.170:0/2911681985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb29815d5d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb29806a890 tx=0x7fb288009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601a048c97dae0ef server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29cc08700 1 -- 172.21.15.170:0/2911681985 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800b860 con 0x7fb298159c60 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29cc08700 1 -- 172.21.15.170:0/2911681985 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28800b9c0 con 0x7fb298159c60 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.589+0000 7fb29cc08700 1 -- 172.21.15.170:0/2911681985 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800bd60 con 0x7fb298159c60 2023-10-08T16:15:57.600 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 -- 172.21.15.170:0/2911681985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 msgr2=0x7fb29815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.601 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/2911681985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb29815d5d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb29806a890 tx=0x7fb288009250 comp rx=0 tx=0).stop 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 -- 172.21.15.170:0/2911681985 shutdown_connections 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/2911681985 >> v2:172.21.15.191:3300/0 conn(0x7fb29815db10 0x7fb298069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/2911681985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb29815d5d0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 -- 172.21.15.170:0/2911681985 >> 172.21.15.170:0/2911681985 conn(0x7fb298068e20 msgr2=0x7fb2980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 -- 172.21.15.170:0/2911681985 shutdown_connections 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.590+0000 7fb29fe6e700 1 -- 172.21.15.170:0/2911681985 wait complete. 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.591+0000 7fb29fe6e700 1 Processor -- start 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.591+0000 7fb29fe6e700 1 -- start start 2023-10-08T16:15:57.602 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb298159890 0x7fb29814caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb29815db10 0x7fb29815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29816bde0 con 0x7fb298159c60 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29816b1d0 con 0x7fb298159890 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29816b4b0 con 0x7fb29815db10 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56066/0 (socket says 172.21.15.170:56066) 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 -- 172.21.15.170:0/201305895 learned_addr learned my addr 172.21.15.170:0/201305895 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 -- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb29815db10 msgr2=0x7fb29815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29e40b700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb29815db10 0x7fb29815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.603 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb29815db10 0x7fb29815caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 -- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb298159890 msgr2=0x7fb29814caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29dc0a700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb298159890 0x7fb29814caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb298159890 0x7fb29814caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 -- 172.21.15.170:0/201305895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880094f0 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29d409700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb288009290 tx=0x7fb288017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288011e60 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28801f070 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29816b790 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.592+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28801a870 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.593+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980b3590 con 0x7fb298159c60 2023-10-08T16:15:57.604 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.593+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29804c3c0 con 0x7fb298159c60 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.595+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb28801abe0 con 0x7fb298159c60 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.595+0000 7fb28dffb700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 0x7fb27c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.595+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fb288021070 con 0x7fb298159c60 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.595+0000 7fb29dc0a700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 0x7fb27c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.595+0000 7fb29dc0a700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 0x7fb27c0697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ce190 tx=0x7fb294009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.605 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.598+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2880539c0 con 0x7fb298159c60 2023-10-08T16:15:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.784+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2980b1aa0 con 0x7fb298159c60 2023-10-08T16:15:57.785 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.785+0000 7fb28dffb700 1 -- 172.21.15.170:0/201305895 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+15558 (secure 0 0 0) 0x7fb288015090 con 0x7fb298159c60 2023-10-08T16:15:57.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:15:57.786 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":203,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:14:27.542831+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.11","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 msgr2=0x7fb27c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 0x7fb27c0697d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ce190 tx=0x7fb294009040 comp rx=0 tx=0).stop 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 msgr2=0x7fb2980cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb288009290 tx=0x7fb288017000 comp rx=0 tx=0).stop 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 shutdown_connections 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb29815db10 0x7fb29815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.788 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb298159890 0x7fb29814caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb27c067320 0x7fb27c0697d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 --2- 172.21.15.170:0/201305895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb298159c60 0x7fb2980cf030 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 >> 172.21.15.170:0/201305895 conn(0x7fb298068e20 msgr2=0x7fb2980c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:57.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 shutdown_connections 2023-10-08T16:15:57.789 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.787+0000 7fb29fe6e700 1 -- 172.21.15.170:0/201305895 wait complete. 2023-10-08T16:15:57.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T16:04:30.263818+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 33, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T16:13:44.428413+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '200', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 34, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T16:13:45.120567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T16:15:57.800 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T16:15:57.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.917+0000 7fc3eab5b700 1 Processor -- start 2023-10-08T16:15:57.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.917+0000 7fc3eab5b700 1 -- start start 2023-10-08T16:15:57.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3eab5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3e40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3eab5b700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e40cfb00 con 0x7fc3e40b16d0 2023-10-08T16:15:57.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3eab5b700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e415e060 con 0x7fc3e40b1b40 2023-10-08T16:15:57.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3eab5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40b5c40 con 0x7fc3e40b4e70 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3e9358700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc3e40b16d0 0x7fc3e415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52676/0 (socket says 172.21.15.170:52676) 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3e9358700 1 -- 172.21.15.170:0/3587170174 learned_addr learned my addr 172.21.15.170:0/3587170174 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3e9b59700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3e40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220924732 0 0) 0x7fc3e40cfb00 con 0x7fc3e40b16d0 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.918+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003570 con 0x7fc3e40b16d0 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3266093477 0 0) 0x7fc3d0003570 con 0x7fc3e40b16d0 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 msgr2=0x7fc3e40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3e40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 >> v1:172.21.15.191:6789/0 conn(0x7fc3e40b1b40 legacy=0x7fc3e415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40b5f20 con 0x7fc3e40b16d0 2023-10-08T16:15:57.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fc3d4001470 con 0x7fc3e40b16d0 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 >> v1:172.21.15.170:6789/0 conn(0x7fc3e40b16d0 legacy=0x7fc3e415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3d0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3d0003fd0 0x7fc3d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3d0004d50 0x7fc3d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40b5c40 con 0x7fc3d0004d50 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000bb40 con 0x7fc3d0003fd0 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000bd00 con 0x7fc3e40b4e70 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3d0004d50 0x7fc3d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e9358700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3d0003fd0 0x7fc3d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e9b59700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3d0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 -- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 msgr2=0x7fc3d0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40b4e70 0x7fc3d0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 -- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3d0003fd0 msgr2=0x7fc3d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3d0003fd0 0x7fc3d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 -- 172.21.15.170:0/3587170174 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0003570 con 0x7fc3d0004d50 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e9358700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3d0003fd0 0x7fc3d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3d0004d50 0x7fc3d00051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0020a0 tx=0x7fc3dc0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b3b41b051d4056 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0019e0 con 0x7fc3d0004d50 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc00c8b0 con 0x7fc3d0004d50 2023-10-08T16:15:57.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.919+0000 7fc3e8b57700 1 -- 172.21.15.170:0/3587170174 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00cc50 con 0x7fc3d0004d50 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 -- 172.21.15.170:0/3587170174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3d0004d50 msgr2=0x7fc3d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/3587170174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3d0004d50 0x7fc3d00051c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0020a0 tx=0x7fc3dc0016d0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3ea35a700 1 -- 172.21.15.170:0/3587170174 reap_dead start 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 -- 172.21.15.170:0/3587170174 shutdown_connections 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 -- 172.21.15.170:0/3587170174 >> 172.21.15.170:0/3587170174 conn(0x7fc3e40d10c0 msgr2=0x7fc3e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 -- 172.21.15.170:0/3587170174 shutdown_connections 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.920+0000 7fc3eab5b700 1 -- 172.21.15.170:0/3587170174 wait complete. 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.921+0000 7fc3eab5b700 1 Processor -- start 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.921+0000 7fc3eab5b700 1 -- start start 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 0x7fc3e4068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 0x7fc3e40688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40ccff0 con 0x7fc3e40c8b90 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40cc3e0 con 0x7fc3e40c9d80 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40cc6c0 con 0x7fc3e40cbf00 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56078/0 (socket says 172.21.15.170:56078) 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 -- 172.21.15.170:0/104884889 learned_addr learned my addr 172.21.15.170:0/104884889 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 0x7fc3e40688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9358700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 0x7fc3e4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.928 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 -- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 msgr2=0x7fc3e40688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 0x7fc3e40688d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 -- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 msgr2=0x7fc3e4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 0x7fc3e4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 -- 172.21.15.170:0/104884889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40cc9a0 con 0x7fc3e40c8b90 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3ea35a700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 0x7fc3e40688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9358700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 0x7fc3e4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:57.929 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3e9b59700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0006ac0 tx=0x7fc3e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e000fa10 con 0x7fc3e40c8b90 2023-10-08T16:15:57.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e000fb70 con 0x7fc3e40c8b90 2023-10-08T16:15:57.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e40ccc80 con 0x7fc3e40c8b90 2023-10-08T16:15:57.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.922+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0014440 con 0x7fc3e40c8b90 2023-10-08T16:15:57.930 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.923+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e40c91a0 con 0x7fc3e40c8b90 2023-10-08T16:15:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.923+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e40cd2d0 con 0x7fc3e40c8b90 2023-10-08T16:15:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.924+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc3e0014870 con 0x7fc3e40c8b90 2023-10-08T16:15:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.924+0000 7fc3d9ffb700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 0x7fc3c80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.924+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fc3e0018030 con 0x7fc3e40c8b90 2023-10-08T16:15:57.932 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.924+0000 7fc3e9358700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 0x7fc3c80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:57.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.925+0000 7fc3e9358700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 0x7fc3c80697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40015a0 tx=0x7fc3d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:57.933 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:57.928+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3e0052420 con 0x7fc3e40c8b90 2023-10-08T16:15:58.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.107+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3e404bb80 con 0x7fc3e40c8b90 2023-10-08T16:15:58.108 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.108+0000 7fc3d9ffb700 1 -- 172.21.15.170:0/104884889 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v203) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3e0055a40 con 0x7fc3e40c8b90 2023-10-08T16:15:58.109 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 1 2023-10-08T16:15:58.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 msgr2=0x7fc3c80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 0x7fc3c80697d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40015a0 tx=0x7fc3d4009040 comp rx=0 tx=0).stop 2023-10-08T16:15:58.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 msgr2=0x7fc3e40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.111 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0006ac0 tx=0x7fc3e00091a0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 shutdown_connections 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc3e40cbf00 0x7fc3e40688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc3c8067320 0x7fc3c80697d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc3e40c8b90 0x7fc3e40c9840 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 --2- 172.21.15.170:0/104884889 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc3e40c9d80 0x7fc3e4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 >> 172.21.15.170:0/104884889 conn(0x7fc3e40d10c0 msgr2=0x7fc3e415a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 shutdown_connections 2023-10-08T16:15:58.112 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.110+0000 7fc3eab5b700 1 -- 172.21.15.170:0/104884889 wait complete. 2023-10-08T16:15:58.122 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T16:15:58.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.240+0000 7f3798503700 1 Processor -- start 2023-10-08T16:15:58.240 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.240+0000 7f3798503700 1 -- start start 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3798503700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3790159890 0x7f379015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3798503700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f379015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3798503700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379015a160 con 0x7f379015db10 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3798503700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900d13a0 con 0x7f3790159890 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3798503700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379015e140 con 0x7f3790159c60 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3790159890 0x7f379015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3796aa0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f379015db10 0x7f3790069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52688/0 (socket says 172.21.15.170:52688) 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3796aa0700 1 -- 172.21.15.170:0/725733101 learned_addr learned my addr 172.21.15.170:0/725733101 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:15:58.242 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f3795a9e700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f379015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 -- 172.21.15.170:0/725733101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 msgr2=0x7f379015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f379015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 -- 172.21.15.170:0/725733101 >> v1:172.21.15.170:6789/0 conn(0x7f379015db10 legacy=0x7f3790069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 -- 172.21.15.170:0/725733101 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379015a3d0 con 0x7f3790159890 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379529d700 1 -- 172.21.15.170:0/725733101 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363681231 0 0) 0x7f379015a160 con 0x7f379015db10 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379629f700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3790159890 0x7f379015abd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f378c0098e0 tx=0x7f378c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baab8cb8be9c2525 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379529d700 1 -- 172.21.15.170:0/725733101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00b640 con 0x7f3790159890 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379529d700 1 -- 172.21.15.170:0/725733101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c00b7a0 con 0x7f3790159890 2023-10-08T16:15:58.243 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.241+0000 7f379529d700 1 -- 172.21.15.170:0/725733101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00bb40 con 0x7f3790159890 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 -- 172.21.15.170:0/725733101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3790159890 msgr2=0x7f379015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3790159890 0x7f379015abd0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f378c0098e0 tx=0x7f378c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 -- 172.21.15.170:0/725733101 shutdown_connections 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3790159890 0x7f379015abd0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 --2- 172.21.15.170:0/725733101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f379015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 -- 172.21.15.170:0/725733101 >> 172.21.15.170:0/725733101 conn(0x7f3790068e20 msgr2=0x7f37900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 -- 172.21.15.170:0/725733101 shutdown_connections 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.242+0000 7f3798503700 1 -- 172.21.15.170:0/725733101 wait complete. 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.243+0000 7f3798503700 1 Processor -- start 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.243+0000 7f3798503700 1 -- start start 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790159890 0x7f379016a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f37900b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900b3ad0 con 0x7f379015db10 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37900b3d50 con 0x7f3790159890 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3798503700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900b4030 con 0x7f3790159c60 2023-10-08T16:15:58.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56106/0 (socket says 172.21.15.170:56106) 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 -- 172.21.15.170:0/589754335 learned_addr learned my addr 172.21.15.170:0/589754335 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3795a9e700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f37900b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 -- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 msgr2=0x7f37900b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f379629f700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790159890 0x7f379016a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f37900b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 -- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790159890 msgr2=0x7f379016a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790159890 0x7f379016a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 -- 172.21.15.170:0/589754335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c009420 con 0x7f379015db10 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f3796aa0700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f378800cc10 tx=0x7f3788001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378800a930 con 0x7f379015db10 2023-10-08T16:15:58.248 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.244+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378800aab0 con 0x7f379015db10 2023-10-08T16:15:58.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.245+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37900b4310 con 0x7f379015db10 2023-10-08T16:15:58.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.245+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378800ae70 con 0x7f379015db10 2023-10-08T16:15:58.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.245+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37900b48d0 con 0x7f379015db10 2023-10-08T16:15:58.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.245+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37900b4af0 con 0x7f379015db10 2023-10-08T16:15:58.249 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.246+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3788007300 con 0x7f379015db10 2023-10-08T16:15:58.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.246+0000 7f37867fc700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 0x7f37740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.246+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f3788085120 con 0x7f379015db10 2023-10-08T16:15:58.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.246+0000 7f379629f700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 0x7f37740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.250 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.247+0000 7f379629f700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 0x7f37740697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f378c003080 tx=0x7f378c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.251+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3788051d50 con 0x7f379015db10 2023-10-08T16:15:58.434 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.434+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f379004c3c0 con 0x7f379015db10 2023-10-08T16:15:58.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.434+0000 7f37867fc700 1 -- 172.21.15.170:0/589754335 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v203) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3788055370 con 0x7f379015db10 2023-10-08T16:15:58.435 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 32 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 msgr2=0x7f37740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 0x7f37740697d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f378c003080 tx=0x7f378c017000 comp rx=0 tx=0).stop 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 msgr2=0x7f37900b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f378800cc10 tx=0x7f3788001970 comp rx=0 tx=0).stop 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 shutdown_connections 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3774067320 0x7f37740697d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.437 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f379015db10 0x7f37900b3590 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3790159890 0x7f379016a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 --2- 172.21.15.170:0/589754335 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3790159c60 0x7f37900b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.436+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 >> 172.21.15.170:0/589754335 conn(0x7f3790068e20 msgr2=0x7f3790153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.437+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 shutdown_connections 2023-10-08T16:15:58.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.437+0000 7f3798503700 1 -- 172.21.15.170:0/589754335 wait complete. 2023-10-08T16:15:58.449 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T16:15:58.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.566+0000 7f09e08d6700 1 Processor -- start 2023-10-08T16:15:58.566 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.566+0000 7f09e08d6700 1 -- start start 2023-10-08T16:15:58.568 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09e08d6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f09dc0cc600 0x7f09dc0b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09e08d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09e08d6700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc150b50 con 0x7f09dc0cd0f0 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09e08d6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc0cede0 con 0x7f09dc0cc600 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09e08d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc0b2260 con 0x7f09dc0cc9d0 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f09dc0cc600 0x7f09dc0b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09db59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f09dc0cd0f0 0x7f09dc159c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52696/0 (socket says 172.21.15.170:52696) 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09db59e700 1 -- 172.21.15.170:0/3948028815 learned_addr learned my addr 172.21.15.170:0/3948028815 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:15:58.569 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09d259c700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 -- 172.21.15.170:0/3948028815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 msgr2=0x7f09dc15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 -- 172.21.15.170:0/3948028815 >> v1:172.21.15.170:6789/0 conn(0x7f09dc0cd0f0 legacy=0x7f09dc159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 -- 172.21.15.170:0/3948028815 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc0b2540 con 0x7f09dc0cc600 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09da59c700 1 -- 172.21.15.170:0/3948028815 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899728346 0 0) 0x7f09dc150b50 con 0x7f09dc0cd0f0 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09dad9d700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc0b1a10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0098e0 tx=0x7f09cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206edd8d1b6bd653 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09da59c700 1 -- 172.21.15.170:0/3948028815 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009e80 con 0x7f09dc0cc600 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09da59c700 1 -- 172.21.15.170:0/3948028815 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc00b860 con 0x7f09dc0cc600 2023-10-08T16:15:58.570 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.567+0000 7f09da59c700 1 -- 172.21.15.170:0/3948028815 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00bbe0 con 0x7f09dc0cc600 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 -- 172.21.15.170:0/3948028815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 msgr2=0x7f09dc0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc0b1a10 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0098e0 tx=0x7f09cc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 -- 172.21.15.170:0/3948028815 shutdown_connections 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc0b1a10 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3948028815 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 -- 172.21.15.170:0/3948028815 >> 172.21.15.170:0/3948028815 conn(0x7f09dc0d0630 msgr2=0x7f09dc0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 -- 172.21.15.170:0/3948028815 shutdown_connections 2023-10-08T16:15:58.571 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.568+0000 7f09e08d6700 1 -- 172.21.15.170:0/3948028815 wait complete. 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.569+0000 7f09e08d6700 1 Processor -- start 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.569+0000 7f09e08d6700 1 -- start start 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc16cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc0cd0f0 0x7f09dc0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc16dea0 con 0x7f09dc0cc600 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc159480 con 0x7f09dc0cd0f0 2023-10-08T16:15:58.574 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09e08d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc16d8e0 con 0x7f09dc0cc9d0 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56122/0 (socket says 172.21.15.170:56122) 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 -- 172.21.15.170:0/3706742247 learned_addr learned my addr 172.21.15.170:0/3706742247 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09d259c700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc16cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 -- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 msgr2=0x7f09dc16cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc16cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 -- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc0cd0f0 msgr2=0x7f09dc0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc0cd0f0 0x7f09dc0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 -- 172.21.15.170:0/3706742247 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc009420 con 0x7f09dc0cc600 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09dad9d700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f09dc071390 tx=0x7f09cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.575 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00bbe0 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.570+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc011420 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.571+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0117e0 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.571+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc16dbc0 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.571+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc16d520 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.571+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc0889d0 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.572+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09cc011bd0 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.572+0000 7f09d37fe700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 0x7f09bc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.572+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f09cc086630 con 0x7f09dc0cc600 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.572+0000 7f09d259c700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 0x7f09bc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.576 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.573+0000 7f09d259c700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 0x7f09bc0697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f09dc153ee0 tx=0x7f09c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.576+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09cc053370 con 0x7f09dc0cc600 2023-10-08T16:15:58.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.759+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09dc0b2820 con 0x7f09dc0cc600 2023-10-08T16:15:58.760 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.760+0000 7f09d37fe700 1 -- 172.21.15.170:0/3706742247 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v203) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09cc056990 con 0x7f09dc0cc600 2023-10-08T16:15:58.760 INFO:teuthology.orchestra.run.smithi170.stdout:pg_num: 64 2023-10-08T16:15:58.762 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 msgr2=0x7f09bc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 0x7f09bc0697d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f09dc153ee0 tx=0x7f09c4008040 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 msgr2=0x7f09dc1699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f09dc071390 tx=0x7f09cc017000 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 shutdown_connections 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09dc0cd0f0 0x7f09dc0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09bc067320 0x7f09bc0697d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09dc0cc600 0x7f09dc1699e0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 --2- 172.21.15.170:0/3706742247 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09dc0cc9d0 0x7f09dc16cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 >> 172.21.15.170:0/3706742247 conn(0x7f09dc0d0630 msgr2=0x7f09dc1501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 shutdown_connections 2023-10-08T16:15:58.763 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.762+0000 7f09e08d6700 1 -- 172.21.15.170:0/3706742247 wait complete. 2023-10-08T16:15:58.774 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:15:58.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.888+0000 7f8691e1a700 1 Processor -- start 2023-10-08T16:15:58.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.888+0000 7f8691e1a700 1 -- start start 2023-10-08T16:15:58.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f8691e1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159c60 0x7f868c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f8691e1a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f868c15db10 0x7f868c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f8691e1a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868c15a160 con 0x7f868c159890 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f8691e1a700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868c0d13a0 con 0x7f868c15db10 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f8691e1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c15e140 con 0x7f868c159c60 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f868c15db10 0x7f868c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f868c159890 0x7f868c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52700/0 (socket says 172.21.15.170:52700) 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f868c15db10 0x7f868c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56126/0 (socket says 172.21.15.170:56126) 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868b7fe700 1 -- 172.21.15.170:0/2711668901 learned_addr learned my addr 172.21.15.170:0/2711668901 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:15:58.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 -- 172.21.15.170:0/2711668901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159c60 msgr2=0x7f868c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 --2- 172.21.15.170:0/2711668901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159c60 0x7f868c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 -- 172.21.15.170:0/2711668901 >> v1:172.21.15.170:6789/0 conn(0x7f868c159890 legacy=0x7f868c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 -- 172.21.15.170:0/2711668901 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868c15a3d0 con 0x7f868c15db10 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868affd700 1 -- 172.21.15.170:0/2711668901 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972383110 0 0) 0x7f868c15a160 con 0x7f868c159890 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868bfff700 1 --2- 172.21.15.170:0/2711668901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c069560 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8684009e60 tx=0x7f8684009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5672a5cba829d26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.889+0000 7f868affd700 1 -- 172.21.15.170:0/2711668901 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868400e070 con 0x7f868c15db10 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f868affd700 1 -- 172.21.15.170:0/2711668901 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400bc30 con 0x7f868c15db10 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f868affd700 1 -- 172.21.15.170:0/2711668901 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684012430 con 0x7f868c15db10 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 -- 172.21.15.170:0/2711668901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 msgr2=0x7f868c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2711668901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c069560 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8684009e60 tx=0x7f8684009780 comp rx=0 tx=0).stop 2023-10-08T16:15:58.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 -- 172.21.15.170:0/2711668901 shutdown_connections 2023-10-08T16:15:58.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2711668901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c069560 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2711668901 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159c60 0x7f868c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 -- 172.21.15.170:0/2711668901 >> 172.21.15.170:0/2711668901 conn(0x7f868c068e20 msgr2=0x7f868c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:58.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 -- 172.21.15.170:0/2711668901 shutdown_connections 2023-10-08T16:15:58.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.890+0000 7f8691e1a700 1 -- 172.21.15.170:0/2711668901 wait complete. 2023-10-08T16:15:58.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.891+0000 7f8691e1a700 1 Processor -- start 2023-10-08T16:15:58.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.891+0000 7f8691e1a700 1 -- start start 2023-10-08T16:15:58.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159890 0x7f868c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c159c60 0x7f868c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0cd5d0 con 0x7f868c15db10 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0cd8b0 con 0x7f868c159c60 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f8691e1a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868c0cdb90 con 0x7f868c159890 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56140/0 (socket says 172.21.15.170:56140) 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 -- 172.21.15.170:0/2775143354 learned_addr learned my addr 172.21.15.170:0/2775143354 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 -- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159890 msgr2=0x7f868c14c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159890 0x7f868c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 -- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c159c60 msgr2=0x7f868c152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c159c60 0x7f868c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 -- 172.21.15.170:0/2775143354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009a20 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.892+0000 7f868bfff700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f86840097c0 tx=0x7f8684019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868401e630 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868401e790 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c151270 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868401eb50 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c14ca40 con 0x7f868c15db10 2023-10-08T16:15:58.897 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.893+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c04c3c0 con 0x7f868c15db10 2023-10-08T16:15:58.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.894+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8684024070 con 0x7f868c15db10 2023-10-08T16:15:58.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.894+0000 7f86837fe700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 0x7f866c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:58.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.894+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f8684087700 con 0x7f868c15db10 2023-10-08T16:15:58.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.894+0000 7f868b7fe700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 0x7f866c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:58.898 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.895+0000 7f868b7fe700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 0x7f866c0697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f867c0011e0 tx=0x7f867c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:58.899 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:58.898+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8684054340 con 0x7f868c15db10 2023-10-08T16:15:59.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.121+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f868c0d15a0 con 0x7f868c15db10 2023-10-08T16:15:59.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.121+0000 7f86837fe700 1 -- 172.21.15.170:0/2775143354 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f8684057960 con 0x7f868c15db10 2023-10-08T16:15:59.122 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:15:59.123 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 msgr2=0x7f866c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 0x7f866c0697d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f867c0011e0 tx=0x7f867c0079d0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 msgr2=0x7f868c0cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f86840097c0 tx=0x7f8684019000 comp rx=0 tx=0).stop 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 shutdown_connections 2023-10-08T16:15:59.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f866c067320 0x7f866c0697d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f868c15db10 0x7f868c0cade0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f868c159890 0x7f868c14c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 --2- 172.21.15.170:0/2775143354 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f868c159c60 0x7f868c152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 >> 172.21.15.170:0/2775143354 conn(0x7f868c068e20 msgr2=0x7f868c14d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 shutdown_connections 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.124+0000 7f8691e1a700 1 -- 172.21.15.170:0/2775143354 wait complete. 2023-10-08T16:15:59.125 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 132 2023-10-08T16:15:59.137 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:15:59.251 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.251+0000 7fc689255700 1 Processor -- start 2023-10-08T16:15:59.252 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.251+0000 7fc689255700 1 -- start start 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc689255700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6841597f0 0x7fc6840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc689255700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fc68415d230 0x7fc68415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc689255700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68415db70 con 0x7fc684159420 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc689255700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840b11d0 con 0x7fc6841597f0 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc689255700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6840b14b0 con 0x7fc68415d230 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6841597f0 0x7fc6840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6841597f0 0x7fc6840b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56156/0 (socket says 172.21.15.170:56156) 2023-10-08T16:15:59.255 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 -- 172.21.15.170:0/158311326 learned_addr learned my addr 172.21.15.170:0/158311326 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc688a54700 1 --2- 172.21.15.170:0/158311326 >> v2:172.21.15.191:3300/0 conn(0x7fc68415d230 0x7fc68415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 -- 172.21.15.170:0/158311326 >> v2:172.21.15.191:3300/0 conn(0x7fc68415d230 msgr2=0x7fc68415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 --2- 172.21.15.170:0/158311326 >> v2:172.21.15.191:3300/0 conn(0x7fc68415d230 0x7fc68415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 -- 172.21.15.170:0/158311326 >> v1:172.21.15.191:6789/0 conn(0x7fc684159420 legacy=0x7fc6840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 -- 172.21.15.170:0/158311326 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6840b1790 con 0x7fc6841597f0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.252+0000 7fc6837fe700 1 --2- 172.21.15.170:0/158311326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6841597f0 0x7fc6840b2270 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc68415d170 tx=0x7fc678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61ec53e23d72a0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc682ffd700 1 -- 172.21.15.170:0/158311326 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678009e70 con 0x7fc6841597f0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc682ffd700 1 -- 172.21.15.170:0/158311326 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc67800b860 con 0x7fc6841597f0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc682ffd700 1 -- 172.21.15.170:0/158311326 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67800bc00 con 0x7fc6841597f0 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 -- 172.21.15.170:0/158311326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6841597f0 msgr2=0x7fc6840b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.256 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 --2- 172.21.15.170:0/158311326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6841597f0 0x7fc6840b2270 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc68415d170 tx=0x7fc678009250 comp rx=0 tx=0).stop 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 -- 172.21.15.170:0/158311326 shutdown_connections 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 --2- 172.21.15.170:0/158311326 >> v2:172.21.15.191:3300/0 conn(0x7fc68415d230 0x7fc68415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 --2- 172.21.15.170:0/158311326 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6841597f0 0x7fc6840b2270 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 -- 172.21.15.170:0/158311326 >> 172.21.15.170:0/158311326 conn(0x7fc6840d0660 msgr2=0x7fc6840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 -- 172.21.15.170:0/158311326 shutdown_connections 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.253+0000 7fc689255700 1 -- 172.21.15.170:0/158311326 wait complete. 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.254+0000 7fc689255700 1 Processor -- start 2023-10-08T16:15:59.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.254+0000 7fc689255700 1 -- start start 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6841597f0 0x7fc6840c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68415d230 0x7fc6840b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840c66f0 con 0x7fc684159420 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6840c6f00 con 0x7fc68415d230 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc689255700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc684154940 con 0x7fc6841597f0 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc683fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56170/0 (socket says 172.21.15.170:56170) 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc683fff700 1 -- 172.21.15.170:0/4103580195 learned_addr learned my addr 172.21.15.170:0/4103580195 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc6837fe700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6841597f0 0x7fc6840c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.255+0000 7fc688a54700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68415d230 0x7fc6840b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 -- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6841597f0 msgr2=0x7fc6840c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6841597f0 0x7fc6840c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 -- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68415d230 msgr2=0x7fc6840b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68415d230 0x7fc6840b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 -- 172.21.15.170:0/4103580195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780094f0 con 0x7fc684159420 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc683fff700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c5c0 tx=0x7fc67400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674011590 con 0x7fc684159420 2023-10-08T16:15:59.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc674011710 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6840c5e50 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc674011ad0 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.256+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6840c6410 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.257+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc68404c3c0 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.257+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fc67401b3b0 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.257+0000 7fc66bfff700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 0x7fc6700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.258+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7fc674084140 con 0x7fc684159420 2023-10-08T16:15:59.262 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.258+0000 7fc6837fe700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 0x7fc6700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.258+0000 7fc6837fe700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 0x7fc6700697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc68415d170 tx=0x7fc678017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.263 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.262+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc674050d80 con 0x7fc684159420 2023-10-08T16:15:59.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.447+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6840b1a70 con 0x7fc684159420 2023-10-08T16:15:59.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.448+0000 7fc66bfff700 1 -- 172.21.15.170:0/4103580195 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+15558 (secure 0 0 0) 0x7fc674017090 con 0x7fc684159420 2023-10-08T16:15:59.448 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:15:59.448 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":203,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:14:27.542831+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.11","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 msgr2=0x7fc6700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 0x7fc6700697d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc68415d170 tx=0x7fc678017000 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 msgr2=0x7fc684154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c5c0 tx=0x7fc67400c8d0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 shutdown_connections 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fc68415d230 0x7fc6840b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fc670067320 0x7fc6700697d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc684159420 0x7fc684154400 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 --2- 172.21.15.170:0/4103580195 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fc6841597f0 0x7fc6840c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.451 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 >> 172.21.15.170:0/4103580195 conn(0x7fc6840d0660 msgr2=0x7fc6840b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 shutdown_connections 2023-10-08T16:15:59.452 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.450+0000 7fc689255700 1 -- 172.21.15.170:0/4103580195 wait complete. 2023-10-08T16:15:59.463 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:15:59.463 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:15:59.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.576+0000 7f878eb51700 1 Processor -- start 2023-10-08T16:15:59.577 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.576+0000 7f878eb51700 1 -- start start 2023-10-08T16:15:59.579 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8788159890 0x7f878815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8788159c60 0x7f878815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f878815db10 0x7f8788069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878815a160 con 0x7f8788159890 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87880d13a0 con 0x7f878815db10 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878eb51700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878815e140 con 0x7f8788159c60 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878c8ed700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8788159890 0x7f878815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878c8ed700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8788159890 0x7f878815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56186/0 (socket says 172.21.15.170:56186) 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f878c8ed700 1 -- 172.21.15.170:0/4271050018 learned_addr learned my addr 172.21.15.170:0/4271050018 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.577+0000 7f877ffff700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8788159c60 0x7f878815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.580 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878d0ee700 1 --2- 172.21.15.170:0/4271050018 >> v2:172.21.15.191:3300/0 conn(0x7f878815db10 0x7f8788069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 -- 172.21.15.170:0/4271050018 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8788159c60 msgr2=0x7f878815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8788159c60 0x7f878815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 -- 172.21.15.170:0/4271050018 >> v2:172.21.15.191:3300/0 conn(0x7f878815db10 msgr2=0x7f8788069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 --2- 172.21.15.170:0/4271050018 >> v2:172.21.15.191:3300/0 conn(0x7f878815db10 0x7f8788069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 -- 172.21.15.170:0/4271050018 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878815a3d0 con 0x7f8788159890 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878c8ed700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f878815abd0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8788089b50 tx=0x7f8784009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d3293a7b2d150 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f877f7fe700 1 -- 172.21.15.170:0/4271050018 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878400e070 con 0x7f8788159890 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f877f7fe700 1 -- 172.21.15.170:0/4271050018 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878400ba50 con 0x7f8788159890 2023-10-08T16:15:59.581 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f877f7fe700 1 -- 172.21.15.170:0/4271050018 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840092e0 con 0x7f8788159890 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 -- 172.21.15.170:0/4271050018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 msgr2=0x7f878815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f878815abd0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f8788089b50 tx=0x7f8784009840 comp rx=0 tx=0).stop 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 -- 172.21.15.170:0/4271050018 shutdown_connections 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 --2- 172.21.15.170:0/4271050018 >> v2:172.21.15.191:3300/0 conn(0x7f878815db10 0x7f8788069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f878815abd0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 --2- 172.21.15.170:0/4271050018 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8788159c60 0x7f878815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 -- 172.21.15.170:0/4271050018 >> 172.21.15.170:0/4271050018 conn(0x7f8788068e20 msgr2=0x7f87880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.582 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.578+0000 7f878eb51700 1 -- 172.21.15.170:0/4271050018 shutdown_connections 2023-10-08T16:15:59.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.579+0000 7f878eb51700 1 -- 172.21.15.170:0/4271050018 wait complete. 2023-10-08T16:15:59.583 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.579+0000 7f878eb51700 1 Processor -- start 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 -- start start 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 0x7f87880cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 0x7f8788158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b4530 con 0x7f8788159890 2023-10-08T16:15:59.587 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b5f40 con 0x7f8788159c60 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878eb51700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87880b6220 con 0x7f878815db10 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.580+0000 7f878c8ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56202/0 (socket says 172.21.15.170:56202) 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 -- 172.21.15.170:0/2316310256 learned_addr learned my addr 172.21.15.170:0/2316310256 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878d0ee700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 0x7f8788158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f877ffff700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 0x7f87880cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 -- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 msgr2=0x7f8788158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 0x7f8788158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 -- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 msgr2=0x7f87880cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 0x7f87880cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 -- 172.21.15.170:0/2316310256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009ae0 con 0x7f8788159890 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878d0ee700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 0x7f8788158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f877ffff700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 0x7f87880cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878c8ed700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8788070e20 tx=0x7f8784018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784008bb0 con 0x7f8788159890 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784008d50 con 0x7f8788159890 2023-10-08T16:15:59.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87880ce350 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87840266b0 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.581+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87880b5b60 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.582+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f878804c3c0 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.582+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8784003040 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.583+0000 7f877cff9700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 0x7f87740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.583+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f8784086630 con 0x7f8788159890 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.583+0000 7f877ffff700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 0x7f87740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.583+0000 7f877ffff700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 0x7f87740697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f878816d290 tx=0x7f8778008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.587+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8784053160 con 0x7f8788159890 2023-10-08T16:15:59.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.814+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f878806a5b0 con 0x7f8788159890 2023-10-08T16:15:59.815 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.815+0000 7f877cff9700 1 -- 172.21.15.170:0/2316310256 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7f8784009940 con 0x7f8788159890 2023-10-08T16:15:59.816 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:15:59.816 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:15:59.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 msgr2=0x7f87740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 0x7f87740697d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f878816d290 tx=0x7f8778008040 comp rx=0 tx=0).stop 2023-10-08T16:15:59.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 msgr2=0x7f87880b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f8788070e20 tx=0x7f8784018000 comp rx=0 tx=0).stop 2023-10-08T16:15:59.818 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 shutdown_connections 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f878815db10 0x7f8788158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8774067320 0x7f87740697d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8788159890 0x7f87880b3f50 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 --2- 172.21.15.170:0/2316310256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8788159c60 0x7f87880cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 >> 172.21.15.170:0/2316310256 conn(0x7f8788068e20 msgr2=0x7f8788153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 shutdown_connections 2023-10-08T16:15:59.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.817+0000 7f878eb51700 1 -- 172.21.15.170:0/2316310256 wait complete. 2023-10-08T16:15:59.820 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 132 2023-10-08T16:15:59.830 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:15:59.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.945+0000 7effabea8700 1 Processor -- start 2023-10-08T16:15:59.946 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.945+0000 7effabea8700 1 -- start start 2023-10-08T16:15:59.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effabea8700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 0x7effa4140df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effabea8700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7effa406c0a0 0x7effa4141330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effabea8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa415c2a0 con 0x7effa40c8e00 2023-10-08T16:15:59.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effabea8700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa40c9900 con 0x7effa406c0a0 2023-10-08T16:15:59.948 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effabea8700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa415b690 con 0x7effa406bcd0 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effa9443700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7effa406c0a0 0x7effa4141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effa9443700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7effa406c0a0 0x7effa4141330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56218/0 (socket says 172.21.15.170:56218) 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effa9443700 1 -- 172.21.15.170:0/3090120991 learned_addr learned my addr 172.21.15.170:0/3090120991 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.946+0000 7effaa445700 1 --1- 172.21.15.170:0/3090120991 >> v1:172.21.15.170:6789/0 conn(0x7effa40c8e00 0x7effa40c91f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52706/0 (socket says 172.21.15.170:52706) 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9c44700 1 --2- 172.21.15.170:0/3090120991 >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 0x7effa4140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9443700 1 -- 172.21.15.170:0/3090120991 >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 msgr2=0x7effa4140df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9443700 1 --2- 172.21.15.170:0/3090120991 >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 0x7effa4140df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9443700 1 -- 172.21.15.170:0/3090120991 >> v1:172.21.15.170:6789/0 conn(0x7effa40c8e00 legacy=0x7effa40c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9443700 1 -- 172.21.15.170:0/3090120991 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa415b970 con 0x7effa406c0a0 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa8c42700 1 -- 172.21.15.170:0/3090120991 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680664034 0 0) 0x7effa415c2a0 con 0x7effa40c8e00 2023-10-08T16:15:59.949 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9c44700 1 --2- 172.21.15.170:0/3090120991 >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 0x7effa4140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa9443700 1 --2- 172.21.15.170:0/3090120991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa406c0a0 0x7effa4141330 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7effa40cafd0 tx=0x7eff94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3acb80b322eb7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa8c42700 1 -- 172.21.15.170:0/3090120991 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400b860 con 0x7effa406c0a0 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa8c42700 1 -- 172.21.15.170:0/3090120991 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9400b9c0 con 0x7effa406c0a0 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.947+0000 7effa8c42700 1 -- 172.21.15.170:0/3090120991 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400bd60 con 0x7effa406c0a0 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 -- 172.21.15.170:0/3090120991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa406c0a0 msgr2=0x7effa4141330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 --2- 172.21.15.170:0/3090120991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa406c0a0 0x7effa4141330 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7effa40cafd0 tx=0x7eff94009250 comp rx=0 tx=0).stop 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 -- 172.21.15.170:0/3090120991 shutdown_connections 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 --2- 172.21.15.170:0/3090120991 >> v2:172.21.15.191:3300/0 conn(0x7effa406bcd0 0x7effa4140df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 --2- 172.21.15.170:0/3090120991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa406c0a0 0x7effa4141330 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 -- 172.21.15.170:0/3090120991 >> 172.21.15.170:0/3090120991 conn(0x7effa40ce600 msgr2=0x7effa40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:15:59.950 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 -- 172.21.15.170:0/3090120991 shutdown_connections 2023-10-08T16:15:59.951 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.948+0000 7effabea8700 1 -- 172.21.15.170:0/3090120991 wait complete. 2023-10-08T16:15:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 Processor -- start 2023-10-08T16:15:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 -- start start 2023-10-08T16:15:59.953 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 0x7effa40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 0x7effa413ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40699e0 con 0x7effa40c8e00 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4069c60 con 0x7effa406c0a0 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.949+0000 7effabea8700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4069f40 con 0x7effa406bcd0 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56222/0 (socket says 172.21.15.170:56222) 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 -- 172.21.15.170:0/3633779718 learned_addr learned my addr 172.21.15.170:0/3633779718 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effa9443700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 0x7effa413ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.954 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effa9c44700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 0x7effa40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 -- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 msgr2=0x7effa40694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 0x7effa40694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 -- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 msgr2=0x7effa413ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:15:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 0x7effa413ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:15:59.955 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 -- 172.21.15.170:0/3633779718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940094f0 con 0x7effa40c8e00 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effa9443700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 0x7effa413ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effa9c44700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 0x7effa40694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effaa445700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00c4e0 tx=0x7eff9c007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c005a70 con 0x7effa40c8e00 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c005bd0 con 0x7effa40c8e00 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa406a220 con 0x7effa40c8e00 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0070d0 con 0x7effa40c8e00 2023-10-08T16:15:59.956 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.950+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa406a5b0 con 0x7effa40c8e00 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.951+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa406aa00 con 0x7effa40c8e00 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.952+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7eff9c007500 con 0x7effa40c8e00 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.952+0000 7eff99ffb700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 0x7eff880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.952+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7eff9c084200 con 0x7effa40c8e00 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.952+0000 7effa9c44700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 0x7eff880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.953+0000 7effa9c44700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 0x7eff880697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7effa00094d0 tx=0x7effa000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:15:59.957 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:15:59.956+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff9c050e40 con 0x7effa40c8e00 2023-10-08T16:16:00.177 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.177+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa404c3c0 con 0x7effa40c8e00 2023-10-08T16:16:00.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.177+0000 7eff99ffb700 1 -- 172.21.15.170:0/3633779718 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+4619 (secure 0 0 0) 0x7eff9c054460 con 0x7effa40c8e00 2023-10-08T16:16:00.178 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:00.178 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":132,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126}],"filesystems":[{"mdsmap":{"epoch":131,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T16:13:46.076345+0000","modified":"2023-10-08T16:13:49.720431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":127,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6837/1430346956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":1430346956},{"type":"v1","addr":"172.21.15.170:6837","nonce":1430346956}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[34],"metadata_pool":33,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":16}]} 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 msgr2=0x7eff880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 0x7eff880697d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7effa00094d0 tx=0x7effa000b040 comp rx=0 tx=0).stop 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 msgr2=0x7effa40b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7eff9c00c4e0 tx=0x7eff9c007f40 comp rx=0 tx=0).stop 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 shutdown_connections 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7eff88067320 0x7eff880697d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa40c8e00 0x7effa40b1e40 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7effa406bcd0 0x7effa40694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 --2- 172.21.15.170:0/3633779718 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7effa406c0a0 0x7effa413ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 >> 172.21.15.170:0/3633779718 conn(0x7effa40ce600 msgr2=0x7effa406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 shutdown_connections 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.179+0000 7effabea8700 1 -- 172.21.15.170:0/3633779718 wait complete. 2023-10-08T16:16:00.181 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 132 2023-10-08T16:16:00.192 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:16:00.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.307+0000 7f0592c55700 1 Processor -- start 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.307+0000 7f0592c55700 1 -- start start 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f0592c55700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f0592c55700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f058c159c60 0x7f058c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f0592c55700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058c15a160 con 0x7f058c15db10 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f0592c55700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058c0d13a0 con 0x7f058c159c60 2023-10-08T16:16:00.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f0592c55700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c15e140 con 0x7f058c159890 2023-10-08T16:16:00.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f05909f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f058bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f058c159c60 0x7f058c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f05909f1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:34514/0 (socket says 172.21.15.170:34514) 2023-10-08T16:16:00.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f058bfff700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f058c159c60 0x7f058c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:41342/0 (socket says 172.21.15.170:41342) 2023-10-08T16:16:00.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f05909f1700 1 -- 172.21.15.170:0/1226837810 learned_addr learned my addr 172.21.15.170:0/1226837810 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.308+0000 7f05911f2700 1 --1- 172.21.15.170:0/1226837810 >> v1:172.21.15.191:6789/0 conn(0x7f058c15db10 0x7f058c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:60262/0 (socket says 172.21.15.170:60262) 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058bfff700 1 -- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 msgr2=0x7f058c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058bfff700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058bfff700 1 -- 172.21.15.170:0/1226837810 >> v1:172.21.15.191:6789/0 conn(0x7f058c15db10 legacy=0x7f058c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058bfff700 1 -- 172.21.15.170:0/1226837810 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c15a3d0 con 0x7f058c159c60 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f05909f1700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058bfff700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c159c60 0x7f058c15d5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05780098e0 tx=0x7f05780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad6ded5b6254fa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058b7fe700 1 -- 172.21.15.170:0/1226837810 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800b640 con 0x7f058c159c60 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058b7fe700 1 -- 172.21.15.170:0/1226837810 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057800b7a0 con 0x7f058c159c60 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f058b7fe700 1 -- 172.21.15.170:0/1226837810 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057800bb40 con 0x7f058c159c60 2023-10-08T16:16:00.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f0592c55700 1 -- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c159c60 msgr2=0x7f058c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.309+0000 7f0592c55700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c159c60 0x7f058c15d5d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f05780098e0 tx=0x7f05780091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 -- 172.21.15.170:0/1226837810 shutdown_connections 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159890 0x7f058c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 --2- 172.21.15.170:0/1226837810 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c159c60 0x7f058c15d5d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 -- 172.21.15.170:0/1226837810 >> 172.21.15.170:0/1226837810 conn(0x7f058c068e20 msgr2=0x7f058c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 -- 172.21.15.170:0/1226837810 shutdown_connections 2023-10-08T16:16:00.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.310+0000 7f0592c55700 1 -- 172.21.15.170:0/1226837810 wait complete. 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.311+0000 7f0592c55700 1 Processor -- start 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.311+0000 7f0592c55700 1 -- start start 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.311+0000 7f0592c55700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159c60 0x7f058c1526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 0x7f058c152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0b2170 con 0x7f058c159890 2023-10-08T16:16:00.315 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0b1560 con 0x7f058c15db10 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058c0b1840 con 0x7f058c159c60 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56224/0 (socket says 172.21.15.170:56224) 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 -- 172.21.15.170:0/895203694 learned_addr learned my addr 172.21.15.170:0/895203694 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f058bfff700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159c60 0x7f058c1526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05911f2700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 0x7f058c152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 -- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159c60 msgr2=0x7f058c1526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159c60 0x7f058c1526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 -- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 msgr2=0x7f058c152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 0x7f058c152bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.316 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 -- 172.21.15.170:0/895203694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800094f0 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05911f2700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 0x7f058c152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f05909f1700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f058000ec50 tx=0x7f0580009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000c920 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058000ca80 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0578009420 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058000ce00 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.312+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058c0b1e00 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.313+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058c04c3c0 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.314+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0580013680 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.314+0000 7f0588ff9700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 0x7f05700697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.314+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f0580085090 con 0x7f058c159890 2023-10-08T16:16:00.317 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.314+0000 7f058bfff700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 0x7f05700697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.315+0000 7f058bfff700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 0x7f05700697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f058c15a090 tx=0x7f0578017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.318 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.318+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0580051cd0 con 0x7f058c159890 2023-10-08T16:16:00.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.502+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f058c06a5b0 con 0x7f058c159890 2023-10-08T16:16:00.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.502+0000 7f0588ff9700 1 -- 172.21.15.170:0/895203694 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+15558 (secure 0 0 0) 0x7f05800552f0 con 0x7f058c159890 2023-10-08T16:16:00.503 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:00.503 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":203,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:14:27.542831+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":33,"pool_name":"cephfs_metadata","create_time":"2023-10-08T16:13:44.428413+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"200","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":34,"pool_name":"cephfs_data","create_time":"2023-10-08T16:13:45.120567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"34.11","mappings":[{"from":0,"to":2}]},{"pgid":"34.21","mappings":[{"from":0,"to":2}]},{"pgid":"34.25","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:16:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 msgr2=0x7f05700697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 0x7f05700697d0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f058c15a090 tx=0x7f0578017000 comp rx=0 tx=0).stop 2023-10-08T16:16:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 msgr2=0x7f058c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f058000ec50 tx=0x7f0580009250 comp rx=0 tx=0).stop 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 shutdown_connections 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f058c15db10 0x7f058c152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0570067320 0x7f05700697d0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f058c159890 0x7f058c0b0d10 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 --2- 172.21.15.170:0/895203694 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f058c159c60 0x7f058c1526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 >> 172.21.15.170:0/895203694 conn(0x7f058c068e20 msgr2=0x7f058c158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 shutdown_connections 2023-10-08T16:16:00.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.505+0000 7f0592c55700 1 -- 172.21.15.170:0/895203694 wait complete. 2023-10-08T16:16:00.518 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T16:16:00.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.633+0000 7f9246686700 1 Processor -- start 2023-10-08T16:16:00.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.633+0000 7f9246686700 1 -- start start 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9240159890 0x7f924015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9240159c60 0x7f924015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f924015db10 0x7f9240069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924015a160 con 0x7f924015db10 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400d13a0 con 0x7f9240159c60 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9246686700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924015e140 con 0x7f9240159890 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f924015db10 0x7f9240069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f924015db10 0x7f9240069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56228/0 (socket says 172.21.15.170:56228) 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 -- 172.21.15.170:0/3715305317 learned_addr learned my addr 172.21.15.170:0/3715305317 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:00.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f923ffff700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9240159890 0x7f924015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 -- 172.21.15.170:0/3715305317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9240159890 msgr2=0x7f924015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9240159890 0x7f924015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 -- 172.21.15.170:0/3715305317 >> v2:172.21.15.191:3300/0 conn(0x7f9240159c60 msgr2=0x7f924015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f923f7fe700 1 --2- 172.21.15.170:0/3715305317 >> v2:172.21.15.191:3300/0 conn(0x7f9240159c60 0x7f924015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 --2- 172.21.15.170:0/3715305317 >> v2:172.21.15.191:3300/0 conn(0x7f9240159c60 0x7f924015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 -- 172.21.15.170:0/3715305317 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924015a3d0 con 0x7f924015db10 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f9244c23700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f924015db10 0x7f9240069560 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f92400650f0 tx=0x7f9238009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5865d7f22af2e4df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f923effd700 1 -- 172.21.15.170:0/3715305317 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800e070 con 0x7f924015db10 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f923effd700 1 -- 172.21.15.170:0/3715305317 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800ba50 con 0x7f924015db10 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.634+0000 7f923effd700 1 -- 172.21.15.170:0/3715305317 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380092e0 con 0x7f924015db10 2023-10-08T16:16:00.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 -- 172.21.15.170:0/3715305317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f924015db10 msgr2=0x7f9240069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f924015db10 0x7f9240069560 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f92400650f0 tx=0x7f9238009840 comp rx=0 tx=0).stop 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 -- 172.21.15.170:0/3715305317 shutdown_connections 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f924015db10 0x7f9240069560 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 --2- 172.21.15.170:0/3715305317 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9240159890 0x7f924015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 --2- 172.21.15.170:0/3715305317 >> v2:172.21.15.191:3300/0 conn(0x7f9240159c60 0x7f924015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 -- 172.21.15.170:0/3715305317 >> 172.21.15.170:0/3715305317 conn(0x7f9240068e20 msgr2=0x7f92400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 -- 172.21.15.170:0/3715305317 shutdown_connections 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.635+0000 7f9246686700 1 -- 172.21.15.170:0/3715305317 wait complete. 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.636+0000 7f9246686700 1 Processor -- start 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.636+0000 7f9246686700 1 -- start start 2023-10-08T16:16:00.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9240159c60 0x7f92400b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f924015db10 0x7f92400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b2d80 con 0x7f9240159890 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b28f0 con 0x7f9240159c60 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9246686700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b1fc0 con 0x7f924015db10 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56244/0 (socket says 172.21.15.170:56244) 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 -- 172.21.15.170:0/1226370204 learned_addr learned my addr 172.21.15.170:0/1226370204 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f9244c23700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f924015db10 0x7f92400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923f7fe700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9240159c60 0x7f92400b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 -- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f924015db10 msgr2=0x7f92400b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f924015db10 0x7f92400b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 -- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9240159c60 msgr2=0x7f92400b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9240159c60 0x7f92400b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.637+0000 7f923ffff700 1 -- 172.21.15.170:0/1226370204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238009ae0 con 0x7f9240159890 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f923ffff700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f923000d890 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300125f0 con 0x7f9240159890 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9230012750 con 0x7f9240159890 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230012af0 con 0x7f9240159890 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92400b22a0 con 0x7f9240159890 2023-10-08T16:16:00.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.638+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92400b1cb0 con 0x7f9240159890 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.639+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924004c3c0 con 0x7f9240159890 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.639+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f923001e460 con 0x7f9240159890 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.639+0000 7f9227fff700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 0x7f922c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.640+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5792+0+0 (secure 0 0 0) 0x7f92300851f0 con 0x7f9240159890 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.640+0000 7f923f7fe700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 0x7f922c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.640+0000 7f923f7fe700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 0x7f922c0697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f924016ceb0 tx=0x7f9234008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:00.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.644+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9230051e30 con 0x7f9240159890 2023-10-08T16:16:00.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.870+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f924006a5b0 con 0x7f9240159890 2023-10-08T16:16:00.871 INFO:tasks.ceph.mon.a.smithi170.stderr:2023-10-08T16:16:00.870+0000 7fc73ff60700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T16:16:00.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.875+0000 7f9227fff700 1 -- 172.21.15.170:0/1226370204 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v133) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9230055450 con 0x7f9240159890 2023-10-08T16:16:00.877 INFO:tasks.ceph.mds.c.smithi170.stderr: -3252> 2023-10-08T16:13:52.003+0000 7fdb92328700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 msgr2=0x7f922c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 0x7f922c0697d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f924016ceb0 tx=0x7f9234008040 comp rx=0 tx=0).stop 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 msgr2=0x7f9240158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f923000d890 tx=0x7f9230008040 comp rx=0 tx=0).stop 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 shutdown_connections 2023-10-08T16:16:00.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f924015db10 0x7f92400b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f922c067320 0x7f922c0697d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9240159890 0x7f9240158900 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 --2- 172.21.15.170:0/1226370204 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9240159c60 0x7f92400b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 >> 172.21.15.170:0/1226370204 conn(0x7f9240068e20 msgr2=0x7f924015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 shutdown_connections 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:00.877+0000 7f9246686700 1 -- 172.21.15.170:0/1226370204 wait complete. 2023-10-08T16:16:00.879 INFO:teuthology.orchestra.run.smithi170.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T16:16:00.890 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T16:16:00.901 INFO:tasks.ceph.mds.c.smithi170.stdout:starting mds.c at 2023-10-08T16:16:01.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.008+0000 7fbd67630700 1 Processor -- start 2023-10-08T16:16:01.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 -- start start 2023-10-08T16:16:01.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd60159b10 0x7fbd600cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.011 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd6015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 0x7fbd6015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd600c74b0 con 0x7fbd60159b10 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6015e0e0 con 0x7fbd60159ee0 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.009+0000 7fbd67630700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600c8700 con 0x7fbd6015d730 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd60159b10 0x7fbd600cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbd60159b10 0x7fbd600cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56278/0 (socket says 172.21.15.170:56278) 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 -- 172.21.15.170:0/32139118 learned_addr learned my addr 172.21.15.170:0/32139118 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd65bcd700 1 --2- 172.21.15.170:0/32139118 >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 0x7fbd6015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd6015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 -- 172.21.15.170:0/32139118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 msgr2=0x7fbd6015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd6015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 -- 172.21.15.170:0/32139118 >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 msgr2=0x7fbd6015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 --2- 172.21.15.170:0/32139118 >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 0x7fbd6015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 -- 172.21.15.170:0/32139118 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600c9070 con 0x7fbd60159b10 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd6015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:01.013 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd65bcd700 1 --2- 172.21.15.170:0/32139118 >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 0x7fbd6015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd653cc700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd600cb3e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fbd600ca830 tx=0x7fbd5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6292399955154f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd57fff700 1 -- 172.21.15.170:0/32139118 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c00e070 con 0x7fbd60159b10 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd57fff700 1 -- 172.21.15.170:0/32139118 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c00bae0 con 0x7fbd60159b10 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.010+0000 7fbd57fff700 1 -- 172.21.15.170:0/32139118 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0092e0 con 0x7fbd60159b10 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 -- 172.21.15.170:0/32139118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 msgr2=0x7fbd600cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd600cb3e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fbd600ca830 tx=0x7fbd5c009840 comp rx=0 tx=0).stop 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 -- 172.21.15.170:0/32139118 shutdown_connections 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 --2- 172.21.15.170:0/32139118 >> v2:172.21.15.191:3300/0 conn(0x7fbd60159ee0 0x7fbd6015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd600cb3e0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 --2- 172.21.15.170:0/32139118 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd6015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 -- 172.21.15.170:0/32139118 >> 172.21.15.170:0/32139118 conn(0x7fbd600c9870 msgr2=0x7fbd600c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 -- 172.21.15.170:0/32139118 shutdown_connections 2023-10-08T16:16:01.015 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.011+0000 7fbd67630700 1 -- 172.21.15.170:0/32139118 wait complete. 2023-10-08T16:16:01.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.012+0000 7fbd67630700 1 Processor -- start 2023-10-08T16:16:01.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.012+0000 7fbd67630700 1 -- start start 2023-10-08T16:16:01.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 0x7fbd60156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd60156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd600c4a10 con 0x7fbd60159b10 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6015b430 con 0x7fbd60159ee0 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6015b710 con 0x7fbd6015d730 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56280/0 (socket says 172.21.15.170:56280) 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 -- 172.21.15.170:0/957536193 learned_addr learned my addr 172.21.15.170:0/957536193 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd65bcd700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd60156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.019 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 0x7fbd60156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 -- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 msgr2=0x7fbd60156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd60156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 -- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 msgr2=0x7fbd60156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 0x7fbd60156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 -- 172.21.15.170:0/957536193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009ae0 con 0x7fbd60159b10 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 0x7fbd60156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd653cc700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fbd6006cc50 tx=0x7fbd5c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0094e0 con 0x7fbd60159b10 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5c008be0 con 0x7fbd60159b10 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6015b9f0 con 0x7fbd60159b10 2023-10-08T16:16:01.020 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.013+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c026460 con 0x7fbd60159b10 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.014+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6015bfb0 con 0x7fbd60159b10 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.014+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6004c3c0 con 0x7fbd60159b10 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.015+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fbd5c003040 con 0x7fbd60159b10 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.015+0000 7fbd557fa700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 0x7fbd440697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.015+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5878+0+0 (secure 0 0 0) 0x7fbd5c086d90 con 0x7fbd60159b10 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.015+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 0x7fbd440697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.016+0000 7fbd64bcb700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 0x7fbd440697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbd60156f10 tx=0x7fbd50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:01.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.019+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd5c053860 con 0x7fbd60159b10 2023-10-08T16:16:01.247 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.247+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbd600633e0 con 0x7fbd60159b10 2023-10-08T16:16:01.875 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.875+0000 7fbd557fa700 1 -- 172.21.15.170:0/957536193 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v134) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbd5c056e80 con 0x7fbd60159b10 2023-10-08T16:16:01.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.877+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 msgr2=0x7fbd440697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:01.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.877+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 0x7fbd440697d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fbd60156f10 tx=0x7fbd50008040 comp rx=0 tx=0).stop 2023-10-08T16:16:01.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.877+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 msgr2=0x7fbd601487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:01.878 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.877+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fbd6006cc50 tx=0x7fbd5c018000 comp rx=0 tx=0).stop 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 shutdown_connections 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fbd6015d730 0x7fbd60156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fbd44067320 0x7fbd440697d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd60159b10 0x7fbd601487b0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 --2- 172.21.15.170:0/957536193 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fbd60159ee0 0x7fbd60156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 >> 172.21.15.170:0/957536193 conn(0x7fbd600c9870 msgr2=0x7fbd60154a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 shutdown_connections 2023-10-08T16:16:01.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:01.878+0000 7fbd67630700 1 -- 172.21.15.170:0/957536193 wait complete. 2023-10-08T16:16:01.890 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T16:16:02.051 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.051+0000 7f8b9941c700 1 Processor -- start 2023-10-08T16:16:02.052 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.051+0000 7f8b9941c700 1 -- start start 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b9941c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b94159ec0 0x7f8b940b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b9941c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 0x7f8b940cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b9941c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b940b5b00 con 0x7f8b940b53d0 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b9941c700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b940b5850 con 0x7f8b94159ec0 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b9941c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b940b5da0 con 0x7f8b940cf350 2023-10-08T16:16:02.054 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b92ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b94159ec0 0x7f8b940b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b8a7fc700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8b940b53d0 0x7f8b940cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52724/0 (socket says 172.21.15.170:52724) 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b8a7fc700 1 -- 172.21.15.170:0/2142081279 learned_addr learned my addr 172.21.15.170:0/2142081279 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b937fe700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 0x7f8b940cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b92ffd700 1 -- 172.21.15.170:0/2142081279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 msgr2=0x7f8b940cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b92ffd700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 0x7f8b940cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b92ffd700 1 -- 172.21.15.170:0/2142081279 >> v1:172.21.15.170:6789/0 conn(0x7f8b940b53d0 legacy=0x7f8b940cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b92ffd700 1 -- 172.21.15.170:0/2142081279 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940b6080 con 0x7f8b94159ec0 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b927fc700 1 -- 172.21.15.170:0/2142081279 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092095366 0 0) 0x7f8b940b5b00 con 0x7f8b940b53d0 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.052+0000 7f8b937fe700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 0x7f8b940cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:02.055 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b92ffd700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b94159ec0 0x7f8b940b4e90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8b840098d0 tx=0x7f8b84009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efab6f3c817de4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b927fc700 1 -- 172.21.15.170:0/2142081279 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84009e70 con 0x7f8b94159ec0 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b927fc700 1 -- 172.21.15.170:0/2142081279 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84005c10 con 0x7f8b94159ec0 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b927fc700 1 -- 172.21.15.170:0/2142081279 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400b650 con 0x7f8b94159ec0 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 -- 172.21.15.170:0/2142081279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b94159ec0 msgr2=0x7f8b940b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b94159ec0 0x7f8b940b4e90 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f8b840098d0 tx=0x7f8b84009120 comp rx=0 tx=0).stop 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 -- 172.21.15.170:0/2142081279 shutdown_connections 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b940cf350 0x7f8b940cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 --2- 172.21.15.170:0/2142081279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b94159ec0 0x7f8b940b4e90 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 -- 172.21.15.170:0/2142081279 >> 172.21.15.170:0/2142081279 conn(0x7f8b940999f0 msgr2=0x7f8b940b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 -- 172.21.15.170:0/2142081279 shutdown_connections 2023-10-08T16:16:02.056 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.053+0000 7f8b9941c700 1 -- 172.21.15.170:0/2142081279 wait complete. 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 Processor -- start 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 -- start start 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b940b53d0 0x7f8b940c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b94159ec0 0x7f8b941530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.059 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9406b010 con 0x7f8b940cf350 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94153750 con 0x7f8b940b53d0 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b9941c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b94153a30 con 0x7f8b94159ec0 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56302/0 (socket says 172.21.15.170:56302) 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.055+0000 7f8b8a7fc700 1 -- 172.21.15.170:0/1449734688 learned_addr learned my addr 172.21.15.170:0/1449734688 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b937fe700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b94159ec0 0x7f8b941530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b92ffd700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b940b53d0 0x7f8b940c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 -- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b94159ec0 msgr2=0x7f8b941530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b94159ec0 0x7f8b941530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 -- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b940b53d0 msgr2=0x7f8b940c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.060 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b940b53d0 0x7f8b940c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 -- 172.21.15.170:0/1449734688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840093c0 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8a7fc700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c5b0 tx=0x7f8b7c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c005b30 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c005c90 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c007320 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b94153d10 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.056+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b94156ae0 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.057+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b94157100 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.058+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f8b7c01b400 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.058+0000 7f8b8b7fe700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 0x7f8b740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.058+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5878+0+0 (secure 0 0 0) 0x7f8b7c087210 con 0x7f8b940cf350 2023-10-08T16:16:02.061 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.058+0000 7f8b92ffd700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 0x7f8b740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:02.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.058+0000 7f8b92ffd700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 0x7f8b740697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009160 tx=0x7f8b84016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:02.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.062+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b94157100 con 0x7f8b940cf350 2023-10-08T16:16:02.246 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.245+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b94157100 con 0x7f8b940cf350 2023-10-08T16:16:02.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.883+0000 7f8b8b7fe700 1 -- 172.21.15.170:0/1449734688 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v205) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b94157100 con 0x7f8b940cf350 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 msgr2=0x7f8b740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 0x7f8b740697d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009160 tx=0x7f8b84016000 comp rx=0 tx=0).stop 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 msgr2=0x7f8b94152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00c5b0 tx=0x7f8b7c001070 comp rx=0 tx=0).stop 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 shutdown_connections 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f8b94159ec0 0x7f8b941530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f8b940b53d0 0x7f8b940c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f8b74067320 0x7f8b740697d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 --2- 172.21.15.170:0/1449734688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b940cf350 0x7f8b94152b70 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:02.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 >> 172.21.15.170:0/1449734688 conn(0x7f8b940999f0 msgr2=0x7f8b940cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:02.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.886+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 shutdown_connections 2023-10-08T16:16:02.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:02.887+0000 7f8b9941c700 1 -- 172.21.15.170:0/1449734688 wait complete. 2023-10-08T16:16:02.888 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_metadata' removed 2023-10-08T16:16:02.903 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T16:16:03.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.022+0000 7f3850179700 1 Processor -- start 2023-10-08T16:16:03.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.022+0000 7f3850179700 1 -- start start 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f3850179700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 0x7f38480699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f3850179700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f38480af8c0 0x7f3848159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f3850179700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3848068900 con 0x7f3848068390 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f3850179700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3848068be0 con 0x7f38480af8c0 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f3850179700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3848068ec0 con 0x7f384806ab40 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f384e716700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f38480af8c0 0x7f3848159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f384e716700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f38480af8c0 0x7f3848159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56308/0 (socket says 172.21.15.170:56308) 2023-10-08T16:16:03.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.023+0000 7f384e716700 1 -- 172.21.15.170:0/1419843475 learned_addr learned my addr 172.21.15.170:0/1419843475 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384df15700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 0x7f38480699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384e716700 1 -- 172.21.15.170:0/1419843475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 msgr2=0x7f38480699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384e716700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 0x7f38480699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384e716700 1 -- 172.21.15.170:0/1419843475 >> v1:172.21.15.191:6789/0 conn(0x7f3848068390 legacy=0x7f38480c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384e716700 1 -- 172.21.15.170:0/1419843475 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480691a0 con 0x7f38480af8c0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384df15700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 0x7f38480699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384e716700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f3848159620 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f38400099b0 tx=0x7f3840009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9387b3f49652850c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384cf13700 1 -- 172.21.15.170:0/1419843475 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384000b820 con 0x7f38480af8c0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384cf13700 1 -- 172.21.15.170:0/1419843475 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384000b980 con 0x7f38480af8c0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f384cf13700 1 -- 172.21.15.170:0/1419843475 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384000bd20 con 0x7f38480af8c0 2023-10-08T16:16:03.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 -- 172.21.15.170:0/1419843475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 msgr2=0x7f3848159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f3848159620 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f38400099b0 tx=0x7f3840009250 comp rx=0 tx=0).stop 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 -- 172.21.15.170:0/1419843475 shutdown_connections 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f3848159620 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 --2- 172.21.15.170:0/1419843475 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f384806ab40 0x7f38480699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.024+0000 7f3850179700 1 -- 172.21.15.170:0/1419843475 >> 172.21.15.170:0/1419843475 conn(0x7f38480cd640 msgr2=0x7f38480cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.025+0000 7f3850179700 1 -- 172.21.15.170:0/1419843475 shutdown_connections 2023-10-08T16:16:03.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.025+0000 7f3850179700 1 -- 172.21.15.170:0/1419843475 wait complete. 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 Processor -- start 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 -- start start 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3848068390 0x7f3848153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f384806ab40 0x7f38481553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38481568c0 con 0x7f38480af8c0 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3848155d40 con 0x7f384806ab40 2023-10-08T16:16:03.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f3850179700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3848156020 con 0x7f3848068390 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f384e716700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.026+0000 7f384e716700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56310/0 (socket says 172.21.15.170:56310) 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 -- 172.21.15.170:0/1895540795 learned_addr learned my addr 172.21.15.170:0/1895540795 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384df15700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3848068390 0x7f3848153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384d714700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f384806ab40 0x7f38481553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 -- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3848068390 msgr2=0x7f3848153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3848068390 0x7f3848153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 -- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f384806ab40 msgr2=0x7f38481553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f384806ab40 0x7f38481553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 -- 172.21.15.170:0/1895540795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38400094f0 con 0x7f38480af8c0 2023-10-08T16:16:03.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f384e716700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3840009290 tx=0x7f3840016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384001c460 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384001c5c0 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3848156300 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384001c960 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.027+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848153a40 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.028+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38480c4bd0 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.029+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f384001cd50 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.029+0000 7f3845ffb700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 0x7f38340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.029+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5440+0+0 (secure 0 0 0) 0x7f3840087140 con 0x7f38480af8c0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.029+0000 7f384df15700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 0x7f38340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:03.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.029+0000 7f384df15700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 0x7f38340697d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f383800d6e0 tx=0x7f3838007850 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:03.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.033+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3840053ee0 con 0x7f38480af8c0 2023-10-08T16:16:03.214 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.214+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38480c4bd0 con 0x7f38480af8c0 2023-10-08T16:16:03.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.885+0000 7f3845ffb700 1 -- 172.21.15.170:0/1895540795 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v206) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3840057500 con 0x7f38480af8c0 2023-10-08T16:16:03.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 msgr2=0x7f38340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:03.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 0x7f38340697d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f383800d6e0 tx=0x7f3838007850 comp rx=0 tx=0).stop 2023-10-08T16:16:03.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 msgr2=0x7f38480c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:03.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3840009290 tx=0x7f3840016000 comp rx=0 tx=0).stop 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 shutdown_connections 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3834067320 0x7f38340697d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38480af8c0 0x7f38480c30e0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3848068390 0x7f3848153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 --2- 172.21.15.170:0/1895540795 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f384806ab40 0x7f38481553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 >> 172.21.15.170:0/1895540795 conn(0x7f38480cd640 msgr2=0x7f38480b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 shutdown_connections 2023-10-08T16:16:03.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:03.889+0000 7f3850179700 1 -- 172.21.15.170:0/1895540795 wait complete. 2023-10-08T16:16:03.893 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephfs_data' removed 2023-10-08T16:16:03.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T16:16:04.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.022+0000 7f3b33c95700 1 Processor -- start 2023-10-08T16:16:04.023 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.022+0000 7f3b33c95700 1 -- start start 2023-10-08T16:16:04.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b33c95700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b2c159890 0x7f3b2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b33c95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c069560 0x7f3b2c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b33c95700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c069ee0 con 0x7f3b2c159c60 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b33c95700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c15a160 con 0x7f3b2c159890 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b33c95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0d13a0 con 0x7f3b2c069560 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b31a31700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b2c159890 0x7f3b2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b31a31700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3b2c159890 0x7f3b2c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56318/0 (socket says 172.21.15.170:56318) 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b31a31700 1 -- 172.21.15.170:0/830527405 learned_addr learned my addr 172.21.15.170:0/830527405 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.023+0000 7f3b32232700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c069560 0x7f3b2c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b31a31700 1 -- 172.21.15.170:0/830527405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c069560 msgr2=0x7f3b2c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b31a31700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c069560 0x7f3b2c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b31a31700 1 -- 172.21.15.170:0/830527405 >> v1:172.21.15.191:6789/0 conn(0x7f3b2c159c60 legacy=0x7f3b2c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b31a31700 1 -- 172.21.15.170:0/830527405 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c15e140 con 0x7f3b2c159890 2023-10-08T16:16:04.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b31a31700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c159890 0x7f3b2c15abd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009f20 tx=0x7f3b28009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd7e769efd4e76d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b30a2f700 1 -- 172.21.15.170:0/830527405 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2800e070 con 0x7f3b2c159890 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b30a2f700 1 -- 172.21.15.170:0/830527405 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2800ba80 con 0x7f3b2c159890 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b30a2f700 1 -- 172.21.15.170:0/830527405 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280092e0 con 0x7f3b2c159890 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 -- 172.21.15.170:0/830527405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c159890 msgr2=0x7f3b2c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c159890 0x7f3b2c15abd0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f3b28009f20 tx=0x7f3b28009840 comp rx=0 tx=0).stop 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 -- 172.21.15.170:0/830527405 shutdown_connections 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c069560 0x7f3b2c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 --2- 172.21.15.170:0/830527405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c159890 0x7f3b2c15abd0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.024+0000 7f3b33c95700 1 -- 172.21.15.170:0/830527405 >> 172.21.15.170:0/830527405 conn(0x7f3b2c068e20 msgr2=0x7f3b2c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.025+0000 7f3b33c95700 1 -- 172.21.15.170:0/830527405 shutdown_connections 2023-10-08T16:16:04.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.025+0000 7f3b33c95700 1 -- 172.21.15.170:0/830527405 wait complete. 2023-10-08T16:16:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 Processor -- start 2023-10-08T16:16:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 -- start start 2023-10-08T16:16:04.030 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c159890 0x7f3b2c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 0x7f3b2c0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0c78a0 con 0x7f3b2c069560 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0b41c0 con 0x7f3b2c159c60 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b33c95700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0b44a0 con 0x7f3b2c159890 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.026+0000 7f3b31a31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56322/0 (socket says 172.21.15.170:56322) 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 -- 172.21.15.170:0/1388094679 learned_addr learned my addr 172.21.15.170:0/1388094679 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31230700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c159890 0x7f3b2c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b32232700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 0x7f3b2c0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 -- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c159890 msgr2=0x7f3b2c0b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c159890 0x7f3b2c0b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 -- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 msgr2=0x7f3b2c0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 0x7f3b2c0b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 -- 172.21.15.170:0/1388094679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28009ae0 con 0x7f3b2c069560 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b32232700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 0x7f3b2c0b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b31a31700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b6c0 tx=0x7f3b28018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b28008bb0 con 0x7f3b2c069560 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b28008d10 con 0x7f3b2c069560 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c0c7af0 con 0x7f3b2c069560 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b280266b0 con 0x7f3b2c069560 2023-10-08T16:16:04.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.027+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c147730 con 0x7f3b2c069560 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.028+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c0c7ff0 con 0x7f3b2c069560 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.029+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f3b28003040 con 0x7f3b2c069560 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.029+0000 7f3b21ffb700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 0x7f3b100697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.029+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f3b28086430 con 0x7f3b2c069560 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.029+0000 7f3b31230700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 0x7f3b100697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.029+0000 7f3b31230700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 0x7f3b100697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0c6ec0 tx=0x7f3b1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.033+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b28053380 con 0x7f3b2c069560 2023-10-08T16:16:04.216 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.216+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b2c0b4660 con 0x7f3b2c069560 2023-10-08T16:16:04.220 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.219+0000 7f3b21ffb700 1 -- 172.21.15.170:0/1388094679 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b280569a0 con 0x7f3b2c069560 2023-10-08T16:16:04.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 msgr2=0x7f3b100697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 0x7f3b100697d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0c6ec0 tx=0x7f3b1c009040 comp rx=0 tx=0).stop 2023-10-08T16:16:04.222 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 msgr2=0x7f3b2c1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b6c0 tx=0x7f3b28018000 comp rx=0 tx=0).stop 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 shutdown_connections 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f3b2c159c60 0x7f3b2c0b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f3b10067320 0x7f3b100697d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3b2c069560 0x7f3b2c1471f0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 --2- 172.21.15.170:0/1388094679 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f3b2c159890 0x7f3b2c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 >> 172.21.15.170:0/1388094679 conn(0x7f3b2c068e20 msgr2=0x7f3b2c0b2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 shutdown_connections 2023-10-08T16:16:04.223 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.222+0000 7f3b33c95700 1 -- 172.21.15.170:0/1388094679 wait complete. 2023-10-08T16:16:04.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm' 2023-10-08T16:16:04.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.348+0000 7efd3f320700 1 Processor -- start 2023-10-08T16:16:04.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.348+0000 7efd3f320700 1 -- start start 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3f320700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efd38159890 0x7efd3815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3f320700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3815a160 con 0x7efd38159c60 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3f320700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380d13a0 con 0x7efd38069560 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3f320700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3815e140 con 0x7efd38159890 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3d0bc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7efd38159890 0x7efd3815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3c8bb700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7efd38159c60 0x7efd3815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52728/0 (socket says 172.21.15.170:52728) 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.349+0000 7efd3c8bb700 1 -- 172.21.15.170:0/2943574242 learned_addr learned my addr 172.21.15.170:0/2943574242 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3d0bc700 1 -- 172.21.15.170:0/2943574242 >> v1:172.21.15.191:6789/0 conn(0x7efd38069560 legacy=0x7efd380ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3d0bc700 1 -- 172.21.15.170:0/2943574242 >> v1:172.21.15.170:6789/0 conn(0x7efd38159c60 legacy=0x7efd3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3d0bc700 1 -- 172.21.15.170:0/2943574242 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3815a3d0 con 0x7efd38159890 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd2ffff700 1 -- 172.21.15.170:0/2943574242 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134455777 0 0) 0x7efd3815a160 con 0x7efd38159c60 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3d0bc700 1 --2- 172.21.15.170:0/2943574242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd3815abd0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e60 tx=0x7efd34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b08ceb9326fe83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd2ffff700 1 -- 172.21.15.170:0/2943574242 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400e070 con 0x7efd38159890 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd2ffff700 1 -- 172.21.15.170:0/2943574242 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400bc30 con 0x7efd38159890 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd2ffff700 1 -- 172.21.15.170:0/2943574242 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34012430 con 0x7efd38159890 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3f320700 1 -- 172.21.15.170:0/2943574242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 msgr2=0x7efd3815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.350+0000 7efd3f320700 1 --2- 172.21.15.170:0/2943574242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd3815abd0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e60 tx=0x7efd34009780 comp rx=0 tx=0).stop 2023-10-08T16:16:04.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.351+0000 7efd3f320700 1 -- 172.21.15.170:0/2943574242 shutdown_connections 2023-10-08T16:16:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.351+0000 7efd3f320700 1 --2- 172.21.15.170:0/2943574242 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd3815abd0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.351+0000 7efd3f320700 1 -- 172.21.15.170:0/2943574242 >> 172.21.15.170:0/2943574242 conn(0x7efd38068e20 msgr2=0x7efd380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.351+0000 7efd3f320700 1 -- 172.21.15.170:0/2943574242 shutdown_connections 2023-10-08T16:16:04.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.351+0000 7efd3f320700 1 -- 172.21.15.170:0/2943574242 wait complete. 2023-10-08T16:16:04.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.352+0000 7efd3f320700 1 Processor -- start 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.352+0000 7efd3f320700 1 -- start start 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd38069560 0x7efd38152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 0x7efd380b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38153190 con 0x7efd38159890 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c96f0 con 0x7efd38069560 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c99d0 con 0x7efd38159c60 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:56336/0 (socket says 172.21.15.170:56336) 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 -- 172.21.15.170:0/996747766 learned_addr learned my addr 172.21.15.170:0/996747766 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:04.356 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3d0bc700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd38069560 0x7efd38152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3d8bd700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 0x7efd380b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 -- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 msgr2=0x7efd380b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 0x7efd380b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 -- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd38069560 msgr2=0x7efd38152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd38069560 0x7efd38152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 -- 172.21.15.170:0/996747766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009a20 con 0x7efd38159890 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3d8bd700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 0x7efd380b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3c8bb700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c5b0 tx=0x7efd28001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28005b30 con 0x7efd38159890 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd28005c90 con 0x7efd38159890 2023-10-08T16:16:04.357 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28007300 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380c9cb0 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.353+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380c93e0 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.354+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3804c3c0 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.355+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7efd2801b3b0 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.355+0000 7efd2d7fa700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 0x7efd1c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.355+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7efd28083eb0 con 0x7efd38159890 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.355+0000 7efd3d0bc700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 0x7efd1c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:04.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.355+0000 7efd3d0bc700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 0x7efd1c0697d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e60 tx=0x7efd34019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:04.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.359+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd28050e40 con 0x7efd38159890 2023-10-08T16:16:04.623 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.623+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]} v 0) v1 -- 0x7efd380b1360 con 0x7efd38159890 2023-10-08T16:16:04.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.886+0000 7efd2d7fa700 1 -- 172.21.15.170:0/996747766 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestVolumes.test_volume_rm"]}]=0 v552) v1 ==== 131+0+0 (secure 0 0 0) 0x7efd280170c0 con 0x7efd38159890 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.888+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 msgr2=0x7efd1c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.888+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 0x7efd1c0697d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e60 tx=0x7efd34019000 comp rx=0 tx=0).stop 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.888+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 msgr2=0x7efd380c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.888+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c5b0 tx=0x7efd28001070 comp rx=0 tx=0).stop 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 shutdown_connections 2023-10-08T16:16:04.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7efd38159c60 0x7efd380b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7efd38069560 0x7efd38152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7efd1c067320 0x7efd1c0697d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 --2- 172.21.15.170:0/996747766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd38159890 0x7efd380c8ea0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 >> 172.21.15.170:0/996747766 conn(0x7efd38068e20 msgr2=0x7efd380b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 shutdown_connections 2023-10-08T16:16:04.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:04.889+0000 7efd3f320700 1 -- 172.21.15.170:0/996747766 wait complete. 2023-10-08T16:16:04.903 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used ... ERROR 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-08T16:16:04.904 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-08T16:16:04.905 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-08T16:16:04.906 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.907 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:Ran 12 tests in 662.147s 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:====================================================================== 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:ERROR: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:That the volume can only be removed when --yes-i-really-mean-it is used 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 497, in test_volume_rm 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname) 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-08T16:16:04.908 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs' 2023-10-08T16:16:04.909 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 503, in test_volume_rm 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: self._fs_cmd("volume", "rm", self.volname, "--yes-i-really-mean-it") 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs/test_volumes.py", line 38, in _fs_cmd 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: return self.mgr_cluster.mon_manager.raw_cluster_cmd("fs", *args) 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1624, in raw_cluster_cmd 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: return self.run_cluster_cmd(**kwargs).stdout.getvalue() 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/ceph_manager.py", line 1615, in run_cluster_cmd 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-10-08T16:16:04.910 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: r.wait() 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it' 2023-10-08T16:16:04.911 INFO:tasks.cephfs_test_runner: 2023-10-08T16:16:04.911 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:16:04.977 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=cd12955a3bbb45efa24670a233ad2067 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes) 2023-10-08T16:16:04.979 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-08T16:16:04.987 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T16:16:05.024 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T16:16:05.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.024 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:16:05.042 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T16:16:05.042 INFO:teuthology.orchestra.run.smithi170.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T16:16:05.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.042 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T16:16:05.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.119 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T16:16:05.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.204 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-08T16:16:05.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.279 DEBUG:teuthology.orchestra.run.smithi170:> ip netns list 2023-10-08T16:16:05.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo ip link delete ceph-brx 2023-10-08T16:16:05.360 INFO:teuthology.orchestra.run.smithi170.stderr:Cannot find device "ceph-brx" 2023-10-08T16:16:05.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:05.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.362 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:16:05.381 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T16:16:05.381 INFO:teuthology.orchestra.run.smithi191.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T16:16:05.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.381 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T16:16:05.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.463 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T16:16:05.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.539 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T16:16:05.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.635 DEBUG:teuthology.orchestra.run.smithi191:> ip netns list 2023-10-08T16:16:05.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T16:16:05.651 DEBUG:teuthology.orchestra.run.smithi191:> sudo ip link delete ceph-brx 2023-10-08T16:16:05.720 INFO:teuthology.orchestra.run.smithi191.stderr:Cannot find device "ceph-brx" 2023-10-08T16:16:05.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:05.722 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-08T16:16:05.731 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-08T16:16:05.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:16:05.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.847+0000 7f537c559700 1 Processor -- start 2023-10-08T16:16:05.848 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.847+0000 7f537c559700 1 -- start start 2023-10-08T16:16:05.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537c559700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f53740b1c20 0x7f53741572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:05.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537c559700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5374157c60 con 0x7f5374157820 2023-10-08T16:16:05.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537c559700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53740cb300 con 0x7f53740b1850 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537c559700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537406fc30 con 0x7f53740b1c20 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537aaf6700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5374157820 0x7f53740abf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43350/0 (socket says 172.21.15.170:43350) 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.848+0000 7f537aaf6700 1 -- 172.21.15.170:0/382947039 learned_addr learned my addr 172.21.15.170:0/382947039 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f5379af4700 1 --2- 172.21.15.170:0/382947039 >> v2:172.21.15.191:3300/0 conn(0x7f53740b1c20 0x7f53741572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092365924 0 0) 0x7f5374157c60 con 0x7f5374157820 2023-10-08T16:16:05.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5360003570 con 0x7f5374157820 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289526016 0 0) 0x7f53740cb300 con 0x7f53740b1850 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374157c60 con 0x7f53740b1850 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f5379af4700 1 -- 172.21.15.170:0/382947039 >> v1:172.21.15.191:6789/0 conn(0x7f53740b1850 legacy=0x7f53740ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f5379af4700 1 -- 172.21.15.170:0/382947039 >> v1:172.21.15.170:6789/0 conn(0x7f5374157820 legacy=0x7f53740abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f5379af4700 1 -- 172.21.15.170:0/382947039 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740cb5e0 con 0x7f53740b1c20 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f5379af4700 1 --2- 172.21.15.170:0/382947039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f53741572e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f53640098e0 tx=0x7f53640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232b6791608ede2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:05.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364011630 con 0x7f53740b1c20 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5364011790 con 0x7f53740b1c20 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.849+0000 7f53792f3700 1 -- 172.21.15.170:0/382947039 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5364011b30 con 0x7f53740b1c20 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 -- 172.21.15.170:0/382947039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 msgr2=0x7f53741572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 --2- 172.21.15.170:0/382947039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f53741572e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f53640098e0 tx=0x7f53640091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 -- 172.21.15.170:0/382947039 shutdown_connections 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 --2- 172.21.15.170:0/382947039 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f53741572e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 -- 172.21.15.170:0/382947039 >> 172.21.15.170:0/382947039 conn(0x7f537406a320 msgr2=0x7f53740cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:05.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 -- 172.21.15.170:0/382947039 shutdown_connections 2023-10-08T16:16:05.854 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.850+0000 7f537c559700 1 -- 172.21.15.170:0/382947039 wait complete. 2023-10-08T16:16:05.856 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.851+0000 7f537c559700 1 Processor -- start 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.851+0000 7f537c559700 1 -- start start 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f5374149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5374157820 0x7f53740ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53740cc260 con 0x7f53740b1850 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537416a2c0 con 0x7f53740b1c20 2023-10-08T16:16:05.857 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537416a5a0 con 0x7f5374157820 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34106/0 (socket says 172.21.15.170:34106) 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 -- 172.21.15.170:0/2119096189 learned_addr learned my addr 172.21.15.170:0/2119096189 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f5379af4700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f5374149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537aaf6700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5374157820 0x7f53740ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 -- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5374157820 msgr2=0x7f53740ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:05.858 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5374157820 0x7f53740ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 -- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 msgr2=0x7f5374149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f5374149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 -- 172.21.15.170:0/2119096189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5364009420 con 0x7f53740b1850 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537a2f5700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f537000cb10 tx=0x7f5370002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000ce20 con 0x7f53740b1850 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537000a7e0 con 0x7f53740b1850 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537416a880 con 0x7f53740b1850 2023-10-08T16:16:05.859 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000ab80 con 0x7f53740b1850 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.852+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5374169f90 con 0x7f53740b1850 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.853+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537404c3c0 con 0x7f53740b1850 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.854+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5370012420 con 0x7f53740b1850 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.854+0000 7f536a7fc700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 0x7f5358069800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.854+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5370084c70 con 0x7f53740b1850 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.854+0000 7f5379af4700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 0x7f5358069800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:05.860 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.855+0000 7f5379af4700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 0x7f5358069800 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f53640093f0 tx=0x7f5364017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:05.861 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:05.859+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5370051c00 con 0x7f53740b1850 2023-10-08T16:16:06.045 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.045+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53740cb8c0 con 0x7f53740b1850 2023-10-08T16:16:06.046 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.046+0000 7f536a7fc700 1 -- 172.21.15.170:0/2119096189 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+12323 (secure 0 0 0) 0x7f5370055220 con 0x7f53740b1850 2023-10-08T16:16:06.046 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:06.047 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":206,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:16:03.882728+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6837/1430346956":"2023-10-09T16:16:00.871385+0000","172.21.15.170:6835/1430346956":"2023-10-09T16:16:00.871385+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 msgr2=0x7f5358069800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 0x7f5358069800 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f53640093f0 tx=0x7f5364017000 comp rx=0 tx=0).stop 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 msgr2=0x7f53740b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f537000cb10 tx=0x7f5370002820 comp rx=0 tx=0).stop 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 shutdown_connections 2023-10-08T16:16:06.049 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5374157820 0x7f53740ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5358067350 0x7f5358069800 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53740b1850 0x7f53740b1020 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 --2- 172.21.15.170:0/2119096189 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f53740b1c20 0x7f5374149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 >> 172.21.15.170:0/2119096189 conn(0x7f537406a320 msgr2=0x7f53740704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 shutdown_connections 2023-10-08T16:16:06.050 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.048+0000 7f537c559700 1 -- 172.21.15.170:0/2119096189 wait complete. 2023-10-08T16:16:06.061 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T16:16:06.061 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:16:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.175+0000 7f49563f7700 1 Processor -- start 2023-10-08T16:16:06.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.175+0000 7f49563f7700 1 -- start start 2023-10-08T16:16:06.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f49563f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 0x7f495015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f49563f7700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500b2980 con 0x7f49501597f0 2023-10-08T16:16:06.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f49563f7700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495015dbb0 con 0x7f4950159420 2023-10-08T16:16:06.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f49563f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49500b11d0 con 0x7f495015d230 2023-10-08T16:16:06.178 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4954bf4700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f49501597f0 0x7f49500b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43366/0 (socket says 172.21.15.170:43366) 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4954bf4700 1 -- 172.21.15.170:0/2320849509 learned_addr learned my addr 172.21.15.170:0/2320849509 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4955bf6700 1 --2- 172.21.15.170:0/2320849509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 0x7f495015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134223956 0 0) 0x7f49500b2980 con 0x7f49501597f0 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c003570 con 0x7f49501597f0 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712162425 0 0) 0x7f495015dbb0 con 0x7f4950159420 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49500b2980 con 0x7f4950159420 2023-10-08T16:16:06.179 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4955bf6700 1 -- 172.21.15.170:0/2320849509 >> v1:172.21.15.191:6789/0 conn(0x7f4950159420 legacy=0x7f49500b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4955bf6700 1 -- 172.21.15.170:0/2320849509 >> v1:172.21.15.170:6789/0 conn(0x7f49501597f0 legacy=0x7f49500b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.176+0000 7f4955bf6700 1 -- 172.21.15.170:0/2320849509 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500b14b0 con 0x7f495015d230 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f4955bf6700 1 --2- 172.21.15.170:0/2320849509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 0x7f495015d670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f494c009910 tx=0x7f494c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1d19766474619d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c011630 con 0x7f495015d230 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c011790 con 0x7f495015d230 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f4947fff700 1 -- 172.21.15.170:0/2320849509 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c011b30 con 0x7f495015d230 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 -- 172.21.15.170:0/2320849509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 msgr2=0x7f495015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.180 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 --2- 172.21.15.170:0/2320849509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 0x7f495015d670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f494c009910 tx=0x7f494c0091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 -- 172.21.15.170:0/2320849509 shutdown_connections 2023-10-08T16:16:06.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 --2- 172.21.15.170:0/2320849509 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f495015d230 0x7f495015d670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 -- 172.21.15.170:0/2320849509 >> 172.21.15.170:0/2320849509 conn(0x7f49500d0660 msgr2=0x7f49500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 -- 172.21.15.170:0/2320849509 shutdown_connections 2023-10-08T16:16:06.181 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.177+0000 7f49563f7700 1 -- 172.21.15.170:0/2320849509 wait complete. 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.178+0000 7f49563f7700 1 Processor -- start 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 -- start start 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 0x7f49500c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f495015d230 0x7f49500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.184 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495015baa0 con 0x7f4950159420 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495015bd80 con 0x7f495015d230 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.179+0000 7f49563f7700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495015c060 con 0x7f49501597f0 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34116/0 (socket says 172.21.15.170:34116) 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 -- 172.21.15.170:0/1080010384 learned_addr learned my addr 172.21.15.170:0/1080010384 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f4954bf4700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 0x7f49500c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.185 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f4955bf6700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f495015d230 0x7f49500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 -- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 msgr2=0x7f49500c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 0x7f49500c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 -- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f495015d230 msgr2=0x7f49500b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f495015d230 0x7f49500b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 -- 172.21.15.170:0/1080010384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009420 con 0x7f4950159420 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f4954bf4700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 0x7f49500c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:06.186 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49553f5700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f494800cb40 tx=0x7f4948002810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800ce50 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800a7e0 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800aba0 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495015c340 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.180+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950155110 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.181+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495015c860 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.181+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f4948012420 con 0x7f4950159420 2023-10-08T16:16:06.187 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.182+0000 7f49457fa700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 0x7f49340697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.182+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4948084bf0 con 0x7f4950159420 2023-10-08T16:16:06.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.182+0000 7f4954bf4700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 0x7f49340697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.182+0000 7f4954bf4700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 0x7f49340697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f49500af1f0 tx=0x7f4940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.188 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.186+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f495015c860 con 0x7f4950159420 2023-10-08T16:16:06.364 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.363+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f495015c860 con 0x7f4934067320 2023-10-08T16:16:06.366 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.366+0000 7f49457fa700 1 -- 172.21.15.170:0/1080010384 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50606 (secure 0 0 0) 0x7f495015c860 con 0x7f4934067320 2023-10-08T16:16:06.367 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:06.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 msgr2=0x7f49340697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.368 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 0x7f49340697d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f49500af1f0 tx=0x7f4940009040 comp rx=0 tx=0).stop 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 msgr2=0x7f4950154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f494800cb40 tx=0x7f4948002810 comp rx=0 tx=0).stop 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 shutdown_connections 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f495015d230 0x7f49500b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4934067320 0x7f49340697d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4950159420 0x7f4950154bd0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.369 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 --2- 172.21.15.170:0/1080010384 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f49501597f0 0x7f49500c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 >> 172.21.15.170:0/1080010384 conn(0x7f49500d0660 msgr2=0x7f49500b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 shutdown_connections 2023-10-08T16:16:06.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.368+0000 7f49563f7700 1 -- 172.21.15.170:0/1080010384 wait complete. 2023-10-08T16:16:06.370 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:16:06.380 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":547,"stamp":"2023-10-08T16:16:04.369386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448840,"kb_used_data":21384,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752525880,"statfs":{"total":773094113280,"available":770586501120,"internally_reserved":0,"allocated":21897216,"data_stored":10488438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001228"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":318,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-10-08T16:14:26.545805+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:14:26.545805+0000","last_peered":"2023-10-08T16:14:26.545805+0000","last_clean":"2023-10-08T16:14:26.545805+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:14:26.545805+0000","last_undegraded":"2023-10-08T16:14:26.545805+0000","last_fullsized":"2023-10-08T16:14:26.545805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574990,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1753390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.80200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.32500000000000001,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":8.0939999999999994},"last":0.435}]},{"osd":1,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.29199999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.083},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.38400000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.175},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.69499999999999995},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.745,"5min":0.79100000000000004,"15min":8.4339999999999993},"last":0.55100000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73599999999999999,"5min":1.677,"15min":1.677},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.752,"5min":0.77200000000000002,"15min":8.3409999999999993},"last":0.58099999999999996}]},{"osd":4,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.55800000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.52400000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.497,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51000000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303908,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067932,"statfs":{"total":96636764160,"available":96325562368,"internally_reserved":0,"allocated":2584576,"data_stored":1162480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.90600000000000003},"last":0.495}]},{"osd":1,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.76700000000000002},"last":0.55900000000000005}]},{"osd":2,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.751,"15min":0.751},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.752,"15min":0.96599999999999997},"last":0.59199999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.45600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.94199999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52300000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.80700000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.89200000000000002},"last":0.63600000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.81899999999999995},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.874},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.79600000000000004,"15min":0.93999999999999995},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.85599999999999998},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308008,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063832,"statfs":{"total":96636764160,"available":96321363968,"internally_reserved":0,"allocated":2588672,"data_stored":1162570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.315},"min":{"1min":0.26300000000000001,"5min":0.19,"15min":0.16900000000000001},"max":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":1.2150000000000001},"last":0.312},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.376,"15min":0.34899999999999998},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.77500000000000002},"last":0.28299999999999997}]},{"osd":2,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.35299999999999998},"min":{"1min":0.30399999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":1.2230000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.38900000000000001},"min":{"1min":0.36799999999999999,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.66400000000000003,"15min":0.70399999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.72699999999999998},"last":0.40500000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":1.077},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.90000000000000002,"15min":1.696},"last":0.76200000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.35099999999999998},"max":{"1min":0.78600000000000003,"5min":0.85299999999999998,"15min":1.657},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.51300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88600000000000001,"15min":1.7310000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.875,"15min":1.1839999999999999},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574990,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308584,"kb_used_data":3104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063256,"statfs":{"total":96636764160,"available":96320774144,"internally_reserved":0,"allocated":3178496,"data_stored":1752884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":5.5599999999999996},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.28999999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":3.3220000000000001},"last":0.308}]},{"osd":2,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.45900000000000002},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":5.6769999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.34599999999999997,"5min":0.26300000000000001,"15min":0.219},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.372}]},{"osd":3,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.42299999999999999},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.188},"max":{"1min":0.56999999999999995,"5min":1.0880000000000001,"15min":3.2160000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28199999999999997,"15min":0.26900000000000002},"max":{"1min":0.51500000000000001,"5min":0.67100000000000004,"15min":5.5970000000000004},"last":0.39000000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72099999999999997,"5min":1.1379999999999999,"15min":3.2519999999999998},"last":0.59599999999999997}]},{"osd":5,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.73699999999999999,"5min":1.1140000000000001,"15min":3.2890000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.33400000000000002},"max":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.58299999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":5.6239999999999997},"last":0.75},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.85699999999999998},"last":0.72399999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.60299999999999998,"5min":0.41699999999999998,"15min":0.313},"max":{"1min":0.753,"5min":0.873,"15min":0.88800000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":3.5049999999999999},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308020,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063820,"statfs":{"total":96636764160,"available":96321351680,"internally_reserved":0,"allocated":2600960,"data_stored":1166319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.499,"5min":0.624,"15min":0.63100000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.34699999999999998},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.189},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.64300000000000002},"last":0.25800000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.58499999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36799999999999999},"min":{"1min":0.309,"5min":0.215,"15min":0.20499999999999999},"max":{"1min":0.56399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.33600000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.221},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.61299999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.378}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.313,"15min":0.254},"max":{"1min":0.871,"5min":0.871,"15min":0.89100000000000001},"last":0.624}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.5,"5min":0.379,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.375,"15min":0.30299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.1029999999999998},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63800000000000001,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.52600000000000002,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.72599999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308004,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063836,"statfs":{"total":96636764160,"available":96321368064,"internally_reserved":0,"allocated":2584576,"data_stored":1162536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.238},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.36599999999999999},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.9470000000000001},"last":0.307}]},{"osd":1,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":1.0269999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.31900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.35299999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":1.1399999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.36199999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":0.58899999999999997,"5min":0.68999999999999995,"15min":1.327},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.316,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.9119999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":1.02,"15min":1.099},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.27},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.879},"last":0.435}]},{"osd":6,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.379,"15min":0.372},"max":{"1min":0.82599999999999996,"5min":0.92400000000000004,"15min":1.0620000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":1.03,"15min":1.2969999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0049999999999999,"15min":1.825},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.51500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":1.115},"last":0.49099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574990,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303916,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067924,"statfs":{"total":96636764160,"available":96325554176,"internally_reserved":0,"allocated":2592768,"data_stored":1165743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.438,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.579},"last":0.432},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53700000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.33300000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.48899999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":1.5629999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.628,"15min":0.58699999999999997},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54900000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.67400000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.60699999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68700000000000006}]}]},{"osd":5,"up_from":13,"seq":55834574990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303908,"kb_used_data":2524,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067932,"statfs":{"total":96636764160,"available":96325562368,"internally_reserved":0,"allocated":2584576,"data_stored":1162516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.375},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53800000000000003,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.25,"15min":0.247},"max":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":4.0119999999999996},"last":0.35699999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.33900000000000002,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.84299999999999997,"15min":12.227},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.747,"5min":0.871,"15min":1.252},"last":0.436}]},{"osd":2,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.46600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":3.9510000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":9.3170000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.879,"15min":3.8719999999999999},"last":0.501}]},{"osd":4,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":0.85999999999999999,"15min":0.93799999999999994},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69999999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54600000000000004}]},{"osd":6,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57499999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.442,"5min":0.312,"15min":0.312},"max":{"1min":0.66800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:16:06.382 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:16:06.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.496+0000 7f41d786d700 1 Processor -- start 2023-10-08T16:16:06.497 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.496+0000 7f41d786d700 1 -- start start 2023-10-08T16:16:06.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d786d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f41d0159890 0x7f41d015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d786d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d786d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d015a160 con 0x7f41d015db10 2023-10-08T16:16:06.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d786d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d00d13a0 con 0x7f41d0159890 2023-10-08T16:16:06.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d786d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d015e140 con 0x7f41d0159c60 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d5e0a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f41d015db10 0x7f41d0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43382/0 (socket says 172.21.15.170:43382) 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d5e0a700 1 -- 172.21.15.170:0/3153400256 learned_addr learned my addr 172.21.15.170:0/3153400256 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d4e08700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41d5609700 1 --2- 172.21.15.170:0/3153400256 >> v2:172.21.15.191:3300/0 conn(0x7f41d0159890 0x7f41d015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.497+0000 7f41c7fff700 1 -- 172.21.15.170:0/3153400256 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566359235 0 0) 0x7f41d015a160 con 0x7f41d015db10 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41c7fff700 1 -- 172.21.15.170:0/3153400256 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41bc003570 con 0x7f41d015db10 2023-10-08T16:16:06.500 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d5609700 1 -- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 msgr2=0x7f41d015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d5609700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d5609700 1 -- 172.21.15.170:0/3153400256 >> v1:172.21.15.170:6789/0 conn(0x7f41d015db10 legacy=0x7f41d0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d5609700 1 -- 172.21.15.170:0/3153400256 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d015a3d0 con 0x7f41d0159890 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d4e08700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d5609700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d015abd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098e0 tx=0x7f41cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12c4c61b9e41ab5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41c7fff700 1 -- 172.21.15.170:0/3153400256 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc011630 con 0x7f41d0159890 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41c7fff700 1 -- 172.21.15.170:0/3153400256 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc011790 con 0x7f41d0159890 2023-10-08T16:16:06.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41c7fff700 1 -- 172.21.15.170:0/3153400256 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc011b30 con 0x7f41d0159890 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d786d700 1 -- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 msgr2=0x7f41d015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.498+0000 7f41d786d700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d015abd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0098e0 tx=0x7f41cc0091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 -- 172.21.15.170:0/3153400256 shutdown_connections 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d015abd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 --2- 172.21.15.170:0/3153400256 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 -- 172.21.15.170:0/3153400256 >> 172.21.15.170:0/3153400256 conn(0x7f41d0068e20 msgr2=0x7f41d00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 -- 172.21.15.170:0/3153400256 shutdown_connections 2023-10-08T16:16:06.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.499+0000 7f41d786d700 1 -- 172.21.15.170:0/3153400256 wait complete. 2023-10-08T16:16:06.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.500+0000 7f41d786d700 1 Processor -- start 2023-10-08T16:16:06.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.500+0000 7f41d786d700 1 -- start start 2023-10-08T16:16:06.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d00b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d0169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b4710 con 0x7f41d015db10 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b3a90 con 0x7f41d0159890 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00b3d70 con 0x7f41d0159c60 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34120/0 (socket says 172.21.15.170:34120) 2023-10-08T16:16:06.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 -- 172.21.15.170:0/1390160181 learned_addr learned my addr 172.21.15.170:0/1390160181 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d4e08700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d0169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5609700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d00b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 -- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 msgr2=0x7f41d0169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d0169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 -- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 msgr2=0x7f41d00b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d00b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 -- 172.21.15.170:0/1390160181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c0009420 con 0x7f41d015db10 2023-10-08T16:16:06.507 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d5e0a700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f41c8001460 tx=0x7f41c8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800a870 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c800a9d0 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc009420 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c800ad50 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.501+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d00b4330 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.502+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d00b4850 con 0x7f41d015db10 2023-10-08T16:16:06.508 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.503+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f41c8007300 con 0x7f41d015db10 2023-10-08T16:16:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.503+0000 7f41c57fa700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 0x7f41b40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.503+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f41c8084c80 con 0x7f41d015db10 2023-10-08T16:16:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.503+0000 7f41d5609700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 0x7f41b40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.503+0000 7f41d5609700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 0x7f41b40697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0091e0 tx=0x7f41cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.507+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41c8051cc0 con 0x7f41d015db10 2023-10-08T16:16:06.686 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.686+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41d00b3f30 con 0x7f41b4067320 2023-10-08T16:16:06.688 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.688+0000 7f41c57fa700 1 -- 172.21.15.170:0/1390160181 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50677 (secure 0 0 0) 0x7f41cc011900 con 0x7f41b4067320 2023-10-08T16:16:06.689 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.690+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 msgr2=0x7f41b40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.690+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 0x7f41b40697d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0091e0 tx=0x7f41cc017000 comp rx=0 tx=0).stop 2023-10-08T16:16:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.690+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 msgr2=0x7f41d016a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.691 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.690+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f41c8001460 tx=0x7f41c8001a40 comp rx=0 tx=0).stop 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 shutdown_connections 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f41b4067320 0x7f41b40697d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41d015db10 0x7f41d016a130 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f41d0159890 0x7f41d00b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 --2- 172.21.15.170:0/1390160181 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f41d0159c60 0x7f41d0169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 >> 172.21.15.170:0/1390160181 conn(0x7f41d0068e20 msgr2=0x7f41d0152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.692 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 shutdown_connections 2023-10-08T16:16:06.693 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.691+0000 7f41d786d700 1 -- 172.21.15.170:0/1390160181 wait complete. 2023-10-08T16:16:06.693 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:16:06.703 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2023-10-08T16:16:06.369630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2449032,"kb_used_data":21576,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752525688,"statfs":{"total":773094113280,"available":770586304512,"internally_reserved":0,"allocated":22093824,"data_stored":10589384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001166"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":321,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T16:16:03.453059+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:16:03.453059+0000","last_peered":"2023-10-08T16:16:03.453059+0000","last_clean":"2023-10-08T16:16:03.453059+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:16:03.453059+0000","last_undegraded":"2023-10-08T16:16:03.453059+0000","last_fullsized":"2023-10-08T16:16:03.453059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304512,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067328,"statfs":{"total":96636764160,"available":96324943872,"internally_reserved":0,"allocated":3203072,"data_stored":1765048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.80200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.32500000000000001,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":8.0939999999999994},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.29199999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.083},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.175},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.69499999999999995},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.745,"5min":0.79100000000000004,"15min":8.4339999999999993},"last":0.496}]},{"osd":3,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73599999999999999,"5min":1.677,"15min":1.677},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.752,"5min":0.77200000000000002,"15min":8.3409999999999993},"last":0.60699999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.55800000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.73199999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.497,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59199999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.90600000000000003},"last":0.502}]},{"osd":1,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.76700000000000002},"last":0.53600000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.751,"15min":0.751},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.752,"15min":0.96599999999999997},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.45600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.94199999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.80700000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.89200000000000002},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.81899999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.874},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.79600000000000004,"15min":0.93999999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.85599999999999998},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308028,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063812,"statfs":{"total":96636764160,"available":96321343488,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.315},"min":{"1min":0.26300000000000001,"5min":0.19,"15min":0.16900000000000001},"max":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":1.2150000000000001},"last":0.378},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.376,"15min":0.34899999999999998},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.77500000000000002},"last":0.40300000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.35299999999999998},"min":{"1min":0.30399999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":1.2230000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.38900000000000001},"min":{"1min":0.36799999999999999,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.66400000000000003,"15min":0.70399999999999996},"last":0.438},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.72699999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":1.077},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.90000000000000002,"15min":1.696},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.35099999999999998},"max":{"1min":0.78600000000000003,"5min":0.85299999999999998,"15min":1.657},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.51300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.497},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88600000000000001,"15min":1.7310000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.875,"15min":1.1839999999999999},"last":0.54400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308608,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063232,"statfs":{"total":96636764160,"available":96320749568,"internally_reserved":0,"allocated":3203072,"data_stored":1765048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":5.5599999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.28999999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":3.3220000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.45900000000000002},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":5.6769999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.34599999999999997,"5min":0.26300000000000001,"15min":0.219},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.436}]},{"osd":3,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.42299999999999999},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.188},"max":{"1min":0.56999999999999995,"5min":1.0880000000000001,"15min":3.2160000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28199999999999997,"15min":0.26900000000000002},"max":{"1min":0.51500000000000001,"5min":0.67100000000000004,"15min":5.5970000000000004},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72099999999999997,"5min":1.1379999999999999,"15min":3.2519999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.73699999999999999,"5min":1.1140000000000001,"15min":3.2890000000000001},"last":0.629},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.33400000000000002},"max":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.73699999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":5.6239999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.60299999999999998,"5min":0.41699999999999998,"15min":0.313},"max":{"1min":0.753,"5min":0.873,"15min":0.88800000000000001},"last":0.874},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":3.5049999999999999},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.499,"5min":0.624,"15min":0.63100000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.34699999999999998},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.189},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.64300000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.58499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36799999999999999},"min":{"1min":0.309,"5min":0.215,"15min":0.20499999999999999},"max":{"1min":0.56399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.221},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.61299999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.313,"15min":0.254},"max":{"1min":0.871,"5min":0.871,"15min":0.89100000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.5,"5min":0.379,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.375,"15min":0.30299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.1029999999999998},"last":0.624}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63800000000000001,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.52600000000000002,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308028,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063812,"statfs":{"total":96636764160,"available":96321343488,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.238},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.36599999999999999},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.9470000000000001},"last":0.307}]},{"osd":1,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":1.0269999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.31900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.35299999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":1.1399999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.36199999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":0.58899999999999997,"5min":0.68999999999999995,"15min":1.327},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.316,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.9119999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":1.02,"15min":1.099},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.27},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.879},"last":0.435}]},{"osd":6,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.379,"15min":0.372},"max":{"1min":0.82599999999999996,"5min":0.92400000000000004,"15min":1.0620000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":1.03,"15min":1.2969999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0049999999999999,"15min":1.825},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.51500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":1.115},"last":0.49099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.438,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.579},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.33300000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":1.5629999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.628,"15min":0.58699999999999997},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.66300000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54900000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.441}]},{"osd":6,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53800000000000003,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.25,"15min":0.247},"max":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":4.0119999999999996},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.33900000000000002,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.84299999999999997,"15min":12.227},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.747,"5min":0.871,"15min":1.252},"last":0.746}]},{"osd":2,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.46600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":3.9510000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.44900000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":9.3170000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.879,"15min":3.8719999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":0.85999999999999999,"15min":0.93799999999999994},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69999999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.442,"5min":0.312,"15min":0.312},"max":{"1min":0.66800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:16:06.704 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T16:16:06.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:16:06.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.818+0000 7ff1de832700 1 Processor -- start 2023-10-08T16:16:06.819 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 -- start start 2023-10-08T16:16:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff1d8159890 0x7ff1d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.821 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8069560 0x7ff1d80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d8069ee0 con 0x7ff1d8159c60 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815a160 con 0x7ff1d8159890 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1de832700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80d13a0 con 0x7ff1d8069560 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.819+0000 7ff1d7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff1d8159890 0x7ff1d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d77fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff1d8159c60 0x7ff1d815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43390/0 (socket says 172.21.15.170:43390) 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff1d8159890 0x7ff1d815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34124/0 (socket says 172.21.15.170:34124) 2023-10-08T16:16:06.822 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d77fe700 1 -- 172.21.15.170:0/282407660 learned_addr learned my addr 172.21.15.170:0/282407660 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 -- 172.21.15.170:0/282407660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8069560 msgr2=0x7ff1d80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1dcdcf700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8069560 0x7ff1d80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8069560 0x7ff1d80699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 -- 172.21.15.170:0/282407660 >> v1:172.21.15.170:6789/0 conn(0x7ff1d8159c60 legacy=0x7ff1d815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d6ffd700 1 -- 172.21.15.170:0/282407660 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839436061 0 0) 0x7ff1d8069ee0 con 0x7ff1d8159c60 2023-10-08T16:16:06.823 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 -- 172.21.15.170:0/282407660 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d815e140 con 0x7ff1d8159890 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d7fff700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159890 0x7ff1d815abd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80098e0 tx=0x7ff1c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4c1d42fefc2d4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d6ffd700 1 -- 172.21.15.170:0/282407660 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8009e80 con 0x7ff1d8159890 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d6ffd700 1 -- 172.21.15.170:0/282407660 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800b860 con 0x7ff1d8159890 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.820+0000 7ff1d6ffd700 1 -- 172.21.15.170:0/282407660 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800bbe0 con 0x7ff1d8159890 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 -- 172.21.15.170:0/282407660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159890 msgr2=0x7ff1d815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159890 0x7ff1d815abd0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80098e0 tx=0x7ff1c80091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.824 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 -- 172.21.15.170:0/282407660 shutdown_connections 2023-10-08T16:16:06.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8069560 0x7ff1d80699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 --2- 172.21.15.170:0/282407660 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159890 0x7ff1d815abd0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 -- 172.21.15.170:0/282407660 >> 172.21.15.170:0/282407660 conn(0x7ff1d8068e20 msgr2=0x7ff1d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:06.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 -- 172.21.15.170:0/282407660 shutdown_connections 2023-10-08T16:16:06.825 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.821+0000 7ff1de832700 1 -- 172.21.15.170:0/282407660 wait complete. 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.822+0000 7ff1de832700 1 Processor -- start 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.822+0000 7ff1de832700 1 -- start start 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1d8069560 0x7ff1d80cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8159890 0x7ff1d80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80ce470 con 0x7ff1d8159c60 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b63e0 con 0x7ff1d8069560 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80ce010 con 0x7ff1d8159890 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34136/0 (socket says 172.21.15.170:34136) 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 -- 172.21.15.170:0/2987178669 learned_addr learned my addr 172.21.15.170:0/2987178669 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:06.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1d77fe700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8159890 0x7ff1d80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1d7fff700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1d8069560 0x7ff1d80cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 -- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8159890 msgr2=0x7ff1d80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8159890 0x7ff1d80b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 -- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1d8069560 msgr2=0x7ff1d80cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1d8069560 0x7ff1d80cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 -- 172.21.15.170:0/2987178669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c8009420 con 0x7ff1d8159c60 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1dcdcf700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000cdb0 tx=0x7ff1d00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000a9d0 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000ab30 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d8156f30 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017430 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.823+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d81574f0 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.824+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d804c3c0 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.825+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7ff1d0003020 con 0x7ff1d8159c60 2023-10-08T16:16:06.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.825+0000 7ff1bffff700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 0x7ff1c40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:06.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.825+0000 7ff1d7fff700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 0x7ff1c40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:06.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.825+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff1d0013070 con 0x7ff1d8159c60 2023-10-08T16:16:06.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.826+0000 7ff1d7fff700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 0x7ff1c40697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80091e0 tx=0x7ff1c8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:06.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:06.829+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1d0051d10 con 0x7ff1d8159c60 2023-10-08T16:16:07.012 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.012+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1d806a5b0 con 0x7ff1c4067320 2023-10-08T16:16:07.014 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.014+0000 7ff1bffff700 1 -- 172.21.15.170:0/2987178669 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50677 (secure 0 0 0) 0x7ff1d806a5b0 con 0x7ff1c4067320 2023-10-08T16:16:07.015 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:07.016 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 msgr2=0x7ff1c40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 0x7ff1c40697d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80091e0 tx=0x7ff1c8017000 comp rx=0 tx=0).stop 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 msgr2=0x7ff1d8147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000cdb0 tx=0x7ff1d00077d0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 shutdown_connections 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7ff1c4067320 0x7ff1c40697d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.017 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1d8159c60 0x7ff1d8147da0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7ff1d8069560 0x7ff1d80cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 --2- 172.21.15.170:0/2987178669 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7ff1d8159890 0x7ff1d80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 >> 172.21.15.170:0/2987178669 conn(0x7ff1d8068e20 msgr2=0x7ff1d80b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:07.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 shutdown_connections 2023-10-08T16:16:07.018 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.016+0000 7ff1de832700 1 -- 172.21.15.170:0/2987178669 wait complete. 2023-10-08T16:16:07.019 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:16:07.028 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":548,"stamp":"2023-10-08T16:16:06.369630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2449032,"kb_used_data":21576,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752525688,"statfs":{"total":773094113280,"available":770586304512,"internally_reserved":0,"allocated":22093824,"data_stored":10589384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4067,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-49,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001166"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":321,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T16:16:03.453059+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:16:03.453059+0000","last_peered":"2023-10-08T16:16:03.453059+0000","last_clean":"2023-10-08T16:16:03.453059+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:16:03.453059+0000","last_undegraded":"2023-10-08T16:16:03.453059+0000","last_fullsized":"2023-10-08T16:16:03.453059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574991,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304512,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067328,"statfs":{"total":96636764160,"available":96324943872,"internally_reserved":0,"allocated":3203072,"data_stored":1765048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.80200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.32500000000000001,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":8.0939999999999994},"last":0.46600000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.29199999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.083},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.175},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.69499999999999995},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.745,"5min":0.79100000000000004,"15min":8.4339999999999993},"last":0.496}]},{"osd":3,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73599999999999999,"5min":1.677,"15min":1.677},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.752,"5min":0.77200000000000002,"15min":8.3409999999999993},"last":0.60699999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.55800000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.73199999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.497,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.44},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.59199999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.90600000000000003},"last":0.502}]},{"osd":1,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.76700000000000002},"last":0.53600000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.751,"15min":0.751},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.752,"15min":0.96599999999999997},"last":0.65500000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.45600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.94199999999999995},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45600000000000002}]},{"osd":4,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.80700000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.89200000000000002},"last":0.56799999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.81899999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.874},"last":0.68100000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.79600000000000004,"15min":0.93999999999999995},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.85599999999999998},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308028,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063812,"statfs":{"total":96636764160,"available":96321343488,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.315},"min":{"1min":0.26300000000000001,"5min":0.19,"15min":0.16900000000000001},"max":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":1.2150000000000001},"last":0.378},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.376,"15min":0.34899999999999998},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.77500000000000002},"last":0.40300000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.35299999999999998},"min":{"1min":0.30399999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":1.2230000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.38900000000000001},"min":{"1min":0.36799999999999999,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.66400000000000003,"15min":0.70399999999999996},"last":0.438},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.72699999999999998},"last":0.42499999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":1.077},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.90000000000000002,"15min":1.696},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.35099999999999998},"max":{"1min":0.78600000000000003,"5min":0.85299999999999998,"15min":1.657},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"last":0.57599999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.51300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.497},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88600000000000001,"15min":1.7310000000000001},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.875,"15min":1.1839999999999999},"last":0.54400000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574991,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308608,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063232,"statfs":{"total":96636764160,"available":96320749568,"internally_reserved":0,"allocated":3203072,"data_stored":1765048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":5.5599999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.28999999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":3.3220000000000001},"last":0.376}]},{"osd":2,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.45900000000000002},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":5.6769999999999996},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.34599999999999997,"5min":0.26300000000000001,"15min":0.219},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.436}]},{"osd":3,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.42299999999999999},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.188},"max":{"1min":0.56999999999999995,"5min":1.0880000000000001,"15min":3.2160000000000002},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28199999999999997,"15min":0.26900000000000002},"max":{"1min":0.51500000000000001,"5min":0.67100000000000004,"15min":5.5970000000000004},"last":0.44700000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72099999999999997,"5min":1.1379999999999999,"15min":3.2519999999999998},"last":0.66300000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.73699999999999999,"5min":1.1140000000000001,"15min":3.2890000000000001},"last":0.629},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.33400000000000002},"max":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.73699999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":5.6239999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.85699999999999998},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.60299999999999998,"5min":0.41699999999999998,"15min":0.313},"max":{"1min":0.753,"5min":0.873,"15min":0.88800000000000001},"last":0.874},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":3.5049999999999999},"last":0.71899999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.499,"5min":0.624,"15min":0.63100000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.34699999999999998},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.189},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.64300000000000002},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.58499999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36799999999999999},"min":{"1min":0.309,"5min":0.215,"15min":0.20499999999999999},"max":{"1min":0.56399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.35099999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.221},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.61299999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48299999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.313,"15min":0.254},"max":{"1min":0.871,"5min":0.871,"15min":0.89100000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.5,"5min":0.379,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.375,"15min":0.30299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.1029999999999998},"last":0.624}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63800000000000001,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.495}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.52600000000000002,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46999999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574990,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308028,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063812,"statfs":{"total":96636764160,"available":96321343488,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.238},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.36599999999999999},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.9470000000000001},"last":0.307}]},{"osd":1,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":1.0269999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.31900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.35299999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":1.1399999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.36199999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":0.58899999999999997,"5min":0.68999999999999995,"15min":1.327},"last":0.41299999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.316,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.9119999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":1.02,"15min":1.099},"last":0.45000000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.27},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.879},"last":0.435}]},{"osd":6,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.379,"15min":0.372},"max":{"1min":0.82599999999999996,"5min":0.92400000000000004,"15min":1.0620000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":1.03,"15min":1.2969999999999999},"last":0.54500000000000004}]},{"osd":7,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0049999999999999,"15min":1.825},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.51500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":1.115},"last":0.49099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.438,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.579},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.33300000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":1.5629999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.628,"15min":0.58699999999999997},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.66300000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54900000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.441}]},{"osd":6,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303932,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067908,"statfs":{"total":96636764160,"available":96325537792,"internally_reserved":0,"allocated":2609152,"data_stored":1174680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53800000000000003,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.25,"15min":0.247},"max":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":4.0119999999999996},"last":0.47799999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.33900000000000002,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.84299999999999997,"15min":12.227},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.747,"5min":0.871,"15min":1.252},"last":0.746}]},{"osd":2,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.46600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":3.9510000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.44900000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":9.3170000000000002},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.879,"15min":3.8719999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":0.85999999999999999,"15min":0.93799999999999994},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69999999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69499999999999995}]},{"osd":6,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.63900000000000001}]},{"osd":7,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.442,"5min":0.312,"15min":0.312},"max":{"1min":0.66800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:16:07.030 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T16:16:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 Processor -- start 2023-10-08T16:16:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 -- start start 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 0x7fde4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4415a160 con 0x7fde4415db10 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde440d13a0 con 0x7fde44159890 2023-10-08T16:16:07.147 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.144+0000 7fde4aa73700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415e140 con 0x7fde44159c60 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34144/0 (socket says 172.21.15.170:34144) 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 -- 172.21.15.170:0/3795965285 learned_addr learned my addr 172.21.15.170:0/3795965285 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde43fff700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde4880f700 1 --2- 172.21.15.170:0/3795965285 >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 0x7fde4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 -- 172.21.15.170:0/3795965285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 msgr2=0x7fde4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 -- 172.21.15.170:0/3795965285 >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 msgr2=0x7fde4415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 --2- 172.21.15.170:0/3795965285 >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 0x7fde4415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 -- 172.21.15.170:0/3795965285 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4415a3d0 con 0x7fde4415db10 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde4880f700 1 --2- 172.21.15.170:0/3795965285 >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 0x7fde4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde49010700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde44069560 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fde440d1090 tx=0x7fde3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcc77bb2a6300a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde437fe700 1 -- 172.21.15.170:0/3795965285 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00b860 con 0x7fde4415db10 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde437fe700 1 -- 172.21.15.170:0/3795965285 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00b9c0 con 0x7fde4415db10 2023-10-08T16:16:07.149 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.145+0000 7fde437fe700 1 -- 172.21.15.170:0/3795965285 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00bd60 con 0x7fde4415db10 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 -- 172.21.15.170:0/3795965285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 msgr2=0x7fde44069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde44069560 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fde440d1090 tx=0x7fde3c009250 comp rx=0 tx=0).stop 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 -- 172.21.15.170:0/3795965285 shutdown_connections 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde44069560 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 --2- 172.21.15.170:0/3795965285 >> v2:172.21.15.191:3300/0 conn(0x7fde44159890 0x7fde4415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 --2- 172.21.15.170:0/3795965285 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde4415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 -- 172.21.15.170:0/3795965285 >> 172.21.15.170:0/3795965285 conn(0x7fde44068e20 msgr2=0x7fde440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:07.150 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 -- 172.21.15.170:0/3795965285 shutdown_connections 2023-10-08T16:16:07.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.146+0000 7fde4aa73700 1 -- 172.21.15.170:0/3795965285 wait complete. 2023-10-08T16:16:07.151 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.147+0000 7fde4aa73700 1 Processor -- start 2023-10-08T16:16:07.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.147+0000 7fde4aa73700 1 -- start start 2023-10-08T16:16:07.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde44159890 0x7fde4414f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde440b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.153 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4414fd10 con 0x7fde4415db10 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b0760 con 0x7fde44159890 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde4aa73700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde440b2540 con 0x7fde44159c60 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34152/0 (socket says 172.21.15.170:34152) 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 -- 172.21.15.170:0/2461708329 learned_addr learned my addr 172.21.15.170:0/2461708329 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 -- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 msgr2=0x7fde440b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.154 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde440b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 -- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde44159890 msgr2=0x7fde4414f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde44159890 0x7fde4414f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 -- 172.21.15.170:0/2461708329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0094f0 con 0x7fde4415db10 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde49010700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0090b0 tx=0x7fde3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.148+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c01a660 con 0x7fde4415db10 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.149+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c01a7c0 con 0x7fde4415db10 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.149+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde440b0a00 con 0x7fde4415db10 2023-10-08T16:16:07.155 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.149+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c01ab80 con 0x7fde4415db10 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.149+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440b2230 con 0x7fde4415db10 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.149+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde440b0f20 con 0x7fde4415db10 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.150+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fde3c01b430 con 0x7fde4415db10 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.150+0000 7fde40ff9700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 0x7fde300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.150+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fde3c0861d0 con 0x7fde4415db10 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.150+0000 7fde4880f700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 0x7fde300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:07.156 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.151+0000 7fde4880f700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 0x7fde300697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fde340094d0 tx=0x7fde340096b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:07.157 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.154+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde440b0f20 con 0x7fde4415db10 2023-10-08T16:16:07.337 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.337+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde440b0f20 con 0x7fde4415db10 2023-10-08T16:16:07.338 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.338+0000 7fde40ff9700 1 -- 172.21.15.170:0/2461708329 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+12323 (secure 0 0 0) 0x7fde440b0f20 con 0x7fde4415db10 2023-10-08T16:16:07.339 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:07.339 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":206,"fsid":"a2f97f1a-22f1-43be-8b23-d4161b44866d","created":"2023-10-08T16:04:18.664273+0000","modified":"2023-10-08T16:16:03.882728+0000","last_up_change":"2023-10-08T16:04:28.513686+0000","last_in_change":"2023-10-08T16:04:21.062592+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":34,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T16:04:30.263818+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"3463d550-5fc6-4910-8450-96e64c710dcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":90868},{"type":"v1","addr":"172.21.15.170:6809","nonce":90868}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":90868},{"type":"v1","addr":"172.21.15.170:6811","nonce":90868}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":90868},{"type":"v1","addr":"172.21.15.170:6815","nonce":90868}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":90868},{"type":"v1","addr":"172.21.15.170:6813","nonce":90868}]},"public_addr":"172.21.15.170:6809/90868","cluster_addr":"172.21.15.170:6811/90868","heartbeat_back_addr":"172.21.15.170:6815/90868","heartbeat_front_addr":"172.21.15.170:6813/90868","state":["exists","up"]},{"osd":1,"uuid":"dbc0cad8-7324-4b72-8d94-7e4937678d1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":90867},{"type":"v1","addr":"172.21.15.170:6801","nonce":90867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":90867},{"type":"v1","addr":"172.21.15.170:6803","nonce":90867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":90867},{"type":"v1","addr":"172.21.15.170:6807","nonce":90867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":90867},{"type":"v1","addr":"172.21.15.170:6805","nonce":90867}]},"public_addr":"172.21.15.170:6801/90867","cluster_addr":"172.21.15.170:6803/90867","heartbeat_back_addr":"172.21.15.170:6807/90867","heartbeat_front_addr":"172.21.15.170:6805/90867","state":["exists","up"]},{"osd":2,"uuid":"5d8aa2f9-d4d9-4962-90ee-4e79432b1e7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":90869},{"type":"v1","addr":"172.21.15.170:6817","nonce":90869}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":90869},{"type":"v1","addr":"172.21.15.170:6819","nonce":90869}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":90869},{"type":"v1","addr":"172.21.15.170:6823","nonce":90869}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":90869},{"type":"v1","addr":"172.21.15.170:6821","nonce":90869}]},"public_addr":"172.21.15.170:6817/90869","cluster_addr":"172.21.15.170:6819/90869","heartbeat_back_addr":"172.21.15.170:6823/90869","heartbeat_front_addr":"172.21.15.170:6821/90869","state":["exists","up"]},{"osd":3,"uuid":"c1d70f9b-1a2f-4956-b336-b50fecb8d223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":90884},{"type":"v1","addr":"172.21.15.170:6825","nonce":90884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":90884},{"type":"v1","addr":"172.21.15.170:6827","nonce":90884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":90884},{"type":"v1","addr":"172.21.15.170:6831","nonce":90884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":90884},{"type":"v1","addr":"172.21.15.170:6829","nonce":90884}]},"public_addr":"172.21.15.170:6825/90884","cluster_addr":"172.21.15.170:6827/90884","heartbeat_back_addr":"172.21.15.170:6831/90884","heartbeat_front_addr":"172.21.15.170:6829/90884","state":["exists","up"]},{"osd":4,"uuid":"541091da-0d2a-4788-a70a-4943fc488905","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":202,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":91066},{"type":"v1","addr":"172.21.15.191:6801","nonce":91066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":91066},{"type":"v1","addr":"172.21.15.191:6803","nonce":91066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":91066},{"type":"v1","addr":"172.21.15.191:6807","nonce":91066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":91066},{"type":"v1","addr":"172.21.15.191:6805","nonce":91066}]},"public_addr":"172.21.15.191:6801/91066","cluster_addr":"172.21.15.191:6803/91066","heartbeat_back_addr":"172.21.15.191:6807/91066","heartbeat_front_addr":"172.21.15.191:6805/91066","state":["exists","up"]},{"osd":5,"uuid":"dc305d76-f069-4520-ac6c-1d7ca891b94e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6824","nonce":91096},{"type":"v1","addr":"172.21.15.191:6825","nonce":91096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6826","nonce":91096},{"type":"v1","addr":"172.21.15.191:6827","nonce":91096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6830","nonce":91096},{"type":"v1","addr":"172.21.15.191:6831","nonce":91096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6828","nonce":91096},{"type":"v1","addr":"172.21.15.191:6829","nonce":91096}]},"public_addr":"172.21.15.191:6825/91096","cluster_addr":"172.21.15.191:6827/91096","heartbeat_back_addr":"172.21.15.191:6831/91096","heartbeat_front_addr":"172.21.15.191:6829/91096","state":["exists","up"]},{"osd":6,"uuid":"fae402bd-abf3-4d4b-9c09-5104a8806aa2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6808","nonce":91067},{"type":"v1","addr":"172.21.15.191:6809","nonce":91067}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6810","nonce":91067},{"type":"v1","addr":"172.21.15.191:6811","nonce":91067}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6814","nonce":91067},{"type":"v1","addr":"172.21.15.191:6815","nonce":91067}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6812","nonce":91067},{"type":"v1","addr":"172.21.15.191:6813","nonce":91067}]},"public_addr":"172.21.15.191:6809/91067","cluster_addr":"172.21.15.191:6811/91067","heartbeat_back_addr":"172.21.15.191:6815/91067","heartbeat_front_addr":"172.21.15.191:6813/91067","state":["exists","up"]},{"osd":7,"uuid":"2221559f-a1b9-470e-9926-439928ae1bce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":199,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6816","nonce":91097},{"type":"v1","addr":"172.21.15.191:6817","nonce":91097}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6818","nonce":91097},{"type":"v1","addr":"172.21.15.191:6819","nonce":91097}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6822","nonce":91097},{"type":"v1","addr":"172.21.15.191:6823","nonce":91097}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6820","nonce":91097},{"type":"v1","addr":"172.21.15.191:6821","nonce":91097}]},"public_addr":"172.21.15.191:6817/91097","cluster_addr":"172.21.15.191:6819/91097","heartbeat_back_addr":"172.21.15.191:6823/91097","heartbeat_front_addr":"172.21.15.191:6821/91097","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.250024+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.258311+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.255890+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.217640+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.021539+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.229244+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.183496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T16:04:26.194939+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:6837/1430346956":"2023-10-09T16:16:00.871385+0000","172.21.15.170:6835/1430346956":"2023-10-09T16:16:00.871385+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T16:16:07.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 msgr2=0x7fde300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 0x7fde300697d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fde340094d0 tx=0x7fde340096b0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.341 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 msgr2=0x7fde440b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fde3c0090b0 tx=0x7fde3c017000 comp rx=0 tx=0).stop 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 shutdown_connections 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde30067320 0x7fde300697d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde4415db10 0x7fde440b1200 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde44159890 0x7fde4414f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.342 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.340+0000 7fde4aa73700 1 --2- 172.21.15.170:0/2461708329 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde44159c60 0x7fde440b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:07.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.341+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 >> 172.21.15.170:0/2461708329 conn(0x7fde44068e20 msgr2=0x7fde44156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:07.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.341+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 shutdown_connections 2023-10-08T16:16:07.343 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:07.341+0000 7fde4aa73700 1 -- 172.21.15.170:0/2461708329 wait complete. 2023-10-08T16:16:08.357 INFO:tasks.ceph:Scrubbing osd.0 2023-10-08T16:16:08.357 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:08.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.480+0000 7fe53e4c9700 1 Processor -- start 2023-10-08T16:16:08.481 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.480+0000 7fe53e4c9700 1 -- start start 2023-10-08T16:16:08.483 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 0x7fe5380b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 0x7fe5380cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe5380cce40 0x7fe5380cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380b5b00 con 0x7fe5380cce40 2023-10-08T16:16:08.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380b5850 con 0x7fe538159ec0 2023-10-08T16:16:08.484 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53e4c9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b5da0 con 0x7fe5380b53d0 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53ca66700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe5380cce40 0x7fe5380cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53ca66700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe5380cce40 0x7fe5380cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34156/0 (socket says 172.21.15.170:34156) 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe53ca66700 1 -- 172.21.15.170:0/2129307467 learned_addr learned my addr 172.21.15.170:0/2129307467 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe5377fe700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 0x7fe5380cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.481+0000 7fe537fff700 1 --2- 172.21.15.170:0/2129307467 >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 0x7fe5380b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.485 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 -- 172.21.15.170:0/2129307467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 msgr2=0x7fe5380cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 0x7fe5380cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 -- 172.21.15.170:0/2129307467 >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 msgr2=0x7fe5380b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 --2- 172.21.15.170:0/2129307467 >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 0x7fe5380b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 -- 172.21.15.170:0/2129307467 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380b6080 con 0x7fe5380cce40 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe5377fe700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 0x7fe5380cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe537fff700 1 --2- 172.21.15.170:0/2129307467 >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 0x7fe5380b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53ca66700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5380cd2a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe538089120 tx=0x7fe530009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2231a96d4c46d19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.486 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe536ffd700 1 -- 172.21.15.170:0/2129307467 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000e070 con 0x7fe5380cce40 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe536ffd700 1 -- 172.21.15.170:0/2129307467 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53000ba50 con 0x7fe5380cce40 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe536ffd700 1 -- 172.21.15.170:0/2129307467 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300092e0 con 0x7fe5380cce40 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 -- 172.21.15.170:0/2129307467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 msgr2=0x7fe5380cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5380cd2a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe538089120 tx=0x7fe530009840 comp rx=0 tx=0).stop 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 -- 172.21.15.170:0/2129307467 shutdown_connections 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5380cd2a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/2129307467 >> v2:172.21.15.191:3300/0 conn(0x7fe538159ec0 0x7fe5380b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.487 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/2129307467 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe5380b53d0 0x7fe5380cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 -- 172.21.15.170:0/2129307467 >> 172.21.15.170:0/2129307467 conn(0x7fe5380999f0 msgr2=0x7fe5380b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 -- 172.21.15.170:0/2129307467 shutdown_connections 2023-10-08T16:16:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.482+0000 7fe53e4c9700 1 -- 172.21.15.170:0/2129307467 wait complete. 2023-10-08T16:16:08.488 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.483+0000 7fe53e4c9700 1 Processor -- start 2023-10-08T16:16:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 -- start start 2023-10-08T16:16:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5380b53d0 0x7fe538155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.491 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5381474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe538159ec0 0x7fe53806ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381575e0 con 0x7fe5380cce40 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe538157150 con 0x7fe5380b53d0 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.484+0000 7fe53e4c9700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe538156540 con 0x7fe538159ec0 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5381474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5381474b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34168/0 (socket says 172.21.15.170:34168) 2023-10-08T16:16:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 -- 172.21.15.170:0/833243237 learned_addr learned my addr 172.21.15.170:0/833243237 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe537fff700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5380b53d0 0x7fe538155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe53ca66700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe538159ec0 0x7fe53806ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 -- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe538159ec0 msgr2=0x7fe53806ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.493 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fe538159ec0 0x7fe53806ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 -- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5380b53d0 msgr2=0x7fe538155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fe5380b53d0 0x7fe538155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530009ae0 con 0x7fe5380cce40 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe5377fe700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5381474b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c880 tx=0x7fe52c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011600 con 0x7fe5380cce40 2023-10-08T16:16:08.494 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c011760 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe538156820 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c011b20 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.485+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe538156de0 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.486+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe538063c90 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.486+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fe52c0193b0 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe51ffff700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe524067320 0x7fe5240697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fe52c0849e0 con 0x7fe5380cce40 2023-10-08T16:16:08.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe537fff700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe524067320 0x7fe5240697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe51ffff700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7fe52406d170 0x7fe52406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe52406fc50 con 0x7fe52406d170 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe52c089220 con 0x7fe5380cce40 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe53ca66700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7fe52406d170 0x7fe52406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe537fff700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe524067320 0x7fe5240697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe52800ab80 tx=0x7fe528008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.487+0000 7fe53ca66700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7fe52406d170 0x7fe52406f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.488+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== osd.0 v2:172.21.15.170:6808/90868 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe52406fc50 con 0x7fe52406d170 2023-10-08T16:16:08.501 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.501+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 --> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe5380718b0 con 0x7fe52406d170 2023-10-08T16:16:08.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe51ffff700 1 -- 172.21.15.170:0/833243237 <== osd.0 v2:172.21.15.170:6808/90868 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe5380718b0 con 0x7fe52406d170 2023-10-08T16:16:08.502 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:08.503 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:08.503 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:08.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7fe52406d170 msgr2=0x7fe52406f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6808/90868,v1:172.21.15.170:6809/90868] conn(0x7fe52406d170 0x7fe52406f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe524067320 msgr2=0x7fe5240697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fe524067320 0x7fe5240697d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe52800ab80 tx=0x7fe528008040 comp rx=0 tx=0).stop 2023-10-08T16:16:08.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 msgr2=0x7fe5381474b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 --2- 172.21.15.170:0/833243237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5380cce40 0x7fe5381474b0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00c880 tx=0x7fe52c008040 comp rx=0 tx=0).stop 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53ca66700 1 -- 172.21.15.170:0/833243237 reap_dead start 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 shutdown_connections 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 >> 172.21.15.170:0/833243237 conn(0x7fe5380999f0 msgr2=0x7fe538155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 shutdown_connections 2023-10-08T16:16:08.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.502+0000 7fe53e4c9700 1 -- 172.21.15.170:0/833243237 wait complete. 2023-10-08T16:16:08.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-08T16:16:08.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.629+0000 7f09c9338700 1 Processor -- start 2023-10-08T16:16:08.630 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.629+0000 7f09c9338700 1 -- start start 2023-10-08T16:16:08.632 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c9338700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c9338700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c9338700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40cfb30 con 0x7f09c40cba70 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c9338700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c40cbf10 con 0x7f09c4069bb0 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c9338700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40cf4e0 con 0x7f09c4159840 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.630+0000 7f09c37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c37fe700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:34400/0 (socket says 172.21.15.170:34400) 2023-10-08T16:16:08.633 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:34660/0 (socket says 172.21.15.170:34660) 2023-10-08T16:16:08.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c37fe700 1 -- 172.21.15.170:0/2284930753 learned_addr learned my addr 172.21.15.170:0/2284930753 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c27fc700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7f09c40cba70 0x7f09c40694a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54408/0 (socket says 172.21.15.170:54408) 2023-10-08T16:16:08.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2284930753 >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 msgr2=0x7f09c40b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:08.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 --2- 172.21.15.170:0/2284930753 >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.634 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2284930753 >> v1:172.21.15.191:6789/0 conn(0x7f09c40cba70 legacy=0x7f09c40694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2284930753 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40cf7c0 con 0x7f09c4159840 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c1ffb700 1 -- 172.21.15.170:0/2284930753 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165634900 0 0) 0x7f09c40cfb30 con 0x7f09c40cba70 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c37fe700 1 --2- 172.21.15.170:0/2284930753 >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c2ffd700 1 --2- 172.21.15.170:0/2284930753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009e60 tx=0x7f09b4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b262e39c2d603a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c1ffb700 1 -- 172.21.15.170:0/2284930753 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b400e070 con 0x7f09c4159840 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c1ffb700 1 -- 172.21.15.170:0/2284930753 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b400bb60 con 0x7f09c4159840 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.631+0000 7f09c1ffb700 1 -- 172.21.15.170:0/2284930753 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4012430 con 0x7f09c4159840 2023-10-08T16:16:08.635 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 -- 172.21.15.170:0/2284930753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 msgr2=0x7f09c40cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 --2- 172.21.15.170:0/2284930753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009e60 tx=0x7f09b4009780 comp rx=0 tx=0).stop 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 -- 172.21.15.170:0/2284930753 shutdown_connections 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 --2- 172.21.15.170:0/2284930753 >> v2:172.21.15.191:3300/0 conn(0x7f09c4069bb0 0x7f09c40b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 --2- 172.21.15.170:0/2284930753 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c4159840 0x7f09c40cb530 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 -- 172.21.15.170:0/2284930753 >> 172.21.15.170:0/2284930753 conn(0x7f09c4068e10 msgr2=0x7f09c40d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 -- 172.21.15.170:0/2284930753 shutdown_connections 2023-10-08T16:16:08.636 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.632+0000 7f09c9338700 1 -- 172.21.15.170:0/2284930753 wait complete. 2023-10-08T16:16:08.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.633+0000 7f09c9338700 1 Processor -- start 2023-10-08T16:16:08.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.633+0000 7f09c9338700 1 -- start start 2023-10-08T16:16:08.639 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c40cba70 0x7f09c40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09c4159840 0x7f09c415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40b5690 con 0x7f09c4069bb0 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c4155750 con 0x7f09c4159840 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c9338700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40b5880 con 0x7f09c40cba70 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.640 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34182/0 (socket says 172.21.15.170:34182) 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2653066533 learned_addr learned my addr 172.21.15.170:0/2653066533 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c27fc700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c40cba70 0x7f09c40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c37fe700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09c4159840 0x7f09c415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c40cba70 msgr2=0x7f09c40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c40cba70 0x7f09c40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09c4159840 msgr2=0x7f09c415a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09c4159840 0x7f09c415a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.641 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 -- 172.21.15.170:0/2653066533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09bc0094f0 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09c2ffd700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b6c0 tx=0x7f09b4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b40070c0 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4012a30 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.634+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4012df0 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.635+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b4009a20 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.635+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c40b5e40 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.635+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c41585c0 con 0x7f09c4069bb0 2023-10-08T16:16:08.642 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.636+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f09b4026600 con 0x7f09c4069bb0 2023-10-08T16:16:08.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.636+0000 7f09baffd700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.636+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f09b4086740 con 0x7f09c4069bb0 2023-10-08T16:16:08.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.636+0000 7f09c27fc700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.637+0000 7f09c27fc700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 0x7f09a40697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f09c40b0150 tx=0x7f09ac007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.640+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09b4053640 con 0x7f09c4069bb0 2023-10-08T16:16:08.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.826+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f09c4155910 con 0x7f09a4067320 2023-10-08T16:16:08.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.827+0000 7f09baffd700 1 -- 172.21.15.170:0/2653066533 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f09c4155910 con 0x7f09a4067320 2023-10-08T16:16:08.829 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 msgr2=0x7f09a40697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 0x7f09a40697d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f09c40b0150 tx=0x7f09ac007d50 comp rx=0 tx=0).stop 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 msgr2=0x7f09c415c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f09b400b6c0 tx=0x7f09b4019000 comp rx=0 tx=0).stop 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 shutdown_connections 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f09c4159840 0x7f09c415a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f09a4067320 0x7f09a40697d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09c4069bb0 0x7f09c415c640 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 --2- 172.21.15.170:0/2653066533 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f09c40cba70 0x7f09c40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 >> 172.21.15.170:0/2653066533 conn(0x7f09c4068e10 msgr2=0x7f09c4154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 shutdown_connections 2023-10-08T16:16:08.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.829+0000 7f09c9338700 1 -- 172.21.15.170:0/2653066533 wait complete. 2023-10-08T16:16:08.831 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 0 to deep-scrub 2023-10-08T16:16:08.841 INFO:tasks.ceph:Scrubbing osd.1 2023-10-08T16:16:08.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:08.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.960+0000 7faad7192700 1 Processor -- start 2023-10-08T16:16:08.961 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.960+0000 7faad7192700 1 -- start start 2023-10-08T16:16:08.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad7192700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad006ab40 0x7faad00699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad7192700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faad00af8c0 0x7faad0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad7192700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad0068900 con 0x7faad0068390 2023-10-08T16:16:08.963 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad7192700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad0068be0 con 0x7faad00af8c0 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad7192700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad0068ec0 con 0x7faad006ab40 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad572f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faad00af8c0 0x7faad0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad572f700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7faad00af8c0 0x7faad0159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34194/0 (socket says 172.21.15.170:34194) 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.961+0000 7faad572f700 1 -- 172.21.15.170:0/3845881696 learned_addr learned my addr 172.21.15.170:0/3845881696 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad4f2e700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad006ab40 0x7faad00699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad572f700 1 -- 172.21.15.170:0/3845881696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad006ab40 msgr2=0x7faad00699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:08.964 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad572f700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad006ab40 0x7faad00699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad572f700 1 -- 172.21.15.170:0/3845881696 >> v1:172.21.15.191:6789/0 conn(0x7faad0068390 legacy=0x7faad00c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad572f700 1 -- 172.21.15.170:0/3845881696 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00691a0 con 0x7faad00af8c0 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad572f700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad00af8c0 0x7faad0159620 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faac80099b0 tx=0x7faac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903bdeb7d448e8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faac77fe700 1 -- 172.21.15.170:0/3845881696 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800b820 con 0x7faad00af8c0 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faac77fe700 1 -- 172.21.15.170:0/3845881696 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac800b980 con 0x7faad00af8c0 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faac77fe700 1 -- 172.21.15.170:0/3845881696 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800bd20 con 0x7faad00af8c0 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad7192700 1 -- 172.21.15.170:0/3845881696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad00af8c0 msgr2=0x7faad0159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.965 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.962+0000 7faad7192700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad00af8c0 0x7faad0159620 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faac80099b0 tx=0x7faac8009250 comp rx=0 tx=0).stop 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 -- 172.21.15.170:0/3845881696 shutdown_connections 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad00af8c0 0x7faad0159620 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 --2- 172.21.15.170:0/3845881696 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad006ab40 0x7faad00699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 -- 172.21.15.170:0/3845881696 >> 172.21.15.170:0/3845881696 conn(0x7faad00cd640 msgr2=0x7faad00cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 -- 172.21.15.170:0/3845881696 shutdown_connections 2023-10-08T16:16:08.966 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.963+0000 7faad7192700 1 -- 172.21.15.170:0/3845881696 wait complete. 2023-10-08T16:16:08.971 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.964+0000 7faad7192700 1 Processor -- start 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.964+0000 7faad7192700 1 -- start start 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.964+0000 7faad7192700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 0x7faad00c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.964+0000 7faad7192700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad006ab40 0x7faad0159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad00af8c0 0x7faad0145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad00c73a0 con 0x7faad0068390 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad015b630 con 0x7faad006ab40 2023-10-08T16:16:08.972 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad015b300 con 0x7faad00af8c0 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 0x7faad00c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 0x7faad00c6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34204/0 (socket says 172.21.15.170:34204) 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 -- 172.21.15.170:0/2115740098 learned_addr learned my addr 172.21.15.170:0/2115740098 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faac7fff700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad006ab40 0x7faad0159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad572f700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad00af8c0 0x7faad0145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 -- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad00af8c0 msgr2=0x7faad0145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7faad00af8c0 0x7faad0145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.973 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 -- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad006ab40 msgr2=0x7faad0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7faad006ab40 0x7faad0159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac80094f0 con 0x7faad0068390 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad4f2e700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 0x7faad00c6260 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e190 tx=0x7faacc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00ee30 con 0x7faad0068390 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc014420 con 0x7faad0068390 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad00c6b00 con 0x7faad0068390 2023-10-08T16:16:08.974 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc0147e0 con 0x7faad0068390 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.965+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad00c70c0 con 0x7faad0068390 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.966+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faad00718b0 con 0x7faad0068390 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7faacc014b90 con 0x7faad0068390 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faabc067320 0x7faabc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7faacc085fb0 con 0x7faad0068390 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac7fff700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faabc067320 0x7faabc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.975 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7faabc06d170 0x7faabc06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faabc06fc50 con 0x7faabc06d170 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7faacc0862c0 con 0x7faad0068390 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.967+0000 7faad572f700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7faabc06d170 0x7faabc06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.968+0000 7faac7fff700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faabc067320 0x7faabc0697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faac00027e0 tx=0x7faac0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.968+0000 7faad572f700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7faabc06d170 0x7faabc06f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:08.976 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.968+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== osd.1 v2:172.21.15.170:6800/90867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faabc06fc50 con 0x7faabc06d170 2023-10-08T16:16:08.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 --> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faad0069480 con 0x7faabc06d170 2023-10-08T16:16:08.982 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faac4ff9700 1 -- 172.21.15.170:0/2115740098 <== osd.1 v2:172.21.15.170:6800/90867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faad0069480 con 0x7faabc06d170 2023-10-08T16:16:08.983 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:08.983 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:08.983 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7faabc06d170 msgr2=0x7faabc06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6800/90867,v1:172.21.15.170:6801/90867] conn(0x7faabc06d170 0x7faabc06f580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faabc067320 msgr2=0x7faabc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7faabc067320 0x7faabc0697d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faac00027e0 tx=0x7faac0009040 comp rx=0 tx=0).stop 2023-10-08T16:16:08.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 msgr2=0x7faad00c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 --2- 172.21.15.170:0/2115740098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faad0068390 0x7faad00c6260 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7faacc00e190 tx=0x7faacc009920 comp rx=0 tx=0).stop 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad572f700 1 -- 172.21.15.170:0/2115740098 reap_dead start 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 shutdown_connections 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.982+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 >> 172.21.15.170:0/2115740098 conn(0x7faad00cd640 msgr2=0x7faad00698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.983+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 shutdown_connections 2023-10-08T16:16:08.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:08.983+0000 7faad7192700 1 -- 172.21.15.170:0/2115740098 wait complete. 2023-10-08T16:16:08.994 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-08T16:16:09.109 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.109+0000 7f67c1bf1700 1 Processor -- start 2023-10-08T16:16:09.110 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.109+0000 7f67c1bf1700 1 -- start start 2023-10-08T16:16:09.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67c1bf1700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f67bc069560 0x7f67bc0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67c1bf1700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc069ee0 con 0x7f67bc159c60 2023-10-08T16:16:09.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67c1bf1700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67bc15a160 con 0x7f67bc159890 2023-10-08T16:16:09.116 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67c1bf1700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67bc0d13a0 con 0x7f67bc069560 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67baffd700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f67bc159c60 0x7f67bc15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43404/0 (socket says 172.21.15.170:43404) 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67baffd700 1 -- 172.21.15.170:0/18286169 learned_addr learned my addr 172.21.15.170:0/18286169 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67bbfff700 1 --2- 172.21.15.170:0/18286169 >> v2:172.21.15.191:3300/0 conn(0x7f67bc069560 0x7f67bc0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.110+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293003274 0 0) 0x7f67bc069ee0 con 0x7f67bc159c60 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67a4003570 con 0x7f67bc159c60 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059496986 0 0) 0x7f67bc15a160 con 0x7f67bc159890 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67bc069ee0 con 0x7f67bc159890 2023-10-08T16:16:09.117 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 366161163 0 0) 0x7f67a4003570 con 0x7f67bc159c60 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 >> v2:172.21.15.191:3300/0 conn(0x7f67bc069560 msgr2=0x7f67bc0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 --2- 172.21.15.170:0/18286169 >> v2:172.21.15.191:3300/0 conn(0x7f67bc069560 0x7f67bc0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 >> v1:172.21.15.191:6789/0 conn(0x7f67bc159890 legacy=0x7f67bc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc15e140 con 0x7f67bc159c60 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f67b0001470 con 0x7f67bc159c60 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 >> v1:172.21.15.170:6789/0 conn(0x7f67bc159c60 legacy=0x7f67bc15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 0x7f67a4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004810 0x7f67a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a40051c0 0x7f67a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0d13a0 con 0x7f67a4003bf0 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400c0b0 con 0x7f67a4004810 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a400c270 con 0x7f67a40051c0 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 0x7f67a4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67baffd700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004810 0x7f67a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bbfff700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a40051c0 0x7f67a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 -- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a40051c0 msgr2=0x7f67a400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a40051c0 0x7f67a400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 -- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004810 msgr2=0x7f67a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67a4004810 0x7f67a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 -- 172.21.15.170:0/18286169 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a4003570 con 0x7f67a4003bf0 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bbfff700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a40051c0 0x7f67a400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bbfff700 1 -- 172.21.15.170:0/18286169 reap_dead start 2023-10-08T16:16:09.120 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 0x7f67a4003fe0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f67ac002120 tx=0x7f67ac0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100f9b2ea0ebf911 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac001ab0 con 0x7f67a4003bf0 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac00c980 con 0x7f67a4003bf0 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.111+0000 7f67ba7fc700 1 -- 172.21.15.170:0/18286169 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd40 con 0x7f67a4003bf0 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 -- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 msgr2=0x7f67a4003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 0x7f67a4003fe0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f67ac002120 tx=0x7f67ac0017a0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 -- 172.21.15.170:0/18286169 shutdown_connections 2023-10-08T16:16:09.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/18286169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67a4003bf0 0x7f67a4003fe0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 -- 172.21.15.170:0/18286169 >> 172.21.15.170:0/18286169 conn(0x7f67bc068e20 msgr2=0x7f67bc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 -- 172.21.15.170:0/18286169 shutdown_connections 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.112+0000 7f67c1bf1700 1 -- 172.21.15.170:0/18286169 wait complete. 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.113+0000 7f67c1bf1700 1 Processor -- start 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.113+0000 7f67c1bf1700 1 -- start start 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a4003bf0 0x7f67bc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc13c370 0x7f67bc0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0ce810 con 0x7f67bc0b3590 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0cfdc0 con 0x7f67bc13c370 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67c1bf1700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67bc0b3a20 con 0x7f67a4003bf0 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34216/0 (socket says 172.21.15.170:34216) 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 -- 172.21.15.170:0/2031684708 learned_addr learned my addr 172.21.15.170:0/2031684708 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a4003bf0 0x7f67bc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67bbfff700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc13c370 0x7f67bc0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 -- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a4003bf0 msgr2=0x7f67bc0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a4003bf0 0x7f67bc0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 -- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc13c370 msgr2=0x7f67bc0b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc13c370 0x7f67bc0b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.114+0000 7f67baffd700 1 -- 172.21.15.170:0/2031684708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67bc0b3d00 con 0x7f67bc0b3590 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67baffd700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007390 tx=0x7f67b0001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b0018030 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67b000f6c0 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67bc0b3fe0 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b000fa60 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.115+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67bc0b45a0 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.116+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67bc04c3c0 con 0x7f67bc0b3590 2023-10-08T16:16:09.125 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.116+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f67b001b070 con 0x7f67bc0b3590 2023-10-08T16:16:09.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.116+0000 7f67a37fe700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 0x7f67a80697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.116+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f67b0019070 con 0x7f67bc0b3590 2023-10-08T16:16:09.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.117+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 0x7f67a80697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.126 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.117+0000 7f67bb7fe700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 0x7f67a80697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f67ac000f80 tx=0x7f67ac015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.121+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67b0051660 con 0x7f67bc0b3590 2023-10-08T16:16:09.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.302+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f67bc06a5b0 con 0x7f67a8067320 2023-10-08T16:16:09.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.303+0000 7f67a37fe700 1 -- 172.21.15.170:0/2031684708 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f67bc06a5b0 con 0x7f67a8067320 2023-10-08T16:16:09.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 msgr2=0x7f67a80697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 0x7f67a80697d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f67ac000f80 tx=0x7f67ac015000 comp rx=0 tx=0).stop 2023-10-08T16:16:09.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 msgr2=0x7f67bc0cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f67b0007390 tx=0x7f67b0001df0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 shutdown_connections 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f67bc13c370 0x7f67bc0b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f67a4003bf0 0x7f67bc0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f67a8067320 0x7f67a80697d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 --2- 172.21.15.170:0/2031684708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67bc0b3590 0x7f67bc0cf880 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 >> 172.21.15.170:0/2031684708 conn(0x7f67bc068e20 msgr2=0x7f67bc15bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 shutdown_connections 2023-10-08T16:16:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.305+0000 7f67c1bf1700 1 -- 172.21.15.170:0/2031684708 wait complete. 2023-10-08T16:16:09.307 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 1 to deep-scrub 2023-10-08T16:16:09.317 INFO:tasks.ceph:Scrubbing osd.2 2023-10-08T16:16:09.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:09.435 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.435+0000 7fba8ef29700 1 Processor -- start 2023-10-08T16:16:09.436 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.435+0000 7fba8ef29700 1 -- start start 2023-10-08T16:16:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8ef29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8ef29700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fba880b0790 0x7fba880b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8ef29700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba880cfa00 con 0x7fba881591e0 2023-10-08T16:16:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8ef29700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba880cb580 con 0x7fba880b0790 2023-10-08T16:16:09.438 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8ef29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880b2de0 con 0x7fba880b03c0 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8d726700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fba880b0790 0x7fba880b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8d726700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fba880b0790 0x7fba880b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34218/0 (socket says 172.21.15.170:34218) 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8d726700 1 -- 172.21.15.170:0/3977732225 learned_addr learned my addr 172.21.15.170:0/3977732225 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.436+0000 7fba8df27700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8d726700 1 -- 172.21.15.170:0/3977732225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 msgr2=0x7fba880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8d726700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba880cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8d726700 1 -- 172.21.15.170:0/3977732225 >> v1:172.21.15.191:6789/0 conn(0x7fba881591e0 legacy=0x7fba88159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8cf25700 1 -- 172.21.15.170:0/3977732225 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800188055 0 0) 0x7fba880cfa00 con 0x7fba881591e0 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8d726700 1 -- 172.21.15.170:0/3977732225 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880cf160 con 0x7fba880b0790 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8d726700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba880b0790 0x7fba880b28a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fba780099b0 tx=0x7fba78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736a4808bf972c3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8cf25700 1 -- 172.21.15.170:0/3977732225 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78009ea0 con 0x7fba880b0790 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8cf25700 1 -- 172.21.15.170:0/3977732225 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7800b860 con 0x7fba880b0790 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8cf25700 1 -- 172.21.15.170:0/3977732225 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7800bc00 con 0x7fba880b0790 2023-10-08T16:16:09.440 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 -- 172.21.15.170:0/3977732225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba880b0790 msgr2=0x7fba880b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba880b0790 0x7fba880b28a0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fba780099b0 tx=0x7fba78009250 comp rx=0 tx=0).stop 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 -- 172.21.15.170:0/3977732225 shutdown_connections 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba880cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 --2- 172.21.15.170:0/3977732225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba880b0790 0x7fba880b28a0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 -- 172.21.15.170:0/3977732225 >> 172.21.15.170:0/3977732225 conn(0x7fba88063230 msgr2=0x7fba88063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 -- 172.21.15.170:0/3977732225 shutdown_connections 2023-10-08T16:16:09.441 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.437+0000 7fba8ef29700 1 -- 172.21.15.170:0/3977732225 wait complete. 2023-10-08T16:16:09.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.438+0000 7fba8ef29700 1 Processor -- start 2023-10-08T16:16:09.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 -- start start 2023-10-08T16:16:09.444 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba8815dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba880b0790 0x7fba8815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba881591e0 0x7fba8815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8815ae00 con 0x7fba881591e0 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba880c64e0 con 0x7fba880b0790 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8ef29700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba880c5bb0 con 0x7fba880b03c0 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8df27700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba8815dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.445 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8d726700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba880b0790 0x7fba8815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8df27700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba8815dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:34730/0 (socket says 172.21.15.170:34730) 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8e728700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba881591e0 0x7fba8815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.439+0000 7fba8df27700 1 -- 172.21.15.170:0/402554729 learned_addr learned my addr 172.21.15.170:0/402554729 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 -- 172.21.15.170:0/402554729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 msgr2=0x7fba8815dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fba880b03c0 0x7fba8815dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 -- 172.21.15.170:0/402554729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba880b0790 msgr2=0x7fba8815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fba880b0790 0x7fba8815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.446 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780094f0 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8e728700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba881591e0 0x7fba8815c330 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fba800012e0 tx=0x7fba800016c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80012630 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80012790 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba880c5e90 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80012b50 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.440+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba880c6970 con 0x7fba881591e0 2023-10-08T16:16:09.447 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.441+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fba880cc9b0 con 0x7fba881591e0 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.441+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fba8001a3a0 con 0x7fba881591e0 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba7dffb700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba6c067320 0x7fba6c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fba80084b00 con 0x7fba881591e0 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba8df27700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba6c067320 0x7fba6c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba7dffb700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7fba6c06d170 0x7fba6c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba6c06fc50 con 0x7fba6c06d170 2023-10-08T16:16:09.448 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba80089220 con 0x7fba881591e0 2023-10-08T16:16:09.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba8d726700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7fba6c06d170 0x7fba6c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba8df27700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba6c067320 0x7fba6c0697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fba84009dd0 tx=0x7fba840091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.442+0000 7fba8d726700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7fba6c06d170 0x7fba6c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.449 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.443+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== osd.2 v2:172.21.15.170:6816/90869 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fba6c06fc50 con 0x7fba6c06d170 2023-10-08T16:16:09.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.456+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 --> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fba8804c3c0 con 0x7fba6c06d170 2023-10-08T16:16:09.457 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba7dffb700 1 -- 172.21.15.170:0/402554729 <== osd.2 v2:172.21.15.170:6816/90869 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fba8804c3c0 con 0x7fba6c06d170 2023-10-08T16:16:09.457 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:09.457 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:09.458 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7fba6c06d170 msgr2=0x7fba6c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6816/90869,v1:172.21.15.170:6817/90869] conn(0x7fba6c06d170 0x7fba6c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba6c067320 msgr2=0x7fba6c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fba6c067320 0x7fba6c0697d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fba84009dd0 tx=0x7fba840091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba881591e0 msgr2=0x7fba8815c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.459 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 --2- 172.21.15.170:0/402554729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba881591e0 0x7fba8815c330 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fba800012e0 tx=0x7fba800016c0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8e728700 1 -- 172.21.15.170:0/402554729 reap_dead start 2023-10-08T16:16:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 shutdown_connections 2023-10-08T16:16:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 >> 172.21.15.170:0/402554729 conn(0x7fba88063230 msgr2=0x7fba880cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 shutdown_connections 2023-10-08T16:16:09.460 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.457+0000 7fba8ef29700 1 -- 172.21.15.170:0/402554729 wait complete. 2023-10-08T16:16:09.469 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-08T16:16:09.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.585+0000 7fde950d6700 1 Processor -- start 2023-10-08T16:16:09.586 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.585+0000 7fde950d6700 1 -- start start 2023-10-08T16:16:09.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.586+0000 7fde950d6700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fde900aca80 0x7fde9006aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.588 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.586+0000 7fde950d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.586+0000 7fde950d6700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9006bdb0 con 0x7fde900c81f0 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.586+0000 7fde950d6700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900c8670 con 0x7fde900aca80 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.586+0000 7fde950d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900cb370 con 0x7fde90069d60 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fde900aca80 0x7fde9006aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fde900aca80 0x7fde9006aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:34480/0 (socket says 172.21.15.170:34480) 2023-10-08T16:16:09.589 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8f59e700 1 --1- >> v1:172.21.15.191:6789/0 conn(0x7fde900c81f0 0x7fde90068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54444/0 (socket says 172.21.15.170:54444) 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 -- 172.21.15.170:0/2858483063 learned_addr learned my addr 172.21.15.170:0/2858483063 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8e59c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:34736/0 (socket says 172.21.15.170:34736) 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 -- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 msgr2=0x7fde900c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 -- 172.21.15.170:0/2858483063 >> v1:172.21.15.191:6789/0 conn(0x7fde900c81f0 legacy=0x7fde90068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.590 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 -- 172.21.15.170:0/2858483063 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde90159f60 con 0x7fde900aca80 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8dd9b700 1 -- 172.21.15.170:0/2858483063 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608322053 0 0) 0x7fde9006bdb0 con 0x7fde900c81f0 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8e59c700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8ed9d700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde900aca80 0x7fde9006aba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fde80009990 tx=0x7fde80009220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91b8a47a9b8c9bed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8dd9b700 1 -- 172.21.15.170:0/2858483063 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80009e80 con 0x7fde900aca80 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8dd9b700 1 -- 172.21.15.170:0/2858483063 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80005c10 con 0x7fde900aca80 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.587+0000 7fde8dd9b700 1 -- 172.21.15.170:0/2858483063 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000b650 con 0x7fde900aca80 2023-10-08T16:16:09.591 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 -- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde900aca80 msgr2=0x7fde9006aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde900aca80 0x7fde9006aba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fde80009990 tx=0x7fde80009220 comp rx=0 tx=0).stop 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 -- 172.21.15.170:0/2858483063 shutdown_connections 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde900aca80 0x7fde9006aba0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 --2- 172.21.15.170:0/2858483063 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde90069d60 0x7fde900c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 -- 172.21.15.170:0/2858483063 >> 172.21.15.170:0/2858483063 conn(0x7fde900cd670 msgr2=0x7fde900cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 -- 172.21.15.170:0/2858483063 shutdown_connections 2023-10-08T16:16:09.592 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.588+0000 7fde950d6700 1 -- 172.21.15.170:0/2858483063 wait complete. 2023-10-08T16:16:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.589+0000 7fde950d6700 1 Processor -- start 2023-10-08T16:16:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.589+0000 7fde950d6700 1 -- start start 2023-10-08T16:16:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde90069d60 0x7fde90157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde900aca80 0x7fde900b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.595 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900b2580 con 0x7fde900c81f0 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900b1970 con 0x7fde90069d60 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde950d6700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900b1c50 con 0x7fde900aca80 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34230/0 (socket says 172.21.15.170:34230) 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 -- 172.21.15.170:0/3259715279 learned_addr learned my addr 172.21.15.170:0/3259715279 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8e59c700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde900aca80 0x7fde900b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.596 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8ed9d700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde90069d60 0x7fde90157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 -- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde900aca80 msgr2=0x7fde900b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde900aca80 0x7fde900b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 -- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde90069d60 msgr2=0x7fde90157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde90069d60 0x7fde90157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.590+0000 7fde8f59e700 1 -- 172.21.15.170:0/3259715279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84009420 con 0x7fde900c81f0 2023-10-08T16:16:09.597 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde8f59e700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ccc0 tx=0x7fde88001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88005df0 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8800a600 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80009480 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8800a9c0 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.591+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900b2210 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.592+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9004c3c0 con 0x7fde900c81f0 2023-10-08T16:16:09.598 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.592+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fde880070f0 con 0x7fde900c81f0 2023-10-08T16:16:09.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.592+0000 7fde76ffd700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 0x7fde7c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.592+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fde88084a50 con 0x7fde900c81f0 2023-10-08T16:16:09.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.592+0000 7fde8ed9d700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 0x7fde7c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.593+0000 7fde8ed9d700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 0x7fde7c0697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fde80009260 tx=0x7fde80016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.599 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.596+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde88051b10 con 0x7fde900c81f0 2023-10-08T16:16:09.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.780+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fde9006b510 con 0x7fde7c067320 2023-10-08T16:16:09.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.780+0000 7fde76ffd700 1 -- 172.21.15.170:0/3259715279 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fde9006b510 con 0x7fde7c067320 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 msgr2=0x7fde7c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 0x7fde7c0697d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fde80009260 tx=0x7fde80016000 comp rx=0 tx=0).stop 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 msgr2=0x7fde9015d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fde8800ccc0 tx=0x7fde88001a70 comp rx=0 tx=0).stop 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 shutdown_connections 2023-10-08T16:16:09.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fde7c067320 0x7fde7c0697d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde900c81f0 0x7fde9015d5e0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fde90069d60 0x7fde90157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 --2- 172.21.15.170:0/3259715279 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fde900aca80 0x7fde900b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 >> 172.21.15.170:0/3259715279 conn(0x7fde900cd670 msgr2=0x7fde900c3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 shutdown_connections 2023-10-08T16:16:09.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.782+0000 7fde950d6700 1 -- 172.21.15.170:0/3259715279 wait complete. 2023-10-08T16:16:09.785 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 2 to deep-scrub 2023-10-08T16:16:09.794 INFO:tasks.ceph:Scrubbing osd.3 2023-10-08T16:16:09.795 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:09.913 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.913+0000 7fcba5a17700 1 Processor -- start 2023-10-08T16:16:09.914 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.913+0000 7fcba5a17700 1 -- start start 2023-10-08T16:16:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5a17700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcba00b03c0 0x7fcba00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5a17700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcba00b0790 0x7fcba00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5a17700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba00cfa00 con 0x7fcba01591e0 2023-10-08T16:16:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5a17700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba00cb580 con 0x7fcba00b0790 2023-10-08T16:16:09.916 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5a17700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba00b2de0 con 0x7fcba00b03c0 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcba00b0790 0x7fcba00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba4a15700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fcba00b03c0 0x7fcba00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcba00b0790 0x7fcba00b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34244/0 (socket says 172.21.15.170:34244) 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 -- 172.21.15.170:0/1202437842 learned_addr learned my addr 172.21.15.170:0/1202437842 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcba5216700 1 --1- 172.21.15.170:0/1202437842 >> v1:172.21.15.191:6789/0 conn(0x7fcba01591e0 0x7fcba0159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:54454/0 (socket says 172.21.15.170:54454) 2023-10-08T16:16:09.917 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 -- 172.21.15.170:0/1202437842 >> v2:172.21.15.191:3300/0 conn(0x7fcba00b03c0 msgr2=0x7fcba00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 --2- 172.21.15.170:0/1202437842 >> v2:172.21.15.191:3300/0 conn(0x7fcba00b03c0 0x7fcba00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 -- 172.21.15.170:0/1202437842 >> v1:172.21.15.191:6789/0 conn(0x7fcba01591e0 legacy=0x7fcba0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.914+0000 7fcb9ffff700 1 -- 172.21.15.170:0/1202437842 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba00cf160 con 0x7fcba00b0790 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcb9ffff700 1 --2- 172.21.15.170:0/1202437842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b0790 0x7fcba00b28a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009f20 tx=0x7fcb90009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a27525bce834f7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcb9f7fe700 1 -- 172.21.15.170:0/1202437842 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000e070 con 0x7fcba00b0790 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcb9f7fe700 1 -- 172.21.15.170:0/1202437842 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9000b990 con 0x7fcba00b0790 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcb9f7fe700 1 -- 172.21.15.170:0/1202437842 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900092e0 con 0x7fcba00b0790 2023-10-08T16:16:09.918 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 -- 172.21.15.170:0/1202437842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b0790 msgr2=0x7fcba00b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 --2- 172.21.15.170:0/1202437842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b0790 0x7fcba00b28a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009f20 tx=0x7fcb90009840 comp rx=0 tx=0).stop 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 -- 172.21.15.170:0/1202437842 shutdown_connections 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 --2- 172.21.15.170:0/1202437842 >> v2:172.21.15.191:3300/0 conn(0x7fcba00b03c0 0x7fcba00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 --2- 172.21.15.170:0/1202437842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b0790 0x7fcba00b28a0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 -- 172.21.15.170:0/1202437842 >> 172.21.15.170:0/1202437842 conn(0x7fcba0063230 msgr2=0x7fcba0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 -- 172.21.15.170:0/1202437842 shutdown_connections 2023-10-08T16:16:09.919 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.915+0000 7fcba5a17700 1 -- 172.21.15.170:0/1202437842 wait complete. 2023-10-08T16:16:09.922 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.916+0000 7fcba5a17700 1 Processor -- start 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 -- start start 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 0x7fcba016a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcba00b0790 0x7fcba00c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba01591e0 0x7fcba00d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba015b970 con 0x7fcba00b03c0 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0158060 con 0x7fcba01591e0 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba5a17700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba016aa70 con 0x7fcba00b0790 2023-10-08T16:16:09.923 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba4a15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 0x7fcba016a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba4a15700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 0x7fcba016a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34248/0 (socket says 172.21.15.170:34248) 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.917+0000 7fcba4a15700 1 -- 172.21.15.170:0/819803629 learned_addr learned my addr 172.21.15.170:0/819803629 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcb9ffff700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcba00b0790 0x7fcba00c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba5216700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba01591e0 0x7fcba00d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 -- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcba00b0790 msgr2=0x7fcba00c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fcba00b0790 0x7fcba00c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.924 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 -- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba01591e0 msgr2=0x7fcba00d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba01591e0 0x7fcba00d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90009ae0 con 0x7fcba00b03c0 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba5216700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fcba01591e0 0x7fcba00d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba4a15700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 0x7fcba016a530 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800ec80 tx=0x7fcb980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800c970 con 0x7fcba00b03c0 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9800cad0 con 0x7fcba00b03c0 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba015a3c0 con 0x7fcba00b03c0 2023-10-08T16:16:09.925 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800ce70 con 0x7fcba00b03c0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.918+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba015aa00 con 0x7fcba00b03c0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.919+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcba004c3c0 con 0x7fcba00b03c0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.919+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fcb98013680 con 0x7fcba00b03c0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.919+0000 7fcb9cff9700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb80067320 0x7fcb800697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fcb98084f80 con 0x7fcba00b03c0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9ffff700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb80067320 0x7fcb800697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9cff9700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fcb8006d170 0x7fcb8006f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:09.926 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb8006fc50 con 0x7fcb8006d170 2023-10-08T16:16:09.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb980874e0 con 0x7fcba00b03c0 2023-10-08T16:16:09.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcba5216700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fcb8006d170 0x7fcb8006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:09.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcb9ffff700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb80067320 0x7fcb800697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c010 tx=0x7fcb90018000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.920+0000 7fcba5216700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fcb8006d170 0x7fcb8006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:09.927 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.921+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== osd.3 v2:172.21.15.170:6824/90884 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcb8006fc50 con 0x7fcb8006d170 2023-10-08T16:16:09.934 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.934+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 --> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcba015ad20 con 0x7fcb8006d170 2023-10-08T16:16:09.935 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcb9cff9700 1 -- 172.21.15.170:0/819803629 <== osd.3 v2:172.21.15.170:6824/90884 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcba015ad20 con 0x7fcb8006d170 2023-10-08T16:16:09.935 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:09.935 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:09.935 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:09.936 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fcb8006d170 msgr2=0x7fcb8006f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6824/90884,v1:172.21.15.170:6825/90884] conn(0x7fcb8006d170 0x7fcb8006f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb80067320 msgr2=0x7fcb800697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fcb80067320 0x7fcb800697d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c010 tx=0x7fcb90018000 comp rx=0 tx=0).stop 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 msgr2=0x7fcba016a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 --2- 172.21.15.170:0/819803629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcba00b03c0 0x7fcba016a530 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fcb9800ec80 tx=0x7fcb980091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5216700 1 -- 172.21.15.170:0/819803629 reap_dead start 2023-10-08T16:16:09.937 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 shutdown_connections 2023-10-08T16:16:09.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 >> 172.21.15.170:0/819803629 conn(0x7fcba0063230 msgr2=0x7fcba0068d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:09.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 shutdown_connections 2023-10-08T16:16:09.938 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:09.935+0000 7fcba5a17700 1 -- 172.21.15.170:0/819803629 wait complete. 2023-10-08T16:16:09.947 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-08T16:16:10.062 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.062+0000 7f55aaf7c700 1 Processor -- start 2023-10-08T16:16:10.063 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.062+0000 7f55aaf7c700 1 -- start start 2023-10-08T16:16:10.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55aaf7c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 0x7f55a40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.065 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55aaf7c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f55a415df50 0x7f55a415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55aaf7c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a415d7e0 con 0x7f55a406c0a0 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55aaf7c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a415dac0 con 0x7f55a415df50 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55aaf7c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a415aa30 con 0x7f55a406bcd0 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a3fff700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f55a406c0a0 0x7f55a415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43406/0 (socket says 172.21.15.170:43406) 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a3fff700 1 -- 172.21.15.170:0/4131960119 learned_addr learned my addr 172.21.15.170:0/4131960119 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4131960119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 0x7f55a40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.066 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a9519700 1 --2- 172.21.15.170:0/4131960119 >> v2:172.21.15.191:3300/0 conn(0x7f55a415df50 0x7f55a415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a37fe700 1 -- 172.21.15.170:0/4131960119 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809500533 0 0) 0x7f55a415d7e0 con 0x7f55a406c0a0 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a37fe700 1 -- 172.21.15.170:0/4131960119 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5594003570 con 0x7f55a406c0a0 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a8d18700 1 -- 172.21.15.170:0/4131960119 >> v2:172.21.15.191:3300/0 conn(0x7f55a415df50 msgr2=0x7f55a415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4131960119 >> v2:172.21.15.191:3300/0 conn(0x7f55a415df50 0x7f55a415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a8d18700 1 -- 172.21.15.170:0/4131960119 >> v1:172.21.15.170:6789/0 conn(0x7f55a406c0a0 legacy=0x7f55a415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.063+0000 7f55a8d18700 1 -- 172.21.15.170:0/4131960119 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a415ad10 con 0x7f55a406bcd0 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4131960119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 0x7f55a40caa90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55980098e0 tx=0x7f55980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2d091d2eb033aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.067 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55a37fe700 1 -- 172.21.15.170:0/4131960119 <== mon.2 v2:172.21.15.191:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598011630 con 0x7f55a406bcd0 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55a37fe700 1 -- 172.21.15.170:0/4131960119 <== mon.2 v2:172.21.15.191:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5598011790 con 0x7f55a406bcd0 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55a37fe700 1 -- 172.21.15.170:0/4131960119 <== mon.2 v2:172.21.15.191:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598011b30 con 0x7f55a406bcd0 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4131960119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 msgr2=0x7f55a40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4131960119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 0x7f55a40caa90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55980098e0 tx=0x7f55980091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4131960119 shutdown_connections 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4131960119 >> v2:172.21.15.191:3300/0 conn(0x7f55a415df50 0x7f55a415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4131960119 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a406bcd0 0x7f55a40caa90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.068 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4131960119 >> 172.21.15.170:0/4131960119 conn(0x7f55a40ce600 msgr2=0x7f55a40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4131960119 shutdown_connections 2023-10-08T16:16:10.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.064+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4131960119 wait complete. 2023-10-08T16:16:10.069 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.065+0000 7f55aaf7c700 1 Processor -- start 2023-10-08T16:16:10.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 -- start start 2023-10-08T16:16:10.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.071 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 0x7f55a41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a415df50 0x7f55a4156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4154c70 con 0x7f55a406bcd0 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40c56b0 con 0x7f55a406c0a0 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55aaf7c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4158270 con 0x7f55a415df50 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55a8d18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55a8d18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34260/0 (socket says 172.21.15.170:34260) 2023-10-08T16:16:10.072 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.066+0000 7f55a8d18700 1 -- 172.21.15.170:0/4228782407 learned_addr learned my addr 172.21.15.170:0/4228782407 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a9519700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a415df50 0x7f55a4156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a3fff700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 0x7f55a41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 -- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a415df50 msgr2=0x7f55a4156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a415df50 0x7f55a4156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 -- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 msgr2=0x7f55a41590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 0x7f55a41590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 -- 172.21.15.170:0/4228782407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598009420 con 0x7f55a406bcd0 2023-10-08T16:16:10.073 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a3fff700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 0x7f55a41590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a8d18700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ed0 tx=0x7f5598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598011b10 con 0x7f55a406bcd0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980253f0 con 0x7f55a406bcd0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a4158550 con 0x7f55a406bcd0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55980257b0 con 0x7f55a406bcd0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.067+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a4158b10 con 0x7f55a406bcd0 2023-10-08T16:16:10.074 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.068+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a4064ed0 con 0x7f55a406bcd0 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.068+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5598025b60 con 0x7f55a406bcd0 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.069+0000 7f55a0ff9700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 0x7f55880697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.069+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5598086170 con 0x7f55a406bcd0 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.069+0000 7f55a3fff700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 0x7f55880697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.069+0000 7f55a3fff700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 0x7f55880697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5590007fd0 tx=0x7f5590009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.075 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.073+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5598053100 con 0x7f55a406bcd0 2023-10-08T16:16:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.256+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a404c3c0 con 0x7f5588067320 2023-10-08T16:16:10.257 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.257+0000 7f55a0ff9700 1 -- 172.21.15.170:0/4228782407 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f55a404c3c0 con 0x7f5588067320 2023-10-08T16:16:10.259 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 msgr2=0x7f55880697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 0x7f55880697d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5590007fd0 tx=0x7f5590009040 comp rx=0 tx=0).stop 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 msgr2=0x7f55a4154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5598009ed0 tx=0x7f5598017000 comp rx=0 tx=0).stop 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 shutdown_connections 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f55a415df50 0x7f55a4156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5588067320 0x7f55880697d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.260 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55a406bcd0 0x7f55a4154730 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 --2- 172.21.15.170:0/4228782407 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f55a406c0a0 0x7f55a41590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 >> 172.21.15.170:0/4228782407 conn(0x7f55a40ce600 msgr2=0x7f55a406a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 shutdown_connections 2023-10-08T16:16:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.259+0000 7f55aaf7c700 1 -- 172.21.15.170:0/4228782407 wait complete. 2023-10-08T16:16:10.261 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 3 to deep-scrub 2023-10-08T16:16:10.271 INFO:tasks.ceph:Scrubbing osd.4 2023-10-08T16:16:10.272 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:10.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.390+0000 7f945761a700 1 Processor -- start 2023-10-08T16:16:10.391 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.390+0000 7f945761a700 1 -- start start 2023-10-08T16:16:10.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f945761a700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9450159ec0 0x7f94500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.395 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f945761a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94500b5b00 con 0x7f94500cf350 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f945761a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94500b5850 con 0x7f94500b53d0 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f945761a700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500b5da0 con 0x7f9450159ec0 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f9455bb7700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f94500cf350 0x7f94500cd010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43416/0 (socket says 172.21.15.170:43416) 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.391+0000 7f9455bb7700 1 -- 172.21.15.170:0/4099037707 learned_addr learned my addr 172.21.15.170:0/4099037707 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f94553b6700 1 --2- 172.21.15.170:0/4099037707 >> v2:172.21.15.191:3300/0 conn(0x7f9450159ec0 0x7f94500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634442830 0 0) 0x7f94500b5b00 con 0x7f94500cf350 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9438003570 con 0x7f94500cf350 2023-10-08T16:16:10.396 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449580539 0 0) 0x7f94500b5850 con 0x7f94500b53d0 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94500b5b00 con 0x7f94500b53d0 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3873525415 0 0) 0x7f9438003570 con 0x7f94500cf350 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 >> v2:172.21.15.191:3300/0 conn(0x7f9450159ec0 msgr2=0x7f94500b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 --2- 172.21.15.170:0/4099037707 >> v2:172.21.15.191:3300/0 conn(0x7f9450159ec0 0x7f94500b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 >> v1:172.21.15.191:6789/0 conn(0x7f94500b53d0 legacy=0x7f94500cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500b6080 con 0x7f94500cf350 2023-10-08T16:16:10.397 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f9440001470 con 0x7f94500cf350 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 >> v1:172.21.15.170:6789/0 conn(0x7f94500cf350 legacy=0x7f94500cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438003bf0 0x7f9438003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 0x7f9438004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94380051c0 0x7f943800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b5da0 con 0x7f9438004810 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943800c0b0 con 0x7f94380051c0 2023-10-08T16:16:10.398 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943800c270 con 0x7f9438003bf0 2023-10-08T16:16:10.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f94553b6700 1 --2- 172.21.15.170:0/4099037707 >> v2:172.21.15.191:3300/0 conn(0x7f9450159ec0 0x7f94500b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T16:16:10.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 0x7f9438004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9455bb7700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94380051c0 0x7f943800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f94553b6700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438003bf0 0x7f9438003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.401 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 -- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438003bf0 msgr2=0x7f9438003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438003bf0 0x7f9438003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 -- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94380051c0 msgr2=0x7f943800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94380051c0 0x7f943800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9454bb5700 1 -- 172.21.15.170:0/4099037707 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438003570 con 0x7f9438004810 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9455bb7700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f94380051c0 0x7f943800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.392+0000 7f9455bb7700 1 -- 172.21.15.170:0/4099037707 reap_dead start 2023-10-08T16:16:10.402 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f9454bb5700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 0x7f9438004c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9444001a10 tx=0x7f9444007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b68abd0dc4333a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400c5d0 con 0x7f9438004810 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400c730 con 0x7f9438004810 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f944ffff700 1 -- 172.21.15.170:0/4099037707 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400caf0 con 0x7f9438004810 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 -- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 msgr2=0x7f9438004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 0x7f9438004c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f9444001a10 tx=0x7f9444007d10 comp rx=0 tx=0).stop 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 -- 172.21.15.170:0/4099037707 shutdown_connections 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 --2- 172.21.15.170:0/4099037707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9438004810 0x7f9438004c80 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.403 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 -- 172.21.15.170:0/4099037707 >> 172.21.15.170:0/4099037707 conn(0x7f94500999f0 msgr2=0x7f94500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 -- 172.21.15.170:0/4099037707 shutdown_connections 2023-10-08T16:16:10.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.393+0000 7f945761a700 1 -- 172.21.15.170:0/4099037707 wait complete. 2023-10-08T16:16:10.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.394+0000 7f945761a700 1 Processor -- start 2023-10-08T16:16:10.404 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.394+0000 7f945761a700 1 -- start start 2023-10-08T16:16:10.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438004810 0x7f9450155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 0x7f945015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f945015d080 0x7f9450160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015f250 con 0x7f945015a830 2023-10-08T16:16:10.407 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015ef20 con 0x7f945015d080 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.395+0000 7f945761a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945015b710 con 0x7f9438004810 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 0x7f945015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 0x7f945015cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34276/0 (socket says 172.21.15.170:34276) 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 -- 172.21.15.170:0/2668211130 learned_addr learned my addr 172.21.15.170:0/2668211130 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f94553b6700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438004810 0x7f9450155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9455bb7700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f945015d080 0x7f9450160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.408 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 -- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438004810 msgr2=0x7f9450155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9438004810 0x7f9450155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 -- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f945015d080 msgr2=0x7f9450160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f945015d080 0x7f9450160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945015b9f0 con 0x7f945015a830 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f9454bb5700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 0x7f945015cb40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f943800c080 tx=0x7f9444015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400c5d0 con 0x7f945015a830 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944400cea0 con 0x7f945015a830 2023-10-08T16:16:10.409 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945015bcd0 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444007af0 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.396+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94501559c0 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.397+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f945015bf10 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.397+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f9444019030 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f944d7fa700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c067320 0x7f943c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f94440860a0 con 0x7f945015a830 2023-10-08T16:16:10.410 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f94553b6700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c067320 0x7f943c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f944d7fa700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f943c06d170 0x7f943c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f943c06fc50 con 0x7f943c06d170 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9444088600 con 0x7f945015a830 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f9455bb7700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f943c06d170 0x7f943c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f94553b6700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c067320 0x7f943c0697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9448012910 tx=0x7f944800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.398+0000 7f9455bb7700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f943c06d170 0x7f943c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.411 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.399+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== osd.4 v2:172.21.15.191:6800/91066 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f943c06fc50 con 0x7f943c06d170 2023-10-08T16:16:10.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.413+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 --> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f945015f090 con 0x7f943c06d170 2023-10-08T16:16:10.413 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.413+0000 7f944d7fa700 1 -- 172.21.15.170:0/2668211130 <== osd.4 v2:172.21.15.191:6800/91066 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f945015f090 con 0x7f943c06d170 2023-10-08T16:16:10.414 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:10.414 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:10.414 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:10.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.413+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f943c06d170 msgr2=0x7f943c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.415 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.413+0000 7f945761a700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.191:6800/91066,v1:172.21.15.191:6801/91066] conn(0x7f943c06d170 0x7f943c06f580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c067320 msgr2=0x7f943c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f943c067320 0x7f943c0697d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9448012910 tx=0x7f944800b040 comp rx=0 tx=0).stop 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 msgr2=0x7f945015cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 --2- 172.21.15.170:0/2668211130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f945015a830 0x7f945015cb40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f943800c080 tx=0x7f9444015000 comp rx=0 tx=0).stop 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f9455bb7700 1 -- 172.21.15.170:0/2668211130 reap_dead start 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 shutdown_connections 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 >> 172.21.15.170:0/2668211130 conn(0x7f94500999f0 msgr2=0x7f94500c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.416 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 shutdown_connections 2023-10-08T16:16:10.417 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.414+0000 7f945761a700 1 -- 172.21.15.170:0/2668211130 wait complete. 2023-10-08T16:16:10.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-08T16:16:10.540 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.540+0000 7f9a04e71700 1 Processor -- start 2023-10-08T16:16:10.541 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.540+0000 7f9a04e71700 1 -- start start 2023-10-08T16:16:10.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f9a04e71700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f9a00159c60 0x7f9a0015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f9a04e71700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0015a160 con 0x7f9a00159890 2023-10-08T16:16:10.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f9a04e71700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a000d13a0 con 0x7f9a0015db10 2023-10-08T16:16:10.544 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f9a04e71700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0015df50 con 0x7f9a00159c60 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fe59c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f9a00159890 0x7f9a0015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43418/0 (socket says 172.21.15.170:43418) 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fe59c700 1 -- 172.21.15.170:0/371399292 learned_addr learned my addr 172.21.15.170:0/371399292 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99f5d9b700 1 --2- 172.21.15.170:0/371399292 >> v2:172.21.15.191:3300/0 conn(0x7f9a00159c60 0x7f9a0015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1301223413 0 0) 0x7f9a0015a160 con 0x7f9a00159890 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ec003570 con 0x7f9a00159890 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206889126 0 0) 0x7f9a000d13a0 con 0x7f9a0015db10 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.541+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a0015a160 con 0x7f9a0015db10 2023-10-08T16:16:10.545 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99f5d9b700 1 -- 172.21.15.170:0/371399292 >> v1:172.21.15.191:6789/0 conn(0x7f9a0015db10 legacy=0x7f9a00069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99f5d9b700 1 -- 172.21.15.170:0/371399292 >> v1:172.21.15.170:6789/0 conn(0x7f9a00159890 legacy=0x7f9a0015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 631003765 0 0) 0x7f99ec003570 con 0x7f9a00159890 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99f5d9b700 1 -- 172.21.15.170:0/371399292 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0015e140 con 0x7f9a00159c60 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99f5d9b700 1 --2- 172.21.15.170:0/371399292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a00159c60 0x7f9a0015d5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31deea537c592d17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8011630 con 0x7f9a00159c60 2023-10-08T16:16:10.546 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8011790 con 0x7f9a00159c60 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f99fdd9b700 1 -- 172.21.15.170:0/371399292 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8011b10 con 0x7f9a00159c60 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 -- 172.21.15.170:0/371399292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a00159c60 msgr2=0x7f9a0015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 --2- 172.21.15.170:0/371399292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a00159c60 0x7f9a0015d5d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098e0 tx=0x7f99e80091a0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 -- 172.21.15.170:0/371399292 shutdown_connections 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 --2- 172.21.15.170:0/371399292 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a00159c60 0x7f9a0015d5d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 -- 172.21.15.170:0/371399292 >> 172.21.15.170:0/371399292 conn(0x7f9a00068e20 msgr2=0x7f9a000d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.547 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 -- 172.21.15.170:0/371399292 shutdown_connections 2023-10-08T16:16:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.542+0000 7f9a04e71700 1 -- 172.21.15.170:0/371399292 wait complete. 2023-10-08T16:16:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 Processor -- start 2023-10-08T16:16:10.548 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 -- start start 2023-10-08T16:16:10.550 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00159c60 0x7f9a000cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0015db10 0x7f9a0015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b4780 con 0x7f9a00159890 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000cf760 con 0x7f9a0015db10 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f9a04e71700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000b3b70 con 0x7f9a00159c60 2023-10-08T16:16:10.551 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.544+0000 7f99fe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34278/0 (socket says 172.21.15.170:34278) 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 -- 172.21.15.170:0/2541453667 learned_addr learned my addr 172.21.15.170:0/2541453667 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99f5d9b700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00159c60 0x7f9a000cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fed9d700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0015db10 0x7f9a0015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 -- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00159c60 msgr2=0x7f9a000cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00159c60 0x7f9a000cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.552 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 -- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0015db10 msgr2=0x7f9a0015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0015db10 0x7f9a0015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 -- 172.21.15.170:0/2541453667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8009420 con 0x7f9a00159890 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99fe59c700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c5e0 tx=0x7f99f0007b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007360 con 0x7f9a00159890 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f00074e0 con 0x7f9a00159890 2023-10-08T16:16:10.553 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0004a80 con 0x7f9a00159890 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000b3e50 con 0x7f9a00159890 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.545+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000b4410 con 0x7f9a00159890 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.546+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0004c3c0 con 0x7f9a00159890 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.546+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f99f0014420 con 0x7f9a00159890 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.547+0000 7f99f6ffd700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 0x7f99e00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.554 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.547+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f99f00845a0 con 0x7f9a00159890 2023-10-08T16:16:10.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.547+0000 7f99f5d9b700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 0x7f99e00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.547+0000 7f99f5d9b700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 0x7f99e00697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99e80093f0 tx=0x7f99e8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.555 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.551+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99f0051770 con 0x7f9a00159890 2023-10-08T16:16:10.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.737+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a000b48c0 con 0x7f99e0067320 2023-10-08T16:16:10.737 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.737+0000 7f99f6ffd700 1 -- 172.21.15.170:0/2541453667 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99e8011900 con 0x7f99e0067320 2023-10-08T16:16:10.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 msgr2=0x7f99e00697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 0x7f99e00697d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f99e80093f0 tx=0x7f99e8017000 comp rx=0 tx=0).stop 2023-10-08T16:16:10.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 msgr2=0x7f9a000b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.740 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c5e0 tx=0x7f99f0007b30 comp rx=0 tx=0).stop 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 shutdown_connections 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9a0015db10 0x7f9a0015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f99e0067320 0x7f99e00697d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a00159890 0x7f9a000b3590 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 --2- 172.21.15.170:0/2541453667 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f9a00159c60 0x7f9a000cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 >> 172.21.15.170:0/2541453667 conn(0x7f9a00068e20 msgr2=0x7f9a00155080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 shutdown_connections 2023-10-08T16:16:10.741 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.739+0000 7f9a04e71700 1 -- 172.21.15.170:0/2541453667 wait complete. 2023-10-08T16:16:10.742 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 4 to deep-scrub 2023-10-08T16:16:10.752 INFO:tasks.ceph:Scrubbing osd.5 2023-10-08T16:16:10.752 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:10.870 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.870+0000 7fdfb117c700 1 Processor -- start 2023-10-08T16:16:10.871 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.870+0000 7fdfb117c700 1 -- start start 2023-10-08T16:16:10.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfb117c700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdfac159ec0 0x7fdfac0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfb117c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdfac0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfb117c700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac0b5b00 con 0x7fdfac0cce40 2023-10-08T16:16:10.879 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfb117c700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac0b5850 con 0x7fdfac159ec0 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfb117c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b5da0 con 0x7fdfac0b53d0 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfab59e700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdfac0cce40 0x7fdfac0cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43432/0 (socket says 172.21.15.170:43432) 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfab59e700 1 -- 172.21.15.170:0/2261630953 learned_addr learned my addr 172.21.15.170:0/2261630953 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfa259c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdfac0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/2261630953 >> v2:172.21.15.191:3300/0 conn(0x7fdfac159ec0 0x7fdfac0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488814486 0 0) 0x7fdfac0b5b00 con 0x7fdfac0cce40 2023-10-08T16:16:10.880 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf98003570 con 0x7fdfac0cce40 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1914313106 0 0) 0x7fdf98003570 con 0x7fdfac0cce40 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 msgr2=0x7fdfac0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdfac0cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 >> v2:172.21.15.191:3300/0 conn(0x7fdfac159ec0 msgr2=0x7fdfac0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 --2- 172.21.15.170:0/2261630953 >> v2:172.21.15.191:3300/0 conn(0x7fdfac159ec0 0x7fdfac0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.881 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac0b6080 con 0x7fdfac0cce40 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.871+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7fdfa40017f0 con 0x7fdfac0cce40 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 >> v1:172.21.15.170:6789/0 conn(0x7fdfac0cce40 legacy=0x7fdfac0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf98003a60 0x7fdf98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98004810 0x7fdf98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdf980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.882 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b5850 con 0x7fdf98003a60 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b5da0 con 0x7fdf98004810 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9800ba90 con 0x7fdfac0b53d0 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf98003a60 0x7fdf98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfab59e700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdf980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfa259c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98004810 0x7fdf98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 -- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 msgr2=0x7fdf980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.883 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac0b53d0 0x7fdf980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 -- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98004810 msgr2=0x7fdf98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdf98004810 0x7fdf98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 -- 172.21.15.170:0/2261630953 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98003570 con 0x7fdf98003a60 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf98003a60 0x7fdf98003e70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009bf0 tx=0x7fdf9c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaec05802f0cd383 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c005a00 con 0x7fdf98003a60 2023-10-08T16:16:10.884 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c005b60 con 0x7fdf98003a60 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.872+0000 7fdfaa59c700 1 -- 172.21.15.170:0/2261630953 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c00e6b0 con 0x7fdf98003a60 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 -- 172.21.15.170:0/2261630953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf98003a60 msgr2=0x7fdf98003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 --2- 172.21.15.170:0/2261630953 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf98003a60 0x7fdf98003e70 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009bf0 tx=0x7fdf9c009120 comp rx=0 tx=0).stop 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfab59e700 1 -- 172.21.15.170:0/2261630953 reap_dead start 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 -- 172.21.15.170:0/2261630953 shutdown_connections 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 -- 172.21.15.170:0/2261630953 >> 172.21.15.170:0/2261630953 conn(0x7fdfac0999f0 msgr2=0x7fdfac0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 -- 172.21.15.170:0/2261630953 shutdown_connections 2023-10-08T16:16:10.885 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.873+0000 7fdfb117c700 1 -- 172.21.15.170:0/2261630953 wait complete. 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.874+0000 7fdfb117c700 1 Processor -- start 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.874+0000 7fdfb117c700 1 -- start start 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac09ab80 0x7fdfac068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac06af40 0x7fdfac156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 0x7fdfac157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.886 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac169b00 con 0x7fdfac09af50 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0695f0 con 0x7fdfac06af40 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfb117c700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0698d0 con 0x7fdfac09ab80 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 0x7fdfac157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 0x7fdfac157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34304/0 (socket says 172.21.15.170:34304) 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 -- 172.21.15.170:0/3887218261 learned_addr learned my addr 172.21.15.170:0/3887218261 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac09ab80 0x7fdfac068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.887 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 -- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac09ab80 msgr2=0x7fdfac068da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac09ab80 0x7fdfac068da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 -- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac06af40 msgr2=0x7fdfac156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfa259c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac06af40 0x7fdfac156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdfac06af40 0x7fdfac156e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac069bb0 con 0x7fdfac09af50 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdfac09ab80 0x7fdfac068da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:10.888 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfab59e700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 0x7fdfac157340 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdfac08ad70 tx=0x7fdfa4002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.875+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa40133f0 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.876+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4013570 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.876+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac069e90 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.876+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4013930 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.876+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac09c0f0 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.876+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfac063c90 con 0x7fdfac09af50 2023-10-08T16:16:10.889 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdfa4013d20 con 0x7fdfac09af50 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf8c067320 0x7fdf8c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdfa40855c0 con 0x7fdfac09af50 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf8c067320 0x7fdf8c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7fdf8c06d170 0x7fdf8c06f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdf8c06fc50 con 0x7fdf8c06d170 2023-10-08T16:16:10.890 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.877+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfa4087b20 con 0x7fdfac09af50 2023-10-08T16:16:10.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.878+0000 7fdfa259c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7fdf8c06d170 0x7fdf8c06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:10.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.878+0000 7fdfaad9d700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf8c067320 0x7fdf8c0697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00a010 tx=0x7fdf9c019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.878+0000 7fdfa259c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7fdf8c06d170 0x7fdf8c06f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:10.891 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.879+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== osd.5 v2:172.21.15.191:6824/91096 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fdf8c06fc50 con 0x7fdf8c06d170 2023-10-08T16:16:10.892 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.892+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 --> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdfac169c40 con 0x7fdf8c06d170 2023-10-08T16:16:10.893 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfa37fe700 1 -- 172.21.15.170:0/3887218261 <== osd.5 v2:172.21.15.191:6824/91096 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdf94009a20 con 0x7fdf8c06d170 2023-10-08T16:16:10.894 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:10.894 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:10.894 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:10.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7fdf8c06d170 msgr2=0x7fdf8c06f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.191:6824/91096,v1:172.21.15.191:6825/91096] conn(0x7fdf8c06d170 0x7fdf8c06f580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:10.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf8c067320 msgr2=0x7fdf8c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.895 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdf8c067320 0x7fdf8c0697d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00a010 tx=0x7fdf9c019000 comp rx=0 tx=0).stop 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 msgr2=0x7fdfac157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 --2- 172.21.15.170:0/3887218261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdfac09af50 0x7fdfac157340 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdfac08ad70 tx=0x7fdfa4002210 comp rx=0 tx=0).stop 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfab59e700 1 -- 172.21.15.170:0/3887218261 reap_dead start 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 shutdown_connections 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.893+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 >> 172.21.15.170:0/3887218261 conn(0x7fdfac0999f0 msgr2=0x7fdfac0ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.894+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 shutdown_connections 2023-10-08T16:16:10.896 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:10.894+0000 7fdfb117c700 1 -- 172.21.15.170:0/3887218261 wait complete. 2023-10-08T16:16:10.906 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-08T16:16:11.021 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.021+0000 7f5e49022700 1 Processor -- start 2023-10-08T16:16:11.022 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.021+0000 7f5e49022700 1 -- start start 2023-10-08T16:16:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e49022700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e44159890 0x7f5e4415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e49022700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.024 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e49022700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e44069ee0 con 0x7f5e44159c60 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e49022700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4415a160 con 0x7f5e44159890 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e49022700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440d13a0 con 0x7f5e44069560 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e44159890 0x7f5e4415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5e44159890 0x7f5e4415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34308/0 (socket says 172.21.15.170:34308) 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 -- 172.21.15.170:0/1592059625 learned_addr learned my addr 172.21.15.170:0/1592059625 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.025 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e4359e700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 -- 172.21.15.170:0/1592059625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 msgr2=0x7f5e440699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 -- 172.21.15.170:0/1592059625 >> v1:172.21.15.191:6789/0 conn(0x7f5e44159c60 legacy=0x7f5e4415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 -- 172.21.15.170:0/1592059625 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4415e140 con 0x7f5e44159890 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e42d9d700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159890 0x7f5e4415abd0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009f20 tx=0x7f5e34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155b152848282ec4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.026 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e41d9b700 1 -- 172.21.15.170:0/1592059625 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400e070 con 0x7f5e44159890 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e41d9b700 1 -- 172.21.15.170:0/1592059625 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3400ba40 con 0x7f5e44159890 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.022+0000 7f5e41d9b700 1 -- 172.21.15.170:0/1592059625 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340092e0 con 0x7f5e44159890 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 -- 172.21.15.170:0/1592059625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159890 msgr2=0x7f5e4415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159890 0x7f5e4415abd0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009f20 tx=0x7f5e34009840 comp rx=0 tx=0).stop 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 -- 172.21.15.170:0/1592059625 shutdown_connections 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.027 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 --2- 172.21.15.170:0/1592059625 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159890 0x7f5e4415abd0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 -- 172.21.15.170:0/1592059625 >> 172.21.15.170:0/1592059625 conn(0x7f5e44068e20 msgr2=0x7f5e440d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 -- 172.21.15.170:0/1592059625 shutdown_connections 2023-10-08T16:16:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.023+0000 7f5e49022700 1 -- 172.21.15.170:0/1592059625 wait complete. 2023-10-08T16:16:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.024+0000 7f5e49022700 1 Processor -- start 2023-10-08T16:16:11.028 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.024+0000 7f5e49022700 1 -- start start 2023-10-08T16:16:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e44159890 0x7f5e440b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.031 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b4790 con 0x7f5e44159c60 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b3730 con 0x7f5e44159890 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e49022700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b5ca0 con 0x7f5e44069560 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34312/0 (socket says 172.21.15.170:34312) 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 -- 172.21.15.170:0/314012491 learned_addr learned my addr 172.21.15.170:0/314012491 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e42d9d700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.032 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4259c700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e44159890 0x7f5e440b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 -- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 msgr2=0x7f5e440b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 -- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e44159890 msgr2=0x7f5e440b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e44159890 0x7f5e440b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 -- 172.21.15.170:0/314012491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009ae0 con 0x7f5e44159c60 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.025+0000 7f5e4359e700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c880 tx=0x7f5e3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.033 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.026+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c011600 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.026+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c011760 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.026+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440b39d0 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.026+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c011b00 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.026+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e440b4010 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.027+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4404c3c0 con 0x7f5e44159c60 2023-10-08T16:16:11.034 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.027+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f5e3c018360 con 0x7f5e44159c60 2023-10-08T16:16:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.027+0000 7f5e2affd700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 0x7f5e300697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.027+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f5e3c084820 con 0x7f5e44159c60 2023-10-08T16:16:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.027+0000 7f5e42d9d700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 0x7f5e300697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.035 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.028+0000 7f5e42d9d700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 0x7f5e300697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009880 tx=0x7f5e34017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.036 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.032+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e3c055130 con 0x7f5e44159c60 2023-10-08T16:16:11.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.215+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e4406a5b0 con 0x7f5e30067320 2023-10-08T16:16:11.215 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.215+0000 7f5e2affd700 1 -- 172.21.15.170:0/314012491 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5e4406a5b0 con 0x7f5e30067320 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 msgr2=0x7f5e300697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 0x7f5e300697d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009880 tx=0x7f5e34017000 comp rx=0 tx=0).stop 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 msgr2=0x7f5e440b5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c880 tx=0x7f5e3c008040 comp rx=0 tx=0).stop 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 shutdown_connections 2023-10-08T16:16:11.218 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f5e30067320 0x7f5e300697d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e44159c60 0x7f5e440b5760 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f5e44069560 0x7f5e440b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 --2- 172.21.15.170:0/314012491 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f5e44159890 0x7f5e440b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 >> 172.21.15.170:0/314012491 conn(0x7f5e44068e20 msgr2=0x7f5e440b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 shutdown_connections 2023-10-08T16:16:11.219 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.217+0000 7f5e49022700 1 -- 172.21.15.170:0/314012491 wait complete. 2023-10-08T16:16:11.220 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 5 to deep-scrub 2023-10-08T16:16:11.229 INFO:tasks.ceph:Scrubbing osd.6 2023-10-08T16:16:11.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:11.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.348+0000 7f926f04e700 1 Processor -- start 2023-10-08T16:16:11.349 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.348+0000 7f926f04e700 1 -- start start 2023-10-08T16:16:11.351 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926f04e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9268159ec0 0x7f92680b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926f04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f92680cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926f04e700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92680b5b00 con 0x7f92680b53d0 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926f04e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92680b5850 con 0x7f9268159ec0 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926f04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92680b5da0 con 0x7f92680cf350 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926cdea700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9268159ec0 0x7f92680b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926cdea700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f9268159ec0 0x7f92680b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34328/0 (socket says 172.21.15.170:34328) 2023-10-08T16:16:11.352 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.349+0000 7f926cdea700 1 -- 172.21.15.170:0/1997669341 learned_addr learned my addr 172.21.15.170:0/1997669341 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926d5eb700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f92680cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926cdea700 1 -- 172.21.15.170:0/1997669341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 msgr2=0x7f92680cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926cdea700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f92680cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926cdea700 1 -- 172.21.15.170:0/1997669341 >> v1:172.21.15.191:6789/0 conn(0x7f92680b53d0 legacy=0x7f92680cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926cdea700 1 -- 172.21.15.170:0/1997669341 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92680b6080 con 0x7f9268159ec0 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f92677fe700 1 -- 172.21.15.170:0/1997669341 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409926913 0 0) 0x7f92680b5b00 con 0x7f92680b53d0 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f926cdea700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9268159ec0 0x7f92680b4e90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32b4995713121d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.353 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f92677fe700 1 -- 172.21.15.170:0/1997669341 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258009e80 con 0x7f9268159ec0 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f92677fe700 1 -- 172.21.15.170:0/1997669341 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9258005c10 con 0x7f9268159ec0 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.350+0000 7f92677fe700 1 -- 172.21.15.170:0/1997669341 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800b650 con 0x7f9268159ec0 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 -- 172.21.15.170:0/1997669341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9268159ec0 msgr2=0x7f92680b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9268159ec0 0x7f92680b4e90 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258009220 comp rx=0 tx=0).stop 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 -- 172.21.15.170:0/1997669341 shutdown_connections 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f92680cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.354 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 --2- 172.21.15.170:0/1997669341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9268159ec0 0x7f92680b4e90 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 -- 172.21.15.170:0/1997669341 >> 172.21.15.170:0/1997669341 conn(0x7f92680999f0 msgr2=0x7f92680b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 -- 172.21.15.170:0/1997669341 shutdown_connections 2023-10-08T16:16:11.355 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.351+0000 7f926f04e700 1 -- 172.21.15.170:0/1997669341 wait complete. 2023-10-08T16:16:11.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.352+0000 7f926f04e700 1 Processor -- start 2023-10-08T16:16:11.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.352+0000 7f926f04e700 1 -- start start 2023-10-08T16:16:11.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 0x7f9268068da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.358 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f9268150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9268159ec0 0x7f926806a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9268150760 con 0x7f92680b53d0 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9268069380 con 0x7f9268159ec0 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9268069660 con 0x7f92680cf350 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 0x7f9268068da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 0x7f9268068da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34330/0 (socket says 172.21.15.170:34330) 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 -- 172.21.15.170:0/3318654094 learned_addr learned my addr 172.21.15.170:0/3318654094 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.359 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 -- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 msgr2=0x7f9268150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f92680cf350 0x7f9268150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 -- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9268159ec0 msgr2=0x7f926806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f9268159ec0 0x7f926806a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9258009480 con 0x7f92680b53d0 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926cdea700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 0x7f9268068da0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9268089390 tx=0x7f9258016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925801dcc0 con 0x7f92680b53d0 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925801de40 con 0x7f92680b53d0 2023-10-08T16:16:11.360 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268069940 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.353+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925801b6d0 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.354+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926806ae20 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.354+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9268063c90 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f925801ba40 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9248067320 0x7f92480697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f9258086970 con 0x7f92680b53d0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9267fff700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9248067320 0x7f92480697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.361 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f924806d170 0x7f924806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f924806fc50 con 0x7f924806d170 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.355+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925808b220 con 0x7f92680b53d0 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.356+0000 7f926d5eb700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f924806d170 0x7f924806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.356+0000 7f9267fff700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9248067320 0x7f92480697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f92680c6970 tx=0x7f9250007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.356+0000 7f926d5eb700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f924806d170 0x7f924806f580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.362 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.357+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== osd.6 v2:172.21.15.191:6808/91067 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f924806fc50 con 0x7f924806d170 2023-10-08T16:16:11.370 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.370+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 --> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f92680718b0 con 0x7f924806d170 2023-10-08T16:16:11.371 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f9264ff9700 1 -- 172.21.15.170:0/3318654094 <== osd.6 v2:172.21.15.191:6808/91067 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f92680718b0 con 0x7f924806d170 2023-10-08T16:16:11.371 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:11.371 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:11.371 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:11.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f924806d170 msgr2=0x7f924806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.372 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.191:6808/91067,v1:172.21.15.191:6809/91067] conn(0x7f924806d170 0x7f924806f580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9248067320 msgr2=0x7f92480697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f9248067320 0x7f92480697d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f92680c6970 tx=0x7f9250007ec0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 msgr2=0x7f9268068da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 --2- 172.21.15.170:0/3318654094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f92680b53d0 0x7f9268068da0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9268089390 tx=0x7f9258016000 comp rx=0 tx=0).stop 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926d5eb700 1 -- 172.21.15.170:0/3318654094 reap_dead start 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 shutdown_connections 2023-10-08T16:16:11.373 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 >> 172.21.15.170:0/3318654094 conn(0x7f92680999f0 msgr2=0x7f9268151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 shutdown_connections 2023-10-08T16:16:11.374 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.371+0000 7f926f04e700 1 -- 172.21.15.170:0/3318654094 wait complete. 2023-10-08T16:16:11.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-08T16:16:11.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.499+0000 7f2e006d0700 1 Processor -- start 2023-10-08T16:16:11.499 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.499+0000 7f2e006d0700 1 -- start start 2023-10-08T16:16:11.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2df8159890 0x7f2df815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159c60 0x7f2df815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 0x7f2df8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df815a160 con 0x7f2df8159890 2023-10-08T16:16:11.502 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df80d13a0 con 0x7f2df815db10 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2e006d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df815e140 con 0x7f2df8159c60 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2df8159890 0x7f2df815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2df8159890 0x7f2df815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34342/0 (socket says 172.21.15.170:34342) 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 -- 172.21.15.170:0/433483444 learned_addr learned my addr 172.21.15.170:0/433483444 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfdc6b700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159c60 0x7f2df815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfec6d700 1 --2- 172.21.15.170:0/433483444 >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 0x7f2df8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.503 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 -- 172.21.15.170:0/433483444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159c60 msgr2=0x7f2df815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159c60 0x7f2df815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 -- 172.21.15.170:0/433483444 >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 msgr2=0x7f2df8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/433483444 >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 0x7f2df8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 -- 172.21.15.170:0/433483444 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df815a3d0 con 0x7f2df8159890 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfec6d700 1 --2- 172.21.15.170:0/433483444 >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 0x7f2df8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:11.504 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df8159890 0x7f2df815abd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f2df8089b50 tx=0x7f2df4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f81891664c524d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfd46a700 1 -- 172.21.15.170:0/433483444 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400b860 con 0x7f2df8159890 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfd46a700 1 -- 172.21.15.170:0/433483444 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df400b9c0 con 0x7f2df8159890 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.500+0000 7f2dfd46a700 1 -- 172.21.15.170:0/433483444 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df400bd60 con 0x7f2df8159890 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 -- 172.21.15.170:0/433483444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df8159890 msgr2=0x7f2df815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df8159890 0x7f2df815abd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f2df8089b50 tx=0x7f2df4009250 comp rx=0 tx=0).stop 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 -- 172.21.15.170:0/433483444 shutdown_connections 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 --2- 172.21.15.170:0/433483444 >> v2:172.21.15.191:3300/0 conn(0x7f2df815db10 0x7f2df8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.505 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df8159890 0x7f2df815abd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 --2- 172.21.15.170:0/433483444 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159c60 0x7f2df815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 -- 172.21.15.170:0/433483444 >> 172.21.15.170:0/433483444 conn(0x7f2df8068e20 msgr2=0x7f2df80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 -- 172.21.15.170:0/433483444 shutdown_connections 2023-10-08T16:16:11.506 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.501+0000 7f2e006d0700 1 -- 172.21.15.170:0/433483444 wait complete. 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.502+0000 7f2e006d0700 1 Processor -- start 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.502+0000 7f2e006d0700 1 -- start start 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 0x7f2df8147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 0x7f2df80ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.509 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80b5570 con 0x7f2df815db10 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8152730 con 0x7f2df8159c60 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2e006d0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80b4960 con 0x7f2df8159890 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34358/0 (socket says 172.21.15.170:34358) 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 -- 172.21.15.170:0/1795727351 learned_addr learned my addr 172.21.15.170:0/1795727351 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 0x7f2df8147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.510 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfdc6b700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 0x7f2df80ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 -- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 msgr2=0x7f2df8147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 0x7f2df8147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 -- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 msgr2=0x7f2df80ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 0x7f2df80ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 -- 172.21.15.170:0/1795727351 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40094f0 con 0x7f2df815db10 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 0x7f2df8147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfdc6b700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 0x7f2df80ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:11.511 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dfec6d700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009fc0 tx=0x7f2df00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000dd40 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.503+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df000f040 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.504+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df80b4c40 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.504+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df0015690 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.504+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df80b5200 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.504+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df80b5720 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.505+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f2df001e3b0 con 0x7f2df815db10 2023-10-08T16:16:11.512 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.505+0000 7f2dee7fc700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 0x7f2ddc0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.505+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f2df0084fc0 con 0x7f2df815db10 2023-10-08T16:16:11.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.505+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 0x7f2ddc0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.506+0000 7f2dfe46c700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 0x7f2ddc0697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2df8089b50 tx=0x7f2df4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.513 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.509+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2df80b5720 con 0x7f2df815db10 2023-10-08T16:16:11.694 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.694+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f2df80cbfc0 con 0x7f2ddc067320 2023-10-08T16:16:11.695 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.694+0000 7f2dee7fc700 1 -- 172.21.15.170:0/1795727351 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2df80cbfc0 con 0x7f2ddc067320 2023-10-08T16:16:11.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.696+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 msgr2=0x7f2ddc0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.697 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.696+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 0x7f2ddc0697d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2df8089b50 tx=0x7f2df4017000 comp rx=0 tx=0).stop 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.696+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 msgr2=0x7f2df8169be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.696+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009fc0 tx=0x7f2df00099b0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 shutdown_connections 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f2ddc067320 0x7f2ddc0697d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2df815db10 0x7f2df8169be0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.698 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f2df8159890 0x7f2df8147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 --2- 172.21.15.170:0/1795727351 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f2df8159c60 0x7f2df80ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 >> 172.21.15.170:0/1795727351 conn(0x7f2df8068e20 msgr2=0x7f2df80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 shutdown_connections 2023-10-08T16:16:11.699 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.697+0000 7f2e006d0700 1 -- 172.21.15.170:0/1795727351 wait complete. 2023-10-08T16:16:11.699 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 6 to deep-scrub 2023-10-08T16:16:11.709 INFO:tasks.ceph:Scrubbing osd.7 2023-10-08T16:16:11.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-08T16:16:11.827 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.827+0000 7fb642534700 1 Processor -- start 2023-10-08T16:16:11.828 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.827+0000 7fb642534700 1 -- start start 2023-10-08T16:16:11.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb642534700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb63c0b55a0 0x7fb63c164400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb642534700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0cfa60 con 0x7fb63c0cd910 2023-10-08T16:16:11.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb642534700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0b5b00 con 0x7fb63c159ec0 2023-10-08T16:16:11.830 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb642534700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0b5da0 con 0x7fb63c0b55a0 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb640ad1700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb63c0b55a0 0x7fb63c164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb63b7fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fb63c0cd910 0x7fb63c0ce340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43448/0 (socket says 172.21.15.170:43448) 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb63b7fe700 1 -- 172.21.15.170:0/1515993685 learned_addr learned my addr 172.21.15.170:0/1515993685 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb640ad1700 1 -- 172.21.15.170:0/1515993685 >> v1:172.21.15.191:6789/0 conn(0x7fb63c159ec0 legacy=0x7fb63c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb640ad1700 1 -- 172.21.15.170:0/1515993685 >> v1:172.21.15.170:6789/0 conn(0x7fb63c0cd910 legacy=0x7fb63c0ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb640ad1700 1 -- 172.21.15.170:0/1515993685 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0b6080 con 0x7fb63c0b55a0 2023-10-08T16:16:11.831 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.828+0000 7fb63affd700 1 -- 172.21.15.170:0/1515993685 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930042438 0 0) 0x7fb63c0cfa60 con 0x7fb63c0cd910 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb640ad1700 1 --2- 172.21.15.170:0/1515993685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c0b55a0 0x7fb63c164400 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb634009e90 tx=0x7fb634009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2887a43fa042a5ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb63affd700 1 -- 172.21.15.170:0/1515993685 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400e070 con 0x7fb63c0b55a0 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb63affd700 1 -- 172.21.15.170:0/1515993685 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63400bbb0 con 0x7fb63c0b55a0 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb63affd700 1 -- 172.21.15.170:0/1515993685 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634012430 con 0x7fb63c0b55a0 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 -- 172.21.15.170:0/1515993685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c0b55a0 msgr2=0x7fb63c164400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 --2- 172.21.15.170:0/1515993685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c0b55a0 0x7fb63c164400 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb634009e90 tx=0x7fb634009780 comp rx=0 tx=0).stop 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 -- 172.21.15.170:0/1515993685 shutdown_connections 2023-10-08T16:16:11.832 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 --2- 172.21.15.170:0/1515993685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c0b55a0 0x7fb63c164400 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 -- 172.21.15.170:0/1515993685 >> 172.21.15.170:0/1515993685 conn(0x7fb63c0999f0 msgr2=0x7fb63c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 -- 172.21.15.170:0/1515993685 shutdown_connections 2023-10-08T16:16:11.833 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.829+0000 7fb642534700 1 -- 172.21.15.170:0/1515993685 wait complete. 2023-10-08T16:16:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.830+0000 7fb642534700 1 Processor -- start 2023-10-08T16:16:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 -- start start 2023-10-08T16:16:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb63c0b55a0 0x7fb63c147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.836 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 0x7fb63c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb63c0cf200 0x7fb63c06a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0cf670 con 0x7fb63c159ec0 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c06a010 con 0x7fb63c0cf200 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb642534700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c069400 con 0x7fb63c0b55a0 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 0x7fb63c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb63b7fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 0x7fb63c0cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34376/0 (socket says 172.21.15.170:34376) 2023-10-08T16:16:11.837 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.831+0000 7fb63b7fe700 1 -- 172.21.15.170:0/2719635101 learned_addr learned my addr 172.21.15.170:0/2719635101 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb640ad1700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb63c0cf200 0x7fb63c06a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63bfff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb63c0b55a0 0x7fb63c147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 -- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb63c0b55a0 msgr2=0x7fb63c147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb63c0b55a0 0x7fb63c147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 -- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb63c0cf200 msgr2=0x7fb63c06a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fb63c0cf200 0x7fb63c06a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.838 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634009a20 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63bfff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fb63c0b55a0 0x7fb63c147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb63b7fe700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 0x7fb63c0cecc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c810 tx=0x7fb630007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630005c00 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630005d60 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c0696e0 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300072e0 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.832+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c069ca0 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.833+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb63c063c90 con 0x7fb63c159ec0 2023-10-08T16:16:11.839 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.833+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fb630007690 con 0x7fb63c159ec0 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb623fff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628067320 0x7fb6280697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fb630083d70 con 0x7fb63c159ec0 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb63bfff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628067320 0x7fb6280697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb623fff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7fb62806d170 0x7fb62806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb62806fc50 con 0x7fb62806d170 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=206) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6300862d0 con 0x7fb63c159ec0 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb63bfff700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628067320 0x7fb6280697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb62c006fd0 tx=0x7fb62c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.840 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.834+0000 7fb640ad1700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7fb62806d170 0x7fb62806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.835+0000 7fb640ad1700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7fb62806d170 0x7fb62806f580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.841 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.835+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== osd.7 v2:172.21.15.191:6816/91097 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fb62806fc50 con 0x7fb62806d170 2023-10-08T16:16:11.849 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.849+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 --> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb63c0718b0 con 0x7fb62806d170 2023-10-08T16:16:11.850 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.849+0000 7fb623fff700 1 -- 172.21.15.170:0/2719635101 <== osd.7 v2:172.21.15.191:6816/91097 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb63c0718b0 con 0x7fb62806d170 2023-10-08T16:16:11.850 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2023-10-08T16:16:11.850 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T16:16:11.850 INFO:teuthology.orchestra.run.smithi170.stdout:} 2023-10-08T16:16:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7fb62806d170 msgr2=0x7fb62806f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.851 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.191:6816/91097,v1:172.21.15.191:6817/91097] conn(0x7fb62806d170 0x7fb62806f580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628067320 msgr2=0x7fb6280697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fb628067320 0x7fb6280697d0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb62c006fd0 tx=0x7fb62c008040 comp rx=0 tx=0).stop 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 msgr2=0x7fb63c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 --2- 172.21.15.170:0/2719635101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb63c159ec0 0x7fb63c0cecc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c810 tx=0x7fb630007db0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb640ad1700 1 -- 172.21.15.170:0/2719635101 reap_dead start 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 shutdown_connections 2023-10-08T16:16:11.852 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 >> 172.21.15.170:0/2719635101 conn(0x7fb63c0999f0 msgr2=0x7fb63c0c6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 shutdown_connections 2023-10-08T16:16:11.853 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.850+0000 7fb642534700 1 -- 172.21.15.170:0/2719635101 wait complete. 2023-10-08T16:16:11.862 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-08T16:16:11.978 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.978+0000 7f1db046d700 1 Processor -- start 2023-10-08T16:16:11.979 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.978+0000 7f1db046d700 1 -- start start 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1db046d700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f1da8159890 0x7f1da815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1db046d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1da80699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1db046d700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da8069ee0 con 0x7f1da8159c60 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1db046d700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da815a160 con 0x7f1da8159890 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1db046d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80d13a0 con 0x7f1da8069560 2023-10-08T16:16:11.983 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1dada08700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1da8159c60 0x7f1da815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43462/0 (socket says 172.21.15.170:43462) 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.979+0000 7f1dada08700 1 -- 172.21.15.170:0/486439840 learned_addr learned my addr 172.21.15.170:0/486439840 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1daea0a700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1da80699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 --2- 172.21.15.170:0/486439840 >> v2:172.21.15.191:3300/0 conn(0x7f1da8159890 0x7f1da815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219936582 0 0) 0x7f1da8069ee0 con 0x7f1da8159c60 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d94003570 con 0x7f1da8159c60 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001536146 0 0) 0x7f1d94003570 con 0x7f1da8159c60 2023-10-08T16:16:11.984 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 msgr2=0x7f1da80699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1da80699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 >> v2:172.21.15.191:3300/0 conn(0x7f1da8159890 msgr2=0x7f1da815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 --2- 172.21.15.170:0/486439840 >> v2:172.21.15.191:3300/0 conn(0x7f1da8159890 0x7f1da815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da815e140 con 0x7f1da8159c60 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f1d98001470 con 0x7f1da8159c60 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 >> v1:172.21.15.170:6789/0 conn(0x7f1da8159c60 legacy=0x7f1da815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1d94003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.985 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d94004810 0x7f1d94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1d940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da815a160 con 0x7f1d94003a60 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80d13a0 con 0x7f1d94004810 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9400ba60 con 0x7f1da8069560 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1d94003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dada08700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d94004810 0x7f1d94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.986 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1daea0a700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1d940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 -- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 msgr2=0x7f1d940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1d940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 -- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d94004810 msgr2=0x7f1d94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1d94004810 0x7f1d94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1dae209700 1 -- 172.21.15.170:0/486439840 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d94003570 con 0x7f1d94003a60 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.980+0000 7f1daea0a700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8069560 0x7f1d940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:11.987 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1dae209700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1d94003e70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1da400e240 tx=0x7f1da400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89202197156f2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400eaf0 con 0x7f1d94003a60 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da400f040 con 0x7f1d94003a60 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1dad207700 1 -- 172.21.15.170:0/486439840 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da400eea0 con 0x7f1d94003a60 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 -- 172.21.15.170:0/486439840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 msgr2=0x7f1d94003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 --2- 172.21.15.170:0/486439840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1d94003e70 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f1da400e240 tx=0x7f1da400b040 comp rx=0 tx=0).stop 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1daea0a700 1 -- 172.21.15.170:0/486439840 reap_dead start 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 -- 172.21.15.170:0/486439840 shutdown_connections 2023-10-08T16:16:11.988 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 -- 172.21.15.170:0/486439840 >> 172.21.15.170:0/486439840 conn(0x7f1da8068e20 msgr2=0x7f1da80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:11.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 -- 172.21.15.170:0/486439840 shutdown_connections 2023-10-08T16:16:11.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.981+0000 7f1db046d700 1 -- 172.21.15.170:0/486439840 wait complete. 2023-10-08T16:16:11.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.982+0000 7f1db046d700 1 Processor -- start 2023-10-08T16:16:11.989 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.982+0000 7f1db046d700 1 -- start start 2023-10-08T16:16:11.991 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 0x7f1da816d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8151530 0x7f1da81519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80b3fc0 con 0x7f1d94003a60 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da8152e70 con 0x7f1da816d3a0 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1db046d700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da816dd50 con 0x7f1da8151530 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.992 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34396/0 (socket says 172.21.15.170:34396) 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 -- 172.21.15.170:0/1069593919 learned_addr learned my addr 172.21.15.170:0/1069593919 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dada08700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 0x7f1da816d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 -- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8151530 msgr2=0x7f1da81519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8151530 0x7f1da81519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 -- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 msgr2=0x7f1da816d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:11.993 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 0x7f1da816d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dae209700 1 -- 172.21.15.170:0/1069593919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da8152260 con 0x7f1d94003a60 2023-10-08T16:16:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.983+0000 7f1dada08700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 0x7f1da816d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T16:16:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1dae209700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004700 tx=0x7f1da401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.994 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da401d420 con 0x7f1d94003a60 2023-10-08T16:16:11.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da401d5a0 con 0x7f1d94003a60 2023-10-08T16:16:11.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da401d960 con 0x7f1d94003a60 2023-10-08T16:16:11.995 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da8152540 con 0x7f1d94003a60 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.984+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da8152b00 con 0x7f1d94003a60 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.985+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da804c3c0 con 0x7f1d94003a60 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.985+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f1da4003040 con 0x7f1d94003a60 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.986+0000 7f1d9e7fc700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 0x7f1d8c069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.986+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f1da4086a40 con 0x7f1d94003a60 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.986+0000 7f1dada08700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 0x7f1d8c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.986+0000 7f1dada08700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 0x7f1d8c069850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001620 tx=0x7f1d98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:11.996 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:11.990+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1da4053880 con 0x7f1d94003a60 2023-10-08T16:16:12.171 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.171+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f1da806a5b0 con 0x7f1d8c0673a0 2023-10-08T16:16:12.172 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.171+0000 7f1d9e7fc700 1 -- 172.21.15.170:0/1069593919 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1da806a5b0 con 0x7f1d8c0673a0 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 msgr2=0x7f1d8c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 0x7f1d8c069850 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d98001620 tx=0x7f1d98009040 comp rx=0 tx=0).stop 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 msgr2=0x7f1da814b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f1d94004700 tx=0x7f1da401a000 comp rx=0 tx=0).stop 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 shutdown_connections 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f1da8151530 0x7f1da81519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.174 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f1d8c0673a0 0x7f1d8c069850 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d94003a60 0x7f1da814b540 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 --2- 172.21.15.170:0/1069593919 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f1da816d3a0 0x7f1da816d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 >> 172.21.15.170:0/1069593919 conn(0x7f1da8068e20 msgr2=0x7f1da815ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 shutdown_connections 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.173+0000 7f1db046d700 1 -- 172.21.15.170:0/1069593919 wait complete. 2023-10-08T16:16:12.175 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 7 to deep-scrub 2023-10-08T16:16:12.186 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:16:12.300 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.300+0000 7f0d3feed700 1 Processor -- start 2023-10-08T16:16:12.301 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.300+0000 7f0d3feed700 1 -- start start 2023-10-08T16:16:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3feed700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0d38069560 0x7f0d380699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3feed700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d38069ee0 con 0x7f0d38159890 2023-10-08T16:16:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3feed700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3815a160 con 0x7f0d38159c60 2023-10-08T16:16:12.303 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3feed700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380d13a0 con 0x7f0d38069560 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3e48a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0d38069560 0x7f0d380699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3e48a700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0d38069560 0x7f0d380699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34412/0 (socket says 172.21.15.170:34412) 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3dc89700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0d38159890 0x7f0d3815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43468/0 (socket says 172.21.15.170:43468) 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.301+0000 7f0d3e48a700 1 -- 172.21.15.170:0/2310045979 learned_addr learned my addr 172.21.15.170:0/2310045979 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3e48a700 1 -- 172.21.15.170:0/2310045979 >> v1:172.21.15.191:6789/0 conn(0x7f0d38159c60 legacy=0x7f0d3815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3e48a700 1 -- 172.21.15.170:0/2310045979 >> v1:172.21.15.170:6789/0 conn(0x7f0d38159890 legacy=0x7f0d3815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3e48a700 1 -- 172.21.15.170:0/2310045979 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3815e140 con 0x7f0d38069560 2023-10-08T16:16:12.304 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3cc87700 1 -- 172.21.15.170:0/2310045979 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641825905 0 0) 0x7f0d38069ee0 con 0x7f0d38159890 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3e48a700 1 --2- 172.21.15.170:0/2310045979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380699a0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009f20 tx=0x7f0d30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aaa220144ca8e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3cc87700 1 -- 172.21.15.170:0/2310045979 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000e070 con 0x7f0d38069560 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3cc87700 1 -- 172.21.15.170:0/2310045979 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3000ba40 con 0x7f0d38069560 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3cc87700 1 -- 172.21.15.170:0/2310045979 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300092e0 con 0x7f0d38069560 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 -- 172.21.15.170:0/2310045979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 msgr2=0x7f0d380699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 --2- 172.21.15.170:0/2310045979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380699a0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009f20 tx=0x7f0d30009840 comp rx=0 tx=0).stop 2023-10-08T16:16:12.305 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 -- 172.21.15.170:0/2310045979 shutdown_connections 2023-10-08T16:16:12.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 --2- 172.21.15.170:0/2310045979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380699a0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 -- 172.21.15.170:0/2310045979 >> 172.21.15.170:0/2310045979 conn(0x7f0d38068e20 msgr2=0x7f0d380d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:12.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 -- 172.21.15.170:0/2310045979 shutdown_connections 2023-10-08T16:16:12.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.302+0000 7f0d3feed700 1 -- 172.21.15.170:0/2310045979 wait complete. 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.303+0000 7f0d3feed700 1 Processor -- start 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 -- start start 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 0x7f0d38154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d38159c60 0x7f0d380ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:12.309 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380c78a0 con 0x7f0d38069560 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380c8420 con 0x7f0d38159890 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3feed700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380c7af0 con 0x7f0d38159c60 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3dc89700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3dc89700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:34424/0 (socket says 172.21.15.170:34424) 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3dc89700 1 -- 172.21.15.170:0/4157479845 learned_addr learned my addr 172.21.15.170:0/4157479845 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3d488700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 0x7f0d38154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:12.310 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.304+0000 7f0d3e48a700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d38159c60 0x7f0d380ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 -- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d38159c60 msgr2=0x7f0d380ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d38159c60 0x7f0d380ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 -- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 msgr2=0x7f0d38154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 0x7f0d38154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 -- 172.21.15.170:0/4157479845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30009ae0 con 0x7f0d38069560 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3d488700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 0x7f0d38154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:12.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3dc89700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c7e0 tx=0x7f0d34007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34007300 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d34007460 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d380c7dd0 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34004830 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.305+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d380c8160 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.306+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3804c3c0 con 0x7f0d38069560 2023-10-08T16:16:12.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.306+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f0d34004be0 con 0x7f0d38069560 2023-10-08T16:16:12.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.307+0000 7f0d2dffb700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 0x7f0d1c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:12.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.307+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f0d34083e90 con 0x7f0d38069560 2023-10-08T16:16:12.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.307+0000 7f0d3d488700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 0x7f0d1c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:12.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.307+0000 7f0d3d488700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 0x7f0d1c0697d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006fd0 tx=0x7f0d28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:12.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.311+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d34050ea0 con 0x7f0d38069560 2023-10-08T16:16:12.490 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.490+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d380c8560 con 0x7f0d1c067320 2023-10-08T16:16:12.492 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.492+0000 7f0d2dffb700 1 -- 172.21.15.170:0/4157479845 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50586 (secure 0 0 0) 0x7f0d2800d630 con 0x7f0d1c067320 2023-10-08T16:16:12.493 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 msgr2=0x7f0d1c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 0x7f0d1c0697d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0d28006fd0 tx=0x7f0d28008040 comp rx=0 tx=0).stop 2023-10-08T16:16:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 msgr2=0x7f0d380b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:12.495 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f0d3400c7e0 tx=0x7f0d34007e10 comp rx=0 tx=0).stop 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 shutdown_connections 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f0d38159c60 0x7f0d380ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f0d1c067320 0x7f0d1c0697d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d38069560 0x7f0d380b3590 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 --2- 172.21.15.170:0/4157479845 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f0d38159890 0x7f0d38154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 >> 172.21.15.170:0/4157479845 conn(0x7f0d38068e20 msgr2=0x7f0d380d0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.494+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 shutdown_connections 2023-10-08T16:16:12.496 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:12.495+0000 7f0d3feed700 1 -- 172.21.15.170:0/4157479845 wait complete. 2023-10-08T16:16:12.497 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:16:12.506 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":551,"stamp":"2023-10-08T16:16:12.370578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449128,"kb_used_data":21672,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752525592,"statfs":{"total":773094113280,"available":770586206208,"internally_reserved":0,"allocated":22192128,"data_stored":10623008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.487682"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":321,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T16:16:03.453059+0000","last_change":"2023-10-08T16:04:32.254252+0000","last_active":"2023-10-08T16:16:03.453059+0000","last_peered":"2023-10-08T16:16:03.453059+0000","last_clean":"2023-10-08T16:16:03.453059+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:16:03.453059+0000","last_undegraded":"2023-10-08T16:16:03.453059+0000","last_fullsized":"2023-10-08T16:16:03.453059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T16:04:30.790840+0000","last_clean_scrub_stamp":"2023-10-08T16:04:30.790840+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-10T03:14:19.806929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1770652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.80200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.32500000000000001,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":8.0939999999999994},"last":0.38400000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.29199999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.083},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.175},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.69499999999999995},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.745,"5min":0.79100000000000004,"15min":8.4339999999999993},"last":0.41299999999999998}]},{"osd":3,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73599999999999999,"5min":1.677,"15min":1.677},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.752,"5min":0.77200000000000002,"15min":8.3409999999999993},"last":0.5}]},{"osd":4,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.55800000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.497,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.312}]},{"osd":6,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.442},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.32700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.90600000000000003},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.76700000000000002},"last":0.60699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.751,"15min":0.751},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.752,"15min":0.96599999999999997},"last":0.56200000000000006}]},{"osd":3,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.45600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.94199999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.55400000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.80700000000000005},"last":0.44},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.89200000000000002},"last":0.51600000000000001}]},{"osd":5,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.81899999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.874},"last":0.58299999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.79600000000000004,"15min":0.93999999999999995},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.85599999999999998},"last":0.59299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.315},"min":{"1min":0.26300000000000001,"5min":0.19,"15min":0.16900000000000001},"max":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":1.2150000000000001},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.376,"15min":0.34899999999999998},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.77500000000000002},"last":0.29699999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.35299999999999998},"min":{"1min":0.30399999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":1.2230000000000001},"last":0.315},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.26100000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.38900000000000001},"min":{"1min":0.36799999999999999,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.66400000000000003,"15min":0.70399999999999996},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.72699999999999998},"last":0.32600000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":1.077},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.90000000000000002,"15min":1.696},"last":0.52000000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.35099999999999998},"max":{"1min":0.78600000000000003,"5min":0.85299999999999998,"15min":1.657},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"last":0.52800000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.51300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88600000000000001,"15min":1.7310000000000001},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.628},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.875,"15min":1.1839999999999999},"last":0.68100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308624,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063216,"statfs":{"total":96636764160,"available":96320733184,"internally_reserved":0,"allocated":3219456,"data_stored":1770652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":5.5599999999999996},"last":0.318},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.28999999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":3.3220000000000001},"last":0.26700000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.45900000000000002},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":5.6769999999999996},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.34599999999999997,"5min":0.26300000000000001,"15min":0.219},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.42299999999999999},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.188},"max":{"1min":0.56999999999999995,"5min":1.0880000000000001,"15min":3.2160000000000002},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28199999999999997,"15min":0.26900000000000002},"max":{"1min":0.51500000000000001,"5min":0.67100000000000004,"15min":5.5970000000000004},"last":0.35499999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72099999999999997,"5min":1.1379999999999999,"15min":3.2519999999999998},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.73699999999999999,"5min":1.1140000000000001,"15min":3.2890000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.33400000000000002},"max":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.498}]},{"osd":6,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":5.6239999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.85699999999999998},"last":0.59499999999999997}]},{"osd":7,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.60299999999999998,"5min":0.41699999999999998,"15min":0.313},"max":{"1min":0.753,"5min":0.873,"15min":0.88800000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":3.5049999999999999},"last":0.48199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.499,"5min":0.624,"15min":0.63100000000000001},"last":0.313},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.34699999999999998},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.189},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.64300000000000002},"last":0.24199999999999999}]},{"osd":1,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.58499999999999996},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36799999999999999},"min":{"1min":0.309,"5min":0.215,"15min":0.20499999999999999},"max":{"1min":0.56399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.32800000000000001}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.221},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.61299999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34599999999999997}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.313,"15min":0.254},"max":{"1min":0.871,"5min":0.871,"15min":0.89100000000000001},"last":0.439}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.5,"5min":0.379,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.375,"15min":0.30299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.1029999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63800000000000001,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.52600000000000002,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.501}]}]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.238},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.36599999999999999},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.9470000000000001},"last":0.29099999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":1.0269999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.31900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.311}]},{"osd":2,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.35299999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":1.1399999999999999},"last":0.373},{"interface":"front","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.36199999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":0.58899999999999997,"5min":0.68999999999999995,"15min":1.327},"last":0.42399999999999999}]},{"osd":4,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.316,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.9119999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":1.02,"15min":1.099},"last":0.60999999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.27},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.879},"last":0.64100000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.379,"15min":0.372},"max":{"1min":0.82599999999999996,"5min":0.92400000000000004,"15min":1.0620000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":1.03,"15min":1.2969999999999999},"last":0.64800000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0049999999999999,"15min":1.825},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.51500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":1.115},"last":0.57199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.438,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.579},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38900000000000001}]},{"osd":1,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.33300000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60799999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":1.5629999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.628,"15min":0.58699999999999997},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.66300000000000003}]},{"osd":3,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.626}]},{"osd":5,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54900000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.441}]},{"osd":6,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.52400000000000002}]},{"osd":7,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.81399999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53800000000000003,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.25,"15min":0.247},"max":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":4.0119999999999996},"last":0.434}]},{"osd":1,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.33900000000000002,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.84299999999999997,"15min":12.227},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.747,"5min":0.871,"15min":1.252},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.46600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":3.9510000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.42499999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":9.3170000000000002},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.879,"15min":3.8719999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":0.85999999999999999,"15min":0.93799999999999994},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69999999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34100000000000003}]},{"osd":6,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.5}]},{"osd":7,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.442,"5min":0.312,"15min":0.312},"max":{"1min":0.66800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.45600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:16:12.510 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-08T16:04:30.790840+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=16, tm_min=4, tm_sec=30, tm_wday=6, tm_yday=281, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=16, tm_min=16, tm_sec=7, tm_wday=6, tm_yday=281, tm_isdst=0) 2023-10-08T16:16:12.510 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-08T16:16:15.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:15.043 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:32.511 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T16:16:33.118 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.118+0000 7fdd5b7c0700 1 Processor -- start 2023-10-08T16:16:33.119 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.118+0000 7fdd5b7c0700 1 -- start start 2023-10-08T16:16:33.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5b7c0700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5b7c0700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd54069ee0 con 0x7fdd54159890 2023-10-08T16:16:33.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5b7c0700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5415a160 con 0x7fdd54159c60 2023-10-08T16:16:33.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5b7c0700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd540d13a0 con 0x7fdd54069560 2023-10-08T16:16:33.121 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5955c700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdd54159890 0x7fdd5415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48456/0 (socket says 172.21.15.170:48456) 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd5955c700 1 -- 172.21.15.170:0/2160573293 learned_addr learned my addr 172.21.15.170:0/2160573293 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd59d5d700 1 --2- 172.21.15.170:0/2160573293 >> v2:172.21.15.191:3300/0 conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81623190 0 0) 0x7fdd54069ee0 con 0x7fdd54159890 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40003570 con 0x7fdd54159890 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1291051911 0 0) 0x7fdd5415a160 con 0x7fdd54159c60 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.119+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 --> v1:172.21.15.191:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54069ee0 con 0x7fdd54159c60 2023-10-08T16:16:33.122 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd59d5d700 1 -- 172.21.15.170:0/2160573293 >> v1:172.21.15.191:6789/0 conn(0x7fdd54159c60 legacy=0x7fdd5415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd59d5d700 1 -- 172.21.15.170:0/2160573293 >> v1:172.21.15.170:6789/0 conn(0x7fdd54159890 legacy=0x7fdd5415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd59d5d700 1 -- 172.21.15.170:0/2160573293 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5415e140 con 0x7fdd54069560 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd59d5d700 1 --2- 172.21.15.170:0/2160573293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54069560 0x7fdd540699a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009e60 tx=0x7fdd4c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135875575b9e074a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00e070 con 0x7fdd54069560 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00bab0 con 0x7fdd54069560 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd4bfff700 1 -- 172.21.15.170:0/2160573293 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00be30 con 0x7fdd54069560 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/2160573293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54069560 msgr2=0x7fdd540699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.123 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/2160573293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54069560 0x7fdd540699a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009e60 tx=0x7fdd4c009780 comp rx=0 tx=0).stop 2023-10-08T16:16:33.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/2160573293 shutdown_connections 2023-10-08T16:16:33.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/2160573293 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54069560 0x7fdd540699a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.120+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/2160573293 >> 172.21.15.170:0/2160573293 conn(0x7fdd54068e20 msgr2=0x7fdd540d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:33.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.121+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/2160573293 shutdown_connections 2023-10-08T16:16:33.124 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.121+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/2160573293 wait complete. 2023-10-08T16:16:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.122+0000 7fdd5b7c0700 1 Processor -- start 2023-10-08T16:16:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.122+0000 7fdd5b7c0700 1 -- start start 2023-10-08T16:16:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.122+0000 7fdd5b7c0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.122+0000 7fdd5b7c0700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54159890 0x7fdd540cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.127 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.122+0000 7fdd5b7c0700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd54159c60 0x7fdd54146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5b7c0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540b3590 con 0x7fdd54069560 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5b7c0700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540ce720 con 0x7fdd54159890 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5b7c0700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540cea00 con 0x7fdd54159c60 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:33542/0 (socket says 172.21.15.170:33542) 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 -- 172.21.15.170:0/1298743185 learned_addr learned my addr 172.21.15.170:0/1298743185 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:33.128 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd59d5d700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd54159c60 0x7fdd54146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd58d5b700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54159890 0x7fdd540cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 -- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd54159c60 msgr2=0x7fdd54146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd54159c60 0x7fdd54146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 -- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54159890 msgr2=0x7fdd540cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54159890 0x7fdd540cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 -- 172.21.15.170:0/1298743185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009a20 con 0x7fdd54069560 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5955c700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c790 tx=0x7fdd50007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.129 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50005b30 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50005c90 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540cece0 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50007300 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.123+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd54146680 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.124+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5404c3c0 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.125+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7fdd5001b400 con 0x7fdd54069560 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.125+0000 7fdd497fa700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 0x7fdd380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.130 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.125+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7fdd50084420 con 0x7fdd54069560 2023-10-08T16:16:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.125+0000 7fdd58d5b700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 0x7fdd380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.126+0000 7fdd58d5b700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 0x7fdd380697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007fd0 tx=0x7fdd44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.131 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.130+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd50054ff0 con 0x7fdd54069560 2023-10-08T16:16:33.306 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.306+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 --> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd540b2880 con 0x7fdd38067320 2023-10-08T16:16:33.308 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.308+0000 7fdd497fa700 1 -- 172.21.15.170:0/1298743185 <== mgr.4104 v2:172.21.15.170:6832/90636 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50652 (secure 0 0 0) 0x7fdd540b2880 con 0x7fdd38067320 2023-10-08T16:16:33.309 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:33.311 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.310+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 msgr2=0x7fdd380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.310+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 0x7fdd380697d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdd44007fd0 tx=0x7fdd44009040 comp rx=0 tx=0).stop 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.310+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 msgr2=0x7fdd540ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.310+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c790 tx=0x7fdd50007ea0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 shutdown_connections 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7fdd54159c60 0x7fdd54146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7fdd38067320 0x7fdd380697d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.312 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd54069560 0x7fdd540ce090 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 --2- 172.21.15.170:0/1298743185 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7fdd54159890 0x7fdd540cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 >> 172.21.15.170:0/1298743185 conn(0x7fdd54068e20 msgr2=0x7fdd54155280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:33.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 shutdown_connections 2023-10-08T16:16:33.313 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.311+0000 7fdd5b7c0700 1 -- 172.21.15.170:0/1298743185 wait complete. 2023-10-08T16:16:33.313 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2023-10-08T16:16:33.323 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":561,"stamp":"2023-10-08T16:16:32.374892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2449128,"kb_used_data":21672,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752525592,"statfs":{"total":773094113280,"available":770586206208,"internally_reserved":0,"allocated":22192128,"data_stored":10623008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002702"},"pg_stats":[{"pgid":"1.0","version":"206'119","reported_seq":334,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-08T16:16:12.183312+0000","last_change":"2023-10-08T16:16:12.183271+0000","last_active":"2023-10-08T16:16:12.183312+0000","last_peered":"2023-10-08T16:16:12.183312+0000","last_clean":"2023-10-08T16:16:12.183312+0000","last_became_active":"2023-10-08T16:04:32.253122+0000","last_became_peered":"2023-10-08T16:04:32.253122+0000","last_unstale":"2023-10-08T16:16:12.183312+0000","last_undegraded":"2023-10-08T16:16:12.183312+0000","last_fullsized":"2023-10-08T16:16:12.183312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"206'119","last_scrub_stamp":"2023-10-08T16:16:12.183142+0000","last_deep_scrub":"206'119","last_deep_scrub_stamp":"2023-10-08T16:16:12.183142+0000","last_clean_scrub_stamp":"2023-10-08T16:16:12.183142+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T17:36:09.712021+0000","scrub_duration":0.0044721739999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1770652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54300000000000004,"15min":0.52600000000000002},"min":{"1min":0.438,"5min":0.32900000000000001,"15min":0.29299999999999998},"max":{"1min":0.80200000000000005,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.65100000000000002},"min":{"1min":0.44,"5min":0.32500000000000001,"15min":0.30499999999999999},"max":{"1min":0.72499999999999998,"5min":0.72899999999999998,"15min":8.0939999999999994},"last":0.55600000000000005}]},{"osd":1,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.40000000000000002,"5min":0.32500000000000001,"15min":0.29199999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.083},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.67300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":8.2040000000000006},"last":0.40200000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.442,"5min":0.34300000000000003,"15min":0.33700000000000002},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":1.175},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.69499999999999995},"min":{"1min":0.40000000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.745,"5min":0.79100000000000004,"15min":8.4339999999999993},"last":0.62}]},{"osd":3,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.53900000000000003,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73599999999999999,"5min":1.677,"15min":1.677},"last":0.439},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.69699999999999995},"min":{"1min":0.40200000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.752,"5min":0.77200000000000002,"15min":8.3409999999999993},"last":0.58399999999999996}]},{"osd":4,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45200000000000001,"15min":0.47099999999999997},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.55800000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53300000000000003,"15min":0.54200000000000004},"min":{"1min":0.39600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.66900000000000004,"5min":0.753,"15min":0.753},"last":0.56999999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.497,"15min":0.51100000000000001},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.59499999999999997,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.505,"15min":0.53500000000000003},"min":{"1min":0.33800000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.48499999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:15:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.39200000000000002,"5min":0.35699999999999998,"15min":0.309},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.84399999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.32300000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.60899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.53800000000000003,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.70599999999999996,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.41599999999999998,"5min":0.33900000000000002,"15min":0.29299999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.90600000000000003},"last":0.314}]},{"osd":1,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.53300000000000003,"15min":0.51000000000000001},"min":{"1min":0.34399999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70999999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38500000000000001,"5min":0.34100000000000003,"15min":0.27300000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.76700000000000002},"last":0.40100000000000002}]},{"osd":2,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.54500000000000004,"15min":0.54000000000000004},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.33100000000000002},"max":{"1min":0.73899999999999999,"5min":0.751,"15min":0.751},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.48799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.69899999999999995,"5min":0.752,"15min":0.96599999999999997},"last":0.495}]},{"osd":3,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.45600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.72799999999999998,"5min":0.75800000000000001,"15min":0.94199999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54500000000000004,"15min":0.53900000000000003},"min":{"1min":0.41799999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41199999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.26200000000000001},"max":{"1min":0.68000000000000005,"5min":0.72499999999999998,"15min":0.80700000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.39400000000000002,"5min":0.314,"15min":0.28899999999999998},"max":{"1min":0.63600000000000001,"5min":0.754,"15min":0.89200000000000002},"last":0.441}]},{"osd":5,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.497,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.60499999999999998,"5min":0.72299999999999998,"15min":0.81899999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.55400000000000005,"15min":0.56999999999999995},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.311},"max":{"1min":0.67700000000000005,"5min":0.82399999999999995,"15min":0.874},"last":0.53400000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:15:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.373,"5min":0.33400000000000002,"15min":0.32700000000000001},"max":{"1min":0.69899999999999995,"5min":0.79600000000000004,"15min":0.93999999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":0.73199999999999998,"5min":0.82899999999999996,"15min":0.85599999999999998},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.32900000000000001,"15min":0.315},"min":{"1min":0.26300000000000001,"5min":0.19,"15min":0.16900000000000001},"max":{"1min":0.48799999999999999,"5min":0.55900000000000005,"15min":1.2150000000000001},"last":0.31},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.376,"15min":0.34899999999999998},"min":{"1min":0.28799999999999998,"5min":0.22500000000000001,"15min":0.20300000000000001},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.77500000000000002},"last":0.38200000000000001}]},{"osd":2,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36499999999999999,"15min":0.35299999999999998},"min":{"1min":0.30399999999999999,"5min":0.23499999999999999,"15min":0.23000000000000001},"max":{"1min":0.52000000000000002,"5min":0.64400000000000002,"15min":1.2230000000000001},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.377},"min":{"1min":0.28100000000000003,"5min":0.23499999999999999,"15min":0.19700000000000001},"max":{"1min":0.56200000000000006,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.33300000000000002}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41299999999999998,"15min":0.38900000000000001},"min":{"1min":0.36799999999999999,"5min":0.23200000000000001,"15min":0.23000000000000001},"max":{"1min":0.57199999999999995,"5min":0.66400000000000003,"15min":0.70399999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.35699999999999998},"min":{"1min":0.317,"5min":0.245,"15min":0.215},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.72699999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.54000000000000004},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.28599999999999998},"max":{"1min":0.76200000000000001,"5min":0.80500000000000005,"15min":1.077},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.436,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.90000000000000002,"15min":1.696},"last":0.57099999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.35099999999999998},"max":{"1min":0.78600000000000003,"5min":0.85299999999999998,"15min":1.657},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.82899999999999996},"last":0.67800000000000005}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58999999999999997,"15min":0.57499999999999996},"min":{"1min":0.51300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.79800000000000004,"5min":0.84999999999999998,"15min":1.099},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.61399999999999999},"min":{"1min":0.51100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.74399999999999999,"5min":0.88600000000000001,"15min":1.7310000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.48399999999999999,"5min":0.377,"15min":0.32800000000000001},"max":{"1min":0.77500000000000002,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61499999999999999,"15min":0.59599999999999997},"min":{"1min":0.51400000000000001,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.875,"15min":1.1839999999999999},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308624,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063216,"statfs":{"total":96636764160,"available":96320733184,"internally_reserved":0,"allocated":3219456,"data_stored":1770652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.432},"min":{"1min":0.26200000000000001,"5min":0.189,"15min":0.17299999999999999},"max":{"1min":0.49299999999999999,"5min":0.56200000000000006,"15min":5.5599999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.35499999999999998,"15min":0.40699999999999997},"min":{"1min":0.28999999999999998,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":3.3220000000000001},"last":0.42799999999999999}]},{"osd":2,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.35799999999999998,"15min":0.45900000000000002},"min":{"1min":0.28499999999999998,"5min":0.25,"15min":0.22600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58399999999999996,"15min":5.6769999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.34599999999999997,"5min":0.26300000000000001,"15min":0.219},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.497}]},{"osd":3,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.42299999999999999},"min":{"1min":0.29599999999999999,"5min":0.255,"15min":0.188},"max":{"1min":0.56999999999999995,"5min":1.0880000000000001,"15min":3.2160000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.40100000000000002,"15min":0.499},"min":{"1min":0.35099999999999998,"5min":0.28199999999999997,"15min":0.26900000000000002},"max":{"1min":0.51500000000000001,"5min":0.67100000000000004,"15min":5.5970000000000004},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58099999999999996,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.33500000000000002,"15min":0.32700000000000001},"max":{"1min":0.76700000000000002,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.44500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72099999999999997,"5min":1.1379999999999999,"15min":3.2519999999999998},"last":0.69799999999999995}]},{"osd":5,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58299999999999996,"15min":0.61699999999999999},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.73699999999999999,"5min":1.1140000000000001,"15min":3.2890000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.59599999999999997,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.36599999999999999,"15min":0.33400000000000002},"max":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":5.6500000000000004},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.70999999999999996},"min":{"1min":0.48399999999999999,"5min":0.39300000000000002,"15min":0.376},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":5.6239999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.625,"15min":0.60199999999999998},"min":{"1min":0.51700000000000002,"5min":0.39700000000000002,"15min":0.38800000000000001},"max":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.85699999999999998},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 16:15:55 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.61099999999999999},"min":{"1min":0.60299999999999998,"5min":0.41699999999999998,"15min":0.313},"max":{"1min":0.753,"5min":0.873,"15min":0.88800000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61799999999999999,"15min":0.67400000000000004},"min":{"1min":0.51000000000000001,"5min":0.41199999999999998,"15min":0.377},"max":{"1min":0.82999999999999996,"5min":1.1699999999999999,"15min":3.5049999999999999},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.316},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.188},"max":{"1min":0.499,"5min":0.624,"15min":0.63100000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.34699999999999998},"min":{"1min":0.316,"5min":0.24199999999999999,"15min":0.189},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.64300000000000002},"last":0.28599999999999998}]},{"osd":1,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36299999999999999,"15min":0.35199999999999998},"min":{"1min":0.27900000000000003,"5min":0.23000000000000001,"15min":0.222},"max":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.58499999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36799999999999999},"min":{"1min":0.309,"5min":0.215,"15min":0.20499999999999999},"max":{"1min":0.56399999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.35899999999999999,"15min":0.34799999999999998},"min":{"1min":0.31900000000000001,"5min":0.23699999999999999,"15min":0.221},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.40999999999999998,"15min":0.39300000000000002},"min":{"1min":0.35799999999999998,"5min":0.28599999999999998,"15min":0.26800000000000002},"max":{"1min":0.61299999999999999,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.38400000000000001}]},{"osd":4,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.57299999999999995,"15min":0.55100000000000005},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33300000000000002},"max":{"1min":0.81499999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59199999999999997,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.313,"15min":0.254},"max":{"1min":0.871,"5min":0.871,"15min":0.89100000000000001},"last":0.73499999999999999}]},{"osd":5,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.5,"5min":0.379,"15min":0.315},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.59399999999999997,"15min":0.628},"min":{"1min":0.45200000000000001,"5min":0.375,"15min":0.30299999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":4.1029999999999998},"last":0.72299999999999998}]},{"osd":6,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.63200000000000001,"15min":0.59899999999999998},"min":{"1min":0.51900000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86899999999999999,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63800000000000001,"15min":0.60499999999999998},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70999999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:15:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.52600000000000002,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.94299999999999995,"15min":0.96899999999999997},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.50700000000000001,"5min":0.44600000000000001,"15min":0.36299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.748}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":308044,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063796,"statfs":{"total":96636764160,"available":96321327104,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33900000000000002},"min":{"1min":0.248,"5min":0.183,"15min":0.183},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":1.238},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.34999999999999998,"15min":0.36599999999999999},"min":{"1min":0.28799999999999998,"5min":0.217,"15min":0.21299999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.9470000000000001},"last":0.317}]},{"osd":1,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.35299999999999998,"15min":0.35099999999999998},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.21199999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":1.0269999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.379,"15min":0.38900000000000001},"min":{"1min":0.31900000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":1.3520000000000001},"last":0.379}]},{"osd":2,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.35299999999999998,"5min":0.26400000000000001,"15min":0.248},"max":{"1min":0.56699999999999995,"5min":0.59399999999999997,"15min":1.1399999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.36199999999999999,"5min":0.27900000000000003,"15min":0.25800000000000001},"max":{"1min":0.58899999999999997,"5min":0.68999999999999995,"15min":1.327},"last":0.437}]},{"osd":4,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.441,"5min":0.316,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.88,"15min":1.9119999999999999},"last":0.871},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82199999999999995,"5min":1.02,"15min":1.099},"last":0.65400000000000003}]},{"osd":5,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59099999999999997,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.27},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.46700000000000003,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.879},"last":0.84199999999999997}]},{"osd":6,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.60299999999999998},"min":{"1min":0.495,"5min":0.379,"15min":0.372},"max":{"1min":0.82599999999999996,"5min":0.92400000000000004,"15min":1.0620000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84799999999999998,"5min":1.03,"15min":1.2969999999999999},"last":0.82399999999999995}]},{"osd":7,"last update":"Sun Oct 8 16:15:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35999999999999999},"max":{"1min":0.87,"5min":1.0049999999999999,"15min":1.825},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.624},"min":{"1min":0.51500000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.90400000000000003,"15min":1.115},"last":0.85899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.438,"5min":0.316,"15min":0.28199999999999997},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.579},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.46200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.74299999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.78500000000000003}]},{"osd":1,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.53900000000000003,"15min":0.50900000000000001},"min":{"1min":0.47499999999999998,"5min":0.309,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.38800000000000001,"5min":0.33300000000000002,"15min":0.33000000000000002},"max":{"1min":0.754,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.65900000000000003}]},{"osd":2,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.45000000000000001,"5min":0.34999999999999998,"15min":0.254},"max":{"1min":0.80600000000000005,"5min":0.83099999999999996,"15min":1.5629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.628,"15min":0.58699999999999997},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.80700000000000005}]},{"osd":3,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40799999999999997,"5min":0.34200000000000003,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57499999999999996,"15min":0.56799999999999995},"min":{"1min":0.46300000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.50900000000000001},"min":{"1min":0.35099999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53200000000000003,"15min":0.54900000000000004},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80500000000000005},"last":0.63600000000000001}]},{"osd":6,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.70299999999999996}]},{"osd":7,"last update":"Sun Oct 8 16:15:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.38800000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.71699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303948,"kb_used_data":2564,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067892,"statfs":{"total":96636764160,"available":96325521408,"internally_reserved":0,"allocated":2625536,"data_stored":1180284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53700000000000003,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.76300000000000001,"5min":0.82499999999999996,"15min":3.7770000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53800000000000003,"15min":0.58699999999999997},"min":{"1min":0.378,"5min":0.25,"15min":0.247},"max":{"1min":0.77500000000000002,"5min":0.80100000000000005,"15min":4.0119999999999996},"last":0.69699999999999995}]},{"osd":1,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54000000000000004,"15min":0.77000000000000002},"min":{"1min":0.47999999999999998,"5min":0.33900000000000002,"15min":0.29699999999999999},"max":{"1min":0.73599999999999999,"5min":0.84299999999999997,"15min":12.227},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.40999999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.747,"5min":0.871,"15min":1.252},"last":0.72699999999999998}]},{"osd":2,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.60699999999999998},"min":{"1min":0.46600000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.70799999999999996,"5min":0.75700000000000001,"15min":3.9510000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54800000000000004,"15min":0.53600000000000003},"min":{"1min":0.41399999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.76500000000000001,"5min":0.88700000000000001,"15min":1.444},"last":0.755}]},{"osd":3,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55700000000000005,"15min":0.75700000000000001},"min":{"1min":0.39200000000000002,"5min":0.34899999999999998,"15min":0.312},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":9.3170000000000002},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59199999999999997,"15min":0.64300000000000002},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.34999999999999998},"max":{"1min":0.73899999999999999,"5min":0.879,"15min":3.8719999999999999},"last":0.80900000000000005}]},{"osd":4,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.48599999999999999,"15min":0.49199999999999999},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":0.85999999999999999,"15min":0.93799999999999994},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69999999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.58099999999999996}]},{"osd":6,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"min":{"1min":0.377,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.60099999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Oct 8 16:15:52 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.442,"5min":0.312,"15min":0.312},"max":{"1min":0.66800000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.59399999999999997},"min":{"1min":0.434,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.84699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T16:16:33.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-08T16:16:33.439 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.439+0000 7f485aefb700 1 Processor -- start 2023-10-08T16:16:33.637 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.439+0000 7f485aefb700 1 -- start start 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485aefb700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4854159a60 0x7f48540c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485aefb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 0x7f48540cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485aefb700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48540c9cb0 con 0x7f48540ccc00 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485aefb700 1 -- --> v2:172.21.15.191:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48540cbbd0 con 0x7f4854159a60 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485aefb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485406dfc0 con 0x7f48540cb230 2023-10-08T16:16:33.643 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485a6fa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 0x7f48540cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f4859ef9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4854159a60 0x7f48540c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f4859ef9700 1 --2- >> v2:172.21.15.191:3300/0 conn(0x7f4854159a60 0x7f48540c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3300/0 says I am v2:172.21.15.170:55336/0 (socket says 172.21.15.170:55336) 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f4859ef9700 1 -- 172.21.15.170:0/527613673 learned_addr learned my addr 172.21.15.170:0/527613673 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f485a6fa700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 0x7f48540cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.191:3301/0 says I am v2:172.21.15.170:59936/0 (socket says 172.21.15.170:59936) 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.440+0000 7f48596f8700 1 --1- 172.21.15.170:0/527613673 >> v1:172.21.15.191:6789/0 conn(0x7f48540ccc00 0x7f4854069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.191:6789/0 says I am v1:172.21.15.170:51318/0 (socket says 172.21.15.170:51318) 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4859ef9700 1 -- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 msgr2=0x7f48540cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4859ef9700 1 --2- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 0x7f48540cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.644 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4859ef9700 1 -- 172.21.15.170:0/527613673 >> v1:172.21.15.191:6789/0 conn(0x7f48540ccc00 legacy=0x7f4854069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4859ef9700 1 -- 172.21.15.170:0/527613673 --> v2:172.21.15.191:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540c9f30 con 0x7f4854159a60 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4858ef7700 1 -- 172.21.15.170:0/527613673 <== mon.1 v1:172.21.15.191:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853092847 0 0) 0x7f48540c9cb0 con 0x7f48540ccc00 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4859ef9700 1 --2- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f48540c9770 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48500099e0 tx=0x7f4850009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4035a1358100426d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4858ef7700 1 -- 172.21.15.170:0/527613673 <== mon.1 v2:172.21.15.191:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000b8b0 con 0x7f4854159a60 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4858ef7700 1 -- 172.21.15.170:0/527613673 <== mon.1 v2:172.21.15.191:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485000ba10 con 0x7f4854159a60 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f4858ef7700 1 -- 172.21.15.170:0/527613673 <== mon.1 v2:172.21.15.191:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485000bdb0 con 0x7f4854159a60 2023-10-08T16:16:33.645 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 -- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 msgr2=0x7f48540c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 --2- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f48540c9770 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48500099e0 tx=0x7f4850009250 comp rx=0 tx=0).stop 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 -- 172.21.15.170:0/527613673 shutdown_connections 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 --2- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540cb230 0x7f48540cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 --2- 172.21.15.170:0/527613673 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f48540c9770 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.441+0000 7f485aefb700 1 -- 172.21.15.170:0/527613673 >> 172.21.15.170:0/527613673 conn(0x7f485406baa0 msgr2=0x7f485406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.442+0000 7f485aefb700 1 -- 172.21.15.170:0/527613673 shutdown_connections 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.442+0000 7f485aefb700 1 -- 172.21.15.170:0/527613673 wait complete. 2023-10-08T16:16:33.646 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 Processor -- start 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 -- start start 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540ccc00 0x7f485416d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f485416d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c73c0 con 0x7f48540cb230 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c67b0 con 0x7f4854159a60 2023-10-08T16:16:33.647 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f485aefb700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48540c6a90 con 0x7f48540ccc00 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.443+0000 7f4859ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:33544/0 (socket says 172.21.15.170:33544) 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 -- 172.21.15.170:0/3265414888 learned_addr learned my addr 172.21.15.170:0/3265414888 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f48596f8700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540ccc00 0x7f485416d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f485a6fa700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f485416d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 -- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540ccc00 msgr2=0x7f485416d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540ccc00 0x7f485416d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.648 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 -- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 msgr2=0x7f485416d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f485416d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 -- 172.21.15.170:0/3265414888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500094f0 con 0x7f48540cb230 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4859ef9700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f485406cdc0 tx=0x7f4850017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850011d30 con 0x7f48540cb230 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850011e90 con 0x7f48540cb230 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850024690 con 0x7f48540cb230 2023-10-08T16:16:33.649 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48540c6d70 con 0x7f48540cb230 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.444+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48540a9a60 con 0x7f48540cb230 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.445+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540ca240 con 0x7f48540cb230 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.446+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f485001a420 con 0x7f48540cb230 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.446+0000 7f4849ffb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 0x7f48380697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.446+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f4850085d70 con 0x7f48540cb230 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.446+0000 7f48596f8700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 0x7f48380697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.446+0000 7f48596f8700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 0x7f48380697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f48540c90d0 tx=0x7f4844007d50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.650 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.450+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4850052d00 con 0x7f48540cb230 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.634+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f48540ca470 con 0x7f48540cb230 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.638+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4850052d00 con 0x7f48540cb230 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.638+0000 7f4849ffb700 1 -- 172.21.15.170:0/3265414888 <== mon.0 v2:172.21.15.170:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v34)=0 v34) v1 ==== 123+0+0 (secure 0 0 0) 0x7f4850056320 con 0x7f48540cb230 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 msgr2=0x7f48380697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 0x7f48380697d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f48540c90d0 tx=0x7f4844007d50 comp rx=0 tx=0).stop 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 msgr2=0x7f48540c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f485406cdc0 tx=0x7f4850017000 comp rx=0 tx=0).stop 2023-10-08T16:16:33.651 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 shutdown_connections 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f4854159a60 0x7f485416d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f4838067320 0x7f48380697d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48540cb230 0x7f48540c5d60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 --2- 172.21.15.170:0/3265414888 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f48540ccc00 0x7f485416d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 >> 172.21.15.170:0/3265414888 conn(0x7f485406baa0 msgr2=0x7f48540ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 shutdown_connections 2023-10-08T16:16:33.652 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.641+0000 7f485aefb700 1 -- 172.21.15.170:0/3265414888 wait complete. 2023-10-08T16:16:33.654 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T16:16:33.654 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T16:16:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.768+0000 7f639662a700 1 Processor -- start 2023-10-08T16:16:33.768 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.768+0000 7f639662a700 1 -- start start 2023-10-08T16:16:33.771 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f639662a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f639015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f639662a700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63900b2980 con 0x7f63901597f0 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f639662a700 1 -- --> v1:172.21.15.191:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639015dbb0 con 0x7f6390159420 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f639662a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63900b11d0 con 0x7f639015d230 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6394e27700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f63901597f0 0x7f63900b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:48468/0 (socket says 172.21.15.170:48468) 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6394e27700 1 -- 172.21.15.170:0/1683804158 learned_addr learned my addr 172.21.15.170:0/1683804158 (peer_addr_for_me v1:172.21.15.170:0/0) 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516931843 0 0) 0x7f63900b2980 con 0x7f63901597f0 2023-10-08T16:16:33.772 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6395e29700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f639015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f637c003570 con 0x7f63901597f0 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2303229171 0 0) 0x7f637c003570 con 0x7f63901597f0 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 msgr2=0x7f639015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f639015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 >> v1:172.21.15.191:6789/0 conn(0x7f6390159420 legacy=0x7f63900b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900b14b0 con 0x7f63901597f0 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1185676764 0 0) 0x7f6380001470 con 0x7f63901597f0 2023-10-08T16:16:33.773 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 >> v1:172.21.15.170:6789/0 conn(0x7f63901597f0 legacy=0x7f63900b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f637c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f637c004010 0x7f637c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c004d50 0x7f637c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900b11d0 con 0x7f637c004010 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c00bbf0 con 0x7f637c004d50 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.769+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c00bdb0 con 0x7f639015d230 2023-10-08T16:16:33.774 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f637c004010 0x7f637c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6395628700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f637c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6395e29700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c004d50 0x7f637c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 -- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 msgr2=0x7f637c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f639015d230 0x7f637c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 -- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c004d50 msgr2=0x7f637c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c004d50 0x7f637c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 -- 172.21.15.170:0/1683804158 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c003570 con 0x7f637c004010 2023-10-08T16:16:33.775 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6395e29700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f637c004d50 0x7f637c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6394e27700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f637c004010 0x7f637c004810 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6390159c70 tx=0x7f6380001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b1b8b498e68629 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000cc30 con 0x7f637c004010 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f638000cd90 con 0x7f637c004010 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.770+0000 7f6387fff700 1 -- 172.21.15.170:0/1683804158 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380005b70 con 0x7f637c004010 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 -- 172.21.15.170:0/1683804158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f637c004010 msgr2=0x7f637c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.776 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 --2- 172.21.15.170:0/1683804158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f637c004010 0x7f637c004810 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f6390159c70 tx=0x7f6380001e50 comp rx=0 tx=0).stop 2023-10-08T16:16:33.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f6395e29700 1 -- 172.21.15.170:0/1683804158 reap_dead start 2023-10-08T16:16:33.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 -- 172.21.15.170:0/1683804158 shutdown_connections 2023-10-08T16:16:33.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 -- 172.21.15.170:0/1683804158 >> 172.21.15.170:0/1683804158 conn(0x7f63900d0660 msgr2=0x7f63900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:33.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 -- 172.21.15.170:0/1683804158 shutdown_connections 2023-10-08T16:16:33.777 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.771+0000 7f639662a700 1 -- 172.21.15.170:0/1683804158 wait complete. 2023-10-08T16:16:33.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.772+0000 7f639662a700 1 Processor -- start 2023-10-08T16:16:33.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.772+0000 7f639662a700 1 -- start start 2023-10-08T16:16:33.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 --2- >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 0x7f6390155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.780 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 --2- >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 0x7f63900cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900d0130 con 0x7f63900c8e10 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 -- --> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63900ca640 con 0x7f63900cf750 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 -- --> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6390155b40 con 0x7f637c004010 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:33562/0 (socket says 172.21.15.170:33562) 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 -- 172.21.15.170:0/1149438946 learned_addr learned my addr 172.21.15.170:0/1149438946 (peer_addr_for_me v2:172.21.15.170:0/0) 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6395628700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 0x7f6390155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6395e29700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 0x7f63900cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.781 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 -- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 msgr2=0x7f6390155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 0x7f6390155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 -- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 msgr2=0x7f63900cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 0x7f63900cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 -- 172.21.15.170:0/1149438946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900c5c30 con 0x7f63900c8e10 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6395e29700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 0x7f63900cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6395628700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 0x7f6390155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T16:16:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f6394e27700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f638000c950 tx=0x7f6380001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000cc30 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6380004e70 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63900c5f10 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380001b90 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.773+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63900c64d0 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.774+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639004c3c0 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.775+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 12) v1 ==== 85097+0+0 (secure 0 0 0) 0x7f63800074f0 con 0x7f63900c8e10 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.775+0000 7f63857fa700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 0x7f63740697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T16:16:33.783 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.775+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 4939+0+0 (secure 0 0 0) 0x7f6380085d30 con 0x7f63900c8e10 2023-10-08T16:16:33.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.775+0000 7f6395628700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 0x7f63740697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T16:16:33.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.775+0000 7f6395628700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 0x7f63740697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T16:16:33.784 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:33.779+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6380052d40 con 0x7f63900c8e10 2023-10-08T16:16:34.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.004+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63900b1790 con 0x7f63900c8e10 2023-10-08T16:16:34.005 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.005+0000 7f63857fa700 1 -- 172.21.15.170:0/1149438946 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+3543 (secure 0 0 0) 0x7f6380017020 con 0x7f63900c8e10 2023-10-08T16:16:34.005 INFO:teuthology.orchestra.run.smithi170.stdout: 2023-10-08T16:16:34.006 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":134,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7082,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6835/3037384933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6833","nonce":3037384933},{"type":"v1","addr":"172.21.15.191:6835","nonce":3037384933}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":7993,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.191:6834/1851360340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6832","nonce":1851360340},{"type":"v1","addr":"172.21.15.191:6834","nonce":1851360340}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6836/1731515056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":1731515056},{"type":"v1","addr":"172.21.15.170:6836","nonce":1731515056}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":126},{"gid":9288,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/111938257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6835","nonce":111938257},{"type":"v1","addr":"172.21.15.170:6837","nonce":111938257}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":134}],"filesystems":[]} 2023-10-08T16:16:34.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 msgr2=0x7f63740697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:34.007 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 0x7f63740697d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f638c000c00 tx=0x7f638c009040 comp rx=0 tx=0).stop 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 msgr2=0x7f63900c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f638000c950 tx=0x7f6380001e50 comp rx=0 tx=0).stop 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 shutdown_connections 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] conn(0x7f63900cf750 0x7f63900cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.191:3301/0,v1:172.21.15.191:6790/0] conn(0x7f637c004010 0x7f6390155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:6832/90636,v1:172.21.15.170:6833/90636] conn(0x7f6374067320 0x7f63740697d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:34.008 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 --2- 172.21.15.170:0/1149438946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63900c8e10 0x7f63900c5400 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T16:16:34.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 >> 172.21.15.170:0/1149438946 conn(0x7f63900d0660 msgr2=0x7f63900ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T16:16:34.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 shutdown_connections 2023-10-08T16:16:34.009 INFO:teuthology.orchestra.run.smithi170.stderr:2023-10-08T16:16:34.007+0000 7f639662a700 1 -- 172.21.15.170:0/1149438946 wait complete. 2023-10-08T16:16:34.009 INFO:teuthology.orchestra.run.smithi170.stderr:dumped fsmap epoch 134 2023-10-08T16:16:34.019 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-08T16:16:34.019 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-08T16:16:34.019 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T16:16:34.020 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.027 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T16:16:34.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T16:16:34.028 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.076 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T16:16:34.077 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T16:16:34.077 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.088 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T16:16:34.088 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T16:16:34.088 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.139 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T16:16:34.139 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-08T16:16:34.140 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-08T16:16:34.140 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.178 INFO:tasks.ceph.osd.0:Stopped 2023-10-08T16:16:34.179 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-08T16:16:34.179 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.243 INFO:tasks.ceph.osd.1:Stopped 2023-10-08T16:16:34.243 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-08T16:16:34.243 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.315 INFO:tasks.ceph.osd.2:Stopped 2023-10-08T16:16:34.316 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-08T16:16:34.316 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.383 INFO:tasks.ceph.osd.3:Stopped 2023-10-08T16:16:34.384 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-08T16:16:34.384 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.418 INFO:tasks.ceph.osd.4:Stopped 2023-10-08T16:16:34.419 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-08T16:16:34.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.491 INFO:tasks.ceph.osd.5:Stopped 2023-10-08T16:16:34.491 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-08T16:16:34.492 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.567 INFO:tasks.ceph.osd.6:Stopped 2023-10-08T16:16:34.567 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-08T16:16:34.567 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.637 INFO:tasks.ceph.osd.7:Stopped 2023-10-08T16:16:34.637 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-08T16:16:34.638 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T16:16:34.638 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.664 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T16:16:34.664 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T16:16:34.665 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.703 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T16:16:34.703 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-08T16:16:34.703 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-08T16:16:34.703 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.715 INFO:tasks.ceph.mon.a:Stopped 2023-10-08T16:16:34.716 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-08T16:16:34.716 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.754 INFO:tasks.ceph.mon.b:Stopped 2023-10-08T16:16:34.754 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-08T16:16:34.755 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T16:16:34.806 INFO:tasks.ceph.mon.c:Stopped 2023-10-08T16:16:34.806 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-08T16:16:34.806 DEBUG:teuthology.orchestra.run.smithi170:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-10-08T16:16:34.835 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:16:34.836 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-08T16:16:35.085 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:16:35.085 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-08T16:16:35.170 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:16:35.170 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-08T16:16:35.240 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:16:35.240 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-08T16:16:35.309 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:16:35.309 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-08T16:16:35.486 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:16:35.486 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-08T16:16:35.577 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:16:35.577 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-08T16:16:35.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:16:35.647 DEBUG:teuthology.orchestra.run.smithi191:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-08T16:16:35.722 INFO:tasks.ceph:Archiving mon data... 2023-10-08T16:16:35.723 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/data/mon.a.tgz 2023-10-08T16:16:35.724 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2023-10-08T16:16:35.739 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.cHsd5uPxh0 2023-10-08T16:16:35.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cHsd5uPxh0 2023-10-08T16:16:35.911 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.cHsd5uPxh0 2023-10-08T16:16:35.992 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.cHsd5uPxh0 is 571KB 2023-10-08T16:16:36.092 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.cHsd5uPxh0 2023-10-08T16:16:36.106 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/data/mon.b.tgz 2023-10-08T16:16:36.107 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-08T16:16:36.123 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.mmfpzG5Miz 2023-10-08T16:16:36.123 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mmfpzG5Miz 2023-10-08T16:16:36.309 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.mmfpzG5Miz 2023-10-08T16:16:36.390 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.mmfpzG5Miz is 579KB 2023-10-08T16:16:36.489 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.mmfpzG5Miz 2023-10-08T16:16:36.503 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/data/mon.c.tgz 2023-10-08T16:16:36.503 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2023-10-08T16:16:36.557 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.D5xB0jicka 2023-10-08T16:16:36.557 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.D5xB0jicka 2023-10-08T16:16:36.720 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.D5xB0jicka 2023-10-08T16:16:36.801 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.D5xB0jicka is 580KB 2023-10-08T16:16:36.898 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.D5xB0jicka 2023-10-08T16:16:36.912 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-08T16:16:36.912 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T16:16:36.914 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T16:16:36.976 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-08T16:16:36.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T16:16:36.979 DEBUG:teuthology.orchestra.run.smithi191:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T16:16:37.002 INFO:teuthology.orchestra.run.smithi170.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T16:16:37.042 INFO:teuthology.orchestra.run.smithi191.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T16:16:37.048 INFO:tasks.ceph:Archiving crash dumps... 2023-10-08T16:16:37.049 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi170/crash 2023-10-08T16:16:37.050 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T16:16:37.075 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi191/crash 2023-10-08T16:16:37.076 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T16:16:37.111 INFO:tasks.ceph:Shutting down logrotate 2023-10-08T16:16:37.112 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:37.119 DEBUG:teuthology.orchestra.run.smithi191:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:37.221 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:37.245 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T16:16:37.268 INFO:tasks.ceph:Compressing logs... 2023-10-08T16:16:37.269 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T16:16:37.287 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T16:16:37.357 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99087.log 2023-10-08T16:16:37.357 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.97931.log 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98691.log 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99087.log.gz 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr:gzip /var/log/ceph/ceph-client.3.97931.log: -5 --verbose -- /var/log/ceph/ceph-client.1.101585.log 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.97931.log.gz 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.98691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90674.log 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98691.log.gz 2023-10-08T16:16:37.358 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90840.log 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101585.log.gz 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90714.log 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90840.log: gzip 88.3%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.90674.log.gz-- 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.log 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90886.log 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.90840.log.gz 2023-10-08T16:16:37.359 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90714.log.gz 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101173.log 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.90886.log.gz 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101944.log 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102335.log 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101173.log.gz 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101944.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101944.log.gz 2023-10-08T16:16:37.360 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.102335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97569.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102335.log.gz 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.97569.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97569.log.gz 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90927.log 2023-10-08T16:16:37.361 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.90927.log: 92.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90927.log.gz 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102331.log 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100772.log 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.102331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99976.log 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102331.log.gz 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.100772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101948.log 2023-10-08T16:16:37.362 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100772.log.gz 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99976.log.gz 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101948.log.gz 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99091.log 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100381.log 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99091.log.gz 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.1.100381.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100381.log.gz 2023-10-08T16:16:37.363 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101581.log 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.97561.log 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101581.log.gz 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98314.log 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.97561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.97561.log.gz 2023-10-08T16:16:37.364 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.98314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102714.log 2023-10-08T16:16:37.365 INFO:teuthology.orchestra.run.smithi191.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98314.log.gz 2023-10-08T16:16:37.365 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-08T16:16:37.365 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.102714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102714.log.gz 2023-10-08T16:16:37.367 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99601.log 2023-10-08T16:16:37.367 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99601.log.gz 2023-10-08T16:16:37.367 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100377.log 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.100377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100377.log.gz 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.101177.log 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.101177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.101177.log.gz 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90970.log 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90970.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90970.log.gz 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99980.log 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99980.log.gz 2023-10-08T16:16:37.368 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98687.log 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.98687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98687.log.gz 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90755.log 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90755.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90755.log.gz 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.100776.log 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.100776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.100776.log.gz 2023-10-08T16:16:37.369 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-08T16:16:37.378 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.98318.log 2023-10-08T16:16:37.391 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.98318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.98318.log.gz 2023-10-08T16:16:37.403 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90798.log 2023-10-08T16:16:37.416 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.admin.90798.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90798.log.gz 2023-10-08T16:16:37.416 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.102710.log 2023-10-08T16:16:37.423 INFO:teuthology.orchestra.run.smithi191.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-08T16:16:37.423 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.102710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.102710.log.gz 2023-10-08T16:16:37.423 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.99597.log 2023-10-08T16:16:37.423 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.1.99597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.99597.log.gz 2023-10-08T16:16:37.423 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.97927.log 2023-10-08T16:16:37.424 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/ceph-client.3.97927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.97927.log.gz 2023-10-08T16:16:37.520 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175121.log 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165613.log 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.99777.log 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157605.log 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.2.99777.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.175121.log.gz 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.165613.log.gz 2023-10-08T16:16:37.521 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156402.log 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.99777.log.gz 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166665.log 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156402.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157605.log.gz 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173148.log 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156402.log.gz 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.166665.log: -- /var/log/ceph/ceph-client.admin.161418.log 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166581.log 2023-10-08T16:16:37.522 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.166665.log.gz 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161418.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173148.log.gz 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103917.log 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161418.log.gz 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97656.log 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166581.log.gz 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159164.log 2023-10-08T16:16:37.523 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99822.log 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103917.log.gz 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159164.log: gzip -5 88.9%--verbose -- replaced with /var/log/ceph/ceph-client.admin.97656.log.gz-- 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96711.log 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.99822.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159164.log.gz 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158183.log 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167200.log 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99822.log.gz 2023-10-08T16:16:37.524 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158183.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96711.log.gz 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162071.log 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158183.log.gz 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162409.log 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167200.log.gz 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161552.log 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171467.log 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162071.log.gz 2023-10-08T16:16:37.525 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.162409.log.gz 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104435.log 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97882.log 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.161552.log.gz 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104435.log: gzip -5 --verbose 88.1%-- /var/log/ceph/ceph-client.admin.106237.log -- replaced with /var/log/ceph/ceph-client.admin.171467.log.gz 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5%/var/log/ceph/ceph-client.admin.97882.log: -- replaced with /var/log/ceph/ceph-client.admin.104435.log.gz 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166413.log 2023-10-08T16:16:37.526 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106237.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.97882.log.gz-5 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.107110.log 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166413.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106237.log.gz 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172162.log 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107110.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166413.log.gz 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170756.log 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172162.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107110.log.gz 2023-10-08T16:16:37.527 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157832.log 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170756.log: gzip -5 --verbose -- 89.3%/var/log/ceph/ceph-client.admin.169192.log 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.172162.log.gz 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170756.log.gz 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161901.log 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105200.log 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175559.log 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169192.log.gz 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.157832.log.gz 2023-10-08T16:16:37.528 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.105200.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161901.log.gz 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91004.log 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105200.log.gz 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106341.log 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.91004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156612.log 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.175559.log.gz 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91004.log.gz 2023-10-08T16:16:37.529 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179852.log 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156612.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106341.log.gz 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161699.log 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179852.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156612.log.gz 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102785.log 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.179852.log.gz 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171933.log 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102785.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161699.log.gz 2023-10-08T16:16:37.530 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163721.log 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171933.log: gzip 88.5%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.102785.log.gz 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.162199.log 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163721.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171933.log.gz 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178771.log 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163721.log.gz 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171363.log 2023-10-08T16:16:37.531 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178771.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162199.log.gz 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107194.log 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.178771.log.gz 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170451.log 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4%/var/log/ceph/ceph-client.admin.107194.log: -- replaced with /var/log/ceph/ceph-client.admin.171363.log.gz 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100074.log 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100116.log 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107194.log.gz 2023-10-08T16:16:37.532 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100074.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170451.log.gz 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166245.log 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100116.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100074.log.gz 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177648.log 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166245.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100116.log.gz 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172676.log 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177648.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166245.log.gz 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165571.log 2023-10-08T16:16:37.533 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172676.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177648.log.gz 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163150.log 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165571.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172676.log.gz 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160023.log 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4%/var/log/ceph/ceph-client.admin.163150.log: -- replaced with /var/log/ceph/ceph-client.admin.165571.log.gz 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163763.log 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163150.log.gz 2023-10-08T16:16:37.534 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107499.log 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163763.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160023.log.gz 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169411.log 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.163763.log.gz 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.107499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169130.log 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107499.log.gz 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164536.log 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8%/var/log/ceph/ceph-client.admin.169130.log: -- replaced with /var/log/ceph/ceph-client.admin.169411.log.gz 2023-10-08T16:16:37.535 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171669.log 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164536.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.169130.log.gz-5 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.175882.log 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.164536.log.gz 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96481.log 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175882.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.171669.log.gz 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104223.log 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175882.log.gz 2023-10-08T16:16:37.536 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106027.log 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104223.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96481.log.gz 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156780.log 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167328.log 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104223.log.gz 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156780.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106027.log.gz 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156780.log.gz 2023-10-08T16:16:37.537 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171092.log 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163889.log 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167328.log.gz 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171237.log 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163889.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.171092.log.gz 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157343.log 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178543.log 88.9% 2023-10-08T16:16:37.538 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.163889.log.gz 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107302.log 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171237.log.gz 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178543.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.157343.log.gz 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90597.log 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178543.log.gz 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173798.log 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.90597.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107302.log.gz 2023-10-08T16:16:37.539 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97997.log 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101829.log 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90597.log.gz 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173798.log.gz 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174215.log 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97997.log.gz 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175517.log 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101829.log: /var/log/ceph/ceph-client.admin.174215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102740.log 2023-10-08T16:16:37.540 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101829.log.gz 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174215.log.gz 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175651.log 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175517.log: /var/log/ceph/ceph-client.0.102740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176879.log 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175517.log.gz 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102740.log.gz 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167013.log 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.175651.log.gz 2023-10-08T16:16:37.541 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165697.log 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102305.log 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176879.log.gz 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165697.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167013.log.gz 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90866.log 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102305.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165697.log.gz 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157005.log 2023-10-08T16:16:37.542 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.90866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103461.log 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102305.log.gz 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157005.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90866.log.gz 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100200.log 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157005.log.gz 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102001.log 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97996.log 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103461.log.gz 2023-10-08T16:16:37.543 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102001.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100200.log.gz 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101661.log 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102001.log.gz 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103629.log 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103251.log 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97996.log.gz 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103629.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101661.log.gzgzip 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.97992.log 2023-10-08T16:16:37.544 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103629.log.gz 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96354.log 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103251.log.gz 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165183.log 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96354.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97992.log.gz 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156822.log 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96354.log.gz 2023-10-08T16:16:37.545 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166623.log 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156444.log 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166623.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.156822.log.gz 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178856.log 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.165183.log.gz 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174543.log 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3%/var/log/ceph/ceph-client.admin.156444.log: -- replaced with /var/log/ceph/ceph-client.admin.166623.log.gz 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156696.log 2023-10-08T16:16:37.546 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156444.log.gz 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174543.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178856.log.gz 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100494.log 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174543.log.gz 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170364.log 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163108.log 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156696.log.gz 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.170364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100494.log.gz 2023-10-08T16:16:37.547 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170364.log.gz 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.177841.log 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96661.log 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.177841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159515.log 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% 88.2% -- replaced with /var/log/ceph/ceph-client.0.177841.log.gz -- replaced with /var/log/ceph/ceph-client.admin.163108.log.gz 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179390.log 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159515.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96661.log.gz 2023-10-08T16:16:37.548 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167706.log 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179390.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159515.log.gz 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176573.log 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.179390.log.gz 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163192.log 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107068.log 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163192.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.176573.log.gz 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176963.log 2023-10-08T16:16:37.549 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.167706.log.gz 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163192.log.gz 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105287.log 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106984.log 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176963.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107068.log.gz 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165249.log 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176963.log.gz 2023-10-08T16:16:37.550 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106984.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105287.log.gz 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174782.log 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165249.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106984.log.gz 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175380.log 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174782.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.165249.log.gz 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104265.log 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.175380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174782.log.gz 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97007.log 2023-10-08T16:16:37.551 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175380.log.gz 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105775.log 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97007.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104265.log.gz 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172372.log 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162451.log /var/log/ceph/ceph-client.admin.105775.log: 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97007.log.gz 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172372.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105775.log.gz 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168962.log 2023-10-08T16:16:37.552 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174417.log 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.172372.log.gz 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168962.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162451.log.gz 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105007.log 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174417.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.168962.log.gz 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105817.log 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.174417.log.gz 2023-10-08T16:16:37.553 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102431.log 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105007.log.gz 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163385.log 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102431.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105817.log.gz 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177498.log 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102431.log.gz 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104941.log 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.163385.log.gz 2023-10-08T16:16:37.554 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158091.log 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104941.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177498.log.gz 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104391.log 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.158091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158091.log.gz 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.160593.log 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.104941.log.gz 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100961.log 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.160593.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104391.log.gz 2023-10-08T16:16:37.555 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.157563.log -- replaced with /var/log/ceph/ceph-client.0.160593.log.gz 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177928.log 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9%/var/log/ceph/ceph-client.admin.157563.log: -- replaced with /var/log/ceph/ceph-client.admin.100961.log.gz 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101171.log 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177928.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.157563.log.gz 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166287.log 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101171.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.177928.log.gz 2023-10-08T16:16:37.556 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169845.log 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166287.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101171.log.gz 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97991.log 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.166287.log.gz/var/log/ceph/ceph-client.admin.169845.log: 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159687.log 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97133.log 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.169845.log.gz 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159687.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97991.log.gz 2023-10-08T16:16:37.557 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157435.log 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.159687.log.gz 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173420.log 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166539.log 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97133.log.gz 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173420.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157435.log.gz 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160914.log 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.173420.log.gz 2023-10-08T16:16:37.558 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172120.log 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163343.log 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.166539.log.gz 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101231.log 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160914.log.gz 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160292.log 89.2% 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.172120.log.gz 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163343.log.gz 2023-10-08T16:16:37.559 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165487.log 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101231.log.gz 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177414.log 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165487.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160292.log.gz 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179096.log 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177414.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.165487.log.gz 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176176.log 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177414.log.gz 2023-10-08T16:16:37.560 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176677.log 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176176.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179096.log.gz 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167538.log 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176677.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.176176.log.gz 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97308.log 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167538.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176677.log.gz 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174740.log 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97308.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167538.log.gz 2023-10-08T16:16:37.561 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102953.log 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97308.log.gz 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168052.log 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174740.log.gz 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176489.log 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96135.log 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102953.log.gz 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5%/var/log/ceph/ceph-client.admin.176489.log: -- replaced with /var/log/ceph/ceph-client.admin.168052.log.gz 2023-10-08T16:16:37.562 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170924.log 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176489.log.gz 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164492.log 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170924.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96135.log.gz 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179642.log 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2%/var/log/ceph/ceph-client.admin.164492.log: -- replaced with /var/log/ceph/ceph-client.admin.170924.log.gz 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96531.log 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164492.log.gz 2023-10-08T16:16:37.563 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168228.log 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162367.log 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.179642.log.gz 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96531.log.gz 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178333.log 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162367.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.168228.log.gz-5 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.167156.log 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178333.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.162367.log.gz 2023-10-08T16:16:37.564 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169625.log 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178333.log.gz 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168920.log 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167156.log.gz 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106730.log 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169625.log.gz 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99906.log 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106730.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168920.log.gzgzip 2023-10-08T16:16:37.565 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.156184.log 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.99906.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106730.log.gz 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106942.log 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99906.log.gz 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163427.log 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7%/var/log/ceph/ceph-client.admin.106942.log: -- replaced with /var/log/ceph/ceph-client.admin.156184.log.gz 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174459.log 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163427.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106942.log.gz 2023-10-08T16:16:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173274.log 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.163427.log.gz 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160806.log 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173274.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.174459.log.gz 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166329.log 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160806.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173274.log.gz 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169929.log 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160806.log.gz 2023-10-08T16:16:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159206.log 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169929.log: gzip 89.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.166329.log.gz--verbose 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.99864.log 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159206.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169929.log.gz 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.159206.log.gz 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.99864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164662.log 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.99864.log.gz 2023-10-08T16:16:37.568 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102051.log 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106858.log 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164662.log.gz 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171627.log 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172246.log 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102051.log.gz 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106858.log: /var/log/ceph/ceph-client.admin.171627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169308.log 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106858.log.gz 2023-10-08T16:16:37.569 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174826.log 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172246.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.171627.log.gz 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169308.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.172246.log.gz 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104521.log 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97304.log 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6%/var/log/ceph/ceph-client.admin.174826.log: -- replaced with /var/log/ceph/ceph-client.admin.169308.log.gz 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166971.log 2023-10-08T16:16:37.570 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174826.log.gz 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.170368.log 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97304.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.104521.log.gz 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160680.log 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97304.log.gz 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.170368.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166971.log.gz 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175924.log 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.170368.log.gz 2023-10-08T16:16:37.571 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176362.log 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160680.log.gz/var/log/ceph/ceph-client.admin.175924.log: 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102389.log 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176362.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.175924.log.gz 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162762.log 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.176362.log.gz 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104001.log 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102389.log.gz 2023-10-08T16:16:37.572 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105371.log 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.172631.log 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105371.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104001.log.gz 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.162762.log.gzgzip 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.179264.log 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105371.log.gz 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.172631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105607.log 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.172631.log.gz 2023-10-08T16:16:37.573 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159771.log 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105607.log: gzip 89.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.179264.log.gz--verbose 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.158515.log 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105607.log.gz 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169675.log 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.159771.log.gz 2023-10-08T16:16:37.574 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98410.log 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100919.log 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158515.log.gz 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98410.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.169675.log.gz 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177886.log 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98410.log.gz/var/log/ceph/ceph-client.admin.100919.log: 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169004.log 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177886.log: gzip 89.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.100919.log.gz--verbose -- 2023-10-08T16:16:37.575 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178711.log 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177886.log.gz 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157477.log 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178711.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.169004.log.gz 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160956.log 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.178711.log.gz 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163637.log 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160956.log: gzip 89.4%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.157477.log.gz-- 2023-10-08T16:16:37.576 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158893.log 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163637.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.160956.log.gz 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158725.log 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158893.log: gzip 88.3%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.163637.log.gz-- 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169759.log 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180088.log 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.158893.log.gz 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169759.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158725.log.gz 2023-10-08T16:16:37.577 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173714.log 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2%/var/log/ceph/ceph-client.admin.180088.log: -- replaced with /var/log/ceph/ceph-client.admin.169759.log.gz 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102911.log 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97305.log 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.180088.log.gz 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175693.log 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173714.log.gz 2023-10-08T16:16:37.578 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102911.log.gz 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174952.log 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175693.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97305.log.gz 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102481.log 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174952.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.175693.log.gz 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162661.log 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102481.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.174952.log.gz 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96795.log 2023-10-08T16:16:37.579 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178627.log 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96795.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162661.log.gz 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- 91.2%/var/log/ceph/ceph-client.admin.157393.log 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102481.log.gz 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178627.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96795.log.gz 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105859.log 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.178627.log.gz 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162828.log 2023-10-08T16:16:37.580 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157393.log.gz 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.172635.log 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2%/var/log/ceph/ceph-client.admin.162828.log: -- replaced with /var/log/ceph/ceph-client.admin.105859.log.gz 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107236.log 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.172635.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162828.log.gz 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163469.log 0.0% 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.0.172635.log.gz 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165141.log 2023-10-08T16:16:37.581 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98452.log 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.107236.log.gz 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165141.log: gzip -5 88.5%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.163469.log.gz/var/log/ceph/ceph-client.admin.164280.log 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165141.log.gz 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171883.log 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164280.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98452.log.gz 2023-10-08T16:16:37.582 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179222.log 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.164280.log.gz 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96228.log 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171883.log.gz 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177582.log 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172894.log 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179222.log.gz 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170055.log 2023-10-08T16:16:37.583 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96228.log.gz 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168878.log 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170055.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.177582.log.gz 89.3% 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.172894.log.gz 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167114.log 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168878.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170055.log.gz 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168626.log 2023-10-08T16:16:37.584 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168878.log.gz/var/log/ceph/ceph-client.admin.167114.log: 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166203.log 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106419.log 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167114.log.gz 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166203.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168626.log.gz 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167412.log 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166203.log.gz 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168270.log 2023-10-08T16:16:37.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171279.log 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106419.log.gz 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104141.log 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167412.log.gz 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171279.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.168270.log.gz 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.97307.log 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179684.log 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171279.log.gz 2023-10-08T16:16:37.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106279.log 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104141.log.gz 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97307.log.gz 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.163021.log 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106279.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.179684.log.gz 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165920.log 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.163021.log: 88.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.106279.log.gz--verbose -- 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98964.log 2023-10-08T16:16:37.587 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.163021.log.gz 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165920.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98964.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165920.log.gz 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170966.log 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.audit.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98964.log.gz 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163679.log 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100621.log 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172288.log 2023-10-08T16:16:37.588 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.170966.log.gz 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.163679.log.gz 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157521.log 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100621.log.gz 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175036.log 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157521.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.172288.log.gz 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166889.log 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175036.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157521.log.gz 2023-10-08T16:16:37.589 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100242.log 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.175036.log.gz 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97659.log 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100242.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.166889.log.gz 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179516.log 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100242.log.gz 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103833.log 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97659.log.gz 2023-10-08T16:16:37.590 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104815.log 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96965.log 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179516.log.gz 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104815.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103833.log.gz 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162027.log 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2%/var/log/ceph/ceph-client.admin.96965.log: -- replaced with /var/log/ceph/ceph-client.admin.104815.log.gz 2023-10-08T16:16:37.591 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159038.log 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157731.log 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96965.log.gz 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.162027.log.gz 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178291.log 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159038.log.gz 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178417.log 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157731.log: /var/log/ceph/ceph-client.admin.178291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158683.log 2023-10-08T16:16:37.592 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157731.log.gz 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171839.log 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.178291.log.gz 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1%/var/log/ceph/ceph-client.admin.158683.log: -- replaced with /var/log/ceph/ceph-client.admin.178417.log.gz 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156864.log 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171839.log: gzip -5 --verbose 88.2%-- /var/log/ceph/ceph-client.admin.100877.log -- replaced with /var/log/ceph/ceph-client.admin.158683.log.gz 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156864.log: gzip 88.4%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.171839.log.gz/var/log/ceph/ceph-client.admin.103419.log 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.593 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101957.log 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156864.log.gz 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7%/var/log/ceph/ceph-client.admin.103419.log: -- replaced with /var/log/ceph/ceph-client.admin.100877.log.gz 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173985.log 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101957.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103419.log.gz 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168542.log 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173985.log: gzip -5 88.0%--verbose -- /var/log/ceph/ceph-client.admin.162325.log -- replaced with /var/log/ceph/ceph-client.admin.101957.log.gz 2023-10-08T16:16:37.594 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173985.log.gz 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97301.log 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162325.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168542.log.gz 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98200.log 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161250.log 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.162325.log.gz 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98200.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97301.log.gz 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100452.log 2023-10-08T16:16:37.595 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98200.log.gz 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179558.log 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168354.log 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.161250.log.gz 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100032.log 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100452.log.gz 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168354.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179558.log.gz 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178375.log 2023-10-08T16:16:37.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179726.log 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.168354.log.gz 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178375.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100032.log.gz 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179432.log 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179726.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.178375.log.gz 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103875.log 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179432.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.179726.log.gz 2023-10-08T16:16:37.597 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167244.log 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.179432.log.gz 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103875.log.gz 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102547.log 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107026.log 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173943.log 89.1% 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.167244.log.gz 2023-10-08T16:16:37.598 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107026.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102547.log.gz 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102263.log 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164620.log 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107026.log.gz 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102263.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173943.log.gz 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103335.log 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102263.log.gz 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165442.log 2023-10-08T16:16:37.599 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103335.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164620.log.gz 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106900.log 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103335.log.gz 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.165442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168710.log 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165442.log.gz 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167370.log 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168710.log: gzip -5 89.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.106900.log.gz/var/log/ceph/ceph-client.admin.159268.log 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.600 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.168710.log.gz 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.177845.log 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159939.log 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167370.log.gz 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.177845.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159268.log.gz 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.0.177845.log.gz/var/log/ceph/ceph-client.admin.178145.log 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104349.log 2023-10-08T16:16:37.601 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178145.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.159939.log.gz 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170535.log 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178145.log.gz 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173901.log 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170535.log: 87.6%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.104349.log.gz--verbose 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.101745.log 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173901.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170535.log.gz 2023-10-08T16:16:37.602 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103092.log 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.173901.log.gz 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170714.log 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165815.log 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101745.log.gz 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170714.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103092.log.gz 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176218.log 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170714.log.gz 2023-10-08T16:16:37.603 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97658.log 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161124.log 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.165815.log.gz 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176218.log.gz 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104605.log 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161124.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97658.log.gz 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178814.log 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104605.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161124.log.gz 2023-10-08T16:16:37.604 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171405.log 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178814.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104605.log.gz 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106518.log 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171405.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178814.log.gz 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156142.log 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.171405.log.gz 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162493.log 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106518.log.gz 2023-10-08T16:16:37.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178669.log 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162493.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.156142.log.gz 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98368.log 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163511.log 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.162493.log.gz 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.178669.log.gz 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169581.log 2023-10-08T16:16:37.606 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163511.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98368.log.gz 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156528.log 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3%/var/log/ceph/ceph-client.admin.169581.log: -- replaced with /var/log/ceph/ceph-client.admin.163511.log.gz 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156310.log 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169581.log.gz 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159346.log 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156310.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156528.log.gz 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176615.log 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159346.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.156310.log.gz 2023-10-08T16:16:37.607 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100410.log 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159346.log.gz 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100536.log 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100410.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.176615.log.gz 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168312.log 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100536.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100410.log.gz 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164406.log 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100536.log.gz 2023-10-08T16:16:37.608 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161636.log 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164406.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.168312.log.gz--verbose 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.97297.log 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161636.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.164406.log.gz 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163973.log 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161636.log.gz 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162619.log 2023-10-08T16:16:37.609 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163973.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97297.log.gz 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159897.log 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.163973.log.gz 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166811.log 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162619.log.gz 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106560.log 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166811.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159897.log.gz 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176447.log 2023-10-08T16:16:37.610 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106560.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.166811.log.gz 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172760.log 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106560.log.gz 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171795.log 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160998.log 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.176447.log.gz 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171795.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.172760.log.gz 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179894.log 2023-10-08T16:16:37.611 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171795.log.gz 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177456.log 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179894.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.160998.log.gz 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102093.log 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177456.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.179894.log.gz 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104647.log 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.177456.log.gz 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161468.log 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104647.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102093.log.gz 2023-10-08T16:16:37.612 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156360.log 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161468.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104647.log.gz 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98536.log 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.161468.log.gz 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105204.log 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98536.log: 87.7%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.156360.log.gz-- 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176260.log 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.105204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.105204.log.gz 2023-10-08T16:16:37.613 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98536.log.gz 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177160.log 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107540.log 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103587.log 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176260.log.gz 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177160.log.gz 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106686.log 2023-10-08T16:16:37.614 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107540.log.gz 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179138.log 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106069.log 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103587.log.gz 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106686.log.gz 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158137.log 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.179138.log.gz 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176008.log 2023-10-08T16:16:37.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158137.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106069.log.gzgzip 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.96923.log 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.158137.log.gz 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101087.log 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.176008.log.gz 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159473.log 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97298.log 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96923.log.gz 2023-10-08T16:16:37.616 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101087.log.gz 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173858.log 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.159473.log.gz 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171050.log 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105943.log 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97298.log.gz 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173858.log.gz 2023-10-08T16:16:37.617 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103377.log 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171050.log.gz 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173106.log 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103377.log: gzip -5 --verbose 88.7%-- /var/log/ceph/ceph-client.admin.163553.log 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105943.log.gz 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103377.log.gz 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159981.log 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173106.log.gz 2023-10-08T16:16:37.618 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102135.log 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159981.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163553.log.gz 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105469.log 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.159981.log.gz 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103293.log 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102135.log.gz 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104773.log 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103293.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.105469.log.gz-5 2023-10-08T16:16:37.619 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.172330.log 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103293.log.gz 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165751.log 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104773.log.gz 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107590.log 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165751.log: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.172330.log.gz--verbose 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.100793.log 2023-10-08T16:16:37.620 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.165751.log.gz 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104063.log 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107590.log.gz 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166707.log 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157773.log 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100793.log.gz 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104063.log.gz 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105733.log 2023-10-08T16:16:37.621 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166707.log.gz 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161777.log 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105733.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.157773.log.gz-5 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.171321.log 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161777.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105733.log.gz 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103774.log 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161777.log.gz 2023-10-08T16:16:37.622 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159729.log 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103774.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171321.log.gz 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103008.log 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103774.log.gz 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157647.log 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4%/var/log/ceph/ceph-client.admin.103008.log: -- replaced with /var/log/ceph/ceph-client.admin.159729.log.gz 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105565.log 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156046.log 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103008.log.gz 2023-10-08T16:16:37.623 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157647.log.gz 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173378.log 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.105565.log.gz 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166119.log 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98578.log 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156046.log.gz 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173378.log.gz 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107674.log 2023-10-08T16:16:37.624 INFO:teuthology.orchestra.run.smithi170.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.166119.log.gz 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105901.log 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107674.log: gzip 87.8%-5 -- replaced with /var/log/ceph/ceph-client.admin.98578.log.gz--verbose 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.180048.log 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107674.log.gz 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156927.log 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105901.log.gz 2023-10-08T16:16:37.625 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.180048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102179.log 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156927.log: 87.4%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.180048.log.gz--verbose 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.98788.log 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156927.log.gz 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164956.log 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102179.log.gz 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101394.log 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164956.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98788.log.gz 2023-10-08T16:16:37.626 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158280.log 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164956.log.gz 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160065.log 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101394.log.gz 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169046.log 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160065.log: gzip -5 --verbose 88.5%-- /var/log/ceph/ceph-client.admin.173630.log -- replaced with /var/log/ceph/ceph-client.admin.158280.log.gz 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160065.log.gz 2023-10-08T16:16:37.627 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107632.log 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169046.log.gz 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164119.log 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175777.log 88.6% 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.admin.173630.log.gz 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107632.log.gz 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157213.log 2023-10-08T16:16:37.628 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175777.log: gzip 89.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.164119.log.gz--verbose 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.157255.log 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157213.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175777.log.gz 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105329.log 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157255.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157213.log.gz 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105413.log 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157255.log.gz 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97049.log 2023-10-08T16:16:37.629 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96839.log 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105329.log.gz 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105413.log.gz 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97049.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97049.log.gz 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168584.log 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101129.log 2023-10-08T16:16:37.630 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96839.log.gz 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170493.log 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101129.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168584.log.gz 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174501.log 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101129.log.gz 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.84618.log 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170493.log.gz 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162720.log 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174501.log: /var/log/ceph/ceph.tmp-client.admin.84618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174648.log 2023-10-08T16:16:37.631 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.84618.log.gz 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174501.log.gz 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106153.log 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.162720.log.gz 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178501.log 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.174648.log.gz 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98830.log 2023-10-08T16:16:37.632 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106153.log: /var/log/ceph/ceph-client.admin.178501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102827.log 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106153.log.gz 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98830.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178501.log.gz 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170097.log 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165082.log 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.98830.log.gz 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170097.log: gzip 89.3%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.102827.log.gz/var/log/ceph/ceph-client.admin.176405.log 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164704.log 2023-10-08T16:16:37.633 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165082.log: /var/log/ceph/ceph-client.admin.176405.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.170097.log.gz 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165082.log.gz 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.176405.log.gz 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101517.log 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164704.log.gz 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175187.log 2023-10-08T16:16:37.634 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.101517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173756.log 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175187.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.101517.log.gz 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176134.log 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105245.log 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9%/var/log/ceph/ceph-client.admin.173756.log: -- replaced with /var/log/ceph/ceph-client.admin.175187.log.gz 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176134.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.173756.log.gz--verbose 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.177372.log 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.176134.log.gz 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158599.log 2023-10-08T16:16:37.635 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105245.log: /var/log/ceph/ceph-client.admin.177372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176755.log 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105245.log.gz 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158599.log: 88.7%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.177372.log.gz-- 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175735.log 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158599.log.gz 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167772.log 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176755.log.gz 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107772.log 2023-10-08T16:16:37.636 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175735.log: /var/log/ceph/ceph-client.admin.167772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102869.log 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.175735.log.gz 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107772.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.167772.log.gz 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162535.log 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107772.log.gz 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174293.log 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102869.log.gz 2023-10-08T16:16:37.637 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171195.log 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162535.log: /var/log/ceph/ceph-client.admin.174293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156268.log 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162535.log.gz 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171195.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174293.log.gz 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171975.log 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.171195.log.gz 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158322.log 2023-10-08T16:16:37.638 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156268.log.gz 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.165446.log 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171975.log: /var/log/ceph/ceph-client.admin.158322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.158095.log 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171975.log.gz 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.165446.log: gzip 89.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.158322.log.gz-- 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177288.log 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.165446.log.gz 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.158095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158953.log 2023-10-08T16:16:37.639 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.158095.log.gz 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.160597.log 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177288.log.gz 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.167969.log 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169453.log 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158953.log.gz/var/log/ceph/ceph-client.0.160597.log: 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.160597.log.gz 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.167969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175078.log 2023-10-08T16:16:37.640 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.167969.log.gz 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164578.log 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169453.log.gz 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161334.log 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.175078.log.gz 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103503.log 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173462.log 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164578.log.gz 2023-10-08T16:16:37.641 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161334.log: /var/log/ceph/ceph-client.admin.103503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167622.log 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161334.log.gz 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180130.log 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173462.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103503.log.gz 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158767.log 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.173462.log.gz 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.180130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159643.log 2023-10-08T16:16:37.642 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167622.log.gz 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158767.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.180130.log.gz 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158851.log 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163931.log 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158767.log.gz 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159643.log: /var/log/ceph/ceph-client.admin.158851.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.159643.log.gz 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101003.log 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166371.log 2023-10-08T16:16:37.643 INFO:teuthology.orchestra.run.smithi170.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.158851.log.gz 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101003.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163931.log.gz 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97091.log 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101003.log.gz 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169537.log 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166371.log: /var/log/ceph/ceph-client.admin.97091.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166371.log.gz 2023-10-08T16:16:37.644 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169537.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.97091.log.gz-5 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr:--verbose -- /var/log/ceph/ceph-client.admin.163234.log 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.169537.log.gz 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106644.log 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173190.log 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163234.log: /var/log/ceph/ceph-client.admin.106644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97648.log 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163234.log.gz 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106644.log.gz 2023-10-08T16:16:37.645 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164364.log 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97648.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173190.log.gz 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97648.log.gz 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164364.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164364.log.gz 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170409.log 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164197.log 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161082.log 2023-10-08T16:16:37.646 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.170409.log.gz 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168136.log 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179768.log 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161082.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164197.log.gz 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168136.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161082.log.gz 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179768.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.168136.log.gz 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.179768.log.gz 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166497.log 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176050.log 2023-10-08T16:16:37.647 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104479.log 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.166497.log.gz 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173064.log 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160191.log 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104479.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.176050.log.gz 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173064.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104479.log.gz 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160191.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173064.log.gz 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160191.log.gz 2023-10-08T16:16:37.648 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166161.log 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101913.log 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169088.log 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.166161.log.gz 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159122.log 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97302.log 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169088.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101913.log.gz 2023-10-08T16:16:37.649 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159122.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169088.log.gz 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97302.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159122.log.gz 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97302.log.gz 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168500.log 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172802.log 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178459.log 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.168500.log.gz 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161594.log 2023-10-08T16:16:37.650 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174868.log 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178459.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.172802.log.gz 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.163025.log 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178459.log.gz 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.161594.log.gz 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164448.log 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174868.log.gz 2023-10-08T16:16:37.651 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.163025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.163025.log.gz 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164448.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164448.log.gz 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158641.log 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174375.log 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96573.log 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158641.log.gz 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161292.log 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167072.log 2023-10-08T16:16:37.652 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174375.log.gz 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96573.log: /var/log/ceph/ceph-client.admin.161292.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96573.log.gz 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161292.log.gz 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167072.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167072.log.gz 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169495.log 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175966.log 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-08T16:16:37.653 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102347.log 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169495.log.gz 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167580.log 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97646.log 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102347.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175966.log.gz 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164872.log 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102347.log.gz 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97646.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167580.log.gz 2023-10-08T16:16:37.654 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107495.log 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97646.log.gz 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100158.log 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97661.log 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.107495.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164872.log.gz 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107495.log.gz 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172017.log 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165529.log 2023-10-08T16:16:37.655 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100158.log.gz 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97661.log.gz 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101871.log 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156486.log 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.172017.log.gz 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163595.log 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165529.log.gz 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156486.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101871.log.gz 2023-10-08T16:16:37.656 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157689.log 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156486.log.gz 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103050.log 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173232.log 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157689.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163595.log.gz 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.167965.log 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157689.log.gz 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173232.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103050.log.gz 2023-10-08T16:16:37.657 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166749.log 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.167965.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173232.log.gz 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102221.log 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.167965.log.gz 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160722.log 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166749.log.gz 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.102221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97995.log 2023-10-08T16:16:37.658 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170013.log 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160722.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102221.log.gz 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177066.log 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.160722.log.gz 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170013.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97995.log.gz 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103671.log 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169971.log 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170013.log.gz 2023-10-08T16:16:37.659 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103671.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177066.log.gzgzip 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.158557.log 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103671.log.gz 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176531.log 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96308.log 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158557.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.169971.log.gz 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165040.log 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158557.log.gz 2023-10-08T16:16:37.660 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.176531.log.gz 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97993.log 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165040.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96308.log.gzgzip 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.172438.log 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97993.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165040.log.gz 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170798.log 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175819.log 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97993.log.gz 2023-10-08T16:16:37.661 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172204.log 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172438.log.gz 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.170798.log.gz 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.175819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103959.log 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177540.log 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175819.log.gz 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172204.log.gz 2023-10-08T16:16:37.662 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173022.log 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106602.log 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103959.log.gz 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.177540.log.gzgzip 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.164914.log 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173022.log.gz 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101045.log 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174910.log 2023-10-08T16:16:37.663 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106602.log.gz 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101045.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.164914.log.gz 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165655.log 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174910.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101045.log.gz 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97652.log 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170672.log 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.174910.log.gz 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179978.log 2023-10-08T16:16:37.664 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.165655.log.gz 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97652.log.gz 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164746.log 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157171.log 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179978.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170672.log.gz 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157898.log 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.179978.log.gz 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164746.log.gz 2023-10-08T16:16:37.665 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166035.log 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158473.log 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157898.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157171.log.gz 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179936.log 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.157898.log.gz 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166035.log.gz 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96881.log 2023-10-08T16:16:37.666 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104857.log 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179936.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158473.log.gz 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162157.log 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.179936.log.gz 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96881.log.gz 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168010.log 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170163.log 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104857.log.gz 2023-10-08T16:16:37.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162157.log: /var/log/ceph/ceph-client.admin.168010.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162157.log.gz 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170163.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.168010.log.gz 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170163.log.gz 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100835.log 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99369.log 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99377.log 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100835.log.gz 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.99369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176302.log 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99369.log.gz 2023-10-08T16:16:37.668 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174587.log 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.99377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.99377.log.gz 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160149.log 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.176302.log.gz 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165857.log 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160149.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174587.log.gz 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.165857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163847.log 2023-10-08T16:16:37.669 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160149.log.gz 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159813.log 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.165857.log.gz 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98620.log 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159813.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163847.log.gz 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167286.log 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159080.log 2023-10-08T16:16:37.670 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.159813.log.gz 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99948.log 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98620.log.gz 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159080.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167286.log.gz 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.99948.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159080.log.gz 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98746.log 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99948.log.gz 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177110.log 2023-10-08T16:16:37.671 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101274.log 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177110.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98746.log.gz 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101274.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.177110.log.gz 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101274.log.gz 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168458.log 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177970.log 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99418.log 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177970.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.168458.log.gz 2023-10-08T16:16:37.672 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.99418.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177970.log.gz 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99418.log.gz 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172980.log 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102744.log 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100284.log 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.102744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164015.log 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172980.log.gz 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102744.log.gz 2023-10-08T16:16:37.673 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168836.log 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164015.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.100284.log.gz 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101475.log 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164015.log.gz 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101475.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.168836.log.gz 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101475.log.gz 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164057.log 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171152.log 2023-10-08T16:16:37.674 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171545.log 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171152.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164057.log.gz 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171545.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.171152.log.gz 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106195.log 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.171545.log.gz 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168752.log 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100368.log 2023-10-08T16:16:37.675 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168752.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106195.log.gz 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100368.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168752.log.gz 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100368.log.gz 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179306.log 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157129.log 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158996.log 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157129.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179306.log.gz 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158996.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157129.log.gz 2023-10-08T16:16:37.676 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106111.log 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158996.log.gz 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105691.log 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173588.log 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106111.log.gz 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173588.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105691.log.gz 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173588.log.gz 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168094.log 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101316.log 2023-10-08T16:16:37.677 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.175384.log 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101316.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.168094.log.gz 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.0.175384.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101316.log.gz 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.175384.log.gz 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171008.log 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174111.log 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101703.log 2023-10-08T16:16:37.678 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174111.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.171008.log.gz 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101703.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.174111.log.gz 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101703.log.gz 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98704.log 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177244.log 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163805.log 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177244.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98704.log.gz 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.163805.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.177244.log.gz 2023-10-08T16:16:37.679 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163805.log.gz 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100326.log 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179600.log 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166455.log 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179600.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100326.log.gz 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166455.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179600.log.gz 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.166455.log.gz 2023-10-08T16:16:37.680 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101619.log 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107716.log 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170577.log 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107716.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.101619.log.gz 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170577.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107716.log.gz 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.170577.log.gz 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167664.log 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179474.log 2023-10-08T16:16:37.681 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174153.log 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.179474.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167664.log.gz 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174153.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.179474.log.gz 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174153.log.gz 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174994.log 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169803.log 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161166.log 2023-10-08T16:16:37.682 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169803.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174994.log.gz 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161166.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169803.log.gz 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.161166.log.gz 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170840.log 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96617.log 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.170840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160400.log 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96617.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170840.log.gz 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160400.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96617.log.gz 2023-10-08T16:16:37.683 INFO:teuthology.orchestra.run.smithi170.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.160400.log.gz 2023-10-08T16:16:37.685 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166077.log 2023-10-08T16:16:37.685 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162283.log 2023-10-08T16:16:37.685 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.166077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159855.log 2023-10-08T16:16:37.685 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162283.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166077.log.gz 2023-10-08T16:16:37.685 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159855.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162283.log.gz 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.159855.log.gz 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178898.log 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178012.log 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176837.log 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171711.log 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.178898.log.gz 2023-10-08T16:16:37.686 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161510.log 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178012.log.gz 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.171711.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.176837.log.gz 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161510.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171711.log.gz 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156654.log 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161510.log.gz 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100751.log 2023-10-08T16:16:37.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mgr.y.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156654.log.gz 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100751.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100751.log.gz 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169887.log 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156570.log 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:gzip /var/log/ceph/ceph-client.admin.169887.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.104899.log 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.169887.log.gz 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104899.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.156570.log.gz 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160107.log 2023-10-08T16:16:37.688 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104899.log.gz 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175467.log 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160107.log: /var/log/ceph/ceph-client.admin.175467.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.160107.log.gz 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.175467.log.gz 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161859.log 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160334.log 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161859.log: /var/log/ceph/ceph-client.admin.160334.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161859.log.gz 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.160334.log.gz 2023-10-08T16:16:37.689 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178940.log 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169369.log 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178940.log: /var/log/ceph/ceph-client.admin.169369.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.178940.log.gz 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.169369.log.gz 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176921.log 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161943.log 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97660.log 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7%/var/log/ceph/ceph-client.admin.161943.log: -- replaced with /var/log/ceph/ceph-client.admin.176921.log.gz 2023-10-08T16:16:37.690 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.161943.log.gz 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97660.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97660.log.gz 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164830.log 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171753.log 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.164830.log: /var/log/ceph/ceph-client.admin.171753.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164830.log.gz 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.171753.log.gz 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107152.log 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176092.log 2023-10-08T16:16:37.691 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.107152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162577.log 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.176092.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107152.log.gz 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162577.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.176092.log.gz 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.162577.log.gz 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159557.log 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170882.log 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159557.log: /var/log/ceph/ceph-client.admin.170882.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.159557.log.gz 2023-10-08T16:16:37.692 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97883.log 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170882.log.gz 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101559.log 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.97883.log: /var/log/ceph/ceph-client.admin.101559.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97883.log.gz 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101559.log.gz 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105649.log 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173546.log 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.105649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174069.log 2023-10-08T16:16:37.693 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173546.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105649.log.gz 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173546.log.gz 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.174069.log.gz 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106774.log 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164788.log 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106774.log: /var/log/ceph/ceph-client.admin.164788.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106774.log.gz 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174027.log 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164788.log.gz 2023-10-08T16:16:37.694 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98284.log 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174027.log: /var/log/ceph/ceph-client.admin.98284.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.174027.log.gz 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98284.log.gz 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157087.log 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162241.log 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.157087.log: /var/log/ceph/ceph-client.admin.162241.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.157087.log.gz 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162241.log.gz 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178585.log 2023-10-08T16:16:37.695 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179180.log 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178585.log: /var/log/ceph/ceph-client.admin.179180.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.178585.log.gz 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.179180.log.gz 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159599.log 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161040.log 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.159599.log: /var/log/ceph/ceph-client.admin.161040.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159599.log.gz 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.161040.log.gz 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106816.log 2023-10-08T16:16:37.696 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179810.log 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.106816.log: /var/log/ceph/ceph-client.admin.179810.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106816.log.gz 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179810.log.gz 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162115.log 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179348.log 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.162115.log: /var/log/ceph/ceph-client.admin.179348.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162115.log.gz 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179348.log.gz 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167496.log 2023-10-08T16:16:37.697 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175425.log 2023-10-08T16:16:37.698 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167496.log: /var/log/ceph/ceph-client.admin.175425.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167496.log.gz 2023-10-08T16:16:37.710 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.175425.log.gz 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174698.log 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158238.log 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.174698.log: /var/log/ceph/ceph-client.admin.158238.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174698.log.gz 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158238.log.gz 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177202.log 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172938.log 2023-10-08T16:16:37.711 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177202.log: /var/log/ceph/ceph-client.admin.172938.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177202.log.gz 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.172938.log.gz 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98662.log 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172078.log 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98662.log: /var/log/ceph/ceph-client.admin.172078.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98662.log.gz 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:gzip 88.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.172078.log.gz--verbose 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:-- /var/log/ceph/ceph-client.admin.103545.log 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98494.log 2023-10-08T16:16:37.712 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103545.log: /var/log/ceph/ceph-client.admin.98494.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103545.log.gz 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98494.log.gz 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104563.log 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168668.log 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168794.log 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104563.log.gz 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.168668.log.gz 2023-10-08T16:16:37.713 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.168794.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168794.log.gz 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177330.log 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103136.log 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.177330.log: /var/log/ceph/ceph-client.admin.103136.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177330.log.gz 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173336.log 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103136.log.gz 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160233.log 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173336.log: /var/log/ceph/ceph-client.admin.160233.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173336.log.gz 2023-10-08T16:16:37.714 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160233.log.gz 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169717.log 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177005.log 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.169717.log: /var/log/ceph/ceph-client.admin.177005.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169717.log.gz 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.177005.log.gz 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155966.log 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105985.log 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.155966.log: /var/log/ceph/ceph-client.admin.105985.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155966.log.gz 2023-10-08T16:16:37.715 INFO:teuthology.orchestra.run.smithi170.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.105985.log.gz 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104307.log 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.99781.log 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104307.log: /var/log/ceph/ceph-client.2.99781.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104307.log.gz 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.99781.log.gz 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158809.log 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159430.log 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.158809.log: /var/log/ceph/ceph-client.admin.159430.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.158809.log.gz 2023-10-08T16:16:37.716 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160638.log 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159430.log.gz 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158376.log 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160638.log: /var/log/ceph/ceph-client.admin.158376.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160638.log.gz 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158376.log.gz 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98242.log 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164322.log 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98242.log: /var/log/ceph/ceph-client.admin.164322.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98242.log.gz 2023-10-08T16:16:37.717 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.164322.log.gz 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178104.log 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156226.log 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.178104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172718.log 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.156226.log: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.178104.log.gz 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.172718.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156226.log.gz 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.172718.log.gz 2023-10-08T16:16:37.718 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161985.log 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156738.log 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161985.log: /var/log/ceph/ceph-client.admin.156738.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.161985.log.gz 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.156738.log.gz 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103731.log 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99990.log 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.103731.log: /var/log/ceph/ceph-client.admin.99990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103731.log.gz 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99990.log.gz 2023-10-08T16:16:37.719 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160764.log 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161376.log 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.160764.log: /var/log/ceph/ceph-client.admin.161376.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160764.log.gz 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.161376.log.gz 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167454.log 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173672.log 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.167454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100579.log 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.173672.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.167454.log.gz 2023-10-08T16:16:37.720 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.100579.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.173672.log.gz 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100579.log.gz 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101787.log 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163066.log 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.101787.log: /var/log/ceph/ceph-client.admin.163066.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101787.log.gz 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.163066.log.gz 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104689.log 2023-10-08T16:16:37.721 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.104689.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104689.log.gz 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96432.log 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157299.log 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96432.log: /var/log/ceph/ceph-client.admin.157299.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96432.log.gz 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.157299.log.gz 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161208.log 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173504.log 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.161208.log: /var/log/ceph/ceph-client.admin.173504.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161208.log.gz 2023-10-08T16:16:37.722 INFO:teuthology.orchestra.run.smithi170.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173504.log.gz 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96753.log 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168416.log 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.96753.log: /var/log/ceph/ceph-client.admin.168416.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96753.log.gz 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.168416.log.gz 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104731.log 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164998.log 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.104731.log: /var/log/ceph/ceph-client.admin.164998.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.104731.log.gz 2023-10-08T16:16:37.723 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.164998.log.gz 2023-10-08T16:16:37.724 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98326.log 2023-10-08T16:16:37.724 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.98326.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98326.log.gz 2023-10-08T16:16:37.727 INFO:teuthology.orchestra.run.smithi191.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-08T16:16:37.791 INFO:teuthology.orchestra.run.smithi191.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-08T16:16:37.926 INFO:teuthology.orchestra.run.smithi191.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-08T16:16:38.043 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-08T16:16:38.089 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-08T16:16:38.096 INFO:teuthology.orchestra.run.smithi191.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-08T16:16:38.171 INFO:teuthology.orchestra.run.smithi170.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-08T16:16:38.195 INFO:teuthology.orchestra.run.smithi191.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-08T16:16:38.271 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-08T16:16:38.303 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-08T16:16:38.346 INFO:teuthology.orchestra.run.smithi170.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-08T16:16:38.505 INFO:teuthology.orchestra.run.smithi170.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-08T16:16:38.506 INFO:teuthology.orchestra.run.smithi170.stderr: 2023-10-08T16:16:38.507 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m1.208s 2023-10-08T16:16:38.507 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m5.096s 2023-10-08T16:16:38.507 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.518s 2023-10-08T16:16:42.364 INFO:teuthology.orchestra.run.smithi191.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-08T16:16:42.365 INFO:teuthology.orchestra.run.smithi191.stderr: 2023-10-08T16:16:42.365 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m5.036s 2023-10-08T16:16:42.366 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m8.545s 2023-10-08T16:16:42.366 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.317s 2023-10-08T16:16:42.366 INFO:tasks.ceph:Archiving logs... 2023-10-08T16:16:42.366 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi170/log 2023-10-08T16:16:42.367 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T16:16:43.588 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi191/log 2023-10-08T16:16:43.589 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T16:16:44.786 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T16:16:44.795 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T16:16:44.796 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T16:16:44.798 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T16:16:44.888 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T16:16:44.897 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T16:16:44.897 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T16:16:44.900 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T16:16:44.912 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2023-10-08T16:16:44.916 INFO:teuthology.orchestra.run.smithi170.stdout:210 Number of sources = 4 2023-10-08T16:16:44.916 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T16:16:44.916 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2023-10-08T16:16:44.916 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 14 +181us[ +181us] +/- 76ms 2023-10-08T16:16:44.917 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 16 -442us[ -438us] +/- 61ms 2023-10-08T16:16:44.917 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +429us[ +429us] +/- 75ms 2023-10-08T16:16:44.917 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:16:44.918 INFO:teuthology.orchestra.run.smithi191.stderr:bash: ntpq: command not found 2023-10-08T16:16:44.919 INFO:teuthology.orchestra.run.smithi191.stdout:210 Number of sources = 4 2023-10-08T16:16:44.919 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T16:16:44.920 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2023-10-08T16:16:44.920 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 17 +172us[ +181us] +/- 76ms 2023-10-08T16:16:44.920 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -457us[ -447us] +/- 61ms 2023-10-08T16:16:44.920 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +419us[ +419us] +/- 75ms 2023-10-08T16:16:44.920 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T16:16:44.921 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T16:16:44.931 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T16:16:44.932 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T16:16:44.978 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T16:16:45.033 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T16:16:45.132 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T16:16:45.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:45.159 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T16:16:45.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:45.190 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T16:16:45.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T16:16:45.228 INFO:teuthology.task.internal:Duration was 1442.885048 seconds 2023-10-08T16:16:45.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T16:16:45.278 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T16:16:45.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T16:16:45.281 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T16:16:45.325 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T16:16:45.326 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T16:16:45.756 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T16:16:45.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2023-10-08T16:16:45.757 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T16:16:45.777 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2023-10-08T16:16:45.777 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T16:16:45.796 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T16:16:45.797 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T16:16:45.819 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T16:16:45.876 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T16:16:45.877 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T16:16:45.879 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T16:16:45.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T16:16:46.003 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T16:16:46.004 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T16:16:46.028 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T16:16:46.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T16:16:46.062 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T16:16:46.070 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T16:16:46.092 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2023-10-08T16:16:46.115 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2023-10-08T16:16:46.129 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T16:16:46.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:46.159 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T16:16:46.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T16:16:46.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T16:16:46.193 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T16:16:46.194 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi170 2023-10-08T16:16:46.194 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T16:16:46.233 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298/remote/smithi191 2023-10-08T16:16:46.233 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T16:16:46.272 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T16:16:46.272 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T16:16:46.275 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T16:16:46.317 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T16:16:46.328 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T16:16:46.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T16:16:46.362 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T16:16:46.363 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T16:16:46.365 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T16:16:46.377 INFO:teuthology.orchestra.run.smithi170.stdout: 263006 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 16:16 /home/ubuntu/cephtest 2023-10-08T16:16:46.379 INFO:teuthology.orchestra.run.smithi191.stdout: 263006 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 16:16 /home/ubuntu/cephtest 2023-10-08T16:16:46.380 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T16:16:46.413 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T16:16:46.513 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T16:16:46.540 DEBUG:teuthology.nuke:shortname: smithi170 2023-10-08T16:16:46.540 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T16:16:46.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:43:39.490959', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK5J24LseHkEJqG75H9ewnuQ/awSKU1uoWWfrr4V/Kg3ryQWLjAKjqT4KloPDDmyWqC8zjxZYvVOpIw63sAzbkQ='} 2023-10-08T16:16:46.568 DEBUG:teuthology.nuke:shortname: smithi191 2023-10-08T16:16:46.568 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T16:16:46.591 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-08_15:41:30-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 15:43:39.492255', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ6W+dSuRSn1eeAOlPjZ9dUCkVo104fXgNegX76mlc9/XMmDaVHiwpXLJotSmstk8zg6q7J4uiG1RHLTRlQ6IRA='} 2023-10-08T16:16:46.624 INFO:teuthology.orchestra.console.smithi170:Power off 2023-10-08T16:16:46.624 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T16:16:46.652 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2023-10-08T16:16:46.652 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:46.669 INFO:teuthology.orchestra.console.smithi191:Power off 2023-10-08T16:16:46.669 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T16:16:46.691 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2023-10-08T16:16:46.691 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:46.699 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-08T16:16:46.714 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-10-08T16:16:50.700 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:50.716 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:50.811 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2023-10-08T16:16:50.825 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-10-08T16:16:54.815 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:54.825 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:54.924 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2023-10-08T16:16:54.935 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2023-10-08T16:16:55.026 INFO:teuthology.orchestra.console.smithi170:Power off completed 2023-10-08T16:16:58.938 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T16:16:59.048 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2023-10-08T16:16:59.149 INFO:teuthology.orchestra.console.smithi191:Power off completed 2023-10-08T16:16:59.253 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/basic}} duration: 1442.885047674179 failure_reason: 'Test failure: test_volume_rm (tasks.cephfs.test_volumes.TestVolumes)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=cd12955a3bbb45efa24670a233ad2067 status: fail success: false 2023-10-08T16:16:59.254 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T16:16:59.333 INFO:teuthology.run:FAIL